If you want to model weather systems, perform advanced computational mechanics, simulate the impact of climate change, study the interaction of lithium and manganese in batteries at the atomic level, or conduct the next experiment of your latest in vitro biomedical technique virtually — and you want to do it in Africa — then there is only one place to go: the Center for High Performance Computing (CHPC) in Cape Town.
Built and operated within the South African Council for Scientific and Industrial Research (CSIR), the CHPC is home to South Africa’s newest (and only) supercomputer. Named “Lengau,” which means “Cheetah” in Setswana, the system became fully operational in May 2016 and was ranked 121 on the June TOP500 list of the world’s fastest supercomputers. Its mission: to make South Africa, and indeed Africa itself, a major player within the international community of HPC-driven scientific researchers while also boosting South Africa’s burgeoning development in scientific and technical education. Such world-class ambitions, however, require equally world-class technology.
Built for Speed
Based on Intel Xeon processors, the new system is comprised of 1,013 Dell PowerEdge servers totaling 19 racks of compute nodes and storage. It has a total storage capacity of five petabytes and uses Dell networking Ethernet switches and Mellanox FDR InfiniBand with a maximum interconnect speed of 56 GB/s. With over 24,000 cores, the machine is the fastest computer on the African continent at roughly one petaflop (a thousand trillion floating point operations per second) — 15 times faster than CHPC’s previous system.
The person leading the effort to make the new supercomputer a reality was CHPC Director, Dr. Happy Sithole. For him, nothing less than world-class supercomputing power would suffice.
“For us, it’s no different from the rest of the world in terms of looking for opportunities where we need to accelerate competitiveness. I think high performance computing is vital for competitiveness in developed countries. In South Africa we also have that ambition to accelerate areas where we are competitive in industry and science.”
Those research domains are quite broad, Dr. Sithole says. “They cover chemistry, bioinformatics, astronomy, computational mechanics, engineering applications or systems, and the earth sciences including climate change. The South African Weather Service is a key collaborator as well as the Agricultural Research Council. It’s quite a broad spectrum of users.”
Key High Performance Computing Indicators
But advancing scientific research is only one of the key benefits high performance computing offers South Africa, Dr. Sithole says. Helping industry is another.
“The first key performance indicator for us is whether we are helping someone solve a problem faster. And the second is whether we demonstrate an impact to non-academic users — whether some of our industries can say we were able to do things much faster, we were able to generate more revenue, because of high performance computing.”
Virtual prototyping is a prime example, he says. “The more you are able to do virtual prototypes the faster you can take your product to market. And here at CHPC we have an ongoing investment in virtual prototyping.”
Unique Challenges … and also Opportunities
But if CHPC shares many of the same goals as other high performance computing centers, it also faces some unique challenges as well as opportunities.
“If you look at most centers around the world,” Dr. Sithole says, “they have the option to focus on a specific area. But we don’t have that luxury. We have some users who don’t have access to any other computing resources. That is our uniqueness — that we are the only center in the country and in the continent. We have all those users with varied needs of computing and also of application requirements. But our unique geographical position also brings us unique opportunities and some very good partnerships.”
A good example is climate change research. Like other countries, South Africa is very concerned about the future impact greenhouse gases will have on public health, agriculture, the availability of fresh water, and other areas. But what makes climate research here different is its focus on the Southern Hemisphere.
“Perhaps our biggest user,” Dr. Sithole says, “is a climate modeling team from the CSIR, which absolutely depends on the CHPC for what they call Variable Resolution Earth System Model or VRESM. This is an earth systems model for climate prediction that contributes to global research efforts. It specifically focuses on the Southern Hemisphere whereas similar modeling efforts elsewhere only focus on the Northern Hemisphere. VRESM relies on the CHPC because of the level of computing resources they are accessing — 9,000 to 10,000 cores at a time — which they cannot get anywhere else. And where before their models were limited to an eight-kilometer resolution, today they are at one-kilometer resolution. This is something they could not do before.”
Another example is materials science, particularly in fields like battery research and minerals differentiation (extracting precious metals from ores). South Africa ranks either very near or at the top in deposits of metals like manganese, platinum, chromite, vanadium, and vermiculite. Here too the new system’s increased computational power is having a clear impact. According to Dr. Sithole, “Materials science models that once took 11 days to finish now only take thee-quarters of a day. That’s a major improvement.”
Modeling Millions of Atoms
On the battery side, scientists use CHPC to model the interaction of atoms from different metals, like lithium and manganese, as a way to predict battery performance. “They’re looking at lithium manganese dioxide,” says Dr. Sithole. “In order to realistically show what happens in the actual battery system, researchers need to simulate a large number of lithium atoms traveling through the manganese. That means scaling the size of the battery system to millions of atoms. Where they could only model hundreds before, they have already surpassed 120,000 atoms and they now see they can push to millions.”
CHPC will also play a key role in support of the world’s largest radio telescope — the Square Kilometer Array (SKA) — scheduled to be deployed in South Africa’s Karoo desert by the year 2020. It will be 50 times more sensitive and survey the sky 10,000 times faster than today’s most powerful radio telescopes — and also generate record-setting amounts of astronomical data. The precursor to SKA is the MeerKAT radio telescope, located in South Africa’s Northern Cape. To enable users to have close proximity to their data and also help balance MeerKAT’s — and soon SKA’s — huge compute load, CHPC will support portions of MeerKAT’s data analysis and hold its archives. CHPC will also participate in efforts to create Africa’s first data-intensive cloud infrastructure as part of the country’s new Inter-University Institute for Data Intensive Astronomy (IDIA).
Partnerships Are Key
Supporting these types of use cases would be impossible, Dr. Sithole says, without the help of vendor partners.
“You would not be able to achieve this through working alone. We worked very closely with the Intel team especially when it came to working with the Lustre vendors but also in looking at the libraries and other Intel related dependencies. For example, some configurations under Intel Manager for Lustre software did not allow a number of files to be written at the same time. During this whole process their people were available all the time and were very helpful in resolving issues. Without companies like Intel we would not be able to achieve benefits like efficient parallelization or the introduction of new technologies. So partnerships with OEMs are very important when you are looking to build at scale.”
That’s just one of many lessons Dr. Sithole and his team learned in building out CHPC’s new supercomputer. Another was the need “to identify low hanging fruit so you can start demonstrating impact early.” Still another was to “start building expertise within your user base early and get users involved early and incrementally during the build-out process.”
Thanks to leadership like that, South Africa now has its own role to play in the global community of high performance computing — while at the same time enjoying the singular opportunities that come from leveraging this continent’s unique and abundant resources.
Randall Cronk of greatwriting, LLC is a technology writer with over 30 years’ experience writing about complex systems, products, and services.