During our interview with Sumit Gupta, VP, High Performance Computing and Data Analytics at IBM, on the importance of OpenPOWER to the HPC community, he noted that, “fundamentally, accelerators are the path forward.” These accelerators are GPUs for compute, storage accelerators for big data and FPGAs for special functions.
Driving this move to accelerators is a convergence of big data and big compute, plus the development of interfaces such as NVLink and CAPI (IBM’s Coherent Accelerator Processor Interface for POWER8 processors). These interfaces are important, as they make application acceleration much easier and transparent for the application programmer. NVLink in particular is important, as it more tightly binds GPUs to make programming more transparent.
Similarly, CAPI opens the door to new capabilities such as ‘storage acceleration’ where a single server can effectively have 56 terabytes of extended memory through the use of accelerated flash memory. Meanwhile, FPGA technology, currently utilized more via accelerated packages rather than programs, increases the throughput of specific applications. For example, FPGAs are becoming popular in genomics. The idea, Gupta says, “is to maximize throughput through the appropriate use of accelerators.” OpenPOWER gives developers “the ability to choose the right acceleration though the use of open standards where one company does not own the agenda,” which Gupta maintains, “gives the HPC community a platform for innovation.”
The use of accelerators immediately raises the issues of porting and offload mode computing. Gupta responded that GPUs have become such an important part of the HPC community that many major scientific computing applications have been adapted to utilize accelerators, and that new software efforts are being designed with the capability to run on accelerators. There are currently over 2350 software ISVs and more than 100,000 open-source packages available for Linux on Power systems, ranging from Redis NoSQL to SAP to Amber and GROMACS. These packages, plus libraries such as the IBM Mathematical Acceleration Subsystem libraries (ESSL), mean that customers can start running much of their workloads on OpenPOWER now, and that this ability will bring more clients to OpenPOWER in the future. Of course, there is also IBM’s Watson cognitive platform, which achieved fame by beating humans in the game of Jeopardy and is now being applied to big cognitive scientific and business problems.
It is the convergence of big data and big computing that is bringing both scientific and commercial customers to OpenPOWER and the use of specialized devices to accelerate computing.
Extreme capabilities are one reason for accelerators. For example, a single POWER8 server can utilize a CAPI-based storage accelerator to effectively present 56 terabytes of extended memory to an application. The claim is, “lower cost and higher workload density than a standard RAM-based system — with equal or better performance than scale out.”
NoSQL (Not Only SQL) in-memory databases can make use of the full scope of this extended memory. One example is the Redis Labs Enterprise Layer (rlec) that can redefine big in big-data unstructured scientific data analysis with 56 terabytes of in-memory databases.
The CAPI interface is the reason for such high accelerator performance, and explains why the OpenPOWER consortium partners have made such a significant investment in accelerator technology. Succinctly, the CAPI interface provides virtual addressing so the accelerator and processor can work with the same memory addresses without any operating system or device driver overhead. CAPI also provides hardware-managed cache coherence so either processors or accelerators can lock regions of memory — an essential requirement in parallel programming. IBM claims this eliminates 97 percent of the code path length, as shown in Figure 4.
GPUs also play an important part in the OpenPOWER story, as they are such a well-established accelerator in the HPC community. NVLink is likely to be a huge part in the future GPU success story. The $325M CORAL awards (which will use NVLink) demonstrate the confidence the Department of Energy (DoE) has in the IBM Power Architecture, NVIDIA’s Volta GPU and Mellanox’s network technologies to advance key research initiatives for national nuclear deterrence, technology advancement and scientific discovery.
When asked about the status of CORAL, Gupta could only say that “CORAL is on-track” and that “there are no new announcements.” Gupta also confirmed that there is currently no NVLink hardware available for testing at this time. However, he pointed out there are OpenPOWER systems based on existing technologies that developers can access over the Internet. Specifically, developers should start at http://developers.openpowerfoundation.org to gain access to the current OpenPOWER Developer ecosystem and explore the existing OpenPOWER tools and hardware.
FPGAs are also part of OpenPOWER. However, at this time, they are more focused on security, video and networking, as well as specialized processing for graph algorithms and genomics data. FPGAs can utilize the CAPI coherent interface. It is expected the use of FPGAs for specialized computing will grow, especially with the tremendous growth in machine-learning for big-data analytics. FPGAs can be a very efficient platform to support feed-forward deep learning neural networks, and they have the potential to play an expanded role in efficiently processing recurrent neural network architectures.
Cloud computing is expected to be a growth area in both commercial and scientific computing. According to Earl Joseph, VP for High Performance Computing at International Data Corporation (IDC), the HPC community appears to have transitioned from the experimental to an applied phase for cloud computing. The expectation is that 2016 will demonstrate an accelerated consumption of cloud computing by the commercial, government and academic HPC communities.
Gupta noted that open-source cloud applications, such as Apache Hadoop and Spark are already running on OpenPOWER hardware. The porting effort was effectively a recompilation; yet, in many cases, the resulting executable runs faster due to the hardware capabilities of the POWER8 processor. Gupta gave one example where an Apache Spark application ran up to 2x faster on POWER8. The acquisition of SoftLayer by IBM also provides an enterprise-grade global cloud infrastructure so HPC (and commercial) projects can utilize internal computational resources, yet create on-demand supercomputers in the cloud when there is a need for more compute.
The message is clear, OpenPOWER believes accelerated computing through the use of accelerators is the path forward to achieve application performance for modern workloads. In particular, the scientific community has already adapted many codes to run in an accelerator-based computing environment, which makes OpenPOWER an interesting alternative to consider for HPC procurements. The high visibility CORAL procurement represents an important next step in HPC that will validate OpenPOWER as a leadership-class HPC platform. Further, CORAL is of extreme interest as the accelerated, IBM Power-based machines will provide concrete data points from which the HPC community can glean essential power, heat, cost and reliability metrics for any future exascale procurements.
For more information:
- Solution Brief for Clients on Data Engine for NoSQL (CAPI-flash for Redis)
- Technical Whitepaper for Implementers — Offers details about the implementation of CAPI Flash acceleration.
- CAPI-FLASH APIs on GitHub — Enables clients / ISVs / partners to extend the CAPI Flash APIs for custom solutions.
- POWER8 Coherent Accelerator Processor Interface (CAPI) Developer Kit — Offers clients / ISVs / partners the ability to build custom FPGA solutions.
- Power Development Platform Cloud — Enables clients to begin development on POWER8 hardware.
- SuperVessel — Enables clients to develop on POWER8 hardware together with accelerators.
- YouTube video — Redis and POWER8: Developing High Performance Databases
Rob Farber is a global technology consultant and author with an extensive background in scientific and commercial HPC plus a long history of working with national labs and corporations. He can be reached at [email protected].
R&D 100 AWARD ENTRIES NOW OPEN: Establish your company as a technology leader! For more than 50 years, the R&D 100 Awards have showcased new products of technological significance. You can join this exclusive community! Learn more.
