ARMONK, NY and HAMBURG, Germany — The German Climate Computing Center is managing the world’s largest climate simulation data archive, used by leading climate researchers worldwide. The archive currently consists of more than 40 petabytes of data and is projected to grow by roughly 75 petabytes annually over the next five years. For comparison, a petabyte of data is equal to roughly 210,000 DVDs. As climate simulations are carried out on increasingly powerful supercomputers, massive amounts of data are produced that must be effectively stored and analyzed.
To efficiently manage and provide quick access to this amount of data for analysis and research through its Hierarchical Storage Management system, the Center has selected the High Performance Storage System software solution developed by IBM and the U.S. Department of Energy. Supported by IBM services, this solution is capable of handling more than 500 petabytes of data.
Organizations including the Intergovernmental Panel on Climate Change, the Max-Planck Institute for Meteorology, the University of Hamburg, the Helmholtz-Zentrum Geesthacht Center for Materials and Coastal Research, the Alfred Wegener Institute Helmholtz Center for Polar and Marine Research, the West African Science Service Center on Climate Change and Adapted Land Use, The Coordinated Energy and Water Cycle Observation Project, the Program For Earth System Modeling, and the Center itself use data in the archive for a range of research and reporting activities, including:
- IPCC research and reporting on the increase in the earth’s temperature and its impact on the climate
- simulations to help predict the geographical and environmental impact of oil spills
- the impact of climate change on West Africa, the Mediterranean, Central Europe, Indonesia and other key global regions to determine how to adapt to it
- analysis for airplane traffic routes and altitudes to reduce the emission of carbon dioxide and other gases that may impact climate
- developing 3-D models and visualizations for creating smarter cities and more energy-efficient buildings
- high-definition modeling for the formation of clouds and precipitation to improve the accuracy of weather and climate prediction
“Addressing the challenges of climate change and its impact on the environment and daily life for everyone on the planet requires gathering, storing and analyzing massive amounts of data,” said Professor Thomas Ludwig, director of the German Climate Computing Center and professor of computer science at the University of Hamburg. “We rely on the technology and the expertise of IBM to help us provide this essential service to the world’s foremost climate and environmental research institutions.”
The upgrade of the Center’s Hierarchical Storage Management system includes a solution consisting of the latest x86 servers and Red Hat Linux operating system running the HPSS software. This solution manages all simulation data and serves as the input and output interface between the Center’s high performance computing systems and the tape storage library where the climate data is stored.
“At IBM, one of the values we try to live up to daily is providing innovation that matters for our company and the world,” said Markus Koerner, vice president, IBM Global Technology Services. “I can think nothing that matters more than helping to effectively predict and mitigate the impact of climate change, and we believe the technology and services we provide for this Big Data challenge will significantly help advance the science of climate change.”
The solution uses a disk cache of five petabytes of storage capacity and is capable of providing data access speeds of up to 12 gigabytes per second, which will be upgraded to 18 gigabytes per second later this year. IBM Global Technology Services supports this solution with data center managed services on site, project management services, and hardware vendor management services, and the HPSS solution incorporates IBM DB2 scalable relational database software.
HPSS technology is the result of a 20-year collaborative effort between IBM and the Lawrence Livermore, Los Alamos, Sandia, Lawrence Berkeley and Oak Ridge National Laboratories of the U.S. Department of Energy. It is used for very large scientific data archives at research facilities and universities, and for weather forecast systems.
For further information: www.dkrz.de/Klimarechner-en/datenarchiv
