R&D 100 winner of the day: SZ: A Lossy Compression Framework for Scientific Data

Scientific data reduction is a critical problem that must be addressed for the success of exascale supercomputers and next-generation facility-size scientific instruments at a time when they are urgently needed to address important societal problems like climate change, water management, advanced manufacturing and the development of new vaccines and drugs. The SZ compression framework for scientific data, from Argonne National Laboratory, demonstrates the widest scope of use-cases and offers the best performance for more applications in compression ratios, speed and accuracy compared to competitive products. SZ is applicable to a large spectrum of scientific simulations and instrument facilities. For example, SZ compresses twice more seismic imaging data for oil and gas than the second-best compressor for the same accuracy. SZ addresses effectively the severe data streaming reduction challenge raised by crystallography in the Linac Coherent Light Source, for example, while keeping critical science details. For quantum chemistry, SZ accelerates the execution by up to 50%. No other lossy compressor offers this diversity of applicability and this consistency in performance.