ZFP: Fast, Accurate Data Compression for Modern Supercomputing Applications is today’s R&D 100 of the day

Lawrence Livermore National Laboratory’s ZFP is a new compressed number format for floating-point and integer arrays intended to reduce in-memory and offline storage and transfer time of large data sets that arise in high-performance computing (HPC). ZFP effectively expands available CPU and GPU memory by as much as 10 times; reduces offline storage by one to two orders of magnitude; and — by reducing data volumes — speeds up data movement between memory and disk, distributed compute nodes, CPU and GPU memory, and even main memory and CPU registers. Such reductions in data movement are critical to today’s HPC applications, whose performance is largely limited by data movement rather than compute power.