Research & Development World

  • R&D World Home
  • Topics
    • Aerospace
    • Automotive
    • Biotech
    • Careers
    • Chemistry
    • Environment
    • Energy
    • Life Science
    • Material Science
    • R&D Management
    • Physics
  • Technology
    • 3D Printing
    • A.I./Robotics
    • Software
    • Battery Technology
    • Controlled Environments
      • Cleanrooms
      • Graphene
      • Lasers
      • Regulations/Standards
      • Sensors
    • Imaging
    • Nanotechnology
    • Scientific Computing
      • Big Data
      • HPC/Supercomputing
      • Informatics
      • Security
    • Semiconductors
  • R&D Market Pulse
  • R&D 100
    • Call for Nominations: The 2025 R&D 100 Awards
    • R&D 100 Awards Event
    • R&D 100 Submissions
    • Winner Archive
    • Explore the 2024 R&D 100 award winners and finalists
  • Resources
    • Research Reports
    • Digital Issues
    • R&D Index
    • Subscribe
    • Video
    • Webinars
  • Global Funding Forecast
  • Top Labs
  • Advertise
  • SUBSCRIBE

Argonne Researchers Demonstrate Extraordinary Throughput at SC14

By R&D Editors | December 3, 2014

A team of researchers from Argonne National Laboratory and DataDirect Networks (DDN) moved 65 terabytes of data in under just 100 minutes at a recent supercomputing conference. Typically, two days are needed to move this volume of data between sites with a 10 Gbps connection.
 
With help from Ciena, Brocade, and ICAIR, the team sustained data transfer rates in excess of 85 Gbps — with peaks at over 90 Gbps — between storage systems in Ottawa, Canada, and New Orleans, Louisiana, over a 100 Gbps wide-area network (WAN) connection. The demonstration took place on November 19, 2014 at SC14, the leading international conference for high performance computing, networking, storage and analysis.
 
This unprecedented achievement required combining the embedded file system and virtual machine capabilities of the DDN storage controller, the high-speed wide-area data transfer capabilities of the Globus GridFTP server, and an advanced 100G wide-area network.
 
“Embedding the GridFTP servers in virtual machines on DDN’s storage controller eliminates the need for external data transfer nodes and network adapters,” explained Raj Kettimuthu, principal software development specialist at Argonne National Laboratory. “We sustained a data transfer rate of 85 Gbps for over 60 minutes — and occasionally for as long as 90 minutes — several times during the SC14 conference.”
 
Achieving 90+ Gbps for memory-to-memory transfers using a benchmarking tool like iperf is straightforward and has been demonstrated several times in the past. Achieving similar rates for disk-to-disk transfers however, presents a number of challenges, including choosing the appropriate block size that works well for both disk I/O and network I/O, and selecting the appropriate combination of parallel storage I/O threads and parallel TCP streams for optimal end-to-end performance.
 
Network experts often claim that storage is the bottleneck in the end-to-end transfers on high-speed networks, while storage experts claim that the network is often the bottleneck on transfers between sites with high-performance parallel file systems. “This demonstration was aimed at bringing together the experts and latest developments in all aspects concerning disk-to-disk WAN data movement, including network, storage, and data movement tools,” said Kettimuthu.
 
The team expects that the approach can be used to achieve 100+ Gbps wide-area transfer rates between storage systems using multiple WAN paths and additional storage resources in the end systems.
 
Team members were Kevin Harms, Eun-Sung Jung, Raj Kettimuthu, Linda Winkler from Argonne National Laboratory and the University of Chicago and Mark Adams from DataDirect Networks, with help from Jim Chen and Joe Mambretti from ICAIR, Doug Hogg and Marc Lyonnais from Ciena, Wilbur Smith from Brocade, Jon Dugan and Brian Tierney from ESnet, Ian Foster and Mike Link from Argonne National Laboratory and the University of Chicago, and Clayton Walker, Laura Shepard, Susan Presley, and Bob Vassar from DDN.

Related Articles Read More >

QED-C outlines road map for merging quantum and AI
Quantum computing hardware advance slashes superinductor capacitance >60%, cutting substrate loss
Hold your exaflops! Why comparing AI clusters to supercomputers is bananas
Why IBM predicts quantum advantage within two years
rd newsletter
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, trends, and strategies in Research & Development.
RD 25 Power Index

R&D World Digital Issues

Fall 2024 issue

Browse the most current issue of R&D World and back issues in an easy to use high quality format. Clip, share and download with the leading R&D magazine today.

Research & Development World
  • Subscribe to R&D World Magazine
  • Enews Sign Up
  • Contact Us
  • About Us
  • Drug Discovery & Development
  • Pharmaceutical Processing
  • Global Funding Forecast

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search R&D World

  • R&D World Home
  • Topics
    • Aerospace
    • Automotive
    • Biotech
    • Careers
    • Chemistry
    • Environment
    • Energy
    • Life Science
    • Material Science
    • R&D Management
    • Physics
  • Technology
    • 3D Printing
    • A.I./Robotics
    • Software
    • Battery Technology
    • Controlled Environments
      • Cleanrooms
      • Graphene
      • Lasers
      • Regulations/Standards
      • Sensors
    • Imaging
    • Nanotechnology
    • Scientific Computing
      • Big Data
      • HPC/Supercomputing
      • Informatics
      • Security
    • Semiconductors
  • R&D Market Pulse
  • R&D 100
    • Call for Nominations: The 2025 R&D 100 Awards
    • R&D 100 Awards Event
    • R&D 100 Submissions
    • Winner Archive
    • Explore the 2024 R&D 100 award winners and finalists
  • Resources
    • Research Reports
    • Digital Issues
    • R&D Index
    • Subscribe
    • Video
    • Webinars
  • Global Funding Forecast
  • Top Labs
  • Advertise
  • SUBSCRIBE