ESnet, Caltech and NERSC Team Up to Send Data Screaming at 90 Gbps

ESnet, the Department of Energy’s high-bandwidth network, is working with Caltech and DOE’s National Energy Research Scientific Computing Center (NERSC) to move large data files at an average 90 gigabits per second (Gbps) to the SC13 conference in Denver.

ESnet’s Eric Pouyoul, who set up the data transfers, said that the transfer rate is both more impressive and more realistic as it is being transferred from disc to memory.  “That’s reality – moving files from memory to memory is faster, but it’s not real life,” Pouyoul said.
To set up the demo, Pouyoul created 24 files, each containing 130 gigabytes of data and stored them at NERSC in Oakland, Calif. He configured them to transfer in loops over ESnet’s link to Denver and into the convention center – and over 15 hours achieved an average of 90 Gbps. The demo used the FDT (Fast Data Transfer) tool forr transferring the files, with support from the FDT team.ESnet is supporting four 100 Gbps links into SC13 at the Colorado Convention Center in Denver.

Working in the Caltech booth in the SC13 Exhibition, network engineer Azher Mughal said the demonstration was created to help people understand how large datasets are becoming and how to efficiently transfer the data. Caltech manages the trans-Atlantic link that brings data from the Large Hadron Collider in Switzerland to the United States.Caltech-portal

“We want to showcase how you can efficiently manage large data sets, the challenges you’re likely to see, and how to optimize the transfer,” Mughal said. By sending the data across a single 100 Gbps line rather than multiple 10 Gbps links, there are fewer transfer problems and  it requires fewer CPUs on the computer systems, making them more efficient in the process.


Network-aware Data Management Workshop @SC13

For those interested in networking topics, you will want to check out the Network-aware Data Management workshop @SC13. This all day workshop on Sunday November 17th includes a number of talks related to high-speed networking, including a talk by ESnet’s Brian Tierney on the ESnet 100G Network Testbed. Other talks include “Supporting Climate Modeling Over Named Data Networking” from Christos Papadopoulos, Colorado State University, “Challenges and Solutions in Large Scale Data Movement” by Martin Swany, Indiana University, and many more. 

The goal of this workshop is to bring together researchers and engineers from academia, government, and industry to discuss emerging trends and new technological developments in dynamic resource provisioning, intelligent data-flow and resource coordination, network-aware application design issues, and 100Gbps network performance problems.

For more information see: 

ESnet to Deliver 100G Connectivity to Demos at SC13 Conference

When the Colorado Convention Center becomes the best-connected site on the planet for SC13 from Nov. 17-22, ESnet will be providing a significant portion of the connectivity, which will be used to support live demos by Caltech (booth 3118), Ciena (booth 1924), NASA (booth 822) and the Laboratory for Advanced Computing / Open Cloud Consortium (booth 828 ). Additionally, Brocade (booth 109) will use OSCARS – ESnet’s on-demand bandwidth reservation system – in demonstrations of multi-layer software defined networking.

For Caltech, ESnet will provide four 100G paths to bring data to Denver from CERN in Switzerland, Fermilab in Illinois, DOE’s NERSC (National Energy Research Scientific Computing Center) in California and the Karlsruhe Institute of Technology in Germany.

For demonstrations in the NASA booth, ESnet will provide two paths – one a northern route and the other a southern route – from NASA’s Goddard Space Flight Center in Maryland.

Ciena, in collaboration with ESnet and Internet2, will be delivering unprecedented support for the SCinet network at SC13 through a high-capacity 400G transport network that will carry three of the 100G links that ESnet brings to the Denver show floor.

In the Laboratory for Advanced Computing / Open Cloud Consortium booth, ESnet will provide two paths for a demonstration by the Naval Research Laboratory and the University of Chicago/International Center for Advanced Internet Research, or iCAIR.

Additionally, Brocade (booth 109) will use OSCARS – ESnet’s on-demand bandwidth reservation system – in demonstrations of multi-layer software defined networking. If you miss seeing this at SC, the demonstration will be showcased live on SDNCentral’s DemoFriday on November 22. Learn more at

Key to all of this are the efforts of the SCinet team that works to provide bandwidth into and throughout the convention center. ESnet staff supporting SCinet are Patrick Dorn, Andy Lake, Lauren Rotman and Jason Zurawski.