During the SC10 conference, NASA, NOAA, ESnet, the Dutch Research Consortium, US LHCNet and CANARIE announced that they would transmit 100Gbps of scientific data between Chicago and New Orleans. Through the use of 14 10GigE interconnects, researchers attempted to completely utilize the full 100 Gbps worth of bandwidth by producing up to twelve 8.5-to-10Gbps individual data flows.
Brian Tierney reports: “We are very excited that a team from NASA Goddard completely filled the 100G connection from the show floor to Chicago. It is certainly the first time for the supercomputing conference that a single wavelength over the WAN achieved 100Gbps. The other thing that is so exciting about it that they used a single sending host to do it.”
“Was this just voodoo?” asked NERSC’s Brent Draney.
Tierney assures us that indeed it must have been… but whatever they did, it certainly works.
We caught up with Tracey Wilson, the DICE program manager, who had this to say about the Data Intensive Computing Environment/Obsidian Strategics SCinet Sandbox project, or DOS3 for short: “In our Sandbox project, we wanted to look at the effects of putting E-100 Infiniband extenders with encryption,” said Wilson. “We wanted to test out the performance, with regular TCP-IP transfer and also RDMA transfers. We worked with Lawrence Livermore and they allowed us to have access to almost 100 nodes of Hyperion, a big test cluster there. Our intent there was to do a lot of wide area luster testing over the wide area Infiniband network. We also worked with BlueArc to attach a Mercury NAS system to an IB gateway and access that remotely from both the Hyperion site and the other sites we were using across the fabric using the Obsidian extenders. ESnet is the primary carrier for Hyperion, but we have four other circuits coming from NLR to our other sites. So we’ve been able to get to NASA Goddard and do a span test across the Obsidian on 10G links using NLR. We had a testing slot scheduled from both NLR and ESnet, but we had 10G available through most of the show, thanks to ESnet.”
“Real thanks go to the ESnet guys; Evangelos Chaniotakis, Jon Dugan, and Eli Dart who were supporting SCinet as well as Linda Winkler from Argonne and Caroline Weilhammer from the global NOC, Andrew Lee from NLR and the rest of the routing team.” said Wilson. “It was so much work getting all the VLANs configured across the show floor, because there were a huge number of demos this year and it was a considerable effort to get all these things configured.”