All network traffic flowing in-and-out of the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) is now moving at 100 Gigabits per second (Gbps)—this includes everything from email to massive scientific datasets. Jason Lee, who leads NERSC’s Network and Security Team, worked with engineers at the DOE’s Energy Sciences Network (ESnet) to set up a 100Gbps Science DMZ, which gives NERSC network engineers the ability to set up multiple private circuits using software-defined networking (SDN). With these tools, NERSC staff can help remote scientists who may see their data transfers slow down due to firewalls at their local campuses achieve a true 100Gbps end-to-end connection. Additionally, ESnet engineers also helped NERSC set up a system to announce their own address space. This allows the center to separately route traffic to any research or education (R&E) site or separate R&E traffic from the commodity Internet.
Keith Jackson, ESnet Guest Blogger
Recently we’ve been testing the ability to move huge amounts of scientific data in and out of commercial cloud providers like Amazon and Google. We were doing this because if you want to do scientific computation in the cloud, you need to be able to move data in and out efficiently or it will never be useful for science.
Recently we’ve been working with engineers at Google to test the performance of their cloud storage solution. We were in the midst of transferring data between Berkeley Lab servers and the Google cloud when we noticed the data wasn’t moving as fast as it should.
We tried to figure out the root of the problem. The Google folks talked to their networking people and we talked to our engineers at ESnet.
We found there was a bottleneck in the path between Berkeley and Google on the ESnet side. One path was still only 1 gigabit and was scheduled to be upgraded to 10 gigabit in the next week or so. But it limited us to no more than a gigabit per second data transfers.
Using OSCARS, not only did we find the bottleneck, but as Vangelis talked about in a prior blogpost, we were able to find a way to reroute traffic to avoid the slow link, completely bypassing the problem. ESnet was not only able to help me diagnose the problem right away, but were able to suggest and quickly deploy a solution.
In thinking about that problem, a few things occurred to me. For a scientist just concerned with getting data through the network, it is probably easier to work with ESnet than a commercial provider for several reasons.
As a research network, ESnet is completely accessible. A commercial provider would have been completely opaque because of proprietary issues and have no incentive to grant access into its network for troubleshooting by outsiders. Since serving scientists is not its main mission, its sense of urgency would be different. Moreover, a commercial network’s interfaces are not designed for the particular needs of scientists.
But ESnet exists solely to support science, and scientists. Sometimes we need to be reminded that to scientists, quite literally, the “network matters.”