Science

Learn how research networks are supporting advances in science discovery.

ESnet recognized for outstanding performance

~ ESnetComms

ESnet’s Evangelos Chaniotakis and Chin Guok received Berkeley Lab’s Outstanding Performance Award for their work in promoting technical standards for international scientific networking. Their work is notable because the implementation of open-source  software development and new technical standards for network interoperability sets the stage for scientists around the world to better share research and collaborate.

Guok and Chaniotakis worked extensively within the DICE community on development of the Inter-domain Controller Protocol (IDCP). They are taking the principles and lessons gained from years of development efforts and applying them to the efforts in international standards bodies such as the Open Grid Forum (OGF), as well as consortia such as the Global Lambda Infrastructure Facility (GLIF).

So far, the IDCP has been adopted by more than a dozen Research and Education (R&E) networks around the world, including Internet2 (the leading US higher education network), GEANT (the trans-European R&E network), NORDUnet (Scandinavian R&E network) and USLHCNet (high speed trans-Atlantic network for the LHC community).

Guok and Chaniotakis have also advanced the widescale deployment of ESnet’s virtual circuits OSCARS (On Demand Secure Circuits and Reservation System). OSCARS, developed with DOE support, enables networks
to schedule and move the increasingly vast amounts of data generated by large-scale scientific collaborations. Since last year, ESnet has seen a 30% increase in the use of virtual circuits. OSCARS virtual circuits now carry over 50% of ESnet’s monthly production traffic.  The increased use of virtual circuits was a major factor enabling ESnet to easily handle a nearly 300% rise in traffic from June 2009 to May 2010.

A few reasons why ESnet matters to scientists.

~ ESnetComms

Keith Jackson, ESnet Guest Blogger

Recently we’ve been testing the ability to move huge amounts of scientific data in and out of commercial cloud providers like Amazon and Google. We were doing this because if you want to do scientific computation in the cloud, you need to be able to move data in and out efficiently or it will never be useful for science.

Recently we’ve been working with engineers at Google to test the performance of their cloud storage solution. We were in the midst of transferring data between Berkeley Lab servers and the Google cloud when we noticed the data wasn’t moving as fast as it should.

We tried to figure out the root of the problem. The Google folks talked to their networking people and we talked to our engineers at ESnet.

We found there was a bottleneck in the path between Berkeley and Google on the ESnet side. One path was still only 1 gigabit and was scheduled to be upgraded to 10 gigabit in the next week or so. But it limited us to no more than a gigabit per second data transfers.

Using OSCARS, not only did we find the bottleneck, but as Vangelis talked about in a prior blogpost, we were able to find a way to reroute traffic to avoid the slow link, completely bypassing the problem. ESnet was not only able to help me diagnose the problem right away, but were able to suggest and quickly deploy a solution.

In thinking about that problem, a few things occurred to me. For a scientist just concerned with getting data through the network, it is probably easier to work with ESnet than a commercial provider for several reasons.

As a research network, ESnet is completely accessible. A commercial provider would have been completely opaque because of proprietary issues and have no incentive to grant access into its network for troubleshooting by outsiders. Since serving scientists is not its main mission, its sense of urgency would be different. Moreover, a commercial network’s interfaces are not designed for the particular needs of scientists.

But ESnet exists solely to support science, and scientists. Sometimes we need to be reminded that to scientists, quite literally, the “network matters.”

Down another pit, looking for the secrets of the universe

~ ESnetComms

The Large Hadron Collider is the world’s largest particle collider. At the bottom of a huge shaft dug into a mountain, two beams of subatomic particles, dubbed “hadrons”, shoot around an enclosed racetrack accelerating with every lap until they collide. The idea is to recreate on a small scale the conditions in the universe immediately after the Big Bang.

I actually saw the LHC under construction. I was visiting another accelerator used to generate antimatter, escorted by a physicist with spiky hair who looked like he played in a band. The question of why the universe is composed of matter versus antimatter could give humans a glimpse of “God’s big toe” according to this recent NYTimes interview http://www.nytimes.com/2010/05/18/science/space/18cosmos.html , and points to a fundamental asymmetry in the universe. The accelerator was wrapped in tinfoil and duct tape as a sort of low-tech insulation.

It is not such a bad way to spend one’s career figuring out how the universe got started. Here at ESnet, we are helping. ESnet is part of the network that carries the data from the LHC in Switzerland to groups of physicists in the U.S.

This view is at the inception of traffic on April 1st, 2010.

The Large Hadron Collider is projected to generate 15 petabytes of data yearly from six different detector experiments. The data, too massive to handle internally, is sent to 12 tier 1 sites around the world. CERN data from the ATLAS and CMS detectors travels the Atlantic on USLHCNet http://lhcnet.caltech.edu/ ESnet then carries data to Brookhaven National Laboratory and Fermilab, US tier 1 sites, for processing and archiving. The data is then distributed to tier 2 facilities, mainly of universities and research institutions around the U.S., including the Berkeley Lab’s National Energy Research Scientific Computing Division (NERSC).–Wendy Tsabba

What we see here is an upward spike in traffic. This view is taken within the last 96 hours.

Looking for cosmic particles deep underground

~ ESnetComms

A mile or so down, activity is afoot in the formerly disused Homestake Mine in South Dakota’s Black Hills.  Instead of digging for gold, researchers will hunt for cosmic particles at the Sanford Underground Laboratory. Going deep underground is a chance to revisit physics experiments, as current neutrino detectors are orders of magnitude more sensitive than their predecessors,  and try some new ones without interference from cosmic radiation. And when scientific data is generated, ESnet will be connecting to the networks that will bring it to researchers all over the country.

The Deep Underground Laboratory (DUSEL) is sponsored by the National Science Foundation. Berkeley physicist Kevin Lesko videolinked to the spring Internet2 members meeting from the Ross Shaft at the 4850 level. Clad in a hard hat, and black protective suit laced with reflective tape, Lesko stood in a drafty rock tunnel festooned with wires and pipes and described the search for the next big discoveries in physics. According to Lesko, the plan is to build laboratory modules in 5-6 stories high and a football field in length. In the Davis Cavity, physics experiments are planned for megacavities filled with detectors to catch neutrinos beamed from Fermilab in Chicago 1000 miles away.   There is also a large underground Xenon detector planned to help image dark matter. Geobiologists are sampling microbes to look for specialized bacteria that could give clues to the evolution of life. See the National Academies Press Report: The Limits of Organic Life in Planetary Systems

For this Big Science initiative, NSF is working with the DOE and building networking capacity into the project. There are 370 miles of tunnel in the mine. Engineering challenges include ensuring structural integrity, and pumping in air and pump out water to create a comfortable and safe working environment.  DUSEL’s network engineering team have already deployed 5 miles of single mode fiber 5000 feet down the mineshafts to provide network connectivity. The mine, filled with dust and humidity, is a harsh environment. Engineering teams are building redundant fiber links down the mine shaft for high-speed network access. The labs are in the design stage now, and a report will be finished by year’s end to submit to the NSF. Construction on the laboratory starts in 2014.–Wendy Tsabba