Routers, the Portal, and Transcontinental 100G Links – Oh My!

Things are moving quickly at ESnet, and we’re not talking only about data transfers.  Our Advanced Networking Initiative (ANI) 100G roll-out is gaining even more momentum and we are really gearing up for the SC11 conference next month.

This week, it was announced that we are working with LGS Innovations to deploy Alcatel-Lucent 7750 Service Routers (SR) on the new ANI prototype network. The first routers were delivered about a month ago and we are already well into acceptance testing and deployment. So far we have routers up and running in Sunnyvale, NERSC, StarLight and Argonne.  After we installed this equipment, our engineering team worked with Internet2 to light the first coast-to-coast network path from Washington, D.C. all the way to Sunnyvale in our backyard  –  the first 100G transcontinental link in the world! We were particularly proud of this milestone, which we reached well ahead of schedule. As next steps, we anticipate that 6 routers of our 10 total will be deployed in time for SC11.

To help the community keep tabs on our rapid ANI deployment progress, the ESnet Tools team has been expanding the MyESnet portal to provide a real-time view of the rollout. You can see the result of their efforts here.

This summer we introduced MyESnet to provide DOE Office of Science researchers and IT staff with a wide range of customized tools intended to greatly improve their ability to understand network issues in real time. The initial functionality includes a dashboard that provides a bird’s-eye view of the local area networks of the DOE facilities connected by ESnet, including their traffic patterns and system status.

Using this new ANI visualization tool, users can see which 100G network links are active, as well as information on each of our node and interconnect locations across the country.  The tool will be expanded to show traffic load in the near future.  In the next few weeks, the portal will also provide a real-time view of the eight community-driven projects that will use the new 100G network for large scale demonstrations at SC11. This interface will allow viewers to view the details and traffic load of these demonstrations.

With fewer than 25 days to go until SC11, our team is gearing up for some exciting announcements and ANI demos . . . check back for info on these activities soon and check in with us in Seattle. ESnet is part of Berkeley Lab booth 512.

perfSONAR pS-Performance Toolkit 3.2.1 released

Need to troubleshoot some network performance problems?  perfSONAR-PS is an open source development effort to create a colletion of easy-to-use and easy-to-install perfSONAR network performance monitoring services and tools. Version 3.2.1 of the pS-Performance Toolkit is now available for download. This update contains: new throughput graphs, new delay/loss graphs, scheduled traceroute tests and numerous bug fixes. Release notes are available to denote all changes since the 3.2 release. Please visit the pS Performance Toolkit page for more information.

–Brian Tierney

Nobel Prize Lesson: Distinguish the Improbable from the Impossible

Saul Perlmutter was woken at 3 am yesterday by a reporter asking how he felt about winning the Nobel Prize. Any confusion was cleared up a few minutes later when the Nobel Committee in Sweden called. Perlmutter, an astrophysicist who holds a joint appointment at Berkeley Lab and UC Berkeley, and his colleagues Brian Schmidt and Adam Reiss received the 2011 Nobel Prize in Physics for their work in the 1990s in using supernovae to measure the accelerating expansion of the universe. They found, independently, improbable evidence that the universe was expanding at ever faster speeds.

In the process they uncovered other mysteries. The universe is made up of only about 5 percent visible (or baryonic) matter. The remaining 25 percent of the universe is dark matter, and 75 percent of the universe is made up of dark energy. Perlmutter described dark energy as making space more bouncy and elastic. It appears as “a negative pressure—we see it in the equations.” Nobody is certain of the relationship between dark matter and dark energy; scientists are looking for a theoretical concept that will solve them both at the same time.

In yesterday’s press conference at Berkeley Lab, Perlmutter said that the award “recognizes what it is possible to do when whole communities of science come together.” He went on to say, “when you are driven to learn things about the world, you find yourself inventing new things.”

Of course, that is what ESnet is all about—the ESnet network links scientists so they can efficiently exchange data, collaborate, and discover new things about the world. ESnet’s Bill Johnston recalls the early days of data transfer. “Saul started out doing observations at Chabot and Hamilton. He wrote the images on Tektronix cartridge tape and ferried them around in his car.  When his tape reader at LBL broke, we would help him out. In the graphics lab that Harvard Holmes and I ran, we had several Tek tape readers. That must have been [the] late 1970s.”

And then there’s Daniel Schectman, a hard-core materials scientist at the Technion-Israel’s Institute of Technology, who today was awarded the Nobel Prize in Chemistry for discovering a new state of matter called quasi-crystals, where atoms make a structured pattern that never repeats.  While on sabbatical at NIST in 1982, Schectman was working at the electron microscope, imaging a sample of aluminum and maganese. He made a measurement that seemed to be a mistake– the symmetry of its five-fold crystal structure violated the laws of physics. It is possible to get an identical pattern from structures with 3,4,6 and 8 sides, just like square floor tiles can be repeated in an identical pattern, no matter how you rotate them. But Schectman knew it was theoretically impossible to tile a space in five-fold rotational symmetry.

A former student who took his class in electron beam crystallography at the Technion recalls that Schectman showed the class the research notebook that he kept at NIST. Beside the measurement of the anomalous sample, Schectman scribbled in Hebrew, “there is no creature like that.” Schectman persisted through years of caustic opposition from his peers (Linus Pauling, a two time Nobel-winner, was particularly hostile), to verify his results—he was even asked to leave his research group. The original paper he attempted to publish in the Journal of Applied Physics in 1984 was immediately rejected, but he later published another paper with collaborators that rocked the field of crystallography.

It turns out that to tile a space and achieve five-fold symmetry uses not a single tile, but two—one distorted, a concept called Penrose tiling. But at the time, Schectman was unfamiliar with Roger Penrose’s work; it was an idea that only pure mathematicians played with–it wasn’t even applied mathematics. On an atomic level, quasi-crystals resemble aperiodic mosaics, such as those found in the medieval Islamic mosaics of the Alhambra Palace in Spain and the Darb-i Imam Shrine in Iran.

Schectman’s discovery caused a paradigm shift in chemistry, the Swedish Academy of Sciences said. But exotic quasi-crystals, since they don’t scratch easily, are now appearing in our everyday lives in the non-stick coatings on razor blades, drill-bits, and pots and pans.

Schectman and Perlmutter both stood on the shoulders of people who came before them, and relied on their communities to test and verify their results.  The pace of scope of modern day science is making international collaboration not a luxury, but a necessity. Although individual persistence and courage drive many new discoveries, they still take place in the context of a global community of scientists.  ESnet and other research and education networks make this community possible, by allowing scientists to share datasets and knowledge and to work together towards even greater discoveries.

“Science is a method, not a finished project; we don’t know where it will lead in the future,” Perlmutter summed up. And networks are forging the way.

Idea Power: Two ESnet Projects are Honored With Internet2 IDEA Awards

We are proud to announce that two of ESnet’s projects have received IDEA (Internet2 Driving Exemplary Applications) awards in Internet2’s 2011 annual competition for innovative network applications that have had the most positive impact and potential for adoption within the research and education community. (see: Internet2’s press release).

Internet2 recognized OSCARS (On-Demand Secure Circuits and Advance Reservation System), developed by the ESnet team led by Chin Guok, including Evangelos Chaniotakis, Andrew Lake, Eric Pouyoul and Mary Thompson. Contributing partners also included Internet2, USC ISI and DANTE.

ESnet’s MAVEN (Monitoring and Visualization of Energy consumed by Networks) proof of concept application was also recognized with an IDEA award in the student category. MAVEN was prototyped by Baris Aksanli during his summer internship at ESnet. Baris is a Ph.D student at the University of California, San Diego conducting research at the System Energy Efficiency Lab with his thesis advisor, Dr. Tajana Rosing. Baris worked closely with his summer advisor, Inder Monga, and Jon Dugan to implement MAVEN as part of ESnet’s new Green Networking Initiative.

The idea behind OSCARS

OSCARS enables researchers to automatically schedule and guarantee end-to-end delivery of scientific data across networks and continents. For scientists, being able to count on reliable data delivery is critical as scientific collaborations become more expansive, often global. Meanwhile, in disciplines ranging from high-energy physics to climate, scientists are using powerful, geographically dispersed instruments like the Large Hadron Collider that are producing increasingly massive bursts of data, challenging the capabilities of traditional IP networks.

OSCARS virtual circuits can reliably schedule time-sensitive data flows – like those from the LHC – round the clock across networks, enabling research and education networks to seamlessly meet user needs. OSCARS code is also being deployed by R&E networks worldwide to support an ever-growing user base of researchers with data-intensive collaboration needs. Internet2, U.S. LHCnet, NORDUNet, RNP in Brazil as well as over 10 other regional and national networks have currently implemented OSCARS for virtual circuit services. Moreover, Internet2’s NSF-funded DyGIR and DYNES projects will in 2012 deploy over 60 more instances of OSCARS at university campuses and regional networks to support scientists involved in LHC, Laser Interferometer Gravitational-Wave Observatory (LIGO), Large Synoptic Survey Telescope (LSST) and Electronic Very-Long Baseline Interferometry (eVLBI) programs.

We are proud of the hard work and dedication the OSCARS development team has demonstrated since the start of this project. Just as importantly we are proud to see this work paying off in with new science collaboration and discoveries.

The potential of MAVEN

The Monitoring and Visualization of Energy consumed by Networks (MAVEN) project is a brand new prototype portal that will help network operators and researchers better track live network energy consumption and environmental conditions. MAVEN – implemented by Baris during his summer internship – is a first major step for ESnet in instrumenting our network with the tools to understand these operational dynamics. As networks continue to get bigger and faster, they will require more power and cooling in an era of decreased energy resources. To address this pressing challenge, ESnet is leading a new generation of research aimed at understanding how networks can operate in a more energy-efficient manner. We are grateful for Baris’ significant contributions in leading the development of MAVEN and glad to see that his talent is being recognized by the R&E networking community through this award.

Baris is now back in school at UCSD, completing his Ph.D in computer science. Congratulations Baris!

ESnet is going directly to the “light” source

Over the past few years, ESnet staff has worked to engage researchers across a variety of disciplines to ensure our services meet their needs. As a part of this ongoing effort, I will be leading a session titled “The Energy Sciences Network: Moving Data, Advancing Science” on October 4th at the Advanced Light Source User Meeting being held at Berkeley Lab. I hope to involve ALS users in a discussion around ways to use the ESnet network to more efficiently exchange and transfer their scientific data. The talk will also review some common problems encountered in the transfer of large scientific data sets, as well as highlight solutions showcasing how research groups have worked with ESnet to effectively use the network to expedite their data sharing and analysis.

ALS has a number of different experiments that can benefit from ESnet’s capabilities in different ways. Computing-intensive disciplines such as protein crystallography, X-ray microdiffraction, and other fields that utilize the ALS are experiencing unprecedented growth in data production. This is straining traditional methods of data distribution and analysis. As a matter of fact, within the next five years, we expect that many ALS user groups will experience an increase in data production of up to a factor of 1000, which will far exceed traditional digital media distribution capabilities. Without better ways of distributing data, scientific productivity could be significantly impeded.

In recent months, ESnet has developed new tools and software innovations as part of our goal of making our network and services more accessible to our users.  If you would like to get tips on how to use ESnet focused on your particular scientific discipline, or just have general questions, please contact us at

–Eli Dart