July | 2011 | Light Bytes: Blogging for Science

ESnet’s Inder Monga and Samrat Ganguly of NEC Corporation made a splash at the Summer 2011 ESCC/Internet2 Joint Techs conference in Fairbanks, Alaska by demonstrating some brand new ways that laboratories, universities, and industry can integrate end-to-end network virtualization across both the local area network (LAN) and wide area network (WAN). Check out the video of their demo “OpenFlow with OSCARS: Bridging the gap between campus, data centers and the WAN.”

Monga and Ganguly combined OpenFlow and On-Demand Secure Circuits and Advance Reservation System (OSCARS) as centralized controller technologies with NEC ProgrammableFlow switches to demonstrate automation and secure service provisioning that will enable scientists to share data and collaborate easier while easing the burden on their campus IT infrastructure staff.

Science networking already extensively employs aspects of network virtualization. Mismatches still exist between the campus networks, HPC data centers and R&E networks that require manual intervention and limit end-to-end automation and control. But OpenFlow-enabled campuses, through integration with OSCARS WAN capabilities, can be used to accomplish automated, end-to-end flow management capabilities, thus closing the gap. There was a need noted to define a higher layer API/Network Virtualization constructs that span both OSCARS and OpenFlow and lead to a end-to-end virtual networking model for applications.

At ESnet we are exploring the potential of OpenFlow because it enables network administrators to “program” flexible manipulation of flows, through a well-defined “forwarding instruction set”. Only certain traffic flows, selected by policies pre-programmed by local network administrators in the OpenFlow controller, seamlessly traverse the dynamic WAN virtual circuits enabled by OSCARS. This demonstrated approach creates a simple, secure and dynamic bridge between multiple OpenFlow enabled campuses, or datacenters instead of requiring statically (and manually) configured VLAN circuits used today.

OSCARS and NEC ProgrammableFlow

OSCARS enables the automated provisioning of guaranteed bandwidth over the WAN in multiple research and engineering around the world. OSCARS allows user applications to reserve bandwidth across multiple wide-area domains in advance and offers reliable end-to-end throughput, and quality of service in managing time-sensitive and large data sets.

NEC’s ProgrammableFlow switches and controller software leverage the OpenFlow protocol to automatically monitor and intelligently distribute network traffic across multiple paths, enabling more efficient use of network resources and multiplying available bandwidth within the network. ProgrammableFlow cuts the complexity as it provides a simple interface to implement complex virtualized networks.

A plug for our testbed

Networking researchers around the country can access OpenFlow’s capabilities on ESnet’s Advanced Networking Initiative (ANI) testbed using NEC ProgrammableFlow switches to test out advanced concepts in networking.

After much work and planning, we proudly introduce you to our new scientific network-to-be. Berkeley Lab just announced the signing of an agreement to begin construction of a 100 gigabits per second (Gbps) prototype network, part of the Lab’s Advanced Networking Initiative funded by the American Recovery and Reinvestment Act (ARRA). Under the terms of the deal, Internet2 and its industry partners will construct the network under ESnet’s direction.

Web — Coming Soon: ANI 100 Gpbs Prototype Network, First Step Towards Nationwide 100 Gbps Network

While our ANI network testbed is already open for business (mark your calendar –the next call for research proposals is October 1^st) in a matter of months researchers can conduct experiments on the 100 Gbps prototype network. This network will link the three supercomputing facilities at national laboratories – the National Energy Research Scientific Computing Center (NERSC), Oak Ridge Leadership Computing Facility (OLCF), and Argonne Leadership Computing Facility (ALCF) to MANLAN – the Manhattan Landing international exchange point.

But more is yet to come; the ANI prototype network will also be framework for ESnet to obtain concrete network energy use data for research into “green networking” – a priority here at ESnet. Also part of the agreement, Berkeley Lab negotiated access to a nation-wide pair of dark fiber for 20 years which will be immediately available to network researchers and industry to experiment with new protocols and disruptive technologies. Stay tuned to our blog to follow construction of the network and the exciting happenings at ESnet.

The 100 Gbps prototype network is just the beginning. Berkeley Lab and ESnet will leverage the experience gained deploying the prototype network to extend these capabilities to the national labs and facilities ESnet currently serves, and connect DOE scientists to university collaborators around the world. When moved to production status, the new network will increase the information carrying capacity of ESnet’s current 10 Gbps network by several orders of magnitude. One terabyte of data, which takes approximately 13 minutes to transfer on ESnet’s present 10 Gbps network, will be delivered in under a minute on the 100 Gbps network.

What we hope you will get out of this

While our network currently meets the capacity needs of its users, we see new challenges on the horizon. On behalf of the DOE Office of Science, we regularly survey scientists about their projected needs in order to better target services for our users. Demand on ESnet’s network has grown by a factor of 10 every 47 months since 1990. Over the last year, we have seen a rise in ESnet network traffic by 70 percent – most of that data traffic coming from the Large Hadron Collider at CERN but with genomics and climate data also picking up steam.

Just as investments in national infrastructure built the Interstate Highway System that sped the delivery of goods and opened up commerce in the United States, high-speed networks will speed the scientific discovery that will drive this nation’s economy in the future. New, large-scale instruments are in the offing, and we anticipate a growing tide of data in various disciplines as scientists work to understand our climate, develop clean fuels, and investigate the basic nature of matter. High-speed networks like this one will have a huge impact on an increasing number of disciplines, as more scientists collaborate in global teams and depend on remote supercomputers to model and solve complex problems.

How to be really thrifty with $62 million.

This is a challenging time when budgets are under severe pressure, both for government and the research and education community. We believe that by pooling resources and expertise we can help curb the costs of scientific research. By working closely with Internet2 to build this prototype network, we are leveraging our ARRA award and their funding for synergistic effect and getting more return on investment for the taxpayer.

We strongly believe this new 100Gbps network infrastructure will allow us to better support data-intensive science by providing more capacity at lower cost, lower energy consumption, and lower carbon emissions per bit. And that makes us feel good about what we do.