Six of the world’s leading research and education networks – ESnet, Internet2, NORDUnet, SURFnet, CANARIE and GÉANT – have announced their intent to build the world’s first 100 gigabits-per-second (Gbps) intercontinental transmission links for research and education.
The project, called the “Advanced North Atlantic 100G Pilot” or ANA-100G, is aimed at stimulating the market for 100 Gbps intercontinental networking and advancing global networks and applications to benefit research and education. In addition to ESnet (the U.S. Department of Energy’s Energy Sciences Network), the other participating networks are Internet2, NORDUnet (the Nordic Infrastructure for Research & Education), SURFnet (the Dutch National Research and Education Network), CANARIE (Canada’s Advanced Research and Innovation Network), and GÉANT (the high speed European communication network dedicated to research and education, operated by DANTE).
The partners are inviting other national research and education networks (NRENs) and their constituencies from around the world to participate in the project. The announcement was made April 24 at the 2013 Internet2 Annual Meeting before 800 technology, education and research leaders.
Some recent articles on new developments in virtual circuits such as Fenius and cloud computing with Google, Internet2’s announcements of its ION service, and the recently funded DYNES proposal are all powered by OSCARS or On Demand Secure Circuits and Reservation System, a software engine developed with DOE funding. This open-source software engine provides us with the capability of building a network with highly dynamic, traffic-engineered flows that meet the research data transport needs of scientists. The current deployed release, 0.5.2, has been deployed as a production service within ESnet for the past 3 years. We are currently enhancing 0.5.3 and plan to release the software in the Q4, 2010 time frame.
In the course of running this software as a production service and interacting with scientists, network researchers, and standards community at OGF, we realized we had to redesign the software architecture to be a much more robust and extensible platform. We wanted to be able to easily add new features to the OSCARS platform that would cater to a variety of network engineers and researchers. With this in mind, the re-architectured OSCARS is planned as release version 0.6. Like any successful product, transitioning from a deployed release to a new one involves thorny issues like backward compatibility and feature parity. Hence, the current balancing act of taking something that is quite good and proven (0.5.2), but making it even better a.k.a. 0.6.
Here are four good reasons why OSCARS 0.6 is the way to go:
1. It can meet production requirements: The modular architecture enables features to be added through the use of distinct modules. This allows specific deployment requirements to be easily integrated into the service. For example, if it is necessary to support a federated AA implementation, the AA modules can be replaced with ones that are compliant with that AA framework (e.g. Shibboleth). Another example would be High Availability (HA). The 0.6 architecture helps provide HA on a component basis, ensuring that the critical components do not fail.
2. It provides new complex features: As end-sites and their operators become comfortable with point to point provisioning of virtual circuits, we are getting increased requests for complex feature enhancements. The OSCARS 0.5 software architecture is not especially suitable for new features like multi-point circuits and/or multi-layer provisioning. But these new feature requests increase the urgency of moving to the 0.6 release that has been designed with such enhancements in mind. Moreover, the multi-layer ARCHSTONE research project funded by DOE will use 0.6 as the base research platform.
3. Research/GENI and other testbeds: The research community is a major constituent for OSCARS and its continuing development. This community is now conducting experiments on real infrastructure testbeds like the ANI and GENI. To really leverage the power of those testbeds, the research community wants to leverage the OSCARS software base/framework, while researching/innovating on certain algorithms and testing them. OSCARS 0.6 platform’s modular architecture enables the researcher to replace any component with new algorithmic research module. For example, with the new PCE engine re-design, one can write a flexible workflow of custom PCE’s. This flexibility does not exist with the purpose-built, but monolithic architecture of the OSCARS 0.5 codebase.
4. NSI Protocol/Standards: As the European and Asian research and education communities move towards interoperability with the US, it is important to leverage a common understanding brought through via standards. The NSI protocol standardization being discussed in the OGF NSI working group (http://ogf.org/gf/group_info/view.php?group=nsi-wg) needs to be implemented by the network middleware open source community like OSCARS. We feel that the 0.6 is the right platform to upgrade to the standard NSI protocol whenever it is ready.
At ESnet, we invest considerable time in new technology development, but balance this with our operational responsibilities. We invite the community to join in developing OSCARS 0.6, which has greatly improved capabilities over OSCARS 0.5.2. With your participation in the development process, we can accelerate the 0.6 architected software to production-quality as soon as possible. If this excites you, we welcome you to contribute to the next stage of the OSCARS open source project.
ESnet has been one of the leading research and education networks in the adoption of virtual circuit technology, which has allowed ESnet customers to sidestep traditional limitations of wide area networking and transfer data at high speed between geographically distant sites at a minimal cost. Each day, tens of terabytes of scientific data flow over ESnet’s Science Data Network between supercomputers, clusters, data storage sites, and experimental data sources like the LHC at CERN.
Essentially, virtual circuits provide an Ethernet pipeline with guaranteed bandwidth between two locations. This traffic is isolated from the rest, allowing our users to run “impolite” protocols like UDP, which would otherwise clog up their regular Internet connection. Our homegrown software code-named OSCARS, enables ESnet to easily monitor this traffic for trends and engineer its route to plan for growth and rearrange capacity according to the needs of our customers.
This is a win-win situation for both us and our customers, and we’re not alone in recognizing this. An increasing number of global research and education backbones and exchange points are deploying such services and writing their own software to manage them: Internet2 is providing the ION service (previously called DCN) based on the OSCARS platform. Across the Atlantic GEANT is developing AutoBAHN, and SURFnet is using Nortel’s DRAC. An international consortium developed Harmony under the Phosphorus project and is now starting up GEYSERS. In Japan, AIST has been developing the G-lambda suite, while Korean KISTI is in the process of coding their DynamicKL project – and there are certainly other projects out there.
Can’t we all just talk?
Now for the bad news: since there isn’t a globally accepted standard for this kind of service, the different software suites don’t quite communicate with one another. OSCARS communicates using the OSCARS application interface, DRAC uses the DRAC interface, and so forth. This, unfortunately, stymies our ambitions to automatically “stitch” virtual circuits across multiple networks. With everyone speaking a different language, this is impossible to accomplish.
A solution is to have a standard software interface; then different implementations would be able to interoperate as long as they were compliant. There is a standards effort in progress by the Open Grid Forum Network Services Interface working group, but an actual standard is probably at least several months away.
A bit of history
Several software developers made an effort to solve the interoperability issue at the GLIF meeting co-located at Joint Techs back in early 2008. After a few presentations, it became evident that all of these APIs, stripped of their cosmetic differences and special features, looked remarkably alike in terms of the raw pieces of information they handled. The consensus of the meeting was that there is no real reason not to have basic interoperability, even if many of the bells and whistles would be stripped. The developers then formed the GNI API task force under the umbrella of the GLIF Control Plane technical group, with the objective of duct-taping an interoperability solution together until actual standards emerged.
A mythical reference
They conceived the Fenius project, dubbed for the legendary king of Scythia, Fenius Farsaid. According to Irish folklore, after the collapse of the Tower of Babel, Fenius collected the best parts of the confused tongues of the world and invented a new language.
The Fenius Project is a fairly simple idea: it defines a bare-bones API for virtual circuit services as an interim pseudo-standard. Then developers can easily write code to automatically translate between the “standard” API and a specific software suite such as OSCARS; several translators already exist. The rest of the project is software “glue” which allows Fenius to run standalone, publishing its API as a web service, and routing incoming requests to the specific translator.
We demonstrated Fenius with good results during last year’s GLIF conference in Daejeon, Korea, as well as during Supercomputing 2009 in Portland, OR, using Fenius to provision virtual circuit services on demand across three networks – via completely different technologies, and two different software suites – from a lab in Japan to the NICT booth on the conference showfloor.
The next step for the project is to update its “standard” API according to some important lessons learned during last year’s demos, and to become the de facto external interface of production virtual circuit facilities. We plan to make an appearance at this year’s GLIF conference in Geneva, as well as in Supercomputing 2010 in New Orleans, LA. Fenius is also slated to become a component of OpenDRAC soon. http://www.opendrac.org/.
We hope that Fenius will be able to provide ESnet customers and the international research and education community wider access to the network infrastructure, and that it will enable virtual circuits to become a truly global infrastructure capability in the service of science, research, and education worldwide.
When organizations are driven by the similar goal of excellence, it is amazing what can be accomplished in two days of non-stop discussions. As I try to consolidate my thoughts from meetings the past two days, the following quotation rings particularly true.
Coming together is a beginning, staying together is progress, and working together is success. ~ Henry Ford
There is still a lot of work to be done, milestones to be hit and unforeseen roadblocks to overcome before we give ourselves a pat on the back, but we are off to a good start.
For the perpetually curious, SURFnet, ESnet and NORDUnet stated their intentions to collaborate on furthering innovation in network research by working on open-source middleware to reserve and seamlessly allocate bandwidth across multiple domains (http://www.lbl.gov/cs/Archive/news030910.html). While our first planned meeting was delayed by volcano ash from Iceland, skies and schedules cleared enough for the Dutch to visit us at our facilities at Berkeley lab. This week’s two day meeting of minds is a baby step taken towards an ambitious goal of open sharing of tools, knowledge, open-source software and other network-related research with the community.
We did let our visitors out of the conference room for just few minutes to enjoy the beautiful, albeit faint, backdrop of the Golden Gate Bridge.