Light Bytes – a new name for the ESnet Blog

You may have noticed that the masthead has changed.  After almost a decade, we have finally decided that the “ESnet Blog” deserves a less literal name. “Light Bytes” was selected to better embody two things about which we at ESnet are especially proud.  

First, getting to build the world’s greatest research and education network, and to support global science is a great honor and a technical challenge.  Through ESnet6 and our continuing research, we are advancing our mission of making scientific data free of geographical constraints, to “make bytes light” in terms of fast transport, and to deploy state of the art optical network for “bytes being transported by light.”

Second, while ESnet is officially a DOE User Facility, it is most importantly a remarkable group of people. We hope that this website will show a bit about the great people who make our mission happen and the interesting problems we get to work on.  In that sense, “Light Bytes” is a small written offering, a collection of features about things that are happening while the cause of “networking for science” and “science of networking” progresses. 

Looking back at ESnet’s 2020

Advancing our strategy and shaping our position on the board.
Some thoughts from Inder on the year-that-was.

Miniature from Alfonso X’s Libro del axedrez dados et tablas (Book of chess, dices and tables), c. 1283. , Public domain, via Wikimedia Commons

Dear Friends, Well-wishers, Colleagues, and all of ESnet,

Chess! 2020 has been much more challenging than this game. It’s also been a year where we communicated through the squares on our zoom screens, filled with faces of our colleagues, collaborators, and loved ones.

In January, Research and Education leaders came together in Hawaii at the Pacific Telecommunications Council meeting to discuss the future of networking across the oceans. It was impossible to imagine then that we would not be able to see each other again for such a long time. Though thanks to those underwater cables, we have been able to communicate seamlessly across the globe.

Looking back at 2020, we not only established a solid midgame position on our ESnet chessboard, but succeeded in ‘winning positions’ despite the profound challenges. The ESnet team successfully moved our network operations to be fully remote (and 24/7) and accomplished several strategic priorities. 

ESnet played some really interesting gambits this year: 

  1. Tackled COVID-related network growth and teleworking issues for the DOE complex
    • We saw a 4x spike in remote traffic and worked closely across several Labs to upgrade their connectivity. We continue to address the ever-growing demand in a timely manner. 

    • As we all shifted to telework from home, ESnet engineers developed an impromptu guide that was valuable to troubleshoot our home connectivity issues. 
  2. Progressed greatly on implementing our next-generation network, ESnet6
    • We deployed and transitioned to the ESnet6 optical backbone network, with 300 new site installations, 100’s of 100G waves provisioned, with just six months of effort, and while following pandemic safety constraints. I am grateful to our partners Infinera (Carahsoft) and Lumen for working with our engineers to make this happen. Check out below how we decommissioned the ESnet5 optical network and lit up the ESnet6 network.
    • Installed a brand new management network and security infrastructure upgrades along with significant performance improvements.
    • We awarded the new ESnet6 router RFP (Congratulations Nokia and IMPRES!); the installs start soon.
    • Issued another RFP for optical transponders, and will announce the winner shortly.
  3. Took initiative on several science collaborations to address current and future networking needs
    • We brainstormed new approaches with the Rubin Observatory project team, Amlight, DOE and NSF program managers to meet the performance and security goals for traffic originating in Chile. We moved across several countries in South America before reaching the continental U.S. in Florida (Amlight), and eventually the U.S. Data Facility at SLAC via ESnet.
    • Drew insights through deep engagement of ESnet engineers with the High Energy Physics program physicists, for serving the data needs of their current and planned experiments expediently.
      Due to the pandemic, a two-day immersive in-person meeting turned into a multi-week series of Zoom meetings, breakouts, and discussions.
    • When an instrument produces tons of data, how do you build the data pipeline reliably? ESnet engineers took on this challenge, and worked closely with the GRETA team to define and develop the networking architecture and data movement design for this instrument. This contributed to a successful CD 2/3 review of the project—a challenging enough milestone during normal times, and particularly tough when done remotely. 
    • Exciting opening positions were created with EMSL, FRIB, DUNE/SURF, LCLS-II…these games are still in progress, more will be shared soon. 
  4. Innovated to build a strong technology portfolio with a series of inspired moves
    • AI/ML
      • We demonstrated Netpredict, a tool using deep learning models and real-time traffic statistics to predict when and where the network will be congested. Mariam’s web page showcases some of the other exciting investigations in progress. 
      • Richard and his collaborators published Real-time flow classification by applying AI/ML to detailed network telemetry.
    • High-touch ESnet6 project
      • Ever dream of having the ability to look at every packet, a “packetscope”, at your fingertips? An ability to create new ways to troubleshoot, performance engineer, and gain application insights? We demonstrated a working prototype of that vision at the SC20 XNET workshop
    • SENSE
      • We deployed a beta version of software that provides science applications the ability to orchestrate large data flows across administrative domains securely. What started as a small research project five years ago (Thanks ASCR!) is now part of the AutoGOLE project initiative in addition to being used for Exascale Computing Project (ECP) project, ExaFEL.
    • TCP
      • Initiated the Q-Factor project this year, a research collaboration with Amlight, funded by NSF. The project will enable ultra-high-speed data transfer optimization by TCP parameter tuning through the use of programmable dataplane telemetry: https://q-factor.io/
      • We testbed thoroughly the interactions between TCP congestion control algorithms, BBRv2 and CUBIC. A detailed conversation with Google, the authors of the BBRv2 implementation, is in progress.
  5. Initiated strategic new games, with a high potential for impact
    • FABRIC/FAB
      • Executed on the vision and design of a nationwide @scale research testbed working alongside a superstar multi-university team.
      • With the new FAB grant, FABRIC went international with plans to put nodes in Bristol, Amsterdam, Tokyo and Geneva. More locations and partners are possibilities for the future.  
    • Edge Computing
      • Created an prototype FPGA-based edge-computing platform for data-intensive science instruments in collaboration with the Computational Research Division and Xilinx. Look for exciting news on the blog as we complete the prototype deployment of this platform.
    • Quantum
    • 5G
      • What are the benefits of widespread deployment of 5G technology on science research? We contributed to the development of this important vision at a DOE workshop. New and exciting pilots are emerging that will change the game on how science is conducted. Stay tuned. 

Growth certainly has its challenges. But, as we grew, we evolved from our old game into an adept new playing style. I am thankful for the trust that all of you placed in ESnet leadership, vital for our numerous, parallel successes. Our 2020 reminds me of the scene in Queen’s Gambit where the young Beth Harmon played all the members of a high-school chess team at the same time. 

Several achievements could not make it to this blog, but are important pieces on the ESnet chess board. They required immense support from all parts of ESnet, CS Area staff, Lab procurement, Finance, HR, IT, Facilities, and Communications partners.

I am especially grateful to the DOE Office of Science, Advanced Scientific Computing Research leadership, NSF, and our program manager Ben Brown, whose unwavering support has enabled us to adapt and execute swiftly despite blockades. 

All this has only been possible due to the creativity, resolve, and resilience of ESnet staff — I am truly proud of each one of you. I am appreciative of the new hires that trusted their careers with us and joined us remotely—without shaking hands or even stepping foot at the lab.

My wish is for all to stay safe this holiday season, celebrate your successes, and enjoy that extra time with your immediate family. In 2021, I look forward to killer moves on the ESnet chessboard, while humanity checkmates the virus. 

Signing off for the year, 

Inder Monga

Faucet – Enterprise SDN from an office to multi-terabit SCinet at SC18

This post is written by ESnet Director, Inder Monga, and shared through a personal lens, other perspectives may vary

I wanted to share a remarkable effort happening in the field of networking that is going to be shown in production at the Supercomputing 2018 as part of SCinet. Most of the people working in the SC booths (or even at SCinet) may never realize the role they are playing in moving the field of networking to the next level. Let me share some history [and you can skip the Background section to go straight to the main essence of the blog], and talk about the international, collaborative effort at play here.

Background: my software-driven journey

This is being shared by the perspective of the writer’s journey, other perspectives may vary

From the early 2000’s, the dream that excited me was building software layer that would manage networks and expose the network black-box as a set of ‘knobs and dials’ that applications could use to request and customize for their purposes. A small group of us at Nortel Networks came up with an idea of CO2 aka Content over Optics in 2001 – see the slide below.

This excitement was very new and led to my most productive set of patents filed during that time. Just as the concept was taking hold, an innovative research and education network, SURFnet, decided to deploy this concept on their newly built network, and the name evolved from CO2 to DRAC aka Dynamic Resource Allocation Controller.

Independently, ESnet was on a similar journey and started working on a concept called OSCARS aka ‘On-demand Secure Circuits and Reservation System’. As the community coalesced on the same direction, multiple initiatives like OSCARS, UCLP, DRAC, MANTICORE and others started merging towards an aligned software-driven network journey for the WAN. In the meantime, the Openflow effort from Stanford driven by an enterprise-based architecture, Ethane, captured the hearts of many, and the ‘Software-Defined Network’ (SDN) wave was born.


I am going to skip discussing the twists and turns of the evolution of SDN in industry, where there have been many recent articles on whether OpenFlow is dead. In fact there is a grassroots and under-the-radar SDN/Openflow effort that started in New Zealand called Faucet that proves these concepts are still alive and well, and are poised to thrive! In fact this work recently won the award in the New Zealand Open Source Software Project Award.

IMG_5690

Faucet at ESnet

Ever since I was invited to join the Faucet Foundation Board, I was insistent that I ‘eat my own dog food’. With the help of Josh Bailey and Simeon Miteff, I began running a faucet controlled switch (OpenFlow is not dead) in my office as my only bridge into the Lab’s network and the Internet, i.e. if faucet did not work, I could not get connected! I am proud to say that other than one small self-inflicted upgrading glitch, the system has been working seamlessly over the past year no matter what I do or use in the office. You can see the couple of raspberry PIs running the SDN controller working with a commercial off-the-shelf switch in the image below.

[I wish it was cleaner picture, but I had to get everything together to click a photograph without my desk in the way.]

Then my colleague, Nick Buraglio, and I decided, why not wire our satellite branch office with faucet based networking than buy traditional switches? With Josh (Bailey)’s help, Nick has now wired the lives of eight of ESnet’s key employees to be dependent on faucet and he has been having a blast (no downtime there either)! Read more about his journey on his own private blog: http://www.forwardingplane.net/2018/11/faucet-enterprise-openflow-in-production/

Faucet at SCinet, SC18

I may have led you on and will keep the suspense on for a bit longer….Faucet is being showcased in SCinet as production SDN controller managing a portion of the booth networks. SC18 is in two weeks and I encourage all of you to stop by SCinet to see it working! Reach out to the faucet champions – Josh Bailey, Brad Cowie, Richard Nelson, Nick Buraglio and Kate Mace who can talk you through the intricacies of the faucet deployment there. Poseidon, as you see printed on one of the raspberry pi’s in the image above, is also being featured at SC as it integrates learning and security with faucet.

More details on this topic will be available after SC is over. In the meantime, do check out Nick’s blog for the engineering details. An image from the recent multi-vendor plugfest at SCinet staging in Dallas to – yes, vendors support this.

Look forward to seeing you all in Dallas as all of us continue on this journey

Inder

ESnet’s Monga Keynotes Two R&E Network Workshops

I’ll be busy in Brazil next week. Sharing the newsletter article about it with the Blog readers:

ESnet’s Monga Keynotes Two R&E Network Workshops

ESnet’s Chief Technology Officer Inder Monga will keynote two workshops and participate in a panel focusing on research and education (R&E) networks in Brazil next week.

 On May 18, Monga opens the National Research and Education Network Workshop (WRNP) hosted by ESnet’s Brazilian counterpart. RNP logo imagesIn his talk, entitled “R&E Networks: Imagining the next generation,” Monga will focus on new ideas in R&E networks, from technologies like software defined networking (SDN) and named data networking (NDN) to collaborative architectures to build an internet of different capabilities for global science collaborations. He will also showcase the challenges R&E networks face and focus on enabling end-to-end architectures, including concepts like the Science DMZ.

On May 22, Monga opens the Experimental Research Workshop of the Future Internet (WPIEF). His keynote is entitled “Moving from SDN demo to operations: Challenges.”

Both workshops are held as part of the Brazilian Symposium on Computer Networks  and Distributed Systems (SBRC) conference where Monga will contribute to a May 19 panel discussing “Challenges in the Development of Network Infrastructure, Testbeds for Software Defined Networks.”

Vote for ESnet5 – a good steward of Petabytes

It has almost been a year since we turned 25, and transferred a “whole universe of data” at Supercomputing 2011 – and that was over a single 100G link between NERSC and Seattle. Now we are close to the end of building out the fifth generation of our network, ESnet5.

In order to minimize the downtime for the sites, we are building ESnet5 parallel to ESnet4, with just a configuration-driven switch of traffic from one network to the other. Since the scientific community we serve depends on the network to be up, it’s important to have assurance that the transition is not disruptive in anyway. The question we have heard over and over again from some of our users – when you switch the ESnet4 production traffic to ESnet5, how confident are you that the whole network will work, and not collapse?

In this blog post, I’d like to introduce an innovative testing concept the ESnet network engineering team (with special kudos to Chris Tracy) developed and implemented to address this very problem.

The goal of our testing was to ensure that the entire set of backbone network ports would perform solidly at full 100 Gbps saturation with no packet loss, over a 24 hour period. However we had some limitations. With only one Ixia test-set with 100 GE cards at hand to generate and receive packets and not enough time to ship that equipment to every PoP and test each link, we had to create a test scenario that would generate confidence that all the deployed routers and optical hardware, optics, the fiber connections, and the underlying fiber would performing flawlessly in production.

This implied creating a scenario where the 100 Gbps traffic stream being generated by the Ixia would be carried bi-directionally over every router interface deployed in ESnet5, traverse it only once and cover the entire ESnet5 topology before being directed back to the test hardware. A creative traffic loop was created that traversed the entire footprint, and we called it the ‘Snake Test’. Even though the first possible solution was used to create the ‘snake’, I am wondering if this could be framed as a NP-hard theoretical computer science and optimization approach known as the traveling salesman problem for more complex topologies?

The diagram below illustrates the test topology:

So after sending around 1.2 petabytes of data in 24 hours, and accounting for surprise fiber maintenance events that caused the link to flap, the engineering team was happy to see a zero loss situation.

Here’s a sample portion of the data collected:

Automation is key – utility scripts had been built to do things like load/unload the config from the routers, poll the firewall counters (to check for loss ingress/egress at every interface), clear stats, parse the resulting log files and turn them into CSV (a snapshot you see in the picture) for analysis.

Phew! – the transition from ESnet4 to ESnet5 continues without a hitch. Watch out for the completion news, it may come quicker than you think…..

On behalf of the ESnet Engineering team

ESnet unveils new dashboard showcasing IPv6 status of its connected sites

Image

Just last month our resident IPv6 expert, Mike Sinatra, discussed the Risks of not deploying IPv6 in the R&E Community. On World IPv6 Launch, ESnet is happy to unveil a simple dashboard that tracks the status of IPv6 deployment across its sites. This page is updated based on summary of tests performed by a v6 connected host within ESnet.

Do visit the page at – https://my.es.net/sites/ipv6

Our policy board member, Vint Cerf, also released a compelling video on why we need IPv6, a must-watch.

Happy World IPv6 Launch!

ECSEL leverages OpenFlow to demonstrate new network directions

ESnet and its collaborators successfully completed three days of demonstrating its End-to-End Circuit Service at Layer 2 (ECSEL) software at the Open Networking Summit held at Stanford a couple of weeks ago. Our goal is to build “zero-configuration circuits” to help science applications seamlessly use networks for optimized end-to-end data transport. ECSEL, developed in collaboration with NEC, Indiana University, and the University of Delaware builds on some exciting new conceptual thinking in networking.

Wrangling Big Data 

To put ECSEL in context, the proliferating tide of scientific data flows – anticipated at 2 petabytes per second as planned large-scale experiments get in motion – is already challenging networks to be exponentially more efficient. Wide area networks have vastly increased bandwidth and enable flexible, distributed, scientific workflows that involve connecting multiple scientific labs to a supercomputing site, a university campus, or even a cloud data center.

Heavy network traffic to come

The increasing adoption of distributed, service-oriented computing means that resource and vendor independence for service delivery is a key priority for users. Users expect seamless end-to-end performance and want the ability to send data flows on demand, no matter how many domains and service providers are involved.  The hitch is that even though the Wide Area Network (WAN) can have turbocharged bandwidth, at these exponentially increasing rates of network traffic even a small blockage in the network can seriously impair the flow of data, trapping users in a situation resembling commute conditions on sluggish California freeways. These scientific data transport challenges that we and other R&E networks face are just a taste of what the commercial world will encounter with the increasing popularity of cloud computing and service-driven cloud storage.

Abstracting a solution

One of the key feedback from application developers, scientists and end-users is that they do not want to deal with the complexity at the infrastructure level while still accomplishing their mission. At ESnet, we are exploring various ways to make networks work better for users. A couple of concepts could be game-changers, according to Open Network Summit conference presenter and Berkeley professor Scott Shenker: 1) using abstraction to manage network complexity, and 2) extracting and exposing simplicity out of the network. Shenker himself cites Barbara Liskov’s Turing Lecture as inspiration.

ECSEL is leveraging OSCARS and OpenFlow within the Software Defined Networking (SDN) paradigm to elegantly prevent end-to-end network traffic jams.  OpenFlow is an open standard to allow application-driven manipulation of network flows. ECSEL is using OSCARS-controlled MPLS virtual circuits with OpenFlow to dynamically stitch together a seamless data plane delivering services over multi-domain constructs.  ECSEL also provides an additional level of simplicity to the application, as it can discover host-network interconnection points as necessary, removing the requirement of applications being “statically configured” with their network end-point connections. It also enables stitching of the paths end-to-end, while allowing each administrative entity to set and enforce its own policies. ECSEL can be easily enhanced to enable users to verify end-to-end performance, and dynamically select application-specific protocol forwarding rules in each domain.

The OpenFlow capabilities, whether it be in an enterprise/campus or within the data center, were demonstrated with the help of NEC’s ProgrammableFlow Switch (PFS) and ProgrammableFlow Controller (PFC). We leveraged a special interface developed by them to program a virtual path from ingress to egress of the OpenFlow domain. ECSEL accessed this special interface programmatically when executing the end-to-end path stitching workflow.

Our anticipated next step is to develop ECSEL as an end-to-end service by making it an integral part of a scientific workflow. The ECSEL software will essentially act as an abstraction layer, where the host (or virtual machine) doesn’t need to know how it is connected to the network–the software layer does all the work for it, mapping out the optimum topologies to direct data flow and make the magic happen. To implement this, ECSEL is leveraging the modular architecture and code of the new release of OSCARS 0.6.  Developing this demonstration yielded sufficient proof that well-architected and modular software with simple APIs, like OSCARS 0.6, can speed up the development of new network services, which in turn validates the value-proposition of SDN. But we are not the only ones who think that ECSEL virtual circuits show promise as a platform for spurring further innovation. Vendors such as Brocade and Juniper, as well as other network providers attending the demo were enthusiastic about the potential of ECSEL.

But we are just getting started. We will reprise the ECSEL demo at SC11 in Seattle, this time with a GridFTP application using Remote Direct Memory Access (RDMA) which has been modified to include the XSP (eXtensible Session Protocol) that acts as a signaling mechanism enabling the application to become “network aware.”  XSP, conceived and developed by Martin Swany and Ezra Kissel of Indiana University and University of Delaware,  can directly interact with advanced network services like OSCARS – making the creation of virtual circuits transparent to the end user. In addition, once the application is network aware, it can then make more efficient use of scalable transport mechanisms like RDMA for very large data transfers over high capacity connections.

We look forward to seeing you there and exchanging ideas. Until Seattle, any questions or proposals on working together on this or other solutions to the “Big Data Problem,” don’t hesitate to contact me.

–Inder Monga

imonga@es.net

ECSEL Collaborators:

Eric Pouyoul, Vertika Singh (summer intern), Brian Tierney: ESnet

Samrat Ganguly, Munehiro Ikeda: NEC

Martin Swany, Ahmed Hassany: Indiana University

Ezra Kissel: University of Delaware

Idea Power: Two ESnet Projects are Honored With Internet2 IDEA Awards

We are proud to announce that two of ESnet’s projects have received IDEA (Internet2 Driving Exemplary Applications) awards in Internet2’s 2011 annual competition for innovative network applications that have had the most positive impact and potential for adoption within the research and education community. (see: Internet2’s press release).

Internet2 recognized OSCARS (On-Demand Secure Circuits and Advance Reservation System), developed by the ESnet team led by Chin Guok, including Evangelos Chaniotakis, Andrew Lake, Eric Pouyoul and Mary Thompson. Contributing partners also included Internet2, USC ISI and DANTE.

ESnet’s MAVEN (Monitoring and Visualization of Energy consumed by Networks) proof of concept application was also recognized with an IDEA award in the student category. MAVEN was prototyped by Baris Aksanli during his summer internship at ESnet. Baris is a Ph.D student at the University of California, San Diego conducting research at the System Energy Efficiency Lab with his thesis advisor, Dr. Tajana Rosing. Baris worked closely with his summer advisor, Inder Monga, and Jon Dugan to implement MAVEN as part of ESnet’s new Green Networking Initiative.

The idea behind OSCARS

OSCARS enables researchers to automatically schedule and guarantee end-to-end delivery of scientific data across networks and continents. For scientists, being able to count on reliable data delivery is critical as scientific collaborations become more expansive, often global. Meanwhile, in disciplines ranging from high-energy physics to climate, scientists are using powerful, geographically dispersed instruments like the Large Hadron Collider that are producing increasingly massive bursts of data, challenging the capabilities of traditional IP networks.

OSCARS virtual circuits can reliably schedule time-sensitive data flows – like those from the LHC – round the clock across networks, enabling research and education networks to seamlessly meet user needs. OSCARS code is also being deployed by R&E networks worldwide to support an ever-growing user base of researchers with data-intensive collaboration needs. Internet2, U.S. LHCnet, NORDUNet, RNP in Brazil as well as over 10 other regional and national networks have currently implemented OSCARS for virtual circuit services. Moreover, Internet2’s NSF-funded DyGIR and DYNES projects will in 2012 deploy over 60 more instances of OSCARS at university campuses and regional networks to support scientists involved in LHC, Laser Interferometer Gravitational-Wave Observatory (LIGO), Large Synoptic Survey Telescope (LSST) and Electronic Very-Long Baseline Interferometry (eVLBI) programs.

We are proud of the hard work and dedication the OSCARS development team has demonstrated since the start of this project. Just as importantly we are proud to see this work paying off in with new science collaboration and discoveries.

The potential of MAVEN

The Monitoring and Visualization of Energy consumed by Networks (MAVEN) project is a brand new prototype portal that will help network operators and researchers better track live network energy consumption and environmental conditions. MAVEN – implemented by Baris during his summer internship – is a first major step for ESnet in instrumenting our network with the tools to understand these operational dynamics. As networks continue to get bigger and faster, they will require more power and cooling in an era of decreased energy resources. To address this pressing challenge, ESnet is leading a new generation of research aimed at understanding how networks can operate in a more energy-efficient manner. We are grateful for Baris’ significant contributions in leading the development of MAVEN and glad to see that his talent is being recognized by the R&E networking community through this award.

Baris is now back in school at UCSD, completing his Ph.D in computer science. http://cseweb.ucsd.edu/~baksanli/. Congratulations Baris!

ESnet 2010 Round-up: Part 2

Our take on ANI, OSCARS, perfSONAR, and the state of things to come.

ANI Testbed

In 2010 ESnet led the technology curve in the testbed by putting together a great multi-layer design, deploying specially tuned 10G IO Testers, became early investors in the Openflow protocol by deploying the NEC switches, and built a research breadboard of end-hosts leveraging open-source virtualization and cloud technologies.

The first phase of the ANI testbed is concluding. After 6+ months of operational life, with exciting research projects like ARCHSTONE, Flowbench, HNTES, climate studies, and more leveraging the facilities, we are preparing to move the testbed to its second phase on the dark fiber ring in Long Island. Our call for proposals that closed October 1st garnered excellent ideas from researchers and was reviewed by the academic and industry stalwarts in the panel. We are tying up loose ends as we light the next phase of testbed research.

OSCARS

This year the OSCARS team has been extremely productive. We added enhancements to create the next version (0.5.3) of currently production OSCARS software, progressed on architecting and developing a highly modular and flexible platform for the next-generation OSCARS (0.6), a PCE-SDK targeted towards network researchers focused on creating complex algorithms for path computation, and developing FENIUS to support the GLIF Automated GOLE demonstrator.

Not only did the ESnet team multitask on various ANI, operational network and OSCARS deliverables, it also spent significant time supporting our R&E partners like Internet2, SURFnet, NORDUnet, RNP and others interested in investigating the capabilities of this open-source software. We also appreciate Internet2’s participation by dedicating testing resources for OSCARS 0.6 starting next year to ensure a thoroughly vetted and stable platform during the April timeframe. This is just one example of the accomplishments possible for the R&E community by commiting to partnership and collaboration.

perfSONAR collaboration

perfSONAR kept up its rapid pace of feature additions and new releases in joint collaboration with Internet2 and others. In addition to rapid progress in software capabilities, ESnet is aggressively rolling out perfSONAR nodes in its 10G and 1G POPs, creating an infrastructure where the network can be tuned to hum. With multiple thorny network problems now solved, perfSONAR has proven to be great tool delivering value. This year we focused on making perfSONAR easily deployable and adding the operational features to transform it into a production service. An excellent workshop in August succinctly captured the challenges and opportunities to leverage perfSONAR for operational troubleshooting and also by researchers in understanding further how to improve networks. Joint research projects continue to stimulate further development with a focus on solving end-to-end performance issues.

The next networking challenge?

2011

Life in technology tends to be interesting, even though people keep warning about the commoditization of networking gear. The focus area for innovation just shifts, but never goes away.  Some areas of interest as we evaluate our longer term objectives next year:

  • Enabling the end-to-end world: What new enhancements or innovations are needed to deploy performance measurement, and control techniques to enable a seamless end-to-end application performance?
  • Life in a Terabit digital world: What network innovations are needed to fully exploit the requirement for Terabit connectivity between supercomputer centers in the 2015-2018 timeframe?
  • Life in a carbon economy: What are the low-hanging fruit for networks to become more energy-efficient and/or enable energy-efficiency in the IT ecosystem they play? Cloud-y or Clear?

We welcome your comments and contributions,

Happy New Year

Inder Monga and the folks at ESnet

ESnet 2010 Round-up: Part 1

Part 1: Considering the state of 100G and the state we’re in

The past year slipped by at a dizzying pace for us at ESnet, as we made new forays into cutting-edge technologies. In this two-part blogpost, we will recap accomplishments of the year, but also consider the challenges facing us in the one to come as we progress towards delivering the Advanced Networking Initiative.

Stimulating 100G

One of our prime directives with ANI funding was to stimulate the 100G market towards increasing spectral efficiency. In the last year, we have had wonderful engagement with the vendors that are moving products in this direction. Coherent receivers and DP-QPSK modulation are now standard fare for the 40G/100G solutions. At the latest conference, IEEE ANTS, in Mumbai last week, the 100G question was considered solved. Researchers are now exploring innovative solutions to deliver a next generation of 100G with higher power efficiency, or jump to the next level in delivering 400G. One researcher at the Indian Institute of Technology, Mumbai, is looking at revolutionizing the power consumption curve of the digital processing paradigm of coherent solutions by investigating analog processing techniques (super secret, so we will just have to wait and see).

IEEE-ANTS Conference

A representative from OFS, the optical fiber company, described research on new fibers which cater to the coherent world that will enable better performance. He quoted hero experiments, papers, and research presented at this years’ OFC, touting the advantages of new fiber captured through the joint work of Alcatel-Lucent Bell Labs and OFS (ex-Lucent) research collaborators. There is a lot of fiber still being laid out in the developing countries and they are well positioned to take advantage of this new research to bring cheaper broadband connectivity in so far underserved communities.

Some selected points raised at the panel regarding 400G and beyond:

  • Raman amplification is coming back in vogue
  • 50GHz ITU-Grid needs to evolve to flexi-grid technology. With flexi-grid, some of the basic modem concepts of negotiation (remember the auto-sensing modems of late 90’s) is back – where based on distance, and loss, the appropriate grid spacing can be negotiated for each wavelength.
  • If the industry sticks with electrical compensation, optical equipment will see increased electricity consumption by the power-hungry Analog-Digital Conversion (ADC) and Digital Signal Processing (DSP) ASICS. With advances in CMOS, the status quo might not suffice in a few years, especially since the whole industry is out there sticking the router vendors with a big “power-hungry” sticker. The equations in power-consumption tradeoffs still need to be studied and appropriate comparisons made. I hope the vendors also develop a perspective in that direction.
  • Comcast, the only other vendor on the panel, mentioned current capacities of 30x40G (coherent) on some links of their backbone and their eagerness to deploy 100G solutions. They are WAY ahead in deploying 100G, though the industry seems to not broadcast such news widely.
  • Comcast felt that coherent optics in the Metro Area is overkill and entreated the vendors not to build one-size-fits-all solutions even if simpler (and, they hope making 100G more affordable, as well).
  • Speeding into next year

    There was little discussion on the 100GE standards, although there was a clear message that LR-10 is here to stay, mainly supported by Data Center customers, though almost all traditional carriers intend deploy LR-4, in the case it starts costing less than a Ferrari.

At Supercomputing 2010, the SCinet community orchestrated and deployed 100G-capable equipment from Brocade, Cisco, Ciena and Juniper, to name a few vendors, and included 100G host demonstrations of data transfers by NASA. It was encouraging to see the Academic and R&E community lead deployment and testing of 100G [See a sample poster below].

100G Testing by German Academia

The SCinet community lives on the “bleeding edge” and supported a demonstration by Internet2, ESnet, and other partners carrying live 100Gbps application data over a 100G wave from Chicago to New Orleans show floor.

We are looking forward to Seattle (SC11) and can already predict multiple 100G’s of bandwidth coming towards the show floor – if you have any cool demonstrations that you would like to collaborate with us, please drop us a note.

— Inder Monga