CERN’s Large Hadron Collider Begins Run 3 

Today, the world’s most powerful particle accelerator, the Large Hadron Collider (LHC), begins Run 3, a new period of data taking! This comes after more than three years of upgrade and maintenance work.

Image: CERN

ESnet is looking forward to continuing to support the US LHC community for Run 3 through its connectivity to CERN. ESnet carries all traffic for LHC from Europe to the US, to the two Tier 1 centers at Fermilab and Brookhaven National Lab, and for all the physicists in Tier 2 centers in Universities. You can see the list of LHCONE collaborators at my.es.net/lhcone/list.

View the traffic on our trans-Atlantic links live at my.es.net.

Check out CERN’s livestream of the event here.

We want YOU  for #ESnet6Week!

Registration is now open for the unveiling of the ESnet6 network, Confab22 and other events planned for #ESnet6Week! Come join us either in person in Berkeley, CA or virtually.  

#ESnet6Week Schedule (October 11-14, 2022):

  • October 11: ESnet6 Unveiling – the Next-Generation Network (Free to attend virtually. In-person attendance is invite only)
  • October 12-13 : Confab22 – the Annual User Meeting ($300 to attend in-person, $150 to attend virtually – save $50 if you register before 30 June!)
  • October 14 : ESnet Site Coordinator Committee Meeting (open to ESCC invitees only)

To register or for more information, please see: https://go.lbl.gov/confab22!

ESnet is heading to Italy to share networking expertise with international colleagues at TNC22

ESnet will be in Trieste, Italy from June 13-17, 2022 sharing their knowledge about software-defined networking at TNC22, the research and education networking conference.

Here’s a look at where to find ESnet at the conference: 

Wednesday, 6/15

CEO Roundtable
Wednesday 6/15 – 8:30-16:30 CEST | Room B
Invitation only

ESnet executive director Inder Monga will join 70 CEOs from research networks around the world for a conversation about best practices, advancements in technology, and the future of scientific networking. 

Data Mobility Exhibition
Wednesday 6/15 – 10:30-11:00 CEST | Demonstration Area
Presenter: Eli Dart

Data mobility is a critical component of the process of science. Being able to predictably and efficiently move scientific data between experimental source, processing facilities, long term storage, and collaborators is a common use case that transcends the boundaries of research disciplines. The demo transfers reference data sets from well-tuned Data Transfer Nodes (DTNs) to existing campus CI components. Participants will upload, download, measure, and potentially improve their scientific data movement capabilities as well as review previous tests.

Thursday 6/16

Session: Real Life Network Orchestration

Building a Realistic Orchestration Validation Environment for netwoRks (ROVER)
Thursday 6/16 – 09:00 – 10:30 CEST | Auditorium
Speakers: Chris Cummings, Nick Buraglio 

Building software that controls network equipment has many similarities to traditional software engineering, however, testing this software introduces many complexities unique to the network orchestration world. Join this talk to learn how we approached these challenges by building a Realistic Orchestration Validation Environment for netwoRks (ROVER) at ESnet.

Session: Data Driven Networking

Superfacility – A Blueprint for Supporting Large-Scale Scientific Workflows
Thursday 6/16 – 11:00 – 12:30 CEST | Auditorium
Speaker: Chin Guok

Data-intensive workflows have been a mainstay of large scale collaborative scientific research for several decades. However, with the growing exponential rate of data that instruments can produce today, there needs to be a paradigm shift in how distributed resources are requested and utilized across the various facilities that contribute to these data-intensive workflows. A “Superfacility” model, which seamlessly integrates instrument, computing, storage, and networking facilities, is required to more effectively support these increasingly demanding workflows. This talk will present on the desired characteristics for a Superfacility, discuss specific use cases, and highlight some activities and initiatives in this area.

Exploring the BBRv2 Congestion Control Algorithm for use on Data Transfer Nodes
Thursday 6/16 – 11:00 – 12:30 CEST | Auditorium
Speaker: Eli Dart 

It is well known that loss-based TCP congestion control algorithms are problematic for high-speed, high-latency flows that are common in Big Science. In 2016 Google released a new congestion control algorithm called ‘BBR’ (Bottleneck Bandwidth and Round-trip time) that uses a model-based approach, and the design has since been refined in an alpha release of BBRv2. In this paper, we describe and perform a set of experiments that assess the suitability of BBRv2 for use on Data Transfer Nodes (DTNs).


Registration deadlines for TNC are coming up! Registration for regular in-person passes is open until June 6 (unless you want to pay via bank transfer – registration for that closed on May 30th). Free online passes are also available!

[[Editors note: This post has been updated to include information about the Data Mobility Exhibition on 6/15 that was previously excluded]].

ESnet staff attend strategic on-site meetings for the first time in years!

Last week, over 50 ESnet employees gathered at Berkeley Lab for a week of strategizing and socializing. Here are some pictures from their adventures!

Jealous of all the fun we had? Want to hang out with us, too? Good news – Registration will open soon for Confab22, ESnet’s first user meeting! Keep an eye on the blog or pre-register for updates!

3Q with ESnet’s newest computer software engineer: Alex Ray!

Before coming to ESnet, Alex was a network software automation engineer for Charter Communications and sometimes a contractor. In another life, he was a professional artist. He lives in Denver, CO with his wife and two Pekingese, which may actually be aliens in a cute disguise. Enjoys hiking and camping in the mountains, biking (not in the mountains), comics, and the occasional video game. 

What brought you to ESnet?

I was working as a contractor for about a year, helping write code for the Network Services Orchestration (NSO) piece of ESnet and various other parts of the software ecosystem. The people are great, and the fact that the network being deployed and maintained supports scientific research is amazing. Couldn’t resist the offer to work here!

What is the most exciting thing going on in your field right now?

I really like where the Rust programming language is headed, and I’m actively trying to learn more about it so I can implement it in both professional and personal projects.

What book, movie, or podcast would you recommend?

I enjoyed Dune (just read it for the first time). I’ve been trying to read more lately, so maybe ask again in a couple months. I would also highly recommend Darknet Diaries podcast for those interested in stories about cybersecurity/hacking, physical security testing, or social engineering.

ESnet Teams Up for Workshop on Programmable Switches

A collaboration between the University of South Carolina, the Great Plains Network (GPN), and EPOC (a joint effort between Indiana University and ESnet) recently sponsored a two-day workshop on programmable data plane switches, with specific emphasis on Programming Protocol-Independent Packet Processors (P4).

Data plane programmability has attracted significant attention from the research community and industry due to its ability to enable programmers to run customized packet processing functions in the data plane, but so far there has been limited training available on P4. 

P4 is a domain-specific language for network devices, specifying how data plane devices (switches, NICs, routers, filters, etc.) process packets. Using P4, application developers and network engineers can implement specific behavior in the network, enabling changes to be made in minutes instead of years.

“With FABRIC coming online and GPN as a host of one of the nodes, this P4 Workshop was a very welcome educational opportunity for our community” said James Deaton, the executive director of the Great Plains Network, one of the sponsors for the workshop.  

More than 200 people attended the February event, which was presented via the University of South Carolina’s cybertraining system and offered free of charge. ESnet’s Jason Zurawski was among the organizers. Sessions covered the fundamentals of P4 programmable switches, FABRIC’s national cyberinfrastructure, use of P4 switches on campus networks. There were also hands-on sessions covering P4 building blocks, parser implementation in the data plane, populating match-action tables at runtime, and others.

3Q with David Sundquist

David comes to us from Concord, CA, where he worked the last 10 years at McAfee as a Project Manager in both the Consumer and IT teams. During his tenure with McAfee, he led a variety of projects, from small 2 month projects to multi-year projects. His last several years have been focused mostly on IT infrastructure, and working with the Server, Storage, Network, and Data Center teams. 

He is a father to five daughters, three of them still living at home with David and his wife. He loves watching football, and during the NFL season, you will find him cheering wildly for his 49ers! 

David Sundquist

What brought you to ESnet?

After spending 10 years with McAfee I decided that I needed a change of scenery. While looking around for jobs, I found the position at ESnet and started reading about what goes on at Berkeley Lab and was immediately impressed. Being able to work with folks who are working on cutting-edge technology was too good of an opportunity to pass up.

I’ve been fascinated by technology and computers since I was a kid. I always liked being around the newest technology. After reading all of the cutting-edge work going on at ESnet, especially their leadership in the quantum networking space, it seemed like a great fit. 

What is the most exciting thing going on in your field right now?

I’m not sure the field of Project Management has exciting changes like some of the tech world. However, I can say some of the best parts about working in Project Management is the ability to constantly be creating something new with a group of individuals, and having everyone share their knowledge to build something great. Having the opportunity to work with people from all over the world on any number of projects is always exciting and rewarding.

What book would you recommend?

This is a tough question as I love to read, and picking one book is tough. But I will go with Ender’s Game, as that was probably one of the earliest books I read that really hooked me and got me interested in reading more.

3Q with Britt Gathright

Britt Gathright has a varied background in education, hospitality, entertainment, warehouse/logistics, and oil and gas. He has over 20 years of experience in information technology operations and project management. Gathright obtained his undergraduate degree from his home state’s flagship university, the University of Arkansas, and completed his Masters in Education at the University of Tulsa—where he also met his wife. He also holds a Masters in Management Information Systems from Oklahoma State University and is a certified project manager (PMP). Britt and his wife live in Tulsa, Oklahoma and have three adult children.

Britt Gathright

What brought you to ESnet? 

I’m at a point in my life where I’m fortunate to be able to choose where I want to be. I’d like my work to outlive me and lend my project management skills to a team that is changing the world.

Education and service to others are at my core. Both my parents are educators in Arkansas, and my wife is a high school science teacher. I have seen the impacts of the pursuit of knowledge and hope my skills can help advance and continue the science conversation. 

What is the most exciting thing going on in your field right now?

Remote work has intensified the need to meet workers where they are and identify how they best intake and process information. Crafting communication styles to support and serve project team members and stakeholders is even more critical with dispersed teams. Project Management is trending towards agile methodologies that shorten delivery cycles and work products/packages. This results in hybrid project methods that blend traditional waterfall with agile for more flexible delivery frameworks.

What book would you recommend? 

While I am normally a podcast junkie, I recently enjoyed Bad Blood about the Theranos saga. It illustrates the importance of data, results, and oversight to support product claims and not be swayed too much by charisma and charm. Adventure and political thrillers from David Baldacci and Vince Flynn are favorites too.

I’m interested in book recommendations about diversity and inclusion, so please reach out if you have suggestions!

3 Questions with Liang Zhang

Liang comes to us from Illinois, where he worked at Fermilab for eight years as computer researcher before joining ESnet. His research interests lie in network protocols, resource management, and high performance software architectures. His previous experience involves designing and developing data movement tools including MDTM and BigData Express. He also developed network management applications using Software Defined Networking (SDN).

A photo of Liang Zhang
Liang Zhang

What brought you to ESnet?

I got to know ESnet during my years with Fermilab when I participated in collaboration projects between ESnet and Fermilab. In my impression, ESnet plays a pioneering  role in research and development in networking and is ahead of most of the field when it comes to using new technology. I’m excited to start a new adventure at ESnet and have fun working with other talented researchers and engineers to contribute important knowledge and findings to this community.

What is the most exciting thing going on in your field right now?

While it isn’t directly related to my work at ESnet, I’ve been intrigued by the revolution known as Web3 that’s rising on the horizon. I think it will change the way people deal with the Internet. I am eager to explore how Web3 might impact the infrastructure and applications.

What book would you recommend?

The Three-body Problem. It is a science fiction novel written by a Chinese writer Cixin Liu. It discusses history, physics, the universe, and human nature.. I also enjoy watching Youtube videos about Chinese and European history, technology, and crazy inventions. 

A word from Inder Monga: The Road to ESnet6 (Part 1)

Inder Monga, Executive Director of ESnet.

Dear Friends, Well-wishers, Colleagues, and all of ESnet,

In October of this year we will launch ESnet6, a next-generation network featuring an entirely new, software-driven network design that enhances the ability to rapidly invent, test, and deploy new innovations to meet the data needs of the Office of Science/DOE.

We put forth the vision for ESnet6 in 2016. Since then, this $151M project (total project cost – DOE 413.3 parlance including contingency) has overcome pandemic-induced issues like site lockdowns, differing vaccination and inter-state travel policies, and variable supply chain delays, and is now in its final stages of implementation. As I prepare this historic unveiling, I can’t help but look back at what the team accomplished last year.

This is the first post in a series of blog posts about the people, partnerships, and innovations that have paved the road to ESnet6.

2021 was a year for growth within ESnet. We have 100+ people in the organization now—a 30% increase from last year—and it has been great to have new employees on-boarded, integrated, and productive in this challenging environment. 

A diagram showing the dimensions of growth within ESnet: Foundations, Innovation, Co-design, and Culture. Foundations, Innovation, and Co-design all point outward in separate directions, while Culture lies alongside all three Axes, growing in tandem with them.
The dimensions of growth for ESnet

Looking towards the future, we think of ESnet growing around four dimensions. The three spatial axes are: 

  • Foundations: Next Generation Network and Services 
  • Innovation: Testbeds and Advanced Research and Development
  • and Co-design: Partnerships with Science for new data and network solutions. 

The fourth axis, Culture, is pervasive across all three dimensions. 

The main reason for choosing this very technical representation is to illustrate that these are not independent thrusts—success in each of these dimensions depends on the capabilities of the other.

In this post, I’d like to focus on that first axis: Foundations. In the next few posts, I will focus on the Innovation and Co-Design dimensions and share more thoughts about our focus for 2022 and beyond.

Major capacity improvements

In 2021, we installed a brand new routing infrastructure on our network backbone, while decommissioning a portion of the previous generation packet processors in parallel. We seamlessly transitioned all ESnet customers and peers onto the forty new backbone routers before the holidays, and the remaining router upgrades at our customer sites are in progress and scheduled through 2022.

The greenfield optical infrastructure (installed at 300 locations in 2020— another noteworthy accomplishment) is getting a wonderful upgrade: 400G wavelengths are being standardized across our national backbone as we complete the second phase of optical upgrades.

In addition to our team’s intricate efforts to decommission the existing network, we added another 100G on the ring in Europe (thanks to our collaboration with GEANT). This ensured that the first Large Hadron Collider Data Challenge had enough bandwidth to accommodate both ESnet scientific data and LHC data challenge (test) streams. We also established a new point of presence in Dallas to support new peerings and the FABRIC project

ESnet network map showing LHC data challenge traffic sending nearly 100Gbps from Amsterdam to Boston
ESnet network map showing LHC data challenge traffic sending nearly 100Gbps from Amsterdam to Boston.

Creating a smarter network

The vision laid out in 2016 focused not only on capacity, but also on improving the essential framework of how we operate with the network. 

We made a significant investment in building out a high-availability site within 10ms of our main data center, in addition to our disaster-recovery site on the east coast. So any planned or unplanned power outages will be handled without a scramble. While the supply chain issues prevented the site from being ready for operations, we are making steady progress and look forward to completing it this year. 

The software orchestration team made tremendous progress in laying down the vision and framework for automation. They were supported by strong internal collaboration with the engineering team. Many repetitious deployments were automated, and I know it took diligent effort to make these tools available in the right time frame, aligned with evolving constraints of the deployments. A few examples of where automation was used include:

  • Deployment of optical wavelengths on our backbone
  • Deployment of routers and base configurations, and service provisioning
  • Customer migration configurations from old network to the new equipment automatically generated from ESnet Database (ESDB)
  • Virtualized test environment was developed to test out new tools and services before actual in-field deployment.

This year, we prepare to bring the official DOE 413.3 ESnet6 project to a close, but as you know the network never sleeps, data never stops growing, and we have to constantly evolve the network. I can proudly say that we have the core foundations of the enduring ESnet user facility ready to handle the next big challenges of Data, AI, and Integrated multi-facility research that the scientists and National Labs are actively pursuing.

Wishing you all a very Happy New Year from ESnet. 

Inder

This post is part of a series of posts reflecting on the road to ESnet6. Check back soon to see upcoming posts from Inder focusing on innovation, co-design, and his vision for ESnet6 and beyond.