ESnet’s Wireless Edge: Extending Our Network to Support Field Science

Throughout the world, earth and environmental scientists are deploying new kinds of sensors to measure and understand how the climate is changing and how we can best manage key infrastructure and resources in response. 

Operation and data analysis of these sensors can often be challenging, as they are deployed in areas with limited power, sometimes with no data connectivity beyond the periodic physical collection of memory cards. Sensors may be in areas where weather and other factors make access laborious and challenging, such as at the top of a mountain, down a borehole, or under dense forest canopy.

Solar-powered meteorological and hydrological sensors deployed at the Snodgrass Field Site, Crested Butte, July 2022 at approximately 9,000 ft. elevation. (Photo: Andrew Wiedlea)

As the number, types, and capabilities of these sensors increases, the U.S. Department of Energy’s (DOE) Energy Sciences Network (ESnet) is working on ways to extend its high-speed network to support the needs of scientists working in remote, resource-challenged environments where our fiber backbone cannot be extended. Using advanced wireless technologies such as low-Earth orbit constellations, 5G, and private citizen band radio system cellular, mmWave, and Internet-of-Things tools like long-range (LoRa) mesh networks, we are developing ways to remove the limits of geographical constraints from field scientists, just as we have traditionally sought to do for laboratory scientists around the DOE complex.

In early July this year, ESnet took a step forward in these efforts by installing a private cellular network near Crested Butte, Colorado, supporting sensor fields being used by Earth and environmental scientists on Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) Surface Atmosphere Integrated Laboratory program.  

The purpose of this effort is to assess requirements for operation of a private 4G/5G wireless network in a remote and changing environment, which can pull ESnet capabilities and services supporting scientific research out beyond our performant 13,000 km optical backbone. We are also using this research to identify specific operational, workflow, and data movement needs for the Earth and environmental science community as part of building ESnet’s logistics, operational, and human capital resources available to support the Earth and environmental science mission.

Our system, which is currently being configured, is built around a Nokia Digital Automation Cloud private cellular capability, with antennas being placed across a valley from sensor fields at the Snodgrass Field Site in Crested Butte. The intent is to use this cellular service to automate and improve the efficiency of data collection from sensors, using cellular routers and radios, depending on the specific capabilities of each sensor system. For those sensor systems that cannot be directly connected to a cellular network, we are establishing solar-powered sensor stations that will provide local area bridge (several hundred meter) connectivity to local sensors via wifi, LoRa, or direct ethernet cable. 

Once data is backhauled from a sensor field through our private cellular network, it will be transmitted back to ESnet via SpaceX’s Starlink low earth orbit satellite system, connecting to ESnet at a peering location in Seattle, Washington, and then through our optical backbone to the National Energy Research Scientific Computing Center at Berkeley Lab for processing and storage.

With fantastic assistance and collaboration from the Atmospheric Radiation Monitoring program, the Rocky Mountain Biological Laboratory, and Dan Feldman and Charulekha Varadarajan in the Watershed Function Science Focus Area at Berkeley Lab, our first field campaign was both great fun and extremely productive. 

We will return later in the Fall to complete network configuration and connection of sensors to the network. Once this is done, we can begin the next phase of this research: studying the operational performance and service requirements necessary to support field science through the demanding conditions provided by winter in the Colorado High Rockies. We will also begin to develop standard deployment equipment specifications and practices that we can use to support ESnet wireless edge deployments supporting science in other regions and for other purposes.  

This effort is being made possible by teamwork across ESnet and Berkeley Lab, including outstanding support at Berkeley Lab from Chris Tracy, Jackson Gor with ESnet network engineering, and Steve Nobles and many others with IT Telephone Services. The Colorado deployment success depended on the hard (often physical) work of Stijn Wielandt-EESA, Kate Robinson (ESnet Network Engineering), Jeff D’Ambrogia (IT-Science IT), and Jeff Chavez with Nokia.

Chris Cummings Speaks About Service Orchestration at Networking Field Day

ESnet’s Chris Cummings from our Orchestration and Core Data team was asked to present at the industry event “Networking Field Day: Service Provider”. 

Networking Field Day (NFD) is a unique event in which industry professionals are invited to join a panel of delegates who are presented with new products and offerings from networking vendors. These presentations then follow an open format where delegates can ask questions of the vendors and understand more about the products being discussed. 

For this particular instance of NFD, Chris was invited as a community member to present on how ESnet has built service orchestration and intent-based networking tooling, which allows us to abstract our service offerings from the technical implementation details that compose them. This is a topic that has a lot of mystique and buzz-words surrounding it in the networking industry, but this presentation contains concrete examples and demonstrations of the software that ESnet uses daily. 

The talk Chris gave explored ESnet’s approach to building a service orchestration software suite. He also gave a few demonstrations of the software in action. This presentation is not an exhaustive explanation of how to build your own intent-based networking environment, but rather an example and overview of a real-world stack that is being used in a production network today and the principles behind it. 

Watch Chris’s presentation here: https://techfieldday.com/appearance/a-real-world-approach-to-intent-based-networking-and-service-orchestration/

3Q with Juan Antunez, ESnet’s new Network Operations Engineer!

Before joining ESnet as a network operations engineer, Juan Antunez was an infrastructure engineer at Lowe’s. He brings seven years of experience in IT help desk support (customer support) and holds a Bachelor of Business Administration in Management Information Systems from the University of Houston-Downtown. Outside of work, he enjoys playing soccer with his seven-year-old daughter and traveling. 

Juan Antunez smiling into the camera
Juan Antunez

Question 1: What brought you to ESnet? 

After working for many years in the aerospace and retail industries, I’ve decided to join ESnet, which has a significant footprint supporting scientific research and development. The opportunity to contribute to the Department of Energy’s large-scale scientific research is exciting and fulfilling. I’m thrilled about joining, and I’m looking forward to continuing my career development!

Question 2: What is the most exciting thing going on in your field right now? 

I would say that Automation is something that many organizations are integrating into their network to avoid daily repetitive tasks that take time away from the workforce. It also helps prevent zero-day attacks with automation and response.

Question 3: What book, movie, or podcast would you recommend? 

I highly recommend The Art of Networking Engineering podcast to anyone interested in listening to other network engineers speak of their real-life experiences in the industry.  Also, Darknet Diaries is another fascinating podcast; you get to listen to and learn about what’s happening in the cybersecurity world.