Quantum Networking Basics With ESnet’s Wenji Wu

Quantum networks may provide new capabilities for information processing and transport, potentially transformative for science, economy and natural science uses. These capabilities, provably impossible for existing “classical” physics based networking technologies, are of key interest to many U.S. Department of Energy (DOE) mission areas, such as climate and Earth system science, astronomy, materials discovery, and life sciences, etc.

In August of 2021, the Advanced Scientific Computing Research (ASCR) division of the US Department of Energy’s Office of Science announced a funding award for several quantum information system projects in support of the U.S. National Quantum Initiative. One of these projects is QUANT-NET (Quantum Application Network Testbed for Novel Entanglement Technology), a collaboration between Berkeley Lab, UC Berkeley, University of Innsbruck, and Caltech.

QUANT-NET research is focused on building a software-controlled quantum computing network, linking Berkeley Lab and UC Berkeley. ESnet executive director Inder Monga is the project principal investigator. The idea for QUANT-NET was born out of the 2020 DOE Quantum Internet Blueprint workshop, where representatives from DOE national laboratories, universities, industry, and other U.S. agencies came together to define a roadmap for building the first nationwide quantum Internet.

In this post, Dr. Wenji Wu, an ESnet networking researcher who is part of the QUANT-NET team, describes what future capabilities quantum networking may provide and why researchers believe quantum networks will transform scientific activities. 


Why Quantum Networks?

In the past thirty years, significant progress has been made in the fields of quantum technologies. The combination of quantum mechanics and information science forms a new area – quantum information science (QIS). In the broad context of QIS, quantum networks have an important role for the physical implementation of quantum computing, communication, and metrology. Quantum networks are envisioned to achieve novel capabilities that are provably impossible using classical networks and could be transformative to science, the economy, and national security. These novel capabilities range from cryptography, sensing and metrology, distributed systems, to secure quantum cloud computing. 

A few examples of this include: 

  • Secure Quantum Communication: Quantum networks take advantage of the laws of quantum physics (i.e., superposition and entanglement) to transmit information, potentially achieving a level of privacy and security that is impossible to achieve with today’s Internet. See Figure 1a.
  • A Quantum Network of Clocks: Recent research shows that a quantum network of atomic clocks can result in a substantial boost of the overall precision if multiple clocks are properly connected by quantum mechanical means. Compared to a single clock, the ultimate precision will improve as much as 1/K, where K is the number of clocks. If the same clocks are connected via a classical network, the precision scales as much as 1/SQRT(K). Ultimately, a quantum network of atomic clocks can surpass the Standard Quantum Limit (SQL) to reach the ultimate precision allowed by quantum theory — the Heisenberg limit. See Figure 1b.
  • Upscaling Quantum Computing: An individual quantum computer is typically limited in size. Connected by quantum networks, multiple quantum computers can work together as one big quantum computer to address larger problems. See Figure 1c.
Diagram

Description automatically generated
Figure 1a: Secure quantum communication (credit: Chen et al. https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.124.070501).Figure 1b: A quantum network of clocks (credit: Komar, Peter et al. “A quantum network of clocks.” Nature Physics 10.8 (2014:582-587).Figure 1c: Upscale quantum computing (credit: Thor Swift, Berkeley Lab).

Quantum Network Basics

Quantum networks are distributed systems of quantum systems, which are able to exchange quantum bits (qubits) and generate and distribute entangled quantum states. As illustrated by Figure 2, a quantum network conceptually consists of three essential quantum components: 

  1. Quantum nodes, which are physical quantum systems (e.g., trapped ions, quantum dots, Nitrogen-vacancy centers) connected to the quantum network. Well-characterized matter qubits are typically defined and created from these physical quantum systems. Quantum information is generated, processed, and stored locally by matter qubits in quantum nodes.  Matter qubits, often referred to as stationary qubits, are typically isolated from the surrounding environment to minimize decoherence and facilitate various quantum operations. 
  2. Quantum channels, which connect physically separated quantum components in the quantum network and transfer quantum states faithfully from place to place using the flying qubits. Optical fibers and free-space communications are typically implemented as quantum channels because they have a reduced chance of decoherence and loss. Photons with polarization or time-bin encoding are the flying qubit of choice. The implementation of quantum channels also requires that information encoded in a stationary qubit is reliably transferred to a flying qubit, and vice versa. 
  3. Quantum repeaters, which allow the end-to-end generation of quantum entanglement, and thus, the end-to-end transmission of qubits by using quantum teleportation. Quantum repeaters typically implement entanglement-related operations such as entanglement swapping and entanglement purification.

Figure 2: A quantum network consists of three essential quantum systems

In quantum networks, qubits cannot be copied due to the no-cloning theorem, which forbids the creation of identical copies of an arbitrary unknown quantum state. Therefore, qubits can not be physically transmitted over long distances without being hindered by the effects of signal loss and decoherence inherent to most transport mediums such as optical fiber. However, qubits can share a special relation known as entanglement. Entangled qubits have interesting non-local properties, even if they are located at distant nodes. Consuming an entangled qubit pair, a data qubit can be sent deterministically to a remote node. Entanglement is the fundamental building block of quantum networks. 

As illustrated in Figure 3, key entanglement-related operations include: 

  • Entanglement Purification: Multiple low-quality entanglements can be purified into a high-quality entanglement. 
  • Entanglement Swapping: Long-distance entanglement can be built from shorter segments, with flying qubits transmitted locally.
  • Teleportation: to enable the end-to-end transmission of qubits.

Figure 3: Key entanglement-related operations

Classic networks typically concern the performance metrics such as bandwidth, throughput, and latency. Likewise, quantum networks care for performance metrics related to quantum operations. Critical quantum quality metrics include entanglement generation rate, decoherence rate, and fidelity. In quantum networks, fidelity is a key indicator to characterize the quality of quantum states or operations. In general, a minimum fidelity (Fmin) is required to support quantum operations.

It is envisioned that quantum networks will operate in parallel with classic networks. Quantum networks are not meant to replace classic networks but rather to supplement them with quantum capabilities.

Current Status

Today, quantum networks are in their infancy. Like the Internet, quantum networks are expected to undergo different stages of research and development until they reach their full functionality. There are many promising R&D efforts underway looking to develop quantum network technologies. The DOE unveiled a quantum Internet blueprint in 2020 to accelerate research in quantum science and technology, with the emphasis on the creation of a quantum Internet.

ESnet’s Wireless Edge: Extending Our Network to Support Field Science

Throughout the world, earth and environmental scientists are deploying new kinds of sensors to measure and understand how the climate is changing and how we can best manage key infrastructure and resources in response. 

Operation and data analysis of these sensors can often be challenging, as they are deployed in areas with limited power, sometimes with no data connectivity beyond the periodic physical collection of memory cards. Sensors may be in areas where weather and other factors make access laborious and challenging, such as at the top of a mountain, down a borehole, or under dense forest canopy.

Solar-powered meteorological and hydrological sensors deployed at the Snodgrass Field Site, Crested Butte, July 2022 at approximately 9,000 ft. elevation. (Photo: Andrew Wiedlea)

As the number, types, and capabilities of these sensors increases, the U.S. Department of Energy’s (DOE) Energy Sciences Network (ESnet) is working on ways to extend its high-speed network to support the needs of scientists working in remote, resource-challenged environments where our fiber backbone cannot be extended. Using advanced wireless technologies such as low-Earth orbit constellations, 5G, and private citizen band radio system cellular, mmWave, and Internet-of-Things tools like long-range (LoRa) mesh networks, we are developing ways to remove the limits of geographical constraints from field scientists, just as we have traditionally sought to do for laboratory scientists around the DOE complex.

In early July this year, ESnet took a step forward in these efforts by installing a private cellular network near Crested Butte, Colorado, supporting sensor fields being used by Earth and environmental scientists on Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) Surface Atmosphere Integrated Laboratory program.  

The purpose of this effort is to assess requirements for operation of a private 4G/5G wireless network in a remote and changing environment, which can pull ESnet capabilities and services supporting scientific research out beyond our performant 13,000 km optical backbone. We are also using this research to identify specific operational, workflow, and data movement needs for the Earth and environmental science community as part of building ESnet’s logistics, operational, and human capital resources available to support the Earth and environmental science mission.

Our system, which is currently being configured, is built around a Nokia Digital Automation Cloud private cellular capability, with antennas being placed across a valley from sensor fields at the Snodgrass Field Site in Crested Butte. The intent is to use this cellular service to automate and improve the efficiency of data collection from sensors, using cellular routers and radios, depending on the specific capabilities of each sensor system. For those sensor systems that cannot be directly connected to a cellular network, we are establishing solar-powered sensor stations that will provide local area bridge (several hundred meter) connectivity to local sensors via wifi, LoRa, or direct ethernet cable. 

Once data is backhauled from a sensor field through our private cellular network, it will be transmitted back to ESnet via SpaceX’s Starlink low earth orbit satellite system, connecting to ESnet at a peering location in Seattle, Washington, and then through our optical backbone to the National Energy Research Scientific Computing Center at Berkeley Lab for processing and storage.

With fantastic assistance and collaboration from the Atmospheric Radiation Monitoring program, the Rocky Mountain Biological Laboratory, and Dan Feldman and Charulekha Varadarajan in the Watershed Function Science Focus Area at Berkeley Lab, our first field campaign was both great fun and extremely productive. 

We will return later in the Fall to complete network configuration and connection of sensors to the network. Once this is done, we can begin the next phase of this research: studying the operational performance and service requirements necessary to support field science through the demanding conditions provided by winter in the Colorado High Rockies. We will also begin to develop standard deployment equipment specifications and practices that we can use to support ESnet wireless edge deployments supporting science in other regions and for other purposes.  

This effort is being made possible by teamwork across ESnet and Berkeley Lab, including outstanding support at Berkeley Lab from Chris Tracy, Jackson Gor with ESnet network engineering, and Steve Nobles and many others with IT Telephone Services. The Colorado deployment success depended on the hard (often physical) work of Stijn Wielandt-EESA, Kate Robinson (ESnet Network Engineering), Jeff D’Ambrogia (IT-Science IT), and Jeff Chavez with Nokia.

ESnet Highlights from ZeekWeek’21

Fatema Bannat Wala presenting at ZeekWeek21

Slides and videos from ZeekWeek have just been made available — here are links to ESnet highlights.


ZeekWeek, an annual Fall conference organized by the Zeek Project, took place online from October 13-15 this year. The conference had over 2000 registered participants from the open source user community this year, who got together to share the latest and greatest about this cyber-security and network monitoring software tool.

Berkeley Lab staff member Vern Paxson developed the precursor to the Zeek intrusion detection software, then called Bro, in 1994. As an early adopter, ESnet’s cybersecurity team has strong relationships with the Zeek community, and this ZeekWeek was an opportunity to showcase advances and uses made by the software by ESnet and the entire Research and Educational Networking Community.


The talk “DNS and Spoofed traffic investigation with Zeek,” presented by Fatema Bannat Wala, discussed how Zeek is being used to do network traffic analysis/investigations at ESnet by triaging abnormal activities when these occur on our network.

The talks “A Better Way to Capture Packets with DPDK” and “Details for DPDK plugin development and performance measurement presented by Vlad Grigorescu and Scott Campbell, detailed the development process of the plugin and the performance enhancements it brings to the network packet capture technology.

Fatema Bannat Wala also did a training session on “Introduction to Zeek,” which provided hands-on experience with Zeek tools and information about how to get involved with the collaboration.

ESnet’s cybersecurity team looks forward to continued collaboration with the Zeek community, attending next year’s ZeekWeek, and to contributing future code enhancements to this great software ecosystem.

ESnet Machine Learning Researchers Win Best Paper at MLN ‘2021!

MLN '2021 Best Paper Award Notification

Sheng Shen, Mariam Kiran, and Bashir Mohammed have just been awarded the Best Paper award at the International Conference on Machine Learning for Networking (MLN). Sponsored by the Conservatoire National des Arts et Métiers (CNAM), the École Supérieure d’Ingénieurs en Électrotechnique et Électronique (ESIEE), and Laboratoire d’Informatique Gaspard-Monge (LIGM), MLN is being held virtually 1-3 December 2021.

The paper, “DynamicDeepFlow: An Approach for Identifying Changes in Network Traffic Flow Using Unsupervised Clustering,” uses a hybrid of deep learning variational autoencoder model and a shallow learning k-means to help identify unique traffic patterns across ESnet. These unique patterns can help identify if a new experiment has started or whether current network bandwidth is changing.

DynamicDeepFlow (DDF) model structure

“We’re very excited to receive this recognition and the conference was a wonderful opportunity to exchange thoughts and ideas with peers in France. MLN is a conference dedicated to discussing machine learning applications in networks. Our next task is to integrate DynamicDeepflow with Netpredict to show real-time information in ESnet data” — Mariam Kiran

Papers from MLN will be published as post-proceedings in Springer’s Lecture Notes in Computer Science (LNCS).

ESnet Highlights from the National Science Foundation’s Cybersecurity Summit ’21

The National Science Foundation (NSF) Cybersecurity Center of Excellence, Trusted CI Project hosts a yearly cybersecurity summit, inviting people from various NSF-funded research organizations to share innovations and ideas. Here are some videos of ESnet presentations.

Scott Campbell presented “ESnet Security Group Impact on Network Architecture” where he discussed some of the social, technical, and architectural outcomes of the ESnet6 network upgrade that were beneficial to the organization. By being involved early, security design elements were incorporated into workflows at early stages and were both tightly integrated and vetted during the core design process. This early involvement also heightened the security group’s visibility, which led to a better understanding of how the various groups interact and their different methods of problem-solving and time management.

Eli Dart and Fatema Bannat Wala presented “Best practices for securing Science DMZ,” focusing on disentangling security policies and enforcement for science flows from traditional security approaches for business systems, and use of the Science DMZ model to protect high-performance science flows. They discussed thinking of the Science DMZ as a security architecture that provides useful and implementable security controls without impacting performance. 

ESnet Scientists awarded best paper at SC21 INDIS!

A combined team from ESnet and Lehigh University was awarded the best paper for Exploring the BBRv2 Congestion Control Algorithm for use on Data Transfer Nodes at the 8th IEEE/ACM International Workshop on Innovating the Network for Data-Intensive Science (INDIS 2021), which was held in conjunction with the 2021 IEEE/ACM International Conference for High Performance Computing, Networking, Storage and Analysis (SC21) on Monday, November 15, 2021.

The team was comprised of:

  • Brian Tierney, Energy Sciences Network (ESnet)
  • Eli Dart, Energy Sciences Network (ESnet)
  • Ezra Kissel, Energy Sciences Network (ESnet)
  • Eashan Adhikarla, Lehigh University

The paper can be found here. Slides from the presentation are here. In this Q+A, ESnet spoke with the award-winning team about their research — answers are from the team as a whole.

INDIS 21 Best Paper Certificate

The paper is based on extensive testing and controlled experiments with the BBR (Bottleneck Bandwidth and Round-trip propagation time), BBRv2 and the Cubic Function Binary Increase Congestion Control (CUBIC) Transmission Control Protocol (TCP) Internet congestion algorithms. What was the biggest lesson from this testing?

BBRv2 represents a fundamentally different approach to TCP congestion control. CUBIC (as well as Hamilton, Reno, and many others) are loss-based, meaning that they interpret packet loss as congestion and therefore require significant network engineering effort to achieve high performance. BBRv2 is different in that it measures the network path and builds a model of the path – it then paces itself to avoid loss and queueing. In practical terms, this means that BBRv2 is resilient to packet loss in a way that CUBIC is not. This comes through loud and clear in our data.

What part of the testing was the most difficult and/or interesting?

We ran a large number of tests in a wide range of scenarios. It can be difficult to keep track of all the test configurations, so we wrote a “test harness” in python that allowed us to keep track of all the testing parameters and resulting data sets.

The harness also allowed us to better compare results collected over real-world paths to those in our testbed environments. Managing the deployment of the testing environment though containers also allowed for rapid setup and improved reproducibility. 

You provide readers with links to great resources so they can do their own testing and learn more about BBRv2. What do you hope readers will learn?

We hope others will test BBRv2 in high-performance research and education environments. There are still some things that we don’t fully understand, for example there are some cases where CUBIC outperforms BBRv2 on paths with very large buffers. It would be great for this to be better characterized, especially in R&E network environments.

What’s the next step for ESnet research into BBRv2? How will you top things next year?

We want to further explore how well BBRv2 performs at 100G and 400G. We would also like to spend additional time performing a deeper analysis of the current (and newly generated) results to gain insights into how BBRv2 performs compared to other algorithms across varied networking infrastructure. Ideally we would like to provide strongly substantiated recommendations on where it makes sense to deploy BBRv2 in the context of research and educational network applications.

Graduate students publish on network telemetry with ESnet

Two graduate students working with ESnet have published their papers recently in IEEE and ACM workshops.

Bibek Shrestha, a graduate student at the University of Nevada, Reno, and his advisor Engin Arslan worked with Richard Cziva from ESnet to publish a work on “INT Based Network-Aware Task Scheduling for Edge Computing”. In the paper, Bibek investigated the use of in-band network telemetry (INT) for real-time in-network task scheduling. Bibek’s experimental analysis using various workload types and network congestion scenarios revealed that enhancing task scheduling of edge computing with high-precision network telemetry can lead up to a 40% reduction in data transfer times and up to 30% reduction in total task execution times by favoring edge servers in uncongested (or mildly congested) sections of the network when scheduling tasks. The paper will appear in the 3rd Workshop on Parallel AI and Systems for the Edge (PAISE) co-conducted with IEEE IPDPS 2021 conference to be held on May 21st, 2021, in Portland, Oregon. 

Zhang Liu, a former ESnet intern and a current graduate student at the University of Colorado at Boulder, worked with the ESnet High Touch Team – Chin Guok, Bruce Mah, Yatish Kumar, and Richard Cziva – on fastcapa-ng, ESnet’s telemetry processing software. In the paper “Programmable Per-Packet Network Telemetry: From Wire to Kafka at Scale,” Zhang showed the scaling and performance characteristics of fastcapa-ng, and highlighted the most critical performance considerations that allow the pushing of 10.4 million telemetry packets per second to Kafka with only 5 CPU cores, which is more than enough to handle 170 Gbit/s of original traffic with 1512B MTU. This paper will appear in the 4th International Workshop on Systems and Network Telemetry and Analytics (SNTA 2021) held at the ACM HPCD 2021 conference in Stockholm, Sweden between 21-25 June 2021.

Congratulations Bibek and Zhang!


If you are a networked systems research student looking to collaborate with us on network measurements, please reach out to Richard Cziva. If you are interested in a summer internship with ESnet, please visit this page.

How the World’s Fastest Science Network Was Built

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

Funded by DOE’s Office of Science and managed by the Lawrence Berkeley National Laboratory (Berkeley Lab), ESnet moves about 51  petabytes of scientific data every month. This is a 13-step guide about how ESnet has evolved over 30 years.

Step 1: When fusion energy scientists inherit a cast-off supercomputer, add 4 dialup modems so the people at the Princeton lab can log in. (1975)

Online3

Step 2: When landlines prove too unreliable, upgrade to satellites! Data screams through space. (1981)

18ogxd

Step 3: Whose network is best? High Energy Physics (HEPnet)? Fusion Physics (MFEnet)?  Why argue? Merge them into one-Energy Sciences Network (ESnet)-run by the Department of Energy!  Go ESnet! (1986)

ESnetListicle

Step 4: Make it even faster with DUAL Satellite links! We’re talking 56 kilobits per second! Except for the Princeton fusion scientists – they get 112 Kbps! (1987)

Satellite

Step 5:  Whoa, when an upgrade to 1.5 MEGAbits per second isn’t enough, add ATM (not the money machine, but Asynchronous Transfer Mode) to get more bang for your buck. (1995)

18qlbh

Step 6: Duty now for the future—roll out the very first IPv6 address to ensure there will be enough Internet addresses for decades to come. (2000)

18s8om

Step 7: Crank up the fastest links in the network to 10 GIGAbits per second—16 times faster than the old gear—a two-generation leap in network upgrades at one time. (2003)

18qlnc

Step 8: Work with other networks to develop really cool tools, like the perfSONAR toolkit for measuring and improving end-to-end network performance and OSCARS (On-Demand Secure Circuit and Advance Reservation), so you can reserve a high-speed, end-to-end connection to make sure your data is delivered on time. (2006)

18qn9e

Step 9: Why just rent fiber? Pick up your own dark fiber network at a bargain price for future expansion. In the meantime, boost your bandwidth to 100G for everyone. (2012)

18on55

Step 10: Here’s a cool idea, come up with a new network design so that scientists moving REALLY BIG DATASETS can safely avoid institutional firewalls, call it the Science DMZ, and get research moving faster at universities around the country. (2012)

18onw4

18oo6c

Step 11: We’re all in this science thing together, so let’s build faster ties to Europe. ESnet adds three 100G lines (and a backup 40G link) to connect researchers in the U.S. and Europe. (2014)

18qnu6

Step 12: 100G is fast, but it’s time to get ready for 400G. To pave the way, ESnet installs a production 400G network between facilities in Berkeley and Oakland, Calif., and even provides a 400G testbed so network engineers can get up to speed on the technology. (2015)

18oogv

Step 13: Celebrate 30 years as a research and education network leader, but keep looking forward to the next level. (2016)

ESnetFireworks

ESnet Co-Leads Washington Workshop on Developing Prototype SDN Network

About 100 networking experts from academia, industry, national labs and federal agencies met for a two-day workshop at the National Science foundation to plan a path forward to develop, deploy and operate a prototype SDN network. SDN, or Software Defined Networking, is an upcoming technology paradigm aimed at making it easier for software applications to automatically configure and control the various layers of the network to improve flexibility, predictability and reliability.

ESnet Chief Technologist Inder Monga was the lead organizer of the workshop and ESnet network engineer Erich Pouyoul gave a talk on science drivers for SDN. Monga also led a breakout session on “Technology and Operational Gap Analysis.” ESnet has been a pioneer in developing and deploying SDN technology in support of data-intensive science for almost a decade, starting with research on virtual network circuits that eventually culminated in the facility’s OSCARS project, recipient of a 2013 R&D100 award.

The invitation-only workshop was held at the National Science Foundation in Arlington, Va., and included speakers from the White House Office on Science and Technology Policy (OSTP), Google, DARPA, Internet2, SRI and Brocade, as well as ESnet. Among the areas covered were transparency and interoperation among SDN domains, security and identity management, and the participation of equipment vendors to advance technology transfer.

The workshop was organized after the OSTP directed federal agencies participating in the Networking and Information Technology Research and Development (NITRD) Subcommittee’s Large Scale Networking (LSN) Coordinating Group to plan and hold an LSN workshop. The goal was to have participation by representatives from federal agencies, the commercial sector, researchers, and other networking and distributed systems research community participants to explore and report on the need for a prototype SDN network.

The workshop participants will draft a report documenting recommendations for needed R&D, resources and collaboration to deploy and operate the prototype SDN network and to identify future SDN research needs.

On the eve of the workshop, Federal Computer Week magazine published an article about federal agencies looking into SDN. Monga was among the sources interviewed for the article, which describes SDN as the next major architectural change looming for the IT community.

Image

  ESnet Chief Technologist Inder Monga

ESnet’s OSCARS Bandwidth Reservation System Wins R&D 100 Award

Widely recognized as a mark of excellence, the R&D 100 Awards are the only industry-wide competition rewarding the practical applications of science. Among the 2013 winners of this prestigious award is the latest version of OSCARS, the On-demand Secure Circuits and Reservation System, the development of which was led by ESnet. OSCARS is a software service that creates dedicated bandwidth channels for scientists who need to move massive, time-critical data sets around the world.

What makes OSCARS so useful is that it can automatically create end-to-end circuits, crossing multiple network domains. Before OSCARS, this was a time consuming process–in 2010, for example, ESnet engineers needed 10 hours of phone calls and about 100 emails over three months to do what one person can do in five minutes using OSCARS. The automation of this complex process—through a technique known as Software Defined Networking—is accelerating scientific discovery in high-energy physics and many other disciplines.

“It’s wonderful to see the innovation that’s gone into the latest version of OSCARS recognized with an R&D100 Award,” said ESnet Director Greg Bell. “This early example of Software Defined Networking is yet another example of research networks taking the lead. But OSCARS is not just an ESnet achievement. We’re grateful for collaborations with many partners over the last decade, as the project matured from a bold idea into a production software suite.”

Image

Read more.