ESnet’s Nick Buragilio Wins Prize at the Annual LBNL IPO Pitch Competition for Hecate

Three questions with Nick Buragilio about the Pitch Competition

Nick Buragilio took third prize at the LBNL Intellectual Property Office’s Annual Pitch Competition on September 9 for his talk on “Hecate: Directing happiness to internet service provider customers.” Hecate is a software tool that leverages machine learning to automate complex network traffic engineering.  The prize includes $1000 for the Hecate team, supporting continued lab-to-market progress.


How did you develop the technology? It’s a team of three: myself, Scott Campbell, and Mariam Kiran. Scott is handling the data collection and curation and pipelining the data into AI algorithms being developed by Mariam. Mariam is also working on porting the algorithms to GPUs. I’m handling the overall technology and product strategy, plus network elements supporting large-scale traffic engineering. 

The idea came from traffic engineering and segment routing conversations; Mariam had some ideas about bringing in machine learning from the SENSE project, so we sat down over Zoom and sketched things out – it was a natural meeting of minds and very much a virtual “mapping out a project on a napkin” moment, despite the pandemic.

What was it like pulling together the pitch? I enjoy public speaking, and I like to challenge myself. The Pitch Competition seemed like a good opportunity to test the waters and experiment to see what would work and what might not. The challenge was to fit a complicated technical topic into a 5-minute elevator pitch. The Intellectual Property Office supplied coaching as well.

What’s next? We continue on with our testing, and we are looking for more opportunities to use the demonstration software on real data, especially research and educational network partners who can give access to their network data. I’m at buraglio@es.net if a reader is interested in learning more!

Summer at ESnet: The view from our students, Part 2

Pratyush Muthukumar is a student intern with our Prototypes & Testbed Team. He is a 2021 Goldwater Scholar at UC Irvine, working toward a BS in Computer Science. 

Pratyush Muthukumar

My time as an intern at ESnet has been thrilling and rewarding. Over the last seven months as an intern here, I have learned many valuable research, scholarly, and professional skills that I will undoubtedly use in the future. 

I have primarily been working on the SENSE (Software Defined Networking [SDN] for End-to-End Networked Science at the Exascale) project with my amazing mentors, Xi Yang and Tom Lehman, and this has been exciting and fascinating. I’ve learned a plethora of new research skills and frameworks, including multi-domain monitoring, software-defined networking, network polling, scripting, and virtualization. My project mentors have provided great guidance and supported me throughout every stage of the project. What I enjoy best about being an intern at ESnet is the resources, guidance, and tools available to support my research. I’m excited to work on such a challenging yet rewarding tool that will soon have real-world impact when deployed on ESnet.

The environment as an ESnet intern has been extremely friendly and inviting. I feel a part of the community within ESnet and I really enjoy attending the frequent seminars and brown bag lunches held multiple times weekly. Learning about new research developments from other teams at ESnet and guest lecturers from the Computer Science and Mathematics department at UC Berkeley has broadened my knowledge and interests greatly.

While my experiences may have been different during an in-person internship at ESnet, I appreciate the numerous networking and social activities that ESnet has facilitated to make remote interns feel connected. I have really enjoyed my time as an ESnet intern thus far, and I’m looking forward to the rest of my time at ESnet!

If you are interested in learning more about future summer opportunities with ESnet, please see this link (https://cs.lbl.gov/careers/summer-student-and-faculty-program/). We typically post notices and accept applications for the next summer starting in January or February.

Five Women Nab a WIN-ing Opportunity for SC21

Women in IT Networking at SC (WINS) Program Selects SC21 SCinet Cohort

Five new participants have been chosen to join the Women in IT Networking program at SC (WINS) program for SC21 – the annual International Conference for High Performance Computing, Networking, Storage, and Analysis (SC) which will take place this year from  November 14-19 in St. Louis, Missouri.

WINS was developed in 2015 to help address the gender gap that exists in information technology, particularly in the fields of network engineering and high performance computing (HPC). Each year WINS welcomes a cohort of early-to-mid career women to join the team that builds SCinet, a multi-terabit network built specially for high-bandwidth demonstrations SC attendees and exhibitors who perform high-bandwidth demonstrations. The program seeks women of all ages, races, backgrounds, IT-related professions, and geographical areas. 

“Lack of diversity in IT particularly with women and minorities is significant and an ongoing challenge and concern.  WINS is a small effort to encourage women in their early and mid-careers to stay in IT and find a professional network to engage and gain technical and non-technical support. Underrepresented minority groups are defined as African-American/Black, Latino/ Hispanic, and Native American/Native Hawaiians. The problem is particularly acute for women of color, which represent less than 2 percent of high-level technical positions,” said Marla Meehl (Manager Networking and Front Range GigaPoP (FRGP), University Corporation for Atmospheric Research (UCAR)) and WINS-PI.

This year, 19 women applied for WINS. Leaders in the research and education (R&E) network and HPC communities reviewed and selected the following awardees:

  • Mary Bull, College of William & Mary, SCinet Wireless Team
  • Karen Lopez, National Renewable Energy Lab (NREL), SCinet DevOps Team
  • Deshon Miguel, Tohono O’odham Community College, SCinet WAN Team
  • Stacie Nixon, North Carolina Central University, SCinet Wireless Team
  • Kimberly Schjang, University of Nevada – Las Vegas (UNLV), SCinet Edge Team

SC21 participant Kimberly Schjang, Associate Network Engineer from UNLV shared her thoughts on being selected. “I am honored to represent UNLV and WINS by participating in the SC Conference this year. I feel that this opportunity will not only help me grow as a network engineer, but it will also give me the opportunity to represent Black women, the LGBT community, and women in STEM.”

As a core component of the program, the WINS awardees are deeply immersed in a multi-month-long, intense engineering experience before the event begins and during the SC conference while SCinet is operated live. They join one of over fifteen SCinet teams, are paired with a mentor, and are involved in all phases of the SCinet process. 

Kimball Sekaquaptewa, Chief Technology Director for the Santa Fe Indian School and SC19 WINS participant can attest to the experience. “In the span of three weeks, amazing highly-skilled and intelligent professionals who are generous with their knowledge come together to build an incredible 4.2 Terabit wide area network spanning the continental United States to Asia and beyond. You won’t know until you do it, how profound the experience really is.” 

When WINS began in 2015, 14% of SCinet professional volunteers were women. By 2017, the number had climbed to 21%, and in 2020, 40 of the 115 SCinet volunteers – 19 from the WINS program – and 40% of the SCinet leadership were women. 

“WINS has had a tremendous impact on the diversity of SCinet,” said Lauren Rotman, ESnet Science Engagement Group Lead, and a co-lead of the WINS program. “Not only has the male-to-female volunteer ratio doubled since the program started – the program has added another dimension of diversity by bringing in participants from community colleges, tribal colleges, and other smaller institutions that are traditionally underrepresented.” 

To that end, Sekaquaptewa added, “Because I had just completed building two fiber optic backbones connecting six tribes in New Mexico for my home institution, I could now see how to grow them to connect to a national backbone more meaningfully than just access to commodity Internet. I saw and better yet met the people behind the networks with whom I now collaborate to build connections that empower Tribal nations in the digital space.”

The WINS program has also had a significant impact on the entire SCinet volunteer team which speaks to the broad benefits of a diverse workforce. 

Lance Hutchinson, Manager, Space Control Applications, Sandia National Laboratories and SC21 SCinet Chair said, “WINS is not just creating a more inclusive environment for SCinet but it is also improving the quality of learning opportunities for all of our volunteers as part of their participation. All of our team leads, management and others involved in supporting the WINS awardees are building brand new hands-on mentoring, teaching and leadership skills. The program has exponential value as it improves the SCinet community on multiple levels.”

Jennifer Kim, IT Manager at the Montgomery County Community College, an SC19 WINS awardee shared her thoughts. “I am looking forward to the much anticipated event that is SC21. Originally awarded for SC19, I deferred attendance as I was about to be a first-time Mom around the same time. I think this is important to note as this offering demonstrates WINS’ support that is unique to women.” 

Kim added, “My gratitude only grew as the following year, SC20, was virtual, and WINS made sure to offer us awardees the opportunity to participate in  SC21. I am excited to experience the energy and networking, in every sense of the word, of all facets of the SCinet event. I can hardly wait to work with fellow awardees, teammates and learn of their journeys to SC and what keeps them coming back. And of course, eager to build and service the fastest network I will have the pleasure of working on, which happens to be one of the fastest networks in the world.”

Carlos Rojas Torres, Network Engineer at UCAR and team lead for the SCinet Inclusivity team added, “The SCinet team are thrilled to welcome the SC21 WINS participants to become part of this growing ecosystem as well as the SC19 and SC20 deferred participants – making this the largest WINS cohort ever! WINS is a special and significant part of the SC21 Inclusivity Program which aims to provide opportunities for people of diverse backgrounds.”

In addition to the SCinet experience, WINS provides ongoing support and career development opportunities for the awardees before, during, and after the conference. This even includes monthly calls for the community of WINS alumni participants which now tops 40 women. 

Wendy Huntoon, co-PI for the WINS NSF grant said, “The WINS program provides significant opportunities for technical skill development, broad professional development, and career advancement for its participants. WINS participants broaden and deepen their technical skills, gain exposure to technologies or equipment outside their standard work environment, expand their professional network, and improve their collaboration and communication skills. All skills the women are able to apply at their home institution or in the broader networking community.”

WINS is a joint effort between the University Corporation for Atmospheric Research (UCAR),the  Department of Energy’s Energy Sciences Network (ESnet), and the Keystone Initiative for Network Based Education and Research (KINBER). The multi-year program is funded primarily by grants from the National Science Foundation and direct funding from ESnet with support from the Department of Energy. Due to COVID travel restrictions, ESnet will provide programmatic support for SC21. Members of the WINS coordinating team include PI Marla Meehl, Belinda Green, Susan Guastella, Carlos Rojas Torres (SCinet Inclusivity Liaison), of UCAR; co-PI Wendy Huntoon of KINBER;  Jason Zurawski, Lauren Rotman, Kate Robinson of ESnet; and Kimball Sekaquaptewa with Santa Fe Indian School (SCinet Co-Chairs of WINS).

High Energy Physics Requirements Review Now Available: The Data Deluge Shows no Sign of Cresting!

Lauren Rotman and Jason Zurawski


Across the physical sciences, new instruments and capabilities are continuing a relentless growth in data production and need for high speed networking and analysis resources. 

ESnet stays on-top of these trends via the Network Requirements Review process, which for the past 15 years has been a remarkable and useful collaboration between the DOE Office of Advanced Supercomputing Research (ASCR), ESnet and science programs across the DOE Office of Science.

The latest Network Requirements Review for the Office of Science High Energy Physics program office (HEP) is now available — among many other findings, this review confirms that the exponential growth of scientific data generation will continue unabated as we proceed into what may well be a new golden age for high energy physics research. Some samples include:

The upcoming High Luminosity era for the Large Hadron Collider (beyond 2027, or Run-4) will require multi-Tbps network speeds to support globally dispersed “Tier 1” HPC resources.  Scientists will use the LHC to uncover how the Higgs-Boson interacts and gives mass to other particles, and explore emerging evidence for particle behaviors not explained by current physics models. Each data-taking year, the experiments, ATLAS and CMS combined, are expected to accumulate roughly 1 EB of new data and it is estimated that complete data set sizes may routinely exceed 100 PB.  

Expected maximum luminosity and integrated luminosity for the LHC as a function of calendar year, data produced tracks with improved luminosity and resolution

Scientists at the Deep Underground Neutrino Experiment (DUNE) in South Dakota and at Fermilab in Illinois, will use high speed data transfer to identify supernova events, as part of ongoing measurement of neutrino interactions. Supernovae measured by DUNE will generate over 200TB of compressed data per event, and Research and Educational Networks (REN) must be able to supply highly reliable, predictable data transfer capabilities to provide telescope targeting data to global arrays.

10kt DUNE Far-Detector SP module, showing the alternating 58 m long (into the page), 12 m high anode (A) and cathode (C) planes, as well as the field cage that surrounds the drift regions between the anode and cathode planes. The blank area on the left side was added to show the profile of a single anode plane assembly (APA). Person included for scale.

The Cosmic Microwave Background, Stage 4 (CMB-S4) experiment will require data management and transfer capabilities in some of the most demanding locations on earth. Operating two observational locations, and multiple telescopes with a combined total of 500,000 cryogenically-cooled superconducting detectors at the South Pole and in the Chilean Atacama Desert, CMB-S4 will provide an unprecedented picture back into the start of the Universe. Operating for seven years in these conditions, 22 TB (~8 TB at the South Pole and ~14 TB in Chile) of data will be generated daily, leading to an accrual of 3 PB annually, and as much as 100 TB over the full program lifecycle.

Two Cross-Dragone (CD) telescopes (one is pictured above) with six meter diameter input apertures will be deployed at the Chilean site to map roughly 70% of the sky every day to support the dark universe, matter-mapping, and time-varying mm-wave sky science goals. Image and caption courtesy of the CMB-S4 Project

Network Requirements Reviews analyze the current, near, and long-term needs of the HEP community, providing a network and data-centric understanding of the scientific process used by the researchers and scientists. These requirements reviews drive ESnet’s investments in new services and capabilities, and enable ESnet to build strong partnerships with Office of Science (SC) programs, PIs, and user facilities. More information on this ESnet requirements review process can be found here.

We would like to thank the 13 HEP projects, and all of the HEP & DOE Office of Science collaborators who generously gave of their time, expertise, and most importantly, their enthusiasm for the future of high energy physics, as part of creating this report.

We want to especially thank the entire Science Engagement team plus Kate Robinson, and Dale Carder with our Network Engineering group who all provided outstanding support and technical expertise.

On the Path to ESnet6—Seeing the Light

ESnet6 Network

Three years ago, ESnet unveiled its plan to build ESnet6, its next-generation network dedicated to serving the Department of Energy (DOE) national lab complex and overseas collaborators. With a projected early finish in 2023, ESnet6 will feature an entirely new software-driven network design that enhances the ability to rapidly invent, test, and deploy new innovations. The design includes:

  • State-of-the-art optical, core and service edge equipment deployed on ESnet’s dedicated fiber optic cable backbone
  • A scalable switching core architecture coupled with a programmable services edge to facilitate high-speed data movement
  • 100–400Gbps optical channels, with up to eight times the potential capacity compared to ESnet5
  • Services that monitor and measure the network 24/7/365 to ensure it is operating at peak performance, and
  • Advanced cybersecurity capabilities to protect the network, assist its connected sites, and defend its devices in the event of a cyberattack

Later this month, ESnet staff will present an online update on ESnet6 to the ESnet Site Coordinators Committee (ESCC). Despite the challenges of deploying new equipment at over 300 distinct sites across the country and lighting up approximately 15,000 of miles of dark fiber during a pandemic, the team is making great progress, according to ESnet6 Project Director Kate Mace.

“We’ve had some delays, but our first priority is making sure the work is being done safely,” Mace said. “We have a lot of subcontractors and we are working closely with them to make sure they’re safe, they’re following local pandemic rules and they’re getting the access they need for installs.

“The bottom line is that we have a lot of pretty amazing people putting in a lot of hours and hard work to keep the project moving forward,” Mace said.

When completed in 2023, ESnet6 will provide the DOE science community with a dedicated backbone capable of carrying at least 400 Gigabits per second (Gbps), with some spans capable of carrying more than 1 Terabit per second.

The current network, known as ESnet5, comprises a series of interconnected backbone rings, each with 100Gbps or higher bandwidth. ESnet5 operates on a fiber footprint owned by and shared with Internet2. Once the switch is complete, Internet2 will take over ESnet’s share of the fiber spectrum to provide more bandwidth to the U.S. education community.

“We’re almost done with the optical layer, which is a big deal,” Mace said. “It’s been a major procurement of new optical line equipment from Infinera to light up the new optical footprint.”

Mapping the road to ESnet6 

Back in 2011, using Recovery Act funds for its Advanced Networking Initiative, ESnet secured the long-term rights to a pair of fibers on a national fiber network that had been built, but not yet used. Because there was a surplus of installed fiber cable at the time, ESnet was able to negotiate advantageous terms for the network.

As part of the ESnet6 project, ESnet and its subcontractors began installing optical equipment along the ESnet fiber footprint starting in November 2019. The optical network consists of seven large fiber rings east to west across the U.S., and smaller “metro” rings in the Chicago and San Francisco Bay areas.

At this point, Infinera has completed the installation of the equipment at all locations. The four large eastern-most rings have passed ESnet’s rigorous testing and verification process ensuring that they are configured and working as designed, and most ESnet services in these areas have been transitioned over to the new optical system.

Infinera has turned over the other three large rings and is working closely with ESnet staff to address a number of minor issues identified during testing.

ESnet and Infinera are collaborating on turning up, testing, and rolling services to the new network in the Chicago and Bay Area rings. The installation in these areas is more complex because it is re-using the ESnet5 fiber going into the DOE Laboratories.  

“The ESnet and Infinera teams have worked really well together to overcome all of the typical challenges we expected on a network build of this scale, as well as some unexpected obstacles,” said Joe Metzger, the ESnet6 Implementation Lead. 

The typical expected challenges ranged from installing thousands of perfectly clean (microscopically verified) fiber connections, to the unexpected, such as engineers driving for hours to get to a remote isolated location to install the equipment only to find the access road is drifted in with snow, or somebody changed the lock.

Most of the unexpected challenges were related to COVID-19.

“It was amazing to see how the facility providers, including the DOE Laboratories, ESnet and Infinera teams worked together to find safe, workable solutions to the COVID-19-related access constraints that we encountered during the installation,” said Metzger.  

The team expects the optical system build to be fully accepted and all services transitioned over to it by Oct. 1, completing what they are calling ESnet5.5, the first major step in the transition from ESnet5 to ESnet6.

To get to this point, ESnet’s network engineers needed extensive, hands-on training on the new Infinera equipment and built a specialized test lab at Berkeley Lab. To do this, a test lab was built at Berkeley Lab to provide hands-on training. Engineers take a weeklong session learning how to configure, operate, and troubleshoot the equipment deployed in the field.

The next major step will be the installation of new routers for the packet layer, which is expected to begin in early 2021, Mace said.

And of course, this is all being carried out while ESnet keeps its production network and services in regular operation and with the undercurrent of stress from the COVID-19 pandemic. 

“We’ve got to keep the network running,” Mace said. “And we are hiring additional network engineers, software engineers and technical project managers.

ESnet is supported by DOE’s Office of Science.

Written by Jon Bashor

How a future-facing ESnet project reaches back to Berkeley Lab’s roots

Eric Pouyoul and Mike Witherell

Eric Pouyoul and Mike Witherell
ESnet’s Eric Pouyoul (left) talks to Berkeley Lab Director Mike Witherell (right) about a specialized network that he’s helping to build for the GRETA experiment, short for Gamma Ray Energy Tracking Array. (Photo: Berkeley Lab)

While ESnet staff are known for building an ever-evolving network that’s super fast and super reliable, along with specialized tools to help researchers make effective use of the bandwidth, there is also a side of the organization where things are pushed, tested, broken and rebuilt: ESnet’s testbed.

For example, in conjunction with the rollout of its nationwide 100Gbps backbone network, the staff opened up a 100Gbps testbed in 2009 with Advanced Networking Initiative funding through the American Reinvestment and Recovery Act. This allowed scientists to test their ideas on a separate but equally fast network so if something crashed, ESnet traffic would continue to flow unimpeded across the network. Six years later, ESnet upped the ante and launched the 400Gbps network — the first science network to hit this speed — to help NERSC move its massive data archive from Oakland to Berkeley Lab.

Eric Pouyoul is the principal investigator for the testbed and the things he’s learned on past projects can be applied to others. His most recent project also pushed the boundaries of what the organization does in supporting DOE science. With funding from the lab’s Nuclear Physics Division, Pouyoul developed a pair of uniquely specialized data processing systems for the GRETA experiment, short for Gamma Ray Energy Tracking Array. The gamma ray detector will be installed at DOE’s Facility for Rare Isotope Beams (FRIB) located at Michigan State University in East Lansing.

When an early version of GRETA  goes online at the end of 2023 it will house an array of 120 detectors that will produce up to 480,000 messages per second—totaling 4 gigabytes of data per second—and send them through a computing cluster for analysis. Not only did Pouyoul write the software for the first stage that will reduce the amount of data by an order of magnitude—in real-time—he also designed the physics simulation software to generate realistic data generation to test the system.

For the second data handling phase of GRETA, called the Global Event Builder, he wrote the software that will take all of the data from the first phase and, using the timestamps, aggregate them in order, as well as sort them by event. This data will then be stored for future analysis.

Even though he designed and built the systems to simulate the behavior of the nuclear physics that will occur inside the detector, “don’t expect me to understand it,” Pouyoul said. “I never did anything like this before.”

A rendering of GRETA, the Gamma-Ray Energy Tracking Array.
A rendering of GRETA, the Gamma-Ray Energy Tracking Array. (Credit: Berkeley Lab)

GRETA is the first of its kind in that it will track the positions of the scattering paths of the gamma rays using an algorithm specifically developed for the project. This capability will help scientists understand the structure of nuclei, which is not only important for understanding the synthesis of heavy elements in stellar environments, but also for applied-science topics in nuclear energy, nuclear forensics, and stockpile stewardship.

“This has been my most exciting project and it only could have happened here,” he said. “I think it takes me back to the origins of the Lab when scientists and engineers worked together to create new physics. We know it will work, but we don’t even know how the results will turn out, we don’t know what will be discovered.”

Before joining ESnet at Berkeley Lab 11 years ago, he had worked in the private sector. At one point in his career, he wrote code for control systems for nuclear power plants. Looking back, he estimates that maybe three lines of his code made it into the final library. He’s quick to point out that he doesn’t consider himself a software engineer, nor does he think of himself as a network engineer. At ESnet, those engineers are responsible for designing and deploying robust systems that keep the data moving in support of DOE’s research missions.

“I really like to work with prototypes, one-time projects like in the testbed,” he said. “I know how to build stuff.”

He developed that skill as a high school student in Paris, where he preferred to roam the sidewalks, looking for discarded electronics he could take home, repair, and sell. He did manage to attend classes often enough to pass his exams and graduate with a degree. That was the only diploma he’s ever received. 

Since then, he’s learned by working on things, not sitting in lecture halls. Some of it he picked up working for a supercomputing startup company. He learned how to tune networks for maximum performance by tweaking data transfer nodes, the equipment that takes in data from experiments, observations, or computations and speeds them on their way to end-users. 

He sees the GRETA project as a pilot and it’s already drawing interest from other researchers. The idea is that if ESnet can work with scientists from the start, it will be more efficient and effective than trying to tack on the networking components afterward. Pouyoul looking forward to the next one.

“I’m really not specialized, but I do understand different aspects of projects,” he said. “I only have fun when I’m not in my comfort zone — and I had a lot of fun working on GRETA.”

Interested in working at ESnet? Apply to our open jobs: http://m.rfer.us/LBLt9j2yC 

Read more about ESnet’s contributions to the GRETA project: https://bit.ly/ESnetGRETA

Written by Jon Bashor

New DOE Blueprint to Pave the Way for a Nationwide Quantum Internet

Quantum Internet Blueprint Workshop Steering Committee

DOE Quantum Internet Blueprint Workshop ReportAs modern computers begin to reach the limit of their processing power, quantum computing has the potential to solve more specialized problems that require immensely robust computing. With that potential capturing the imagination of many, a consensus is building that a communication system using quantum mechanics represents one of the most important technological frontiers of the 21st century. To harness the full promise of quantum computing and sensing, we need to build scalable quantum communication networks that can support applications across science, industry, and national security. Scientists now believe that the construction of a prototype “quantum Internet” will be within reach over the next decade.

Toward this end, on July 23 the U.S. Department of Energy (DOE) unveiled a report that lays out a blueprint strategy to accelerate research in quantum science and technology, with an emphasis on the creation of a quantum Internet. “The Department of Energy is proud to play an instrumental role in the development of the national quantum Internet,” U.S. Secretary of Energy Dan Brouillette said in a news release. “By constructing this new and emerging technology, the United States continues with its commitment to maintaining and expanding our quantum capabilities.”

In support of this and related efforts, the Energy Sciences Network (ESnet) – a DOE Office of Science user facility managed by Lawrence Berkeley National Laboratory – is actively tracking multiple quantum networking projects and collaborating with the research community to help lay the groundwork for scalable quantum communication networks and a quantum Internet. In this interview with ESnet Director Inder Monga, he talks about the future of quantum networking and its role in facilitating quantum information science across the DOE and beyond.

How will the DOE’s quantum networking blueprint impact the development of quantum communications and applications?

Quantum Internet Blueprint Workshop Steering Committee
The Quantum Internet Blueprint Workshop chairs. Top row left to right: Inder Monga (ESnet) and Gabriella Carini (BNL). Bottom row left to right: Nicolas Peters (ORNL), Kerstin Kleese van Dam (BNL), Joseph Lykken (Fermilab), Thomas Schenkel (Berkeley Lab).

Researchers believe that a quantum Internet could have a profound impact on a number of application areas critical to science, national security, and industry. Application areas include upscaling quantum computing by connecting distributed quantum computers, quantum sensing through a network of quantum telescopes, quantum metrology, and secure communications.

DOE’s 17 National Laboratories will serve as the backbone of a proposed nationwide quantum Internet, which will rely on the laws of quantum mechanics to control and transmit information more securely than ever before. Earlier this year I co-chaired the first Quantum Internet Blueprint Workshop, designed to begin laying the groundwork for this complex undertaking. (Details can be found in the workshop report released July 23.) This meeting was a great first step in articulating what challenges need to be addressed in order to create a quantum Internet with a coordinated research roadmap. The diversity of attendance at the meeting itself was an example of how such grand challenges can be tackled with collaboration across DOE national labs, universities, industry, and various government entities, including  NASA, NIST, NSF, and others.

The workshop explored the specific research and engineering advances needed to build a quantum Internet in the near term, along with the path to move from today’s limited tabletop experiments to a viable, metro-to-wide-area, secure quantum Internet. Participants identified four priority research opportunities and five key milestones that must be achieved to build the foundation for a quantum network:

  • Research Priorities
    • Provide the foundational building blocks for a quantum Internet
    • Integrate multiple quantum networking devices
    • Create repeating, switching, and routing for quantum entanglement
    • Enable error correction of quantum networking functions
  • Roadmap Milestones
    • Verification of secure quantum protocols over fiber networks
    • Inter-campus and intra-city entanglement distribution
    • Intercity quantum communication using entanglement swapping
    • Interstate quantum entanglement distribution using quantum repeaters
    • Build a multi-institutional ecosystem to transition from demonstration to operational infrastructure

What role will a high-speed, high-performance, dedicated science research network like ESnet play in facilitating the adoption of quantum technologies?

Inder Monga and Eden Figueroa
ESnet engineers worked with researchers at Brookhaven National Laboratory and Stony Brook University to test quantum entanglement across the Stony Brook campus leveraging existing ESnet fiber pairs, achieving long-distance entanglement of 18 km using an existing ESnet communications fiber network. Since that initial work, Stony Brook and Brookhaven have established an 80-mile quantum network testbed.ESnet’s Inder Monga (right) talks with Stony Brook’s Eden Figueroa, lead investigator of the quantum networking testbed project.

ESnet is a DOE user facility that connects all of the DOE national labs with a high-speed “classical” network built over leased fiber pairs that span more than 15,000 miles nationwide. In order to build an operational and manageable quantum network, we need to research and build new quantum devices and build new protocols and control systems to integrate, control, manage, and monitor those devices and systems to achieve end-to-end communications. While the classical networks will not have access to the secure data being carried by the quantum channel, using the classical network for control and management is critical to a quantum network’s operational success.

ESnet not only brings the existing nationwide infrastructure and connections to National Labs, it also brings its operational expertise and protocol knowledge to work hand-in-hand with the quantum physicists, scientists, and device and system manufacturers to ensure the right mechanisms are in place to realize DOE’s vision of a quantum Internet.

What research is ESnet helping to enable, and what more still needs to happen to make long-distance quantum communications a practical reality?

Just like the first modems developed in the 1960s leveraged the well-established copper-based telephone network to connect teletype terminals and send data at 110 bits per second (compared to 800 gigabits-per-second modems over fiber today), quantum networking technologies are at an early stage of development. Most current quantum networking research uses photons over either fiber or free-space as the preferred medium of transmission.

At this point, there are many significant investments in small but highly collaborative quantum networking research and prototype deployments in different areas of the country, including Brookhaven/Stony Brook, MIT/Harvard/Lincoln Labs, ANL/University of Chicago, FermiLab/Caltech/JPL/Northwestern, Oak Ridge National Laboratory, and more.

As researchers across the nation build and test fundamental technologies needed to build the quantum internet, ESnet is helping build the infrastructure and provide support to those projects. For example, ESnet engineers worked with researchers at Brookhaven and Stony Brook to test quantum entanglement across the Stony Brook campus leveraging existing ESnet fiber pairs, achieving long-distance entanglement of 18 km using unique quantum entanglement sources and an existing ESnet communications fiber network. Since that initial work, Stony Brook and Brookhaven have established an 80-mile quantum network testbed. ESnet is also working closely with Stony Brook, FERMI, ANL, and Caltech researchers to see how we can support the infrastructure build to expand the reach of their quantum networking research, and we are in conversation with other exploratory testbed projects.

One of the key building blocks to scale these regional demonstrations is the quantum repeater. The repeater is an essential piece of the quantum network that will enable transmission of quantum information across large distances. Many of the testbeds mentioned above are aggressively building breadboard versions of the quantum repeater, competing with other nations to create a first viable repeater system that can be deployed widely. Each of them have a different scientific approach to the problem, for example, the use of quantum memories, and that diversity of research ideas at this stage is extremely important for us to find the right solution that will scale.

What do you see as the next steps to realizing the goal of building a nationwide quantum Internet?

The blueprint report describes in detail the five key milestones that will demonstrate progress toward the ultimate goal of building a nationwide quantum Internet. Here I provide my own perspective on the next steps that will help us realize this goal.

First, we need capable quantum network devices that blend the quantum protocols with classical control. In addition to the grand challenge of building a deployable quantum repeater, an ecosystem of quantum devices from efficient quantum memory; transducers for quantum sources; high-speed, low-loss quantum switches; and much more are needed. Many research labs across the U.S. are working on these technologies, and the first big milestone will be to take these devices, and reliable quantum entanglement/distribution, from laboratory-level readiness to acceptable field-level readiness. Following the “team science” philosophy of Berkeley Lab, this step will not just involve the researchers and physicists but will require collaboration with engineers who have experience deploying and managing components in the field.

Second, once we have deployable and supportable components, we need to gain experience running and operating these devices. It is wonderful that Brookhaven, Fermi, Argonne, UChicago, Northwestern, Oakridge, Caltech, MIT, and others have built or are planning to build capable free-space and/or fiber-based regional testbeds. We can also think about expanding these testbeds using existing dark fiber from Berkeley to SLAC to Caltech, or Brookhaven/Stonybrook to MIT/Harvard, or the Argonne/Fermi/Chicago regional testbed to Oakridge as the capabilities of the devices expand.

In addition to physical devices, quantum entanglement, and teleportation techniques, the classical networking protocols and techniques to control, manage, and operate the quantum network are extremely important. At this stage of development, it is critical to let a thousand flowers bloom. ESnet, with its practical experience, can help design testbeds, connect these testbeds to quantum applications, and support end-to-end tests to help the researchers focus on the techniques that are most viable and easily scalable across the nation. This principle of co-design has been impactful across the DOE Office of Science projects and I hope will be applied to the quantum Internet efforts as well.

Finally, I remember working on one of the leading multi-protocol routers in the 90s with protocols like X.25, Appletalk, IPX, OSI, and many others that have now faded away. With the many approaches to quantum routing and potential protocols to control these devices, we will need a testbed that allows neutral testing of these various research approaches and maybe even determines the interoperability of the various free-space, satellite, and fiber-based techniques. We will also need to build strong collaborations not only between the DOE labs, quantum centers, and the science applications, but also with other agencies like NASA, NIST, NSF, and others that have investments in this space. This is the only way we can leverage the knowledge and expertise of the broader scientific community to reach the vision outlined in the Quantum Internet Blueprint report.

Interview by Kathy Kincade, Berkeley Lab Computing Sciences

 

5G For Science: How Research Will Benefit from Advanced Wireless?

5G Andrew

5G is the next-generation wireless network that will give you much faster Internet connections. That means massive files, like high-definition movies, that take you about six minutes to download over a 4G LTE network, could be downloaded in a matter of seconds over the 5G network. And because of its innovative design, 5G is about to change the way things like cars, TVs, and even buildings connect to the Internet.

The Department of Energy’s national labs, sponsored by the Office of Science, are currently working to identify opportunities on how science can leverage 5G and other advanced wireless technologies. The Office of Science recently published a report on its findings.     

ESnet Computer Systems Engineer Andrew Wiedlea helped facilitate discussions and report findings. We recently caught up with him to talk about the benefits of 5G and other advanced wireless technologies for science, and what it will take to make it available for research. 

What is 5G and Advanced Wireless? And, how could science benefit from it?

Floating Robots
Berkeley Lab and UC Berkeley researchers unleashed 100 floating sensors to understand how water flows through the Sacramento-San Joaquin Delta on its way to pumping stations and San Francisco Bay.  The sensors transmitted data to the National Energy Research Scientific Computing Center for assimilation and analysis. (Photo by Roy Kaltschmidt).

Scientific data movement is on the cusp of a new era for flexible, low-cost deployment of scientific sensors and data mobility. Advanced wireless capabilities offer the promise of solving the “last mile problem” for science, which is creating new ways for scientists to connect data from sensors, vehicles, and isolated locations, with U.S. Department of Energy’s world-class supercomputers. It’s important to note that advanced wireless will not replace high speed scientific optical networks for large-scale wired “backbone” connectivity, rather we will solve the last mile problem through the integration of advanced wireless- and wired- backhaul. 

5G technology is one part of this emerging wireless data connectivity era. In addition to emerging low-orbit satellite constellation non-terrestrial networks, terrestrial millimeter wireless (mmWave), 5G “New Radio” capabilities will be deployed both by commercial vendors and non-commercial entities (using open parts of the radio frequency spectrum-space) to support myriad uses. Because 5G operates over a very wide range of radio frequencies (600 MHz to 27 GHz) and also leverages advances made since the deployment of earlier cellular radio communication standards, such as software defined networking, beam steering, and improved signal processing, 5G will allow users (including the scientific community) to engineer wireless data transmission supporting novel sensing applications for the world around us.

What makes 5G different from previous wireless standards for science? 

accessibility-browsing-5g-business5G is built around three standards, each of which leverages network resources in different ways.  Each of these application models will be leveraged by scientists depending on their needs:

Enhanced Mobile Broadband: The main benefit of 5G comes from a great increase in the ability to spatially reuse the radio spectrum. In comparison to previous cellular network standards, 5G networks will support higher data rates, and an ability to support many more subscribing devices wherever this is needed.  For scientists, this will mean much improved options for sensor networks, Internet of Things (IoT) applications, lower wireless data costs, and (hopefully) less reliance on “sneakernet” or other improvised methods for data collection and movement.

Ultra Reliable and Low Latency Communications: 5G supports deployment modes based around defined service levels, which means users will be able to reserve “slices” of capacity in a way similar to reserving circuits on a wired network. This, combined with other capabilities, will allow 5G to support scientific uses where communications reliability is essential, such as when measurements depend on near-real-time interaction with instrument control systems or as part of operating mobile systems such as unmanned aerial vehicles.

Massive Machine Type Communications: 5G is also built to support deployment modes in support for low power, automated systems.  These capabilities will be of benefit for all kinds of urban applications, but particularly so for scientists leveraging 5G for urban or building applications.  Leveraging this standard, scientists will be able to deploy hundreds or even thousands of small, very power-efficient, sensors throughout buildings or other areas to measure energy or environmental factors.

Taken as a whole, the capabilities provided by advanced wireless (5G, non-terrestrial networks, and mmWave) will allow new kinds of science, both within the confines of the laboratory and outside in a world via commercial and national laboratory dense sensor networks. Both the types and amounts of data generated will greatly increase – as will the scientific opportunities to learn new things.

What role is ESnet playing in creating a 5G network for scientists?

Cori Supercomputer
Advanced wireless capabilities are creating new ways for scientists to connect data from sensors, vehicles, and isolated locations, with world-class supercomputers like the National Energy Research Scientific Computing Center’s (NERSC’s) Cori system. (Picture by Roy Kaltschmidt)

ESnet’s mission is to ensure that science collaborations—at every scale and in every scientific domain—have the information and tools they need to achieve maximum benefit from global networks. This mission is not defined by a particular technology. ESnet works to integrate the compute, storage, and analytic resources operated by sites within the Department of Energy complex, and our scientific customers. 

Unlike previous generations of sensor or data infrastructure development, such as the Internet, Advanced Wireless and 5G advances are largely occurring without the US National Laboratory system playing lead roles. The challenge for scientific users is primarily one of connecting wireless technology (when needed) into the toolset made available by the Department of Energy to support US and global science objectives.  

ESnet inherently must support these customer efforts because we operate the high-speed scientific data network upon which the community depends now, and in the future as next-generation capabilities (ESnet 6) come to life.  We are also at the forefront of thinking about next-generation data movement and analytics through leadership roles with the National Science Foundation’s FABRIC program, software defined networking, and other projects supporting the Department of Energy’s future vision for the science laboratory system. 

At the Lawrence Berkeley National Laboratory (Berkeley Lab), where ESnet is headquartered, we are working to develop a community of interest on 5G and advanced wireless applications, and have been using this as a forum to develop ideas, and bring in external speakers to provide technical talks on 5G state of the art.  

ESnet’s Science Engagement Team is also starting to work with the Applied Physics Program and others to test aspects of advanced wireless technology, as well as how we can connect this to ESnet work in edge computing, our ScienceDMZ architecture, and other Berkeley Lab resources.  We have also started to develop research relationships with the UC Berkeley advanced wireless community, especially the Wireless Research Center to explore mmWave capabilities.  Outside of Berkeley Lab, we have been very active in the Department of Energy’s Enabled Energy Innovation Workshop (5GEEIW)  and related discussions for science uses of 5G and future requirements, as well as discussions with other labs and commercial entities about collaboration on testbeds and prototyping use cases.  These efforts will grow over the next year and hopefully, the report just released from the 5GEEIW will contribute to this progress. [link here]

Are there any experiments looking to use 5G? 

16538042638_cf555c1932_k
Argonne National Laboratory’s  Waggle platform is a novel wireless sensor with advanced edge computing capabilities to enable a new breed of sensor-driven environmental science and smart city research. (Photo by Mark Lopez, Argonne National Laboratory)

Around the Department of Energy complex, many teams are starting to look at the use of 5G to support experiments.  There are also developing applications for inside building and laboratory use as well using unlicensed 5G spectrum—some of this application space is now served by either Wifi or wired connectivity. There is a need for some general networking research to explore how ESnet wired capabilities, such as caching and data transfer nodes, should be deployed as part of wired-wireless interfaces, and to develop patterns for scientific support for projects making use of advanced wireless technologies as part of ESnet support for science.

We, along with Argonne National Laboratory, Pacific Northwest National Laboratory, and other Labs, are developing ideas for 5G/Advanced Wireless testbed and prototype application testing environments.  At present, the availability of equipment and service is limited, but this is expected to change rapidly as the first generation of 5G handsets and other devices begin to flood the market.

What is the state of 5G now? How long will it be until scientists can access it?

5G is being commercially rolled out by carriers now, and the build-out of this service is expected to take several years.  Other resources, such as IoT 5G toolsets and hardware are also beginning to reach the market from Ericsson and other vendors.  Similarly, non-terrestrial network constellations such as StarLink are beginning to support limited communities of beta-testers, and mmWave resources are also becoming commercially available.  

Thanks to Berkeley Lab IT’s stellar work with Verizon, however, we hope that there will be options over this next year for Berkeley-community access to 5G testing resources, and similar opportunities to explore mmWave or non-terrestrial networks tools as we build relationships and capabilities.  We also believe that opportunities and resources will start to become available over this next year from the Department of Energy, and other funding sources to support science user testing and the uptake of advanced wireless.

How did you get into this work and what do you enjoy most about it?

I got into this area at the start of my career working on satellite mobile telephony, and later with the Department of Defense working on sensors and analysis systems. When I was supporting military forces in the field with analytics, the problem was always how to handle really data thin-pipes, and as part of this, we had a lot of trouble with existing radio, cellular and satellite options. 

As part of the research-support community, I’m most interested in how we can use 5G and advanced wireless technologies to allow scientists to do new things. It is really fascinating to be at a point of inflection, for RF wireless technology and the ability for almost anyone to be able to affordably collect data from the world, backhaul that data globally, and make sense of it.  

I think that we are in a great position to lead the way with open science “out in the world” which will leverage these new technologies and ESnet is a wonderful place to serve that cause.

Interviewed by Linda Vu, Berkeley Lab Computing Sciences

Meet Todd Anderson, ESnet’s New Director of Systems & Software

Todd Anderson may be new to his role as thTodd Croppede Energy Sciences Network’s (ESnet) Director of Systems and Software Engineering, but he’s already made history.

Anderson is the first ESnet staff member to be hired and onboarded completely virtually. And because of the Bay Area’s extended shelter-in-place order, he will be spending his first months on the job managing his team remotely from home in Lafayette, California.

Before coming to ESnet, he spent 20 years working at the executive level of a technology company that provided software-as-a-service (SaaS) solutions to financial institutions, including services in account identity risk management, fraud prevention, and digital payments, specifically the Zelle payment platform.

“It was a good gig. We were preventing fraud, but at the end of the day, our mission was to help big banks optimize their bottom line,” said Anderson. “I have an engineering background and I wanted to do something a little more meaningful. So, I looked for organizations around the Bay Area working in the fields of sustainability, cleantech, and renewable energy. That’s when I saw this opportunity at ESnet.”

At ESnet, he gets to apply his experience to manage the teams that develop and deploy tools to allow scientific users to optimize their use of the network.

“It’s really exciting to hear about the science projects I will be supporting as a member of the ESnet staff,” said Anderson. “And ESnet is doing its own networking research, too. It’s cool to be on a conference call and hear people talking about quantum networking and 5G.”

As a child during the space race, Anderson remembers watching the Apollo and space shuttle missions. That experience inspired him to major in mechanical engineering at the University of Colorado, Boulder, so he could one day build spacecraft for NASA. But after graduation, life took a different turn. Following in the footsteps of a friend he admired, he joined the Peace Corps and spent two years teaching math and science to middle school children in Botswana’s Kalahari Desert.

“Service really makes you feel like you are doing something worthwhile and making the world a better place,” said Anderson.

When he returned to the United States, a friend asked him to help write a software application to detect merchant credit card fraud. This move kicked-off of his 30-year career in technology.

In his free time, Anderson enjoys doing things around the house. When the world isn’t in the midst of a pandemic, he likes to be out in nature and to sample the diverse culture and geography of the Bay Area with his family.

Written by Linda Vu

ESnet Builds Morale and Community With a Zoom Competition

Nearly two months into California’s shelter-in-place order, we’ve all been in more than our fair share of video conferences. To boost morale during this difficult time, the Energy Sciences Network (ESnet) staff held a Zoom Background Competition during their all-to-all meeting on Monday, April 27. 

Staff were encouraged to create their own backgrounds and display them during the meeting. There were 21 entries. ESnet employees voted. Submissions were judged on overall artistry, functionality (not too distracting as a background), whether it elevated the voter’s mood, and if it made them feel included in the ESnet community. 

The top three winners got bragging rights. Here they are:

First place: Jeff Berman, NOC Engineer

This Zoom challenge inspired Berman, an avid sailoJeffrey Bermanr, to take to the sea. He won this competition with an hour video of the San Francisco skyline, one he filmed while sailing on the Bay. Although he typically likes to go sailing with friends and family, he says that sailing solo brings him a sense of peace, calm, and tranquility.  

“What is sailing? Most books define it as hours of sheer boredom scattered with white knuckle periods of terror. On a good day, both are true. Both give you an equal sense of accomplishment. How to be with yourself with nothing to do, good training for our current situation,” said Berman.

Second Place: Sartaj Baveja, Software Engineer

This challenge inspired Baveja to create a background meme of office life. In the background, someone (Baveja) is looking over your shoulder to catch a glimpse of your screen and make sure you don’t procrastinate.

Sartaj Baveja

Third Place: Joe Metzger, Network Engineer

Joe_interviewThis challenge inspired Metzger to use a picture that he took in Barcelona. The focal point of the picture (the blur) is a little girl in a red coat, black dress and white tights who was just running back and forth between the pools of light and shadow created by the stone arches and rosette windows, while her family was sitting in the cafe. 

“I used this as my zoom background because I think it is a really cool picture. It brings to mind a fun evening strolling around the little squares and back streets in Barcelona and sitting in cafes with a good glass of wine relaxing,” said Metzger.

Girl in Red

Written by Linda Vu, Berkeley Lab Computing Sciences.