Three questions with a new staff member on our Software Engineering – Orchestration and Core Data Team!
Karim comes to us from Carnegie Mellon University, where he served as a Software Engineer in the Network Services group. In that role, he designed, implemented, deployed, and maintained numerous applications to provide support to the campus network infrastructure. He has worked on a diverse set of network computing problems with a focus on automation and self-service utilities. Karim is proficient in a multitude of application development stacks but has a special place in his heart for those that put Python in the mix.
What brought you to ESnet?
I’ve always had a profound curiosity for the intersection of mathematics, science, and technology. Starting with a strong foundation in mathematics, I learned how to better apply my problem-solving skills by pursuing graduate work in computational biology. It was there that I discovered how next-generation computing technologies could radically transform and elevate entire scientific fields. I’ve been seeking to utilize the skills I’ve built up over my 15 years of industry experience to help build tools for scientists, to empower them, and help them achieve discoveries in a world that is becoming ever increasingly more complex. The work being done at ESnet lines up perfectly with this goal in mind.
What is the most exciting thing going on in software engineering right now?
I would say the rapid proliferation of containerization technologies and the use of cloud infrastructure for distributed computing problems, as well as advancements in machine learning libraries and toolkits that let scientists more easily simplify the manipulation and analysis of large datasets. Many of these concepts were in their infancy or early stages only a decade ago, and now they’re everywhere and I’m happy to see how fast they’ve been adopted.
What book would you recommend?
Time Travel in Einstein’s Universe by J. Richard Gott. An accessible read for laymen like me, about how one would — given some ridiculous assumptions — go about creating various time machines.
You may have noticed that the masthead has changed. After almost a decade, we have finally decided that the “ESnet Blog” deserves a less literal name. “Light Bytes” was selected to better embody two things about which we at ESnet are especially proud.
First, getting to build the world’s greatest research and education network, and to support global science is a great honor and a technical challenge. Through ESnet6 and our continuing research, we are advancing our mission of making scientific data free of geographical constraints, to “make bytes light” in terms of fast transport, and to deploy state of the art optical network for “bytes being transported by light.”
Second, while ESnet is officially a DOE User Facility, it is most importantly a remarkable group of people. We hope that this website will show a bit about the great people who make our mission happen and the interesting problems we get to work on. In that sense, “Light Bytes” is a small written offering, a collection of features about things that are happening while the cause of “networking for science” and “science of networking” progresses.
As a Network Engineer at ESnet, I am no stranger to the importance of designing and maintaining a robust fiber-optic network. To operate a network that will “enable and accelerate scientific discovery by delivering unparalleled network infrastructure, capabilities, and tools,” ESnet has acquired an impressive US continental footprint of more than 21,000 kilometers of leased fiber-optic cable. We spend a great deal of effort designing and sourcing redundant fiber-optic paths to support network data connectivity between scores of DOE Office of Science facilities and research collaborators across the country.
But network data transfer is only one of the uses for fiber-optic cable. What about using buried fiber-optic cable for some truly “ground-shaking” science? The answer is “Yes, absolutely!” – and I was fortunate to play a part in exploring new uses for fiber-optic cable networks this past year.
Back in 2017, the majority of our 21,000 km fiber footprint was still considered “dark fiber,” meaning it was not yet in use. At that time, ESnet was actively working on the design to upgrade from our current production network “ESnet5” to our next-generation network “ESnet6,” but we hadn’t yet put our fiber into production.
At the same time, Dr. Jonathan Ajo-Franklin, then graduate students Nate Lindsey and Shan Dou, and the Berkeley Lab’s Earth and Environmental Science Area (EESA) were exploring the use of distributed acoustic sensing (DAS) technology to detect seismic waves by using laser pulses across buried fiber optic cable. The timing was perfect to try and expand on the short-range tests that Dr. Ajo-Franklin and his team had been performing at the University of California’s Richmond Field Station by using a section of the unused ESnet dark fiber footprint in the West Sacramento area for more extensive testing. ESnet’s own Chris Tracy worked with Dr. Ajo-Franklin and team to demonstrate how the underground fiber-optic cables running from West Sacramento northwest toward Woodland in California’s Central Valley made an excellent sensor platform for early earthquake detection, monitoring groundwater, and mapping new sources of potential geothermal energy.
Fast forward to May 2019, and Dr. Ajo-Franklin was heading up a new collaborative scientific research project for the DOE’s Geothermal Technology Office based on his prior DAS experimentation successes using ESnet fiber. The intent was to map potential geothermal energy locations in the California Imperial Valley south of the Salton Sea, near Calipatria and El Centro. The team, including scientists in EESA, Lawrence Livermore National Laboratory (LLNL), and Rice University needed a fiber path to conduct the experiment. It would make sense to assume that ESnet’s fiber footprint, which runs through that area, would be an excellent candidate for this experiment. Fortunately for ESnet’s other users, but unfortunately for the DAS team, by 2018 the ESnet6 team was already “lighting” this previously dark fiber.
However, just because ESnet fiber in the Imperial Valley was no longer a candidate for DAS-based experiments, that didn’t mean there weren’t ways to gain access to unused dark fiber. For every piece of fiber that has been put into production to support ESnet6, there are dozens if not hundreds of other fibers running right alongside it. When fiber-optic providers install new fiber paths, they pull large cables consisting of many individual fibers to lease or sell to as many customers as possible. Because the ESnet fiber footprint was running right through the Imperial Valley, we knew that there was likely unused fiber in the ground, and only had to find a provider that would be willing to lease a small section to Berkeley Lab for Dr. Ajo-Franklin’s experiment.
Making the search a little more complicated, the DAS equipment utilized for this experiment has an effective sensing range that is limited to less than 30 kilometers. Most fiber providers expect to lease long sections of fiber connecting metropolitan areas. For example, the fiber circuits that run through the Imperial Valley are actually intended to connect metropolitan areas of Arizona to large cities in Southern California. Finding a provider that would be willing to break up a continuous 600 km circuit connecting Phoenix to Los Angeles just to sell a 30 km piece for a year-long research project would be a difficult task.
One of my contributions to the ESnet6 project was sourcing new dark fiber circuits and data center colocation spaces to “fill out” our existing footprint and get ready for our optical system deployments. Because of those efforts, I knew that there were often entire sections of fiber that had been damaged across the country and would likely not be repaired until there was a new customer that wanted to lease the fiber. I was asked to assist Dr. Ajo-Franklin and his team to engineer a new fiber solution for the experiment. I just had to find someone willing to lease us one of these small damaged sections.
After speaking with many providers in the area, the communications company Zayo was able to find a section of fiber starting in Calipatria, heading south through El Centro and then west to Plaster City, that was a great candidate for DAS use. This section of fiber had been accidentally cut near Plaster City and was considered unusable for networking purposes. Working with Zayo, we were able to negotiate a lease on this “broken” fiber span along with a small amount of rack space and power to house the DAS equipment that Dr. Ajo-Franklin’s team would need to move forward with their research.
This cut fiber segment was successfully “turned up” for the project on November 10, 2020 by a team including Co-PI Veronica Rodriguez Tribaldos, Michelle Robertson, and Todd Wood (EESA/LBNL), and seismic data collection equipment is now up and running. The figure above (D) shows some great initial data recorded on the array, a small earthquake many miles to the north. There will be many more articles and reports from the Imperial Valley Dark Fiber Team as they continue to gather data and perform their experiments, and I’m sure we’ll begin to see fiber across the country put to use for this type of sensing and research.
I’ve had a great experience working with the different groups that were assembled for this project. By seeing how new technologies and methods are being developed to use fiber-optic cable for important research outside of computing science, I’ve developed a greater appreciation for how our labs and universities are tackling some of our biggest energy and public safety challenges.
ESnet is proud to welcome Rede Nacional de Ensino e Pesquisa (RNP), the national research and education network of Brazil, as an official collaboration partner on the perfSONAR project. The official announcement on the perfSONAR website is here. RNP joins five other organizations (ESnet, GEANT, Indiana University, Internet2, and the University of Michigan) committed to providing dedicated resources that develop and maintain the perfSONAR software.
Even though RNP is now becoming an official project member, they have been part of the perfSONAR community for the past 15 years. RNP has used their own perfSONAR fork for eight years and recently moved to the baseline version of perfSONAR across RNP’s 27 points of presence. The use of the perfSONAR branch code will provide network engineers and customers with improved abilities to maintain their network and validate on-demand circuit use.
Going forward, RNP will be a key contributor in several areas. Iara Machado will be working in conjunction with perfSONAR’s existing steering committee members to provide executive level guidance for the project. Additionally, Marcos Schwarz will join the perfSONAR leadership team to not only spearhead the RNP development team, but also help manage the day-to-day development of the project with existing partner institutions. Initial interests include containerization, perfSONAR as microservices, standard display and analysis packages, and possibly making their circuit validation tool available to the broader community. RNP’s experience and insight will be invaluable to the perfSONAR project going forward.
ESnet and the entire perfSONAR collaboration are excited to officially have them on the team. Having partners like RNP strengthens the perfSONAR initiative and helps ensure a bright future in its continued role as a critical piece of infrastructure for high-performance scientific networks.