ESnet is proud to welcome Rede Nacional de Ensino e Pesquisa (RNP), the national research and education network of Brazil, as an official collaboration partner on the perfSONAR project. The official announcement on the perfSONAR website is here. RNP joins five other organizations (ESnet, GEANT, Indiana University, Internet2, and the University of Michigan) committed to providing dedicated resources that develop and maintain the perfSONAR software.
Even though RNP is now becoming an official project member, they have been part of the perfSONAR community for the past 15 years. RNP has used their own perfSONAR fork for eight years and recently moved to the baseline version of perfSONAR across RNP’s 27 points of presence. The use of the perfSONAR branch code will provide network engineers and customers with improved abilities to maintain their network and validate on-demand circuit use.
Going forward, RNP will be a key contributor in several areas. Iara Machado will be working in conjunction with perfSONAR’s existing steering committee members to provide executive level guidance for the project. Additionally, Marcos Schwarz will join the perfSONAR leadership team to not only spearhead the RNP development team, but also help manage the day-to-day development of the project with existing partner institutions. Initial interests include containerization, perfSONAR as microservices, standard display and analysis packages, and possibly making their circuit validation tool available to the broader community. RNP’s experience and insight will be invaluable to the perfSONAR project going forward.
ESnet and the entire perfSONAR collaboration are excited to officially have them on the team. Having partners like RNP strengthens the perfSONAR initiative and helps ensure a bright future in its continued role as a critical piece of infrastructure for high-performance scientific networks.
Scientific discovery increasingly relies on the ability to perform large data transfers across networks operated by many different providers (including ESnet) around the globe. But what happens when a researcher initiates one of these large data transfers and data movement is slow? What does “slow” even mean? These can be surprisingly complex questions and it is important to have the right tools to help answer them. perfSONAR is an open source software tool designed to measure network performance and pinpoint issues that occur as data travels across many different networks on the way to a destination.
perfSONAR has been around for more than 15 years and is primarily maintained today by a collaboration of ESnet, GEANT, Indiana University, Internet2, and the University of Michigan. perfSONAR has an active community that extends well beyond the five core organizations that maintain the software with more than 2000 public deployments that span six continents and hundreds of organizations. perfSONAR deployments are capable of scheduling and running tests that calculate metrics including (but not limited to) how fast a transfer can be performed (throughput), if a unit of information makes it to a desired destination (packet loss), if so how long did it take (latency) and what path did it take to get there (traceroute). What is novel about perfSONAR is not just these metrics, but the set of tools to feature these metrics in dashboards built by multiple collaborating organizations. These dashboards aim to clearly identify patterns that signify potential issues and provide the means to drill-down into graphs that give more information.
While perfSONAR has had great success in providing the current set of capabilities, there is more that can be done. For example, perfSONAR is very good at correlating metrics it collects with the other perfSONAR metrics with at least one similar endpoint. But what if we want to correlate the metrics by location, intermediate network or with non-perfSONAR collected statistics like flow statistics and interface counters? These are all key questions the perfSONAR project is looking to answer.
Building upon a strong foundation
PerfSONAR has the ability to add analytics from other software tools using a plug-in framework. Recently, we have begun to use Elastic Search via this framework, to ingest log data and enable improved search and analytics on perfSONAR data.
For example, traditionally perfSONAR has viewed an individual measurement as something between a pair of IP addresses. But what do these IP addresses represent and where are they located? Using off-the-shelf tools Elastic Search in combination with Logstash, perfSONAR is able to answer questions like “What geographic areas are showing the most packet loss?”.
Additionally, we can apply this same principle to traceroute (and similar tools) that yield a list of IP addresses giving an idea of the path a measurement takes between source and destination. Each IP address is a key to more information about the path including not only geographic information but also the organization at each point. This means you can ask questions such as “What is the throughput of all results that transit a given organization?”. Previously a user would not only have to know the exact address of the IPs, but it would have to be the first (source) or last (destination) address in the path.
Integration with non-perfSONAR data is another area the project is looking to expand. By putting perfSONAR data in a well established data store like Elasticsearch, the door is open to leverage other off-the-shelf open source tools like Grafana to display results. What’s interesting about this platform is not only its ability to build new visualizations, but also the diverse set of backends it is capable of querying. If data such as host metrics, network interface counters and flow statistics are kept in any of the supported data stores, then there is a means to present this information along perfSONAR data.
These efforts are very much still in their early stages of development, but initial indicators are promising. Leveraging the perfSONAR architecture in conjunction with the wealth of off-the-shelf open source tools available on the market today create opportunities to gain new insights from the network, like those described above, not previously possible with the traditional perfSONAR tools.
Getting involved and learning more
The perfSONAR project will continue to provide updates as this work progresses. You can also see the perfSONAR web site for updates and more information on keeping in touch through our mailing lists. The perfSONAR project looks forward to working with the community to provide exciting new network measurement capabilities.
Attending SC15? Get a Close-up Look at Virtualized Science DMZs as a Service
ESnet, NERSC and RENCI are pooling their expertise to demonstrate “Virtualized Science SMZs as a Service” at the SC15 conference being held Nov. 15-20 in Austin. They will be giving the demos at 2:30-3:30 p.m. Tuesday and Wednesday and 1:30-2:30 p.m. Thursday in the RENCI booth #181.
Here’s the background: Many campuses are installing ScienceDMZs to support efficient large-scale scientific data transfers. There’s a need to create custom configurations of ScienceDMZs for different groups on campus. Network function virtualization (NFV) combined with compute and storage virtualization enables a multi-tenant approach to deploying virtual ScienceDMZs. It makes it possible for campus IT or NREN organizations to quickly deploy well-tuned ScienceDMZ instances targeted at a particular collaboration or project. This demo shows a prototype implementation of ScienceDMZ-as-a-Service using ExoGENI racks (ExoGENI is part of NSF GENI federation of testbeds) deployed at StarLight facility in Chicago and at NERSC.
The virtual ScienceDMZs deployed on-demand in these racks use the SPOT software suite developed at Lawrence Berkeley National Laboratory to connect to a data source at Argonne National Lab and a compute cluster at NERSC to provide seamless end-to-end high-speed data transfers of data acquired from Argonne’s Advanced Photon Source (APS) to be processed at NERSC. The ExoGENI racks dynamically instantiate necessary compute virtual resources for ScienceDMZ functions and connect to each other on-demand using ESnet’s OSCARS and Internet2’s AL2S system.
Jason Zurawski of ESnet’s Science Engagement Team gave presentations on the Science DMZ architecture and perfSONAR network measurement toolkit at a two-day workshop held last month at Penn State. The workshop, which aimed to strengthen campus cyberinfrastucture, drew more than 30 higher education network engineers representing 11 higher education institutions.
The workshop was a collaboration of ESnet, the Keystone Initiative for Network Based Education and Research (KINBER) and Penn State with funding from the National Science Foundation. Zurawski and other members of ESnet’s Science Engagement Team regularly participate in similar workshops and give webinars to share ESnet’s expertise and experience with campuses and regional networks as they handle increasingly large data flows.
Jason Zurawski of ESnet’s Science Engagement team will lead a March 4 webinar on “Upgrading Campus Cyberinfrastructure: An Introduction to the Science DMZ Architecture” for research and education organizations in Pennsylvania.
Zurawski will introduce ESnet’s Science DMZ Architecture, a network design pattern designed to streamline the process of science and improve the outcomes for researchers. Included in this design are network monitoring concepts via the perfSONAR framework, as well as functional components used to manage security and manage the transfer of data. This design pattern has roots in high speed networks at major computing facilities, but is flexible enough to be deployed and used by institutions of any size. This solution has been successfully deployed on numerous campuses involved in the NSF CC-IIE and CC-NIE programs, and is a focus area for the upcoming CC-DNI program.
ESnet has deployed the first public 40Gbps production perfSONAR host directly connected to an R&E backbone network, allowing research organizations to test and diagnose the performance of network links up to 40 gigabits per second.
The host, located in Boston, Mass., is available to any organization in the R&E (research and education) networking community. More and more, organizations are setting up their own 40 Gbps data transfer nodes to help systems keep up with the increasing size of research datasets.
The next installment of the Operating Innovative Networks (OIN) workshop series will be held Feb. 27-28 at Lawrence Berkeley National Laboratory, drawing 50 university and laboratory network engineers from as far away as New Jersey, Texas, Illinois and Colorado. The workshop is co-hosted by ESnet, Berkeley Lab and CENIC, the Corporation for Education Network Initiatives in California. The program is designed to give attendees the knowledge and training needed to build next-generation campus networks that are optimized for data-intensive science.
Presented by experts from the Department of Energy’s ESnet, Indiana University and Internet2, the workshop series will focus on Science DMZnetwork architectures, perfSONAR performance measurement software, data transfer nodes and emerging software defined networking techniques. Combined, these technologies are proven to support high-performance, big data science applications, including high-volume bulk data transfer, remote experiment control, and data visualization. The workshops will consist of two days of presentation material, along with hands-on sessions to encourage immediate familiarity with these technologies.
The Berkeley workshop is the fourth in the series and future sessions are planned in Atlanta (March), Boston (May) and Oregon (July). Read more at: http://oinworkshop.com/
In an interview with HPCwire editor Nicole Hemsoth, Brian Tierney, leader of the ESnet Advanced Network Technologies Group at ESnet’s 100G Network Testbed Project, discusses network performance issues, including the 1,000th deployment of the perfSONAR network measurement software.
In the 20-minute interview, Brian talks about how he originally got interested in network performance, the current state of ESnet and how institutions can assess and improve the performance of their network connections, such as referring to the fasterdata.es.net knowledge base. He also describes how Science DMZs can safely transfer massive datasets around firewalls and touches on the potential of Software Defined Networking.
ESnet’s Jon Dugan will lead a Bof on network measurement 12:15, Thurs in room 278-279 at SC10. Functional networks are critical to high performance computing, but to achieve optimal performance, it is necessary to accurately measure networks. Jon will open up the session to discuss ideas in measurement tools such as perfSONAR, emerging standards, and the latest in research directions.
1. Being an essential scientific resource for DOE. ESnet is making great strides in providing optimal connectivity between DOE labs as well as further developing dedicated network resources, such as our securing of dark fiber at Brookhaven. We are laying the groundwork to manage rapidly accelerating increases in DOE scientific networking traffic. The first afternoon, Steve Cotter will give a more detailed update on ESnet’s activities at 2:10 pm and Greg Bell will lead the discussion about the ESnet implications of site reliance on cloud or externally-hosted services at 3:55 pm.
2. Knowing our users better than anyone. Steve Cotter will talk about new ways we will be reaching out to and listening to our users needs during his talk.
3. Setting a global standard for user experience. We may not have invented the seamless user experience, but end to end data transmission is all our users care about. To that end we will be talking about our work on Graphite, URL and Weathermap. Also, Thursday starting at 9:40 am Joe Metzger will report on the PerfSONAR Joint Interagency Demonstration Project followed by Evangelos Chaniotakis’s presentation on ESnet’s virtual circuit services status.
4. Efficiency. Helping our users optimize their networking resources in collaborations, accessing instrumentation and exascale computing needs in the most energy efficient ways possible. Be sure not to miss Wednesday evening’s Focus Session on improving WAN network performance with Eli Dart and Joe Metzger beginning at 6:30 pm.