Data flows have certain characteristics that make them suited for certain types of networks. It is a complex problem to match flows with the “right” networks. In the ESnet core network, one can identify flows by looking at multiple fields in packet headers, according to Veeraraghavan, but you can’t know the size of the flow (bytes) or whether a flow is long or short. A challenge of this project is to predict characteristics of data flows based on prior history.  To do this, the researchers are using machine learning techniques. Flows are classified based on size and duration. Large-sized (“elephant”) flows are known to consume a higher share of bandwidth and thus adversely affect small-sized (“mice”) flows. Therefore, they are good candidates to redirect to the SDN. If SDN circuits are to established dynamically, i.e., after a router starts seeing packets in a flow that history indicates is a good candidate for SDN, then the flow needs to not only be large-sized but also of long-duration (“tortoise”) because circuit setup takes minutes. Short-duration (“dragonfly”) flows are not good candidates for dynamic circuits, but if they are of large size and occur frequently, static circuits could be used.

The concept of different lanes handling different types of traffic is seen commonly in other contexts. For example, Veeraraghavan notes that on some urban streets with mixed traffic, “separate lanes are set aside for buses, cars, motorcycles, and bicycles.” Also, grocery store checkouts have separate express lanes for the equivalent of “mice flows”.  To support this concept, the researchers developed several modules of a system called Hybrid Network Traffic Engineering Software (HNTES), and tested these modules on the ANI Testbed.

Their experiments use two computers loaded with the HNTES software, and two Juniper routers. The HNTES software configures the routers to mirror packets of certain pre-determined flows (this determination is made with an offline flow analysis tool that analyzes previously collected Netflow data to find flow identifiers of elephant and tortoise flows) to a server that runs a flow-monitoring module (part of HNTES). Upon detecting such a flow, HNTES reconfigures the router to redirect packets from this flow to a circuit (different path). In future versions, if dynamic circuits are deemed feasible, a HNTES module called IDC Interface Module will send messages to an OSCARS IDC server to reserve and provision a circuit before reconfiguring the router.  So far Veeraraghavan and her colleagues have completed phase I of the software implementation, and demonstrated it. The demonstrations were presented in a Oct. 2010 DOE annual review meeting with previously recorded CAMtasia video. The next step will be to improve several features in the software to understand what happens to the flow when it is redirected; do packets get lost, or does redirection cause out of order arrivals at the destination? They are also doing a theoretical study with flow simulations, to see if taking the trucks off the parkway and putting them on the freeway really benefits the flow of “traffic.”