Experimenting with Programmable Routers in Real Networks





Stanford and Internet2 have collaborated on an experimental backbone network using Internet2's underlying infrastructure. Figure 1 shows the topology of this experimental network of programmable routers. Eventually, we will deploy two experimental networks. One network will be run a stable version of our IPv4 reference router and will carry some production traffic (e.g. PlanetLab traffic). The other network will be more experimental and less stable, running modified routers, and carrying experimental traffic that need not be IPv4.


NetFPGA Routers
Each of our NetFPGA routers consists of a rack-mounted Linux PC (Dell 2950 server), with a NetFPGA PCI card. The NetFPGA board contains an FPGA, four Gigabit Ethernet ports, SRAM and DRAM for packet buffers and lookup tables, and can be programmed to act as a router (or an Ethernet switch, or pretty much any packet processor or NIC card). NetFPGA was originally developed for teaching networking hardware and the NetFPGA website carries several standard reference designs, gateware, software and courseware. The board is available from a 3rd party manufacturer, and is used by a number of colleges and universities around the world for teaching and research. Typically, NetFPGA is installed in a laboratory of 10-20 PCs and the canonical lab exercise is to create a 4x1GE Ethernet switch or Internet router.

Status of our Internet2 deployment
At the time of writing, two NetFPGA routers are installed in each of two Internet2 POPs - in Los Angeles and two in Houston. In summer 2008, we plan to install four more routers - two in New York and two in Washington D.C. The network will expand over time to other Internet2 POPs.

Routing traffic
We use two techniques to force traffic to pass over our backbone network. For outgoing traffic (from our subnet) we tunnel traffic using IP-in-IP. When a NetFPGA router receives encapsulated packets, it decapsulates then processes the packet as usual. For incoming traffic, Internet2 advertises the prefix of our subnet and routes traffic across our backbone. Hence outbound traffic is tunneled, and incoming traffic is not. Today, the routers are interconnected by dedicated 100Mb/s circuits in Internet2, which means we can easily achieve end-to-end performance of 100Mb/s. Shortly, the circuits will be upgraded to 1Gb/s. We expect our own access network to be the bottleneck (although preliminary experiments suggest that several hundred Mb/s are available).

Generating Traffic
In our experiments, we will generate a number of different types of traffic. We will generate small amounts of regular traffic using a mix of applications and iperf on PCs in subnets at Stanford and Rice universities. We will also generate large numbers of flows using Linux-based traffic generators (e.g., Harpoon ). Finally, we hope to divert some of PlanetLab traffic to our network allowing us to study network performance with large amounts of real traffic.