Packet generator

This project provides a simple packet generator and capture tool. It is designed for use by anyone who wants to inject packets into a network and/or observe packets exiting a network.

The generation feature in the current implementation is a "playback" mechanism—a sequence of Ethernet frames is loaded from a PCAP dump file into the generator which then proceeds to transmit that sequence of frames. Delay between individual frames can be tightly controlled; by default the generator will use the delay specified in the pcap source file. A user can optionally specify a maximum data rate and/or a set delay between frames.

The packet generator also be used to report statistics on packets exiting the network and optionally to record the packets exiting the network. These feature are useful for analyzing throughput through a device and for analyzing whether the data exiting a network is correct.

Show Contents...Hide Contents...

Project summary

Status

Released

Current version

1.1.1

NetFPGA package requirement

2.1.1

Authors

Hardware: Glen Gibb (grg@stanford.edu)
Software: Adam Covington, Glen Gibb

Release date

June 18th, 2010

Specifications

Replay timing accuracy

Typically within a few hundred microseconds.
The location of the delay module within the pipeline limits the precision because of buffers that follow. This is a known issue and will be addressed at some point in the future.

Capture timestamp precision

nanoseconds

Capture timestamp accuracy

nominally ± 8ns (+ any added inaccuracy in the NetFPGA clock)
The NetFPGA's core clock is nominally 125MHz (8ns period). Uncertainty in introduced into the measurement due to a clock-domain crossing. (See the datasheets for the clock part/Virtex II Pro for accuracy of the original clock signal.)

Revision history

See Packet generator revision history for the revision history and to download older versions.

Download/Installation

Install from NetFPGA Yum repository

WARNING: The yum package is not yet available for 1.1.1.

1. Install the NetFPGA Base Package
2. Install the packet generator

 yum install netfpga-packet_generator

Install from tarball

Download from here: http://yuba.stanford.edu/netfpga/releases/netfpga_packet_generator_1_1_1.tar.gz

Usage

Preparation

Stand-alone package

WARNING: Standalone package not currently available for 1.1.1.
Run the following command:

 packet_generator_setup.pl

NetFPGA source package

1. Ensure that the NetFPGA kernel driver is loaded and that the CPCI has been reprogrammed.
2. Download the packet generator bitfile:

 nf_download packet_generator.bit

Packet generation/capture

The generator is controlled via the packet_generator.pl command:

   packet_generator.pl
        -q 
   [-r ] (Kbps)
   [-i ]
   [-d ] (ns)
   [-c ]
   [--pad]
   [--nodrop]
   [--wait]
   [--ns]

Options

Note: queue number should be a number between 0 and 3 inclusive, corresponding to nf2c0 through nf2c3.

-q<queue number> <pcap file>

Specify the pcap file to load in and send from a queue

-r<queue number> <rate>

Specify the rate for each queue in Kbps

-i<queue number> <number of iterations>

Specify the number of iterations per queue

-d<queue number> <delay between packets>

Specify the delay between packets in ns. If not specified then the delay recorded in the pcap file is used. A value of 0 will disable the delay.

-c<queue number> <capture file>

Specify the capture file.

--pad

Shorten all packets to 64 bytes maximum and allow the NetFPGA to pad the packets when sending them.

--nodrop

Don't drop packets on input ports for which we're not capturing

--wait

Wait for USR1 signal; helps with synchronizing multiple generators.

--ns

Report times with nanosecond precision (default in microsecond). Stanard PCAP format only supports microsecond resolution -- nanosecond resolution is provided by multiplying all packet times by 1000. Thus, 1000s in the PCAP file corresponds to 1s realtime.

Sample output

The example below is the output produced when sending data out port 2 (with port 2 linked directly to port 3). In this example the rate is limited to 1.000 Mbps:

NetFPGA environment:
  Root dir:       /home/grg/netfpga
  Project name:   packet_generator
  Project dir:    /home/grg/netfpga/projects/packet_generator
  Work dir:       /tmp/grg

Limiting MAC Queue 2 to 1.000 Mbps (tokens = 1, clks = 1000)
Loaded 1 packet(s) into MAC Queue 2
Limiting CPU Queue 0 to 200.000 Mbps (tokens = 1, clks = 5)
Limiting CPU Queue 1 to 200.000 Mbps (tokens = 1, clks = 5)
Limiting CPU Queue 2 to 200.000 Mbps (tokens = 1, clks = 5)
Limiting CPU Queue 3 to 200.000 Mbps (tokens = 1, clks = 5)
Sending packets...
Last packet scheduled for transmission at 0.512 seconds
0 seconds elapsed...

Transmit statistics:
====================

MAC Queue 2:
   Packets: 1000
   Completed iterations: 1000


Receive statistics:
===================

MAC Queue 0:
   Packets: 0
MAC Queue 1:
   Packets: 0
MAC Queue 2:
   Packets: 0
MAC Queue 3:
   Packets: 1000
   Bytes: 64000
   Time: 0.511488512 s
   Rate: 1.001 Mbps (packet data only)
   Rate: 1.314 Mbps (including preamble/inter-packet gap)

The output shows that the last packet is estimated to be transmitted at time 0.512 seconds. The transmit statistics for queue 2 indicate that 1000 packets were sent and 1000 iterations were completed. The receive statistics for MAC queue 3 (nf2c3) indicate that 1000 packets were received which totaled 64000 bytes. (In this example each packet was 64 bytes.) The time between the reception of the first byte to the reception of the last byte was approximately 0.511 s. This corresponds to a rate of 1.001 Mbps (or 1.314 Mbps if you include the preamble/inter-packet gap in the rate calculation).

Examples

Simple packet replay

Send the packets contained in the file http.pcap out nf2c0:

 packet_generator.pl -q0 http.pcap

Packet replay and capture

Send 10 iterations of the packets contained in http.pcap out nf2c0. Capture the data coming back in port nf2c1 into a file named nf2c1.pcap:

 packet_generator.pl -q0 http.pcap -i0 10 -c1 nf2c1.pcap

Packet replay with rate limiting

Send the packets contained in http.pcap out nf2c0. Limit the rate to 1 Mbps (1000 Kbps):

 packet_generator.pl -q0 http.pcap -r0 1000

Packet replay with zero delay between packets

Send the packets contained in http.pcap out nf2c0. Set the delay between packets to 0 ns:

 packet_generator.pl -q0 http.pcap -d0 0

Simultaneous replay and capture on all four ports

Send the packets contained in http0.pcap, http1.pcap, udp2.pcap, and udp3.pcap out interfaces nf2c0, nf2c1, nf2c2 and nf2c3 respectively. Capture the data coming back on all four ports to files nf1c0.pcap, nf2c1.pcap, nf2c2.pcap and nf2c3.pcap:

 packet_generator.pl -q0 http0.pcap -q1 http1.pcap -q2 udp2.pcap -q3 udp3.pcap -c0 nf2c0.pcap -c1 nf2c1.pcap -c2 nf2c2.pcap -c3 nf2c3.pcap

Packet replay with padding

Send the packets contained in http.pcap out nf2c0. Only load the headers of the packets into SRAM in the NetFPGA.

 packet_generator.pl -q0 http.pcap --pad

Measuring delay through a switch/router

Delay through a switch or router can be measured using the packet generator. Delay is measured using all four ports simultaneously: two ports are connected directly to one another and used to provide a "baseline" measurement, the other two are connected to the device under test and are used to measure the difference from the baseline.

For example, ports nf2c0 and nf2c1 are directly connected and ports nf2c2 and nf2c3 are connected to the device under test. The same packet stream is sent out nf2c0 and nf2c2 and the packets are captured on nf2c1 and nf2c3; the difference in receive times between nf2c1 and nf2c3 is the delay through the device under test.

 packet_generator.pl -q0 test_pkts.pcap -c1 reference.pcap -q2 test_pkts.pcap -c3 device_test.pcap --ns

Note: The program used to display the pcap file must display absolute times to allow the comparison between the two receive ports. The difference in absolute time is the delay through the device under test. Some programs show times relative to the first packet by default. The following table lists some popular programs that read pcap files and how to display absolute times.

Program	Default time format	How to display absolute times
tcpdump	absolute	N/A
wireshark	relative to first packet	Select View -> Time Display Format -> Time of Day
tshark	relative to first packet	Specify "-t a" on the command line

Capture/replay details

Timestamping

Timestamping of received packets is performed in hardware. This allows packets to be timestamped with the accuracy of the clock on the NetFPGA board. (The accuracy of timestamps produced by software tools such as tcpdump is significantly less since each packet must be transferred from hardware to software. The process of notifying the kernel of an available packet and then transferring that packet into the kernel introduces a (small) variable delay thus limiting the accuracy of timestamps reported by tcpdump .)

Timestamps in the output pcap file can be recorded with millisecond (default) or nanosecond precision. Timestamps recorded with nanosecond precision are multiplied by 1000, thus a time of 1000s in the output file represents 1s in hardware. This is done because the standard pcap file format only supports millisecond resolution. (A variant of the pcap file format does support nanosecond resoultion however this is not compatible with all programs.)

Note that timestamps recorded in the captured pcap file are based upon the hardware timestamp.

Measurement

All measurements are performed in hardware. The packet counter is 32 bits and the byte counter is 40 bits. The time reported during a measurement is reported with nanosecond precision—this is only as accurate as the clock on the NetFPGA. The time reported is the time from the beginning of the first bit of data to the end of the last bit of data received—the preamble before the first packet is not included.

An example of a measurement on MAC queue 3 (nf2c3) is:

 MAC Queue 3:
         Packets: 1000
         Bytes: 1518000
         Time: 12.131868144 s
         Rate: 1.001 Mbps

Capture

The hardware attempts to send all packets received (on the relevant ports) to software for processing when performing packet capture. Since the PCI bandwidth is less than the maximum combined receive bandwidth across all four Ethernet ports it is possible that received packets will be dropped. Dropping of packets occurs when the buffers on the NetFPGA board overflow.

The measurement statistics are correct regardless of whether packet are dropped.

pcap files

Many programs have the ability to generate pcap files. Two such programs are tcpdump and Wireshark.

Regression Tests

See Packet generator regression tests

Additional Information

A more in depth description of the architecture can be found in the following paper.

FCCM 2009 Packet Generator Paper