To overcome this fairness problem, each topology now has its own queue into which incoming packets are placed. These topology-specific queues are then served in a round-robin fashion to ensure that each topology receives prompt attention, even if one topology is experiencing heavier traffic. Currently, this scheme assumes each packet requires a similar amount of work on the simulator’s part. In the future, this scheme might be extended to monitor the time used by each job and then use deficit round robin to serve these queues. For now, however, the simpler round robin scheme seems to have greatly improved fairness. In particular, an ongoing download of a large file on one topology now has almost no perceptible impact the round trip times on another topology.

Implementing this scheme incurred some performance costs (mostly due to new synchronization constructs) — maximum throughput decreased by 25%. However, these performance costs have been offset by two new improvements. First, Psyco is now used to run the simulator. It only required a dozen or so new lines of code for a 6% performance boost. Second, we utilized a script for automatically commenting out logging statements and replacing them with pass statements. This script completely removes the impact of any logging statements which are only useful when debugging the system. Since we make liberal use of the logging module, this was a big win — about a 25% boost in performance. When combined with Psyco, the new system is actually about 5% faster (in terms of throughput) than it was before the topology-specific queues were introduced.

• Logging. The VNS Simulator verbosely logs various events to aid in development and debugging efforts. Of course this affects performance, even when logging is disabled. This makes me miss Haskell with its lazy evaluation of parameters (and maybe even cpp’s #define‘s a little bit). However, since VNS is written in Python neither of these avenues were available. This means that the cost of constructing the messages being logged can still negatively impact performance even if they are never actually logged. Thankfully, only one particular kind of logging was causing a significant performance hit — the logging of individual packets. This was expensive in part because decoding packets and stringifying their contents is a bit CPU-intensive (especially in Python). This was easy to improve by simply bypassing packet decoding. This alone provided a speedup of about 2x.
• Unnecessary array reallocation. The new TCP stack implementation we use for our virtual web server nodes (more here, code here) poorly maintained the outgoing data queue. In particular, whenever an ACK was received the ACK’ed data was actively deleted instead of passively ignored. This lead to the outgoing queue to being resized far too often — though it was convenient from an implementation standpoint (one line of code), it was an expensive operation behind the scenes which involved allocating a new queue and then copying the remaining portion of the old queue into the new one. Eliminating this inefficiency increased the complexity a little (~20 lines of code) but paid huge dividends — large file transfers improved by nearly 5x.

The next biggest user of CPU time is largely located within the twisted framework itself (to receive incoming packets), so for now this is where we’ll stop optimizing. After these optimizations, a single client is now able to transfer large files from an otherwise unloaded VNS server at over a megabit per second!

In the future, our number one performance goal is to provide fairness and isolation between topologies. Ideally, if one user downloads a large file from one topology, then the impact on the round trip time to another topology should not be significantly affected.

Another solution I considered was using twisted’s built-in HTTP server to handle HTTP requests to our virtual web servers. Unfortunately, there was no easy way to ask twisted’s framework to process raw packets once they had arrived in the VNS simulator — the twisted web server sanely builds on the operating system’s TCP stack.

I finally gave in and decided that VNS virtual web servers would use a lightweight, user-level TCP stack to process these raw packets. Though lwip exists for C/C++, no similar framework was natively available for the Python-based VNS simulator. Thus we ended up rolling our own (simplified) TCP stack and used it to piece together incoming HTTP requests and to manage all of the TCP details. Surprisingly, this solution turned out to be much simpler to integrate into the virtual web servers (~20 lines of code instead of ~200). It also works much more reliably and quicker than the proxy solution, and has the advantage of enabling VNS to function as a standalone application with no reliance on external web servers.

This is just the beginning of the interface. In the future, I’d like students to be able to see a graphical representation of their topology and tweak it in real-time (e.g., take down a link or node to better test an implementation of a dynamic routing protocol). Eventually, students should also be able to create their own topologies.

The Using VNS page has been updated with information about how to use this web interface. Enjoy!

In the near future, we also hope to improve the assignment pages by factoring out common material (many students participate in more than one of our assignments).