Expedient Plugin Tutorial

Expedient has two types of plugins that interact with each other: Aggregate Plugins and ui-plugins. This tutorial will guide through the process of creation of both types of plugins. You’ll first need to install expedient. Please see Installing Expedient.

Aggregate Plugins

Aggregate plugins have three main tasks:

  1. Describe resources and their types to the Expedient database
  2. Offer an API for other plugins to consume
  3. Keep information about resources in an aggregate up-to-date

In this tutorial, we will go through the process of writing a plugin for a very simple type of aggregate, an SSH access aggregate consisting of a set of SSH servers. We will write an aggregate plugin that allows users on Expedient to request SSH access to SSH servers. The admin of the SSH aggregate will get an email with a message from the user, information about the user, and a link on Expedient for approving or denying the request.

If the admin approves the request, the plugin will create a login for the user on each machine the user asks for, and add a public key provided by the user. The plugin does that by storing a private key that can be used to login to the server and execute the required commands.

Preliminaries

First, make sure that you have Expedient installed and that its packages are in your python library path (PYTHONPATH environment variable). Then go through the Django tutorial here. Create a package directory called sshaggregate with the following files:

  • sshaggregate/__init__.py: An empty file
  • sshaggregate/models.py: Will contain descriptions of our resources
  • sshaggregate/views.py: Will contain the plugin’s views

Also the directory hierarchy sshaggregate/templates/sshaggregate that will hold the templates for the plugin.

Writing Models

There are several models that will need to be written. Mainly, the aggregate model, the resources’ models, and any additional info that needs to be stored.

Edit the sshaggregate/models.py so it looks like this:

import shlex, subprocess
from StringIO import StringIO
from django.db import models
from django.db.models.fields import IPAddressField
import paramiko
from paramiko.rsakey import RSAKey
from expedient.clearinghouse.aggregate.models import Aggregate
from expedient.clearinghouse.resources.models import Resource, Sliver
from expedient.common.utils.modelfields import LimitedIntegerField
from expedient.common.middleware import threadlocals
from expedient.clearinghouse.utils import post_message_to_current_user
from expedient.common.messaging.models import DatedMessage
from expedient.clearinghouse.slice.models import Slice

# SSHServer class
class SSHServer(Resource):
    # SSHServer fields
    ip_address = IPAddressField(
        "IP address",
        help_text="Specify the server's IP address.",
    )
    ssh_port = LimitedIntegerField(
        "SSH port number",
        min_value=1,
        max_value=2**16-1,
        default=22,
        help_text="Specify the SSH port number to use."
    )
    # end

    def is_alive(self):
        """Ping the server and check if it's alive.
        
        @return: True if ping succeeds, False otherwise.
        """
        ret = subprocess.call(
            shlex.split("ping -c 1 -W 2 %s" % self.ip_address),
            stdout=open('/dev/null', 'w'),
            stderr=subprocess.STDOUT,
        )
        
        if ret == 0:
            return True
        else:
            return False
        
    def exec_command(self, command, **connection_info):
        """Connect to the server using an SSH session and execute a command.
        
        @param command: The command to execute
        @type command: C{str}
        @param username: The username to use to connect to the server.
        @type username: C{str}
        @keyword connection_info: A dict of other info to pass to
            C{paramiko.SSHClient.exec_command}.
        @return: A (out, err) tuple that is the output read on the
            stdout and stderr channels.
        @rtype: C{tuple(str, str)}
        """

        client = paramiko.SSHClient()
        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        client.connect(
            str(self.ip_address),
            port=int(self.ssh_port),
            **connection_info
        )
        _, sout, serr = client.exec_command(command)
        o = sout.read()
        e = serr.read()
        client.close()
        return o, e
    
    def __unicode__(self):
        return u"SSH server at IP %s" % self.ip_address
        
class SSHServerSliver(Sliver): pass

class SSHSliceInfo(models.Model):
    slice = models.OneToOneField(Slice)
    public_key = models.TextField()
    
# SSHAggregate class
class SSHAggregate(Aggregate):
    # SSHAggregate information field
    information = "An aggregate of SSH servers that are controlled" \
        " by a single administrator, to which users can request" \
        " access. Once approved, users get SSH access to all" \
        " machines using a public key they provide."
    # SSHAggregate end information field
    
    # SSHAggregate meta
    class Meta:
        verbose_name = "SSH Aggregate"
    # SSHAggregate end meta
    
    # SSHAggregate required fields
    admin_username = models.CharField(max_length=255)
    private_key = models.TextField()
    # SSHAggregate end required fields
    
    # SSHAggregate optional fields
    add_user_command = models.TextField(
        default="sh -c 'sudo useradd -m %(username)s'",
        help_text="Specify the command to create a new user. " \
            "'%(username)s' will be replaced by the user's " \
            " username. The command should return non-zero on failure " \
            " and 0 on success.",
    )
    del_user_command = models.TextField(
        default="sh -c 'sudo userdel -r -f %(username)s'",
        help_text="Specify the command to delete an existing user. " \
            "'%(username)s' will be replaced by the user's " \
            " username. The command should return non-zero on failure " \
            " and 0 on success.",
    )
    add_pubkey_user_command = models.TextField(
        default="sudo -u %(username)s mkdir /home/%(username)s/.ssh; "
            "sudo -u %(username)s chmod 700 /home/%(username)s/.ssh; "
            "sh -c 'sudo -u %(username)s echo %(pubkey)s >> "
            "/home/%(username)s/.ssh/authorized_keys'",
        help_text="Specify the command to add a public key to a user's " \
            "account. '%(username)s' will be replaced by the user's " \
            " username and '%(pubkey)s' will be replaced by the public key." \
            " The command should return non-zero on failure " \
            " and 0 on success.",
    )
    # SSHAggregate end optional fields
    
    def _op_user(self, op, server, cmd_subs, quiet=False):
        """common code for adding/removing users."""
        
        pkey_f = StringIO(self.private_key)
        pkey = RSAKey.from_private_key(pkey_f)
        pkey_f.close()
        
        cmd = getattr(self, "%s_user_command" % op) % cmd_subs
        cmd = cmd + "; echo $?"
        out, err = server.exec_command(
            cmd,
            username=str(self.admin_username),
            pkey=pkey,
        )
        
        lines = out.strip().split("\n")
        ret = int(lines[-1])
        
        if ret != 0:
            error = "".join(lines[:-1])
            if not quiet:
                # msg example
                msg = "Failed to %s user on %s. Output was:\n%s" \
                    % (op, server, error),
                post_message_to_current_user(
                    msg,
                    msg_type=DatedMessage.TYPE_ERROR,
                )
                # end msg example
            raise Exception(msg)

    def add_user(self, server, username, pubkey, quiet=False):
        """Add a user to a server.
        
        Add a user with username C{username} with public key C{pubkey} to
        server C{server}.
        
        @param server: The server to add the user to.
        @type server: L{SSHServer}
        @param username: the new user's username
        @type username: C{str}
        @param pubkey: The public key to add to the user's account.
        @type pubkey: the public key's value a C{str} 
        @keyword quiet: If True, no messages will be sent on failure.
            Defaults to False.
        @type quiet: C{boolean}
        """
        self._op_user("add", server, {"username": username}, quiet)
        self._op_user(
            "add_pubkey",
            server,
            {"username": username, "pubkey": pubkey},
            quiet,
        )
    
    def del_user(self, server, username, quiet=False):
        """Remove user from a server.
        
        Remove user with username C{username} from server C{server}.
        
        @param server: The server to remove the user from.
        @type server: L{SSHServer}
        @param username: the user's username
        @type username: C{str}
        @keyword quiet: If True, no messages will be sent on failure.
            Defaults to False.
        @type quiet: C{boolean}
        """
        self._op_user("del", server, {"username": username}, quiet)
        
    def check_status(self):
        return self.available and reduce(
            lambda x, y: x and y.is_alive(),
            SSHServer.objects.filter(aggregate__id=self.id),
    )
    
    # start_slice func
    def start_slice(self, slice):
        # start_slice call super
        super(SSHAggregate, self).start_slice(slice)
        # start_slice end call super
        
        # start_slice get info
        slice_info = SSHSliceInfo.objects.get(slice=slice)
        # start_slice get user
        user = slice.owner
        # start_slice get slivers
        slivers = SSHServerSliver.objects.filter(
            slice=slice, resource__aggregate__id=self.id)
        # start_slice end info
        
        # start_slice loop
        succeeded = []
        for sliver in slivers:
            # Execute the command on the server and get status
            server = sliver.resource.as_leaf_class()
            # start_slice add user
            try:
                self.add_user(server, user.username, slice_info.public_key)
            except:
                for s in succeeded:
                    try:
                        self.del_user(s, user.username)
                    except:
                        pass
                raise
            
            succeeded.append(server)
            # start_slice end loop
            
    def stop_slice(self, slice):
        super(SSHAggregate, self).start_slice(slice)
        user = threadlocals.get_thread_locals()["user"]
        for sliver in SSHServerSliver.objects.filter(slice=slice):
            server = sliver.resource.as_leaf_class()
            try:
                self.del_user(server, user.username)
            except:
                pass

Let’s go through the code section by section. We use paramiko in order to communicate with our SSH servers over SSH. Paramiko is a python SSH library.

SSHServer

The first class in our module is the SSHServer class which extends Resource class. The Resource class defines a few common fields and operations for resources. All resources that can be reserved must inherit from the Resource class:

class SSHServer(Resource):

Most importantly, the resource is related to an Aggregate by a foreign key relationship. Take a look at the Resource class documentation before continuing.

In the SSHServer class, we just define some extra fields and functions. An SSHServer instance has an IP Address and an SSH port number:

    ip_address = IPAddressField(
        "IP address",
        help_text="Specify the server's IP address.",
    )
    ssh_port = LimitedIntegerField(
        "SSH port number",
        min_value=1,
        max_value=2**16-1,
        default=22,
        help_text="Specify the SSH port number to use."
    )

We also define two extra functions: is_alive() and exec_command().

    def is_alive(self):
        """Ping the server and check if it's alive.
        
        @return: True if ping succeeds, False otherwise.
        """
        ret = subprocess.call(
            shlex.split("ping -c 1 -W 2 %s" % self.ip_address),
            stdout=open('/dev/null', 'w'),
            stderr=subprocess.STDOUT,
        )
        
        if ret == 0:
            return True
        else:
            return False
        
    def exec_command(self, command, **connection_info):
        """Connect to the server using an SSH session and execute a command.
        
        @param command: The command to execute
        @type command: C{str}
        @param username: The username to use to connect to the server.
        @type username: C{str}
        @keyword connection_info: A dict of other info to pass to
            C{paramiko.SSHClient.exec_command}.
        @return: A (out, err) tuple that is the output read on the
            stdout and stderr channels.
        @rtype: C{tuple(str, str)}
        """

        client = paramiko.SSHClient()
        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        client.connect(
            str(self.ip_address),
            port=int(self.ssh_port),
            **connection_info
        )
        _, sout, serr = client.exec_command(command)
        o = sout.read()
        e = serr.read()
        client.close()
        return o, e
    

The is_alive() function pings the server and checks that it is up, while the exec_command() function executes a command on the server using paramiko.

SSHServerSliver

A slice (represented by the Slice class) in Expedient is a container of slivers of different types of resources and across different aggregates. A sliver (represented by the generic class Sliver) is the reservation of one resource instance, and it relates the resource to the slice. A sliver describes information about the reservation of that particular instance. For example, if reserving a virtual machine, a sliver might describe the CPU percentage reserved for the VM.

When an aggregate is about to create a slice across its resources, it looks at the slice and all the slivers that are for resources it controls. It then uses those slivers to create the slice. We will see more information on creating a slice later.

In our example, we don’t have any per sliver information, so our SSHServerSliver is empty:

class SSHServerSliver(Sliver): pass

We could have also not created the class at all, but it makes our code clearer. Take a look at the Sliver and Slice classes documentation before continuing.

SSHSliceInfo

Some types of resources might require some per-slice info. In our example, creating a slice requires a public key for the user, so the SSHSliceInfo class will store that required information:

class SSHSliceInfo(models.Model):
    slice = models.OneToOneField(Slice)
    public_key = models.TextField()
    
# SSHAggregate class

SSHAggregate

This is the most involved class. The SSHAggregate class extends the generic Aggregate class. The Aggregate class defines some functions and fields that are shared among all aggregate classes. Aggregate plugins must always define an Aggregate class child.

class SSHAggregate(Aggregate):

The SSHAggregate class overrides the information field that contains information about the aggregate and describes the aggregate type. This field is used in the information page that describes the aggregate type:

    information = "An aggregate of SSH servers that are controlled" \
        " by a single administrator, to which users can request" \
        " access. Once approved, users get SSH access to all" \
        " machines using a public key they provide."

We have modified the verbose name used in for aggregate to make it easier to understand what the class is:

    class Meta:
        verbose_name = "SSH Aggregate"

It also adds a private_key and an admin_username fields that are used to login to the servers for administering them. These must be the same for all servers in the aggregate:

    admin_username = models.CharField(max_length=255)
    private_key = models.TextField()

We also have three additional fields that specify the commands that should be used for creating a user (add_user_command()), deleting a user (del_user_command()), and adding a public key to a user (add_pubkey_command()). These commands will be executed in an SSH shell when creating or deleting users. Make sure that SSH servers you add allow you to execute these commands non-interactively (i.e. either login as root or give no-password sudo access to the commands for the user logging in to execute the commands) :

    add_user_command = models.TextField(
        default="sh -c 'sudo useradd -m %(username)s'",
        help_text="Specify the command to create a new user. " \
            "'%(username)s' will be replaced by the user's " \
            " username. The command should return non-zero on failure " \
            " and 0 on success.",
    )
    del_user_command = models.TextField(
        default="sh -c 'sudo userdel -r -f %(username)s'",
        help_text="Specify the command to delete an existing user. " \
            "'%(username)s' will be replaced by the user's " \
            " username. The command should return non-zero on failure " \
            " and 0 on success.",
    )
    add_pubkey_user_command = models.TextField(
        default="sudo -u %(username)s mkdir /home/%(username)s/.ssh; "
            "sudo -u %(username)s chmod 700 /home/%(username)s/.ssh; "
            "sh -c 'sudo -u %(username)s echo %(pubkey)s >> "
            "/home/%(username)s/.ssh/authorized_keys'",
        help_text="Specify the command to add a public key to a user's " \
            "account. '%(username)s' will be replaced by the user's " \
            " username and '%(pubkey)s' will be replaced by the public key." \
            " The command should return non-zero on failure " \
            " and 0 on success.",
    )

We have also defined some helper functions to add and delete users from particular server (add_user() and del_user()).

    def add_user(self, server, username, pubkey, quiet=False):
        """Add a user to a server.
        
        Add a user with username C{username} with public key C{pubkey} to
        server C{server}.
        
        @param server: The server to add the user to.
        @type server: L{SSHServer}
        @param username: the new user's username
        @type username: C{str}
        @param pubkey: The public key to add to the user's account.
        @type pubkey: the public key's value a C{str} 
        @keyword quiet: If True, no messages will be sent on failure.
            Defaults to False.
        @type quiet: C{boolean}
        """
        self._op_user("add", server, {"username": username}, quiet)
        self._op_user(
            "add_pubkey",
            server,
            {"username": username, "pubkey": pubkey},
            quiet,
        )
    
    def del_user(self, server, username, quiet=False):
        """Remove user from a server.
        
        Remove user with username C{username} from server C{server}.
        
        @param server: The server to remove the user from.
        @type server: L{SSHServer}
        @param username: the user's username
        @type username: C{str}
        @keyword quiet: If True, no messages will be sent on failure.
            Defaults to False.
        @type quiet: C{boolean}
        """
        self._op_user("del", server, {"username": username}, quiet)
        

These functions use the private method _op_user(). Note that in case of error, we post a message to the user:

                msg = "Failed to %s user on %s. Output was:\n%s" \
                    % (op, server, error),
                post_message_to_current_user(
                    msg,
                    msg_type=DatedMessage.TYPE_ERROR,
                )

This uses the messaging module and a utility function post_message_to_current_user to post a message to the user indicating an error has occurred. This message will be shown in the list of messages for the user.

The check_status() method overrides the Aggregate class’s check_status() method to also make sure that all the servers in the aggregate are up by calling their is_alive() method.

    def check_status(self):
        return self.available and reduce(
            lambda x, y: x and y.is_alive(),
            SSHServer.objects.filter(aggregate__id=self.id),
    )
    
    # start_slice func

At a minimum any child that inherits from Aggregate must override start_slice and stop_slice methods. Our SSHAggregate class does that too:

    def start_slice(self, slice):

The start_slice() method calls the parent class’s start_slice() method because the parent class has some permission checking that we would rather not copy or redo:

        super(SSHAggregate, self).start_slice(slice)

It then gets needed information about the slice:

        slice_info = SSHSliceInfo.objects.get(slice=slice)

And the owner of the slice whose username will be used:

        user = slice.owner

Then we get the slivers in the slice that are for resources in the aggregate:

        slivers = SSHServerSliver.objects.filter(
            slice=slice, resource__aggregate__id=self.id)

Note that we don’t just do resource__aggregate=self because that the resource is related to SSHAggregate‘s parent class. So we need to compare them using ids. We could have instead done resource__aggregate=self.aggregate_ptr.

Now we add the user to the server pointed to by each sliver, keeping track of our successes for rollback in case of error:

        succeeded = []
        for sliver in slivers:

The SSHServerSliver‘s parent class has a pointer to the generic resource. To obtain the leaf child that the sliver is pointing to, we need to use a special function. Otherwise, sliver.resource returns an object of type generic Resource:

            server = sliver.resource.as_leaf_class()

Then we add the user, paying attention to roll back the changes in case of errors:

            try:
                self.add_user(server, user.username, slice_info.public_key)
            except:
                for s in succeeded:
                    try:
                        self.del_user(s, user.username)
                    except:
                        pass
                raise
            
            succeeded.append(server)

stop_slice() is very similar to start_slice() but a bit simpler since we don’t rollback changes in case of errors.

    def stop_slice(self, slice):
        super(SSHAggregate, self).start_slice(slice)
        user = threadlocals.get_thread_locals()["user"]
        for sliver in SSHServerSliver.objects.filter(slice=slice):
            server = sliver.resource.as_leaf_class()
            try:
                self.del_user(server, user.username)
            except:
                pass

Relationships

Below we show a summary of the relationships between slices, resources, aggregates, and slivers.

../_images/slice-aggregate-resource-relationships.png

Each aggregate is connected to a number of resources. Each slice is also related to a number of resources through a sliver. In our example, an SSHAggregate consists of a number of SSHServer instances. A slice can have a number of SSHServerSliver instances that are each part of an SSHServer instance.

Documenting Code

You’ll notice that the code we wrote uses something very similar to javadoc for documenting our methods and classes. We use Epydoc for documenting code and automatically generating API docs from the code. Feel free to use whatever you like, but please document your code thoroughly.

Writing Views and Templates

The next step is writing some views and HTML templates for managing the SSH aggregate in Expedient. This includes pages for adding the aggregate to Expedient or editing it.

Views

The add/edit aggregate view is the page that the user gets redirected to when she wants to add an SSH aggregate to Expedient. There will be two steps for adding an aggregate. In the first, we store information about the aggregate as a whole. Below, we sketch out what it looks like. We recommend you always sketch out what your views will look like before writing them:

                             +-------------------------+
Admin Username:              |                         |
                             +-------------------------+
                             +-------------------------+
Private Key:                 |                         |
                             +-------------------------+
                             +-------------------------+
Add user command:            |                         |
                             +-------------------------+
                             +-------------------------+
Del user command:            |                         |
                             +-------------------------+
                             +-------------------------+
Add pubkey user command:     |                         |
                             +-------------------------+

                   +------+
                   | Next | | Cancel
                   +------+

The next view is where the user specifies the SSH servers to add to this aggregate. All these servers can be administered using the same information entered in the previous step:

Add all servers that can be administered using the same information
here. You can click save to get more rows once you fill the existing
ones.

+--------------------------------------------------------+
|                             +-------------------------+|
|Name:                        |                         ||
|                             +-------------------------+|
|                             +-------------------------+|
|IP Address:                  |                         ||
|                             +-------------------------+|
|                             +-------------------------+|
|SSH Port:                    |                         ||
|                             +-------------------------+|
|                             +-------------------------+|
|Delete:                      |                         ||
|                             +-------------------------+|
+--------------------------------------------------------+

+--------------------------------------------------------+
|                             +-------------------------+|
|Name:                        |                         ||
|                             +-------------------------+|
|                             +-------------------------+|
|IP Address:                  |                         ||
|                             +-------------------------+|
|                             +-------------------------+|
|SSH Port:                    |                         ||
|                             +-------------------------+|
|                             +-------------------------+|
|Delete:                      |                         ||
|                             +-------------------------+|
+--------------------------------------------------------+

                   +------+
                   | Save | | Home
                   +------+

These are our two views. We will use a very similar view for updating aggregates. In fact, we will use the same view with a minor change: The information about the aggregate would already be preloaded into the tables. Now, create the file sshaggregate/views.py and modify it to look like the following:

from django.core.urlresolvers import reverse
from django.forms.models import modelformset_factory
from django.shortcuts import get_object_or_404
from django.http import HttpResponseRedirect
from django.views.generic import simple
from expedient.common.utils.views import generic_crud
from expedient.common.messaging.models import DatedMessage
from expedient.clearinghouse.aggregate.models import Aggregate
from expedient.clearinghouse.utils import post_message_to_current_user
from sshaggregate.models import SSHAggregate, SSHServer

def aggregate_crud(request, agg_id=None):
    """Show a page for the user to add/edit an SSH aggregate in Expedient."""
    
    return generic_crud(
        request,
        obj_id=agg_id,
        model=SSHAggregate,
        template_object_name="aggregate",
        template="sshaggregate/aggregate_crud.html",
        redirect=lambda inst: reverse(
            aggregate_add_servers, args=[inst.id]),
        success_msg=lambda inst: \
            "Successfully created/updated SSH Aggregate %s" % inst.name,
    )
    
def aggregate_add_servers(request, agg_id):
    """Show a page that allows user to add SSH servers to the aggregate."""
    
    agg = get_object_or_404(SSHAggregate, id=agg_id)
    servers = SSHServer.objects.filter(aggregate__id=agg_id)
    ServerFormSet = modelformset_factory(
        SSHServer, can_delete=True, extra=3,
        fields=["name", "ip_address", "ssh_port"],
    )

    if request.method == "POST":
        formset = ServerFormSet(
            request.POST, queryset=servers)
        if formset.is_valid():
            instances = formset.save(commit=False)
            for instance in instances:
                instance.aggregate = agg
                instance.save()
            formset.save_m2m()
            post_message_to_current_user(
                "Successfully added/updated servers "\
                "to aggregate %s" % agg.name,
                msg_type=DatedMessage.TYPE_SUCCESS)
            
            return HttpResponseRedirect(request.path)
        
    else:
        formset = ServerFormSet(queryset=servers)

    return simple.direct_to_template(
        request, template="sshaggregate/aggregate_add_servers.html",
        extra_context={"aggregate": agg, "formset": formset})

The aggregate_crud() function uses the generic_crud generic view to add or update existing aggregates. generic_crud() simplifies many of the operations that are needed for creating or updating database objects. Take a look at generic_crud‘s documentation before continuing.

The aggregate_add_servers() view uses model formsets. See the Django documentation about modelformset_factory. It is a convenient way to create and modify a number of objects:

    ServerFormSet = modelformset_factory(

Finally when returning a response we should use one of Django’s generic views because they use RequestContext when generating the context for templates. This allows us to use context processors to add more variables into template contexts, one of which is the messages that you see at the top of each page:

    return simple.direct_to_template(
        request, template="sshaggregate/aggregate_add_servers.html",
        extra_context={"aggregate": agg, "formset": formset})

Templates

We need three templates:

  • sshaggregate/templates/sshaggregate/sshaggregate_base.html
  • sshaggregate/templates/sshaggregate/aggregate_crud.html
  • sshaggregate/templates/sshaggregate/aggregate_add_servers.html

There is quite a bit that happens behind the scenes in templates, and there are is a base template that you should use for creating your own templates. It is a good practice to create a base template for each plugin or django app from which all other templates in the plugin or app inherit. Here the base template is called sshaggregate_base.html.

Edit sshaggregate_base.html to look like the following:

{% extends "base.html" %}

The base right now includes only one line specifying that it extends the base.html template that is used as the base template for all the templates in Expedient. The base template has a few style headers and some javascript code that does magic like changing help text in tables that have the formtable class into a question mark. You will see it in action here.

Next edit the aggregate_crud.html file to look like the following:

{% extends "sshaggregate/sshaggregate_base.html" %}

{% block title %}Add/Update SSH Aggregate.{% endblock title %}

{% block head %}
{% endblock %}

{% block content %}
<div class="main">
	{% if aggregate %}
	<h1>Update aggregate {{ aggregate.name }}</h1>
	{% else %}
	<h1>Add aggregate</h1>
	{% endif %}
	<form method="post" action="">{% csrf_token %}
		<table class="formtable_noborder">
		{{ form.as_table }}
	    </table>
		<div class="center">
			<input type="submit" value="Next" /> | 
			{% if aggregate %}
			<a href="{% url aggregate_delete aggregate.pk %}">Delete</a> |
			{% endif %} 
			<a href="{% url home %}">Cancel</a>
		</div>
	</form>
</div>
{% endblock content %}

Let’s go through the file. We want our template to extend our plugin’s base template:

{% extends "sshaggregate/sshaggregate_base.html" %}

Expedient’s base template base.html has a number of blocks that can be extended.

  • title: Gets used as the page’s title.
  • head: Any code in this block is added to the <head> element in the html document. You can use this space to add javascript or css style blocks.
  • content: This is where you should put your template’s body.

We set the title:

{% block title %}Add/Update SSH Aggregate.{% endblock title %}

There’s nothing in the head:

{% block head %}
{% endblock %}

Then our content start:

{% block content %}
<div class="main">

We specified that the template object name is aggregate when we used the generic_crud() function. So if it is specified in the template, it means that the template is being used to update an existing instance:

	{% if aggregate %}
	<h1>Update aggregate {{ aggregate.name }}</h1>

Otherwise, we are creating a new instance:

	{% else %}
	<h1>Add aggregate</h1>
	{% endif %}

We then add the form to get the info about the aggregate:

	<form method="post" action="">{% csrf_token %}
		<table class="formtable_noborder">
		{{ form.as_table }}
	    </table>
		<div class="center">
			<input type="submit" value="Next" /> | 
			{% if aggregate %}
			<a href="{% url aggregate_delete aggregate.pk %}">Delete</a> |
			{% endif %} 
			<a href="{% url home %}">Cancel</a>
		</div>
	</form>
</div>
{% endblock content %}

Note that we use the {% csrf_token %} tag because Django’s CSRF protection is enabled. Also note that we give the table in the form the formtable_noborder class which defines some css settings and lets javascript do its magic in moving help text into balloon tips.

Next, create the sshaggregate/templates/sshaggregate/aggregate_add_servers.html and edit it to look like the following:

{% extends "sshaggregate/sshaggregate_base.html" %}

{% block title %}Add/Update SSH Servers in {{ aggregate.name }}.{% endblock title %}

{% block head %}
{% endblock %}

{% block content %}
<div class="main">
	<h1>Add/Update SSH Servers in {{ aggregate.name }}</h1>
	
	<div> Add all servers that can be administered using the same information
    here. You can click save to get more rows once you fill the existing
    ones.</div>
	
	<form method="post" action="">{% csrf_token %}
    	{{ formset.management_form }}
		{% for form in formset.forms %}
			{% for hidden in form.hidden_fields %}
				{{ hidden }}
			{% endfor %}
		<table class="formtable">
			{{ form.as_table }}
	    </table>
	    {% endfor %}
		<div class="center">
			<input type="submit" value="Save" /> | 
			<a href="{% url home %}">Home</a>
		</div>
	</form>
</div>
{% endblock content %}

This is similar to aggregate_crud.html, so we won’t go through it in detail.

Connecting to Expedient

To add the plugin to Expedient, there are a few things we need to do:

  1. Write the URLs
  2. Modify Expedient settings to include the plugin

Write the URLs

Create the file sshaggregate/urls.py and edit it to look like the following:

from django.conf.urls.defaults import *

urlpatterns = patterns('sshaggregate.views',
    url(r'^aggregate/create/$', 'aggregate_crud', name='sshaggregate_aggregate_create'),
    url(r'^aggregate/(?P<agg_id>\d+)/edit/$', 'aggregate_crud', name='sshaggregate_aggregate_edit'),
    url(r'^aggregate/(?P<agg_id>\d+)/servers/$', 'aggregate_add_servers', name='sshaggregate_aggregate_servers'),
)

We need three urls:

  • Creating the aggregate
  • Updating the aggregate
  • Adding/removing servers from the aggregate

The Aggregate class by default expects the create and update urls to have a particular format: <app_name>_aggregate_create and <app_name>_aggregate_update. If the plugin specified an additional URL <app_name>_aggregate_delete, the user would be redirected to that URL when requesting to delete the aggregate instead of the default deletion confirmation page.

Note that for both the create and update, we use the same aggregate_crud() function. In the create url, it will be called with no arguments, while in the edit url, it will be called with the agg_id argument.

Update Settings

First, we need to add the plugin to the list of installed apps. Edit your localsettings.py file to do so. Eventually, if the plugin makes it into the Expedient distribution, the settings that you modify here will be added to the default settings. We will need to edit two settings.

Default Settings

All the settings you put in the localsettings.py appear in Django’s settings module. So if your application has some default settings that you would like to add, you will need to add a line similar to:

from sshaggregate.defaultsettings import *

at the top of the localsettings.py file.

You can append additional list items to almost all of the default settings that are lists. Below we show an example of how this is done for INSTALLED_APPS and AGGREGATE_PLUGINS. If you add an EXTRA_ at the beginning of the name, the list you specify next is appended to the setting’s list value.

INSTALLED_APPS

Add the following line to your localsettings.py:

EXTRA_INSTALLED_APPS = [
    'sshaggregate',
]
AGGREGATE_PLUGINS

The AGGREGATE_PLUGINS setting describes the aggregate plugins that are installed. See the AGGREGATE_PLUGINS setting documentation for more information:

EXTRA_AGGREGATE_PLUGINS = [
    ('sshaggregate.models.SSHAggregate', 'sshaggregate', 'sshaggregate.urls'),
]

Testing

Now, you will need to test your plugin to make everything works. The first phase of testing is going to be manual prelimiary testing to make sure that the plugin works well. Before you do so, make sure that you have performed a manage.py syncdb on the running Expedient instance that you will use. Here we will cover automated testing.

The importance of automated testing cannot be stressed enough. Anytime you make a change, you will need to run the tests, and nothing makes your job of fixing bugs and adding new features as easy as automated tests. Django has extensive facilities for testing that come in quite handy. Create the sshaggregate/tests.py and edit it to look like the following:

import getpass, os
import re
from StringIO import StringIO
from paramiko.rsakey import RSAKey
from django.test import TestCase
from django.contrib.auth.models import User
from django.core.urlresolvers import reverse
from expedient.clearinghouse.slice.models import Slice
from expedient.common.permissions.models import ExpedientPermission
from expedient.clearinghouse.project.models import Project
from expedient.common.tests.client import test_get_and_post_form
from sshaggregate.views import aggregate_add_servers, aggregate_crud
from sshaggregate.models import SSHAggregate, SSHServer, SSHServerSliver,\
    SSHSliceInfo

# Tests class
class Tests(TestCase):
    # start constants
    CREATED_USER_FNAME = "/tmp/created_user"
    '''Where do we store the created user's username?'''
    DELETED_USER_FNAME = "/tmp/deleted_user"
    '''Where do we store the deleted user's username?'''
    PUBKEY_USER_FNAME = "/tmp/pubkey_user"
    '''Where do we store the created user's public key?'''
    
    TEST_SSH_KEY_NAME = "id_rsa_sshaggregate"
    TEST_KEY_COMMENT = "ssh_aggregate_test_key"
    # end
    def create_test_ssh_key(self):
        self.test_key = RSAKey.generate(1024)
        self.test_pubkey = "\nssh-rsa %s %s\n" % \
            (self.test_key.get_base64(), Tests.TEST_KEY_COMMENT)
        f = open(os.path.expanduser("~/.ssh/authorized_keys"), "a")
        f.write(self.test_pubkey)
        f.close()
        
    def delete_test_ssh_key(self):
        f = open(os.path.expanduser("~/.ssh/authorized_keys"), "r")
        auth_keys = f.read()
        f.close()
        new_auth_keys = re.sub(
            r"\n.*%s\n" % Tests.TEST_KEY_COMMENT,"", auth_keys)
        f = open(os.path.expanduser("~/.ssh/authorized_keys"), "w")
        f.write(new_auth_keys)
        f.close()

    def setUp(self):
        # create local user
        self.su = User.objects.create_superuser(
            "su", "su@stanford.edu", "password")
        # end
        
        # disable permissions
        ExpedientPermission.objects.disable_checks()
        # end
        
        # delete existing temp files
        for f in Tests.CREATED_USER_FNAME, Tests.DELETED_USER_FNAME,\
        Tests.PUBKEY_USER_FNAME:
            if os.access(f, os.F_OK):
                os.unlink(f)
        # end
        
        # create ssh key
        self.create_test_ssh_key()
        # end
        
        # login user
        self.client.login(username="su", password="password")
        # end
        
    def tearDown(self):
        for f in Tests.CREATED_USER_FNAME, Tests.DELETED_USER_FNAME,\
        Tests.PUBKEY_USER_FNAME:
            if os.access(f, os.F_OK):
                os.unlink(f)

        self.delete_test_ssh_key()

        ExpedientPermission.objects.enable_checks()
    
    # add aggregate tests
    def test_add_aggregate(self):
        # end
        
        # check nothing is there
        self.assertEqual(SSHAggregate.objects.count(), 0)
        # end
        
        # get private key as string
        pkey_f = StringIO()
        self.test_key.write_private_key(pkey_f)
        pkey = pkey_f.getvalue()
        pkey_f.close()
        # end
        
        # Add the aggregate
        response = test_get_and_post_form(
            self.client,
            url=reverse(aggregate_crud),
            params=dict(
                name="Test Aggregate",
                description="Aggregate on localhost",
                location="right here",
                admin_username=getpass.getuser(),
                private_key=pkey,
                add_user_command="sh -c 'echo %(username)s >> " +
                    Tests.CREATED_USER_FNAME + "'",
                del_user_command="sh -c 'echo %(username)s >> " +
                    Tests.DELETED_USER_FNAME + "'",
                add_pubkey_user_command="sh -c 'echo %(pubkey)s >> " +
                    Tests.PUBKEY_USER_FNAME + "'",
            ),
        )
        self.assertEqual(SSHAggregate.objects.count(), 1)
        # end
        
        # where do we go next?
        next_url = reverse(aggregate_add_servers, args=[1])
        self.assertRedirects(
            response, next_url)
        # end
        
        # Add the localhost as a server
        response = test_get_and_post_form(
            self.client,
            url=next_url,
            params={
                "form-0-name": "localhost",
                "form-0-ip_address": "127.0.0.1",
                "form-0-ssh_port": "22",
            },
        )
        # end
        
        # check that localhost added
        self.assertRedirects(
            response, reverse(aggregate_add_servers, args=[1]),
            msg_prefix="Response was %s" % response)
        self.assertEqual(SSHServer.objects.count(), 1)
        # end
        
    # slice tests
    def test_create_delete_slice(self):
        # end
        
        # Add the aggregate
        self.test_add_aggregate()
        agg = SSHAggregate.objects.all()[0]
        # end
        
        # Create the project
        self.client.post(
            reverse("project_create"),
            data=dict(name="Test Project", description="Blah blah"),
        )
        project = Project.objects.get(name="Test Project")
        # end

        # Add the aggregate to the project
        url = reverse("project_add_agg", args=[project.id])
        response = self.client.post(
            path=url,
            data={"id": agg.id},
        )
        self.assertTrue(project.aggregates.count() == 1)
        self.assertRedirects(response, url)
        # end
        
        # Create the slice
        self.client.post(
            reverse("slice_create", args=[project.id]),
            data=dict(
                name="Test Slice",
                description="Blah blah",
            )
        )
        slice = Slice.objects.get(name="Test Slice")
        # end
        
        # Add the aggregate to the slice
        slice_add_agg_url = reverse("slice_add_agg", args=[slice.id])
        response = self.client.post(
            path=slice_add_agg_url,
            data={"id": agg.id},
        )
        self.assertRedirects(response, slice_add_agg_url)
        self.assertEqual(
            slice.aggregates.count(), 1,
            "Did not add aggregate to slice.")
        # end
        
        # Create a sliver
        server = SSHServer.objects.all()[0]
        SSHServerSliver.objects.create(resource=server, slice=slice)
        # end
        
        # Create the SSHSliceInfo
        SSHSliceInfo.objects.create(
            slice=slice, public_key=self.test_pubkey)
        # end
        
        # Start the slice
        self.client.post(reverse("slice_start", args=[slice.id]))
        # end
        
        # Check that the add command was executed correctly
        self.assertTrue(os.access(Tests.CREATED_USER_FNAME, os.F_OK))
        f = open(Tests.CREATED_USER_FNAME, "r")
        username = f.read()
        f.close()
        self.assertEqual(
            username.strip(), self.su.username,
            "Add user command not executed correctly. Expected to find '%s' "
            "in %s, but found '%s' instead."
            % (self.su.username, Tests.CREATED_USER_FNAME, username))
        # end
        
        # check that the add pub key command executed correctly
        f = open(Tests.PUBKEY_USER_FNAME, "r")
        pubkey = f.read()
        f.close()
        self.assertEqual(
            username.strip(), self.su.username,
            "Add pub key command not executed correctly. Expected to find "
            "'%s' in %s, but found '%s' instead."
            % (self.test_pubkey, Tests.PUBKEY_USER_FNAME, pubkey))
        # end
        
        # Stop the slice
        self.client.post(reverse("slice_stop", args=[slice.id]))
        # end
        
        # Check that the del user command executed correctly
        f = open(Tests.DELETED_USER_FNAME, "r")
        username = f.read()
        f.close()
        self.assertEqual(
            username.strip(), self.su.username,
            "Del user command not executed correctly. Expected to find '%s' "
            "in %s, but found '%s' instead."
            % (self.su.username, Tests.DELETED_USER_FNAME, username))
        # end

The tests that we have written test two things:

  • Adding an SSH aggregate to Expedient
  • Creating and deleting a slice

This is not a comprehensive test suite, and your tests should include more exhaustive testing. The test class does the following. It first sets up the environment by creating an SSH key for the user running the test, adding it to the user’s authorized keys, and using the user as the admin for the aggregate. The tests can now SSH into localhost using the created key.

When adding an aggregate the test suite uses custom commands for creating users, deleting users, and adding a public key to a user’s account. Instead of these commands actually doing what they are supposed to do, we do the following. When creating a user, we write the user’s username into a temprary file and we write the user’s public key into another file. Then we check that the commands executed correctly and that these files were written as expected. Similarly, when deleting a user, we write the user’s username into a temporary file and we check that the file was written as expected. So let’s go through the test code section by section.

First, we create a class that inherits from the django.test.TestCase:

class Tests(TestCase):

Expedient also provides a different class you can inherit from, expedient.common.tests.manager.SettingsTestCase. This test case allows you to easily change settings in the test case to, for example, add new django apps that are used in testing. Take a look at expedient.clearinghouse.aggregate.tests.tests for an example on how it is used.

Next, we define some constants that are used in testing. These are the names of the temporary files used and information about the temporary SSH key created for the tests:

    CREATED_USER_FNAME = "/tmp/created_user"
    '''Where do we store the created user's username?'''
    DELETED_USER_FNAME = "/tmp/deleted_user"
    '''Where do we store the deleted user's username?'''
    PUBKEY_USER_FNAME = "/tmp/pubkey_user"
    '''Where do we store the created user's public key?'''
    
    TEST_SSH_KEY_NAME = "id_rsa_sshaggregate"
    TEST_KEY_COMMENT = "ssh_aggregate_test_key"

Next, we define a method to create an SSH key and add it to the current user’s authorized_keys file. This function uses paramiko to generate a key.

    def create_test_ssh_key(self):
        self.test_key = RSAKey.generate(1024)
        self.test_pubkey = "\nssh-rsa %s %s\n" % \
            (self.test_key.get_base64(), Tests.TEST_KEY_COMMENT)
        f = open(os.path.expanduser("~/.ssh/authorized_keys"), "a")
        f.write(self.test_pubkey)
        f.close()
        

Note that we’ve given the key a special comment that we can use to delete the key in the delete_test_ssh_key() method:

    def delete_test_ssh_key(self):
        f = open(os.path.expanduser("~/.ssh/authorized_keys"), "r")
        auth_keys = f.read()
        f.close()
        new_auth_keys = re.sub(
            r"\n.*%s\n" % Tests.TEST_KEY_COMMENT,"", auth_keys)
        f = open(os.path.expanduser("~/.ssh/authorized_keys"), "w")
        f.write(new_auth_keys)
        f.close()

The setUp() function first creates a local user:

        self.su = User.objects.create_superuser(
            "su", "su@stanford.edu", "password")

We chose to create a superuser because it bypasses some of the security restrictions built into Expedient. You might want to do the testing with a regular user instead. Then we disable permission checking in Expedient:

        ExpedientPermission.objects.disable_checks()

This disables most of the permission checks that Expedient performs. In more exhaustive testing, you will want to keep permissions enabled. Then we create an ssh key to ssh into localhost:

        self.create_test_ssh_key()

And finally we log in as the user we just created so we can use Django’s test client to make html requests:

        self.client.login(username="su", password="password")

The tearDown() function does many of setUp()‘s operations in reverse:

    def tearDown(self):
        for f in Tests.CREATED_USER_FNAME, Tests.DELETED_USER_FNAME,\
        Tests.PUBKEY_USER_FNAME:
            if os.access(f, os.F_OK):
                os.unlink(f)

        self.delete_test_ssh_key()

        ExpedientPermission.objects.enable_checks()
    
    # add aggregate tests

Adding Aggregates

The first test checks that we can add an aggregate:

    def test_add_aggregate(self):

Make sure that nothing is in Expedient already:

        self.assertEqual(SSHAggregate.objects.count(), 0)

Get the private key that we created as a string to use in adding the aggregate:

        pkey_f = StringIO()
        self.test_key.write_private_key(pkey_f)
        pkey = pkey_f.getvalue()
        pkey_f.close()

Then we add the aggregate using an html post request using django’s client. Note that here we use a custom function test_get_and_post_form that gets a form, and fills in some of the parameters. We use this function to save some of the trouble of having to add fields with existing defaults or for fields that are arriving from the server with custom values. After we do the post request, we check that the aggregate has indeed been added.

        response = test_get_and_post_form(
            self.client,
            url=reverse(aggregate_crud),
            params=dict(
                name="Test Aggregate",
                description="Aggregate on localhost",
                location="right here",
                admin_username=getpass.getuser(),
                private_key=pkey,
                add_user_command="sh -c 'echo %(username)s >> " +
                    Tests.CREATED_USER_FNAME + "'",
                del_user_command="sh -c 'echo %(username)s >> " +
                    Tests.DELETED_USER_FNAME + "'",
                add_pubkey_user_command="sh -c 'echo %(pubkey)s >> " +
                    Tests.PUBKEY_USER_FNAME + "'",
            ),
        )
        self.assertEqual(SSHAggregate.objects.count(), 1)

Note the custom commands we have used to add a user, delete a user, and add a public key to a user. We also need to check that after adding the aggregate we were redirected to the page where we add servers:

        next_url = reverse(aggregate_add_servers, args=[1])
        self.assertRedirects(
            response, next_url)

Now we add the localhost as a server in the aggregate:

        response = test_get_and_post_form(
            self.client,
            url=next_url,
            params={
                "form-0-name": "localhost",
                "form-0-ip_address": "127.0.0.1",
                "form-0-ssh_port": "22",
            },
        )

And finally we check that the server was added correctly:

        self.assertRedirects(
            response, reverse(aggregate_add_servers, args=[1]),
            msg_prefix="Response was %s" % response)
        self.assertEqual(SSHServer.objects.count(), 1)

Creating and Deleting Slices

Next we check that functionality of our aggregate.

    def test_create_delete_slice(self):

We call the previous function to add the aggregate to Expedient first.

        response = test_get_and_post_form(
            self.client,
            url=reverse(aggregate_crud),
            params=dict(
                name="Test Aggregate",
                description="Aggregate on localhost",
                location="right here",
                admin_username=getpass.getuser(),
                private_key=pkey,
                add_user_command="sh -c 'echo %(username)s >> " +
                    Tests.CREATED_USER_FNAME + "'",
                del_user_command="sh -c 'echo %(username)s >> " +
                    Tests.DELETED_USER_FNAME + "'",
                add_pubkey_user_command="sh -c 'echo %(pubkey)s >> " +
                    Tests.PUBKEY_USER_FNAME + "'",
            ),
        )
        self.assertEqual(SSHAggregate.objects.count(), 1)

Then we create a project.

        self.client.post(
            reverse("project_create"),
            data=dict(name="Test Project", description="Blah blah"),
        )
        project = Project.objects.get(name="Test Project")

We add the aggregate to the project, and we check that the project now has the aggregate:

        url = reverse("project_add_agg", args=[project.id])
        response = self.client.post(
            path=url,
            data={"id": agg.id},
        )
        self.assertTrue(project.aggregates.count() == 1)
        self.assertRedirects(response, url)

Then we create the slice:

        self.client.post(
            reverse("slice_create", args=[project.id]),
            data=dict(
                name="Test Slice",
                description="Blah blah",
            )
        )
        slice = Slice.objects.get(name="Test Slice")

We add the aggregate to the slice, and check that the addition worked.

        slice_add_agg_url = reverse("slice_add_agg", args=[slice.id])
        response = self.client.post(
            path=slice_add_agg_url,
            data={"id": agg.id},
        )
        self.assertRedirects(response, slice_add_agg_url)
        self.assertEqual(
            slice.aggregates.count(), 1,
            "Did not add aggregate to slice.")

Now, if you recall, slices are composed of slivers. To tell Expedient that we want an SSHServer from a particular aggregate in our slice, we need to a create a sliver for that resource. We do this by creating an SSHServerSliver instance:

        server = SSHServer.objects.all()[0]
        SSHServerSliver.objects.create(resource=server, slice=slice)

Next we add more information about the slice using the SSHSliceInfo class:

        SSHSliceInfo.objects.create(
            slice=slice, public_key=self.test_pubkey)

Now we can start the slice:

        self.client.post(reverse("slice_start", args=[slice.id]))

Now we need to check that the command to add users was executed correctly on the localhost. We do this by checking that the username was written correctly to the temporary file.

        self.assertTrue(os.access(Tests.CREATED_USER_FNAME, os.F_OK))
        f = open(Tests.CREATED_USER_FNAME, "r")
        username = f.read()
        f.close()
        self.assertEqual(
            username.strip(), self.su.username,
            "Add user command not executed correctly. Expected to find '%s' "
            "in %s, but found '%s' instead."
            % (self.su.username, Tests.CREATED_USER_FNAME, username))

We do a similar check to see that the public key for the user was added:

        f = open(Tests.PUBKEY_USER_FNAME, "r")
        pubkey = f.read()
        f.close()
        self.assertEqual(
            username.strip(), self.su.username,
            "Add pub key command not executed correctly. Expected to find "
            "'%s' in %s, but found '%s' instead."
            % (self.test_pubkey, Tests.PUBKEY_USER_FNAME, pubkey))

Next, we stop the slice:

        self.client.post(reverse("slice_stop", args=[slice.id]))

And we make sure that the command executed correctly:

        f = open(Tests.DELETED_USER_FNAME, "r")
        username = f.read()
        f.close()
        self.assertEqual(
            username.strip(), self.su.username,
            "Del user command not executed correctly. Expected to find '%s' "
            "in %s, but found '%s' instead."
            % (self.su.username, Tests.DELETED_USER_FNAME, username))

And that’s it. You now have a basic test suite that you can run. To run the test, you need to make sure that sshaggregate is in your PYTHONPATH. Then you can use the test management command to run the sshaggregate test suite.