Writing Tests for Puppet

Contributions to Puppet need to be accompanied by appropriate unit tests, especially if those contributions add new functionality (such as new types or providers) or modify existing core functionality.

Puppet currently (as of May 2008) has a large, existing code base of tests written in test/unit, in the test/ subdirectory of Puppet's SCM repository. These were largely written while Luke was still learning how to write tests, and their quality varies widely.

There is also an ever-growing list of tests written in RSpec, in the spec/ subdirectory. These tests are all relatively modern, and most of them are well-written.

The directory structure under each directory is set up to reflect the directory structure of the rest of the project. Each test is individually executable.

In general, only new RSpec tests should be added. In fact, you'll often find it's easier to start from scratch with RSpec tests than it is to modify existing test/unit tests.

Testing Tools

During testing, it's recommended that you run autotest; you can get a configuration and instructions in the ext/autotest directory in the Puppet git repository.

You can also run all of either test type by cd'ing into the appropriate directory and running rake. The test/unit tests can be run per-directory, by running rake <directory>.

Mocking

Both test/unit and RSpec tests use Mocha for mocking tests. See its documentation for more information.

Writing Tests

Puppet tests should be written using RSpec and placed in the spec/ subdirectory.

The existing tests are a very good source of information about writing new tests. Each new test should open with the following stanza:

#!/usr/bin/env ruby

require File.dirname(__FILE__) + '/../spec_helper'

The first line is what allows tests to be executed individually. The require statement loads up the required RSpec libraries. Note you'll have to change the number of /.. entries depending on the path of your test file.

Next you'll want to require whatever section of Puppet you are testing, as well as any Ruby libraries you will need to perform your tests. For example, when I wrote spec/unit/util/loadedfile.rb I used:

require 'tempfile'
require 'puppet/util/loadedfile'

Examples and Example Groups

It's best to look at RSpec's documentation for how to do this, but generally, you will have one or more "Example Groups", written like this (again, using LoadedFile):

describe Puppet::Util::LoadedFile do
    ... tests go here ...
end

It will contain one or more examples, written like this:

it "should load files" do
    ... test code goes here ...
end

Default Providers

When writing a type specifically designed for a certain platform you will have to mock the type's defaultprovider class method since there will likely be no suitable providers on other platforms.

In order to do this you should look for the first provider rather than make a call to the defaultprovider method:

before :each do
    provider_class = mcx_type.provider(mcx_type.providers[0])
    mcx_type.stubs(:defaultprovider).returns provider_class
end

When used in your Example Groups, this will mock the defaultprovider class method and return the first provider available to that type. This will allow these spec tests to be run on platforms other than those suitable for the provider.

Writing Good Tests

This is the part that's really hard. In general, your tests should be as simple and as short as possible (this is where most of the existing test/unit tests go wrong), preferably testing only one feature at a time.

See existing RSpec tests for examples, and preferentially look at more recent tests, and especially those written by Rick Bradley.