This post about unit testing is part of the Setting up a full stack for web testing series.

Introduction

In functional testing, also referred to system testing or blackbox testing, you don’t just test the individual units but the whole application. So you’re testing if all of the small units you built and tested actually work together.

For functional testing a web site I talk to the application in HTTP. As mentioned already in Unit testing, an example for an address book would be to create an address using the provided web form, then check that the detail page renders and that it shows up in a list or search result.

I explicitly exclude JavaScript tests from this category, as I’ll cover those later in a separate post. The reason I don’t include them in functional tests is that Javascript tests tend to be more fickle and I try to have as stable functional tests as possible.

Tools

For functional testing you should get a tool which allows you to test the HTTP and HTML part of your applications.

As with the generic testing framework this is something that your web framework probably already provides. So check your framework documentation, some pointers:

If you’re outside of a framework there are still great tools that you can use:

I really like the simplicity of the WebTest solution and think that more such tools should be provided. HttpUnit seems to have the same capabilities (using ServletUnit) but SimpleTest Web tester can only work against URLs.

Ground rules

These are some rules I made for myself over the years. They aren’t definitive and to be honest I break most of them from time to time. But if you adhere to those rules your functional tests will be a lot more manageable.

In addition to these the general rules of unit testing also apply of course.

Isolate the application under test

Generally it’s very useful for the tests to run in the same process as the web application under test. This way in-memory databases can be used which makes test suites a lot more stable. At the same time this means that only your tests can access the application and change the state.

If you can’t use an in-memory application, at least try to start up the server at the beginning of the test suite. The worst thing you can do from a stability perspective is to test against some shared deployment of your site.

For example I’ve been responsible for testing a classifieds listing site in the past. The tests created a classified listing using the web form, then checked if it was in the search result, etc. All of this was running against a central test server which also was used by the developers to try out the current version of the code. While this mostly worked, from time to time the tests would fail because some user or another test run was changing the listings. This caused test failures that were very hard to reproduce and were bugs in the tests - not in the code.

Don’t mock

In functional tests you should mock as little as possible. You should really treat the system under test as a blackbox and not know about all the method calls it will execute. In this context, mocks will make your tests very fragile.

To isolate the system you may use test databases (using sqlite’s :memory: for example) or dummy services instead of mocks.

Use Page Objects

You probably want to test the HTML output. For that you’ll need some selectors such as XPath expressions or CSS selectors to access the page contents. You’ll rely on parts of the DOM structure such as element names, CSS classes or IDs for that. Instead of hard-coding those in every test, create page objects. So then when the page layout changes, you’ll only have to adjust the page objects, not every test.

A very simple way to create a page object is to have one class per page type and define property on that. For example:

from lxml import etree

class Page(object):
    """Page object base class. Instantiated with some HTML."""
    def __init__(self, body):
        self._body = body
        self._doc = etree.fromstring(body)

    def __str__(self):
        return self._body

    def _get_text(self, xpath):
        return self._doc.xpath('string(' + xpath + ')').strip()

class DetailPage(Page):
    def get_title(self):
        return self._get_text('/html/head/title')

    def get_page_title(self):
        return self._get_text('//h1')

This is just a simple example of how to do this. It expects a valid XML document as input as it uses the lxml parser. You could instead use BeautifulSoup or something similar.

An example for using these page objects:

def test_get_item(self):
    """app is an instance of webtest.TestApp"""
    response = app.get('/item/id')
    page = DetailPage(response.body)
    assert page.get_title() == 'Test page - Memonic'
    assert page.get_page_title() == 'Test page'

This assumes an existing app object where the test executes a request on. It then checks that the head title and page title are of the expected value.

Conclusion

Functional tests can mean a lot of satisfaction as they test the actual application. Some of my best success moments are when doing test-driven development with functional tests without ever opening a browser. It really allows me to get in a state where I just focus on what I want to accomplish and stay in a very productive mode.

If you have followed my tutorial so far, you now have a solid testing framework, good unit tests and can now progress with testing the meat of your application with functional tests.

In the next installment I’ll cover testing of the Javascript in your application.