Unit testing

Posted by Patrice Neff Wed, 14 Oct 2009

This post about unit testing is part of the Setting up a full stack for web testing series.

Introduction

A modern application is split up in many functions, classes or modules. In unit testing you test the individual components and make sure they work as specified. This contrasts with functional or integration testing where you test multiple components at once.

As an example let’s assume you’re writing a web based address book. There will be functions to get a list of addresses from the database, update an address in the database, verify credentials of a user who wants to log in, etc. All of these will be tested with unit tests. A functional test on the other hand would for example test the web frontend by creating an address using the web form and then check if it can by found by the integrated search. So a functional test will rely on many of the base units.

To be able to write good unit tests, testability is an important factor. It basically means that you write your units completely independent of each-other. Testability will be covered in detail in a later post.

Now let’s get started with unit testing. Nowadays the basic unit of abstraction is a class. So I usually write one unit test class for each class in the code.

Example

Again all examples are in Python, but should be easily translated into your programming language of choice.

For starters let’s assume we have this model class which handles database access for addresses:

class AddressModel(object):
    """This is a very incomplete address book model."""
    def __init__(self, db):
        """Constructor. db is a sqlite3 object."""
        self.db = db

    def get_address(self, id):
        sql = "SELECT name FROM adr WHERE id=?"
        c = self.db.cursor()
        c.execute(sql, (id,))
        row = c.fetchone()
        if row:
            return {'name': row[0]}
        else:
            return None

    def setup_db(self):
        """Create the database table."""
        self.db.execute("""CREATE TABLE adr (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            name VARCHAR(300)
        )""")

Forgive the ugly code, but it’s the shortest working example I could come up with.

The database is explicitly passed into the model. This might seem strange but this kind of dependency injection is the best way to ensure testability. Also the model is responsible for creating the database model. This will come in very handy for tests where the best way to test is using an in-memory database.

Now let’s write a simple test case. For this demo I just appended the following code to the same file as the address model (test.py) – though in a real project I would of course separate those.

import unittest
import sqlite3

class TestAddressModel(unittest.TestCase):
    def setUp(self):
        self.db = sqlite3.connect(':memory:')
        self.model = AddressModel(self.db)
        self.model.setup_db()

    def create_user(self):
        """Create a simple test user."""
        self.db.execute("INSERT INTO adr (id, name) VALUES (2, 'Tester')")

    def test_get_no_data(self):
        """Database is empty, get_address returns None."""
        self.assertEqual(self.model.get_address(1), None)

    def test_get_other_id(self):
        """Database has one entry. Requesting a different id returns None."""
        self.create_user()
        self.assertEqual(self.model.get_address(1), None)

    def test_get_name(self):
        """Database has one entry, requesting that entry returns a dict."""
        self.create_user()
        self.assertEqual(self.model.get_address(2), {'name': 'Tester'})

    def test_get_sql_injection(self):
        """Ensure SQL Injection is not possible. See BUG-123 for details."""
        self.create_user()
        self.assertEqual(self.model.get_address('id --'), None)

if __name__ == '__main__':
    unittest.main()

The resulting file can be executed using python: python test.py

Ground rules

There are a few very simple rules I try to follow when writing unit tests and I wrote these tests accordingly.

1. Don’t repeat yourself (DRY)

This is a general coding guidelines, but one that should also apply to tests. Make liberal use of the setUp method that most testing frameworks provide. Also extract common test scenarios (fixtures) into methods or even external data files. The create_user method above is an example.

2. Document the tests

Each of your tests should be documented. Where you have an entry in your bug tracking system, make sure you link to that. This way when the bug pops up again, the next developer has a lot more context.

The docstrings I use here in Python are particularly useful as they get printed when the test fails. For example:

======================================================================
FAIL: Database has one entry, requesting that entry returns a dict.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pneff/Desktop/test.py", line 50, in test_get_name
    self.assertEqual(self.model.get_address(2), {'name': 'Tester1'})
AssertionError: {'name': u'Tester'} != {'name': 'Tester1'}

3. Keep your tests small

Each test method should be as small as possible. If you make them too big it will become very hard to find out what exactly is failing. As a rule of thumb you should call only one method from the unit you’re testing.

4. Isolate tests

Do not rely on the order your tests are executed in. This happens more often than you’d think. For example let’s take this test class:

db = sqlite3.connect(':memory:')
model = AddressModel(db)
model.setup_db()

class TestAddressModelBadlyWritten(unittest.TestCase):
    def test_get_empty(self):
        """Database is empty, get_address returns None."""
        self.assertEqual(model.get_address(2), None)

    def test_get_name(self):
        """Database has one entry, requesting that entry returns a dict."""
        db.execute("INSERT INTO adr (id, name) VALUES (2, 'Tester')")
        self.assertEqual(model.get_address(2), {'name': 'Tester'})

On my machine this works at the moment. But when I rename test_get_name to test_get_no_data it stops working. So this is a test where isolation failed.

This example might be clear and easy to spot and is also easy to fix (set up the model in the setUp method as was done in the original example). But this problem may sometimes be hidden behind a few layers of abstraction. A few examples of where you can encounter this:

  • You may be using a central database (the same sqlite file, MySQL database, etc.) for all tests.
  • You may be using some global variables or static class variables in your code.
  • A lot of code saves some state on the file system.
  • When relying on a web service that service can store central state.

You can often solve those problems with dependency injection and by using mocks objects.

If your tests are properly isolated you can also parallelize them. Try for example the nose multiprocess plugin.

Conclusion

Writing tests for a big web application is not easy. But if you follow a few simple rules and properly isolate the core units you have a good foundation to build on.

On the basis of good unit tests you can then go on and write functional tests.