Python: Unit Testing

Unit Testing

This is part of my Python & Django Series which can be found here including information on how to download all the source code.

The ability for software to test and diagnose itself is a powerful feature.

A Simple Example

Lets take a simple function which adds two numbers together.

Code:

def add_two_numbers(a, b):
    """
    A simple method to test
    """

    return a + b

We can create a test to ensure that add_two_numbers works as expected by comparing the result of a call to the function with our expected result.

Code:

from unittest import TestCase, main

class MyTestClass(TestCase):
    """
    A simple unit test example
    """

    def test_add_two_numbers(self):
        self.assertEqual(add_two_numbers(1, 2), 3)

A test class inherits from unittest.TestCase. All functions which are prefixed with test_ will be determined as tests which are required to be run.

Above we call add_two_numbers with parameters 1 and 2. We then use the returned value as a parameter to the assertEqual function along with our expected result of 3.

If the assertion validates as expected the assertion returns allowing control to carry on, otherwise an error is raised and the test is marked as failed.

A test function can have any number of assertions called.

We can run our test function with the main function from unittest.

Code:

if __name__ == '__main__':
    main()

Output:

.py::MyTestClass true
Testing started at 12:42 …

Process finished with exit code 0

If a bug appeared in our code we would see a result similar to the following.

Output:

.py::MyTestClass true
Testing started at 12:44 …

Process finished with exit code 0

Failure
Traceback (most recent call last):
File “/data/data/Dropbox/Development/SandBox/Git/ThePythonPit/PythonSandBox/Testing/unittest_examples/simple_example.py”, line 26, in test_add_two_numbers
self.assertEqual(add_two_numbers(1, 2), 4)
AssertionError: 3 != 4

Assertions

In the previous section we saw the assertEqual assertion. The unittest module provides many assertion functions to cater for a range of possible test criteria.

Equals Assertions

Equality assertion can be made with the assertEqual and inequality assertion can be made with the assertNotEqual function. Both functions take two parameters; the result and the expected result.

Code:

self.assertEqual(1, 1)
self.assertNotEqual(1, 2)

For numerical results the assertAlmostEqual and assertNotAlmostEqual functions allow equality assertion within a tolerance of error. The tolerance is passed in as the third parameter and represents the number of decimal places to be used when determining equality.

The call to assertAlmostEqual takes 1.1 and 1.11 with a tolerance of 1 d.p. This would fail if we used assertEqual but as 1.11 becomes 1.1 when rounding to 1 d.p and therefore the assertion passes.

Code:

self.assertAlmostEqual(1.1, 1.11, 1)  # 3rd argument is the precession
self.assertNotAlmostEqual(1.1, 1.11, 2)  # 3rd argument is the precession

The assertEqual function can take most types. All of the following asserts for lists, tuples, sets, dictionaries and multi-line strings pass assertion.

When being called for collections, the test requires both collections to be of the same type, contain the same number of elements and the elements at the same ordinal position to be equal.

Code:

self.assertEqual([1, 2, 3], [1, 2, 3]) # list
self.assertEqual((1, 2, 3), (1, 2, 3)) # tuple
self.assertEqual({1, 2, 3}, {1, 2, 3}) # set
self.assertEqual({'a': 1}, {'a': 1})  # dictionary
self.assertEqual("onentwo", "onentwo") # multi-line string

Unittest does provide specific assert equal functions for each type though these are implicitly called via the assertEquals function. You should favour using the assertEquals functions.

Code:

self.assertListEqual([1, 2, 3], [1, 2, 3])
self.assertTupleEqual((1, 2, 3), (1, 2, 3))
self.assertSetEqual({1, 2, 3}, {1, 2, 3})
self.assertDictEqual({'a': 1}, {'a': 1})
self.assertMultiLineEqual("onentwo", "onentwo")

Code:

The assertEqual function works upon equality; as such an integer of value 1 and a float of value 1.0 will pass an assertion check together.

self.assertEqual(1.0, 1)

Booleans Assertions

The assertFalse and assertTrue functions for ensuring that a boolean type is either false or true respectively.

Code:

self.assertFalse(False)
self.assertTrue(True)

Collections Assertions

A number of assertions specifically for collections are provided.

We have already seen the assertEqual function which determines if two parameters are equal.

When working with collections this performs the following checks

  • The collection types are equal
  • The collections contain the same number of elements
  • Each element at the same ordinal position equals that in the other collection.

The elements can be of another type as long as their values are equal. In the example below one list contains integers and the other floats but the assertion passes as the elements are equal.

Code:

self.assertEqual([1.0, 2.0, 3.0], [1, 2, 3]) 

The assertSequenceEqual function works the same as assertEqual though it will not fail if the collections are of different types. Below we ensure that the contents of a list and a tuple are equal.

Code:

self.assertSequenceEqual((1, 2, 3), [1, 2, 3])  # Checks only the sequence

The assertIn and assetNotIn funcitons allows checks to see if an element is contained or not contained within a collection. The check is based upon equality.

Here we check that 1 is in 1,2,3 and that 4 is not in 1, 2, 3.

Code:

self.assertIn(1, (1, 2, 3))
self.assertNotIn(4, (1, 2, 3))

The assertCountEqual function has to be a contender for the worst named function in history. This function ensures that two collections contain exactly the same elements though their order is not important.

Code:

self.assertCountEqual((1, 2, 3), (3, 2, 1))  # Badly named. This checked elements and not their order

Comparison Assertions

Python provides the comparison checks in the form of less than, less than or equal to, greater than and greater than or equal to.

Code:

self.assertLess(1, 10)
self.assertLessEqual(1, 1)
self.assertGreater(10, 1)
self.assertGreaterEqual(1, 1)

Identity Assertions

Identity ensures that two parameters point to the same object instance.

In Python each type instance is assigned it’s own object id upon creation. More information can be found here .

The assertIs and assertIsNot can ensure that two objects are and are not the same instance respectively.

Code:

self.assertIs(1, 1)
self.assertIsNot(1, 2)

For parameters which are not referencing any data or have not been initialised they will point to the None type. Here we can check to see if a parameter is pointing to or not pointing to None with the assertIsNone and assertIsNotNone functions.

Code:

self.assertIsNone(None)
self.assertIsNotNone(1)

The assertIsInstance and assertNotIsInstance functions can be used to see if a parameter holds a specific type. Here we pass a parameter holding an instance of a type along with the class name of the type that we want to insure it references or does not reference.

Code:

self.assertIsInstance((), tuple)
self.assertNotIsInstance((), set)

Regular Expressions Assertions

Code:

We can use regular expressions to ensure the format of a string is as expected with the assertRegex and assertNotRegex functions

self.assertRegex('Luke', "^[a-zA-Z]{3,4}$")
self.assertNotRegex('Lukey', "^[a-zA-Z]{3,4}$")

Exceptions Assertions

Code should throw exceptions when we want it to or when it is called incorrectly. We can use the assertRaises function to assert that not only an exception is raised but it is of a certain type.

Below we ensure that a ZeroDivisionError error is raised.

Code:

with self.assertRaises(ZeroDivisionError) as ex:
    result = 1 / 0

self.assertEqual(str(ex.exception), "division by zero")

In the above example we assign the raised exception to a variable ex, we can then run assertions upon the exception to make sure it is as expected. We check the string representation of the object is as expected. The latter check can be enforced with the assertRaisesRegex function.

Code:

with self.assertRaisesRegex(ZeroDivisionError, "^division by [a-zA-z]{4}$"):
    result = 1 / 0

We can also annotate a test with the @expectedFailure attribute. Here the test will fail if an error is not raised.

Output:

  @expectedFailure
    def test_expectedFailure(self):
        self.fail("This is an expected failure")

Warnings Assertions

Python provides the same functions for warnings as it does for exceptions; they work in exactly the same way

Code:

with self.assertWarns(DeprecationWarning) as wn:
    warn("deprecated", DeprecationWarning)

self.assertEqual(str(wn.warning), "deprecated")

with self.assertWarnsRegex(DeprecationWarning, "^deprecate[a-z]$"):
    warn("deprecated", DeprecationWarning)

Assertions Messages

Each assertion can optionally take a string to be used as an error message when the test fails.

Code:

self.assertFalse(False, "False is not false!")

Would report as the following:

Output:

AssertionError: True is not false : False is not false

The following would be reported if the error message had not been provided.

Output:

AssertionError: True is not false

Failing Tests

We can fail a test in code with the fail method.

Code:

self.fail("Fail!!!")

Test Fixture

If a test class has a function called setUp, it will be run before every test function within it. If an error is raised within the setUp function then no test functions will be run.

If a test class has a function called tearDown, it will be run after every test function within it. This function will always be run after each test function regardless if the test passes or fails.

Code:

from unittest import TestCase


class TestFixtureExample(TestCase):

    def setUp(self):
        # Set up / initialise before a test
        # If this fails then no tests will be run
        print("In the setUp")

    def tearDown(self):
        # Destroy any resources required during the test
        # Will always be run if setUp runs regardless of tests successes
        print("In the tearDown")

    def test_fixture_one(self):
        self.assertTrue(True)

    def test_fixture_two(self):
        self.assertTrue(True)

    def test_fixture_three(self):
        self.assertTrue(True)

Output:

.py::TestFixtureExample true
Testing started at 14:17 …
In the setUp
In the tearDown
In the setUp
In the tearDown
In the setUp
In the tearDown

Test Suite

The TestSuite class can be used to register tests which can then be run with the TextTestRunner.

The addTest can be used to add an individual test method into a TestSuite instance.

The TestLoader().loadTestsFromTestCase() can be used to create a TestSuite with all test functions of a test class.

The TextTestRunner().run() function can then run all TestSuites passed in.

**Code:

from unittest import TestSuite, TextTestRunner, TestLoader

# Test Suite
def my_test_suite():
    suite_one= TestSuite()
    suite_one.addTest(MyTestClass('test_add_two_numbers')) # Adds MyTestClass.test_add_two_numbers()

    suite_two = TestLoader().loadTestsFromTestCase(TestAssertsExample)

    return TestSuite([suite_one, suite_two])

# Run the test suite
if __name__ == '__main__':
    TextTestRunner().run(my_test_suite())

Skipping Tests

Test functions can be annotated with specific unittest attributes.

Skip can be used to stop a test from running. This can also be done in code with the SkipTest function

SkipIf can be used to stop a test from running if a boolean statement evaluates to true.

SkipUnless can be used to stop a test from running unless a boolean statement evaluates to true.

Code:

class TestAttributes(TestCase):

    @skip("Test is not run")
    def test_skip(self):
        self.fail("This should not be run")

    @skipIf(True, "This is not run")
    def test_skipIf(self):
        self.fail("This should not be run")

    @skipUnless(False, "This is not run")
    def test_skipUnless(self):
        self.fail("This should not be run")

    def test_skipTest(self):
        SkipTest("This should not be run")
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s