/documentations/python_libraries/testing/doctest/

Doctest

The doctest module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown. (see Test Interactive Python Examples)

More concretely, lines where the first non blank entry is >>>, are recognized as python input. All lines not starting with >>> or ..., following an input line, are recognized as output lines. doctest executes the input lines and checks whether the output corresponds to the given output lines.

Text Files

Let's test python from this text file:
>>> a = 1

Now some real test:
    >>> 0 + a
    1

If example.txt contains the lines above, then executing:

python -m doctest -v example.txt

produces the following lines as output:

Trying:
    a = 1
Expecting nothing
ok
Trying:
    0 + a
Expecting:
    1
ok
1 items passed all tests:
   2 tests in example.txt
2 tests in 1 items.
2 passed and 0 failed.
Test passed.

The flag -m doctest tells python to load the module doctest and -v to be verbose. Without the verbose flag, doctest will only produce an output in case of an error. Changing a = 1 to a = 2 in example.txt and issuing python -m doctest example.txt will produce the output:

**********************************************************************
File "example.txt", line 5, in example.txt
Failed example:
    0 + a
Expected:
    1
Got:
    2
**********************************************************************
1 items had failures:
   1 of   2 in example.txt
***Test Failed*** 1 failures.

Docstrings

Executing text files is really useful when writing documentation in a lightweight markup language like Markdown, Textile or reStructuredText. But doctest allows also to integrate examples in a program:

"""
This file implements is_prime:
>>> is_prime(7)
True
"""
def is_prime(n):
    """
    Primality test up to 13
    >>> is_prime(2)
    True
    >>> [is_prime(n) for n in range(6)]
    [False, False, True, True, False, True]
    >>> is_prime(101)
    True
    """
    return n in [2,3,5,7,11,13]

If example.py contains the lines above, then the output of python -m doctest example.py contains:

File "example.py", line 12, in example.is_prime
Failed example:
is_prime(101)
Expected:
    True
Got:
    False

When executing a python file, doctest searches all docstrings for lines starting with >>>. Each docstring is executed in a new context, hence examples in one docstring can not see names defined in other docstrings.

How to Execute doctest

Adding the following lines at the end of example.py, allows to directly execute example.py to test the file with doctest:

if __name__ == "__main__":
    import doctest
    doctest.testmod()

See executing-modules-as-scripts for more information about __name__ == "__main__". More generally doctest can be used in any python program by issuing the functions:

The only mandatory argument is filename for testfile. A useful argument for both functions is verbose=True.

Long Lines

Input code can be split into multiple lines with ...:

>>> if 1 < 0:
...     print "ERROR"
... else:
...     print "OK"
OK

ELLIPSIS allows to only check part of a long output by replacing the rest with .... The example below should succeed when tested with doctest:

>>> import this # doctest: +ELLIPSIS
The Zen of Python, by Tim Peters
<BLANKLINE>
Beautiful is better than ugly.
Explicit is better than implicit.
...
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Note that empty output lines must be replaced with <BLANKLINE>. To check for exceptions see: What About Exceptions.