Python `doctest` Module

doctest is a Python standard-library module that searches the code at hand for text that looks like an interactive Python session, executes it, and verifies that the actual output exactly matches the expected output recorded next to the prompt. It is most commonly used to run the examples embedded in module- and function-level docstrings, which ensures the documentation does not silently drift out of sync with the code as the implementation evolves.

This pattern is closely related to three ideas:

• Executable documentation. The examples in the documentation are the tests, so they cannot be left to rot.
• Literate testing. Tests are written as prose, interleaved with the explanation of what the code is doing.
• Literate programming. Donald Knuth’s idea that source code should be written primarily as an explanation directed at a human reader, with the machine-runnable program as a secondary artefact.

There are many real-world examples in the standard Python test suite and libraries. The Fluent Python 2e book uses doctest for all its example code.

Example

A docstring with embedded examples:

def add(a, b):
    """Return the sum of two numbers.
 
    >>> add(2, 3)
    5
    >>> add(-1, 1)
    0
    """
    return a + b

Run with:

python -m doctest mymodule.py -v

Running doctests under pytest

The popular Python testing tool pytest can discover and execute doctests by adding the --doctest-modules flag. To apply it by default, place it in the pytest.ini configuration file:

# pytest.ini
[pytest]
addopts = --doctest-modules

Tradeoffs

doctest is brittle on output formatting. Whitespace, dictionary ordering, floating-point precision and exception tracebacks can all break a doctest even when the underlying behaviour is correct. Use directives such as # doctest: +ELLIPSIS, # doctest: +NORMALIZE_WHITESPACE, or # doctest: +IGNORE_EXCEPTION_DETAIL to soften the comparison when needed.

It is also not a replacement for unit tests. Doctests are best suited to small, illustrative examples that live next to the code they describe. Complex setup, fixtures, mocking and parametrisation belong in pytest or unittest.

Finally, doctests couple the documentation to the behaviour. Any behavioural change forces a corresponding update to the docs. This is usually a feature rather than a bug.

Related

• Python Development Environment Setup