1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
|
=======================
Using zope.testrunner
=======================
.. IMPORTANT: This document is included in the long_description for
rendering on PyPI, so it cannot use Sphinx-only features.
Installation
============
Buildout-based projects
-----------------------
zope.testrunner is often used for projects that use buildout_::
[buildout]
develop = .
parts = ... test ...
[test]
recipe = zc.recipe.testrunner
eggs = mypackage
The usual buildout process ::
python bootstrap.py
bin/buildout
creates a ``bin/test`` script that will run the tests for *mypackage*.
.. tip::
zc.recipe.testrunner_ takes care to specify the right
``--test-path`` option in the generated script. You can add
other options (such as ``--tests-pattern``) too; check
zc.recipe.testrunner_'s documentation for details.
Virtualenv-based projects
-------------------------
``pip install zope.testrunner`` and you'll get a ``zope-testrunner``
script. Run your tests with ::
zope-testrunner --test-path=path/to/your/source/tree
Your source code needs to be available for the testrunner to import,
so you need to run ``python setup.py install`` or ``pip install -e
.`` into the same virtualenv_.
Some useful command-line options to get you started
===================================================
-p show a progress indicator
-v increase verbosity
-c colorize the output
-t test specify test names (one or more regexes)
-m module specify test modules (one or more regexes)
-s package specify test packages (one or more regexes)
--list-tests show names of tests instead of running them
-x stop on first error or failure
-D, --pdb enable post-mortem debugging of test failures
--xml path generate XML reports to be written at the given path
--help show *all* command-line options (there are many more!)
For example ::
bin/test -pvc -m test_foo -t TestBar
runs all TestBar tests from a module called test_foo.py.
Writing tests
=============
``zope.testrunner`` expects to find your tests inside your package
directory, in a subpackage or module named ``tests``. Test modules
in a test subpackage should be named ``test*.py``.
.. tip::
You can change these assumptions with ``--tests-pattern`` and
``--test-file-pattern`` test runner options.
Tests themselves should be classes inheriting from
``unittest.TestCase``, and if you wish to use doctests, please tell
the test runner where to find them and what options to use for them
in by supplying a function named ``test_suite``.
Example::
import unittest
import doctest
class TestArithmetic(unittest.TestCase):
def test_two_plus_two(self):
self.assertEqual(2 + 2, 4)
def doctest_string_formatting():
"""Test Python string formatting
>>> print('{} + {}'.format(2, 2))
2 + 2
"""
def test_suite():
return unittest.TestSuite([
unittest.defaultTestLoader.loadTestsFromName(__name__),
doctest.DocTestSuite(),
doctest.DocFileSuite('../README.txt',
optionflags=doctest.ELLIPSIS),
])
Test grouping
=============
In addition to per-package and per-module filtering, zope.testrunner
has other mechanisms for grouping tests:
* **layers** allow you to have shared setup/teardown code to be used
by a group of tests, that is executed only once, and not for each
test. Layers are orthogonal to the usual package/module structure
and are specified by setting the ``layer`` attribute on test
suites.
* **levels** allow you to group slow-running tests and not run them
by default. They're specified by setting the ``level`` attribute
on test suites to an int.
Other features
==============
zope.testrunner can profile your tests, measure test coverage,
check for memory leaks, integrate with subunit_, shuffle the
test execution order, and run multiple tests in parallel.
.. _buildout: https://buildout.readthedocs.io
.. _virtualenv: https://virtualenv.pypa.io/
.. _zc.recipe.testrunner: https://pypi.python.org/pypi/zc.recipe.testrunner
.. _subunit: https://pypi.python.org/pypi/python-subunit
|
