1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
|
.. highlightlang:: python
.. _`goodpractises`:
Good Integration Practises
=================================================
Work with virtual environments
-----------------------------------------------------------
We recommend to use virtualenv_ environments and use easy_install_
(or pip_) for installing your application dependencies as well as
the ``pytest`` package itself. This way you will get a much more reproducible
environment. A good tool to help you automate test runs against multiple
dependency configurations or Python interpreters is `tox`_.
.. _`virtualenv`: http://pypi.python.org/pypi/virtualenv
.. _`buildout`: http://www.buildout.org/
.. _pip: http://pypi.python.org/pypi/pip
Use tox and Continuous Integration servers
-------------------------------------------------
If you frequently release code to the public you
may want to look into `tox`_, the virtualenv test automation
tool and its `pytest support <http://codespeak.net/tox/example/pytest.html>`_.
The basic idea is to generate a JUnitXML file through the ``--junitxml=PATH`` option and have a continuous integration server like Jenkins_ pick it up
and generate reports.
.. _standalone:
.. _`genscript method`:
Create a py.test standalone script
-------------------------------------------
If you are a maintainer or application developer and want others
to easily run tests you can generate a completely standalone "py.test"
script::
py.test --genscript=runtests.py
generates a ``runtests.py`` script which is a fully functional basic
``py.test`` script, running unchanged under Python2 and Python3.
You can tell people to download the script and then e.g. run it like this::
python runtests.py
.. _`Distribute for installation`: http://pypi.python.org/pypi/distribute#installation-instructions
.. _`distribute installation`: http://pypi.python.org/pypi/distribute
Integrating with distutils / ``python setup.py test``
--------------------------------------------------------
You can integrate test runs into your distutils or
setuptools based project. Use the `genscript method`_
to generate a standalone py.test script::
py.test --genscript=runtests.py
and make this script part of your distribution and then add
this to your ``setup.py`` file::
from distutils.core import setup, Command
# you can also import from setuptools
class PyTest(Command):
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
import sys,subprocess
errno = subprocess.call([sys.executable, 'runtest.py'])
raise SystemExit(errno)
setup(
#...,
cmdclass = {'test': PyTest},
#...,
)
If you now type::
python setup.py test
this will execute your tests using ``runtest.py``. As this is a
standalone version of ``py.test`` no prior installation whatsoever is
required for calling the test command. You can also pass additional
arguments to the subprocess-calls such as your test directory or other
options.
.. _`test discovery`:
.. _`Python test discovery`:
Integration with setuptools/distribute test commands
----------------------------------------------------
Distribute/Setuptools support test requirements,
which means its really easy to extend its test command
to support running a pytest from test requirements::
from setuptools.command.test import test as TestCommand
class PyTest(TestCommand):
def finalize_options(self):
TestCommand.finalize_options(self)
self.test_args = []
self.test_suite = True
def run_tests(self):
#import here, cause outside the eggs aren't loaded
import pytest
pytest.main(self.test_args)
setup(
#...,
tests_require=['pytest'],
cmdclass = {'test': pytest},
)
Now if you run::
python setup.py test
this will download py.test if needed and then run py.test
as you would expect it to.
Conventions for Python test discovery
-------------------------------------------------
``py.test`` implements the following standard test discovery:
* collection starts from the initial command line arguments
which may be directories, filenames or test ids.
* recurse into directories, unless they match :confval:`norecursedirs`
* ``test_*.py`` or ``*_test.py`` files, imported by their `package name`_.
* ``Test`` prefixed test classes (without an ``__init__`` method)
* ``test_`` prefixed test functions or methods are test items
For examples of how to customize your test discovery :doc:`example/pythoncollection`.
Within Python modules, py.test also discovers tests using the standard
:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.
Choosing a test layout / import rules
------------------------------------------
py.test supports common test layouts:
* inlining test directories into your application package, useful if you want to
keep (unit) tests and actually tested code close together::
mypkg/
__init__.py
appmodule.py
...
test/
test_app.py
...
* putting tests into an extra directory outside your actual application
code, useful if you have many functional tests or want to keep
tests separate from actual application code::
mypkg/
__init__.py
appmodule.py
tests/
test_app.py
...
In both cases you usually need to make sure that ``mypkg`` is importable,
for example by using the setuptools ``python setup.py develop`` method.
You can run your tests by pointing to it::
py.test tests/test_app.py # for external test dirs
py.test mypkg/test/test_app.py # for inlined test dirs
py.test mypkg # run tests in all below test directories
py.test # run all tests below current dir
...
.. _`package name`:
.. note::
If py.test finds a "a/b/test_module.py" test file while
recursing into the filesystem it determines the import name
as follows:
* find ``basedir`` -- this is the first "upward" (towards the root)
directory not containing an ``__init__.py``. If both the ``a``
and ``b`` directories contain an ``__init__.py`` the basedir will
be the parent dir of ``a``.
* perform ``sys.path.insert(0, basedir)`` to make the test module
importable under the fully qualified import name.
* ``import a.b.test_module`` where the path is determined
by converting path separators ``/`` into "." characters. This means
you must follow the convention of having directory and file
names map directly to the import names.
The reason for this somewhat evolved importing technique is
that in larger projects multiple test modules might import
from each other and thus deriving a canonical import name helps
to avoid surprises such as a test modules getting imported twice.
.. include:: links.inc
|
