1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
|
.\" SPDX-FileCopyrightText: Peter Pentchev <roam@ringlet.net>
.\" SPDX-License-Identifier: BSD-2-Clause
.Dd May 13, 2023
.Dt TOX-STAGES 1
.Os
.Sh NAME
.Nm tox-stages
.Nd run Tox environments in groups, stop on failure
.Sh SYNOPSIS
.Nm
.Op Fl f Ar filename
.Cm available
.Nm
.Op Fl f Ar filename
.Cm run
.Op Fl Fl arg Ar arg... | Fl A Ar arg...
.Op Fl Fl match\-spec Ar spec | Fl m Ar spec
.Op Fl Fl parallel Ar spec | Fl p Ar spec
.Op Ar stage...
.Sh DESCRIPTION
The
.Nm
tool is used to run Tox test environments in several
stages, one or more environments running in parallel at each stage.
If any of the test environments run at some stage should fail,
.Nm
will stop, not run anything further, and exit with
a non-zero exit code.
This allows quick static check tools like e.g.
.Cm ruff
to stop the testing process early, and also allows scenarios like running
all the static check tools before the package's unit or functional
tests to avoid unnecessary failures on simple errors.
.Sh TAGGING TOX TEST ENVIRONMENTS
The
.Nm
tool expects to be able to invoke an installation of
Tox that will load the
.Cm tox_trivtags
plugin module distributed as part of
the
.Nm test-stages
library.
This module will add a
.Va tags
list of strings to the definition of each
Tox environment; those tags can be specified in the
.Pa tox.ini
file as follows:
.Pp
.Dl [testenv:format]
.Dl skip_install = True
.Dl tags =
.Dl " check"
.Dl " format"
.Dl deps =
.Dl " ..."
.Sh SUBCOMMANDS
.Ss available - can the tox-stages tool be run on this system
The
.Nm
.Cm available
subcommand exits with a code of zero (indicating success) if there is
a suitable version of Tox installed in the same Python execution environment
as the
.Nm
tool itself.
.Ss run - run some Tox environments in stages
The
.Nm
.Cm run
subcommand starts the process of running Tox test
environments, grouped in stages.
If any of the test environments run at some stage should fail,
.Nm
will stop, not run anything further, and exit with
a non-zero exit code.
.Pp
The
.Cm run
subcommand accepts the following options:
.Bl -tag -width indent
.It Fl Fl arg Ar argument | Fl A Ar argument
Pass an additional command-line argument to each Tox invocation.
This option may be specified more than once, and the arguments will be
passed in the order given.
.It Fl Fl match\-spec Ar spec | Fl m Ar spec
Pass an additional specification for Tox environments to satisfy,
e.g.
.Dq -m '@check'
to only run static checkers and not unit tests.
.It Fl Fl parallel Ar spec | Fl p Ar spec
Specify which stages to run in parallel.
The
.Ar spec
parameter is a list of stage indices (1, 2, etc.) or
ranges (4-6); the tests in the specified stages will be run in
parallel, while the tests in the rest of the stages will not.
By default, all tests are run in parallel.
The special values
.Dq ""
(an empty string),
.Dq 0
(a single character, the digit zero), or
.Dq none
will be treated as an empty set, and
no tests will be run in parallel.
.El
.Pp
The positional arguments to the
.Cm run
subcommand are interpreted as
test stage specifications as described in
the parse-stages library's documentation.
If no stage specifications are given on the command line,
.Nm
will read the
.Pa pyproject.toml
file in the same
directory as the
.Pa tox.ini
file, and will look for a
.Va tool.test-stages.stages
list of strings to use.
.Sh FILES
If no stage specifications are given on the command line,
.Nm
will read the
.Pa pyproject.toml
file in the same
directory as the
.Pa tox.ini
file, and will look for a
.Va tool.test-stages.stages
list of strings to use.
.Sh EXAMPLES
Run all the stages as defined in the
.Pa pyproject.toml
file's
.Va tool.test-stages.stages
parameter:
.Pp
.Dl tox-stages run
.Pp
Group Tox environments into stages as defined in the
.Pa pyproject.toml
file, but then only run the ones marked with the "check" tag that also have
names containing the string "format":
.Pp
.Dl tox-stages run -m '@check and format'
.Pp
Run a specific set of stages, passing
.Ar -- -k slug
as additional
Tox arguments so that e.g. a
.Cm pytest
environment that uses the Tox
.Va {posargs}
variable may only run a selected subset of tests:
.Pp
.Dl tox-stages -A -- -A -k -A slug @check unit-tests
.Pp
Execute a somewhat more complicated recipe:
.Bl -tag -width \-
.It -
first, run all test environments with names containing "ruff" in parallel
.It -
then, run the rest of the test environments marked with the "check" tag,
but not marked with the "manual" tag, one by one
.It -
then, run all test environments with names containing "unit" in parallel
.It -
finally, run the rest of the test environments marked with the "tests" tag,
but not marked with the "manual" tag, in parallel
.El
.Pp
.Dl tox-stages -p 1,3-4 ruff '@check and not @manual' unit '@tests and not @manual'
.Sh AUTHORS
The
.Nm
tool, along with its documentation, is developed as part of
the
.Nm test-stages library by
.An Peter Pentchev
.Aq roam@ringlet.net .
|
