1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
|
.TH SHELLTEST 1 "March 18 2012" "shelltestrunner" "version 1.2.1"
.SH NAME
.PP
shelltestrunner - test command-line programs or arbitrary shell commands
.SH SYNOPSIS
.PP
shelltest [\f[I]options\f[]] {\f[I]testfiles\f[]|\f[I]testdirs\f[]}
.SH DESCRIPTION
.PP
shelltestrunner tests command-line programs (or arbitrary shell
commands).
It reads simple declarative tests specifying a command, some input, and
the expected output, and can run them run in parallel, selectively, with
a timeout, in color, and/or with differences highlighted.
.SH OPTIONS
.TP
.B -a, --all
Show all failure output, even if large
.RS
.RE
.TP
.B -c, --color
Show colored output if your terminal supports it
.RS
.RE
.TP
.B -d, --diff
Show failures in diff format
.RS
.RE
.TP
.B -p, --precise
Show failure output precisely (good for whitespace)
.RS
.RE
.TP
.B -x \f[I]STR\f[], --exclude=\f[I]STR\f[]
Exclude test files whose path contains \f[I]STR\f[]
.RS
.RE
.TP
.B --execdir
Run tests from within the test file\[aq]s directory.
Test commands normally run within your current directory;
\f[C]--execdir\f[] makes them run within the directory where they are
defined, instead.
.RS
.RE
.TP
.B --extension=\f[I]EXT\f[]
Filename suffix of test files (default: \f[I]\&.test\f[])
.RS
.RE
.TP
.B -w, --with=\f[I]EXECUTABLE\f[]
Replace the first word of (unindented) test commands.
This option replaces the first word of all test commands with something
else, which can be useful for testing alternate versions of a program.
Commands which have been indented by one or more spaces will not be
affected by this option.
.RS
.RE
.TP
.B --debug
Show debug info, for troubleshooting
.RS
.RE
.TP
.B --debug-parse
Show test file parsing info and stop
.RS
.RE
.TP
.B --help-format
Display test format help
.RS
.RE
.TP
.B -?, --help
Display help message
.RS
.RE
.TP
.B -V, --version
Print version information
.RS
.RE
.TP
.B -- \f[I]TFOPTIONS\f[]
Set extra test-framework options like \f[C]-j\f[]/\f[C]--threads\f[],
\f[C]-t\f[]/\f[C]--select-tests\f[], \f[C]-o\f[]/\f[C]--timeout\f[],
\f[C]--hide-successes\f[].
Use \f[C]--\ --help\f[] for a list.
Avoid spaces.
.RS
.RE
.SH DEFINING TESTS
.PP
Test files, typically named \f[C]tests/*.test\f[], contain one or more
tests consisting of:
.IP \[bu] 2
a one-line command
.IP \[bu] 2
optional standard input (\f[C]<<<\f[]), standard output (\f[C]>>>\f[])
and/or standard error output (\f[C]>>>2\f[]) specifications
.IP \[bu] 2
an exit status (\f[C]>>>=\f[]) specification
.PP
\f[B]Test format:\f[]
.IP
.nf
\f[C]
#\ optional\ comment
the\ command\ to\ test
<<<
zero\ or\ more\ lines\ of\ standard\ input
>>>
zero\ or\ more\ lines\ of\ expected\ standard\ output
(or\ /REGEXP/\ added\ to\ the\ previous\ line)
>>>2
zero\ or\ more\ lines\ of\ expected\ standard\ error\ output
(or\ /REGEXP/\ added\ to\ the\ previous\ line)
>>>=\ EXITCODE\ (or\ /REGEXP/)
\f[]
.fi
.IP \[bu] 2
A \f[C]/REGEXP/\f[] pattern may be used instead of explicit data.
In this case a match anywhere in the output allows the test to pass.
The regular expression syntax is
regex-tdfa (http://hackage.haskell.org/package/regex-tdfa)\[aq]s.
.IP \[bu] 2
\f[C]EXITCODE\f[] is a numeric exit
status (http://en.wikipedia.org/wiki/Exit_status), eg \f[C]0\f[] for a
successful exit.
.IP \[bu] 2
You can put \f[C]!\f[] before a \f[C]/REGEXP/\f[] or \f[C]EXITCODE\f[]
to negate the match.
.IP \[bu] 2
Comment lines beginning with \f[C]#\f[] may be used between tests.
.SH EXAMPLES
.PP
Here\[aq]s \f[C]example.test\f[], a file containing two simple tests:
.IP
.nf
\f[C]
#\ 1.\ let\[aq]s\ test\ that\ echo\ runs.\ Numbering\ your\ tests\ can\ be\ helpful.
echo
>>>=\ 0
#\ 2.\ and\ now\ the\ cat\ command.\ On\ windows,\ this\ one\ should\ fail.
cat
<<<
foo
>>>
foo
>>>=\ 0
\f[]
.fi
.PP
Run it with \f[C]shelltest\f[]:
.IP
.nf
\f[C]
$\ shelltest\ example.test
:t.test:1:\ [OK]
:t.test:2:\ [OK]
\ \ \ \ \ \ \ \ \ Test\ Cases\ \ Total
\ Passed\ \ 2\ \ \ \ \ \ \ \ \ \ \ 2
\ Failed\ \ 0\ \ \ \ \ \ \ \ \ \ \ 0
\ Total\ \ \ 2\ \ \ \ \ \ \ \ \ \ \ 2
\f[]
.fi
.SH AUTHORS
Simon Michael.
|
