1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Using and writing Ghostscript testing scripts</title>
<!-- $Id: Testing.htm,v 1.14.2.1 2004/01/30 06:57:37 giles Exp $ -->
<link rel="stylesheet" type="text/css" href="gs.css" title="Ghostscript Style">
</head>
<body>
<!-- [1.0 begin visible header] ============================================ -->
<!-- [1.1 begin headline] ================================================== -->
<h1>Using and writing Ghostscript testing scripts</h1>
<!-- [1.1 end headline] ==================================================== -->
<!-- [1.2 begin table of contents] ========================================= -->
<!-- [1.2 end table of contents] =========================================== -->
<h2>Table of contents</h2>
<blockquote><ul>
<li><a href="#General_overview">General overview</a>
<li><a href="#Running_tests">Running tests</a>
<ul>
<li><a href="#Individual_tests">Individual tests</a>
<li><a href="#Regression_testing">Regression testing</a>
</ul>
<li><a href="#Writing_new_tests">Writing new tests</a>
</ul></blockquote>
<!-- [1.3 begin hint] ====================================================== -->
<p>
This document describes how to use the scripts located in the <a
href="../toolbin/tests">toolbin/tests</a> directory, and conventions for
writing new testing scripts.
<p>For other information, see the <a href="Readme.htm">Ghostscript
overview</a> and the documentation related to <a
href="Maintain.htm">maintaining Ghostscript</a>.
<!-- [1.3 end hint] ======================================================== -->
<hr>
<!-- [1.0 end visible header] ============================================== -->
<!-- [2.0 begin contents] ================================================== -->
<h2><a name="General_overview"></a>General overview</h2>
<p>
The test scripts discussed here are written in Python, a language whose
implementation is freely available from <a class="offsite"
href="http://www.python.org"><tt>http://www.python.org</tt></a>. The
scripts require Python version 2.1 or later.
<h2><a name="Running_tests"></a>Running tests</h2>
<p>
On Unix and Linux systems, test scripts written in Python can be executed
directly simply by typing their name into the shell, e.g.,
<blockquote><pre>
toolbin/tests/check_source.py
</pre></blockquote>
<p>
On other systems, it may be necessary to invoke Python explicitly, e.g.,
<blockquote><pre>
python toolbin/tests/check_source.py
</pre></blockquote>
<p>
The test scripts will print information about any failures that occur.
<h3><a name="Individual_tests"></a>Individual tests</h3>
<p>
The individual test scripts are named
<b><tt>toolbin/tests/check_</tt></b><em>xxx</em><b><tt>.py</tt></b> (if they
do not run Ghostscript) or
<b><tt>toolbin/tests/gscheck_</tt></b><em>xxx</em><b><tt>.py</tt></b> (if
they do run Ghostscript). Any script with such a name can be run
individually, and is also normally run as part of regression testing
(described in the next section).
<p>
We don't list the individual test scripts here, because any such
documentation would inevitably be out of date most of the time. Each of
these scripts contains documentation about what it tests: we suggest you
read the documentation in the scripts.
<h3><a name="Regression_testing"></a>Regression testing</h3>
<p>
We run a nightly regression test to discover any obvious problems caused by
code checked in the previous day. Here is a list of the scripts and
supporting files that make up the regression test.
<h4>Top-level scripts</h4>
<dl>
<dt><a href="../toolbin/tests/dump_testdb"><tt>dump_testdb</tt></a>
<dd>This script creates an initial test database. It uses gsconf.baselinegs
to create raster data from the test files and computes their MD5 sums and
stores them in the gsconf.testdatadb database.
</dl>
<dl>
<dt><a href="../toolbin/tests/make_testdb"><tt>make_testdb</tt></a>
<dd>A debugging script that will print the contents of the database defined
by gsconf.testdatadb.
</dl>
<dl>
<dt><a href="../toolbin/tests/make_two_versions"><tt>make_two_versions</tt></a>
<dd>A helper script to make two versions of a particular file for visual
diffing or manual analysis. When a test fails nightly regression, this is
generally the first investigative step.
</dl>
<dl>
<dt><a href="../toolbin/tests/make_two_pdfversions"><tt>make_two_pdfversions</tt></a>
<dd>Same as above, except for pdfwrite regressions.
</dl>
<dl>
<dt><a href="../toolbin/tests/testdiff"><tt>testdiff</tt></a>
<dd>this script provides the difference between two sets of regression results.
if end date is omitted, the current date will be used.
<dl>
<dt><a href="../toolbin/tests/revert_baseline"><tt>revert_baseline</tt></a>
<dd>This script is invoked to update the MD5 sum in the test database when a
nightly regression is really a progression. Generally after noticing that
the output from make_two_versions is acceptable or better, this script is
run to log the changes to the database.
</dl>
<dl>
<dt><a href="../toolbin/tests/revert_pdfbaseline"><tt>revert_pdfbaseline</tt></a>
<dd>Same as above, except for pdfwrite baselines.
</dl>
<dl>
<dt><a href="../toolbin/tests/run_nightly"><tt>run_nightly</tt></a>
<dd>In cases where a baseline has been accidentally updated, this script
will revert the database entry to the MD5 sum computed with
gsconf.baselinegs.
</dl>
<dl>
<dt><a href="../toolbin/tests/run_regression"><tt>run_regression</tt></a>
<dd>This script runs the full gamut of regression tests using files from
gsconf.comparedir. It differentiates files by extension and controls what
tests get run and with what options.
</dl>
<dl>
<dt><a href="../toolbin/tests/update_baseline"><tt>update_baseline</tt></a>
<dd>This is the control script (usually invoked by cron) that controls the
nightly test run. It's responsible for checking the latest code out of CVS,
building a new Ghostscript to compare with and launching the test suite via
run_regression.
</dl>
<dl>
<dt><a href="../toolbin/tests/update_pdfbaseline"><tt>update_pdfbaseline</tt></a>
<dd>Same as above, except for pdfwrite baselines.
</dl>
<h4>Support files</h4>
<dl>
<dt><a href="../toolbin/tests/fuzzy.c"><tt>fuzzy.c</tt></a>
<dd>A fuzzy comparison tool appropriate for tests where exact binary matches
aren't appropriate.
</dl>
<dl>
<dt><a href="../toolbin/tests/gsconf.py"><tt>gsconf.py</tt></a>
<dd>This is the master configuration for the scripts above. It controls
where files are found, where Ghostscript executables are and the location of
the test database. Most test configuration will be in this file.
</dl>
<dl>
<dt><a href="../toolbin/tests/gstestgs.py"><tt>gstestgs.py</tt></a>
<dd>This provides classes for running tests that actually execute
Ghostcript.
</dl>
<dl>
<dt><a href="../toolbin/tests/gsparamsets.py"><tt>gsparamsets.py</tt></a>
<dd>What parameters tests get run with by default are stored in this file.
Between this configuration information and the information in gsconf.py all
configurable testing parameters should be covered.
</dl>
<dl>
<dt><a href="../toolbin/tests/gssum.py"><tt>gssum.py</tt></a>
<dd>Helper functions that compute, compare and store MD5 sums.
</dl>
<h2><a name="Writing_new_tests"></a>Writing new tests</h2>
<p>
Some of Ghostscript's test scripts follow a set of conventions that allow
them to be run either stand-alone or as part of a suite; in particular, they
can be run as part of the nightly regression test suite. In this section,
we provide pointers to documentation on how to write new tests that follow
these conventions, since that will make them the most useful.
<p>
The test scripts are based on Python's <b><tt>unittest</tt></b> module. We
suggest that if you are not familiar with this module, you read the
documentation, which is available at <a class="offsite"
href="http://www.python.org/doc/current/lib/module-unittest.html">http://www.python.org/doc/current/lib/module-unittest.html</a>.
<p>
Ghostscript specializes the <b><tt>unittest</tt></b> module by defining
subclasses, which all individual tests use in place of those in
<b><tt>unittest</tt></b>. These subclasses are defined in <a
href="../toolbin/tests/gstestutils.py">toolbin/tests/gstestutils.py</a>.
<p>
Since code documentation separate from the code itself is always out of
date, we have decided to maintain the primary documentation for writing new
tests in <a href="../toolbin/tests/gstestutils.py">gstestutils.py</a> rather
than here in a separate document. Please read that file for more
information.
<!-- [2.0 end contents] ==================================================== -->
<!-- [3.0 begin visible trailer] =========================================== -->
<hr>
<p>
<small>Copyright © 2002 artofcode LLC. All rights reserved.</small>
<p>
<small>This file is part of AFPL Ghostscript. See the <a
href="Public.htm">Aladdin Free Public License</a> (the "License") for full
details of the terms of using, copying, modifying, and redistributing AFPL
Ghostscript.</small>
<p>
<small>Ghostscript version 8.01, 30 January 2004
<!-- [3.0 end visible trailer] ============================================= -->
</body>
</html>
|
