1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
|
NEWS
====
This file describes user-visible changes to QMTest.
QMTest 2.4.1
============
Address a couple of critical issues preventing QMTest from working
with Python 2.6.
QMTest 2.4
==========
A new "expectation_database" extension kind has been added, with two
built-in implementations, previous_testrun.PreviousTestRun, and
xml_expectation_database.XMLExpectationDatabase. The command-line
interface has been enhanced to support this.
A new CompilationTestDatabase has been added, and various related
test classes and utility functions have been improved to support it.
The QMTest documentation has been much improved. The tutorial has been
restructured to contain three chapters:
1) concepts, 2) customization, 3) extension.
Changes have been made to make QMTest work with Python 2.5.
QMTest now installs its data into <prefix>/share/qmtest, and its
documentation into <prefix>/share/doc/qmtest.
QM 2.3
======
There is now a "qmtest report" command-line option that can be used to
create test reports from multiple result files.
There is a new "host" extension kind (with a built-in set of predefined
host types such as 'localhost.LocalHost', 'ssh_host.SSHHost', etc.)
QMTest now installs its data into <prefix>/share/qm, and its documentation
into <prefix>/share/doc/qm.
QMTest has been made robust and more flexible. In particular,
QMTest now runs with Python 2.4, and is now suitable for scripting
as a programmatic way to access and manipulate result (and expectation)
files, instead of the command line interface.
QM 2.2
======
Under Windows, tests derived from command.ExecTestBase that explicitly
set environment variables for the child process did not operate
correctly. This problem has been corrected.
Resources are now destroyed in reverse order of creation.
Signal masks are now set correctly for processes spawned by QMTest,
even when QMTest is running with multiple threads.
QMTest can provide output reports compatible with that from TET.
QMTest's DejaGNU emulation has been updated and improved.
QM 2.1.2
========
A problem with the Windows packaging script that resulted in qmtest.py
not being installed in the Python Scripts directory has been
corrected.
QM 2.1.1
========
A problem that prevent ShellScriptTest from working on Windows with
the native Windows command-interpreter has been fixed.
QM 2.1
======
QMTest requires Python 2.2 or greater.
On Windows, we recommend that Mozilla be used with QMTest. Internet
Explorer appears to handle incorrectly some forms that QMTest creates.
CodeSourcery has validated the HTML using automated and manual
methods, and believes that the problems lie with Internet Explorer and
not with QMTest.
There is now a "qmtest create" command-line option that can be used to
create extension instances (such as tests, resources, or databases)
programmatically. The "--class" option to the qmtest create-tdb"
command now supports the same descriptor syntax as "qmtest create".
There is a new "executable.TimeoutExecutable" class that can be used
to run a child process with a timeout. The "command" test classes now
make use of this functionality to support running child processes with
a timeout.
There is now a "-O" or "--outcomes" option for setting the expected
outcomes, even when working within in the GUI.
There is a new "--result-stream" option to "qmtest run" and "qmtest
summarize" that allows users to specify custom result streams.
The exit code returned by QMTest is now documented, and now
distinguishes between a failure of QMTest to carry out the action
requested by the user and the failure of a test run by QMTest.
Comparisions between the expected standard output and actual standard
output done by test classes in the "command" module now ignore line
endings. This change facilitates moving tests between operating
systems.
A defect that could cause a multi-threaded or multi-process tests run
to hang if a resource could not be set up has been corrected.
A defect that resulted in resources being created only per run of the
GUI, rather than once per test run, has been corrected.
The XML results file format has been improved to make it more
readable, and to make it easier to postprocess the XML.
QMTest now correctly uses ";" as the separator in QMTEST_CLASS_PATH on
Windows.
A defect that could cause crashes in the RSHTarget class has been
corrected.
The QM_PYTHON environment variable is no longer honored by QMTest. To
control which Python interpreter is used, run:
/path/to/python /path/to/prefix/bin/qmtest
explicitly.
QM 2.0.3
========
A JavaScript issue that prevented the pop-up menus in the GUI from
working correctly with Mozilla 1.0.1 was corrected.
A bug that could cause QMTest to hang when running tests in parallel
has been corrected.
QM 2.0.2
========
When a test fails due to dependency on a resource that could not be
set up, the GUI now displays a link to the resource result from the
test result. (Previously, there was no way to get information about
resource failures in the GUI.)
The GUI now offers a choice of available prerequisite tests and
resources when editing a test.
The substitutions field for the FileContentsTest now works correctly
in the GUI. This is an incompatible change; tests that use this test
class will require hand-editing.
An argument in a derived test or resource class which has the same
name as an argument in a base class now overrides the base class
argument. Therefore, every test class will have only one argument
with a given name. (Previously, both the base and derived class
arguments were present.)
QM 2.0.1
========
A bug that prevented the "New Resource" entry on the "File" menu from
working has been fixed.
QM 2.0
======
Netscape 4 is no longer supported by the QMTest GUI. Mozilla 1.x and
Internet Explorer 6 are the recommended browsers for use with the
QMTest GUI. On UNIX-like systems, QMTest will now try to run
"mozilla", rather than "netscape", by default.
Tests are no longer run in a random order by default. To run tests in
a random order, you must now use the "--random" option to "qmtest run".
QMTest now has a "ProcessTarget" target class that can be used to
run tests in parallel on systems that do not support threads.
QMTest now requires that all extension classes be listed in the
classes.qmc file in the directory that contains the extension class.
QMTest now uses a uniform XML file format for all extension classes,
including tests, resources, databases, and targets. This format will
remain stable for the forseeable future. While QMTest 2.0 can read
the formats created by previous versions of QMTest, it will only write
files in the new format. Most users will be unaffected by these
changes.
New command-line options:
-------------------------
QMTest now has a "--version" option.
The "qmtest run" command now has a "--rerun" option to rerun only
those tests which had unexpected outcomes when the tests were last
run.
The "qmtest run" command now has a "--random" option to run the tests in
a random order.
The "qmtest extensions" command can be used to list the available
extension classes.
The "qmtest register" command can be used to register new extension
classes and to try loading an extension class.
The "qmtest help" command is a synonym for "qmtest --help".
The "qmtest run" and "qmtest gui" commands now use a file named
"context" in the current directory as a context file if no explict -C
option is provided.
QM 1.1.5
========
When loading a test, resource, or database class, the directory
containing the extension class is placed at the front of the path used
to search for modules. In previous versions of QMTest, it was placed
at the back.
QM 1.1.4
========
The QMTest GUI no longer crashes when the "Cancel" button is hit when
creating a new test suite.
Commands like "qmtest create-tdb --attribute X=Y" where "Y" contains
spaces are now handled correctly.
QM 1.1.3
========
The "-c" option to "qmtest run" did not work correctly in QM 1.1.2.
This problem has been corrected.
QM 1.1.2
========
QM is now licensed under the GNU General Public License.
The QMTest GUI has been substantially improved.
It is now possible to load and save the context in the GUI.
Numerous small bugs have been fixed.
QM 1.1
======
QMTest now works on systems that do not have support for threads, or,
more precisely, when using a Python interpreter for which "import
thread" fails.
The Makefiles and build tree have been reorganized so as to make
installation from source simpler.
QMTest should now build on Debian GNU/Linux without configure options.
|
