1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
"http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<meta content="text/html; charset=iso-8859-1" http-equiv="Content-Type">
<title>The UML test suite</title>
</head>
<body alink="#FF0000" vlink="#55188A" link="#0000EF" bgcolor="#FFFFFF" text="#000099">
<table border="0">
<tr align="left">
<td valign="top">
<table border="0">
<tr align="left"><td valign="top" >
<img src="http://user-mode-linux.sourceforge.net/uml-small.png" height="171" width="120"/>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/index.html">Site Home Page</a></font>
<br>
<font size="-1"><a href="http://uml.harlowhill.com">The UML Wiki</a></font>
<br>
<font size="-1"><a href="http://usermodelinux.org">UML Community Site</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/roadmap.html">The UML roadmap</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/uses.html">What it's good for</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/case-studies.html">Case Studies</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/kernel.html">Kernel Capabilities</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/dl-sf.html">Downloading it</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/run.html">Running it</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/compile.html">Compiling</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/install.html">Installation</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/skas.html">Skas Mode</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/patches.html">Incremental Patches</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/tests.html">Test Suite</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/devanon.html">Host memory use</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/fs_making.html">Building filesystems</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/faq.html">Troubles</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/contrib.html">User Contributions</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/links.html">Related Links</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/todo.html">The ToDo list</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/projects.html">Projects</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/diary.html">Diary</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/thanks.html">Thanks</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/contacts.html">Contacts</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">Tutorials<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/UserModeLinux-HOWTO.html">The HOWTO (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/UserModeLinux-HOWTO.txt">The HOWTO (text)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/hostfs.html">Host file access</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/input.html">Device inputs</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/shared_fs.html">Sharing filesystems</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/fs.html">Creating filesystems</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/resize.html">Resizing filesystems</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/networking.html">Virtual Networking</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/mconsole.html">Management Console</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/debugging.html">Kernel Debugging</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/honeypots.html">UML Honeypots</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/gprof.html">gprof and gcov</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/xtut.html">Running X</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/trouble.html">Diagnosing problems</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/config.html">Configuration</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slack_readme.html">Installing Slackware</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/arch-port.html">Porting UML</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/iomem.html">IO memory emulation</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/2G-2G.html">UML on 2G/2G hosts</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/lksct/index.html">Adding a UML system call</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/nesting.html">Running nested UMLs</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">How you can help<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/help-gen.html">Overview</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/help-doc.html">Documentation</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/help-userspace.html">Utilities</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/help-kernel-v1.html">Kernel bugs</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/projects.html">Kernel projects</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">Screenshots<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/net.html">A virtual network</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/x.html">An X session</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">Transcripts<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/login.html">A login session</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/debug-session.html">A debugging session</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slackinst.html">Slackware installation</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">Reference<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/switches.html">Kernel switches</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slack_readme.html">Slackware README</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">Papers<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/als2000/index.html">ALS 2000 paper (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/als2000.tex">ALS 2000 paper (TeX)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/als2000/slides.html">ALS 2000 slides</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/lca2001/lca.html">LCA 2001 slides</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/ols2001/index.html">OLS 2001 paper (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/ols2001.tex">OLS 2001 paper (TeX)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/als2001/index.html">ALS 2001 paper (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/als2001.tex">ALS 2001 paper (TeX)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/ists2002/umlsec.htm">UML security (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/lca2002/lca2002.htm">LCA 2002 (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/wvu2002/wvu2002.htm">WVU 2002 (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/ists_rt/ists_rt.htm">Security Roundtable (html)</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/ols2002/ols2002.html">OLS 2002 slides</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/slides/lwe2005/LWE2005.html">LWE 2005 slides</a></font>
</td></tr>
<tr align="left"><td valign="top" bgcolor="#e0e0e0">Fun and Games<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/cgi-bin/hangman">Kernel Hangman</a></font>
<br>
<font size="-1"><a href="http://user-mode-linux.sourceforge.net/sdotm.html">Disaster of the Month</a></font>
</td></tr>
</table>
</td>
<td valign="top" align="left">
<center>
<h3>The UML test suite</h3>
</center>
The UML test suite is a set of Perl scripts which I use as a
regression test on new UML patches. It comprises a test harness,
which organizes and runs the suite, and the suite itself, which is a
directory hierarchy which groups similar tests.
<p>
The tests to be done during a particular test run are specified by the
parent directory, if you want all of the tests in that directory to
run, an individual test, or any combination of directories and files.
<p>
The tests are able to specify the properties of the UML in which they
are to run. The test harness compiles and runs UMLs according to the
test requirements, trying to minimize the number of builds and boots
required to satisfy all of the tests' requirements.
<p>
The tests vary widely in what they test and how they work. The
simplest ones run a command (which they may have compiled themselves)
and check the return value. Others run gdb and make it step through a
test program to check that breakpoints work correctly. These use an
already-booted UML, and don't need any special setup of the UML.
<p>
In contrast, there are tests which have requirements of the UML. For
example, the boot_filesystems test checks that a user-specified set of
filesystems boots correctly. So, it requests an unbooted UML for its
use, but puts no more requirements on it. Others test various
configurable options, and request a UML with certain options enabled,
which they boot.
<p>
Finally, there are tests which stress the kernel generally in order to
make sure that it is stable under a hard workload. The kbuild test,
which runs a kernel build, is an example of this.
<a name="Why you should run it"/><table width="100%" bgcolor="#e0e0e0">
<tr>
<td>
<b>
<font color="black">Why you should run it</font>
</b>
</td>
</tr>
</table>
<blockquote head="Why you should run it">
The main reason would be to do a sanity-check on UML when compiled
locally, with whatever unusual config options you favor, and run on
your hardware. UML is somewhat sensitive to toolchain changes, and,
to a less extent, to the host hardware. Users regularly shake out
bugs that don't appear during development, and these are often exposed
by different toolchains and hardware.
<p>
Secondly, if UML fails mysteriously somehow, a test may exercise the
same bug. In this case, the test is likely to be much more specific
than your workload, and the search for the bug much more constrained.
<p>
It would also be a help to the UML project in general to have the
tests run as often as possible, and on as a great a variety of hosts
as possible. This will expose bugs, and get them fixed, more quickly.
</blockquote>
<a name="How to run it"/><table width="100%" bgcolor="#e0e0e0">
<tr>
<td>
<b>
<font color="black">How to run it</font>
</b>
</td>
</tr>
</table>
<blockquote head="How to run it">
First, download the latest version from the <a href="dl-sf.html#The UML test suite">test suite</a> section of
the UML download page.
<blockquote>
<tt>
<font color="#000000" where="host">host%
wget http://www.user-mode-linux.org/mirror/uml_tests_20040726.tar.bz2
</font>
</tt>
</blockquote>
Uncompress and extract it
<blockquote>
<tt>
<font color="#000000" where="host">host% bunzip2 uml_tests_20040726.tar.bz2</font>
</tt>
</blockquote>
<blockquote>
<tt>
<font color="#000000" where="host">host% tar xf uml_tests_20040726.tar</font>
</tt>
</blockquote>
Set up a ~/.umltest, which is the config file needed to tell the
harness about the external resources, such as filesystems, and other
information it will need in order to run. The file contains a perl
hash, with different types of information under different keys. If
you don't know perl, don't panic, because the syntax is simple
enough. Mine looks like this:
<tt>
<pre>
<font size="-1">
{
kernel_pool => "/home/jdike/linux/2.4/um",
uml => {
ubd => { 0 => "/home/jdike/roots/debian_22" },
mem => "64M",
cow_ubds => 1,
prompts => { "/home/jdike/roots/debian_22" => "usermode:.*#" },
},
tests => { "filesystems/boot_filesystems" =>
{ filesystems => [ "/home/jdike/roots/debian_22" ],
time => 0
},
"stress/kbuild" =>
{ kernel_pool => "/home/jdike/roots/kernel",
time => 0 },
},
}
</font>
</pre>
</tt>
The kernel_pool tells the harness the location of a UML source pool.
This will be where it builds whatever UMLs it needs.
<p>
The uml section specifies the default UML command. This will be used
for any tests which don't have special requirements. The ubd hash
within that is a mapping of device numbers to files, and this one will
turn into "ubd0=/home/jdike/roots/debian_22" on the command line. The
prompts hash specifies a regular expression which matches the shell
prompts for the filesystems that will be booted.
<p>
The tests section provides test-specific information. The keys here
are the filenames of the tests under the tests/ directory, and without
the ".pl" extension. They can ask for arbitrary information here. I
specify one filesystem for the boot_filesystems test to boot, and an
image of a filesystem containing a kernel source tree for the kernel
build test.
<p>
Now, the tests can be run as follows
<blockquote>
<tt>
<font color="#000000" where="host">host% perl test.pl</font>
</tt>
</blockquote>
This will run the full test suite, reporting an overall status when
it's done. A full log of the run will appear in tests.log. This will
contain all output from the UMLs, including boot logs and anything the
tests do.
<p>
Specific tests can be run by adding them to the command line
<blockquote>
<tt>
<font color="#000000"> perl test.pl stress/kbuild</font>
</tt>
</blockquote>
Adding a directory to the command line will cause all of the tests
within the that directory to be run. So,
<blockquote>
<tt>
<font color="#000000"> perl test.pl functional</font>
</tt>
</blockquote>
will run all the tests under functional/, but nothing else.
</blockquote>
<a name="How to add tests"/><table width="100%" bgcolor="#e0e0e0">
<tr>
<td>
<b>
<font color="black">How to add tests</font>
</b>
</td>
</tr>
</table>
<blockquote head="How to add tests">
The tests are all under the "tests" directory, each in a separate perl
file. Here is a very simple test, commented:
<tt>
<pre>
<font size="-1">
# Virtually all tests will need these two modules. The first defines
# the Test object, an instance of which is returned from here. The
# second is a library of useful utilities which essentially all of the
# tests use.
use UML::Test;
use UML::Testlib;
# strict is good
use strict;
# This function is the actual test. It will be run inside a booted
# UML, at the shell. run_exit_0 is from TestLib. It runs the dd
# command within the UML object given to the test, returning if the
# command had an exit status of 0, and dying if not. In this case,
# the harness will trap that and report the failure.
sub run {
my $test = shift;
my $uml = shift;
run_exit_0($uml, "dd if=/dev/mem of=/dev/null");
}
# This constructs the Test object and returns it. This specifies that
# this test be run in a booted UML (state => "up") and that the test
# is contained in the 'run' function.
UML::Test->new(state => "up", run => \&run);
</font>
</pre>
</tt>
This illustrates the two essential features of a test
<ul>
<li>
The function which executes the actual test
</li>
<li>
The construction of the test object, which specifies the test function
and any requirements that the test makes on the UML.
</li>
</ul>
The test can also request an unbooted UML with 'state => "up"', if the
purpose of the test is to check that UML boots. It can also request a
UML with special configuration options. The initrd test does this
like so:
<tt>
<pre>
<font size="-1">
UML::Test->new(state => "up",
run => \&run,
kconfig => { MMAPPER => "y" },
config => { "uml/iomem" => "mmapper,$file" } );
</font>
</pre>
</tt>
The kconfig option requests that it be given a UML with CONFIG_MMAPPER
set to "y", and the config option adds 'iomem=mmapper,$file' to the
UML command line.
</blockquote>
</td>
</tr>
</table>
<center>
<font size="-1">Hosted at </font>
<a href="http://sourceforge.net">
<img src="http://sourceforge.net/sflogo.php?group_id=429" width="88" height="31" border="0" alt="SourceForge Logo">
</a>
</center>
</body>
</html>
|
