1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<meta http-equiv="CONTENT-TYPE"
content="text/html; charset=iso-8859-1">
<title></title>
<meta name="GENERATOR" content="StarOffice/5.2 (Linux)">
<meta name="AUTHOR" content=" ">
<meta name="CREATED" content="20000320;9474300">
<meta name="CHANGEDBY" content="Ed Falis">
<meta name="CHANGED" content="20001129;15395100">
</head>
<body>
<h1><font color="#33ff33">A</font><font color="#cc0000">U</font>nit
Cookbook</h1>
<p
style="border-style: none none double; border-color: -moz-use-text-color -moz-use-text-color rgb(128, 128, 128); border-width: medium medium 1.1pt; padding: 0in 0in 0.02in;">
<font style="font-size: 11pt;" size="2">Version 1.03</font></p>
<p><br>
This is a short guide for using the AUnit test framework.
AUnit is an adaptation of the Java JUnit (Kent Beck, Erich Gamma) unit
test framework for Ada code. This document is adapted from the JUnit
Cookbook document contained in the JUnit release package.</p>
<h2>Simple Test Case</h2>
<p>How do you write testing code? </p>
<p>The simplest way is as an expression in a debugger. You can change
debug expressions without recompiling, and you can wait to decide
what to write until you have seen the running objects. You can also
write test expressions as statements which print to the standard
output stream. Both styles of tests are limited because they require
human judgment to analyze their results. Also, they don't compose
nicely- you can only execute one debug expression at a time and a
program with too many print statements causes the dreaded "Scroll
Blindness". </p>
<p>AUnit tests do not require human judgment to interpret, and it is
easy to run many of them at the same time. When you need to test
something, here is what you do: </p>
<ol>
<li>
<p style="margin-bottom: 0in;">Declare a package for a test case -
a set of logically related test routines. A template for such a package
is in /AUnit/Template/pr_xxxx_xxx.ad*.</p>
</li>
<li>
<p style="margin-bottom: 0in;">Derive from <i><span style="">AUnit</span><b>.</b>Test_Cases.Test_Case</i>
in the new package. </p>
</li>
<li>
<p style="margin-bottom: 0in;">The new derived type must provide
implementations of <i>Register_Tests</i> and <i>Name.</i></p>
</li>
<li>
<p style="margin-bottom: 0in;">Write each test routine (see below)
and register it with a line in routine <i>Register_Tests</i>, of the
form:</p>
<p style="margin-bottom: 0in;"><br>
<font size="2"><font face="courier, monospace">Register_Routine (T,
Test_Name'Access, "Description of test routine");</font></font></p>
</li>
<li>
<p>When you want to check a value, use:<br>
<br>
<font size="2"><font face="courier, monospace">AUnit.Assertions.A</font></font><tt><font
size="2"><font face="courier, monospace">ssert</font></font></tt><font
size="2"><font face="courier, monospace">(Boolean_Expression,
String_Description);</font></font></p>
</li>
<li>
<p>Create a suite function to gather together test cases and
sub-suites.</p>
</li>
<li>
<p>At any level at which you wish to run tests, create a harness
instantiating Aunit.Test_Runner with a suite function collecting
together test cases and sub-suites to execute.</p>
</li>
<li>
<p>Be sure to initialize the source path for GNAT to include
/aunit-1.01/aunit and its subdirectories. In GLIDE, this can be done
using the "Load Recursive Directory" command in the project editing
window. For other compilation systems, make these source file
subdirectories known to them using their specific conventions.</p>
</li>
<li>
<p align="left">Build the harness routine using gnatmake. <font
size="3"><font face="Times New Roman, serif">The Glide project file
aunit-1.01/aunit/aunit.adp contains all the necessary links and
switches for building test cases. When testing a new compiler, as
opposed to incremental unit tests, the GNAT "-f" switch should be set
for gnatmake. One can then use GLIDE to build and run the tests, making
sure that aunit.adp is the default project file, setting "-f" if
needed, and building (^C-C) and running (^C-R). For other compilation
systems, use their standard build commands, ensuring that the
subdirectories under aunit-1.01/aunit are known to the compilation
system as containing sources.</font></font></p>
</li>
</ol>
<p>For example, to test that the sum of two Moneys with the same
currency contains a value which is the sum of the values of the two
Moneys, the test routine would look like: </p>
<pre><font face="Courier"><font size="2">procedure Test_Simple_Add</font></font>
<font face="Courier"><font size="2"> (T : Aunit.Test_Cases.Test_Case'Class) is</font></font>
<font face="Courier"><font size="2"> X, Y: Some_Currency;</font></font>
<font face="Courier"><font size="2">begin</font></font>
<font face="Courier"><font size="2"> X := 12; Y := 14;</font></font>
<font face="Courier"><font size="2"> Assert (X + Y = 24, "Addition is incorrect");</font></font>
<font face="Courier"><font size="2">end;</font></font></pre>
<p style="margin-bottom: 0in;">The package spec (taken almost directly
from pr_xxxx_xxx.ads) looks
as follows. The only modification was to remove support for a test
fixture (next section), and to provide a name for the unit. Changes
to "boilerplate code" are in bold:</p>
<p style="margin-bottom: 0in;"><br>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">with
Ada.Strings.Unbounded;</font></font></p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">use
Ada.Strings.Unbounded;</font></font></p>
<p style="margin-bottom: 0in;"><br>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">with
AUnit.Test_Cases;</font></font></p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">use
AUnit.Test_Cases;</font></font></p>
<p style="margin-bottom: 0in;"><br>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">package
<b>PR_xxxx_xxx</b> is </font></font>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">type
Test_Case is new AUnit.Test_Cases.Test_Case </font></font>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">with
null record;</font></font></p>
<p style="margin-bottom: 0in;"><br>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">--
Register routines to be run:</font></font></p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">procedure
Register_Tests (T: in out Test_Case);</font></font></p>
<p style="margin-bottom: 0in;"><br>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">--
Provide name identifying the test case:</font></font></p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">function
Name (T: Test_Case) return String_Access;</font></font></p>
<p style="margin-bottom: 0in;"><br>
</p>
<p style="margin-bottom: 0in;"><font face="Courier, monospace"><font
size="2">end
<b>PR_xxxx_xxx</b>;</font></font></p>
<p><br>
<br>
</p>
<p style="margin-bottom: 0in;"><font face="Times New Roman, serif"><font
size="3">The
package body, constructed by modifying pr_xxxx_xxx.adb is:</font></font></p>
<p style="margin-bottom: 0in;"><br>
</p>
<pre><font size="2">with AUnit.Test_Cases.Registration;</font>
<font size="2">use AUnit.Test_Cases.Registration;</font>
<font size="2">with AUnit.Assertions; use AUnit.Assertions;</font>
<font size="2">-- Template for test case body.</font>
<font size="2">package body <b>PR_xxxx_xxx</b> is</font>
<font face="Courier, monospace"><font size="2"><b> -- Simple test routine:</b></font></font>
<font face="Courier, monospace"><font size="2"><b> procedure Test_Simple_Add</b></font></font>
<font face="Courier, monospace"><font size="2"><b><font face="Courier"> (T : Aunit.Test_Cases.Test_Case'Class)</font> is</b></font></font>
<font face="Courier, monospace"><font size="2"><b> X, Y: Some_Currency;</b></font></font>
<font face="Courier, monospace"><font size="2"><b> begin</b></font></font>
<font face="Courier, monospace"><font size="2"><b> X := 12; Y := 14;</b></font></font>
<font face="Courier, monospace"><font size="2"><b> Assert </b></font></font>
<font face="Courier, monospace"><font size="2"><b> (X + Y = 26, "Addition is incorrect");</b></font></font>
<font face="Courier, monospace"><font size="2"><b> end;</b></font></font>
<font size="2"> -- Register test routines to call:</font>
<font size="2"> procedure Register_Tests (T: access Test_Case) is</font>
<font size="2"> begin</font>
<font size="2"> -- Repeat for each test routine:</font>
<font size="2"> <b> Register_Routine (T, Test_Simple_Add'Access, "Test Addition");</b></font>
<font size="2"> end Register_Tests;</font>
<font size="2"> -- Identifier of test case. Just change the string</font>
<font size="2"> -- result of the function.</font>
<font size="2"> function Name (T: Test_Case) return String_Access is</font>
<font size="2"> begin</font>
<font size="2"> return new String'(<b>"Money Tests"</b>);</font>
<font size="2"> end Name;</font>
<font size="2">end <b>PR_xxxx_xxx</b>;</font></pre>
<p><font size="3">The corresponding harness code, adapted from
aunit-1.01/template/harness.adb is:</font></p>
<pre><font size="2">with AUnit.Test_Suites; use AUnit.Test_Suites;</font>
<font size="2">with AUnit.Test_Runner;</font>
<font size="2">-- List of tests and suites to run:</font>
<font size="2"><b>with PR_XXXX_XXX;</b></font>
<font size="2">procedure Harness is</font>
<font size="2"><b> function Suite return Access_Test_Suite is</b></font>
<font size="2"><b> Result : Access_Test_Suite := new Test_Suite;</b></font>
<font size="2"><b> begin</b></font>
<font size="2"><b> -- You may add multiple tests or suites here:</b></font>
<font size="2"><b> Add_Test (Result, new PR_XXXX_XXX.Test_Case);</b></font>
<font size="2"><b> return Result;</b></font>
<font size="2"><b> end Suite;</b></font>
<font size="2"> procedure Run is new AUnit.Test_Runner (Suite);</font>
<font size="2">begin</font>
<font size="2"> Run;</font>
<font size="2">end Harness;</font></pre>
<h2>Fixture</h2>
<p>Tests need to run against the background of a known set of
objects. This set of objects is called a test fixture. When you are
writing tests you will often find that you spend more time writing
the code to set up the fixture than you do in actually testing
values. </p>
<p>To some extent, you can make writing the fixture code easier by
paying careful attention to the constructors you write. However, a
much bigger savings comes from sharing fixture code. Often, you will
be able to use the same fixture for several different tests. Each
case will send slightly different messages or parameters to the
fixture and will check for different results. </p>
<p>When you have a common fixture, here is what you do: </p>
<ol>
<li>
<p style="margin-bottom: 0in;">Create a package as in the previous
section, starting from the templates pr_xxxx_xxx.ad*</p>
</li>
<li>
<p style="margin-bottom: 0in;">Add fields for elements of the
fixture into package body.</p>
</li>
<li>
<p>Override <i><font face="courier, monospace">Set_Up</font></i>
to initialize the variables </p>
</li>
<li>
<p>Override <font face="courier, monospace"><i>Tear_Down</i> </font>to
release any permanent resources you allocated in <i>S<font
face="courier, monospace">et_Up</font></i><font
face="courier, monospace"> </font> </p>
</li>
</ol>
<p>For example, to write several test cases that want to work with
different combinations of 12 French Francs, 14 French Francs, and 26
US Dollars, first create a fixture. The package spec is now:</p>
<pre>with Ada.Strings.Unbounded;<br>use Ada.Strings.Unbounded;<br><br>with AUnit.Test_Cases;<br>use AUnit.Test_Cases;<br>package PR_xxxx_xxx is<br> type Test_Case is new AUnit.Test_Cases.Test_Case with null record;<br><br> -- Register routines to be run:<br> procedure Register_Tests (T: in out Test_Case);<br><br> -- Provide name identifying the test case:<br> function Name (T: Test_Case) return String_Access;<br><br><br><b> -- Preparation performed before each routine:</b>
<b> Procedure Set_Up (T: in out Test_Case);</b>
end PR_xxxx_xxx;</pre>
<p>The body becomes:</p>
<pre><font size="2">with AUnit.Test_Cases.Registration;</font>
<font size="2">use AUnit.Test_Cases.Registration;</font>
<font size="2">with AUnit.Assertions; use AUnit.Assertions;</font>
<font size="2"><b>with Currencies; use Currencies;</b></font>
<font size="2">package body PR_xxxx_xxx is</font>
<font size="2"> <b>-- Fixture elements:</b></font>
<font size="2"> <b> FR_12, FR_14: French_Franc; </b></font>
<font size="2"><b> US_26: US_Dollar;</b></font>
<font size="2"> <b>-- Preparation performed before each routine:</b></font>
<font size="2"><b> Procedure Set_Up (T: in out Test_Case) is</b></font>
<font size="2"><b> begin</b></font>
<font size="2"><b> FR_12 := 12; FR_14 := 14;</b></font>
<font size="2"><b> US_26 := 26;</b></font>
<font size="2"><b> end Set_Up;</b></font>
<font face="Courier, monospace"><font size="2"> -- Simple test routine:</font></font>
<font face="Courier, monospace"><font size="2"> procedure Test_Simple_Add</font></font>
<font face="Courier, monospace"><font size="2"><font face="Courier"> (T : Aunit.Test_Cases.Test_Case'Class)</font> is</font></font>
<font face="Courier, monospace"><font size="2"> begin</font></font>
<font face="Courier, monospace"><font size="2"><b> Assert </b></font></font>
<font face="Courier, monospace"><font size="2"><b> (FR_12 + FR_14 /= US_26, </b></font></font>
<font face="Courier, monospace"><font size="2"><b> "US and French currencies not diffentiated");</b></font></font>
<font face="Courier, monospace"><font size="2"><b> </b> end;</font></font>
<font size="2"> -- Register test routines to call:</font>
<font size="2"> procedure Register_Tests (T: in out Test_Case) is</font>
<font size="2"> begin</font>
<font size="2"> -- Repeat for each test routine:</font>
<font size="2"> Register_Routine (T, Test_Simple_Add'Access, "Test Addition");</font>
<font size="2"> end Register_Tests;</font>
<font size="2"> -- Identifier of test case. Just change the string</font>
<font size="2"> -- result of the function.</font>
<font size="2"> function Name (T: Test_Case) return String_Access is</font>
<font size="2"> begin</font>
<font size="2"> return new String'("Money Tests");</font>
<font size="2"> end Name;</font>
<font size="2">end PR_xxxx_xxx;</font></pre>
<p>Once you have the fixture in place, you can write as many test
routines as you like. Calls to <font face="courier, monospace">Set_Up</font>
and <font face="courier, monospace">Tear_Down</font><font
face="times, serif">
</font><font face="Times New Roman, serif">bracket the invocation of
each test routine.</font></p>
<p><font face="Times New Roman, serif">Note that as of AUnit 1.01 a
parameter of type AUnit.Test_Cases.Test_Case'Class has been added to
test routines. This parameter allows access to the current Test_Case
instance, so that a test routine can access per-instance (rather than
package body global) data. This can be useful when deriving one test
case from another, which introduces the need to separate data of an
instance of the parent type from that of derived types. This is
different than the normal case of a set of tests where each Test_Case
derived type almost certainly will have a singleton instance,
allowing the safe use of package body global data.</font></p>
<p>Once you have several test cases, organize them into a Suite. </p>
<h2>Suite</h2>
<p>How do you run several test cases at once? </p>
<p>As soon as you have two tests, you'll want to run them together.
You could run the tests one at a time yourself, but you would quickly
grow tired of that. Instead, AUnit provides an object, <font
face="courier, monospace">Test_Suite</font>
which runs any number of test cases together. </p>
<p>For test routines that use the same fixture (i.e. those declared
in the same package), the <font face="courier, monospace">Register_Routine
</font><font face="Times New Roman, serif">procedure is used to
collect them into the single test case.</font></p>
<p><font face="Times New Roman, serif">A single Test_Case and its
collection of routines can be executed directly in a harness like so:</font></p>
<pre style="margin-bottom: 0.2in;"><font face="Times New Roman, serif">...</font>
<font face="Times New Roman, serif">Test : PR_XXXX_XXX.Test_Case;</font>
<font face="Times New Roman, serif">Result : Aunit.Test_Results.Result;</font>
<font face="Times New Roman, serif">...</font>
<font face="times, serif">run (Test, Result);</font></pre>
<p>To create a suite of two test cases and run them together, execute: </p>
<pre>with AUnit.Test_Suites; use AUnit.Test_Suites;<br>with AUnit.Test_Runner;<br><br>-- List of tests and suites to run:<br>with Test_Case_1, Test_Case_2;<br><br>procedure Harness is<br><br><b> function Suite return Access_Test_Suite is</b>
<b> Result : Access_Test_Suite := new Test_Suite;</b>
<b> begin</b>
<b> -- You may add multiple tests or suites here:</b>
<b> Add_Test (Result, new Test_Case_1.Test_Case);</b>
<b> Add_Test (Result, new Test_Case_2.Test_Case);</b>
<b> return Result;</b>
<b> end Suite;</b>
procedure Run is new AUnit.Test_Runner (Suite);
begin
Run;
end Harness;</pre>
<h2>Composition of Suites</h2>
<p>Typically, one will want the flexibility to execute a complete set
of tests, or some subset of them. In order to facilitate this, we can
reorganize the harness so that the composition of test cases and
suites is done in a separate library function, and each composition
level can have its own harness:</p>
<pre>-- Composition function:<br>with AUnit.Test_Suites; use Aunit.Test_Suites;<br><br>-- List of tests and suites to compose:<br>with Test_Case_1;<br>with Test_Case_2;<br>function This_Suite return Access_Test_Suite is<br> Result : Access_Test_Suite := new Test_Suite;<br>begin<br> Add_Test (Result, new Test_Case_1.Test_Case);<br> Add_Test (Result, new Test_Case_2.Test_Case);<br> return Result;<br>end Suite;<br><br><br>-- More general form of harness for a given level:<br>with AUnit.Test_Runner;<br><br>-- Composition function for this level:<br>with This_Suite;<br><br>procedure Harness is<br> procedure Run is new AUnit.Test_Runner (This_Suite);<br>begin<br> Run;<br>end Harness;</pre>
<p>At a higher level, we may wish to combine two suites of units tests
that are composed with functions <font size="2"><font
face="Courier, monospace">This_Suite</font></font>
and <font size="2"><font face="Courier, monospace">That_Suite</font></font>.</p>
<p>The corresponding composition function and harness would be:</p>
<pre>-- Composition function:<br>with AUnit.Test_Suites; use Aunit.Test_Suites;<br><br>-- List of tests and suites to compose:<br>with Suite_1;<br>with Suite_2;<br>function Composition_Suite return Access_Test_Suite is<br> Result : Access_Test_Suite := new Test_Suite;<br>begin<br> Add_Test (Result, Suite_1);<br> Add_Test (Result, Suite_2);<br> return Result;<br>end Composition_Suite;<br><br><br>-- More general form of harness for a given level:<br>with AUnit.Test_Runner;<br><br>-- Composition function for this level:<br>with Composition_Suite;<br><br>procedure Harness is<br> procedure Run is new AUnit.Test_Runner (Composition_Suite);<br>begin<br> Run;<br>end Harness;</pre>
<p style="">As can be seen, this is a very flexible way of composing
test cases
into execution runs.</p>
<p style="">Note that the Aunit.Test_Runner.Run
routine has a defaulted parameter to control whether timing
information is reported. Its speficiation is:</p>
<pre>procedure Run (Timed : Boolean := True);</pre>
<p style=""><br>
<br>
</p>
<p style="margin-bottom: 0in;">By default the execution time for a
harness is reported. If you are running some number of harnesses
from a scripting language, and comparing the result to an existing
file, using <span style="font-style: normal;"><font size="2"><font
face="Courier, sans-serif">Timed
=> False</font></font></span> ensures that the output will be
identical across successful runs.</p>
<h2>Reporting</h2>
<p style="">Currently test results are reported
using a simple console reporting routine:</p>
<pre style="margin-bottom: 0.2in;"> Test_Results.Text_Reporter.Report (Result);</pre>
<p>A sample run on a set of problem reports submitted to ACT prints the
following to the console when executed:</p>
<pre>[efalis@dogen AUnit]$ ./harness<br> Total Tests Run: 10<br><br> Failed Tests: 1<br> PR 7503-008.Allocation_Test:: Bad discriminant check<br><br> Unexpected Errors: 0</pre>
<p>The switch "-v" may be used with any harness to cause the
list of successful tests to be printed along with any failures or
errors:</p>
<pre>[efalis@dogen AUnit]$ ./harness -v<br> Total Tests Run: 17<br><br> Successful Tests: 17<br> PR 7112-001: Record_Initialization<br> PR 7210-005: Test_1<br> PR 7210-005: Test_2<br> PR 7210-005: Test_3<br> PR 7210-005: Test_4<br> PR 7210-005: Test_5<br> PR 7210-005: Test_6<br> PR 7210-005: Test_A<br> PR 7210-005: Test_B<br> PR 7503-008: Allocation_Test<br> PR 7605-009: Modular_Bounds<br> PR 8010-001b: Test calculation of constant with modular sub-expression<br> PR 7522-012: Subtype not recognized in initialization<br> PR 7617-011: Test renaming in instantiation I<br> PR 7624-003: Use of multi-dimensional aggregate as generic actual parameter<br> PR 7813-010: Test -gnatc for bogus semantic error<br> PR 8010-009: Overload resolution with enumeration literals<br><br> Failed Tests: 0<br><br> Unexpected Errors: 0<br><br>Time: 0.001011000 seconds</pre>
</body>
</html>
|
