1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
Sparse test suite
~~~~~~~~~~~~~~~~~
Sparse has a number of test cases in its validation directory. The test-suite
script aims at making automated checking of these tests possible. It works by
embedding tags in C comments in the test cases.
check-name: (mandatory)
Name of the test.
check-description: (optional)
A description of what the test checks.
check-command: (optional)
There are different kinds of tests. Some can validate the sparse
preprocessor, while others will use sparse, cgcc, or even other backends
of the library. check-command allows you to give a custom command to
run the test-case.
The '$file' string is special. It will be expanded to the file name at
run time.
It defaults to "sparse $file".
check-exit-value: (optional)
The expected exit value of check-command. It defaults to 0.
check-output-start / check-output-end (optional)
The expected output (stdout and stderr) of check-command lies between
those two tags. It defaults to no output.
check-known-to-fail (optional)
Mark the test as being known to fail.
Using test-suite
~~~~~~~~~~~~~~~~
The test-suite script is called through the check target of the Makefile. It
will try to check every test case it finds (find validation -name '*.c').
It can be called to check a single test with:
$ cd validation
$ ./test-suite single preprocessor/preprocessor1.c
TEST Preprocessor #1 (preprocessor/preprocessor1.c)
preprocessor/preprocessor1.c passed !
Writing a test
~~~~~~~~~~~~~~
test-suite comes with a format command to make a test easier to write:
test-suite format file [name [cmd]]
name:
check-name value. If no name is provided, it defaults to the file name.
cmd:
check-command value. If no cmd is provided, it defaults to
"sparse $file".
The output of the test-suite format command can be redirected into the
test case to create a test-suite formated file.
$ ./test-suite format bad-assignment.c Assignment >> bad-assignment.c
$ cat !$
cat bad-assignment.c
/*
* check-name: bad assignment
*
* check-command: sparse $file
* check-exit-value: 1
*
* check-output-start
bad-assignment.c:3:6: error: Expected ; at end of statement
bad-assignment.c:3:6: error: got \
* check-output-end
*/
You can define the check-command you want to use for the test. $file will be
extended to the file name at run time.
$ ./test-suite format validation/preprocessor2.c "Preprocessor #2" \
"sparse -E \$file" >> validation/preprocessor2.c
$ cat !$
cat validation/preprocessor2.c
/*
* This one we happen to get right.
*
* It should result in a simple
*
* a + b
*
* for a proper preprocessor.
*/
#define TWO a, b
#define UNARY(x) BINARY(x)
#define BINARY(x, y) x + y
UNARY(TWO)
/*
* check-name: Preprocessor #2
*
* check-command: sparse -E $file
* check-exit-value: 0
*
* check-output-start
a + b
* check-output-end
*/
|
