1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
|
This file describes the automated testing framework for mairix
===============================================================================
Most important commands
===============================================================================
The most important commands are:
make test -- runs all tests but stops at the first test
make check -- runs all tests but does not stop at failed tests
===============================================================================
How to write new tests:
===============================================================================
Simply create a new file ending in .test-spec in the directory of this README
file. The required syntax of the test specification is documented in
README.format. Upon "make test" or "make check" the new test is automatically
included and executed in the test suite. To run the test specification
FILE.test-spec individually, issue
make test-FILE
.
===============================================================================
More detailed look at the test suite:
===============================================================================
Each of "make test" or "make check" looks for all available tests (all
.test-spec files in this directory) an runs them.
Each test is stored in a .test-spec file within this directory and to run
the test of FILE.test-spec isolated, typically
make test-FILE
is used. Thereby, the script scripts/test.sh interprets the test specification
given in FILE.test-spec and outputs all logging, databases, etc in the
directory FILE.data. Hence, if a test fails, FILE.data is the place to find
more information.
All relevant information about a test specification FILE.test-spec can be found
in the following files:
FILE.data/log contains the detailed log of what happened and in which line
of FILE.test-spec, the test failed.
FILE.data/database is the current state of the database
FILE.data/database.dump is the most up-to-date dump of the database. Note
that this dump is automatically updated whenever
searching for a message.
FILE.data/mairixrc is the current configuration file used for mairix.
FILE.data/messages contains the messages added in the current test.
FILE.data/search_result may be a file or a directory and contains the most
recent search result in the requested format
(maildir, mh, mbox). Already asserted matches are
not removed from this directory. Hence it allows to
get a picture of the previous search, even if some,
but not all matches have been asserted.
FILE.data/search_result_split is only used for searches conducted in mbox
format. For such searches, this directory
contains the individual messages of the
FILE.data/search_result mbox. The 1st message
of FILE.data/search_result is stored in
FILE.data/search_result_split/part.0
The 2nd in
FILE.data/search_result_split/part.1
and so on.
FILE.test-spec is the test specification itself
FILE.status contains the status of the most-recent invocation of this test.
Typically either "failed" or "passed".
Of the above files, all (except for FILE.test-spec) are automatically
generated upon need, when running
make test-FILE
.
Besides above test specific files, there are the following files and
directories:
messages contains the messages used in the tests
dumps contains the databases' dumps used for checking the validity of the
databases.
scripts contains the relevant scripts used in the test suite.
scripts/generate_big_message.sh is used to generate big messages, which
would unnecessarily increase the released
tarball. Those messages are however useful
when searching messages by the file size.
scripts/print_test_statistics.sh summarizes all the .status files
scripts/split_mbox.sh splits an mbox into individual messages, to allow
fine grained message assertion even for mboxen.
scripts/test.sh is the intepreter of a test specification
README is the file you are currently reading.
README.format gives the syntax of a test specification.
|
