1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
|
HOWTO handle external (blackbox) tests
======================================
Quick and dirty howto on how we do "blackbox" tests, the kind of tests
that are run against the mtn executable as a whole, and live in the
test/func/ directory. Very early draft. Feel free to improve.
See also the "tester.txt" and "testsuite.txt" files, which
respectively describe the basic testing framework's API, and the
monotone-specific addons.
Running tests:
--------------
- Starting in the monotone main dir. After having './configure'd monotone you
can do 'make testers' to create the needed test infrastructure, such as
the 'tester' program that runs the tests, the 'unit_tester' binary for the
unit tests and some auxiliary scripts.
- Execute './run_func_tests' to run the functional tests, './run_unit_tests'
for the C++ unit tests and './run_tester_tests' for tests of the test framework
itself. Each script accepts the following options and arguments:
'-l' : lists the names and numbers of all available tests in the suite
'<n>' : runs only test number n from the suite, negative n counts from the end
'foobar': runs tests with "foobar" in the name (it's actually a regex)
'-d' : keeps the work files for post-test debugging in test/work;
summary of test logs are saved under test/work/{func,unit,tester}.log,
detailed test logs are available under
test/work/{func,unit,tester}/<testname>/tester.log
Creating new functional tests:
------------------------------
- Copy and paste is your friend :)
- Make a new directory test/func/<testname>
- Create a new file __driver__.lua inside this directory
- Sometimes you need to canonicalize things
Template for a functional test:
-------------------------------
-- Initialize our workspace
mtn_setup()
-- run monotone
-- we want return value 0
-- we want to save the standard output in file "stdout"
-- we want to ignore the standard error
check(mtn("ls", "unknown"), 0, true, false)
-- we want "mtn foobar" to work, but since we know it doesn't
-- we tell the test program that this is expected to fail
xfail_if(true, mtn("foobar"), 0)
Debugging
---------
compile with -O0
run the offending command under gdb with something like gdb --args mtn ...
put a breakpoint on 'sanity::invar<TAB> (note the leading single quote)
|
