File: README.test_maintainers.txt

package info (click to toggle)
darcs 2.0.2-3
  • links: PTS
  • area: main
  • in suites: lenny
  • size: 6,400 kB
  • ctags: 1,048
  • sloc: haskell: 24,937; perl: 9,736; sh: 3,369; ansic: 1,913; makefile: 17; xml: 14
file content (133 lines) | stat: -rw-r--r-- 4,428 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

The following is provided as a reference for those interested in understanding
how the test suite works, and how to add and maintain tests.

Overview of how tests are run
=============================

Running tests starts in "GNUmakefile" which has a "make test" target.

This causes all the functional tests in "tests/" to be run against repositories
in the old, hashed, and darcs2 formats. Additionally, scripts in "bugs/" are
also run on all three types of repository, with the expectation that they
currently fail. 

Running fewer tests
-----------------------------

Because "make test" can take a long time to run, it's useful to run fewer tests
at once.  To help with that, the following specific make targets are also
available through GNUmakefile:

  make test_shell
  make test_shell_format2
  make test_shell_hashed
  make test_perl
  make test_perl_format2
  make test_perl_hashed
  make bugs_shell
  make bugs_shell_format2
  make bugs_shell_hashed
  make bugs_perl
  make bugs_perl_format2
  make bugs_perl_hashed


Running a single test
---------------------

  ./bin/prove script.pl
  perl shell_harness script.sh

or alternatively to run a couple of tests

  echo test-name.sh > tests/tests_to_run
  echo test-name2.sh >> tests/tests_to_run
  make test_shell # or whichever you like!


Overview of types of tests
==========================

Darcs has tests in three formats: Functional tests that test the darcs binary
are written in Shell and Perl. Unit tests are  written in Haskell and
directly test Haskell functions. These can be run via "make unit", but are
not otherwise documented here. 

Shell tests
---------------------------

Shell tests and are useful because they are easy to create from a copy/paste of
actual shell commands executed. They are considered successful if no bad exit
codes are returned from the commands in the script. 

Perl scripts
---------------------------

The Perl scripts are useful because of the featureful testing framework that
Perl provides, and the potential they offer to create more portable tests. 

Right now both kinds of tests are equally accepted by the project, but
shell tests are preferred by the main developers, as they are generally
more useful and easier to debug.


Tips for writing tests
=======================

- Avoid including a repo format type to "darcs init"
  This insures that all three repo formats will be tested.
  However, if you know that the test only passes under some
  repo formats, *do* explicity include a format option to "darcs init". 


Tips for writing Shell tests
-------------------------------

- Copy an existing test, which will already have the following properties:

- Simply call darcs using "darcs" as you would in the shell.  It is the
  responsibility of the test harness to ensure that the darcs we are
  testing is first in the path.

- Always use bash explicitly--this improves the portability of our tests.

- Always add this near the top of the script:

   set -ev 

 The "v" causes the contents of the script to be printed as part of the run,
 which is helpful for debugging.  The "e" causes the script to exit as soon as
 their is an error. 


Tips for writing Perl tests
-------------------------------

- Be aware of the testing modules that are available. See the docs for each of
  these, in order of importance. They are in the tests/lib/perl directory.
  (Use perldoc file.pm to see the docs). 

   Test::More
   Test::Darcs
   Shell::Command
   File::Slurp

- Avoid direct system calls. It will be tempting to use backticks to drop in direct shell
  commands. By writing as much of the tests as possible in pure Perl, they will remain
  more portable. The modules above cover the common cases. 

- ./bin/prove has many options for running the Perl tests. 


The future of testing darcs
=================================

We would prefer to have all tests written in bash, as these tests are both
easier to write (since you ordinarily run darcs from a shell in any case),
and are more helpful when trying to track down bugs, for the same reason:
it is easy to run bash tests by hand in a shell, examining each
intermediate state.  For this same reason, try to avoid defining functions
in your test scripts.  This makes them harder to run and generally harder
to use.  There are certainly cases where it is appropriate to define a
function, but please do not do this just to avoid a little duplication.