1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
|
# Bisect_ppx advanced usage
<br>
#### Table of contents
- [Exhaustiveness checking](#Exhaustiveness)
- [Excluding generated files from coverage](#Excluding)
- [Environment variables](#EnvironmentVariables)
- [Naming the output files](#OutFiles)
- [Logging](#Logging)
- [SIGTERM handling](#SIGTERM)
- [Setting at compile time](#CompileTime)
<br>
<a id="Exhaustiveness"></a>
## Exhaustiveness checking
It is easy to accidentally fail to preprocess part of your project, by leaving
out `(preprocess (pps bisect_ppx))` in one of its subdirectories. If this
happens, and goes unnoticed, you may see misleading coverage statistics, because
Bisect_ppx will simply not be aware of the missing files at all. It will not
report them as either covered or not covered — they will just not be
present in the report.
To sanity-check this, you can pass the `--expect` option to `bisect-ppx-report`.
For example,
```
bisect-ppx-report html --expect src/
```
`bisect-ppx-report` will then recursively scan `src/` for any `.ml` and `.re`
files, and check that all of them were included in the report.
You may have a subdirectory of `src/` that should not be included. You can
exclude it from the recursive scan with `--do-not-expect`:
```
bisect-ppx-report html --expect src/ --do-not-expect src/build_tool/
```
You can also specify individual files with `--expect` and `--do-not-expect` by
omitting the trailing path separator:
```
bisect-ppx-report html --expect src/ --do-not-expect src/build_tool.ml
```
<br>
<a id="Excluding"></a>
## Excluding generated files from coverage
Whole files can be excluded by placing `[@@@coverage exclude_file]` anywhere in
their top-level module.
If you have generated code that you cannot easily place an attribute into, nor
is it easy to avoid preprocessing it, you can pass the `--exclusions` option to
the Bisect_ppx preprocessor:
```
(instrumentation
(backend bisect_ppx --exclusions bisect.exclude)
(deps bisect.exclude))
```
This requires Dune 2.9 or later.
Note that the paths to `bisect.exclude` might be different between the
`preprocess` and `preprocessor_deps` stanzas, because `pps bisect_ppx` looks for
the file relative to the root directory of your project, while
`preprocessor_deps` looks in the same directory that the `dune` file is in.
Here is what the `bisect.exclude` file can look like:
```
(* OCaml-style comments are okay. *)
(* Exclude the file "foo.ml": *)
file "foo.ml"
(* Exclude all files whose names start with "test_": *)
file regexp "test_.*"
(* Exclude the top-level values "foo" and "bar" in "baz.ml": *)
file "baz.ml" [
name "foo"
name "bar"
]
(* Exclude all top-level values whose names begin with "dbg_" in all
files in "src/": *)
file regexp "src/.*" [ regexp "dbg_.*" ]
```
All regular expressions are in the syntax of the [`Str`][Str] module.
<br>
<a id="EnvironmentVariables"></a>
## Environment variables
A program instrumented by Bisect_ppx writes `.coverage` files, which contain the
numbers of times various points in the program's code were visited during
execution. Two environment variables are available to control the writing of
these files.
<a id="OutFiles"></a>
#### Naming the output files
By default, the counts files are called `bisect0001.coverage`,
`bisect0002.coverage`, etc. The prefix `bisect` can be changed by setting the
environment variable `BISECT_FILE`. In particular, you can set it to something
like `_coverage/bisect` to put the counts files in a different directory, in
this example `_coverage/`.
`BISECT_FILE` can also be used to control the prefix programmatically. For
example, the following code bases the prefix on the program name, and puts the
`.coverage` files into the system temporary directory:
let () =
let (//) = Filename.concat in
let tmpdir = Filename.get_temp_dir_name () in
Unix.putenv "BISECT_FILE"
(tmpdir // Printf.sprintf "bisect-%s-" Sys.executable_name)
<a id="Logging"></a>
#### Logging
If the instrumented program fails to write an `.coverage` file, it will log a
message. By default, these messages go to a file `bisect.log`. `BISECT_SILENT`
can be set to `YES` to turn off logging completely. Alternatively, it can be set
to another filename, or to `ERR` in order to log to `STDERR`.
<a id="SIGTERM"></a>
#### SIGTERM handling
If `BISECT_SIGTERM` is set to `yes`, the Bisect runtime will install a SIGTERM
handler that will write `.coverage` files and exit. This is useful if you have a
large number of test programs, and they can be killed by another process.
<a id="CompileTime"></a>
#### Setting at compile time
If your testing environment doesn't allow you to easily specify these
environment variables at testing time, you can specify default values for them
at compile time by passing the `--bisect-file`, `--bisect-silent`, and/or
`--bisect-sigterm` options to the Bisect_ppx instrumenter:
```
(instrumentation
(backend bisect_ppx
--bisect-file /tmp/mycoverage
--bisect-silent /tmp/coverage.log
--bisect-sigterm))
```
If different values are specified in different `dune` files for code that is
then linked into one binary, one set of values is chosen arbitrarily.
Passing arguments to Bisect in this way requires Dune version 2.8.0 or higher.
[Str]: http://caml.inria.fr/pub/docs/manual-ocaml/libref/Str.html#VALregexp
|
