1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
|
<div class="hidden-warning"><a href="https://docs.haskellstack.org/"><img src="https://cdn.jsdelivr.net/gh/commercialhaskell/stack/doc/img/hidden-warning.svg"></a></div>
# The `stack hpc` commands
~~~text
stack hpc COMMAND
Available commands:
report Generate unified HPC coverage report from tix files
and project targets
~~~
Code coverage is a measure of the degree to which the source code of a program
is executed when a test suite is run.
[Haskell Program Coverage (HPC)](https://ku-fpg.github.io/software/hpc/) is a
code coverage tool for Haskell that is provided with GHC. Code coverage is
enabled by passing the flag `--coverage` to `stack build`.
`stack hpc` provides commands specific to HPC. Command `stack hpc` for the
available commands.
The following refers to the local HPC root directory. Its location can be
obtained by command:
~~~text
stack path --local-hpc-root
~~~
## The `stack hpc report` command
~~~text
stack hpc report [TARGET_OR_TIX] [--all] [--destdir DIR] [--open]
~~~
The `stack hpc report` command generates a report for a selection of targets and
`.tix` files.
Pass the flag `--all` for a report that uses all stored results.
Pass the flag `--open` to open the HTML report in your browser.
## The `extra-tix-files` directory
During the execution of the build, you can place additional tix files in the
`extra-tix-files` subdirectory in the local HPC root directory, in order for
them to be included in the unified report. A couple caveats:
1. These tix files must be generated by executables that are built against the
exact same library versions. Also note that, on subsequent builds with
coverage, the local HPC root directory will be recursively deleted. It
just stores the most recent coverage data.
2. These tix files will not be considered by `stack hpc report` unless listed
explicitly by file name.
## Examples
If we have three different packages with test suites, packages `A`, `B`, and
`C`, the default unified report will have coverage from all three. If we want a
unified report with just two, we can instead command:
~~~text
stack hpc report A B
~~~
This will output to the standard output stream a summary report for the combined
coverage from `A` and `B`'s test suites. It will also log the path to the HTML
for the corresponding full report.
This command also supports taking extra `.tix` files. If you've also built an
executable, against exactly the same library versions of `A`, `B`, and `C`, then
you could command the following:
~~~text
stack exec -- an-exe
stack hpc report A B C an-exe.tix
~~~
or, equivalently:
~~~text
stack exec -- an-exe
stack hpc report --all an-exe.tix
~~~
This report will consider all test results as well as the newly generated
`an-exe.tix` file.
## Usage
`stack test --coverage` is quite streamlined for the following use-case:
1. You have test suites which exercise your project packages.
2. These test suites link against your library, rather than building the
library directly. Coverage information is only given for libraries, ignoring
the modules which get compiled directly into your executable. A common case
where this doesn't happen is when your test suite and library both have
something like `hs-source-dirs: src/`. In this case, when building your test
suite you may also be compiling your library, instead of just linking
against it.
When your project has these properties, you will get the following:
1. Summary coverage reports, sent to the standard output stream in the build
output, and a log of the paths to the HTML for the corresponding full
reports.
2. A summary unified report, sent to the standard output stream, and a log of
the path to the HTML for the corresponding full report. These reports
consider the coverage on all local libraries, based on all of the tests that
were run.
3. An index of all generated HTML reports, in `index.html` in the local
HPC root directory, and a log of the path to the HTML for that index.
## Implementation details
Most users can get away with just understanding the above documentation.
However, advanced users may want to understand exactly how `--coverage` works:
1. The GHC option `-fhpc` gets passed to all project packages. This tells GHC to
output executables that track coverage information and output them to `.tix`
files. `the-exe-name.tix` files will get written to the working directory of
the executable.
When switching on this flag, it will usually cause all project packages to be
rebuilt (see issue
[#1940](https://github.com/commercialhaskell/stack/issues/1940)).
2. Before the build runs with `--coverage`, the contents of the local HPC root
directory gets deleted. This prevents old reports from getting mixed
with new reports. If you want to preserve report information from multiple
runs, copy the contents of this path to a new directory.
3. Before a test run, if a `test-name.tix` file exists in the package directory,
it will be deleted.
4. After a test run, it will expect a `test-name.tix` file to exist. This file
will then get loaded, modified, and outputted to
`pkg-name/test-name/test-name.tix` in the local HPC root directory.
The `.tix` file gets modified to remove coverage file that isn't associated
with a library. So, this means that you won't get coverage information for
the modules compiled in the `executable` or `test-suite` stanza of your Cabal
file. This makes it possible to directly union multiple `*.tix` files from
different executables (assuming they are using the exact same versions of the
project packages).
If there is enough popular demand, it may be possible in the future to give
coverage information for modules that are compiled directly into the
executable. See issue
[#1359](https://github.com/commercialhaskell/stack/issues/1359).
5. Once we have a `.tix` file for a test, we also generate a summary report and
a corresponding full report using HTML. The summary report is sent to the
standard output stream. The index of the test-specific HTML report is
available at `pkg-name/test-name/index.html` in the local HPC root directory.
6. After the build completes, if there are multiple output `*.tix` files, they
get combined into a unified report. The index of this report will be
available at `combined/all/index.html` in the local HPC root directory.
7. Finally, an index of the resulting coverage reports is generated. It links to
the individual coverage reports (one for each test-suite), as well as the
unified report. This index is available at `index.html` in the local HPC root
directory.
|
