File: 10-code_coverage_of_ttcn-3_modules.adoc

package info (click to toggle)
eclipse-titan 6.5.0-1
  • links: PTS
  • area: main
  • in suites: buster
  • size: 101,128 kB
  • sloc: cpp: 259,139; ansic: 47,560; yacc: 22,554; makefile: 14,074; sh: 12,630; lex: 9,101; xml: 5,362; java: 4,849; perl: 3,784; awk: 48; php: 32; python: 13
file content (89 lines) | stat: -rw-r--r-- 6,286 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
= Code Coverage of TTCN-3 Modules

Note: the feature described here is deprecated; please use instead the coverage tool described in <<4-ttcn3_language_extensions.adoc#profiling-and-code-coverage, Profiling and code coverage>>.

The TTCN-3 compiler is able to instrument the generated C/{cpp} code for a set of TTCN-3 modules (= or files) to generate code coverage information during runtime. To enable this feature the `-K file` option needs to be used. For convenience this option is available for `ttcn3_makefilegen` as well. It’s possible to generate code coverage information only for a given set of TTCN-3 modules listed on the command line. In that case the set of files in `file` needs to be a subset of the files listed on the command line. If `file` contains a file which is not listed on the command line an error will be issued.

== Generating Code Coverage

Assuming a project with the following files: `a.ttcn`, `b.ttcn`, `c.ttcn` in parallel mode and some PTCs during runtime were created. The scenario is the following:

. Install LCOV. It’s an external tool necessary to generate HTML output. Available here: http://ltp.sourceforge.net/coverage/lcov.php or can be installed using the package manager of your OS. This step needs to be performed only once.

. Create an ASCII text file listing all the TTCN-3 modules to generate code coverage data for, one file name in a line. In this case we want to generate code coverage data for `a.ttcn` and `b.ttcn` and we’re not interested in `c.ttcn`. Our `tcov_file_list.txt` will look like: `a.ttcn, b.ttcn`

. Generate a `Makefile` using `tcov_file_list.txt`:
+
[source]
ttcn3_makefilegen -K tcov_file_list.txt -g -e mytest *.ttcn

. Run: make

. Then: `ttcn3_start ./mytest`
+
(During runtime some .tcd XML files will be generated. Namely a `tcov-<component_id>.tcd` file per PTC and one for MTC.)

. Collect and merge all .tcd files using `tcov2lcov` with default parameters:
+
[source]
tcov2lcov
+
(An `output.info` file will be generated in the current working directory. For more detailed information about `tcov2lcov` see <<command-line-syntax-of-tcov2lcov, here>>.)

. Generate HTML using LCOV’s `genhtml` and the generated `output.info` (see <<converting-code-coverage-data-from-xml-to-html, here>> for more information):
+
[source]
genhtml –no-branch-coverage –t "Titan Coverage" –legend output.info –o titan_coverage

. Open `titan_coverage/index.html` in a browser.

[[converting-code-coverage-data-from-xml-to-html]]
== Converting Code Coverage Data from XML to HTML

The `tcov2lcov` tool (binary tool) shipped with Titan and LCOV’s `genhtml` tool (Perl script) are provided to generate human readable HTML from the Titan generated .tcd XML code coverage data files. LCOV’s `genhtml` is not part of the Titan distribution and needs to be installed separately. The basic process is the following:

. Execute `tcov2lcov` to collect and merge all the .tcd files generated during test execution. One .tcd file per component. By default an `output.info` file will be generated in the current working directory, which can be processed directly by LCOV’s `genhtml`. More detailed information about `tcov2lcov` can be found in 10.3.
. Execute `genhtml` with `output.info` as its input parameter. The recommended parameters are the following:` genhtml –no-branch-coverage -t "Titan Coverage" –legend output.info -o titan_coverage`. For more detailed introduction to `genhtml` and LCOV in general please read their user manuals at http://ltp.sourceforge.net/coverage/lcov.php.

[[command-line-syntax-of-tcov2lcov]]
== Command Line Syntax of tcov2lcov

[source]
tcov2lcov [-h][-b dir][-d dir][-o file][-x xsd]

or

[source]
tcov2lcov -v

The Titan code coverage data to LCOV coverage data converter collects all valid .tcd XML files from a given directory recursively and generates a single ASCII text file suitable to be further processed by LCOV’s `genhtml` tool to produce human readable HTML output. Please note that the output format generated by `tcov2lcov` is also human readable ASCII, but its only purpose is to provide an input for LCOV’s `genhtml` tool. This format is LCOV specific and not documented here.

The following command line options are available (listed in alphabetical order):

* `-b dir` (optional)
+
Set code base directory for source files. `dir` is usually the absolute path to the directory of the source files. This path is used as a common prefix for `genhtml`. The default value is the absolute path of the current working directory.

* `-d dir` (optional)
+
`.tcd` files will be collected from `dir` recursively. By default the current working directory will be examined.

* `-h` (optional)
+
Display help.

* `-o file` (optional)
+
Set the name of the output file. The default file name is `output.info`.

* `-x xsd` (optional)
+
Path to an XSD to validate all .tcd XML files found against. By default `$TTCN3_DIR/include/tcov.xsd` is used. This XSD file is part of the Titan distribution. If any of the .tcd files doesn’t conform to the XSD an error will be given.

== Limitations

The Titan compiler implements some optimizations which can affect the accuracy of the generated code coverage information. These optimizations cannot be turned off. The compile time evaluation of constant expressions can lead to untracked lines and statements (white in LCOV’s HTML output) invisible to the code coverage information generator. These lines and statements are not counted as never-executed lines and statements (orange in LCOV’s HTML output), so the statistics are not affected, but the end result can be confusing.

External functions are not yet supported and they’re not shown in the statistics.

The Titan code coverage implementation is based on Titan’s internal location tracking mechanism enabled by the `–L` compiler flag, which must be used together with `-K`. Sometimes it can lead to a little bit confusing or strange code coverage output. E.g. multiple location objects are generated when multiple variable declarations appear on the same line in the TTCN-3 source code. In this case the code coverage output will show that the given line is executed twice. For more complex statements like `for` three location objects are generated etc.