1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
This document is targeted at cppTango developers only.
## General considerations
We have a large variety of distributions, and OS's, to check cppTango against.
In order to not create a mess in CI in the following we have listed a few
general ideas of how the tests are, and should be, organized:
- Each job MUST use one of the predefined templates:
- `job-template-with-tests` if you want to run the tests or
- `job-template-no-test-run` if you do not want to run the tests.
If you have been longer with us, the variable `RUN_TESTS` has been replaced by this
job template selection.
- Each job MUST inherit the global list of variables via `<<: *variables`
- Each job MUST use one of the predefined docker images, see [here]( https://gitlab.com/tango-controls/docker/ci/cpptango).
If none of the available ones fit your purpose, check with the rest of the
developers if one should either extend an existing one or create a new one.
Best place to discuss this, is an issue.
- If you have one of the following commands in your `.gitlab-ci.yml`, then this
indicates bad practice. Try to avoid this under all circumstances!
- `docker exec`: Should not be needed because you start already with the correct image
- `cmake`: Configuration should always be done via `ci/config.sh`
(`cmake-with-locations-and-min-versions-XXX` is an exception as it does test cmake configuration specialities.)
- `make`: Building should always be done via `ci/build.sh`
- `git clone`: The docker image should already include all required dependencies
- `apt-get`/`dnf`/`apk`: The docker image should already have all required
packages installed
- All docker images use a non-root user `tango` as default user, `sudo` is
available in all images which also have docker installed and are able to run the tests.
### Useful scripts in CI and their supported environment variables
- `ci/build.sh`:
- `MAKEFLAGS`, see `man make`, defaults to `-j <numProcessors>`
- `TANGO_MAKE_TARGET`, target to build, defaults to `all`, a list of
available targets can be outputted via `cmake --build . --target help`
(only available when using the makefile generator)
- `ci/check-abi-api-compliance-check.sh`:
- `CI_TARGET_BRANCH`, defaults to `main`
- `ci/config.sh`:
This script accepts all cmake options listed in [Install.md](./Install.md)
as environment variables and will pass them to cmake.
Additionally supported:
- `CMAKE_EXPORT_COMPILE_COMMANDS`, defaults to `off`
- `CMAKE_UNITY_BUILD`, defaults to `off`
- `CMAKE_UNITY_BUILD_BATCH_SIZE`, defaults to `0`
- `OMNIORB_VERSION` for overriding the version check
- `ci/print_coredumps.sh`:
- None
- `ci/test.sh`:
- `GITLAB_CI`, set by gitlab CI only to enable specific tweaks, don't set yourself
- `SKIP_TESTS`, regular expression passed to `ctest -E` for tests to skip
### Updating the CI docker images
- Check with the other developers if this is the right approach
- Change the docker images, see
[here](https://gitlab.com/tango-controls/docker/ci/cpptango/super) for the
super project with all of them
- Make a merge request (MR1) on the docker image project in
https://gitlab.com/tango-controls/docker/ci/cpptango. The CI will build a
docker image and push that to the registry. It is named like the branch.
- Make a merge request (MR2) against cppTango and use the branch of MR1
- Once review has passed on MR1 and MR2, get MR1 merged and tag the repo
- Update MR2 to point to the newly created tag
- Pass review again and get MR2 merged
### Running the tests
Running the tests requires docker. See
[here](https://docs.docker.com/engine/install) for install instructions.
Once you have installed docker, you can run the tests from `build/` via:
```bash
ctest
```
#### Excluding IDL5 tests
All of our test executables are run against two versions of the device servers.
The default "IDL6 servers", where device classes inherit from
`Tango::Device_6Impl` and the "IDL5 servers", where device classes inherit from
`Tango::Device_5Impl`.
There are two copies of each test registered with `ctest`: One with the
`idl5::` prefix, which runs against the IDL5 servers; and, one without the
prefix, which runs against the IDL6 servers.
To avoid running the IDL5 versions of the tests, pass `-E "idl5::"` to ctest to
exclude these tests:
```bash
ctest -E "idl5::"
```
#### Run individual tests
For debugging a single test execute the following:
```
ctest -R event::per_event --output-on-failure -V
```
Test output and device server logs are collected in `build/tests/results`.
For more details on testing with CTest, see [here](https://cmake.org/Wiki/CMake/Testing_With_CTest).
#### Setting environment up manually
The test runner automatically starts database and all required
device servers for each test. If you want to set up the environment
manually, from `build/` directory run:
```bash
source ./tests/environment/setup_database.sh # source to get TANGO_HOST
./tests/environment/setup_devices.sh
# attach the debugger or perform some additional configuration
TANGO_TEST_CASE_SKIP_FIXTURE=1 ctest -V -R ds_cache -E "idl5::"
killall DevTest FwdTest
docker stop tango_cs mysql_db
```
The above starts the latest IDL version (i.e. IDL6) of the servers and, in
general, the "idl5::" versions of the tests will fail against these servers. So,
we pass `-E "idl5::"` to `ctest` to avoid running these tests.
To start IDL5 versions of the servers run `./tests/environment/setup_devices.sh`
with `IDL_SUFFIX=_IDL5` set:
```bash
source ./tests/environment/setup_database.sh # source to get TANGO_HOST
IDL_SUFFIX=_IDL5 ./tests/environment/setup_devices.sh
# attach the debugger or perform some additional configuration
TANGO_TEST_CASE_SKIP_FIXTURE=1 ctest -V -R ds_cache -R "idl5::"
killall DevTest FwdTest
docker stop tango_cs mysql_db
```
Here the IDL6 versions of the tests will fail, so we must exclude them by
passing `-R "idl5::" to `ctest`.
#### Execucting gitlab CI jobs locally
For faster development it is possible to execute gitlab CI jobs locally on the development machine.
* Install gitlab runner, see https://docs.gitlab.com/runner/install
* `gitlab-runner exec docker --env "MAKEFLAGS=-j NUMBER_OF_CPUS" JOBNAME`
|
