1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
|
# Testing astcenc
The repository contains a small suite of tests which can be used to sanity
check source code changes to the compressor. It must be noted that this test
suite is relatively limited in scope and does not cover every feature or
bitrate of the standard.
# Required software
Running the tests requires Python 3.7 to be installed on the host machine, and
an `astcenc-avx2` release build to have been previously compiled and installed
into an directory called `astcenc` in the root of the git checkout. This
can be achieved by configuring the CMake build using the install prefix
`-DCMAKE_INSTALL_PREFIX=../` and then running a build with the `install` build
target.
# Running C++ unit tests
We support a small (but growing) number of C++ unit tests, which are written
using the `googletest` framework and integrated in the CMake "CTest" test
framework.
To build unit tests pull the `googletest` git submodule and add `-DUNITTEST=ON`
to the CMake command line when configuring.
To run unit tests use the CMake `ctest` utility from your build directory after
you have built the tests.
```shell
cd build
ctest --verbose
```
# Running command line tests
To run the command line tests, which aim to get coverage of the command line
options and core codec stability without testing the compression quality
itself, run the command line:
python3 -m unittest discover -s Test -p astc_test*.py -v
# Running image tests
To run the image test suite run the following command from the root directory
of the repository:
python3 ./Test/astc_test_image.py
This will run though a series of image compression tests, comparing the image
PSNR against a set of reference results from the last stable baseline. The test
will fail if any reduction in PSNR above a set threshold is detected. Note that
performance information is reported, but regressions will not flag a failure.
For debug purposes, all decompressed test output images and result CSV files
are stored in the `TestOutput` directory, using the same test set structure as
the `Test/Images` folder.
## Test selection
The runner supports a number of options to filter down what is run, enabling
developers to focus local testing on the parts of the code they are working on.
* `--encoder` selects which encoder to run. By default the `avx2` encoder is
selected. Note that some out-of-tree reference encoders (older encoders, and
some third-party encoders) are supported for comparison purposes. These will
not work without the binaries being manually provided; they are not
distributed here.
* `--test-set` selects which image set to run. By default the `Small` image
test set is selected, which aims to provide basic coverage of many different
color formats and color profiles.
* `--block-size` selects which block size to run. By default a range of
block sizes (2D and 3D) are used.
* `--color-profile` selects which color profiles from the standard should be
used (LDR, LDR sRGB, or HDR) to select images. By default all are selected.
* `--color-format` selects which color formats should be used (L, XY, RGB,
RGBA) to select images. By default all are selected.
## Performance tests
To provide less noisy performance results the test suite supports compressing
each image multiple times and returning the best measured performance. To
enable this mode use the following options:
* `--repeats <M>` : Run M test compression passes which are timed.
**Note:** The reference CSV contains performance results measured on an Intel
Core i5 9600K running at 4.3GHz, running each test 5 times.
## Updating reference data
The reference PSNR and performance scores are stored in CSVs committed to the
repository. This data is created by running the tests using the last stable
release on a standard test machine we use for performance testing builds.
It can be useful for developers to rebuild the reference results for their
local machine, in particular for measuring performance improvements. To build
new reference CSVs, download the current reference `astcenc` binary (1.7) from
GitHub for your host OS and place it in to the `./Binaries/1.7/` directory.
Once this is done, run the command:
python3 ./Test/astc_test_image.py --encoder 1.7 --test-set all --repeats 5
... to regenerate the reference CSV files.
**WARNING:** This can take some hours to complete, and it is best done when the
test suite gets exclusive use of the machine to avoid other processing slowing
down the compression and disturbing the performance data. It is recommended to
shutdown or disable any background applications that are running.
## Valgrind memcheck
It is always worth running the Valgrind memcheck tool to validate that we have
not introduced any obvious memory errors. Build a release build with symbols
information with `-DCMAKE_BUILD_TYPE=RelWithDebInfo` and then run:
valgrind --tool=memcheck --track-origins=yes <command>
- - -
_Copyright © 2019-2022, Arm Limited and contributors. All rights reserved._
|
