1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
|
.. _sec_runningtests:
Running PETSc Tests
-------------------
Quick start with the tests
~~~~~~~~~~~~~~~~~~~~~~~~~~
For testing builds, the general invocation from the ``PETSC_DIR`` is:
.. code-block:: bash
make [-j <n>] -f gmakefile test PETSC_ARCH=<PETSC_ARCH>
For testing ``./configure`` that used the ``--prefix`` option, the
general invocation from the installation (prefix) directory is:
.. code-block:: bash
make [-j <n>] -f share/petsc/examples/gmakefile test
For a full list of options, use
.. code-block:: bash
make -f gmakefile help-test
Understanding test output and more information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As discussed in :any:`sec_runningtests`, users should set
``PETSC_DIR`` and ``PETSC_ARCH`` before running the tests, or can
provide them on the command line as below.
To check if the libraries are working do:
.. code-block:: bash
make PETSC_DIR=<PETSC_DIR> PETSC_ARCH=<PETSC_ARCH> test
A comprehensive set of tests can be run with
.. code-block:: bash
make PETSC_DIR=<PETSC_DIR> PETSC_ARCH=<PETSC_ARCH> alltests
or
.. code-block:: bash
make [-j <n>] -f gmakefile test PETSC_ARCH=<PETSC_ARCH>
Depending on your machine’s configuration running the full test suite
(above) can take from a few minutes to a couple hours. Note that
currently we do not have a mechanism for automatically running the test
suite on batch computer systems except to obtain an interactive compute
node (via the batch system) and run the tests on that node (this assumes
that the compilers are available on the interactive compute nodes.
The test reporting system classifies them according to the Test Anywhere
Protocal (TAP) [11]_. In brief, the categories are
- ``ok`` The test passed.
- ``not ok`` The test failed.
- ``not ok #SKIP`` The test was skipped, usually because build
requirements were not met (for example, an external solver library
was required, but PETSc was not ``./configure`` for that library.)
compiled against it).
- ``ok #TODO`` The test is under development by the developers.
The tests are a series of shell scripts, generated by information
contained within the test source file, that are invoked by the makefile
system. The tests are run in ``${PETSC_DIR}/${PETSC_ARCH}/tests`` with
the same directory as the source tree underneath. For testing installs,
the default location is ``${PREFIX_DIR}/tests`` but this can be changed
with the ``TESTDIR`` location. (See :any:`sec_directory`). A
label is used to denote where it can be found within the source tree.
For example, test ``vec_vec_tutorials-ex6``, which can be run e.g. with
.. code-block:: bash
make -f gmakefile test search='vec_vec_tutorials-ex6'
(see the discussion of ``search`` below), denotes the shell script:
.. code-block:: bash
${PETSC_DIR}/${PETSC_ARCH}/tests/vec/vec/tutorials/runex6.sh
These shell scripts can be run independently in those directories, and
take arguments to show the commands run, change arguments, etc. Use the
``-h`` option to the shell script to see these options.
Often, you want to run only a subset of tests. Our makefiles use
``gmake``\ ’s wildcard syntax. In this syntax, ``%`` is a wild card
character and is passed in using the ``search`` argument. Two wildcard
characters cannot be used in a search, so the ``searchin`` argument is
used to provide the equivalent of ``%pattern%`` search. The default
examples have default arguments, and we often wish to test examples with
various arguments; we use the ``argsearch`` argument for these searches.
Like ``searchin``, it does not use wildcards, but rather whether the
string is within the arguments.
Some examples are:
.. code-block:: bash
make -f gmakefile test search='ts%' # Run all TS examples
make -f gmakefile test searchin='tutorials' # Run all tutorials
make -f gmakefile test search='ts%' searchin='tutorials' # Run all TS tutorials
make -f gmakefile test argsearch='cuda' # Run examples with cuda in arguments
make -f gmakefile test test-fail='1'
make -f gmakefile test query='requires' queryval='*MPI_PROCESS_SHARED_MEMORY*'
It is useful before invoking the tests to see what targets will be run.
The ``print-test`` target helps with this:
.. code-block:: bash
make -f gmakefile print-test argsearch='cuda'
To see all of the test targets which would be run, this command can be
used:
.. code-block:: bash
make -f gmakefile print-test
For testing in install directories, some examples are:
.. code-block:: bash
cd ${PREFIX_DIR}; make -f share/petsc/examples/gmakefile.test test TESTDIR=mytests
or
.. code-block:: bash
cd ${PREFIX_DIR}/share/petsc/examples; make -f gmakefile test TESTDIR=$PWD/mytests
where the latter is needed to make have it run in the local directory
instead of ``$PREFIX_DIR``.
To learn more about the test system details, one can look at the
`the PETSc developers documentation <https://docs.petsc.org/en/latest/developers>`__.
.. [11]
See https://testanything.org/tap-specification.html
|
