1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<!--#include file="Head.html"-->
<body bgcolor="#FFFFFF" leftmargin=0 topmargin=0 text="black" link="#0000FF"
vlink="#000000" alink="#000000">
<!--#include file="Table.html"-->
<tr>
<!--#include file="SideBar.html"-->
<td width="550" valign="top" bgcolor="#FFFFFF">
<div align="left">
<a name="general"></a><font size=5>Defining Tests</font>
<p>
Tests are specified to Dart using the ADD_TEST() command.
<pre>
ADD_TEST(TestIdentifier ExecutableName [Arguments])
</pre>
Each test must have a unique "TestIdentifier". The TestIdentifier is
a string presented on the Dartboard. The ExecutableName can be a
program built by your system project or may be any other executable on
our computer (for instance Tcl, Python, etc.). Tests may have
additional arguments.
<p>
The ADD_TEST() command provides a lot of flexibility. Each test
could be a separate executable or each test could use the same
executable but provide different arguments. The only restriction
is the TestIdentifiers be unique.
<p>
If using CMake, the ADD_TEST() command is put in a CMakeList.txt file
in the project source tree. If not using CMake, the ADD_TEST()
command is put in a DartTestfile.txt file in the project build
tree. (Under CMake, the ADD_TEST() commands in CMakeLists.txt
files in the source tree are automatically written into
DartTestfile.txt files in the build tree).
<p>
<a name="general"></a><font size=5>Test Status</font>
<p>The exit status of a test is used to determine whether a test passes
or fails.
<p>
<a name="general"></a><font size=5>Test Results</font>
<p>A test's standard output and standard error are captured and
presented on the DartBoard. More elaborate presentations can created
by embedding formating commands for Dart in the standard output of
your test. There are two formatting commands, DartMeasurement and
DartMeasurementFile, which can be placed like XML tags in the standard
output of a test
<pre>
<DartMeasurement name="foo" type="numeric/double" <br> encoding="none" compression="none">
data
</DartMeasurement>
<DartMeasurementFile name="foo" type="image/jpeg">
filename
</DartMeasrementFile>
</pre>
<ul>
<li><b>DartMeasurement - </b> is used to display standard data types like numbers and strings.
<li><b>DartMeasurementFile - </b> is used to display bulk data items like images. The data in the specified file will be compressed and encoded and placed in the Test.xml file. Once on the Dart server, the file will be extracted, decoded, and uncompressed.
</ul>
The "type" can be one of the following:
<ul>
<li> text/plain
<li> text/string
<li> text/html
<li> text/xml
<li> link/url
<li> link/image
<li> numeric/integer
<li> numeric/float
<li> numeric/double
<li> numeric/boolean
<li> image/png
<li> image/jpeg
</ul>
The "encoding" can be:
<ul>
<li> none (default, does not have to be specified)
<li> base64
</ul>
The "compression" can be:
<ul>
<li> none (default, does not have to be specified)
<li> gzip
</ul>
The "encoding" and "compression" attributes specify how the data
within the tag is encoded or compressed. In most cases, the encoding
and compression will be "none".
<p> The mechanims used to extract DartMeasurement and
DartMeasurementFile tags from the standard output of a test places a
few restrictions on the contents of the tags. The contents of a
DartMeasurement tag cannot contain a "<" character.
<p>
DartMeasurements will be presented in a table. One column will
present the "name"'s of the measurements, a second column will present
the "values" (or image). After the DartMeasurements, the test's
standard out (with all DartMeasurement tags stripped out) and standard
error buffers will be displayed.
<p>
Along with DartMeasurements specified by the test, a standard set of DartMeasurements are also displayed
<ul>
<li><b>Execution Time - </b> Amount of time it took Dart to run the test. This includes the time start the executable.
<li><b>Completion Status - </b> How the test exited. This can be "Completed" or "TimeOut".
<li><b>Exit Code - </b> Displayed only if a test failed. This is either "CHILDSTATUS" indicating the test failed and outputted an non-zero exit value or "CHILDKILLED" indicating the test was aborted.
<li><b>Exit Explanation - </b> Displayed only if the exit code is "CHILDKILLED". This measurement will convey whether the program crashed (segmentation violation) or was killed by the user.
<li><b>Exit Value - </b> The exit value returned by the test. If a test completed, this will be numeric value return by the test's call to exit(). If the test crashed, this will be a code indicating the type of runtime error (for instance SIGSEGV).
</div>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>
|
