1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta name="GENERATOR" content="Microsoft FrontPage 4.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<title>Test Tools Library</title>
</head>
<body bgcolor="#FFFFFF" text="#000000">
<h1><img border="0" src="../../../c++boost.gif" width="277" height="86">Boost
Test Library: Test Tools</h1>
<p><a href="#Introduction">Introduction</a><br>
<a href="#Introduction">Example</a><br>
<a href="#Sample">Test/Sample Programs</a><br>
<a href="#Header Imp">Header Implementation Option</a><br>
<a href="#Rationale">Rationale</a><br>
<a href="#Design">Design<br>
<br>
</a>Also see: <a href="execution_tools.htm">Execution Tools</a></p>
<h2><a name="Introduction">Introduction</a></h2>
<p>The Boost Test Library's Test Tools supply several components to ease creation and maintenance of
test programs.</p>
<ul>
<li>Header <a href="../../../boost/test/test_tools.hpp"> <b> boost/test/test_tools.hpp</b></a> supplies macros and
function declarations to simplify
detecting and reporting errors. While the functions can be called
directly, the usual usage is via the convenience macros:<pre> BOOST_TEST( bool ); // continue after a failure
BOOST_CRITICAL_TEST( bool ); // stop after a failure
BOOST_ERROR( char* ); // continue after reporting error
BOOST_CRITICAL_ERROR( char* ); // stop after reporting error</pre>
</li>
</ul>
<ul>
<li>A <b>cpp_main()</b> function is provided which calls a user-supplied <b>test_main()</b>
function. The library supplied <b>cpp_main()</b> relieves users from messy
error detection and reporting duties. The default implementation is layered
on top of <a href="execution_tools.htm">Execution Tools</a> which supplies the program's <b>main()</b> function,
and handles errors not specific to the Boost Test Tools.
</li>
</ul>
<p>Boost Test Tools are intended for fairly simple test programs. <a href="execution_tools.htm">Execution Tools</a> may be more suitable for production (non-test) programs.
Unit Test
Tools may be more suitable for complex test
programs. </p>
<p>The boost/test_tools.hpp macros and functions are intended for
test code rather than library or applications code, where throwing exceptions,
assert(), or BOOST_STATIC_ASSERT() may be more suitable ways to
detect and report errors.</p>
<h2><a name="Example">Example</a></h2>
<p>The example program shows four different ways to detect and report an error in the
add() function.</p>
<blockquote>
<pre>#include <<a href="../../../boost/test/test_tools.hpp">boost/test/test_tools.hpp</a>> // see "<a href="#Header Imp">Header Implementation Option</a>"
int add( int i, int j ) { return i+j; }
int test_main( int, char *[] ) // note the name!
{
// six ways to detect and report the same error:
BOOST_TEST( add(2,2) == 4 ); // #1 continues on error
BOOST_CRITICAL_TEST( add(2,2) == 4 ); // #2 throws on error
if ( add(2,2) != 4 )
BOOST_ERROR( "Ouch..."); // #3 continues on error
if ( add(2,2) != 4 )
BOOST_CRITICAL_ERROR("Ouch..."); // #4 throws on error
if ( add(2,2) != 4 ) throw "Oops..."; // #5 throws on error
return add(2,2) == 4 ? 0 : 1; // #6 returns error code
}</pre>
</blockquote>
<p><b>Approach #1</b> uses the BOOST_TEST macro, which displays an error message
on std::cout that includes the expression that failed, the source file name, and
the source file line number. It also increments an error count. At program termination,
the error count will be displayed automatically by the Boost Test Tools supplied cpp_main() function..</p>
<p><b>Approach #2</b> using the BOOST_CRITICAL_TEST macro, is similar to #1, except that after
displaying the error, an exception is thrown, to be caught by the library
supplied cpp_main()
function. This approach is suitable when writing a
explicit test program, and the error would be so severe as to make further
testing impractical. BOOST_CRITICAL_TEST differs from the C++ Standard Library's
assert() macro in that it is always generated, and channels error detection into
the uniform <a href="../../../boost/test/test_main.cpp">boost/test/test_main.cpp</a> reporting procedure.</p>
<p><b>Approaches #3 and #4</b> are similar to #1 and #2 respectively, except
that the error detection is coded separately. This is most useful when the
specific condition being tested is not indicative of the reason for failure.</p>
<p><b>Approach #5</b> throws an exception, which will be caught and reported by the
main() function supplied by either the Execution Tools or Test Tools. This approach is suitable for both production and test code, in
libraries or not. The error message displayed when the exception is caught
will be most meaningful if the exception is derived from std::exception, or is a
char * or std::string.</p>
<p><b>Approach #6</b> uses a return value to inform the caller of the error. This approach is particularly suitable for
integrating existing test code with the test tools library. Although it
works fine with the Boost Program Execution or Test libraries, and is very
useful for running existing code under them, most C++ experts prefer using
exceptions for error reporting.</p>
<h2>Test/<a name="Sample">Sample</a> and Test Programs</h2>
<blockquote>
<pre><a href="../example/test_tools_example.cpp">test_tools_example.cpp</a>
<a href="../test/test_tools_fail1.cpp">test_tools_fail1.cpp</a>
<a href="../test/test_tools_fail2.cpp">test_tools_fail2.cpp</a>
<a href="../test/test_tools_fail3.cpp">test_tools_fail3.cpp</a>
<a href="../test/test_tools_fail4.cpp">test_tools_fail4.cpp</a></pre>
</blockquote>
<h2><a name="Header Imp">Header Imp</a>lementation Option</h2>
<p>The Boost Execution Tools <b>main()</b> function and Boost Test Tools supplied
<b> cpp_main()</b> functions may be stored in an object library
and linked into the final program just like any other supplied function.
If this is not convenient, they can be compiled directly into the user's program
by defining a macro:</p>
<blockquote>
<pre>#define BOOST_INCLUDE_MAIN
#include <a href="../../../boost/test/test_tools.hpp"><boost/test/test_tools.hpp></a></pre>
</blockquote>
<p>Although this usage is unusual, it is quite useful in some test environments
where it may be difficult or undesirable to link to an object library.</p>
<h2><a name="Rationale">Rationale</a></h2>
<p>How should a test program report errors?</p>
<p>Displaying an error message is an obvious possibility:</p>
<blockquote>
<pre>if ( something_bad_detected )
std::cout << "something bad has been detected" << std::endl;</pre>
</blockquote>
<p>But that requires inspection of the program's output after each run to
determine if an error occurred. Since test programs are often run as part
of a regression test suite, human inspection of output to detect error messages
is too time consuming and unreliable. Test frameworks like GNU/expect can do the
inspections automatically, but are overly complex for simple testing.</p>
<p>A better simple way to report errors is for the test program to return
EXIT_SUCCESS (normally 0) if the test program completes satisfactorily, and
EXIT_FAILURE
if an error is detected. This allows a simple regression test script to
automatically and unambiguous detect success or failure. Further
appropriate actions such as creating an HTML table or emailing an alert can be
taken by the script, and can be modified as desired without having to change the
actual C++ test programs.</p>
<p>A testing protocol based on a policy of test programs returning EXIT_SUCCESS or
EXIT_FAILURE does not require any supporting tools; the C++ language and standard
library are sufficient. The programmer must remember, however, to catch
all exceptions and convert them to program exits with non-zero return codes. The
programmer must also remember to not use the standard library assert() macro for
test code, because on some systems it results in undesirable side effects like a message
requiring manual intervention.</p>
<p>The Test Library automates those tasks, yet can be ignored by
programmers who prefer to implement the zero return testing protocol themselves.</p>
<h2><a name="Design">Design</a></h2>
<p>The <a href="test_lib_design.htm">Boost Test Library Design</a>
document describes the relationship between the Boost Test Library components.</p>
<hr>
<p> Copyright Beman Dawes 2000</p>
<p>Revised <!--webbot bot="Timestamp" S-Type="EDITED"
S-Format="%d %b %Y" startspan -->28 Feb 2001<!--webbot bot="Timestamp" endspan i-checksum="14390" -->
</p>
</body>
</html>
|
