File: howto.html

package info (click to toggle)
libtut 0.0.20070706-2
links: PTS
area: main
in suites: buster
size: 1,148 kB
ctags: 607
sloc: cpp: 3,588; xml: 137; makefile: 61; ansic: 9
file content (799 lines) | stat: -rw-r--r-- 24,803 bytes
parent folder | download | duplicates (2)
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>TUT How-To - TUT: C++ Unit Test Framework</title>
<meta name="keywords" content="C++ test-driven unit tests TUT framework STL">
<meta name="description" content="TUT How-To - minimum steps to make TUT work for you">
<style type="text/css">
<!--

body, p, td
{
	scrollbar-base-color: #eeeeee;
	scrollbar-track-color: #ffffff;
	scrollbar-arrow-color: #cccccc;
	scrollbar-shadow-color: #cccccc;
	scrollbar-highlight-color: #cccccc;
	scrollbar-darkshadow-color: #ffffff;
	scrollbar-3dlight-color: #ffffff;
	
	/*
	scrollbar-base-color: #cccccc;
	scrollbar-track-color: #ffffff;
	scrollbar-arrow-color: #eeeeee;
	*/
	
	font-family : Verdana, Arial, Helvetica, sans-serif;
	font-size : 14px;
	color: #000000;
}

pre
{
	font-family : Courier, sans-serif;
	font-size : 12px;
}

strong, b
{
	font-weight : bold;
}

ul
{
	list-style-type : square;
}

.header
{
	font-size : 24px;
	font-weight : bold;
}

.subheader
{
	font-size : 16px;
	font-weight : bold;
}

.logo
{
	font-size : 24px;
	font-weight : bold;
}

.question
{
	font-size : 16px;
	font-weight : bold;
	color : #000000;
}

a
{
	font-weight : bold;
	color : #ff9900;
	text-decoration : none;
}

a:hover
{
	font-weight : bold;
	color : #ff9900;
	text-decoration : underline;
}

a:visited
{
	font-weight : bold;
	color : #ff9900;
}

a.menu
{
	font-weight : bold;
	color : #ff9900;
	text-decoration : none;
}

a.menu:hover
{
	font-weight : bold;
	color : #ff9900;
	text-decoration : underline;
}

a.menu:visited
{
	font-weight : bold;
	color : #ff9900;
}

//-->
</style>
</head>

<body text="#000000" link="#ff9900" alink="#ffcc00" vlink="#ff9900" bgcolor="#ffffff" leftmargin=0 topmargin=0 marginheight=0 marginwidth=0>

<img src="/_img/pixel.gif" alt="" width=1 height=20><br>
<table width="100%" cellspacing=0 cellpadding=0>
<tr>
<td width=20 align=left valign=top>
<img src="/_img/pixel.gif" alt="" width=20 height=1><br>
</td>
<td width="100%" align=left valign=top>
<!-- home table >> -->
<table width="100%" cellspacing=0 cellpadding=0>
<tr>
<td width="100%" align=center valign=center bgcolor="#cccccc">
<!-- >> -->
<table width="100%" cellspacing=1 cellpadding=10>
<tr>
<td width="100%" align=left valign=top bgcolor="#eeeeee">
<p class="logo"><a href="/">TUT: C++ Unit Test Framework</a></p>
</td>
</tr>
</table>
<!-- << -->
</td>
</tr>
</table>
<!-- home table << -->
</td>
<td width=20 align=left valign=top>
<img src="/_img/pixel.gif" alt="" width=20 height=1><br>
</td>
</tr>
</table>

<table border=0 width="100%" cellspacing=20 cellpadding=0>
<tr>
<td width="30%" align=left valign=top>
<!-- menu table >> -->
<table width="100%" cellspacing=0 cellpadding=0>
<tr>
<td width="100%" align=center valign=center bgcolor="#cccccc">
<!-- >> -->
<table width="100%" cellspacing=1 cellpadding=10>
<tr>
<td width="100%" align=left valign=top bgcolor="#eeeeee">
<p class="subheader">Documentation</p>
<ul>
<li><p><strong>TUT How-To</strong><br>minimum steps to make TUT work for you</p></li>
<li><p><a href="/design/" class="menu">TUT Design</a><br>what's hidden under the hood of tut.h, and why things are such as they
		are</p></li>
<li><p><a href="/example/" class="menu">TUT Usage Example</a><br>it's better to see once...</p></li>
<li><p><a href="/whole/" class="menu">TUT As Is</a><br>complete source of TUT</p></li>
</ul>
<p class="subheader">Distribution</p>
<ul>
<li><p><a href="/copyright/" class="menu">The TUT License</a><br>almost no restrictions to use</p></li>
<li><p><a href="/download/" class="menu">TUT Downloads</a><br>latest version of TUT as well as other related stuff</p></li>
</ul>
<p class="subheader">Support</p>
<ul>
<li><p><a href="/faq/" class="menu">TUT Frequently Asked Questions</a><br>often asked questions and answers for them</p></li>
<li><p><a href="/links/" class="menu">Links</a><br>related projects and concepts</p></li>
<li><p><a href="/author/" class="menu">TUT Author</a><br>who is the author</p></li>
</ul>
</td>
</tr>
</table>
<!-- << -->
</td>
</tr>
</table>
<!-- menu table << -->
</td>
<td width="70%" valign=top>
<!-- content table >> -->
<table width="100%" cellspacing=0 cellpadding=0>
<tr>
<td width="100%" align=center valign=center bgcolor="#cccccc">
<!-- >> -->
<table width="100%" cellspacing=1 cellpadding=10>
<tr>
<td width="100%" align=left valign=top bgcolor="#eeeeee">
<p class="header">TUT How-To</p>
<p class="subheader">About this document</p>		
		
		<p>
			This text contains an explanation how you can start to use TUT in shortest possible time.
			It also contains some introductory information on unit testing itself for those who are
			new in the field.
		</p>
				
		<p class="subheader">What is TUT</p>
		
		<p> 
			TUT is a pure C++ unit test framework. 
			It's name - TUT - stands for Template Unit Tests. 
		</p>
		
		<p>
			Unit test frameworks help to organize and run 
			<a href="http://en.wikipedia.org/wiki/Unit_tests">unit tests</a>. 
			Unit tests are used	to check application correctness according to requirements. 
			Those tests are usually run	often (nightly or hourly) to achieve 
			<a href="http://www.martinfowler.com/articles/continuousIntegration.html">continuous integration</a>.
		</p>
		
		<p class="subheader">TUT tests organization</p>
		
		<p>
			C++ produces executable code, so tests have to be compiled into a single binary
			called test application. The application can be built in automated mode to
			perform nightly tests. It can also be built manually when a developer hunts for
			bugs.
		</p>
		
		<p>
			The test application contains tests, organized into test groups.
			The functionality of a tested application can be divided into a few separate
			functional blocks (e.g. Access Rights, Export, Processing, ...). It is natural to
			group tests for each block together. Each test group has a unique human-readable 
			name and normally is located in a separate file.
		</p>
		
		<p class="subheader">Tests</p>
		
		<p>
			Test is a function (method) that implements some specific scenario and checks
			if the code (unit) behaves as required. Each test usually checks only one 
			specific element of functionality. In almost any test we have a preparation
			phase, execution phase and verification phase. For example, if we need to
			test our container's clear() method is correct, we need:
			<ul>
				<li>create a container instance and fill it with some data (preparation phase)</li>
				<li>call clear() method (execution phase)</li>
				<li>ensure that size() now returns zero (verification phase)</li>
			</ul>
		</p>
		
		<p class="subheader">What we are going to test?</p>		
		
		<p>
			Suppose we need to create a shared_ptr class for our application. 
			We need to write tests for the class to be sure it works as it should.
			The same tests would also guide someone who will maintain the code.
		</p>
		
		<p>
			Prior to test writing we should decide what to test. 
			Maximalist approach requires to write so many tests that altering any single 
			line of your code will break at least one test. Minimalist approach allows 
			us to write tests only for the most general or the most complex use cases. The 
			truth lies somewhere in between. We should consider common correct and 
			incorrect usage scenarios, and use them as a basis for our tests.
		</p>
		
		<p>
			For our shared_ptr we obviosly should test constructors, assignment operators,
			referencing and passing ownership. Later we may come to some other test scenarios.
		</p>
		
		<p class="subheader">Skeleton</p>
		
		<p>
			If you don't have any implemented class to test yet, it would be good to implement it
			as a set of stub methods first. Thus you'll get an interface, and be able to write
			your tests. Yes, this is correct: 
			<a href="http://en.wikipedia.org/wiki/Test_driven_development">you should write your tests before writing the code</a>! 
			Writing tests often helps to understand oddities in the	current interface, and fix it. 
			Besides, with the stubs all your tests will fail, so you'll be sure tests do their job.
		</p>
		
		<p class="subheader">Creating Test Group</p>
		
		<p>
			Since we're writing unit tests, it would be a good idea to group the tests for our
			class in one place to be able to run them separately. It's also natural in C++ to
			place all the tests of a group into one compilation unit (i.e. source file). So, to
			begin, we should create a new file. Let's call it test_shared_ptr.cpp. 
			(<i>Final variant of the test group can be found in TUT distribution in directory examples/shared_ptr </i>)
		</p>
		
		<pre> 
// test_shared_ptr.cpp 
#include &lt;tut.h&gt;
namespace tut { }; 
		</pre>
		
		<p>
			As you see, you need to include TUT header file (as expected) and use namespace tut
			for tests. You may also use anonymous namespace if your compiler allows it (you
			will need to instantiate methods from tut namespace and some compilers refuse to
			place such instantiations into the anonymous namespace).</p>
		
		<p>
			A test group in TUT framework is described by the special template
			test_group&lt;T&gt;. The template parameter T is a type that will hold all
			test-specific data during the test execution. Actually, the data stored in T are
			member data of the test class. Test class is inherited from T, so any test can refer to
			the data in T as it's member data. Complex? Not really, as you'll see.
		</p>
		
		<p>
			For simple test groups (where all data are stored in test local variables) type T 
			could be an empty struct.
		</p>
		
		<pre> 
#include &lt;tut.h&gt;
namespace tut 
{ 
	struct shared_ptr_data { }; 
} 
		</pre>
		
		<p>
			But when tests have complex or repeating creation phase, you may put data members
			into T and provide constructor (and, if required, destructor) for it. For
			each test, a new instance of T will be created. To prepare your test for execution
			TUT will use default constructor. Similarly, after the test has been finished,
			the destructor is called to clean up T. I.e.:
		</p>
		
		<pre> 
#include &lt;tut.h&gt;
namespace tut 
{ 
	struct complex_data 
	{ 
		connection* con; 
		complex_data()
		{ 
			con = db_pool.get_connection(); 
		} 
		~complex_data()
		{
			db_pool.release_connection(con); 
		} 
	};
	
	// each test from now on will have con data 
	// member initialized by constructor
	...
	con-&gt;commit(); 
	... 
		</pre>
		
		<p>
			What will happen if the constructor throws an exception? TUT will treat it as if
			test itself failed with exception, so this test will not be executed. You'll see
			an exception mark near the test, and if the constructor throwed something
			printable, the message will appear.
		</p>
		
		<p>
			Exception in destructor is threated a bit different. Reaching destruction phase
			means that the test is passed, so TUT marks test with warning status meaning that
			test itself was OK, but something bad has happend <i>after</i> the test. 
		</p>
		
		<p>
			Well, all we have written so far is just a type declaration. To work with a group we
			have to have an object, so we must create the test group object. Since we need only
			one test group object for each unit, we can (and should, actually) make this
			object static. To prevent name clash with other test group objects in the
			namespace tut, we should provide an unique descriptive name (or, alternatively, we may
			put it into the anonymous namespace). The former is more correct, but the
			descriptive name usually works well too, unless you're too terse in giving names
			to objects.
		</p>
		
		<pre> 
#include &lt;tut.h&gt;
namespace tut 
{ 
	struct shared_ptr_data {};
	typedef test_group&lt;shared_ptr_data&gt; tg; 
	tg shared_ptr_group("shared_ptr"); 
}; 
		</pre>
		
		<p>
			As you see, any test group accepts a single parameter - it's human-readable name.
			This name is used to identify the group when a programmer wants to execute all
			tests or a single test within the group. So this name shall also be descriptive
			enough to avoid clashes. Since we're writing tests for a specific unit, it's
			enough to name it after the unit name.
		</p>
		
		<p>
			Test group constructor will be called at unspecified moment at the test
			application startup. The constructor performs self-registration; it calls
			tut::runner and asks it to store the test group object name and location. Any
			other test group in the system undergoes the same processing, i.e. each test
			group object registers itself. Thus, test runner can iterate all test groups or
			execute any test group by its name.
		</p>
		
		<p>
			Newly created group has no tests associated with it. To be more precise, it has
			predefined set of dummy tests. By default, there are 50 tests in a group,
			including dummy ones. To create a test group with higher volume (e.g. when tests
			are generated by a script and their number is higher) we must provide a higher
			border of test group size when it is instantiated:
		</p>
		
		<pre> 
#include &lt;tut.h&gt;
namespace tut 
{ 
	struct huge_test_data { };
	// test group with maximum 500 tests 
	typedef	test_group&lt;huge_test_data,500&gt; testgroup; 
	testgroup huge_test_testgroup("huge group"); 
}; 
		</pre>
		
		<p>
			Note also that your compiler would possibly need a command-line switch or pragma
			to enlarge recursive instantiation depth. For g++, for example, you should
			specify at least --ftemplate-depth-501 to increase the depth to 500. For more
			information see your compiler documentation.
		</p>
		
		<p class="subheader">Creating Tests</p>
		
		<p>Now it's time to fill our test group with content.</p>
		
		<p>
			In TUT, all tests have unique <b>numbers</b> inside the test group. 
			Some people believe	that textual names better describe failed tests in reports. 
			I agree. But C++ templates work with numbers because they are compile-time 
			constants and refuse to do the same with strings, since strings are in fact
			addresses of character buffers, i.e. run-time data. So I had no other choice.
		</p>
		
		<p>
			As I mentioned above, our test group already has a few dummy tests; and we can
			replace any of them with something real just by writing our own version:
		</p>
		
		<pre> 
#include &lt;tut.h&gt;
namespace tut 
{ 
	struct shared_ptr_data{};
	typedef test_group&lt;shared_ptr_data&gt; testgroup; 
	typedef	testgroup::object testobject; 
	testgroup shared_ptr_testgroup("shared_ptr");

	template&lt;&gt; template&lt;&gt; 
	void testobject::test&lt;1&gt;() 
	{ 
		// do nothing test 
	} 
}; 
		</pre>
		
		<p>
			So far this test does nothing, but it's enough to illustrate the way we may
			create our own test methods. Note that your shouldn't call anything like
			REGISTER_TEST() macro or do anything similarely annoying. Just write a test,
			and it will be executed.
		</p>
		
		<p>
			All tests in the group are methods of the type test_group&lt;T&gt;::object. This
			class is directly inherited from our test data structure. In our case, it is
		</p>
		
		<pre> 
class object : public shared_ptr_data { };
		</pre>
		
		<p>
			This allows to access members of the shared_ptr_data structure directly, 
			since at the same time they are members of the object type itself. We also 
			typedef the type with testobject for brevity.
		</p>
		
		<p>
			We mark our test with number 1. Previously, test group had a dummy test with the same
			number, but now, since we've defined our own version, it replaced the dummy test
			as more specialized one. It's how C++ template ordering works.
		</p>
		
		<p>
			The test we've written always succeeds. Successful test returns with no
			exceptions. Unsuccessful one either throws an exception, or fails at fail() or
			ensure() methods (which anyway just throw the exception when failed).
		</p>
		
		<p class="subheader">First real test</p>
		
		<p>
			Now we know enough to write the first real working test. This test will create
			shared_ptr instances and check their state. We will define a small structure
			(keepee) to use it as shared_ptr stored object type.
		</p>
		
		<pre> 
#include &lt;tut.h&gt; 
#include &lt;shared_ptr.h&gt;
namespace tut 
{ 
	struct shared_ptr_data 
	{ 
		struct keepee{ int data; }; 
	};
	
	typedef test_group&lt;shared_ptr_data&gt; testgroup; 
	typedef	testgroup::object testobject; 
	testgroup shared_ptr_testgroup("shared_ptr");
			
	// checks default constructor. 
	template&lt;&gt; template&lt;&gt; 
	void testobject::test&lt;1&gt;() 
	{ 
		shared_ptr&lt;keepee&gt; def;
		ensure("null",def.get()== 0); 
	} 
}; 
		</pre>
		
		<p>
			That's all! The first line creates shared_ptr. If constructor throws an
			exception, test will fail (exceptions, including '...', are catched by the TUT
			framework). If the first line succeeds, we must check whether the kept object is
			null one. To do this, we use test object member function ensure(), which throws
			std::logic_error with a given message if its second argument is not true.
			Finally, if destructor of shared_ptr fails with exception, TUT also will report
			this test as failed.
		</p>
		
		<p>
			It's equally easy to write a test for the scenario where we expect to get an
			exception: let's consider our class should throw an exception if it has no stored
			object, and the operator -&gt; is called.
		</p>
		
		<pre> 
	// checks operator -&gt; throws instead of returning null.
	template&lt;&gt; template&lt;&gt; 
	void testobject::test&lt;2&gt;() 
	{ 
		try 
		{
			shared_ptr&lt;keepee&gt; sp; 
			sp-&gt;data = 0; 
			fail("exception expected"); 
		}
		catch( const std::runtime_error&amp; ex ) 
		{ 
			// ok 
		} 
	} 
		</pre>
		
		<p>
			Here we expect the std::runtime_error. If operator doesn't throw it, we'll force
			the test to fail using another member function: fail(). It just throws
			std::logic_error with a given message. If operator throws anything else, our
			test will fail too, since we intercept only std::runtime_error, and any other
			exception means the test has failed. 
		</p>
		
		<p>
			NB: our second test has number 2 in its name; it can, actually, be any in range
			1..Max; the only requirement is not to write tests with the same numbers. But if you
			did, compiler will force you to fix it anyway. 
		</p>
		
		<p>
			And finally, one more test to demonstrate how to use the ensure_equals template
			member function:
		</p>
		
		<pre> 
	/** checks keepee counting. 
	template&lt;&gt; template&lt;&gt; 
	void testobject::test&lt;3&gt;() 
	{ 
		shared_ptr&lt;keepee&gt; sp1(new keepee()); 
		shared_ptr&lt;keepee&gt; sp2(sp1); 
		ensure_equals("second copy at sp1",sp1.count(),2);
		ensure_equals("second copy at sp2",sp2.count(),2);
	} 
		</pre>
		
		<p>
			The test checks if the shared_ptr correctly counts references during copy
			construction. What's interesting here is the template member ensure_equals.
			It has an additional functionality comparing with similar call
			ensure("second_copy",sp1.count()==2); it uses operator == to check the
			equality of the two passed parameters and, what's more important, it uses
			std::stringstream to format the passed parameters into a human-readable message
			(smth like: "second copy: expected 2, got 1"). It means that ensure_equals
			cannot be used with the types that don't have operator
			&lt;&lt;; but for those having the operator it provides much more informational
			message.
		</p>
		
		<p>
			In contrast to JUnit assertEquals, where the expected value goes before the
			actual one, ensure_equals() accepts the expected after the actual value. I
			believe it's more natural to read ensure_equals("msg", count, 5) as "ensure
			that count equals to 5" rather than JUnit's "assert that 5 is the value of the
			count".
		</p>
		
		<p class="subheader">Running tests</p>
		
		<p>
			Tests are written, but an attempt to run them will be unsuccessful. We need
			a few other bits to complete the test application.
		</p>
		
		<p>
			First of all, we need a main() method, simply because it must be in every
			application. Second, we need a test runner singleton. Remember I said each
			test group should register itself in singleton? So, we need that singleton. And,
			finally, we need a kind of a callback handler to visualize our test results.
		</p>
		
		<p>
			The design of TUT doesn't strictly set a way the tests are visualized; instead, it
			provides an opportunity to get the test results by means of callbacks. Moreover
			it allows user to output the results in any format he prefers. Of course, there is a
			"reference implementation" in the example/ subdirectory of the project.
		</p>
		
		<p>
			Test runner singleton is defined in tut.h, so all we need to activate it is to
			declare an object of the type tut::test_runner_singleton in the main module
			with a special name tut::runner.
		</p>
		
		<p>
			Now, with the test_runner we can run tests. Singleton has method get() returning a
			reference to an instance of the test_runner class, which in turn has methods
			<ul>>
			<li>run_tests() to run all tests in all groups</li>
			<li>run_tests(const std::string&amp; groupname) to run all tests in a given group</li>
			<li>run_test(const std::string&amp; grp,int n) to run one test in the specified group.</li>
			</ul>
		</p>
		
		<p>
			So here goes our main module:
		</p>
		
		<pre> 
// main.cpp 
#include &lt;tut.h&gt;
namespace tut 
{ 
	test_runner_singleton runner; 
}

int main() 
{ 
	// run all tests in all groups 
	runner.get().run_tests();
	
	// run all tests in group "shared_ptr"
	runner.get().run_tests("shared_ptr");
			
	// run test number 5 in group "shared_ptr"
	runner.get().run_test("shared_ptr",5);
	
	return 0; 
} 
		</pre>
		
		<p>
			It's up to user to handle command-line arguments or GUI messages and map those
			arguments/messages to actual calls to test runner. Again, as you see, TUT
			doesn't restrict user here.
		</p>
		
		<p>
			But, the last question is still unanswered: how do we get our test results? The
			answer lies inside tut::callback interface. We could create it's subclass,
			and write a few simple methods. We also can omit any method since they have
			default no-op implementation. Each corresponding method is called in the
			following cases:
			<ul>
				<li>a new test run has started;</li>
				<li>test is finished; </li>
				<li>test run is finished.</li>
			</ul> 
		</p>
		
		<p>
			Here is a minimal implementation:
		</p>
		
		<pre> 
class visualizator : public tut::callback 
{ 
public: 
	void run_started(){ }
	void test_completed(const tut::test_result&amp; tr) 
	{ // ... show test result here ... }
	void run_completed(){ } 
}; 
		</pre>
		
		<p>
			The most important is the test_completed() method; its parameter has type
			test_result, and contains everything about the finished test, from its group
			name and number to the exception message, if any. Member variable result is an 
			enum which contains status of the test: ok, fail or ex. Take a look at the
			examples/basic/main.cpp for more complete visualizator. 
		</p>
		
		<p>
			Visualizator should be passed to the test_runner before the run. Knowing that, we are
			ready to write the final version of our main module: 
		</p>
		
		<pre> 
#include &lt;tut.h&gt;
namespace tut 
{ 
	test_runner_singleton runner; 
}

class callback : public tut::callback 
{ 
public: 
	void run_started(){ std::cout &lt;&lt; "\nbegin"; }
	void test_completed(const tut::test_result&amp; tr) 
	{ std::cout &lt;&lt; tr.test_pos &lt;&lt; "=" &lt;&lt; tr.result &lt;&lt; std::flush; }
	void run_completed(){ std::cout &lt;&lt; "\nend"; } 
};

int main() 
{ 
	callback clbk; 
	runner.get().set_callback(&amp;clbk);
	runner.get().run_tests(); 
	return 0; 
} 
		</pre>
		
		<p>
			That's it. We are now ready to link and run our test application. Do it as often as
			possible; once a day is a definite must. I hope TUT will help you to make your
			application more robust and relieve your testing pain. Feel free to send your
			questions, suggestions and critical opinions to me; I'll do my best to address
			them asap. 
		</p>
<img src="/_img/pixel.gif" alt="" width=1 height=300><br>

<center>
<!--RAX counter-->
<script language="JavaScript">
<!--
document.write('<a href="http://www.rax.ru/click" '+
'target=_blank><img src="http://counter.yadro.ru/hit?t13.4;r'+
escape(document.referrer)+((typeof(screen)=='undefined')?'':
';s'+screen.width+'*'+screen.height+'*'+(screen.colorDepth?
screen.colorDepth:screen.pixelDepth))+';'+Math.random()+
'" alt="rax.ru:     24 ,   24    \" '+
'border=0 width=88 height=31></a><br>')
//-->
</script>
<!--/RAX-->
</center>

</td>
</tr>
</table>
<!-- << -->
</td>
</tr>
</table>
<!-- content table << -->
</td>
</tr>
</table>

</body>
</html>