File: tutorial.html

package info (click to toggle)
cl-unit 1.3.1-2
links: PTS
area: main
in suites: etch, etch-m68k, sarge
size: 160 kB
ctags: 42
sloc: lisp: 259; makefile: 49; sh: 27
file content (461 lines) | stat: -rw-r--r-- 15,365 bytes
parent folder | download | duplicates (2)
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=windows-1252">
<TITLE>Using CLUnit: An Introduction and Tutorial</TITLE>
</HEAD>
<BODY>
<table>
<TR>
<td>
<H1>Using CLUnit: An Introduction and Tutorial</H1>
<H2>F. A. Adrian<br>ancar technology</H2>
</td>
<td>
<a HREF="CLUnit.html">
<img src="clunit-med.jpg" align="top" width="368" height="92" border="0" alt="CLUnit">
</a>
</td>
</tr>
</table>
<p>Insuring that code runs and continues to run after changes is
extremely important in developing systems in a timely and effective
manner.  Unit testing is well known way to ensure that code works
properly. Unit testing provides functions that are run to verify
proper operation of the system under test.  The CLUnit package
provides a unit test environment for Common Lisp Implementations.
It provides facilities for defining, organizing, and running tests.</P>
<P>CLUnit is provided under the terms of the
<A HREF="http://www.opensource.org/licenses/lgpl-license.html">
LGPL</A>.  As such,
it may be loaded and compiled into a product for general release.
However, the code is distributed without any warranty (see the text
of LGPL for all applicable licensing conditions) and the user assumes
all risk of usage.</P>
<P>To start using CLUnit, load the file
<code>CLUnit.lisp</code>:</P>
<pre>
<B>(load "CLUnit.lisp")</B>
</pre>
<p>In this document, we will show whatever the user types in <B>bold</B>
and how the computer responds in normal face.</p>
<PRE>
Dummy error occurred in test "Error test"
1 test run; 0 tests passed; 1 test failed.
Dummy error occurred in test "Error test"
1 test run; 0 tests passed; 1 test failed.
2 tests run; 2 tests passed; 0 tests failed.
11 tests run; 11 tests passed; 0 tests failed.
CLUnit self-test passed.
</pre>
<p>The first thing that CLUnit does is to run a self-test to ensure
that it is running properly. The ancilary messages shown are outputs due to
testing of error trapping and running of test sets.  If the message "CLUnit
self-test passed." is showing, then CLUnit is working!</p>
<H2>Basic Testing</H2>
<P>Tests are defined using the <code><A HREF="deftest.html">
deftest</A></code> macro.  In its simplest form, the macro takes a test
description and a test function that is invoked when the test is run.
If the test function returns a non-nil value, the test is assumed to
have succeeded.  If it throws an error that escapes from the function
or returns nil, the test is assumed to have failed:</P>
<pre>
<B>(deftest "Test car 1" :test-fn #'(lambda () (eq (car '(a b)) 'a)))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car 1/*UNCATEGORIZED* #x1011B20&gt;

<B>(run-named-test "Test car 1")</B>
T

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>As we can see, a test can be defined and run by name and all tests
thus far defined can be run.  The <code><A HREF="run-all-tests.html">
run-all-tests</a></code> function returns three values -- a boolean,
telling us if all tests have succeeded or not, the number of failed
tests and the number of tests that passed.  If we define a test
that fails and then <code><A HREF="run-all-tests.html">run-all-tests
</a></code> again:</p>
<pre>
<B>(deftest "Test car 2" :test-fn #'(lambda () (error "A test error")))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car 2/*UNCATEGORIZED* #x10AD190&gt;

<B>(run-all-tests)</B>
A test error occurred in test "Test car 2"
2 tests run; 1 test passed; 1 test failed.
NIL
1
1</pre>
<p>In this case, the error is trapped and the
<code><A HREF="run-all-tests.html">run-all-tests</A></code> function
returns the appropriate values.</p>
<P>Let's redefine the second test for a more graceful failure.
By defining the test with the same name, the original test is
replaced:</P>
<pre>
<B>(deftest "Test car 2" :test-fn #'(lambda () nil))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car 2/*UNCATEGORIZED* #x10C85B0&gt;

<B>(run-all-tests)</B>
Output did not match expected output in test "Test car 2"
2 tests run; 1 test passed; 1 test failed.
NIL
1
1</pre>
<p>Again, the error is detected.  Note that, when a test fails, a
message is printed giving the description of the failed test and
the reason for failure.  In addition, the <code><A HREF="failed-tests.html">
failed-tests</A></code> function will return a list of all tests that
failed:</p>
<pre>
<B>(failed-tests)</B>
(#&lt;ORG.ANCAR.CLUNIT::TEST Test car 2/*UNCATEGORIZED* # x10C85B0&gt;)</pre>

<p>Now we'll get rid of the failing test by using the <code>
<a href="remove-test.html">remove-test</a></code> function:</P>

<pre>
<B>(remove-test "Test car 2")</B>
(#&lt;ORG.ANCAR.CLUNIT::TEST Test car 1/*UNCATEGORIZED* #x1097258&gt;)

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>And we see that the removed test is no longer in the system.</p>
<P>Finally, we can get rid of all of the tests by using the function
<code><A href="clear-tests.html">clear-tests</A></code> function:</P>
<pre>
<B>(clear-tests)</B>
NIL

<B>(run-all-tests)</B>
0 tests run; 0 tests passed; 0 tests failed.
T
0
0</pre>
<H2>Categorizing Tests</H2>
<p>Categories are a useful tool to organize sets of tests.  A category for
a test is specified by using the <code>:category</code>
keyword in the <code><a href="deftest.html">deftest</a></code> macro and tests
from a specific category are run using the
<code><a href="run-category.html">run-category</a></code> function:</P>
<pre>
<B>(deftest "Test car of nil" :category "Test car"
    :test-fn #'(lambda () (eq (car nil) nil)))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car of nil/Test car #x10F8630&gt;

<B>(deftest "Test car of dotted cons" :category "Test car"
    :test-fn #'(lambda () (eq (car '(a . b)) 'a)))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car of dotted cons/Test car #x110F688&gt;

<B>(deftest "Another test" :category "General"
    :test-fn #'(lambda () t))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Another test/General #x11255C0&gt;

<B>(run-category "Test car")</B>
2 tests run; 2 tests passed; 0 tests failed.
T
0
2

<B>(run-category "General")</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1

<B>(run-all-tests)</B>
3 tests run; 3 tests passed; 0 tests failed.
T
0
3</pre>
<p>By default, tests are categorized under the category named *UNCATEGORIZED*.
Let's clear the tests again:</P>
<pre>
<B>(clear-tests)</B>
NIL</pre>
<H2>Better Test Specifications</H2>
<p>All of the work of the tester can be done using only this
functionality, but sometimes the testing becomes a bit clunky.
Suppose we had a function returning multiple values:</p>
<pre>
<B>(defun my-func () (values 1 2 3))</B>
MY-FUNC

<B>(my-func)</B>
1
2
3</pre>
To test this function, a test would have to be written as follows:
<pre>
<B>(deftest "Test my-func"
    :test-fn #'(lambda () (equal (multiple-value-list (my-func)) '(1 2 3))))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test my-func/*UNCATEGORIZED* #x103EA68&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>In order to handle multiple values more gracefully, this test can be
written using the <code>:output-fn</code>
keyword:</p>
<pre>
<B>(deftest "Test my-func" :test-fn #'my-func :output-fn #'(lambda () (values 1 2 3)))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test my-func/*UNCATEGORIZED* #x1096640&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1 

<B>(clear-tests)</B>
NIL</pre>
<p>This feature can also be used for more mundane uses:</p>
<pre>
<B>(deftest "Test car of list" :category "Test car"
    :test-fn (lambda () (car '(a b))) :output-fn #'(lambda () 'a))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car of list/Test car #x10CA018&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>The expected output function is always a function of zero arguments
that returns the values that the output of the test function is
compared against.  The comparison is done by turning each set of
values into a list using <code>
multiple-value-list</code> and comparing the list with a comparison
function.  This comparison function defaults to
<code>#'equal</code>, but the user can
specify another comparison function using the
<code>:compare-fn</code> keyword:</p>
<pre>
<B>(clear-tests)</B>
NIL

<B>(deftest "Test compare-fn"
    :test-fn #'(lambda () "abc")
    :output-form "abc"
    :compare-fn #'(lambda (rlist1 rlist2)
                      (reduce #'and (mapcar #'string-equal rlist1 rlist2))))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test compare-fn/*UNCATEGORIZED* #xFB5500&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>Note that the comparison function is comparing two lists holding the
results of the output and test functions.  As such, comparison
functions can sometimes be quite hairy.</p>

<P>Rather than specifying the <code>
:output-fn</code> keyword, one can use the
<code>:output-form</code> keyword.
With respect to the test above, the test would change as follows:</P>
<pre>
<B>(clear-tests)</B>
NIL

<B>(deftest "Test car of list" :category "Test car"
    :test-fn (lambda () (car '(a b))) :output-form 'a)</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car of list/Test car #x10E6370&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>If both <code>:output-fn </code>and
<code>:output-form</code> keywords are
specified, the <code>:output-fn</code>
keyword takes precedence.</p>

<P>Analogously, <code>:input-fn </code>
and/or <code>:input-form </code>functions
can be specified:</P>
<pre>
<B>(clear-tests)</B>
NIL

<B>(deftest "Test +"
    :input-fn #'(lambda () (values 1 2 3))
    :test-fn #'+
    :output-form 6)</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test +/*UNCATEGORIZED* #x1105888&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>The test function is applied to the multiple values returned
by the input function.  The <code>:input-form
</code>keyword returns only one value so the test function, in this
case, must be unary:</p>
<pre>
<B>(clear-tests)</B>
NIL

<B>(deftest "Test car"
    :input-form '(a b)
    :test-fn #'car
    :output-form 'a)</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test car/*UNCATEGORIZED* #x1121CD0&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1</pre>
<p>Specifying an incorrect arity in the test function will result
in an error:</p>
<pre>
<B>(clear-tests)</B>
NIL

<B>(deftest "Test cons"
    :input-form '(a b)
    :test-fn #'cons
    :output-form '(a . b))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test cons/*UNCATEGORIZED* #xFC0EB8&gt;

<B>(run-all-tests)</B>
Wrong number of arguments occurred in test "Test cons"
1 test run; 0 tests passed; 1 test failed.
NIL
1
0

<B>(deftest "Test cons"
    :input-form (values 'a 'b)
    :test-fn #'cons
    :output-form '(a . b))</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test cons/*UNCATEGORIZED* #xFDA318&gt;

<B>(run-all-tests)</B>
1 test run; 1 test passed; 0 tests failed.
T
0
1

<B>(clear-tests)</B>
NIL</pre>
<p>After a test failure, one may want to run a test outside the error
trapping code of <code><a href="run-all-tests.html">run-all-tests</a></code>
or <code><a href="run-category.html">run-category</a></code> to debug a
test failure.  We use the function <code><a href="run-named-test.html">
run-named-test</a></code> to do this:</p>
<pre>
<B>(run-named-test "Test cons")</B>
;;; An error occurred in function _WRONG-NUMBER-OF-ARGS-ERROR:
;;; Error: Wrong number of arguments
;;; Entering Corman Lisp debug loop. 
;;; Use :C followed by an option to exit. Type :HELP for help.
;;; Restart options:
;;; 1   Abort to top level.

;;; Debugging stuff...</pre>
<p>The function <code><a href="run-named-test.html">
run-named-test</a></code>
also takes an optional "protected mode" flag to allow the
user to try an individual test:</p>
<pre>
<B>(run-named-test "Test cons" t)</B>
Wrong number of arguments occurred in test "Test cons"
(NIL #&lt;Simple-error #x101EDB8&gt;)
</pre>
<p>This form of the <code><a href="run-named-test.html">
run-named-test</a></code>
function returns two values, the first telling whether the function
succeeded, the second giving the error in case of failure.</p>
<H2>Miscellaneous Features</H2>
<p>Other miscellaneous functions include <code>
<a href="list-tests.html">list-tests</a></code> and <code>
<a href="list-categories.html">list-categories</a></code>:</P>
<pre>
<B>(clear-tests)</B>
NIL

<B>(deftest "Test 1" :category "Category 1")</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test 1/Category 1 #x1037060&gt;

<B>(deftest "Test 1.2" :category "Category 1")</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test 1.2/Category 1 #x1048DF0&gt;

<B>(deftest "Test 2" :category "Category 2")</B>
#&lt;ORG.ANCAR.CLUNIT::TEST Test 2/Category 2 #x105BB90&gt;

<B>(list-tests)</B>
("Test 2/Category 2" "Test 1.2/Category 1" "Test 1/Category 1")

<B>(list-tests "Category 1")</B>
("Test 1.2/Category 1" "Test 1/Category 1")

<B>(list-categories)</B>
("Category 1" "Category 2")

<B>(run-all-tests)</B>
3 tests run; 3 tests passed; 0 tests failed.
T
0
3</pre>
<p>Note the other (somewhat useless) feature demonstrated above -- a test
without a test function specified always succeeds.</p>
<H2>Tips and Usage Hints</h2>
There are a few general usage hints that will help you use CLUnit to
its best advantage:
<UL>
<LI>Test names and categories are strings and compared with
<code>string-equal</code> in the CLUnit
code.  As such, case sensitivity is not necessarily an issue, but
proper spelling of category and test names (including embedded spaces)
is.  It may be helpful to define constants for category names:
<pre>
<B>(defconstant +category-name+ "XXX")</B>
+CATEGORY-NAME+

<B>(deftest "A test" :category +category-name+)</B>
#&lt;ORG.ANCAR.CLUNIT::TEST A test/XXX #x1093B58&gt;</pre></LI></UL>
<UL>
<LI>Use <code><a href="remove-test.html">remove-test</a></code> sparingly.
If the feature was important enough to test before, it's probably
important enough to tests now.  Only in the cases when the
functionality has been removed is it usually necessary to remove the
test.  When in doubt, redefine.</LI>
<LI>Use <code><a href="run-named-test.html">run-named-test</a></code> and
<code><a href="run-category.html">run-category</a></code> sparingly.  These
functions are provided for the convenience of running a subset of tests in
isolation for use in short development sessions.  In general, you should run
all the system's unit tests as often as possible.
The more often you run all tests, the more often you insure that you
haven't broken anything else with your changes.  Prefer
<code><a href="run-all-tests.html">run-all-tests</a></code>.</LI>
<LI>Use <code>:input-fn</code>,
<code>:input-form</code>,
<code>:output-fn</code>,
and <code>:output-form</code> to keep test
functions as simple as possible.</LI></UL>
<H2>Final Word</H2>
I hope you enjoy <a href="CLUnit.html">CLUnit</a> and that it helps you to
deliver error-free
code in a timely manner.  If you have any suggestions for improvements
or changes, please contact me at <A HREF="mailto:fadrian@qwest.net">
<tt>fadrian@qwest.net</tt></A>.<hr>
<address>
  <a href="http://validator.w3.org/check/referer"><img
     src="http://www.w3.org/Icons/valid-html32" height="31" width="88"
     align=right border="0" alt="Valid HTML 3.2!"></a>
  <a href="mailto:fadrian@qwest.net">Frank A. Adrian</a><br>
  Last modified: 12 June, 2001 16:04 
</address>

</BODY>
</HTML>