File: Test-Functions.html

package info (click to toggle)
octave 3.8.2-4
links: PTS, VCS
area: main
in suites: jessie, jessie-kfreebsd
size: 84,396 kB
ctags: 45,547
sloc: cpp: 293,356; ansic: 42,041; fortran: 23,669; sh: 13,629; objc: 7,890; yacc: 7,093; lex: 3,442; java: 2,125; makefile: 1,589; perl: 1,009; awk: 974; xml: 34
file content (537 lines) | stat: -rw-r--r-- 22,486 bytes
parent folder | download | duplicates (3)
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
<head>
<title>GNU Octave: Test Functions</title>

<meta name="description" content="GNU Octave: Test Functions">
<meta name="keywords" content="GNU Octave: Test Functions">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Concept-Index.html#Concept-Index" rel="index" title="Concept Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Test-and-Demo-Functions.html#Test-and-Demo-Functions" rel="up" title="Test and Demo Functions">
<link href="Demonstration-Functions.html#Demonstration-Functions" rel="next" title="Demonstration Functions">
<link href="Test-and-Demo-Functions.html#Test-and-Demo-Functions" rel="prev" title="Test and Demo Functions">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.indentedblock {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
div.smalllisp {margin-left: 3.2em}
kbd {font-style:oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nocodebreak {white-space:nowrap}
span.nolinebreak {white-space:nowrap}
span.roman {font-family:serif; font-weight:normal}
span.sansserif {font-family:sans-serif; font-weight:normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
<a name="Test-Functions"></a>
<div class="header">
<p>
Next: <a href="Demonstration-Functions.html#Demonstration-Functions" accesskey="n" rel="next">Demonstration Functions</a>, Up: <a href="Test-and-Demo-Functions.html#Test-and-Demo-Functions" accesskey="u" rel="up">Test and Demo Functions</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Test-Functions-1"></a>
<h3 class="section">B.1 Test Functions</h3>

<a name="XREFtest"></a><dl>
<dt><a name="index-test"></a>Command: <em></em> <strong>test</strong> <em><var>name</var></em></dt>
<dt><a name="index-test-1"></a>Command: <em></em> <strong>test</strong> <em><var>name</var> quiet|normal|verbose</em></dt>
<dt><a name="index-test-2"></a>Function File: <em></em> <strong>test</strong> <em>(&quot;<var>name</var>&quot;, &quot;quiet|normal|verbose&quot;, <var>fid</var>)</em></dt>
<dt><a name="index-test-3"></a>Function File: <em></em> <strong>test</strong> <em>([], &quot;explain&quot;, <var>fid</var>)</em></dt>
<dt><a name="index-test-4"></a>Function File: <em><var>success</var> =</em> <strong>test</strong> <em>(&hellip;)</em></dt>
<dt><a name="index-test-5"></a>Function File: <em>[<var>n</var>, <var>max</var>] =</em> <strong>test</strong> <em>(&hellip;)</em></dt>
<dt><a name="index-test-6"></a>Function File: <em>[<var>code</var>, <var>idx</var>] =</em> <strong>test</strong> <em>(&quot;<var>name</var>&quot;, &quot;grabdemo&quot;)</em></dt>
<dd>
<p>Perform tests from the first file in the loadpath matching <var>name</var>.
<code>test</code> can be called as a command or as a function.  Called with
a single argument <var>name</var>, the tests are run interactively and stop
after the first error is encountered.
</p>
<p>With a second argument the tests which are performed and the amount of
output is selected.
</p>
<dl compact="compact">
<dt><code>&quot;quiet&quot;</code></dt>
<dd><p>Don&rsquo;t report all the tests as they happen, just the errors.
</p>
</dd>
<dt><code>&quot;normal&quot;</code></dt>
<dd><p>Report all tests as they happen, but don&rsquo;t do tests which require
user interaction.
</p>
</dd>
<dt><code>&quot;verbose&quot;</code></dt>
<dd><p>Do tests which require user interaction.
</p></dd>
</dl>

<p>The argument <var>fid</var> can be used to allow batch processing.  Errors
can be written to the already open file defined by <var>fid</var>, and
hopefully when Octave crashes this file will tell you what was happening
when it did.  You can use <code>stdout</code> if you want to see the results as
they happen.  You can also give a file name rather than an <var>fid</var>, in
which case the contents of the file will be replaced with the log from
the current test.
</p>
<p>Called with a single output argument <var>success</var>, <code>test</code> returns
true if all of the tests were successful.  Called with two output arguments
<var>n</var> and <var>max</var>, the number of successful tests and the total number
of tests in the file <var>name</var> are returned.
</p>
<p>If the second argument is the string <code>&quot;grabdemo&quot;</code>, the contents of
the demo blocks are extracted but not executed.  Code for all code blocks
is concatenated and returned as <var>code</var> with <var>idx</var> being a vector
of positions of the ends of the demo blocks.
</p>
<p>If the second argument is <code>&quot;explain&quot;</code>, then <var>name</var> is ignored
and an explanation of the line markers used is written to the file
<var>fid</var>.
</p>
<p><strong>See also:</strong> <a href="#XREFassert">assert</a>, <a href="#XREFfail">fail</a>, <a href="Raising-Errors.html#XREFerror">error</a>, <a href="Demonstration-Functions.html#XREFdemo">demo</a>, <a href="Demonstration-Functions.html#XREFexample">example</a>.
</p></dd></dl>


<p><code>test</code> scans the named script file looking for lines which start
with the identifier &lsquo;<samp>%!</samp>&rsquo;.  The prefix is stripped off and the rest
of the line is processed through the Octave interpreter.  If the code
generates an error, then the test is said to fail.
</p>
<p>Since <code>eval()</code> will stop at the first error it encounters, you must
divide your tests up into blocks, with anything in a separate
block evaluated separately.  Blocks are introduced by valid keywords like
<code>test</code>, <code>function</code>, or <code>assert</code> immediately following &lsquo;<samp>%!</samp>&rsquo;.
A block is defined by indentation as in Python.  Lines beginning with
&lsquo;<samp>%!&lt;whitespace&gt;</samp>&rsquo; are part of the preceeding block.
</p> 
<p>For example:
</p>
<div class="example">
<pre class="example">%!test error (&quot;this test fails!&quot;);
%!test &quot;test doesn't fail. it doesn't generate an error&quot;;
</pre></div>

<p>When a test fails, you will see something like:
</p>
<div class="example">
<pre class="example">  ***** test error (&quot;this test fails!&quot;)
!!!!! test failed
this test fails!
</pre></div>

<p>Generally, to test if something works, you want to assert that it
produces a correct value.  A real test might look something like
</p>
<div class="example">
<pre class="example">%!test
%! <var>a</var> = [1, 2, 3; 4, 5, 6]; B = [1; 2];
%! expect = [ <var>a</var> ; 2*<var>a</var> ];
%! get = kron (<var>b</var>, <var>a</var>);
%! if (any (size (expect) != size (get)))
%!   error (&quot;wrong size: expected %d,%d but got %d,%d&quot;,
%!          size (expect), size (get));
%! elseif (any (any (expect != get)))
%!   error (&quot;didn't get what was expected.&quot;);
%! endif
</pre></div>

<p>To make the process easier, use the <code>assert</code> function.  For example,
with <code>assert</code> the previous test is reduced to:
</p>
<div class="example">
<pre class="example">%!test
%! <var>a</var> = [1, 2, 3; 4, 5, 6]; <var>b</var> = [1; 2];
%! assert (kron (<var>b</var>, <var>a</var>), [ <var>a</var>; 2*<var>a</var> ]);
</pre></div>

<p><code>assert</code> can accept a tolerance so that you can compare results
absolutely or relatively.  For example, the following all succeed:
</p>
<div class="example">
<pre class="example">%!test assert (1+eps, 1, 2*eps)          # absolute error
%!test assert (100+100*eps, 100, -2*eps) # relative error
</pre></div>

<p>You can also do the comparison yourself, but still have assert
generate the error:
</p>
<div class="example">
<pre class="example">%!test assert (isempty ([]))
%!test assert ([1, 2; 3, 4] &gt; 0)
</pre></div>

<p>Because <code>assert</code> is so frequently used alone in a test block, there
is a shorthand form:
</p>
<div class="example">
<pre class="example">%!assert (&hellip;)
</pre></div>

<p>which is equivalent to:
</p>
<div class="example">
<pre class="example">%!test assert (&hellip;)
</pre></div>

<p>Occasionally a block of tests will depend on having optional
functionality in Octave.  Before testing such blocks the availability of
the required functionality must be checked.  A <code>%!testif HAVE_XXX</code>
block will only be run if Octave was compiled with functionality
&lsquo;<samp>HAVE_XXX</samp>&rsquo;.  For example, the sparse single value decomposition,
<code>svds()</code>, depends on having the <small>ARPACK</small> library.  All of the tests
for <code>svds</code> begin with
</p>
<div class="example">
<pre class="example">%!testif HAVE_ARPACK
</pre></div>

<p>Review <samp>config.h</samp> or <code>octave_config_info (&quot;features&quot;)</code> to see some
of the possible values to check.
</p>
<p>Sometimes during development there is a test that should work but is
known to fail.  You still want to leave the test in because when the
final code is ready the test should pass, but you may not be able to
fix it immediately.  To avoid unnecessary bug reports for these known
failures, mark the block with <code>xtest</code> rather than <code>test</code>:
</p>
<div class="example">
<pre class="example">%!xtest assert (1==0)
%!xtest fail (&quot;success=1&quot;, &quot;error&quot;)
</pre></div>

<p>In this case, the test will run and any failure will be reported.
However, testing is not aborted and subsequent test blocks will be
processed normally.  Another use of <code>xtest</code> is for statistical
tests which should pass most of the time but are known to fail
occasionally.
</p>
<p>Each block is evaluated in its own function environment, which means
that variables defined in one block are not automatically shared
with other blocks.  If you do want to share variables, then you
must declare them as <code>shared</code> before you use them.  For example, the
following declares the variable <var>a</var>, gives it an initial value (default
is empty), and then uses it in several subsequent tests.
</p>
<div class="example">
<pre class="example">%!shared <var>a</var>
%! <var>a</var> = [1, 2, 3; 4, 5, 6];
%!assert (kron ([1; 2], <var>a</var>), [ <var>a</var>; 2*<var>a</var> ]);
%!assert (kron ([1, 2], <var>a</var>), [ <var>a</var>, 2*<var>a</var> ]);
%!assert (kron ([1,2; 3,4], <var>a</var>), [ <var>a</var>,2*<var>a</var>; 3*<var>a</var>,4*<var>a</var> ]);
</pre></div>

<p>You can share several variables at the same time:
</p>
<div class="example">
<pre class="example">%!shared <var>a</var>, <var>b</var>
</pre></div>

<p>You can also share test functions:
</p>
<div class="example">
<pre class="example">%!function <var>a</var> = fn (<var>b</var>)
%!  <var>a</var> = 2*<var>b</var>;
%!endfunction
%!assert (fn(2), 4);
</pre></div>

<p>Note that all previous variables and values are lost when a new 
shared block is declared.
</p>
<p>Remember that <code>%!function</code> begins a new block and that 
<code>%!endfunction</code> ends this block.  Be aware that until a new block
is started, lines starting with &lsquo;<samp>%!&lt;space&gt;</samp>&rsquo; will be discarded as comments.
The following is nearly identical to the example above, but does nothing.
</p>
<div class="example">
<pre class="example">%!function <var>a</var> = fn (<var>b</var>)
%!  <var>a</var> = 2*<var>b</var>;
%!endfunction
%! assert (fn(2), 4);
</pre></div>

<p>Because there is a space after &lsquo;<samp>%!</samp>&rsquo; the <code>assert</code> statement does
not begin a new block and this line is treated as a comment.
</p>
<p>Error and warning blocks are like test blocks, but they only succeed 
if the code generates an error.  You can check the text of the error
is correct using an optional regular expression <code>&lt;pattern&gt;</code>.  
For example:
</p>
<div class="example">
<pre class="example">%!error &lt;passes!&gt; error (&quot;this test passes!&quot;);
</pre></div>

<p>If the code doesn&rsquo;t generate an error, the test fails.  For example:
</p>
<div class="example">
<pre class="example">%!error &quot;this is an error because it succeeds.&quot;;
</pre></div>

<p>produces
</p>
<div class="example">
<pre class="example">  ***** error &quot;this is an error because it succeeds.&quot;;
!!!!! test failed: no error
</pre></div>

<p>It is important to automate the tests as much as possible, however
some tests require user interaction.  These can be isolated into
demo blocks, which if you are in batch mode, are only run when 
called with <code>demo</code> or the <code>verbose</code> option to <code>test</code>.
The code is displayed before it is executed.  For example,
</p>
<div class="example">
<pre class="example">%!demo
%! <var>t</var> = [0:0.01:2*pi]; <var>x</var> = sin (<var>t</var>);
%! plot (<var>t</var>, <var>x</var>);
%! # you should now see a sine wave in your figure window
</pre></div>

<p>produces
</p>
<div class="example">
<pre class="example">funcname example 1:
 <var>t</var> = [0:0.01:2*pi]; <var>x</var> = sin (<var>t</var>);
 plot (<var>t</var>, <var>x</var>);
 # you should now see a sine wave in your figure window

Press &lt;enter&gt; to continue: 
</pre></div>

<p>Note that demo blocks cannot use any shared variables.  This is so
that they can be executed by themselves, ignoring all other tests.
</p>
<p>If you want to temporarily disable a test block, put <code>#</code> in place
of the block type.  This creates a comment block which is echoed
in the log file but not executed.  For example:
</p>
<div class="example">
<pre class="example">%!#demo
%! <var>t</var> = [0:0.01:2*pi]; <var>x</var> = sin (<var>t</var>);
%! plot (<var>t</var>, <var>x</var>);
%! # you should now see a sine wave in your figure window
</pre></div>

<p>The following trivial code snippet provides examples for the use of
fail, assert, error and xtest:
</p>
<div class="example">
<pre class="example">function output = must_be_zero (<var>input</var>)
  if (<var>input</var> != 0)
    error (&quot;Non-zero input!&quot;)
  endif
  output = input;
endfunction

%!fail (&quot;must_be_zero (1)&quot;);
%!assert (must_be_zero (0), 0);
%!error &lt;Non-zero&gt; must_be_zero (1);
%!xtest error (&quot;This code generates an error&quot;);
</pre></div>

<p>When putting this a file <samp>must_be_zero.m</samp>, and running the test, we see
</p>
<div class="example">
<pre class="example">test must_be_zero verbose

&rArr;
&gt;&gt;&gt;&gt;&gt; /path/to/must_be_zero.m
  ***** fail (&quot;must_be_zero (1)&quot;);
  ***** assert (must_be_zero (0), 0);
  ***** error &lt;Non-zero&gt; must_be_zero (1);
  ***** xtest error (&quot;This code generates an error&quot;);
!!!!! known failure
This code generates an error
PASSES 4 out of 4 tests (1 expected failures)
</pre></div>

<a name="Block-type-summary_003a"></a>
<h4 class="subsubheading">Block type summary:</h4>

<dl compact="compact">
<dt><code>%!test</code></dt>
<dd><p>check that entire block is correct
</p>
</dd>
<dt><code>%!testif HAVE_XXX</code></dt>
<dd><p>check block only if Octave was compiled with feature HAVE_XXX.
</p>
</dd>
<dt><code>%!xtest</code></dt>
<dd><p>check block, report a test failure but do not abort testing.
</p>
</dd>
<dt><code>%!error</code></dt>
<dd><p>check for correct error message
</p>
</dd>
<dt><code>%!warning</code></dt>
<dd><p>check for correct warning message
</p>
</dd>
<dt><code>%!demo</code></dt>
<dd><p>demo only executes in interactive mode
</p>
</dd>
<dt><code>%!#</code></dt>
<dd><p>comment: ignore everything within the block
</p>
</dd>
<dt><code>%!shared x,y,z</code></dt>
<dd><p>declare variables for use in multiple tests
</p>
</dd>
<dt><code>%!function</code></dt>
<dd><p>define a function for use in multiple tests
</p>
</dd>
<dt><code>%!endfunction</code></dt>
<dd><p>close a function definition
</p>
</dd>
<dt><code>%!assert (x, y, tol)</code></dt>
<dd><p>shorthand for <code>%!test assert (x, y, tol)</code>
</p></dd>
</dl>

<p>You can also create test scripts for built-in functions and your own C++
functions.  To do so, put a file with the bare function name (no .m
extension) in a directory in the load path and it will be discovered by
the <code>test</code> function.  Alternatively, you can embed tests directly in your
C++ code:
</p>
<div class="example">
<pre class="example">/*
%!test disp (&quot;this is a test&quot;)
*/
</pre></div>

<p>or
</p>
<div class="example">
<pre class="example">#if 0
%!test disp (&quot;this is a test&quot;)
#endif
</pre></div>

<p>However, in this case the raw source code will need to be on the load
path and the user will have to remember to type
<code>test (&quot;funcname.cc&quot;)</code>.
</p>
<a name="XREFassert"></a><dl>
<dt><a name="index-assert"></a>Function File: <em></em> <strong>assert</strong> <em>(<var>cond</var>)</em></dt>
<dt><a name="index-assert-1"></a>Function File: <em></em> <strong>assert</strong> <em>(<var>cond</var>, <var>errmsg</var>, &hellip;)</em></dt>
<dt><a name="index-assert-2"></a>Function File: <em></em> <strong>assert</strong> <em>(<var>cond</var>, <var>msg_id</var>, <var>errmsg</var>, &hellip;)</em></dt>
<dt><a name="index-assert-3"></a>Function File: <em></em> <strong>assert</strong> <em>(<var>observed</var>, <var>expected</var>)</em></dt>
<dt><a name="index-assert-4"></a>Function File: <em></em> <strong>assert</strong> <em>(<var>observed</var>, <var>expected</var>, <var>tol</var>)</em></dt>
<dd>
<p>Produce an error if the specified condition is not met.  <code>assert</code> can
be called in three different ways.
</p>
<dl compact="compact">
<dt><code>assert (<var>cond</var>)</code></dt>
<dt><code>assert (<var>cond</var>, <var>errmsg</var>, &hellip;)</code></dt>
<dt><code>assert (<var>cond</var>, <var>msg_id</var>, <var>errmsg</var>, &hellip;)</code></dt>
<dd><p>Called with a single argument <var>cond</var>, <code>assert</code> produces an
error if <var>cond</var> is zero.  When called with more than one argument the
additional arguments are passed to the <code>error</code> function.
</p>
</dd>
<dt><code>assert (<var>observed</var>, <var>expected</var>)</code></dt>
<dd><p>Produce an error if observed is not the same as expected.  Note that
<var>observed</var> and <var>expected</var> can be scalars, vectors, matrices,
strings, cell arrays, or structures.
</p>
</dd>
<dt><code>assert (<var>observed</var>, <var>expected</var>, <var>tol</var>)</code></dt>
<dd><p>Produce an error if observed is not the same as expected but equality
comparison for numeric data uses a tolerance <var>tol</var>.
If <var>tol</var> is positive then it is an absolute tolerance which will produce
an error if <code>abs (<var>observed</var> - <var>expected</var>) &gt; abs (<var>tol</var>)</code>.
If <var>tol</var> is negative then it is a relative tolerance which will produce
an error if <code>abs (<var>observed</var> - <var>expected</var>) &gt;
abs (<var>tol</var> * <var>expected</var>)</code>.  If <var>expected</var> is zero <var>tol</var> will
always be interpreted as an absolute tolerance.  If <var>tol</var> is not scalar
its dimensions must agree with those of <var>observed</var> and <var>expected</var>
and tests are performed on an element-wise basis.
</p></dd>
</dl>

<p><strong>See also:</strong> <a href="#XREFtest">test</a>, <a href="#XREFfail">fail</a>, <a href="Raising-Errors.html#XREFerror">error</a>.
</p></dd></dl>


<a name="XREFfail"></a><dl>
<dt><a name="index-fail"></a>Function File: <em></em> <strong>fail</strong> <em>(<var>code</var>)</em></dt>
<dt><a name="index-fail-1"></a>Function File: <em></em> <strong>fail</strong> <em>(<var>code</var>, <var>pattern</var>)</em></dt>
<dt><a name="index-fail-2"></a>Function File: <em></em> <strong>fail</strong> <em>(<var>code</var>, &quot;warning&quot;, <var>pattern</var>)</em></dt>
<dd>
<p>Return true if <var>code</var> fails with an error message matching
<var>pattern</var>, otherwise produce an error.  Note that <var>code</var>
is a string and if <var>code</var> runs successfully, the error produced is:
</p>
<div class="example">
<pre class="example">          expected error &lt;.&gt; but got none
</pre></div>


<p>Code must be in the form of a string that may be passed by
<code>fail</code> to the Octave interpreter via the <code>evalin</code> function,
that is, a (quoted) string constant or a string variable.
</p>
<p>If called with two arguments, the behavior is similar to
<code>fail (<var>code</var>)</code>, except the return value will only be true if
code fails with an error message containing pattern (case sensitive).
If the code fails with a different error to that given in pattern,
the message produced is:
</p>
<div class="example">
<pre class="example">          expected &lt;pattern&gt;
          but got &lt;text of actual error&gt;
</pre></div>

<p>The angle brackets are not part of the output.
</p>
<p>Called with three arguments, the behavior is similar to
<code>fail (<var>code</var>, <var>pattern</var>)</code>, but produces an error if no
warning is given during code execution or if the code fails.
</p>
<p><strong>See also:</strong> <a href="#XREFassert">assert</a>.
</p></dd></dl>


<hr>
<div class="header">
<p>
Next: <a href="Demonstration-Functions.html#Demonstration-Functions" accesskey="n" rel="next">Demonstration Functions</a>, Up: <a href="Test-and-Demo-Functions.html#Test-and-Demo-Functions" accesskey="u" rel="up">Test and Demo Functions</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p>
</div>



</body>
</html>