1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274
|
Test units for pasdoc.
Some short and difficult Pasdoc sources. Some of these are taken
from bugreports, to make sure we don't reintroduce an old bug again.
Some of these are written to test new features.
To add a new testcase,
1. first of all just add the file here
(follow the naming conventions below, usually just ok_xxx.pas).
2. make it used by the "make html" and such tests by adding
an appropriate line at the end of scripts/mk_tests.sh.
3. after you make sure it works, you may upload new
version of "correct tests", see below. Or just notify Michalis about it :)
Using these tests fully requires having some standard Unix tools
installed and available on $PATH. You need GNU `make`, bash, diff.
Also wget is used for downloading correct tests for comparison.
And tar, scp are used for uploading correct tests by developers.
Of course, you also need the `pasdoc` binary available on $PATH.
--------------------
Naming of Pascal unit files in this directory:
ok_*
Means that this unit should be parsed by pasdoc without any warnings.
warning_*
Means that this unit should be parsed by pasdoc (i.e. some documentation
for it should be generated), but some warnings should be reported
by pasdoc.
error_*
Means that pasdoc should not be able to parse this unit,
documentation for this unit shouldn't be possible to generate.
pasdoc should generate proper error message for this case.
Note that pasdoc may report errors as warnings,
e.g. "Warning[2]: Error EPasDoc: error_line_number.pas(26):
could not open include file not_existing.inc ..."
Pasdoc calls this a warning, since, after all, it can continue
it's work by simply skipping to the next unit.
But for the sake of this distinction, this is an *error*,
not merely a *warning*.
The precise difference between an error and a warning is:
"error makes impossible to generate documentation for this unit".
Units are divided into these 3 groups because
-- These groups are precisely defined, so there shouldn't be any concern
about "where this test unit belongs".
-- We should be eventually able to use output messages and exit status
of pasdoc to automate testing as much as we can.
Notes:
-- Please keep prefixes "ok_", "warning_", "error_" lowercase so that
e.g. the file-mask `ok_*' works as expected on case-sensitive file-systems.
Try to follow the convention
"prefix_description_of_test_lowercase_with_underscores.pas".
-- Most of these units will have empty implementation.
They are not supposed to be ever compiled by anything.
-- There is no requirement here that the interface of units
placed here must be correct Pascal code.
Pasdoc should be able to handle even incorrect units.
Usually it should break with some parsing error on such units,
but it doesn't have to (after all, pasdoc is not meant to
exactly reimplement some part of a compiler that carefully checks
syntax and everything), it may try to generate some docs.
But even with incorrect units, it obviously shouldn't fail with some
miserable error (like sigsegv :) or do anything that it shouldn't,
like removing some files or so.
-- If you want to test unit with a special pasdoc's command-line,
you must add appropriate line at the end of ./mk_tests.sh script.
--------------------
Possible tests to be done:
1. Generate tests output by simple
make
Default output is html, you should also try other output
formats, e.g. latex or even pdf (made from latex), like
make latex
make pdf
Tests are created in subdirectories,
e.g. html/ok_const_1st_comment_missing/ and
latex/ok_const_1st_comment_missing/ will contain output for
ok_const_1st_comment_missing.pas unit.
To automatically test this output, see below,
for "correct tests output", "validate_html", "validate_simplexml".
But you can of course just manually look at units'
sources and comments there to know what generated documentation should
look like (in case of ok_* and warning_* files) and what
warnings/errors should be reported (in case of warning_* and error_* files).
Of course, even briefly checking that all ok_* units generate no warnings,
all warning_* units generate some warnings (and produce some docs)
and all error_* units generate errors (and no docs)
is still a better test than nothing...
Note that pasdoc messages (printed on stdout) will not be shown
when you will make tests using mentioned `make ...' commands.
Instead they will be saved to files named PASDOC-OUTPUT
in appropriate subdirectories of tests output.
This way we treat messages printed by pasdoc as important part
of pasdoc's output, they are included in "correct tests output"
(see below) so comparing with "correct tests output" also checks
that pasdoc messages are correct (warnings are as they should be
and where they should be etc.)
2. "Correct tests output" try to give more automated way to exactly
check documentation of test units here.
We maintain on [http://pasdoc.sourceforge.net/correct_tests_output/]
something that we call "correct output of tests".
Everyone can download them simply by entering scripts/ directory
and executing
./download_correct_tests_output.sh html
where you can replace `html' with any pasdoc's output format name.
See documentation inside scripts/download_correct_tests_output.sh
for more detailed docs.
Then whenever you want you can compare current documentation generated
by pasdoc to downloaded "correct output"
e.g. using
diff -ur correct_output/html/ html/
Note that *failure* of comparison does not necessarily mean that
current pasdoc *has* a bug, since it's possible that current output of
pasdoc is different (because e.g. we improved and changed something)
but still correct.
Also *success* of comparison does not mean that current pasdoc does
*not* have a bug, because this "current output" was just
accepted at some point by human as "correct".
So, this is unfortunately no ultimate test.
But in practice it often works great, and allows us to quickly
check that nothing is broken :) When no output changes are expected,
or the changes are very local, then looking at "diff" output is a great
and simple way to test.
If pasdoc's developer sees at any point that "current output" differs
from "correct output" and also he carefully checked that "current
output" is still correct, than he should upload his "current output"
as new "correct output" by entering scripts/ directory and executing
./upload_correct_tests_output.sh username html
See documentation inside scripts/upload_correct_tests_output.sh
for more detailed docs. Note that before uploading you should
be relatively sure that you compared against most recent version
of "correct output", to save yourself trouble of unnecessary
uploading.
Beware that script `upload_correct_tests_output.sh'
will upload everything that you will have inside subdirectory
of given format. This is painful in case of output like LaTeX,
where you can accidentally upload files like docs.pdf or docs.dvi
(that shouldn't be uploaded).
In this case you can do something like `make clean latex' before
uploading to be sure that you upload only "docs.tex" file.
3.1. make html validate_html
This is an automatic test that makes html docs for all test units
and validates them using sgml validator.
onsgmls program must be installed for this to work.
Links how to install onsgmls may be found here
[http://pasdoc.sipsolutions.net/HtmlOutput].
3.2. make simplexml validate_simplexml
This is an automatic test that makes simplexml docs for all test units
and validates them using xmllint.
xmllint must be installed for this to work.
We do not have any DTD, so it doesn't check that our XML files
conform to anything. But at least it checks that they are well-formed
(all open elements are closed and such).
4. cd scripts/
./check_cache.sh html
./check_cache.sh latex
$1 arg for this script is required and is any valid pasdoc format
(as for pasdoc's --format option).
Rest of args are just passed to pasdoc as they are.
This runs pasdoc twice using the cache,
1st time the cache directory is empty and pasdoc writes the cache,
2nd time pasdoc should read everything from the cache.
The script then checks that pasdoc's output was identical each time
(comparing them with `diff -u', so you get diff on output if something
does not match).
This somehow checks that writing the cache and reading the cache
and writing documentation basing on information obtained from the cache
(instead of from parsing units) works OK.
Everything is read/written to a temporary directory scripts/check_cache_tmp/,
that is removed at the beginning and at the end of the script.
(It's removed at the beginning, and also by `make clean', just to
make sure that no garbage is left there, in case script failed
for whatever reason.)
So this script is mostly independent from the rest of the tests
here -- it just happens to use the same test units.
In case comparison between two outputs failed both outputs
and left in scripts/check_cache_tmp/, so developer can inspect
them closer.
5. cd scripts/
./check_cache_format_independent.sh html latex
./check_cache_format_independent.sh latex html
Requires two arguments, two names of pasdoc output format.
These two formats should be different for this test to be really sensible
(and better than check_cache.sh), but they can also be equal and
test should pass anyway.
This is similar to ./check_cache.sh test, but it checks
that cache format is independent of pasdoc's output format.
1st it generates output with format 1, without using any cache.
2nd it generates output with format 2, writing the cache.
3rd it generates output with format 1, this time reading the cache.
Then it checks that 1st and 3rd output are equal.
This way it checks that cache generated while doing format 2
may be reused while making format 1. So it tests that cache
format is really independent from pasdoc's chosen output format.
--------------------
Random notes:
`make clean' will clean this directory.
Note that make used must be GNU make.
Under Linux this is standard, under FreeBSD this is called `gmake',
under Win32 you can get this with e.g. FPC, MinGW or Cygwin.
scripts/ subdirectory contains some helpful things for running tests.
These should be scripts that perform some additional tests
on test units available here (like check_cache.sh),
but also some helper scripts/files for Makefile in this directory.
--------------------
Subdirectory todo/
It contains units that are known to
be incorrectly handled by pasdoc by now. "Incorrectly handled"
may mean that generated documentation is incorrect, or that pasdoc
fails with some error on correct input, but "incorrectly handled"
may also mean that pasdoc fails to write a proper warning/error
in case when input (unit's sources) is obviously wrong.
Files inside todo/ should follow exactly the same naming convention
as units in this directory (ok_*, warning_*, error_*).
In this case, unit's name tells what pasdoc *should* do with such unit,
even if it doesn't do it for now.
The idea is that when developer fixes a problem with some unit
in tests/todo/ directory, he can simply move this unit to tests/.
These files are in separate todo/ subdirectory, because otherwise
every time we would like to check our tests we would have to
remember "oh, such-and-such test fails but it's a known problem,
so I can ignore it". This would be troublesome,
because *usually* we will want to test whether we did not break
anything that previously worked, and we will not care that there
are still some unresolved problems in pasdoc.
--------------------
And that's all for now.
|