		PLPLOT DOCUMENTATION IN THE DOCBOOK FORMAT
		    (Notes for contributors/developers)

Introduction
============

You will find in this directory tree the latest PLplot documentation in
DocBook 4.1 XML format.  You will find in the current file
(README.developers) instructions for building the documentation from the
DocBook source files in this directory. The documentation that is built
includes man pages of the API as well as complete versions of the
documentation in info, HTML, and PDF forms.

Configuring and Building
========================

If you would like to help with improving the content, you will want to build
the documentation from the DocBook source so you can immediately see and
evaluate the results of your efforts.

Here is how.  Simply add the cmake option -DBUILD_DOC=ON to your
normal cmake command-line options for the PLplot build.  The resulting
cmake step looks for all the tools required for the documentation
build and generates a WARNING message and turns off all or some
component of the documentation build if any of those required tools
are missing.  The tools searched for by our build system are onsgmls
(validation); db2x_texixml, db2x_xsltproc, and makeinfo (info);
XML::Parser and XML::DOM (man); xmlto (html); and dblatex and xelatex
(pdf).

After the cmake step using -DBUILD_DOC=ON you must run either

make -j4 all >& all.out

or

make -j4 build_docbook_documentation >& build_docbook_documentation.out

to build the DocBook documentation.  (The former builds all aspects of
PLplot including the documentation, while the latter is quicker
because it just builds the documentation and its PLplot
prerequisites.)  Furthermore, at build-time xmlto and dblatex
automatically check for required tools, and dblatex checks for missing
fonts when generating the pdf results.  (Those required fonts are
chosen by doc/docbook/source/dblatex_stylesheet.xsl and are currently
FreeSans, FreeSerif, and FreeMono).

In sum, cmake WARNING messages with the -DBUILD_DOC=ON option _and_
build-time messages from xmlto and dblatex should be sufficient for you
to figure out what DocBook/XML related tools and fonts you need to
install in order to build the documentation.

N.B. As far as we know, the complete set of required tools is only
available on Linux and Cygwin so you will need access to one of those
platforms to do a documentation build.  For more information on
those platforms see "Linux Specific" and "Cygwin Specific" below.

Linux Specific
==============

Installing the following packages on Debian should allow you
to build the documentation (this may not be a complete list,
but it should be close to complete since installation
of one Debian package also automatically installs all
Debian packages it depends on).

You can use, e.g., apt-file search bin/xmlto to discover
that

/usr/bin/onsgmls is contained in the "opensp" Debian package;
/usr/bin/db2x_texixml is contained in the "docbook2x" Debian package;
/usr/bin/db2x_xsltproc is contained in the "docbook2x" Debian package;
/usr/bin/makeinfo is contained in the "texinfo" Debian package;
/usr/lib/perl5/XML/Parser.pm (which implements XML::Parser) is contained in the "libxml-parser-perl" Debian package;
/usr/share/perl5/XML/DOM.pm (which implements XML::DOM) is contained in the "libxml-parser-perl" Debian package;
/usr/bin/xmlto is contained in the "xmlto" Debian package;
/usr/bin/dblatex is contained in the "dblatex" Debian package;
/usr/bin/xelatex is contained in the "texlive-xetex" Debian package;
/usr/share/texlive/texmf-dist/fonts/truetype/public/gnu-freefont/FreeSans.ttf is in the "texlive-fonts-extra" Debian package
/usr/share/texlive/texmf-dist/fonts/truetype/public/gnu-freefont/FreeSerif.ttf is in the "texlive-fonts-extra" Debian package
/usr/share/texlive/texmf-dist/fonts/truetype/public/gnu-freefont/FreeMono.ttf is in the "texlive-fonts-extra" Debian package

With these in place the documentation should build with the apropriate
CMake flag set on Debian and other Debian-based distros such as
Ubuntu.

PLplot developers have no recent experience with rpm-based Linux
distros, but Orion Poplawski (the Fedora packager for PLplot and a lot
of other free software) has recently had success building the PLplot
documentation on Fedora.  We presume he used command-line rpm tools
similar to Debian's apt-file or else usrf generic rpm-finding software
such as <http://www.rpmfind.net/linux/RPM/> to search rpm packages
for particular filenames similar to the above.  Presumably a
similar situation would apply for other rpm-based distros such
as openSUSE.

Cygwin Specific
===============

Installing the following packages on Cygwin will allow you
to build the documentation (note this may include more
packages than strictly required, but this list definitely
works):
openjade, OpenSP, texlive-collection-htmlxml,
texlive-collection-genericrecommended,
texlive-collection-latexrecommended,
texlive-collection-fontsextra, docbook-dsssl, lynx,
ImageMagick, libxml2.

You must then make the gnu FreeFonts available to fontconfig. You can do
this by creating a file /etc/fonts/local.conf which should contain the
following:
     <?xml version="1.0"?>
     <!DOCTYPE fontconfig SYSTEM "fonts.dtd">

     <fontconfig>

     <!-- Font directory list -->

             <dir>/usr/share/texmf-dist/fonts/opentype</dir>
             <dir>/usr/share/texmf-dist/fonts/truetype</dir>
             <dir>/usr/share/texmf-dist/fonts/type1</dir>


     </fontconfig>
Note also that if you wish you could copy the FreeFonts from the opentype
subdirectory of the folder listed above to anywhere else (e.g. ~/.fonts or
your Windows fonts directory) and add this to the local.conf file or you can
copy them to /usr/share/fonts/OTF in which case a local.conf file is not
needed.
Finally run fc-cache to update the fontconfig cache. Now the documentation
should build with the apropriate CMake flag set.


The DocBook Back-end Tool Chains
================================

PLplot uses a number of different applications to generate
PLplot documentation in various formats from our source DocBook XML files.

1.  man pages.

Our man pages are generated with a configured home-brew perl script
bin/api2man.pl(.in). That script uses XML::DOM::Parser to parse the
combination of plplotdoc.xml (for entity definitions) and api.xml
(the subset of our DocBook source files which describes our core
library's API) to obtain the information used to generate the man
pages.

(Aside. The bin subdirectory also contained at one time api2text.pl which
was used by the non-swig version of the Octave bindings to generate
text documentation for the octave PLplot commands.  That script has
now been removed and replaced by the closely related
api2swigdoc.pl which generates bindings/swig-support/swig_documentation.i
API documentation in a form which can be used by swig (for both
the Python and Octave bindings).

2. info pages.

Our info pages are generated by a combination of 

bin/info-clean.pl --> db2x_xsltproc --> db2x_texixml --> makeinfo

where info-clean.pl is a home-brew perl script required (as far as I
can tell from reading comments in it) to get around some of the
db2x_xsltproc limitations.  db2x_xsltproc and db2x_texixml are from
the docbook2x package.

3. Our HTML results are generated with xmlto and configured at run
time with a CSS stylesheet, see doc/docbook/src/stylesheet.css.xsl.in.
The xmlto application is actually a convenience script that uses
XML/XSL to generate the HTML results as opposed to the previous
SGML/DSSSL approach used to create the HTML results for plplot-5.9.9
and previous.

4. Our PDF results are generated with "dblatex --backend=xetex" using
a combination of XML/XSL and xelatex to generate the PDF results (in
contrast to the SGML/DSSL tools used for that job for plplot-5.9.9 and
previous, and the XML/XSL "xmlto --with-fop" method used with
PLplot-5.9.10 to do that job.

5. Generation of dvi and PostScript forms of our documentation is no longer available.


Validation
==========

Validation checks that the PLplot Docbook files in doc/docbook/src are
well-formed XML and correspond to the DocBook schema.  You can perform
such validation by typing the

make validate

command at the top of the build tree.  This make target is only available if
the PLplot CMake-based build system has found the onsgmls software
application (which is distributed as part of OpenSP) on your system. This
quick check works regardless of whether you decide to build the
documentation with -DBUILD_DOC=ON or not. Using "make validate" is
especially useful if you are just making a series of simple changes to the
files in doc/docbook/src, and you don't really feel it is necessary to check
every change by doing a complete documentation build.

In addition, the xmlto commands used to build the HTML and print
part of the documentation automatically include a validation step
with xmllint.  It turns out that xmllint is more sensitive than onsgmls
to DocBook XML errors.  On the other hand, onsgmls handles any
errors it finds with ease while xmllint tends to segfault when there
is a validation error.  So use "make validate"
first to detect any obvious validation errors using onsgmls to avoid
situations where xmlto's call to xmllint will segfault.

Testing the documentation that has been built.
==============================================

All tests are performed in the doc/dockbook/src subdirectory _of the
build tree_.  The given test commands are for the bash shell, and
$version is currently 5.9.10.  For the man pages, look carefully at
the style of the results.  For the rest, look carefully at the style
of the api chapter and the style/results near Table-3.4 in the
advanced chapter.  In particular look at how the following examples
render: the overline-underline example just prior to Table-3.4,
Table-3.4 of Greek symbols, Table-3.5 of the "Peace" word expressed in
various languages, and the mathematical symbols occurring in the
paragraph just after Table-3.5.

1. man pages.

I cannot figure out how to get nroff to work properly with utf8 man pages
so use the man command to test the generated man pages as follows:

From the top of the build tree....

mkdir -p man/man3
ln -s $(find $(pwd) -name "*.3plplot") man/man3
for MANPAGE in $(find man/man3 -type l|sort|grep -E '(plot3dcl|plsurf3dl)'); do MANPAGE1=${MANPAGE##*/}; man -M./man ${MANPAGE1%%.3plplot} ; done

Two of our man pages (plot3dcl.3plplot and plsurf3dl.3plplot)
currently contain utf8 symbols (for the math symbols ≤ and ≥) so I have selected those in the
above example, but if you prefer to look at all man pages that "|grep -E '(plot3dcl|plsurf3dl)'"
stanza should be dropped.

2. info pages.

info --file doc/docbook/src/plplotdoc.info

3. web pages.

Browse with your favorite browser the
doc/docbook/src/index.html file in your
build tree.

4. pdf file

Browse with your favorite browser the 
doc/docbook/src/plplot-$version.pdf file
that is generated in your build tree.

UTF-8 limitations of the current set of backend tools
=====================================================

The above set of DocBook backend tools has been chosen with the goal
of allowing essentially arbitrary UTF-8 strings into our DocBook
source.  And the advanced.xml part of our DocBook source constitutes a
simple test of how close we have come to that goal by including the
"Peace" word in all the human languages expressed in example 24 and by
also including some UTF-8 forms of math symbols.  Here are the current
results for how well those UTF-8 strings render for our various
backends.

Man pages.

These results are not affected since the man pages ignore the advanced
chapter, but (see discussion above) the api.xml file currently
contains two functions whose documentation uses UTF-8 math symbols,
and for those man pages the man command renders those symbols without
issues.  I assume what is going on here is the UTF-8 math symbols
are just being cleanly passed through to the man viewer ("less" in
this case) so with some exceptions (see comments on less below) good
rendering of arbitrary UTF-8 symbols will occur.

Info files.

These results are reasonably good.  All math glyphs and all but the
glyphs occurring in the Korean, Hindi, and Mandarin "Peace" words are
rendered correctly.  That's exactly the same set of missing glyphs
that occurs when I use "less" on advanced.xml so it is possible some
configuration adjustment for the non-GUI component of my system will
fix the missing glyphs that occur both for the info form of our
documentation and when using "less".

HTML results.

These are outstanding results with no issues that we are aware of.
All math glyphs and all the glyphs occurring in the set of "Peace"
words render without issues including the CTL languages like Hebrew,
Arabic, and Hindi.

PDF results.

These results are reasonably good.  All math glyphs and all but the
glyphs occurring in the Korean and Mandarin "Peace" words are present.
I attribute the missing glyph issues to missing Korean and Mandarin
glyphs in the chosen FreeSans, FreeSerif, and FreeMono fonts (more
about that choice below). The order of the glyphs in the Hindi Peace
word is not correct (last two glyphs switched) which is a common
complex text layout (CTL) issue when using unsophisticated software to
render Hindi.  However, the Peace word is laid out in the correct
(right-to-left) order for Arabic and Hebrew so there is some CTL
sophistication in the xelatex layout engine, and probably hope that
further layout issues such as the one for Hindi will be fixed.

More on the reason for the missing PDF glyphs.

It was a huge step forward in the tex world for the combination of
xelatex and the fontspec tex package to make a very large number of
UTF-8 glyphs potentially available in pdf results.  So I think the
above method of using dblatex with the xetex backend (which uses
xelatex and fontspec internally) is the best we can do to generate the
pdf form of our documentation as free as possible from unicode issues.
For example, the situation is much improved over the previous xmlto
method which severely limited the glyphs in the PDF form of our
results.

However, the fontspec package still has a fundamental limitation which
is you must use a specific rather than generic font for the sans,
serif, and mono cases. Specific fonts are never a good way to go for
documents containing different languages (such as the table of "Peace"
words I recently introduced into advanced.xml) since the user is
forced to compromise between font quality and comprehensiveness.  For
example, I have tried using the unifont font (which gives complete
coverage of glyphs for the basic multilingual plane), and the
resulting pdf form of our documentation has no missing glyphs.
However, that font is of extremely poor quality (scaled,
low-resolution bitmapped, monospaced fonts) which makes our
documentation look ugly so I am not going to use it.  Instead, I have
chosen (via doc/docbook/src/dblatex_stylesheet.xsl) to use FreeSans,
FreeSerif, and FreeMono fonts which have reasonable quality but which
have some missing glyphs (notably CJK glyphs that leave our Korean and
Mandarin "peace" words empty in the documentation).

The proper way to solve this issue (which would bring tex completely
into the modern unicode world) would be to modify fontspec so that
certain given font names are considered generic, (e.g., the
"sans-serif", "serif", and "mono-space" font names could be adopted
for this purpose to follow what is done in the SVG world).  For those
generic font names, the idea is the fontspec package would simply hand
off font selection to fontconfig which does a very nice job of
automatically selecting the ideal sans, serif, or mono system font to
provide a particular UTF-8 glyph.  I have asked the fontspec
maintainer about this potential feature several days before the
current writing, but so far he has not replied.  Therefore, it appears
at this time there will be no quick fixes to xelatex/fontspec with
regard to automatic selection of fonts for the generic case, and we
will be stuck for the foreseeable future with the specific font
approach.  That approach forces a compromise between complete glyph
coverage and reasonable quality.  Thus, as a result of avoiding truly
ugly fonts in our generated pdf documentation there are missing CJK
glyphs in the results.

Installing the generated documentation (and the rest of the generated website) at plplot.sf.net
===============================================================================================

Follow the directions in README.Release_Manager_Cookbook.  That file is
located in the top-level source tree.
