% \documentclass[journal]{IEEEtran}
%\documentclass[twocolumn]{IEEEtran}
\documentclass{InsightBook}

% If the IEEEtran.cls has not been installed into the LaTeX system files,
% manually specify the path to it:
% \documentclass[journal]{../sty/IEEEtran}

% \usepackage{graphicx}
\usepackage{cite}
\usepackage{subfigure}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{fancyhdr}
% \usepackage{fncychap}
% \usepackage{float}
\usepackage{amsmath}
\usepackage[tight]{units}

%check if running PDFLaTeX
\newif\ifpdf%
\ifx\pdfoutput\undefined%
  \pdffalse% we are not running PDFLaTeX
  \DeclareGraphicsExtensions{.eps}%
\else%
  % \pdfoutput=1% we are running PDFLaTeX
  \DeclareGraphicsExtensions{.pdf}%
  \pdftrue%
\fi%

\ifpdf%
% yes we have PDFLaTeX, so use .pdf images
\else%
% sorry no PDFLaTeX, use .eps images
\fi%

\begin{document}
\title{Dart: Testing, Reports and Dashboards}
\author{Daniel J. Blezek}
\maketitle

\chapter{Dart v2}
This chapter describes the requirements and design criteria for the
next version of Dart, Dart2 (or Dart v2).

\section{Dart2 Statement of Purpose}
\begin{quote}
\emph{Dart2 shall aggregate data across many independant distributed
build and test hosts, summarizing the software quality aspects of the project
in a concise and informative fashion cross-sectionly and longitudinally.}
\end{quote}

\section{User Requirements}
\begin{enumerate}
\item A single server instance shall process multiple projects, with
simple, flexible configuration and management.
% \item Server configuration and management shall be simple but
% flexible.
\item Presentation of results shall be configurable, allowing results
to persist on the dashboard for different periods.  For instance,
coverage information is time consuming to produce but slowly
changing and ought to persist for more than one day.
\item Dashboards may be aggregated into Meta-Dashboards.  For
instance, Slicer depends on VTK, ITK, gsl and Tcl/Tk.  The Slicer
Meta-Dashboard shall present summary information from these
dependancies.
\item Dart2 shall support submission authentication and selectivly
reject or expire unauthenticated submissions.
\item Dart2 shall provide resource management tools for disk space,
bandwidth and processing time allowing both Clients and Servers to
efficiently manage resources.
\end{enumerate}

\section{Design Requirements}
\paragraph{Basic}
\begin{enumerate}
\item The server shall contain all components required and shall not
require any external packages, nor operating system applications.
The server shall run as a daemon and shall include these components:
	\begin{enumerate}
	\item Scheduler: Dart2 shall include an internal scheduling
system for routine systems tasks, \emph{etc}\ldots
	\item RDBMS: Dart2 shall include an embedded database to
handle small Projects.
	\item Web Server: Dart2 shall include an enmedded web server
to publish dashboard pages.
	\item Web Services: Dart2 shall communicate using an
established protocol for web services, allowing Results submissions
and query of Project status from remote, homogeneous clients.
	\end{enumerate}
\item The server shall be extensible with user supplied components,
including:
	\begin{enumerate}
	\item RDBMS: Dart2 shall use JDBC compliant drivers for all DB
access allowing different database systems such as MySQL, Postgres,
Oracle, \emph{etc.}
	\item Web Server: Apache and other web servers shall be
capable of serving Dart2 generated pages.
	\item Web Services: Dart2 shall allow the ability to
communicate using external web servers such as Apache, Tomcat,
\emph{etc.}
	\item Portal Server: If desired, a Portal server such as
Jetspeed may be used to interface with Dart2 results database.  This
capability is currently unspecified.
	\end{enumerate}
\end{enumerate}
\paragraph{Resource Management (Section~\ref{Sec:ResourceManagement})}
\begin{enumerate}
\item Dart2 shall, as an option, maintain compressed XML files, using
on-the-fly decompression.  This will result in approximately 10:1
spacing savings for the XML. 
\item Dart2 shall provide a policy mechanism to selectively delete
or archive unnecessary Builds.  An archived Build shall consume less
than 10K of disk space by retaining only summary information.
\end{enumerate}

\paragraph{Storage, Processing and Presentation Engines}
\begin{enumerate}
\item Dart2 shall comprise three engines: Storage, Processing and
Presentation.
\item The Storage engine shall accept submissions from clients parse
the input and store results in a generic format with large data items
stored in the file system, \emph{i.e.} images, with numeric and
shorter text information stored in a database system.
\item The Processing engine shall process and summarize the results
organized by the Storage engine at regular intervals and upon
user-defined event triggering processing actions.
\item The Presentation engine shall provide a customizable view of
data: both ``raw'' data from the Storage engine, and from the
Processing engine.  In the first instance, the Presentation engine
shall simply be HTML, potentially migrating to a Portal based server.
\end{enumerate}

\paragraph{Customization}
\begin{enumerate}
\item Dart2 shall provide an easy to modify template engine for
summarizing results.
\item Dart2 shall provide a server side plug in mechanism allowing
custom data aggregration and flexible reporting.
\item Dart2 shall make provide mechanisms for simple localization and
internationalization, where appropriate.
\end{enumerate} 


\subsection{Resource Management}
\label{Sec:ResourceManagement}
\subsubsection{Disk Space}
The Insight toolkit is the largest Dart project to date.  Currently,
with compressed HTML files, one day consumes over 650M of disk space.
This includes (from November 4, 2004):
\begin{itemize}
\item One Doxygen run (400K for XML, 13K for HTMLZ)
\item One Master Update (13K for XML, 4K for HTMLZ)
\item One Dashboard (12M for XML, 9K for HTMLZ)
\item One BuildOverview (250K for XML, 8K for HTMLZ)
\item One TestOverview (25M for XML, 2 x 29K for HTMLZ)
\item One Coverage build (24M for XML, 21M for HTMLZ)
\item Builds (average of 9M for XML and HTMLZ)
\begin{itemize}
 \item 52 Nightly Builds
 \item 19 Continuous Builds
 \item 22 Experimental Builds
\end{itemize}
\end{itemize}

Breaking down an example day, we have:

\begin{centering}
\begin{tabular}{|l|l|l|}
\hline
File & Size & Notes \\
\hline
Build.xml & 140K & 111 Warnings, average of 1.2K per warning \\
Configure.xml & 0.8K & \\
Test.xml & 4.3M & 859 Tests, average of 5K per test \\
Uptate.xml & 1.4K & 1 Updake, 1.4K per updated file \\
TestSummary.xml & 215K & \\
All.htmlz & 86K & Total for 8 HTMLZ files \\
\hline
\end{tabular}
\end{centering}

The largest generator of data is test output.  Errors/Warnings and
Update information are rather verbose, capturing context information.
In general, XML is verbose with low entropy.  A 4.3M Test.xml file
is 522K compressed with gzip.

\subsubsection{Dashboard Generation Time}

\subsubsection{Bandwidth}

\subsection{Historical Data}
Dart currently perserves data from previous days, it is not
linked across temporal Builds on the same system.  While simple, this
 restriction increases the difficulty of monitoring the quality of a
project.  To overcome this limitation, Dart2 shall link data in a
temporal fashion.
\subsection{Hierarchical Data}
  \subsubsection{Dashboards}
  \subsubsection{Tests}
  \subsubsection{Builds}

\subsection{Persistance of Builds/Results}
\subsubsection{Stream Concept}

\subsection{Documentation}

\subsection{Submissions}
\subsubsection{Incremental Submission}
\subsubsection{Mechanisms}
\subsubsection{Authentication}

\subsection{Configuration}
\subsubsection{Initial Setup}
\subsubsection{Options}

\subsection{Customization}
  \subsubsection{Dashboard presentation}
\subsection{Extensibility}

\chapter{Implementation Ideas}
This section captures some implementation ideas.
\section{Server}

\paragraph{Language}
Of all cross-platform languages, Java provides the most robust set of libraries suitable for
Dart2.  Java also allows simple distribution of compiled libraries,
\emph{i.e.} jar files, as plug-ins.  Potentially, a client could query
the server for a list of available plug-ins downloading and installing
as needed.

\paragraph{RDBMS}
There are several embedable Java RDBMS available,
two of the more interesting projects are Cloudscape, recently released
from IBM, and renamed Derby on the Apache site and Hypersonic SQL
(HSQLDB) project hosted on SourceForge.  Dart2 is envisioned to have a
RDBMS holding summary data; embedding a database into the server
should help to make it transparent and invisible to the casual Dart2
user.  For more scalability, the backing store could be any
RDBMS with a JDBC driver.  MySQL and Postgres come to mind.

\paragraph{Transport} 
Though over-designed and complex, SOAP has the elements need to
transmit XML files to the server from the client.  Specifically, SOAP
with attachements could deliver chunks of compressed XML to the server
via HTTP, since most (all?) firewalls allow HTTP traffic. SOAP could
also be used for Dashboard to Dashboard (D2D?)  communication and
remote management and monitoring of Dart2 servers.  XML-RPC is a much
simpler API, and identically suitable.  XML-RPC will be considered at
the same level as SOAP. Another possible use is dissimination of
plug-ins for clients.  The Java Messaging Service~(JMS) is another
possibility.  JMS gives great flexibility to transport mechanisms and
can operate asyncroniously.

\paragraph{Scheduling}
Quartz is an open source enterprise strength
scheduling system for Java.  Quartz will drive scheduled events such
as Dashboard roll ups, DB tasks, and archiving/deletion of old
results.  Quartz will replace cron.

\paragraph{Template Engine}
There are several competing Template engines for Java.  Velocity is an
Apache sponsored project and has some great features including close
integration with other Apache projects.  FreeMarker is another engine
that is more sophisticated than Velocity, but not as integrated.  The
Template engine will be the driver to produce HTML and other reports
replacing XSLT.

\paragraph{Jakarta}  The Apache Jakarta project provides several
packages of immediate use.
\begin{itemize}
\item Digester builds objects from XML,
greatly simplifying configuration from XML files.  Each object is
constructed as needed and automatically configured.
\item CLI should provide a great command line parsing interface.
\item Commons eMail provides a simple java email client.
\item ORO and RegExp, two regular expression packages.
\end{itemize}

\paragraph{Portal}  Though the current Dart HTML pages serve the
purpose well, adding a portal on top would allow custom portlets to be
developed for specific purposes.  For instance, one portlet could be
configured to show a particular build over the last several days, or
perhaps graph the performance of a Test or Result through time across
several architectures.  Dynamic generation of all the Dart2 results
places undo burden on the server, where a Portal could dynamically
generate limited data in an efficient manner.  One Portal project that
is interesting is Jetspeed 2, an Apache sponsored project.

Portals do add administrative overhead.  It is preferable to
have the ability to use Dart2 without a Portal, but easily being able
to add the increased utility if desired.


\section{Client}


\end{document}