Quixote Programming Overview
============================

This document explains how a Quixote application is structured.  Be sure
you have read the "Understanding the demo" section of demo.txt first --
this explains a lot of Quixote fundamentals.

There are three components to a Quixote application:

1) A driver script, usually a CGI or FastCGI script.  This is
   the interface between your web server (eg., Apache) and the bulk of
   your application code.

   The driver script is responsible for creating a Quixote publisher
   customized for your application and invoking its publishing loop.

2) A configuration file.  This file specifies various features of the
   Publisher class, such as how errors are handled, the paths of
   various log files, and various other things.  Read through
   quixote/config.py for the full list of configuration settings.
   
   The most important configuration parameters are:

      ``ERROR_EMAIL``
        e-mail address to which errors will be mailed
      ``ERROR_LOG``
        file to which errors will be logged

   For development/debugging, you should also set ``DISPLAY_EXCEPTIONS``
   true and ``SECURE_ERRORS`` false; the defaults are the reverse, to
   favour security over convenience.

3) Finally, the bulk of the code will be in a Python package or
   module, called the root namespace.  The Quixote publisher will be set
   up to start traversing at the root namespace.


Driver script
-------------

The driver script is the interface between your web server and Quixote's
"publishing loop", which in turn is the gateway to your application
code.  Thus, there are two things that your Quixote driver script must
do:

* create a Quixote publisher -- that is, an instance of the Publisher
  class provided by the quixote.publish module -- and customize it for
  your application

* invoke the Quixote publishing loop by calling the 'publish_cgi()'
  method of the publisher

The publisher is responsible for translating URLs to Python objects and
calling the appropriate function, method, or PTL template to retrieve
the information and/or carry out the action requested by the URL.

The most important application-specific customization done by the driver
script is to set the root namespace of your application.  Broadly
speaking, a namespace is any Python object with attributes.  The most
common namespaces are modules, packages, and class instances.  The root
namespace of a Quixote application is usually a Python package, although
for a small application it could be a regular module.

The driver script can be very simple; for example, here is a
trimmed-down version of demo.cgi, the driver script for the Quixote
demo::

    from quixote import enable_ptl, Publisher
    enable_ptl()
    app = Publisher("quixote.demo")
    app.setup_logs()
    app.publish_cgi()

(Whether you install this as ``demo.cgi``, ``demo.fcgi``, ``demo.py``,
or whatever is up to you and your web server.)

That's almost the simplest possible case -- there's no
application-specific configuration info apart from the root namespace.
(The only way to make this simpler would be to remove the
``enable_ptl()`` and ``setup_logs()`` calls.  The former would remove
the ability to import PTL modules, which is at least half the fun with
Quixote; the latter would disable Quixote's debug and error logging,
which is very useful.)

Here's a slightly more elaborate example, for a hypothetical database of
books::

    from quixote import enable_ptl, Publisher
    from quixote.config import Config

    # Install the PTL import hook, so we can use PTL modules in this app
    enable_ptl()

    # Create a Publisher instance with the default configuration.
    pub = Publisher('books')

    # Read a config file to override some default values.
    pub.read_config('/www/conf/books.conf')

    # Setup error and debug logging (do this after read_config(), so
    # the settings in /www/conf/books.conf have an effect!).
    pub.setup_logs()

    # Enter the publishing main loop
    pub.publish_cgi()

The application code is kept in a package named simply 'books' in this
example, so its name is provided as the root namespace when creating the
Publisher instance.

The SessionPublisher class in quixote.publish can also be used; it
provides session tracking.  The changes required to use
SessionPublisher would be::

    ...
    from quixote.publish import SessionPublisher
    ...
    pub = SessionPublisher(PACKAGE_NAME)
    ...

For details on session management, see session-mgmt.txt.

Getting the driver script to actually run is between you and your web
server.  See the web-server.txt document for help, especially with
Apache (which is the only web server we currently know anything about).


Configuration file
------------------

In the ``books.cgi`` driver script, configuration information is read
from a file by this line::

    pub.read_config('/www/conf/books.conf')

You should never edit the default values in quixote/config.py, because
your edits will be lost if you upgrade to a newer Quixote version.  You
should certainly read it, though, to understand what all the
configuration variables are.

The configuration file contains Python code, which is then evaluated
using Python's built-in function ``execfile()``.  Since it's Python code,
it's easy to set config variables::

    ACCESS_LOG = "/www/log/access/books.log" 
    DEBUG_LOG = "/www/log/books-debug.log"
    ERROR_LOG = "/www/log/books-error.log"

You can also execute arbitrary Python code to figure out what the
variables should be.  The following example changes some settings to
be more convenient for a developer when the ``WEB_MODE`` environment
variable is the string ``DEVEL``::

    web_mode = os.environ["WEB_MODE"]
    if web_mode == "DEVEL":
        DISPLAY_EXCEPTIONS = 1
        SECURE_ERRORS = 0
        RUN_ONCE = 1
    elif web_mode in ("STAGING", "LIVE"):
        DISPLAY_EXCEPTIONS = 0
        SECURE_ERRORS = 1
        RUN_ONCE = 0
    else:
        raise RuntimeError, "unknown server mode: %s" % web_mode

At the MEMS Exchange, we use this flexibility to display tracebacks in
``DEVEL`` mode, to redirect generated e-mails to a staging address in
``STAGING`` mode, and to enable all features in ``LIVE`` mode.


Logging
-------

Every Quixote application can have up to three log files, each of
which is selected by a different configuration variable:

* access log (``ACCESS_LOG``)
* error log (``ERROR_LOG``)
* debug log (``DEBUG_LOG``)

If you want logging to work, you must call ``setup_logs()`` on your
Publisher object after creating it and reading any application-specific
config file.  (This only applies for CGI/FastCGI driver scripts, where
you are responsible for creating the Publisher object.  With mod_python
under Apache, it's taken care of for you.)

Quixote writes one (rather long) line to the access log for each request
it handles; we have split that line up here to make it easier to read::

    127.0.0.1 - 2001-10-15 09:48:43
      2504 "GET /catalog/ HTTP/1.1"
      200 'Opera/6.0 (Linux; U)' 0.10sec

This line consists of:

* client IP address
* current user (according to Quixote session management mechanism,
  so this will be "-" unless you're using a session manager that
  does authentication)
* date and time of request in local timezone, as YYYY-MM-DD hh:mm:ss
* process ID of the process serving the request (eg. your CGI/FastCGI
  driver script)
* the HTTP request line (request method, URI, and protocol)
* response status code
* HTTP user agent string (specifically, this is
  ``repr(os.environ.get('HTTP_USER_AGENT', ''))``)
* time to complete the request

If no access log is configured (ie., ``ACCESS_LOG`` is ``None``), then
Quixote will not do any access logging.

The error log is used for two purposes:

* all application output to standard error (``sys.stderr``) goes to
  Quixote's error log
* all application tracebacks will be written to Quixote's error log

If no error log is configured (with ``ERROR_LOG``), then both types of
messages will be written to the stderr supplied to Quixote for this
request by your web server.  At least for CGI/FastCGI scripts under
Apache, this winds up in Apache's error log.

The debug log is where any application output to stdout goes.  Thus, you
can just sprinkle ``print`` statements into your application for
debugging; if you have configured a debug log, those print statements
will wind up there.  If you don't configure a debug log, they go to the
bit bucket (``/dev/null`` on Unix, ``NUL`` on Windows).


Application code
----------------

Finally, we reach the most complicated part of a Quixote application.
However, thanks to Quixote's design, everything you've ever learned
about designing and writing Python code is applicable, so there are no
new hoops to jump through.  The only new language to learn is PTL, which
is simply Python with a novel way of generating function return values
-- see PTL.txt for details.

An application's code lives in a Python package that contains both .py
and .ptl files.  Complicated logic should be in .py files, while .ptl
files, ideally, should contain only the logic needed to render your Web
interface and basic objects as HTML.  As long as your driver script
calls ``enable_ptl()``, you can import PTL modules (.ptl files) just as
if they were Python modules.

Quixote's publisher will start at the root of this package, and will
treat the rest of the URL as a path into the package's contents.  Here
are some examples, assuming that the ``URL_PREFIX`` is ``"/q"``, your
web server is setup to rewrite ``/q`` requests as calls to (eg.)
``/www/cgi-bin/books.cgi``, and the root package for your application is
'books'::

  http://.../q/         call         books._q_index()
  http://.../q/other    call         books.other(), if books.other
                                     is callable (eg. a function or
                                     method)
  http://.../q/other    redirect to  /q/other/, if books.other is a
                                     namespace (eg. a module or sub-package)
  http://.../q/other/   call         books.other._q_index(), if books.other
                                     is a namespace

One of Quixote's design principles is "Be explicit."  Therefore there's
no complicated rule for remembering which functions in a module are
public; you just have to list them all in the _q_exports variable, which
should be a list of strings naming the public functions.  You don't need
to list the ``_q_index()`` function as being public; that's assumed.
Eg. if ``foo()`` is a function to be exported (via Quixote to the web)
from your application's namespace, you should have this somewhere in
that namespace (ie. at module level in a module or __init__.py file)::

    _q_exports = ['foo']

At times it is desirable for URLs to contain path components that are
not valid Python identifiers.  In these cases you can provide an
explicit external to internal name mapping.  For example::

    _q_exports = ['foo', ('stylesheet.css', 'stylesheet_css')]

When a function is callable from the web, it must expect a single
parameter, which will be an instance of the HTTPRequest class.  This
object contains everything Quixote could discover about the current HTTP
request -- CGI environment variables, form data, cookies, etc.  When
using SessionPublisher, request.session is a Session object for the user
agent making the request.

The function should return a string; all PTL templates return a string
automatically.  ``request.response`` is an HTTPResponse instance, which
has methods for setting the content-type of the function's output,
generating an HTTP redirect, specifying arbitrary HTTP response headers,
and other common tasks.  (Actually, the request object also has a method
for generating a redirect.  It's usually better to use this -- ie. code
``request.redirect(...)`` because generating a redirect correctly
requires knowledge of the request, and only the request object has that
knowledge.  ``request.response.redirect(...)`` only works if you supply
an absolute URL, eg. ``"http://www.example.com/foo/bar"``.)

Use ::

    pydoc quixote.http_request
    pydoc quixote.http_response

to view the documentation for the HTTPRequest and HTTPResponse classes,
or consult the source code for all the gory details.

There are a few special functions that affect Quixote's 
traversal of a URL to determine how to handle it: ``_q_access()``,
``_q_lookup()``, and  ``_q_resolve()``.


``_q_access(request)``
----------------------

If this function is present in a module, it will be called before
attempting to traverse any further.  It can look at the contents of
request and decide if the traversal can continue; if not, it should
raise quixote.errors.AccessError (or a subclass), and Quixote will
return a 403 ("forbidden") HTTP status code.  The return value is
ignored if ``_q_access()`` doesn't raise an exception.

For example, in the MEMS Exchange code, we have some sets of pages that
are only accessible to signed-in users of a certain type.  The
``_q_access()`` function looks like this::

    def _q_access (request):
        if request.session.user is None:
            raise NotLoggedInError("You must be signed in.")
        if not (request.session.user.is_admin() or
                request.session.user.is_fab()):
            raise AccessError("You don't have access to the reports page.")

This is less error-prone than having to remember to add checks to 
every single public function.


``_q_lookup(request, name)``
-----------------------------

This function translates an arbitrary string into an object that we
continue traversing.  This is very handy; it lets you put user-space
objects into your URL-space, eliminating the need for digging ID
strings out of a query, or checking ``PATH_INFO`` after Quixote's done
with it.  But it is a compromise with security: it opens up the
traversal algorithm to arbitrary names not listed in ``_q_exports``.
(``_q_lookup()`` is never called for names listed in ``_q_exports``.)
You should therefore be extremely paranoid about checking the value of
``name``.

``request`` is the request object, as it is everywhere else; ``name`` is
a string containing the next component of the path.  ``_q_lookup()`` should
return either a string (a complete document that will be returned to the
client) or some object that can be traversed further.  Returning a
string is useful in simple cases, eg. if you want the ``/user/joe`` URI
to show everything about user "joe" in your database, you would define a
``_q_lookup()`` in the namespace that handles ``/user/`` requests::

    def _q_lookup [plain] (request, name):
        if not request.session.user.is_admin():
            raise AccessError("permission denied")
        user = get_database().get_user(name)
        if user is None:
            raise TraversalError("no such user: %r" % name)
        else:
            "<h1>User %s</h1>\n" % html_quote(name)
            "<table>\n"
            " <tr><th>real name</th><td>%s</td>\n" % user.real_name
            # ...

(This assumes that the namespace in question is a PTL module, not a
Python module.)

To publish more complex objects, you'll want to use ``_q_lookup()``'s
ability to return a new namespace that Quixote continues traversing.
The usual way to do this is to return an instance of a class that
implements the web front-end to your object.  That class must have a
``_q_exports`` attribute, and it will almost certainly have a
``_q_index()`` method.  It might also have ``_q_access()`` and
``_q_lookup()`` (yes, ``_q_lookup()`` calls can nest arbitrarily
deeply).

For example, you might want ``/user/joe/`` to show a summary,
``/user/joe/history`` to show a login history, ``/user/joe/prefs`` to be
a page where joe can edit his personal preferences, etc.  The
``_q_lookup()`` function would then be ::

     def _q_lookup (request, name):
         return UserUI(request, name)

and the UserUI class, which implements the web interface to user
objects, might look like ::

    class UserUI:
        _q_exports = ['history', 'prefs']

        def __init__ (self, request, name):
            if not request.session.user.is_admin():
                raise AccessError("permission denied")
            self.user = get_database().get_user(name)
            if self.user is None:
                raise TraversalError("no such user: %r" % name)

        def _q_index (self, request):
            # ... generate summary page ...

        def history (self, request):
            # ... generate history page ...

        def prefs (self, request):
            # ... generate prefs-editing page ...

``_q_resolve(name)``
--------------------

``_q_resolve()`` looks a bit like ``_q_lookup()``, but is intended for
a different purpose.  Quixote applications can be slow to start up 
because they have to import a large number of Python and PTL modules.
``_q_resolve()`` is a hook that lets time-consuming imports 
be postponed until the code is actually needed

``name`` is a string containing the next component of the path.
``_q_resolve()`` should do whatever imports are necessary and return a
module that will be traversed further.  (Nothing enforces that this
function return a module, so you could also return other types, such
as a class instance, a callable object, or even a string) if the last
component of the path is being resolved.  Given ``_q_resolve()``'s
memoization feature, though, returning a module is the most useful
thing to do.)

``_q_resolve()`` is only ever called for names that are in
``_q_exports`` and that don't already exist in the containing
namespace.  It is not passed the request object, so its return value
can't depend on the client in any way.  Calls are also memoized; after
being called the object returned will be added to the containing
namespace, so ``_q_resolve()`` will be called at most once for a given
name.

Most commonly, ``_q_resolve()`` will look something like this::

    _q_exports = [..., 'expensive', ...]

    def _q_resolve(name):
        if name == 'expensive':
            from otherpackage import expensive
            return expensive

Let's say this function is in ``app.ui``.  The first time
``/expensive`` is accessed, ``_q_resolve('expensive')`` is called, the
``otherpackage.expensive`` module is returned and traversal continues.
The imported module is also saved as ``app.ui.expensive``, so future
references to ``/expensive`` won't need to invoke the ``_q_resolve()``
hook.

``_q_exception_handler(request, exception)``
--------------------------------------------

Quixote will display a default error page when a ``PublishError``
exception is raised.  Before displaying the default page, Quixote will
search back through the list of namespaces traversed looking for an
object with a ``_q_exception_handler`` attribute.  That attribute is
expected to be a function and is called with the request and exception
instance as arguments and should return the error page (e.g. a
string).  If the handler doesn't want to handle a particular error it
can re-raise it and the next nearest handler will be found.  If no
``_q_exception_handler`` is found, the default Quixote handler is
used.


$Id: programming.txt 23893 2004-04-06 19:31:20Z nascheme $