Expat (XML Parser Toolkit) Module for Ruby
		       version 0.5.16

		       Yoshida Masato

- Introduction

This is the module to access to James Clark's XML Parser
Toolkit "expat" (http://www.jclark.com/xml/expat.html) from
Ruby.

Supported version of expat is 1.1 and a test version
19990728 (ftp://ftp.jclark.com/pub/test/expat.zip).


- Installation

This can work with ruby-1.3. I recommend you to use
ruby-1.4.0 or later. And you need the source code of
expat-1.1.

Extract the archive of expat under this package. And apply
the patch 'expat-1.1.diff' (to modify Makefile) to it.

In some system, you have to modify CC, CFLAGS and RANLIB
of Makefile.


  cd ext
  gzip -dc < xmlparser-0.x.x.tar.gz | tar xvf -
  cd xmlparser
  unzip expat.zip
  cd expat
  patch -p1 < ../expat-1.1.diff
  make
  cd ..

After making expat, install this module usually.
If you want to use encoding maps of XML::Parser of Perl, set
the proper directory to -DXML_ENC_PATH.
For example, when Ruby supports dynamic linking on your OS,

  ruby extconf.rb
  make
  make install


- Usage

If you do not link this module with Ruby statically, 

  require "xmlparser"

before using.

There is two styles to get parsing result. One is to
define instance methods as event handlers, another is
to use iterator.

To define event handlers is like SAX (Simple API for XML).

If you use event handlers, inherit XMLParser class and
define instance methods as event handlers.
Or you may use the instance of XMLParser class (or derived)
with singleton instance methods as event handlers.

When no event handlers are defined, this parser does
non-validating syntax checking only.

  method name           | event
  ----------------------+---------------------------
  startElement          | element start tag
  endElement            | element end tag
  character             | character data
  processingInstruction | processing instruction
  unparsedEntityDecl    | unparsed entity declaration
  notationDecl          | notation declaration
  externalEntityRef     | external entity reference
  comment               | comment (*1)
  startCdata            | CDATA section start (*1)
  endCdata              | CDATA section end (*1)
  startNamespaceDecl    | Namespace declaration start (*1)
  endNamespaceDecl      | Namespace declaration end (*1)
  startDoctypeDecl      | DOCTYPE declaration start (*3)
  endDoctypeDecl        | DOCTYPE declaration end (*3)
  notStandalone         | document is not standalone (*1)
  default               | other data
  defaultExpand         | same as default (*2)
  unknownEncoding       | unknown character encoding

  *1 expat-1.1 or later.
  *2 expat-1.1 or later. inhibits expansion of internal
     entities. defaultExpand have higher priority
     than default.
  *3 expat-19990728 or later.

To use iterator is probably a ruby-ish manner.

If you use iterator, this parser ignores event handlers
even if they are defined.
The iterator evaluates the iterator block with three
variables, event type, name, and data.

  event type                          | name            | data
  ------------------------------------+-----------------+-------------------
  XMLParser::START_ELEM               | element name    | hash of attributes
  XMLParser::END_ELEM                 | element name    | nil
  XMLParser::CDATA                    | nil             | string
  XMLParser::PI                       | PI name         | string
  XMLParser::UNPARSED_ENTITY_DECL     | entity name     | array (*1)
  XMLParser::NOTATION_DECL            | notation name   | array (*2)
  XMLParser::EXTERNAL_ENTITY_REF      | entity names(*5)| array (*2)
  XMLParser::COMMENT (*3)             | nil             | string
  XMLParser::START_CDATA (*3)         | nil             | nil
  XMLParser::END_CDATA (*3)           | nil             | nil
  XMLParser::START_NAMESPACE_DECL (*3)| prefix          | URI
  XMLParser::END_NAMESPACE_DECL (*3)  | prefix          | nil
  XMLParser::START_DOCTYPE_DECL (*6)  | doctype name    | nil
  XMLParser::END_DOCTYPE_DECL (*6)    | nil             | nil
  XMLParser::DEFAULT (*4)             | nil             | string

  *1 [URL base, system ID, public ID, notation name]
      URL base and notation name may be nil.
  *2 [URL base, system ID, public ID]
      URL base, system ID and public ID may be nil.
  *3 expat-1.1 or later.
  *4 defaultExpand enables this event in expat-1.1 or later.
  *5 It may be nil on expat-19990626 or later.
  *6 expat-19990728 or later.

XMLParser::UNPARSED_ENTITY_DECL, XMLParser::NOTATION_DECL,
XMLParser::EXTERNAL_ENTITY_REF, XMLParser::COMMENT,
XMLParser::START_CDATA, XMLParser::END_CDATA,
XMLParser::START_NAMESPACE_DECL,
XMLParser::END_NAMESPACE_DECL and XMLParser::DEFAULT events
are generated only if each dummy methods,
"unparsedEntityDecl", "notationDecl", "externalEntityRef",
"comment", "startCdata", "endCdata", "startNamespaceDecl",
"endNamespaceDecl" and "default" (or "defaultExpand") are
defined.

Supported input character encodings are UTF-8 and UTF-16.
Output character encoding is UTF-8.
If XML_ENC_PATH is set on compiling, you can use the
encoding maps of XML::Parser of Perl. This package not
include them, you must get XML::Parser or XML::Encoding from
CPAN, and install .enc files into the proper directory.


XMLParser class:

 Class method
    new(encoding = nil, nssep = nil)
      Create a XML parser object. The failure of the
      creation raises a XMLParserError exception.

      The "encoding" parameter can specify the character
      encoding. Expat can recognize ISO-8859-1, UTF-8,
      US-ASCII and UTF-16, and expat_ja can also EUC-JP and
      Shift_JIS.

      The "nssep" parameter enables namespace extension.
      The namespace-prefixed element and attribute names are
      concatenated with the namespace's URI and a separator
      (the first byte of nssep).

      For example, with sssep = '!',

         <hoge:test xmlns:hoge="http://tristan.inse.co.jp/hoge/">

      is parsed into

         http://tristan.inse.co.jp/hoge/!test

      The object that finish parsing cannot be reused,
      so you must create a new one for every parsing.

    new(parser, context, encoding = nil)
      Create a XML parser object that can parse an external
      general entity. The failure of the creation raises a
      XMLParserError exception.

      This can be called at any point after the first call
      to an externalEntityRef event.
      
      The "context" parameter can be passed from the parse
      context of externalEntityRef event.

      The "encoding" parameter can specify the character
      encoding.

      The object that finish parsing cannot be reused,
      so you must create a new one for every parsing.


 Method
    parse(str, isFinal = true)
      Parse a string. This method can be an
      iterator. Parsing results can be processed by event
      handlers or an iterator block.

      "IsFinal" parameter must true on last call of this
      method. Default is true. No parameter call of this
      method indicates the end of parsing.

      "Str" can be the stream object. It must be the object
      with "gets" method. In this case, "isFinal" is
      ignored, the parsing is repeated until the stream
      returns nil.

      The failure to parse raises a XMLParserError
      exception.

    done
      Free the parser. Usually you can trust the GC, but
      after parsing the external parameser entity, you must
      free the parser in the externalEntityRef event.

    defaultCurrent
      Raise a "default" event within any event handlers or
      an iterator block.
      You can get the corresponding markup.

      If within a event handler, it raise a default event
      immediately. But within an iterator block, the next
      yielding will be XMLParser::DEFUALT.

    setBase
      Set URL base. The setting value can get the parameter
      'base' of the external entity methods, such as
      unparsedEntityDecl.

    line
    column
    byteIndex
      Get current parsing location.

      When a "parse" method raises XMLParserError, these
      method return the position of the error detected.

    byteCount
      Get the number of bytes in the current event.

      When the event is is in an internal entity, this
      method returns 0.

      This method is for expat-1.1 or later.

    getSpecifiedAttributes
      Check the attributes whether specified or defaulted.

      Return value is a hash, the keys are the attribute's
      name, the values are specified or not (boolean).

      This method should be used in startElement event
      handler.

      This method is for expat-1.1 or later.

    setParamEntityParsing(parsing)
      Controls parsing of parameter entities .

      "Parsing" parameter is
         PARAM_ENTITY_PARSING_NEVER or
         PARAM_ENTITY_PARSING_UNLESS_STANDALONE or
         PARAM_ENTITY_PARSING_ALWAYS.

      This method is for the expat-19990626.


 Method (event handler)
    startElement(name, attrs)
      This method is called when element start tags are
      detected.
      "Name" is the element name, attrs is a hash of
      attributes, the keys are the attribute's name, the
      values are attribute's values.

    endElement(name)
      This method is called when element end tags are
      detected.
      "Name" is the element name.

    character(data)
      This method is called when texts or CDATA sections are
      detected.
      Internal entities are expanded as long as "default"
      handler is not defined.

    processingInstruction(target, data)
      This method is called when processing instructions are
      detected.

    unparsedEntityDecl(entityName, base, systemId, publicId, notationName)
      This methods is called when parsed entity declarations
      are detected.
      "EntityName", "base", "systemId", "publicId" and
      "notationName" are the entity name, the URL base, the
      system identifier, the public identifier and the
      notation name.
      The URL base and the notation name can be nil.

      If you use iterator, this method is not called, but to
      define this affects to cause
      XMLParser::UNPARSED_ENTITY_DECL event.

    notationDecl(notationName, base, systemId, publicId)
      This methods is called when notation declarations are
      detected.
      "NotationName", "base", "systemId", and "publicId" are
      the notation name, the URL base, the system identifier
      and the public identifier.
      The URL base, the system identifier and the public
      identifier can be nil.

      If you use iterator, this method is not called, but to
      define this affects to cause XMLParser::NOTATION_DECL
      event.

    externalEntityRef(context, base, systemId, publicId)
      This methods is called when external entity references
      are detected.
      "context", "base", "systemId", and "publicId" are the
      parsing context, the URL base, the system identifier
      and the public identifier.
      The URL base and the public identifier can be nil.
      The context can use the 'context' parameter of the
      constructor of the external entity parser.

      If you do not parse the external entities by this
      event, the external entities are never parsed.

      If you use iterator, this method is not called, but to
      define this affects to cause
      XMLParser::EXTERNAL_ENTITY_REF event.

      On expat-19990626, it is called when external
      parameter entities (including external DTD subset) are
      detected. In this case, "context" will be nil. The
      parser for the external parameter entitiy must be
      created, "parse" and "done" in this event.

    comment(data)
      This methods is called when comments are detected.
      This method is for expat-1.1 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause XMLParser::COMMENT event.

    startCdata()
      This methods is called when CDATA sections start.
      The contents of the CDATA sections are reported by
      character event.
      This method is for expat-1.1 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause XMLParser::START_CDATA
      event.

    endCdata()
      This methods is called when CDATA sections end.
      This method is for expat-1.1 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause XMLParser::END_CDATA
      event.

    startNamespaceDecl(prefix, uri)
      This methods is called before the element that has
      namespace declaration.

      Prefix and uri can be nil.

      This method is for expat-1.1 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause
      XMLParser::START_NAMESPACE_DECL event.

    endNamespaceDecl(prefix)
      This methods is called after the element that has
      namespace declaration.

      Prefix can be nil.

      This method is for expat-1.1 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause
      XMLParser::END_NAMESPACE_DECL event.

    startDoctypeDecl(doctypeName)
      This methods is called when the name of the DOCTYPE is
      encountered.

      This method is for expat-19990728 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause
      XMLParser::START_DOCTYPE_DECL event.

    endDoctypeDecl()
      This methods is called when the closing > is
      encountered, but after processing any external subset.

      This method is for expat-19990728 or later.

      If you use iterator, this method is not called, but to
      define this affects to cause
      XMLParser::END_DOCTYPE_DECL event.

    default(data)
      This method is called when there is no applicable
      event handler.

      If this method is defined, expansion of internal
      entities are inhibited.

      If you use iterator, this method is not called, but to
      define this affects to cause XMLParser::DEFAULT event
      and to inhibit expansion of internal entities.

    defaultExpand(data)
      This method is called when there is no applicable
      event handler.

      If you use iterator, this method is not called, but to
      define this affects to cause XMLParser::DEFAULT event.

      This method have higher priority than default method.

    unknownEncoding(name)
      This method is called when unknown encoding is
      detected.

      XMLEncoding object (or nil to reject) must be returned.

      Even if parse method is used as the iterator, this
      method is called.

    notStandalone()
      This methods is called if the document is not standalone
      (it has an external subset or a reference to a
       parameter entity, but does not have standalone="yes").

      If you may return 0 to raise an error, or return 1 to
      continue the parsing.

      This method is for expat-1.1 or later.

      Even if parse method is used as the iterator, this
      method is called.


XMLEncoding class:
  To convert the character encoding, you must define map and
  convert method.

  Method
    map(code)
      This method is called to define byte stream
      information. Code is the first byte of stream, 00h to
      ffh. You must return the following value.

        >= 0     : treat as Unicode value
        -1       : the byte sequence is malformed
        -n (n>=2): n-byte sequence

    convert(s)
      This method is called to convert the byte sequence
      into a Unicode.
      The byte sequence is n-byte string (n is defined by
      map), you must return an integer value (treat as
      Unicode), an ASCII  character or two byte string
      (treat as a little endian UCS-2 character).


- Additional Library

XML::SimpleTree module and XML::SimpleTreeBuilder module are
added since version 0.3.1.
These module are not well documented, and API specification is
not fixed, so they are for experts only.


XML::SimpleTree module (xmltree.rb)
  This module is a library for making and manipulating XML
  trees.
  The APIs are like Document Object Model (DOM) of W3C.
 
  Classes
    NameNodeMap
    NodeList
    Node
    DocumentFragment<Node
    Document<Node
    CharacterData<Node
    Attr<Node
    Element<Node
    Text<CharacterData
    Comment<Data
    CDATASection<Text
    DocumentType<Node
    Notation<Node
    Entity<Node
    EntityReference<Node
    ProcessingInstruction<Node

XML::SimpleTreeBuilder (xmltreebuilder.rb)
  This module is a library for parsing XML file and building
  XML tree.

XML::JapaneseTreeBuilder class (xmltreebuilder-ja.rb)

XML::DOM::Visitor (xmltreevisitor.rb)

XMLEncoding_ja class (xmlencoding-ja.rb)

WGET module (wget.rb)

DOMHASH module (xmldigest.rb)

SAX module (sax.rb, saxdriver.rb)


- Samples

These sample scripts are required the optional "uconv"
module for Japanese XML files.

  xmlchack.rb - a sample for syntax checking
  xmlevent.rb - a sample for defining event handlers
  xmliter.rb  - a sample for iterator
  xmlgrep.rb  - a simple XML grep application
  treetest.rb - a sample for XML::SimpleTree
  buildertest.rb - a sample for XML::SimpleTreeBuilder
  videolist.rb - a simple XML application for video tape management
  gtktree.rb  - a sample with GTK
  xmlcomment.rb - a sample comes from XML::Parser of Perl
  visitortest.rb - a visitor sample comes from XML::Grove of Perl
  my-html.rb  - a visitor sample comes from XML::Grove of Perl
  visitor.rb  - a sample to access the tree like visitor.
  namespaces/ - files to test namespaces
  xpointer.rb - a simple application of XPointer with GTK
  digesttest.rb - a sample for DOMHASH
  digesttest2.rb - a sample for DOMHASH without DOM
  saxtest.rb - a sample for SAX


- Copying

This extension module is copyrighted free software by
Yoshida Masato.

You can redistribute it and/or modify it under the same term as
Ruby or expat.

encoding.h and the functions of encoding map are part of
XML::Parser for Perl.

  Copyright (c) 1998 Larry Wall and Clark Cooper.
  All rights reserved.
  This program is free software; you can redistribute it and/or modify it
  under the same terms as Perl itself. 


- Author

 Yoshida Masato <yoshidam@inse.co.jp>, <yoshidam@yoshidam.net>

 XPointer support is contributed by Masaki Fukushima 
     <fukusima@goto.info.waseda.ac.jp>


- History

 Oct 14, 1999 version 0.5.16 change some samples
 Aug 18, 1999 version 0.5.15 support start/endDoctypeDecl event of
                             expat-19990728.
                             fix SAX driver bug.
 Jun 29, 1999 version 0.5.14 support to parse external
                             parameter entities for
                             expat-19990626
 Jun 10, 1999 version 0.5.13 support experimental SAX driver
                             support expat-1.1
 May 13, 1999 version 0.5.12 fix extconf.rb bug
 Apr 28, 1999 version 0.5.11 for expat-19990425, add NotStandalone
                             event, getSpecifiedAttributes method
                             and byteCount method
 Apr 20, 1999 version 0.5.10 change xmldigest.rb for xss4j
 Mar 29, 1999 version 0.5.9  change the object structure for Ruby 1.3
 Mar 23, 1999 version 0.5.8  support the omission of keywords of XPointer
                             support to parse external parsed entities in 
                             XML::DOM::Builder
 Mar  8, 1999 version 0.5.7  support start/endNamespaceDecl event of
                             expat-19990307.
 Jan 25, 1999 version 0.5.6  class name aliases are defined in C module.
                             support cygwin.
 Jan 14, 1999 version 0.5.5  support start/endCdataSection event of
                             expat-19981231
 Jan 13, 1999 version 0.5.4  modify xmltree and xmltreebuilder
 Jan 10, 1999 version 0.5.3  encoding map support
 Dec  1, 1998 version 0.5.1  fix some bugs
 Nov 24, 1998 version 0.5.0  support the test version of expat
 Nov  5, 1998 version 0.4.18 fix some bugs, class name alias
                               XMLParserErorr        -> XML::Parser::Error
                             and change some internal functions.
 Oct 28, 1998 version 0.4.17 mIDs are stored into static vars
 Oct 28, 1998 version 0.4.16 change ID attribute support of XPointer.
                             Node#trim is now xml:space-aware
 Oct 23, 1998 version 0.4.15 fix some bugs, add class name alias
                               XMLParser              -> XML::Parser
                               XML::SimpleTree        -> XML::DOM
                               XML::SimpleTreeBuilder -> XML::DOM::Builder
 Oct 20, 1998 version 0.4.14 add better XPointer support by Masaki Fukushima
 Sep 17, 1998 version 0.4.5 add methods to SimpleTree
 Sep  8, 1998 version 0.4.4 change parser object type from
                            T_DATA to T_OBJECT (now can use
                            instance variables)
 Sep  3, 1998 version 0.4.3 add isFinal flag, and stream
                            parsing facility
 Sep  2, 1998 version 0.4.2 add external entity event and parser
 Aug 14, 1998 version 0.3.3 support expat 1.0
 Aug 12, 1998 version 0.3.2
 Aug  4, 1998 version 0.3.1
 Jul 17, 1998 version 0.3
 Jul  3, 1998 version 0.2
 Jul  1, 1998 version 0.1