File: UsersGuide.html

package info (click to toggle)
libopentoken 3.0b-5
links: PTS
area: main
in suites: etch, etch-m68k
size: 1,416 kB
ctags: 51
sloc: ada: 8,561; makefile: 120
file content (1537 lines) | stat: -rw-r--r-- 90,335 bytes
parent folder | download | duplicates (3)
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="Author" content="Ellie Dennison">
   <meta name="GENERATOR" content="Mozilla/4.72 [en] (Windows NT 5.0; I) [Netscape]">
   <title>OpenToken User's Guide</title>
</head>
<body>

<center>
<h1>
OpenToken User Guide</h1></center>

<h2>
User Manual</h2>

<h3>
Index</h3>

<ul>
<li>
<a href="#Introduction">Introduction</a></li>

<li>
<a href="#Lexical">Lexical Analysis</a></li>

<ul>
<li>
<a href="#LA_Introduction">Introduction</a></li>

<li>
<a href="#Step 1: Creating an enumeration of">Step 1: Creating an enumeration
of tokens</a></li>

<li>
<a href="#Step 2: Instantiate a token analyzer">Step 2: Instantiate a token
analyzer class</a></li>

<li>
<a href="#Step 3: Create a token recognizers for each">Step 3: Create a
token recognizers for each token</a></li>

<li>
<a href="#Step 3.5: Creating a custom token recognizer">Step 3.5: Creating
a custom token recognizer type</a></li>

<li>
<a href="#Step 4: Map the recognizers to their">Step 4: Map the recognizers
to their tokens</a></li>

<li>
<a href="#Step 5: Create a Token Analyzer">Step 5: Create a Token Analyzer
object</a></li>

<ul>
<li>
<a href="#Advanced: Using your own Text">Advanced: Using your own Text
Feeder</a></li>
</ul>

<li>
<a href="#Use">Use</a></li>
</ul>

<li>
<a href="#Parsing">Parsing</a></li>

<ul>
<li>
<a href="#Table-Driven">Table-Driven Parsing</a></li>

<ul>
<li>
<a href="#Step 1: Creating an Enumeration of">Step 1: Creating an Enumeration
of Token IDs</a></li>

<li>
<a href="#Step 2: Instantiate Your Token">Step 2: Instantiate Your Token
Packages</a></li>

<li>
<a href="#Step 3: Create the">Step 3: Create the Tokens</a></li>

<li>
<a href="#Step 4: Map the Terminal Token ID's to their">Step 4: Map the
Terminal Token ID's to their recognizers and tokens</a></li>

<li>
<a href="#Step 5: Define a Lexical">Step 5: Define a Lexical Analyzer</a></li>

<ul>
<li>
<a href="#Advanced: Declaring a Custom Text">Advanced: Declaring a Custom
Text Feeder</a></li>
</ul>

<li>
<a href="#Step 6: Creating a">Step 6: Creating a Grammar</a></li>

<li>
<a href="#Step 7: Generating a">Step 7: Generating a parser</a></li>

<li>
<a href="#Parser Use">Use</a></li>

<li>
<a href="#Synthesized">Synthesized Attributes</a></li>

<ul>
<li>
<a href="#Custom">Custom Nonterminals</a></li>

<li>
<a href="#Defining the">Defining the Parser</a></li>
</ul>
</ul>

<li>
<a href="#Recursive Descent">Recursive Descent Parsing</a> (beta)</li>
</ul>
</ul>

<h3>
<a NAME="Introduction"></a>Introduction</h3>
OpenToken is a token analysis system that is at once powerful, extensible,
and easy to use. You should have to do far less work to get a running lexical
analyzer or parser than you would with a traditional lerxer/parser generator.
In many cases using it is just a matter of picking the right components
and plugging them together properly.
<p>A <i>token</i> is a consecutive sequence of characters that have a collective
meaning. As the name implies, this facility revolves around tokens. The
basic facilities provided are for <i><a href="#Lexical">lexical analysis</a></i>,
which is the breaking up of a stream of text into tokens, and <i><a href="#Parsing">parsing</a></i>,
which is the grouping of tokens into grammatical phrases.
<h3>
<a NAME="Lexical"></a>Lexical Analysis</h3>

<h4>
<a NAME="LA_Introduction"></a>Introduction To Lexical Analysis</h4>
The OpenToken lexical analyzer generator packages consist of two parts:
<ol>
<li>
The lexical analysis engine itself (OpenToken.Token.Analyzer)</li>

<li>
A set of token recognizer packages (OpenToken.Recognizer.Line_Comment,
etc.)</li>
</ol>
There are 5 phases to creating your own lexical analyzer using OpenToken.
<ol>
<li>
<a href="#Step 1: Creating an enumeration of">Define an enumeration</a>
containing all your <i>tokens</i>.</li>

<li>
<a href="#Step 2: Instantiate a token analyzer">Instantiate a <i>token
analyzer class</i></a> for your tokens.</li>

<li>
<a href="#Step 3: Create a token recognizers for each">Create a <i>token
recognizer</i></a> for each token.</li>

<li>
Map the recognizers to their tokens to <a href="#Step 4: Map the recognizers to their">create
a <i>syntax</i></a>.</li>

<li>
<a href="#Step 5: Create a Token Analyzer">Create a <i>token analyzer object</i></a>
initialized with your syntax</li>
</ol>
The following sections will walk you through each of these steps in detail,
using an example from chapter 3 of <u>Compilers, Principles, Techniques,
&amp; Tools</u>.<sup><a href="#*">*</a></sup> (aka: the "dragon book").
<h4>
<a NAME="Step 1: Creating an enumeration of"></a>Step 1: Creating an enumeration
of tokens</h4>
This step is fairly simple. Just create an enumerated type containing one
entry for each token you want to be recognized in the input. For our example,
we will assume the grammar in Example 3.6 of the dragon book.<sup><a href="#*">*</a></sup>
<br>&nbsp;
<blockquote><tt>type Example_Token_ID is (If_ID, Then_ID, Else_ID, ID_ID,
Num, Relop, Whitespace);</tt></blockquote>
Again, this is a very simple step once you know the list of tokens you
need. But of course figuring that out is not always so simple!
<h4>
<a NAME="Step 2: Instantiate a token analyzer"></a>Step 2: Instantiate
a token analyzer class</h4>
This step is trivial. Simply instantiate the generic OpenToken.Token.Enumerated
package with your enumerated type. Then use that package to instantiate
an OpenToken.Token.Enumerated.Analyzer package.
<br>&nbsp;
<blockquote><tt>package Example_Token is new Opentoken.Token.Enumerated
(Example_Token_ID);</tt>
<br><tt>package Tokenizer is new Example_Token.Analyzer;</tt></blockquote>

<h4>
<a NAME="Step 3: Create a token recognizers for each"></a>Step 3: Create
a token recognizer for each token</h4>
Each enumerated token needs a recognizer object. Recognizer objects can
be created in one of two ways. The easy way is to use one of the recognizer
classes in the OpenToken.Recognizer.* hierarchy of packages.
<blockquote><tt>If_Recognizer&nbsp;&nbsp; : constant Tokenizer.Recognizable_Token
:=</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Get(Opentoken.Recognizer.Keyword.Get ("if"));</tt>
<br><tt>Then_Recognizer : constant Tokenizer.Recognizable_Token :=</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Get(Opentoken.Recognizer.Keyword.Get ("then"));</tt>
<br><tt>Else_Recognizer : constant Tokenizer.Recognizable_Token :=</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Get(Opentoken.Recognizer.Keyword.Get ("else"));</tt>
<br><tt>ID_Recognizer&nbsp;&nbsp; : constant Tokenizer.Recognizable_Token
:=</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Get(Opentoken.Recognizer.Identifier.Get);</tt>
<br><tt>Num_Recognizer&nbsp; : constant Tokenizer.Recognizable_Token :=</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Get(Opentoken.Recognizer.Real.Get);</tt>
<br><tt>Whitesp_Recognizer : constant Tokenizer.Recognizable_Token :=</tt>
<br><tt>&nbsp; Tokenizer.Get(Opentoken.Recognizer.Character_Set.Get</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(Opentoken.Recognizer.Character_Set.Standard_Whitespace)</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt></blockquote>
The source above creates token recognizers for
<blockquote>
<li>
The keywords "if", "then", and "else". Case does not matter for keywords,
so "IF" or "If" would also be recognized.</li>

<li>
An identifier. Any series of characters starting with a letter and containing
only letters, number, and underscores will be recognized..</li>

<li>
A real (floating or fixed point) literal.</li>

<li>
A series of "whitespace" characters. The Standard_Whitespace is used, which
includes spaces, tabs, and line terminators.</li>
</blockquote>

<h5>
<a NAME="Step 3.5: Creating a custom token recognizer"></a>Step 3.5: Creating
a custom token recognizer type</h5>
If you have a token that cannot be recognized by any of the default recognizers,
there is an extra step. You have to create your own recognizer routine.
That may sound like a lot of work, but really it is not significantly more
complicated that creating a regular expression in lex would be.
<p>A recognizer is a tagged type that is derived from the type OpenToken.Recognizer.Instance.
You should extend the type to provide yourself state information and to
keep track of any settings that your recognizer type may allow. Other routines
and information about this specific type of token may be placed in there
too. In our example the token <i>Relop</i> cannot be recognized by any
of the provided token recognizers, so we declare it as follows. The part
that can be cut-and-paste is in black. The part that was custom for this
recognizer is is <font color="#3366FF">blue</font>&nbsp; (if your browser
supports colors).
<br>&nbsp;
<blockquote><tt>with OpenToken.Recognizer;</tt>
<br><tt>package <font color="#3366FF">Relop_Example_Token</font> is</tt>
<p><tt>&nbsp;&nbsp; type Instance is new Opentoken.Recognizer.Instance
with private;</tt>
<p><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; -- This function will be called to create an Identifier
token. Note that</tt>
<br><tt>&nbsp;&nbsp; -- this is a simple recognizer, so Get doesn't need
any parameters.</tt>
<br><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; function Get return Instance;</tt>
<p><tt>private</tt>
<p><tt>&nbsp;&nbsp; type State_ID is (First_Char, <font color="#3366FF">Equal_or_Greater,
Equal,</font> Done);</tt>
<p><tt>&nbsp;&nbsp; type Instance is new Opentoken.Recognizer.Instance
with record</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; State : State_ID := First_Char;</tt>
<br><tt>&nbsp;&nbsp; end record;</tt>
<p><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; -- This procedure will be called when analysis on
a new candidate string</tt>
<br><tt>&nbsp;&nbsp; -- is started. The Token needs to clear its state
(if any).</tt>
<br><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; procedure Clear (The_Token : in out Instance);</tt>
<br>&nbsp;
<p><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; -- This procedure will be called to perform further
analysis on a token</tt>
<br><tt>&nbsp;&nbsp; -- based on the given next character.</tt>
<br><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; procedure Analyze (The_Token : in out Instance;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Next_Char : in&nbsp;&nbsp;&nbsp;&nbsp; Character;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp; :&nbsp;&nbsp;&nbsp; out Opentoken.Recognizer.Analysis_Verdict);</tt>
<p><tt>end <font color="#3366FF">Relop_Example_Token</font>;</tt></blockquote>
Note that very little code is in blue; just the name of the package and
the states between the first and last state. Of course more routines and
fields in <i>Instance</i> may be added at your discretion depending on
the needs of your recognizer.
<p>To help avoid confusion, when naming states, I have found it easiest
to stick to the following standard:
<ul>
<li>
The first and last states are named <i>First_Char</i> and <i>Done</i> respectively.</li>

<li>
The intervening states are named for the current part of the token we are
expecting to recognize, <b>not</b> for the item that was just recognized.</li>
</ul>
The package body requires a bit more thought. You will have to implement
a state machine for recognizing your token. At the end of any state you
will need to set the new state for the recognizer (if it changed) and return
the match result for the given character.
<p>The result will be one of the enumeration values in OpenToken.Recognizer.Analysis_Verdict.
<i>Matches</i>
indicates that the string you have been fed so far (since the last <i>Clear</i>
call) does fully qualify as a token. <i>So_Far_So_Good</i> indicates that
the string in its current state does not match a token, but it could possibly
in the future match, depending on the next characters that are fed in.
Note that it is quite possible for the verdict to be Matches on one call,
and <i>So_Far_So_Good</i> on a later call, depending on the definition
of the token. The final verdict, <i>Failed</i>, is different. You return
it to indicate that the string is not a legal token of your type, and can
never be one no matter how many more characters are fed in. Whenever you
return this, you should set the recognizer's state to <i>Done</i> as well.
<blockquote><tt>package body <font color="#3366FF">Relop_Example_Token</font>
is</tt>
<p><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; -- This procedure will be called when analysis on
a new candidate string</tt>
<br><tt>&nbsp;&nbsp; -- is started. The Token needs to clear its state
(if any).</tt>
<br><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; procedure Clear (The_Token : in out Instance) is</tt>
<br><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; The_Token.State := First_Char;</tt>
<br><tt>&nbsp;&nbsp; end Clear;</tt>
<p><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; -- This procedure will be called to create a <font color="#3366FF">Relop</font>
token recognizer</tt>
<br><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; function Get return Instance is</tt>
<br><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return (Report => True,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
State&nbsp; => First_Char);</tt>
<br><tt>&nbsp;&nbsp; end Get;</tt>
<p><tt>&nbsp;&nbsp; --------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; -- This procedure will be called to perform further
analysis on a token</tt>
<br><tt>&nbsp;&nbsp; -- based on the given next character.</tt>
<br><tt>&nbsp;&nbsp; ---------------------------------------------------------------------------</tt>
<br><tt>&nbsp;&nbsp; procedure Analyze (The_Token : in out Instance;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Next_Char : in Character;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp; : out Opentoken.Recognizer.Analysis_Verdict) is</tt>
<br><tt>&nbsp;&nbsp; begin</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case The_Token.State is</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; when First_Char
=></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
-- If the first char is a &lt;, =, or >, its a match</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
case Next_Char is</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when '&lt;' =></font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Matches;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Equal_Or_Greater;</font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when '>' =></font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Matches;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Equal;</font></tt>
<br>&nbsp;
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when '=' =></font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Matches;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Done;</font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when others =></font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Failed;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Done;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
end case;</font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when Equal_Or_Greater =></font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
-- If the next char is a > or =, its a match</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
case Next_Char is</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when '>' | '=' =></font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Matches;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Done;</font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when others =></font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Failed;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Done;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
end case;</font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
when Equal =></font></tt>
<p><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
-- If the next char is a =, its a match</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
if Next_Char = '=' then</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Matches;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Done;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
else</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; := Opentoken.Recognizer.Failed;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
The_Token.State := Done;</font></tt>
<br><tt><font color="#3366FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
end if;</font></tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; when Done =></tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Verdict := Opentoken.Recognizer.Failed;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; end case;</tt>
<br><tt>&nbsp;&nbsp; end Analyze;</tt>
<p><tt>end <font color="#3366FF">Relop_Example_Token</font>;</tt>
<br>&nbsp;</blockquote>
Now the only thing that remains is to create a token recognizer object
of your new recognizer type, just like you did for the predefined recognizer
types.
<blockquote><tt>Relop_Recognizer&nbsp; : constant Tokenizer.Recognizable_Token
:=</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Get(Relop_Example_Token.Get);</tt></blockquote>

<h4>
<a NAME="Step 4: Map the recognizers to their"></a>Step 4: Map the recognizers
to their tokens</h4>
This step is quite simple. Just declare an object of type <i>Tokenizer.Syntax
</i>(assuming
your instantiation of the analyzer package in step 2 was named <i>Tokenizer</i>).
Initialize the array with the proper token recognizers for each token index.
For our example it would look like this:
<br>&nbsp;
<blockquote><tt>Syntax : constant Tokenizer.Syntax :=</tt>
<br><tt>&nbsp;&nbsp; (If_ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; => If_Recognizer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; Then_ID&nbsp;&nbsp;&nbsp; => Then_Recognizer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; Else_ID&nbsp;&nbsp;&nbsp; => Else_Recognizer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; ID_ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; => ID_Recognizer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; Num&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
=> Num_Recognizer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; Relop&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; => Relop_Recognizer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; Whitespace => Whitesp_Recognizer</tt>
<br><tt>&nbsp;&nbsp; );</tt></blockquote>
To make things a little easier, we can easily combine steps 3 and 4 into
one step. eg:
<blockquote><tt>Syntax : constant Tokenizer.Syntax :=</tt>
<br><tt>&nbsp; (If_ID&nbsp;&nbsp; => Tokenizer.Get(Opentoken.Recognizer.Keyword.Get
("if")),</tt>
<br><tt>&nbsp;&nbsp; Then_ID => Tokenizer.Get(Opentoken.Recognizer.Keyword.Get
("then")),</tt>
<br><tt>&nbsp;&nbsp; Else_ID => Tokenizer.Get(Opentoken.Recognizer.Keyword.Get
("else")),</tt>
<br><tt>&nbsp;&nbsp; ID_ID&nbsp;&nbsp; => Tokenizer.Get(Opentoken.Recognizer.Identifier.Get),</tt>
<br><tt>&nbsp;&nbsp; Int&nbsp;&nbsp;&nbsp;&nbsp; => Tokenizer.Get(Opentoken.Recognizer.Integer.Get),</tt>
<br><tt>&nbsp;&nbsp; Real&nbsp;&nbsp;&nbsp; => Tokenizer.Get(Opentoken.Recognizer.Real.Get),</tt>
<br><tt>&nbsp;&nbsp; Relop&nbsp;&nbsp; => Tokenizer.Get(Relop_Example_Token.Get),</tt>
<br><tt>&nbsp;&nbsp; Whitespace => Tokenizer.Get(Opentoken.Recognizer.Character_Set.Get</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(Opentoken.Recognizer.Character_Set.Standard_Whitespace))</tt>
<br><tt>&nbsp;&nbsp; );</tt></blockquote>

<h4>
<a NAME="Step 5: Create a Token Analyzer"></a>Step 5: Create a Token Analyzer
object</h4>
Now we are ready to create our token analyzer. All we have to do is declare
an object of type <i>Tokenizer.Instance</i> (again, assuming that <i>Tokenizer</i>
is the name of the analyzer instantiated back in step 2) , and initialize
it via the <i>Tokenizer.Initialize</i> call. For this call we supply the
syntax object from step 4.
<blockquote><tt>Analyzer : Tokenizer.Instance := Tokenizer.Initialize (Syntax);</tt></blockquote>
This creates an analyzer that will read input from Ada.Text_IO.Current_Input,
and attempt to match it to the given syntax. By default this will be standard
input, but than can be redirected to the file of your choice using <tt>Ada.Text_IO.Set_Input.</tt>
<h5>
<a NAME="Advanced: Using your own Text"></a>Advanced: Using your own Text
Feeder</h5>
In the majority of cases the above will be sufficient. However, if you
want to preserve the ability to read user input from standard input, you
can instead create your own Text_IO-based text feeder and pass a pointer
to it when you create the Analyzer:
<br>&nbsp;
<blockquote><tt>File&nbsp;&nbsp; : aliased Ada.Text_IO.File_Type;</tt>
<br><tt>Feeder : aliased OpenToken.Text_Feeder.Text_IO.Instance :=</tt>
<br><tt>&nbsp;&nbsp; OpenToken.Text_Feeder.Text_IO.Create (File'Unchecked_Access);</tt></blockquote>

<blockquote><tt>Analyzer : Tokenizer.Instance := Tokenizer.Initialize</tt>
<br><tt>&nbsp;&nbsp; (Language_Syntax => Syntax,</tt>
<br><tt>&nbsp;&nbsp;&nbsp; Feeder&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
=> Feeder'access);</tt></blockquote>
The text feeder is tagged type in <i><tt>OpenToken.Text_Feeder.Instance'Class</tt></i>.
It has a primitive (overrideable) routine named <i>Get</i> that fills a
string with characters. Whenever the analyzer runs out of characters to
process, it will request more from the feeder's <i>Get</i> function. If
you do not supply a text feeder, a default one is used which reads input
from the current input file.
<p>If you want to change the file the default text feeder reads from, you
can directly modify <i>Tokenizer.Input_Feeder</i>, either along with changing
Text_IO's current input file...
<blockquote><tt>&nbsp;&nbsp; Ada.Text_IO.Set_Input (File);</tt>
<br><tt>&nbsp;&nbsp; Tokenizer.Input_Feeder := OpenToken.Text_Feeder.Text_IO.Create;</tt></blockquote>
...or independently from Text_IO's current input file:
<blockquote><tt>&nbsp;&nbsp; Tokenizer.Input_Feeder := OpenToken.Text_Feeder.Text_IO.Create(File'Unchecked_Access);</tt></blockquote>
If you want to change your analyzer's text feeder during analysis, you
can also use the function Set_Text_Feeder:
<ul><tt>&nbsp;&nbsp; Tokenizer.Set_Text_Feeder (Analyzer => Analyzer,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Feeder&nbsp;&nbsp; => My_New_Text_Feeder</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt></ul>
Finally, if you want to use an input feeder that does not rely on Text_IO
files, there are other feeders available in the <i>OpenToken.Text_Feeder.*</i>
package hierarchy. If none of those suit your purposes, you can derive
a type from <i>OpenToken.Text_Feeder.Instance</i>, and override its <i>Get</i>
procedure with your own version.
<h4>
<a NAME="Use"></a>Use</h4>
Now we have our own token analyzer. To use it, all we have to do is call
<i>Tokenizer.Find_Next
</i>once
for each token we want to find. <i>Tokenizer.ID</i> will return the ID
of the token that was found. <i>Tokenizer.Lexeme</i> returns the actual
string that was matched.
<p>The full source that was used for this tutorial is available in the
Examples/ASU_Example_3_6 directory, along with a sample input file. To
run it using Gnat, issue the "make" command in that directory. (For other
compilers, consult your compiler documentation to see how to build a program).
When the command completes, type in "asu_example_3_6" to run it. You should
see the following list of tokens recognized:
<blockquote><tt>Found IF_ID</tt>
<br><tt>Found ID_ID</tt>
<br><tt>Found RELOP</tt>
<br><tt>Found ID_ID</tt>
<br><tt>Found THEN_ID</tt>
<br><tt>Found ELSE_ID</tt>
<br><tt>Found RELOP</tt>
<br><tt>Found REAL</tt>
<br><tt>Found RELOP</tt>
<br><tt>Found INT</tt></blockquote>
The text is read in from the file <i>Example.txt</i> in that directory.
If you want you can modify the contents of that file to produce a different
list of token recognitions.
<br>&nbsp;
<h3>
<a NAME="Parsing"></a>Parsing</h3>
OpenToken has to facilities to support parsing. The traditional method
is table-driven parsing, which relies on extra "non-terminal" token enumerations,
as well as the OpenToken.Production tree of packages.
<p>The newest method is recursive-descent parsing. This method relies on
enumerated tokens, as well as non-enumerated tokens from the OpenToken.Token
hierarchy.
<br>&nbsp;
<h4>
<a NAME="Table-Driven"></a>Table-Driven Parsing</h4>
The OpenToken table-driven parsing facility packages consists of 4 major
parts.
<ol>
<li>
The lexical analyzer packages</li>

<li>
The parser</li>

<li>
Tokens</li>

<li>
A list of productions (aka: a Grammar)</li>
</ol>
There are five basic phases to creating your own parser with OpenToken
<ol>
<li>
<a href="#Step 1: Creating an Enumeration of">Create an enumeration</a>
listing all your tokens.</li>

<li>
Specify your tokens using an enumeration of token IDs, and actual token
objects.</li>

<li>
Create the lexical analyzer.</li>

<li>
Define your Grammar to specify how your tokens go together.</li>

<li>
Generate a parser for your grammar.</li>
</ol>
For our example this time, we will assume the grammar in Example 4.46 of
the dragon book.<sup><a href="#*">*</a></sup>:
<blockquote><tt>S' -> S</tt>
<br><tt>S&nbsp; -> L = R | R</tt>
<br><tt>L&nbsp; -> * R | id</tt>
<br><tt>R&nbsp; -> L</tt></blockquote>
For our implementation we will also use two extra tokens to designate the
end of the file, and whitespace in the text stream.
<h5>
<a NAME="Step 1: Creating an Enumeration of"></a>Step 1: Creating an Enumeration
of Token IDs</h5>
This step is the same as outlined in the previous section, with one twist:
No longer do we just need to identify ID's of tokens that will physically
appear in the input (aka: "terminals"). We will also have to identify ID's
for tokens that will be created by the parser from other tokens (aka: "nonterminals").
For reasons that will hopefully become clear later, the terminal ID's will
need to appear in the enumeration before any of the nonterminal ID's.
<br>&nbsp;
<blockquote><tt>type Token_IDs is (Asterix_ID, ID_ID, Equals_ID, EOF_ID,
Whitespace_ID, S_ID, L_ID, R_ID, S_Prime_ID);</tt></blockquote>
Again, this is a very simple step once you know the list of tokens you
need. But of course figuring that out is not always so simple!
<br>Just like any other program, you cannot expect to sit down at the keyboard
and pound out a well-working parser without doing any design beforehand.
<br>&nbsp;
<h5>
<a NAME="Step 2: Instantiate Your Token"></a>Step 2: Instantiate Your Token
Packages</h5>
This step is much like step 2 in the previous section, except that now
there are a whole lot more packages to instantiate. Start by instantiating
the generic OpenToken.Token.Enumerated package with the token enumerated
type. Then use that package to instantiate an OpenToken.Token.Enumerated.Analyzer
package. You may have noticed before that the Analyzer package has a generic
parameter and wondered why it wasn't mentioned. Its purpose is to designate
to the analyzer which Token ID enumeration is the <i>last</i> terminal.
The analyzer doesn't care about nonterminals because they won't be found
in the input stream.
<br>&nbsp;
<blockquote><tt>package Master_Token is new OpenToken.Token.Enumerated(Token_IDs);</tt>
<br><tt>package Tokenizer is new Master_Token.Analyzer(Whitespace_ID);</tt></blockquote>
Next you will need to use the instantiated token package to create a token
list package for use in creating productions. That package is also used
to instantiate the OpenToken.Token.Nonterminal package.
<blockquote><tt>package Token_List is new Master_Token.List;</tt>
<br><tt>package Nonterminal is new Master_Token.Nonterminal(Token_List);</tt></blockquote>
Next we need access to the packages that allow us to create a Grammar.
OpenToken.Production gets instantiated with our root token package, our
root nonterminal package, and our token list package. Its child .List is
instantiated after that.
<blockquote><tt>package Production is new OpenToken.Production(Master_Token,
Token_List, Nonterminal);</tt>
<br><tt>package Production_List is new Production.List;</tt></blockquote>
<tt>Next we instantiate the root package in the parser hierarchy with the
production list package and the analyzer package. And then at last we create
our parser package.</tt>
<blockquote><tt>package Parser is new Production.Parser(Production_List,
Tokenizer);</tt>
<br><tt>package LALR_Parser is new Parser.LALR;</tt></blockquote>
<tt>That's it for the generic instantiations. That wasn't really so bad,
was it? There is one last package visibility chore though. In order to
use the infix production operators to define our grammar, we'll probably
want to perform a few use type's</tt>
<blockquote><tt>-- Allow infix operators for building productions</tt>
<br><tt>use type Token_List.Instance;</tt>
<br><tt>use type Production.Right_Hand_Side;</tt>
<br><tt>use type Production.Instance;</tt>
<br><tt>use type Production_List.Instance;</tt></blockquote>

<h5>
<a NAME="Step 3: Create the"></a>Step 3: Create the Tokens</h5>
Next we need to declare token variables for all our tokens that will appear
in a production. Unreported tokens, like the Whitespace token in this example,
can just as easily be created on the fly in the next step. The terminals
must be declared as objects derived from Master_Token.Instance. The nonterminals
must be declared as objects derived from Nonterminal.Instance.
<blockquote><tt>Asterix : aliased Master_Token.Class := Master_Token.Get
(Asterix_ID);</tt>
<br><tt>ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : aliased Master_Token.Class :=
Master_Token.Get (ID_ID);</tt>
<br><tt>Equals&nbsp; : aliased Master_Token.Class := Master_Token.Get (Equals_ID);</tt>
<br><tt>EOF&nbsp;&nbsp;&nbsp;&nbsp; : aliased Master_Token.Class := Master_Token.Get
(EOF_ID);</tt>
<br><tt>S&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : aliased Nonterminal.Class&nbsp;
:= Nonterminal.Get (S_ID);</tt>
<br><tt>L&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : aliased Nonterminal.Class&nbsp;
:= Nonterminal.Get (L_ID);</tt>
<br><tt>R&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : aliased Nonterminal.Class&nbsp;
:= Nonterminal.Get (R_ID);</tt>
<br><tt>S_Prime : aliased Nonterminal.Class&nbsp; := Nonterminal.Get (S_Prime_ID);</tt></blockquote>

<h5>
<a NAME="Step 4: Map the Terminal Token ID's to their"></a>Step 4: Map
the Terminal Token ID's to their recognizers and tokens</h5>
An object of type <i>Tokenizer.Syntax</i> must now be initialized. This
object will map all Terminal token ID's to both their recognizer and their
token object. The routine Tokenizer.Get creates a structure (called a <i>Recognizable_Token</i>)
which can be assigned into an analyzer syntax element to create the mapping.
<p>Additionally, there's a special version of the <i>Get</i> routine which
has no <i>New_Token</i> parameter. This form can be used to create a mapping
without first creating a token object. This is useful for unreported tokens
like our whitespace token, which do not take part in the grammar. In this
case a Master_Token.Instance will be dynamically allocated and assigned
into the mapping.
<blockquote><tt>Syntax : constant Tokenizer.Syntax :=</tt>
<br><tt>&nbsp; (Asterix_ID&nbsp;&nbsp;&nbsp; => Tokenizer.Get (Recognizer
=> OpenToken.Recognizer.Keyword.Get ("*"),</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
New_Token&nbsp; => Asterix),</tt>
<br><tt>&nbsp;&nbsp; ID_ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
=> Tokenizer.Get (Recognizer => OpenToken.Recognizer.Keyword.Get ("id"),</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
New_Token&nbsp; => ID),</tt>
<br><tt>&nbsp;&nbsp; Equals_ID&nbsp;&nbsp;&nbsp;&nbsp; => Tokenizer.Get
(Recognizer => OpenToken.Recognizer.Keyword.Get ("="),</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
New_Token&nbsp; => Equals),</tt>
<br><tt>&nbsp;&nbsp; EOF_ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =>
Tokenizer.Get (Recognizer => OpenToken.Recognizer.End_Of_File.Get,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
New_Token&nbsp; => EOF),</tt>
<br><tt>&nbsp;&nbsp; Whitespace_ID => Tokenizer.Get</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp; (OpenToken.Recognizer.Character_Set.Get</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (OpenToken.Recognizer.Character_Set.Standard_Whitespace))</tt>
<br><tt>&nbsp;&nbsp; );</tt></blockquote>

<h5>
<a NAME="Step 5: Define a Lexical"></a>Step 5: Define a Lexical Analyzer</h5>

<h6>
<a NAME="Advanced: Declaring a Custom Text"></a>Advanced: Declaring a Custom
Text Feeder</h6>
In this case we would like to use a text feeder tied to our own file, rather
than <i>Ada.Text_IO.Current_Input</i>. To do this, first declare an (aliased)
object to serve as the file for <i>Text_IO.Open</i>s and <i>Close</i>s.
Then declare an <i>OpenToken.Text_Feeder.Text_IO.Instance</i>. Initialize
the text feeder with the <i>Create</i> routine, using the file object as
the <i>File_Ptr</i>.
<blockquote><tt>Input_File : aliased Ada.Text_IO.File_Type;</tt>
<br><tt>Feeder&nbsp;&nbsp;&nbsp;&nbsp; : aliased OpenToken.Text_Feeder.Text_IO.Instance
:=</tt>
<br><tt>&nbsp; OpenToken.Text_Feeder.Text_IO.Create (Input_File'Unchecked_Access);</tt></blockquote>
Now to create the analyzer itself, we just make an object of type <i>Tokenizer.Instance</i>,
and initialize it with our syntax and text feeder. This is trivial.
<blockquote><tt>Analyzer : Tokenizer.Instance := Tokenizer.Initialize (Syntax,
Feeder'access);</tt></blockquote>
So far this should have all looked pretty familiar if you have gone through
the previous example. Yes, there were a lot more packages to instantiate,
and that bit about creating tokens was new. But still most of the steps
were familiar, and the result is pretty much the same so far: an analyzer.
With the next step this will start to change, so get ready. Here it comes...
<h5>
<a NAME="Step 6: Creating a"></a>Step 6: Creating a Grammar</h5>
A <i>grammar</i> is a notation for describing how the tokens of the language
interrelate. Another way of looking at it is that it is a way of describing
a language in terms of its tokens.
<p>A grammar is created using the <i>OpenToken.Production.List.Instance</i>
type. It is a series of <i>productions</i> strung together using <i>and</i>
operators.
<p>So in that case, what is a production? A production is a relation on
the <i>&lt;=</i> operator between a nonterminal token on the left hand
side and a right hand side consisting of a list of tokens (and perhaps
an attribute synthesization routine). Token lists on the right hand side
are built using the <i>&amp;</i> operator, and synthesization routines
can be tacked on using the <i>+</i> operator. Note that the <i>&lt;=</i>
operator in this context is the "is derived from" operator. It has nothing
to do with one side being less than or equal to another.
<p>The idea is that the nonterminal on the left-hand side of the first
production symbolizes the entire language. The tokens on its right hand
side represent tokens that the language may be composed of (or that it
may be decomposed into). For each token in that list that is a non-terminal
(derived from Nonterminal.Instance), there must be one or more productions
describing what tokens it is composed of. Some of those tokens may also
be nonterminals, in which case there may need to be more productions describing
what tokens they are derived from. Ultimately, all nonterminals must somehow
derive from a series of terminals. It is also important that left-hand
token of the first production does not appear in any other production.
Depending on the parser used, there may be extra restrictions on the grammar
as well. For instance, many parsers cannot handle grammars that are ambiguous
(could produce more than one derivation sequence using the parser's derivation
method).
<p>Terminals are <i>not</i> described using productions, as they will be
generated by the lexical analyzer from series of characters.
<p>Below is the grammar definition for our example language. I have also
included the textual definition provided by the dragon book for comparison
purposes
<blockquote><tt>--------------------------------------------------------------------------</tt>
<br><tt>-- Define the Grammar. The text in the example in the book looks
something</tt>
<br><tt>-- like:</tt>
<br><tt>--</tt>
<br><tt>-- S' -> S</tt>
<br><tt>-- S&nbsp; -> L = R | R</tt>
<br><tt>-- L&nbsp; -> * R | id</tt>
<br><tt>-- R&nbsp; -> L</tt>
<br><tt>--</tt>
<br><tt>Grammar : constant Production_List.Instance :=</tt>
<br><tt>&nbsp; S_Prime &lt;= S &amp; EOF and</tt>
<br><tt>&nbsp; S&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;= L &amp; Equals
&amp; R and</tt>
<br><tt>&nbsp; S&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;= R and</tt>
<br><tt>&nbsp; L&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;= Asterix &amp;
R + Nonterminal.Synthesize_Self and</tt>
<br><tt>&nbsp; L&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;= ID + Nonterminal.Synthesize_Self
and</tt>
<br><tt>&nbsp; R&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;= L;</tt></blockquote>

<h5>
<a NAME="Step 7: Generating a"></a>Step 7: Generating a parser</h5>
Now that we have a grammar defined, we can use it to generate a parser.
For this example we will use the lalr(1) parser type in our instantiation
of OpenToken.Production.Parser.LALR. We just declare an object of that
type, and initialize it by calling its Generate function with the grammar
and the token analyzer.
<blockquote><tt>-- The lalr parser instance.</tt>
<br><tt>Test_Parser : LALR_Parser.Instance :=</tt>
<br><tt>&nbsp; LALR_Parser.Generate (Grammar&nbsp; => Grammar,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Analyzer => Analyzer</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt></blockquote>

<h5>
<a NAME="Parser Use"></a>Use</h5>
To use the parser, we just call its Parse routine, and the parser will
handle the rest. For our example program we want to first open up the file
that the token analyzer's text feeder uses.
<blockquote><tt>&nbsp;&nbsp; Test_File_Name : constant String := "Example.txt";</tt>
<br><tt>begin</tt>
<p><tt>&nbsp;&nbsp; Ada.Text_IO.Put ("Parsing file " &amp; Test_File_Name
&amp; "...");</tt>
<br><tt>&nbsp;&nbsp; Ada.Text_IO.Flush;</tt>
<p><tt>&nbsp;&nbsp; Ada.Text_IO.Open (File => Input_File,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Name => Test_File_Name,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Mode => Ada.Text_IO.In_File</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt>
<p><tt>&nbsp;&nbsp; LALR_Parser.Parse (Test_Parser);</tt>
<br>&nbsp;</blockquote>

<h5>
<a NAME="Synthesized"></a>Synthesized Attributes</h5>
So now that you can create a parser, lets look at how you get it to <i>do
</i>something.
<p>If you take a peek at one of the Token.Nonterminal packages, you will
notice some primitive operations with the word "synthesize" in them. One
of these routines will be dispatched to whenever a token is recognized.
Their job is to initialize a nonterminal based on their input parameters,
which are typically a list of tokens. Whenever the right hand side of a
production is matched a process called "reduction" occurs. This involves
all the tokens matching the right hand side of the production being deleted
and a new nonterminal token of the same type as the production's left hand
side token being created. The standard Nonterminal token is a tagged record
with only one field; the token ID. But you can derive from it your own
tokens with extra fields (called <i>attributes</i> in compiler lingo).
These attributes can store useful information like numeric values, symbol
table pointers, object code, etc.
<p>When a reduction of a production occurs, a synthesization routine of
your choosing is called with the list of tokens found on the right hand
side. If you didn't specify a routine when you defined the production,
the routine OpenToken.Token.Nonterminal.Synthesize_Default is called. Synthesize_Default
dispatches to the Default_Synthesize routine for your type. If you don't
override Default_Synthesize, it in turn dispatches to the Synthesize_By_Copying
routine for your type, passing it the first token on the right hand side
list. So by default nonterminals are created as copies of the first token
on the right hand side of the production, but you can override that default
on several levels, or just explicitly specify a different method. This
is important since you will probably have to explicitly override some synthesizations
in any grammar. Trying to use the default Synthesize_By_Copying will cause
a constraint error if the source token isn't in the target token's class.
<p>To help show how all this works, lets try a more complicated example
from the Dragon Book. Example 5.10 defines a simple desk calculator. It
has addition and multiplication operators, as well as parenthesis for association.
<br>&nbsp;
<h6>
<a NAME="Custom"></a>Custom Nonterminals</h6>
First off, we need to create a custom integer-valued nonterminal, as OpenToken
currently doesn't have such a thing in its nonterminal library. We do that
by creating a new type derived from Nonterminal. Since we want it in its
own package for modularity purposes, we need to make it a generic package
with the token generics as parameters. We want it to be able to synthesize
from integer literal tokens, so that package needs to be a parameter too.
<blockquote><tt>generic</tt>
<br><tt>&nbsp;&nbsp; with package Token&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
is new OpenToken.Token(&lt;>);</tt>
<br><tt>&nbsp;&nbsp; with package Token_List&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
is new Token.List;</tt>
<br><tt>&nbsp;&nbsp; with package Nonterminal&nbsp;&nbsp;&nbsp;&nbsp; is
new Token.Nonterminal (Token_List);</tt>
<br><tt>&nbsp;&nbsp; with package Integer_Literal is new Token.Integer_Literal;</tt>
<br><tt>package Simple_Integer_Token is</tt>
<p><tt>&nbsp;&nbsp; type Instance is new Nonterminal.Instance with private;</tt>
<p><tt>&nbsp;&nbsp; subtype Class is Instance'Class;</tt>
<p><tt>&nbsp;&nbsp; type Handle is access all Class;</tt>
<br>&nbsp;</blockquote>
Every nonterminal needs a constructor to create an initial token for the
grammar.
<blockquote><tt>function Get (ID&nbsp;&nbsp;&nbsp;&nbsp; : in Token.Token_ID;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value&nbsp; : in Integer := 0</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
) return Instance'Class;</tt></blockquote>
We also need a routine to retrieve the value of the token (so we can print
it on the screen when we are done). Additionally, we'd like to provide
custom implementations for the Synthesize_By_Copying routine.
<blockquote><tt>function Value (Subject : in Instance) return Integer;</tt></blockquote>

<blockquote><tt>procedure Synthesize_By_Copying (New_Token : out Instance;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID);</tt></blockquote>
This token also needs three new synthesization methods: One that synthesizes
by adding the values of the first and third tokens, one that synthesizes
by multiplying them, and one that synthesizes by copying the value of the
second token (for expressions enclosed in parenthesis). These are declared
as Nonterminal.Synthesize constants.
<blockquote>&nbsp;
<p><tt>&nbsp;&nbsp; Add_Integers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : constant
Nonterminal.Synthesize;</tt>
<br><tt>&nbsp;&nbsp; Multiply_Integers : constant Nonterminal.Synthesize;</tt>
<br><tt>&nbsp;&nbsp; Synthesize_Second : constant Nonterminal.Synthesize;</tt></blockquote>
In the private section, we make the full declaration of our instance, with
its attributes that we want to keep track of. In this case, that would
be an integer value for the calculator.
<blockquote>&nbsp;
<br><tt>private</tt>
<br><tt>&nbsp;&nbsp; type Instance is new Nonterminal.Instance with record</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Value : Integer;</tt>
<br><tt>&nbsp;&nbsp; end record;</tt></blockquote>
We also declare the actual attribute synthesization routines that the constants
refer to. Since they will be pointed to by procedure access objects, their
parameter profile must be identical to that of Nonterminal.Synthesize.
Note that since they operate on objects of Nonterminal.Class, they will
not dispatch. Hang around for a little longer, and I'll show you how we
get around that problem.
<blockquote>&nbsp;
<br><tt>&nbsp;&nbsp; procedure Synthesize_Add (New_Token : out Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID);</tt>
<br><tt>&nbsp;&nbsp; procedure Synthesize_Multiply (New_Token : out Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID);</tt>
<br><tt>&nbsp;&nbsp; procedure Synthesize_From_Second_Argument (New_Token
: out Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID);</tt>
<p><tt>&nbsp;&nbsp; Add_Integers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : constant
Nonterminal.Synthesize := Synthesize_Add'Access;</tt>
<br><tt>&nbsp;&nbsp; Multiply_Integers : constant Nonterminal.Synthesize
:= Synthesize_Multiply'Access;</tt>
<br><tt>&nbsp;&nbsp; Synthesize_Second : constant Nonterminal.Synthesize
:= Synthesize_From_Second_Argument'Access;</tt></blockquote>
For the package body, we now need to implement all of the routines we declared
in the spec. Get and Value are both quite trivial. For get we just return
an instance with all its fields properly initialized. For value we return
the value of the token.
<blockquote><tt>package body Simple_Integer_Token is</tt>
<p><tt>&nbsp;&nbsp; function Get (ID&nbsp;&nbsp;&nbsp;&nbsp; : in Token.Token_ID;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value&nbsp; : in Integer := 0) return Instance'Class is</tt>
<br><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return Instance'Class(Instance'(Nonterminal.Instance(Nonterminal.Get(ID))
with Value => Value));</tt>
<br><tt>&nbsp;&nbsp; end Get;</tt>
<p><tt>&nbsp;&nbsp; function Value (Subject : in Instance) return Integer
is</tt>
<br><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return Subject.Value;</tt>
<br><tt>&nbsp;&nbsp; end Value;</tt></blockquote>
Next is our overload of the inherited Synthesize_By_Copying routine. We
basically do what the default Nonterminal version of the routine did. But
we also want to copy the value of an integer terminal. To do that we check
the tag of the source token. If the source isn't in either our class or
Integer_Literal's class, we raise an Invalid_Synth_Argument with a descriptive
error message.
<blockquote><tt>&nbsp;&nbsp; procedure Synthesize_By_Copying (New_Token
: out Instance;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID) is</tt>
<br><tt>&nbsp;&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if Source in Integer_Literal.Class
then</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; New_Token := (Nonterminal.Instance(Nonterminal.Get(To_ID))
with</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value => Integer_Literal.Value(Integer_Literal.Class(Source)));</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; elsif Source in Class then</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; New_Token
:= (Nonterminal.Instance(Nonterminal.Get(To_ID)) with</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value => Instance(Source).Value);</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; else</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Exceptions.Raise_Exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(Nonterminal.Invalid_Synth_Argument'Identity,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"Token " &amp; Token.Token_ID'Image(To_ID) &amp; " cannot be synthesized
" &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"solely from a " &amp; Token.Token_ID'Image(Token.ID(Source)) &amp; "."</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; end if;</tt>
<br><tt>&nbsp;&nbsp;&nbsp; end Synthesize_By_Copying;</tt></blockquote>
Now we implement our new synthesization routines. Note that as classwide
routines, they will not dispatch. If we want dispatching behavior, the
way to do it is to call a primitive operation on Instance from within one
of these routines.
<p>The routines require us to have the handles of the first, second, and
third tokens in the source list. This is accomplished using a list iterator
from the <i>Token_List</i> package (OpenToken.Token.List). The first token
will be pointed to by the <i>Initial_Iterator</i> routine. Subsequent tokens
in the list can be reached by using the <i>Next_Token</i> procedure.
<p>If Source is not of type Instance, then the assignment will cause a
Constraint_Error. We trap that and raise Invalid_Synth_Argument with a
descriptive message for debugging.
<blockquote><tt>&nbsp;&nbsp; procedure Synthesize_Add (New_Token : out
Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID) is</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Left&nbsp; : Token_List.List_Iterator
:= Token_List.Initial_Iterator(Source);</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Right : Token_List.List_Iterator
:= Token_List.Initial_Iterator(Source);</tt>
<p><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -- Move "Right" over to the third
item;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Token_List.Next_Token (Right);</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Token_List.Next_Token (Right);</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; New_Token := Class(Instance'(Token.Instance(Token.Get
(To_ID)) with</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(Value (Class (Token_List.Token_Handle(Left).all)) +</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value (Class (Token_List.Token_Handle(Right).all))</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
)</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
)</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt>
<br><tt>&nbsp;&nbsp; exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; when Constraint_Error =></tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Exceptions.Raise_Exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (Nonterminal.Invalid_Synth_Argument'Identity,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"Token " &amp; Token.Token_ID'Image(To_ID) &amp; " cannot be synthesized
" &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"from a " &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Token.Token_ID'Image (Token.ID (Token_List.Token_Handle(Left).all) ) &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
" and a " &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Token.Token_ID'Image (Token.ID (Token_List.Token_Handle(Right).all) ) &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"."</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt>
<br><tt>&nbsp;&nbsp; end Synthesize_Add;</tt>
<p><tt>&nbsp;&nbsp; procedure Synthesize_Multiply (New_Token : out Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID) is</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Left&nbsp; : Token_List.List_Iterator
:= Token_List.Initial_Iterator(Source);</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Right : Token_List.List_Iterator
:= Token_List.Initial_Iterator(Source);</tt>
<p><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -- Move "Right" over to the third
item;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Token_List.Next_Token (Right);</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Token_List.Next_Token (Right);</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; New_Token := Class(Instance'</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(Token.Instance(Token.Get (To_ID)) with</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value => (Value (Class(Token_List.Token_Handle(Left).all)) *</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value (Class(Token_List.Token_Handle(Right).all))</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
)</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
)</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt>
<p><tt>&nbsp;&nbsp; exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; when Constraint_Error =></tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Exceptions.Raise_Exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (Nonterminal.Invalid_Synth_Argument'Identity,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"Token " &amp; Token.Token_ID'Image(To_ID) &amp; " cannot be synthesized
" &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"from a " &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Token.Token_ID'Image (Token.ID (Token_List.Token_Handle(Left).all) ) &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
" and a " &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Token.Token_ID'Image (Token.ID (Token_List.Token_Handle(Right).all) ) &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"."</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt>
<br><tt>&nbsp;&nbsp; end Synthesize_Multiply;</tt>
<p><tt>&nbsp;&nbsp; procedure Synthesize_From_Second_Argument (New_Token
: out Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Token.Token_ID) is</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Second&nbsp; : Token_List.List_Iterator
:= Token_List.Initial_Iterator(Source);</tt>
<p><tt>&nbsp;&nbsp; begin</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -- Move "Second" over to the second
item;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Token_List.Next_Token (Second);</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; New_Token := Class(Instance'(Nonterminal.Instance(Nonterminal.Get(To_ID))
with</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Value => Class(Token_List.Token_Handle(Second).all).Value));</tt>
<p><tt>&nbsp;&nbsp; exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; when Constraint_Error =></tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Exceptions.Raise_Exception</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (Nonterminal.Invalid_Synth_Argument'Identity,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"Token " &amp; Token.Token_ID'Image(To_ID) &amp; " cannot be synthesized
" &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
"solely from a " &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Token.Token_ID'Image</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(Token.ID (Token_List.Token_Handle(Second).all) ) &amp; ".");</tt>
<br><tt>&nbsp;&nbsp; end Synthesize_From_Second_Argument;</tt>
<p><tt>end Simple_Integer_Token;</tt>
<br>&nbsp;</blockquote>

<h6>
<a NAME="Defining the"></a>Defining the Parser</h6>
So now that we have declared our custom Nonterminal token, let's define
a parser. The first few steps are very much as they were in the previous
example. The description of the grammar from the dragon book that we are
trying to match is this:
<br>&nbsp;
<blockquote><tt>L -> E&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
print (L.val)</tt>
<br><tt>E -> E + T&nbsp;&nbsp;&nbsp;&nbsp; E.val := E1.val + T.val</tt>
<br><tt>E -> T</tt>
<br><tt>T -> T * F&nbsp;&nbsp;&nbsp;&nbsp; T.val := T1.val * F.val</tt>
<br><tt>T -> F</tt>
<br><tt>F -> ( E )&nbsp;&nbsp;&nbsp;&nbsp; F.val := E.val</tt>
<br><tt>F -> digit</tt></blockquote>
The stuff on the right is the synthesization actions that are to occur
when a production is reduced. The absence of an action implies that the
attributes of the first token on the right are used to create the new token.
<p>We start by declaring our Token_IDs:
<blockquote><tt>type Token_IDs is (Integer_ID, Left_Paren_ID, Right_Paren_ID,
Plus_Sign_ID,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Multiply_ID, EOF_ID, Whitespace_ID, L_ID, E_ID, T_ID, F_ID);</tt></blockquote>
We instantiate all our packages as in the previous example, with two new
additions. We'll use OpenToken's <i>Integer_Literal</i> token package for
our literal numbers, and a Nonterminal <i>Simple_Integer_Token</i> package
that we will write ourselves to represent numeric-valued non-terminals.
<blockquote><tt>package Integer_Literal is new Master_Token.Integer_Literal;</tt>
<br><tt>package Simple_Integer is new Simple_Integer_Token(Master_Token,
Token_List, Nonterminal, Integer_Literal);</tt></blockquote>
For our calculator we want to read lines from the terminal. But each line
should be a complete parse in and of itself. We could have done that by
making an end-of-line the final token in our top production. But instead
we'll use the custom text feeder <i>OpenToken.Text_Feeder.String.Instance</i>.
It returns to the analyzer strings we manually feed into it, with end of
file tacked on the end. We'll also need a couple of variables for reading
the strings from standard input.
<blockquote><tt>Line&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; :&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
String (1..1024);</tt>
<br><tt>Line_Length :&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Natural;</tt>
<br><tt>Feeder&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : aliased OpenToken.Text_Feeder.String.Instance;</tt></blockquote>
The tokens, analyzer are made much like the previous ones were, so we won't
dwell on that. We do want to create one last synthesization routine though.
This is the routine that gets run when the final (first) production is
reduced. Its job is to print the value of that nonterminal to the screen.
<blockquote><tt>&nbsp;&nbsp; procedure Print_Value (New_Token : out Nonterminal.Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Source&nbsp;&nbsp;&nbsp; : in&nbsp; Token_List.Instance'Class;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
To_ID&nbsp;&nbsp;&nbsp;&nbsp; : in&nbsp; Master_Token.Token_ID);</tt></blockquote>
Now here's where those synthesization routines come in. We define our grammar
in much the same way we defined the previous one. Just as in the dragon
book's example, productions that don't have a synthesization routine will
by synthesized by copying the attributes from the first token on the right.
Obviously if that token is not the the left hand side token's class, a
constraint error would normally occur. But we took care of that for the
case of Integer_Literals to Simple_Integers by overloading the default
<i><tt>Synthesize_By_Copying</tt></i>routine
for Simple_Integers. In cases where that is not the behavior we want, as
in the case of addition productions, we explicitly specify an appropriate
synthesization routine.
<blockquote>&nbsp;
<br><tt>Grammar : constant Production_List.Instance :=</tt>
<br><tt>&nbsp; L &lt;= E &amp; EOF&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ Print_Value'Access&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
and</tt>
<br><tt>&nbsp; E &lt;= E &amp; Plus &amp; T&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ Simple_Integer.Add_Integers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; and</tt>
<br><tt>&nbsp; E &lt;= T&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
and</tt>
<br><tt>&nbsp; T &lt;= T &amp; Times &amp; F&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ Simple_Integer.Multiply_Integers and</tt>
<br><tt>&nbsp; T &lt;= F&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
and</tt>
<br><tt>&nbsp; F &lt;= Left_Paren &amp; E &amp; Right_Paren + Simple_Integer.Synthesize_Second
and</tt>
<br><tt>&nbsp; F &lt;= Int_Literal;</tt></blockquote>
After doing this and creating our parser, (and implementing Print_Value)
we are ready to use the parser. The code below shows how this parser can
be used to repeatedly perform calculations on strings entered from the
keyboard. It will terminate when a blank line is entered).
<blockquote><tt>&nbsp;&nbsp; loop</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Text_IO.Get_Line(Line, Line_Length);</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exit when Line_Length = 0;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; OpenToken.Text_Feeder.String.Set</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (Feeder => Feeder,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Value&nbsp; =>
Line (1..Line_Length));</tt>
<p><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; LALR_Parser.Parse (Test_Parser);</tt>
<br><tt>&nbsp;&nbsp; end loop;</tt></blockquote>
When this code is run and the user types in
<blockquote><tt>3 * (5 + 7 * 2)</tt></blockquote>
The program responds with
<blockquote><tt>57</tt></blockquote>

<h4>
<a NAME="Recursive Descent"></a>Recursive Descent Parsing</h4>
When using this kind of parsing, you should consider a Token to be a parsable
entity. Each token has Parse routine which verifies that the input contains
a valid version of that token. A Token is either made up of combinations
of other tokens, or comes from the lexical analyzer directly (a "terminal"
token). So the entire language that your parser handles is itself one big
Token. Your job as a language designer is to define how your language's
token breaks down into its terminal tokens.
<p>Although there is this division between terminal and non-terminal tokens,
there really aren't a lot of separate parts and facilities to a recursive-decent
parser. Terminal tokens are parsed by reading them from the analyzer, while
non-terminals are parsed by looking for the right combination of other
tokens. But parsing is just an activity that is performed on tokens, and
everything is a token.
<p>This conceptual simplification should bring you a lot more freedom.
You could take an existing parser and:
<ul>
<li>
reuse some of its existing tokens in another parser.</li>

<li>
add a new token or two to it to extend the token's language.</li>

<li>
extend some of its tokens with new attributes and action routines so that
it parses the same language, but gathers different information in the process.</li>
</ul>

<p><br>The general process you follow in creating a recursive descent parser
is:
<ul>
<li>
<a href="#Step 1: Creating an Enumeration of">Create an enumeration</a>
listing all your tokens.</li>

<li>
Specify your tokens using an enumeration of token IDs, and actual token
objects.</li>

<li>
Create the lexical analyzer.</li>

<li>
Create a token to represent your entire language.</li>

<li>
(Iterative) Create any tokens you need to represent the token(s) you created
in the previous step</li>
</ul>
For our example this time, we will assume the grammar in Example 4.46 of
the dragon book.<sup><a href="#*">*</a></sup>:
<blockquote><tt>S' -> S</tt>
<br><tt>S&nbsp; -> L = R | R</tt>
<br><tt>L&nbsp; -> * R | id</tt>
<br><tt>R&nbsp; -> L</tt></blockquote>
For our implementation we will also use two extra tokens to designate the
end of the file, and whitespace in the text stream.
<h5>
Step 1: Creating an Enumeration of Token IDs</h5>
This step is the same as outlined in the <a href="#Lexical">section on
lexical analysis</a>.
<blockquote><tt>type Token_IDs is (Asterix_ID, ID_ID, Equals_ID, EOF_ID,
Whitespace_ID);</tt></blockquote>
Again, this is a very simple step once you know the list of tokens you
need. But of course figuring that out is not always so simple!
<br>Just like any other program, you cannot expect to sit down at the keyboard
and pound out a well-working parser without doing any design beforehand.
<h5>
Step 2: Instantiate Your Token Packages</h5>
This step is the same as outlined in the <a href="#Lexical">section on
lexical analysis</a> (and is still trivial). Simply instantiate the generic
OpenToken.Token.Enumerated package with your enumerated type. Then use
that package to instantiate an OpenToken.Token.Enumerated.Analyzer package.
<blockquote><tt>package Master_Token is new Opentoken.Token.Enumerated
(Token_IDs);</tt>
<br><tt>package Tokenizer is new Master_Token.Analyzer;</tt></blockquote>

<h5>
Step 3: Map the Terminal Token ID's to their recognizers and tokens</h5>
An object of type <i>Tokenizer.Syntax</i> must now be initialized. This
object will map all Terminal token ID's to both their recognizer and their
token object. The routine Tokenizer.Get creates a structure (called a <i>Recognizable_Token</i>)
which can be assigned into an analyzer syntax element to create the mapping.
<p>Additionally, there's a special version of the <i>Get</i> routine which
has no <i>New_Token</i> parameter. This form can be used to create a mapping
without first creating a token object. This saves us some work, at the
expense of a little heap storage. A Master_Token.Instance will be dynamically
allocated and assigned into the mapping.
<blockquote><tt>Syntax : constant Tokenizer.Syntax :=</tt>
<br><tt>&nbsp; (Asterix_ID&nbsp;&nbsp;&nbsp; => Tokenizer.Get (OpenToken.Recognizer.Keyword.Get
("*")),</tt>
<br><tt>&nbsp;&nbsp; ID_ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
=> Tokenizer.Get (OpenToken.Recognizer.Keyword.Get ("id")),</tt>
<br><tt>&nbsp;&nbsp; Equals_ID&nbsp;&nbsp;&nbsp;&nbsp; => Tokenizer.Get
(OpenToken.Recognizer.Keyword.Get ("=")),</tt>
<br><tt>&nbsp;&nbsp; EOF_ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =>
Tokenizer.Get (OpenToken.Recognizer.End_Of_File.Get),</tt>
<br><tt>&nbsp;&nbsp; Whitespace_ID => Tokenizer.Get (OpenToken.Recognizer.Character_Set.Get</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(OpenToken.Recognizer.Character_Set.Standard_Whitespace))</tt>
<br><tt>&nbsp;&nbsp; );</tt></blockquote>

<h5>
Step 4: Create the Tokens</h5>
Next we need to declare token handles for all our tokens. The terminals
can simply be assigned straight out of the Syntax (a renames would probably
work just as well). The nonterminals could be initialized later, but since
they will always point to the same token objects, we will save space and
initialize them now. If you initialize them here, you will want to do so
with the token type that they will be using. In later steps you will see
why the instance types I chose below were used. If you remember the example
we are using, the names of the nonterminal tokens (the ones on the left
side of a production) were S', S, L, and R. So following is their declarations:
<blockquote><tt>S_Prime :&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
OpenToken.Token.Handle;</tt>
<br><tt>S&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : constant OpenToken.Token.Handle
:= new OpenToken.Token.Selection.Instance;</tt>
<br><tt>L&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : constant OpenToken.Token.Handle
:= new OpenToken.Token.Selection.Instance;</tt>
<br><tt>R&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; : constant OpenToken.Token.Handle
:= new OpenToken.Token.Selection.Instance;</tt></blockquote>
Later I will also explain why S_Prime is not initialized and the others
are.
<blockquote>&nbsp;</blockquote>

<h5>
Step 5: Define a Lexical Analyzer</h5>

<h6>
Advanced: Declaring a Custom Text Feeder</h6>
In this case we would like to use a text feeder tied to our own file, rather
than <i>Ada.Text_IO.Current_Input</i>. To do this, first declare an (aliased)
object to serve as the file for <i>Text_IO.Open</i>s and <i>Close</i>s.
Then declare an <i>OpenToken.Text_Feeder.Text_IO.Instance</i>. Initialize
the text feeder with the <i>Create</i> routine, using the file object as
the <i>File_Ptr</i>.
<blockquote><tt>Input_File : aliased Ada.Text_IO.File_Type;</tt>
<br><tt>Feeder&nbsp;&nbsp;&nbsp;&nbsp; : aliased OpenToken.Text_Feeder.Text_IO.Instance
:=</tt>
<br><tt>&nbsp; OpenToken.Text_Feeder.Text_IO.Create (Input_File'Unchecked_Access);</tt></blockquote>
Now to create the analyzer itself, we just make an object of type <i>Tokenizer.Instance</i>,
and initialize it with our syntax and text feeder. This is trivial.
<blockquote><tt>Analyzer : Tokenizer.Instance := Tokenizer.Initialize (Syntax,
Feeder'access);</tt></blockquote>
In this example program, we're going to be reading from a set filename.
Let's also define that filename here in a constant, so its easy to change:
<blockquote><tt>Test_File_Name : constant String := "Example.txt";</tt></blockquote>
So far this should have all looked pretty familiar if you have gone through
the first example. That bit about creating tokens was new. But still most
of the steps were familiar, and the result is pretty much the same so far:
an analyzer. With the next step this will start to change, so get ready.
Here it comes...
<br>&nbsp;
<h5>
Step 6: Define the top-level (Language) token:</h5>
Let's recall the first (top-level) production in our language's grammar.
It was:
<blockquote><tt>S' -> S</tt></blockquote>
Now you may think this looks like a pretty silly rule. Well, you're absolutely
right, it is a silly rule! It was artificially put into this example in
order to make a table parser building algorithm work right. We are using
a table to parse, so its really just a waste. The smart thing to do would
be to just ditch this rule and start with S. But that wouldn't exactly
be playing fair, because then we aren't really doing anything for the production.
We could create a token that just looks for an S token when it parses.
But that's really a bit of a waste of time.
<p>So what we will do is just make S_Prime point to the same object as
S:
<blockquote><tt>S_Prime := S;</tt></blockquote>
Incidentally, this is the reason we didn't make an object for S_Prime,
like we did for the others.
<br>&nbsp;
<h5>
Step 7: Define an undefined token used in a previous step</h5>
We used S in the previous step. Its grammar rule was:
<blockquote><tt>S&nbsp; -> L = R | R</tt></blockquote>
Note that since this it the top level token, there is an implied EOF token
at the end of the production. So in other words, it is <b>either</b> the
sequence of tokens L, =, R, and EOF <b>or</b> it is a sequence of an R
and an EOF token. We could craft a custom token with a parser that recognizes
that pattern. But a sequence of tokens, and a selection between tokens
happen to be very common token types. For that reason there are prebuilt
Token.Sequence and Token.Selection tokens. Using those packages (with appropriate
"use type" declarations of course), we can define S thusly:
<blockquote><tt>S.all&nbsp;&nbsp; := OpenToken.Token.Selection.Class</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(OpenToken.Token.Sequence.New_Instance(L &amp; Equals &amp; R &amp; EOF)
or</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
OpenToken.Token.Sequence.New_Instance(R &amp; EOF));</tt></blockquote>

<h5>
Step 8: Define an undefined token used in a previous step</h5>
We used the nonterminal tokens L and R in the previous step. Let's define
L next. Its production was:
<blockquote><tt>L&nbsp; -> * R | id</tt></blockquote>
That's either a sequnce of * and R, or an id. Using the same two prebuilt
tokens, we get:
<blockquote><tt>L.all&nbsp;&nbsp; := OpenToken.Token.Selection.Class</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
(OpenToken.Token.Sequence.New_Instance(Asterix &amp; R) or ID);</tt></blockquote>

<h5>
Step 9: Define an undefined token used in a previous step</h5>
The only remaining undefined token is R, so we'll finish with it. Its production
was:
<blockquote><tt>R&nbsp; -> L</tt></blockquote>
Here is another of those silly identity productions again. But let's try
another tactic this time and make a separate R token that is a copy of
the L token.
<blockquote><tt>R.all&nbsp;&nbsp; := L.all;</tt></blockquote>
Use
<p>To use the parser, we just call the top-level token's Parse routine.
All our token's built-in parse routines will handle the rest. For our example
program we want to first open up the file that the token analyzer's text
feeder uses. Then we must call the token analyzer once to load up the first
token. Token parser routines assume the currently loaded token is the first
one they are to look at.
<blockquote><tt>Ada.Text_IO.Open (File => Input_File,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Name => Test_File_Name,</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Mode => Ada.Text_IO.In_File</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
);</tt><tt></tt>
<p><tt>-- Load up the first token</tt>
<br><tt>Tokenizer.Find_Next (Analyzer);</tt><tt></tt>
<p><tt>OpenToken.Token.Parse</tt>
<br><tt>&nbsp; (Match&nbsp;&nbsp;&nbsp; => S_Prime.all,</tt>
<br><tt>&nbsp;&nbsp; Analyzer => Analyzer</tt>
<br><tt>&nbsp;&nbsp; );</tt></blockquote>
That's it! If you want to perform error handling, you may consider adding
something like the following exception handler:
<blockquote><tt>exception</tt>
<br><tt>&nbsp;&nbsp; when Error : OpenToken.Token.Parse_Error =></tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Text_IO.Put_Line ("failed at
line" &amp; Integer'Image(Tokenizer.Line (Analyzer)) &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
", column" &amp; Integer'Image(Tokenizer.Column (Analyzer)) &amp;</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
" due to parse exception:");</tt>
<br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Ada.Text_IO.Put_Line (Ada.Exceptions.Exception_Information
(Error));</tt></blockquote>

<h5>
Caveats:</h5>
Note that recursive descent parsers do not handle "left-recursion" very
well. This is a situation where the definition of a token has that same
token as its first possiblity. This would normally cause infinite recursion
in the parser. If you are implementing the parse routine yourself, this
situation is quite obvious, but its something to be aware of when using
a prebuilt Token package.
<p>
<hr WIDTH="100%"><a NAME="*"></a><sup>*</sup>&nbsp; This is the classic
text on compiler theory. Note that for this example we have some minor
modifications to the syntax to keep things simple. For instance, the "num"
terminal has been split into the following 2 terminals:
<blockquote><tt><b>integer</b> -> (+ | -)? <b>digit</b>+</tt>
<br><tt><b>real</b> -> (+ | -)? (<b>digit</b> | _)* <b>digit</b> . (<b>digit</b>
| _)<sup>*</sup> ( (e | E) (- | +)? (<b>digit</b>)<sup>+</sup> )?</tt></blockquote>
This change has been made simply because it matches the definition used
for the Integer and Real tokens provided with the OpenToken package. A
joint "num" token could have been created to exactly match the num specified
in ASD, but we will leave that as an exercise for the reader.
<br>
<hr WIDTH="100%">
<h3>
Revisions</h3>

<pre><font size=-1>$Log: UsersGuide.html,v $
<pre><font size=-1>Revision 1.5  2000/08/14 03:05:08  Ted
<pre><font size=-1>Add recursive-descent example for 3.0b release
<pre><font size=-1></font></pre>

<pre><font size=-1>Revision 1.4&nbsp; 2000/02/05 04:08:10&nbsp; Ted</font></pre>

<pre><font size=-1>Fix typo</font></pre>

<pre><font size=-1>Revision 1.3 2000/01/27 21:08:56 Ted Add two examples to illustrate the new parsing facility. Spell check.</font></pre>

<pre><font size=-1>Revision 1.2 1999/08/17 03:21:41 Ted Add log line</font></pre>

</body>
</html>