1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
The lexical analyzer's job is to convert characters or sequences of characters
read from the input stream into tokens. B()'s parser obtains its tokens by
calling its lexical analyzer member (tt(lex)), which is a predefined member
of the parser class. See section ref(LEX).
Our calculator only needs a simple lexical analyzer. It skips blanks and tabs,
then reads numbers, returning them as tt(NUM) tokens. Any other character that
isn't part of a number is a separate token. Note that the token-value for such
a single-character token is the character itself.
The lexical analyzer function's return value is a numeric value representing a
token. If the token is a literal character, then its numeric code is the
character's tt(ASCII) code; if the token is the name of a token defined by b()
in a grammar specification file, then such named tokens are defined by b() in
a bf(C++) enumeration. In the current example, therefore, tt(NUM) becomes an
enumeration value, returned by the tt(lex) member as the `value' tt(NUM).
Semantic values of nonterminals are stored in the parser's data member
tt(d_val_) (comparable to the variable tt(yylval) used by, e.g., Bison). This
data member has tt(int) as its default type, but by specifying tt(%stype) in
the grammar file's directive section this default type is modified (to, e.g.,
tt(double)).
A token value of zero is returned once end-of-file is encountered (the parsing
function produced by b() interprets any nonpositive token value as
end-of-input).
Here is the lexical scanner's implementation:
verbinclude(rpn/parser/lex.cc)
|
