1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
|
Parser Toolkit
==============
PTK - (c) Jérôme Laheurte 2015-2019
.. contents:: **Table of contents**
What is PTK ?
-------------
PTK is a LR(1) parser "generator" for Python. It is not actually a
"generator" in the sense that it doesn't output source code, using
Python's dynamic nature to build everything it needs at runtime
instead. Also, it supports asynchronous parsing; see the API
documentation for details.
This code is licensed under the `GNU LGPL version 3 or, at your
option, any later version
<https://www.gnu.org/copyleft/lesser.html>`_.
Why another one of those ?
--------------------------
There are a number of parser generators for Python out there. Most of
them only support LL(1) or PEG. The other ones are either
- Unmaintained
- Straight translations from Yacc/Bison, and thus use an ugly syntax
- All of the above
The main goals of PTK are
- Clean, compact, Python-friendly syntax
- Support for asynchronous input streams: why would you need the
whole input string to start working when the underlying system is
actually an automaton ?
- Play nice in 'special' cases, like when the underlying
'filesystem' is a PyZipFile archive.
- Don't use hacks like module-level introspection to compensate for
an ugly design (I'm looking at you PLY). Those tend to produce
subtle and headache-inducing bugs when running from compiled code.
Supported platforms
-------------------
All unit tests pass on the following platforms/Python version:
+-----+-------+-----+---------+
| | Linux | OSX | Windows |
+=====+=======+=====+=========+
| 3.3 | | | X |
+-----+-------+-----+---------+
| 3.4 | | | X |
+-----+-------+-----+---------+
| 3.5 | | | X |
+-----+-------+-----+---------+
| 3.6 | X | | X |
+-----+-------+-----+---------+
| 3.7 | | X | X |
+-----+-------+-----+---------+
Installation
------------
Using pip::
$ pip install -U ptk
From source::
$ wget https://pypi.python.org/packages/source/p/ptk/ptk-%(version)s.tar.gz
$ tar xjf ptk-%(version)s.tar.bz2; cd ptk-%(version)s
$ sudo python3 ./setup.py install
Sample usage
------------
Four-operations integer calculator:
.. code-block:: python
from ptk.parser import LRParser, production, leftAssoc
from ptk.lexer import ReLexer, token
import operator
@leftAssoc('+', '-')
@rightAssoc('*', '/')
class Parser(LRParser, ReLexer):
@token('[1-9][0-9]*')
def number(self, tok):
tok.value = int(tok.value)
@production('E -> number<n>')
def litteral(self, n):
return n
@production('E -> "-" E<val>', priority='*')
def minus(self, val):
return -val
@production('E -> "(" E<val> ")"')
def paren(self, val):
return val
@production('E -> E<left> "+"<op> E<right>')
@production('E -> E<left> "-"<op> E<right>')
@production('E -> E<left> "*"<op> E<right>')
@production('E -> E<left> "/"<op> E<right>')
def binaryop(self, left, op, right):
return {
'+': operator.add,
'-': operator.sub,
'*': operator.mul,
'/': operator.floordiv
}[op](left, right)
parser = Parser()
while True:
expr = input('> ')
print parser.parse(expr)
Code samples
------------
The *samples* subdirectory in the source tree contains the
aforementioned calculator and a script that generates a skeleton
Python file from a Yacc or Bison grammar file.
API documentation
-----------------
The full documentation is hosted `here <http://ptk.readthedocs.io/en/release-%(version)s/>`_.
Changelog
---------
Version 1.3.8:
- Add more error handling stuff
Version 1.3.7:
- Add token and production positions in source
Version 1.3.6:
- Drop support for Python versions older than 3.3.
Version 1.3.5:
- Update copyright notices
- Fix some packaging issues
Version 1.3.4:
- Added SkipToken for consumers
Version 1.3.3:
- Fix a number of problems when working with bytes in Python 3
- One couldn't mix asynchronous and synchronous parsers in the same program
Version 1.3.2:
- Fix Python regular expression based lexer (use match instead of search)
Version 1.3.1:
- Fix version number in README.rst
Version 1.3.0:
- Added deferred_lexer and deferred_parser (asynchronous parsing using
Twisted Deferred objects)
- Asynchronous classes cannot be imported from 'regular' modules
anymore, import them explicitely from 'ptk.async_lexer' and 'ptk.async_parser'.
Version 1.2.0:
- Production methods cannot have the same name any more. This was
idiotic to begin with. Inheritance thus works as expected.
- Add AsyncLexer and AsyncLRParser for asynchronous parsing.
Version 1.1.0:
- Added repeat operators ('*', '+', '?') in production syntax.
- Support for more yacc/bison declarations in yacc2py sample (most are
actually ignored)
|
