1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
|
/*!
\page parsing "Console output parsing"
This atricle describes, how does MkS parses console output of a compiler, interpreter, etc; and how can you create own parser.
\section introduction "Introduction"
MkS cabable to parse output from console probramms, such as builders, compilers, interpreters, running applications and others, executed by it.
Output parsed for display it in good readable form on "Build steps" dock, and, probably may be used for some other reasons.
For example, when project C++ builded on Linux, "GNU Make" and "GCC" parsers collects errors from build output and display it in good readable form.
An extension module, which supports parsing output of some programm, named "Parser".
This guide gives hint, how to develop own parsers. As base, used MkS revison 3518, at svn//svn.tuxfamily.org/svnroot/monkeystudio/mks/v2/branches/parsing. Remeber, that some things may be changed at the time, when you reading this guide.
\section creating "Creating own parsers"
There are two ways for create own parser.
\subsection subclassabstraceparser "Subclass AbstractCommandParser"
The most powerful way.
Subclass AbstractCommandParser, implement own
\code
virtual int AbstractCommandParser::processParsing(QString* text) = 0;
\endcode
Then add parser using
\code
void pConsoleManager::addParser( AbstractCommandParser* p )
\endcode
Advance of this method - you are able to do with console output whatever you won't in your parser.
\subsection useCommandParser "Use CommandParser class"
Easier.
This class it implementation of AbstractCommandParser, currently (23-12-2009) used for parse GCC and GNU Make output.
It is based on idea of "pattern".
Pattern is regular expression, which searches some phrases in output, and set of ruses, which allows to convert regular expression capture to message, displayable on "Build Steps" dock. See structure CommandParser::Pattern.
Example of contents of such structure:
\code
{
// middle part of error
// src/views/TreeViewModel.h:9: note: because the following virtual functions are pure within 'TreeViewModel':
QRegExp("^([\\w\\\\/\\.\\:\\d\\-]+):(\\d+): note: ([^\\n]+)",
Qt::CaseSensitive,
QRegExp::RegExp2), //reg exp
"%1", //file name
"0", //column
"%2", //row
pConsoleManager::stError, //type
"%3", //text
"%0", //full text
},
\endcode
This pattern intended for search string
\verbatim
src/views/TreeViewModel.h:9: note: because the following virtual functions are pure within 'TreeViewModel':
\endverbatim
extract from it filename, "src/views/TreeViewModel.h", line "9", text of note "because the following virtual functions are pure within 'TreeViewModel'"
When processing pattern, %<number> (%1, %2, ...) replaced with subcapture from regular expression, and %0 replaced with full text of capture.
As result, when parsing
\verbatim
src/views/TreeViewModel.h:9: note: because the following virtual functions are pure within 'TreeViewModel':
\endverbatim
will be generated entry for "Build steps" dock, with text "because the following virtual functions are pure within 'TreeViewModel'", and hint "src/views/TreeViewModel.h:9: note: because the following virtual functions are pure within 'TreeViewModel':". When entry clicked, src/views/TreeViewModel.h file will be opened, cursor will jump to line 9 column 0.
For create own parser, create CommandParser instance, and fill it with patterns, using function
\code
void CommandParser::addPattern(const Pattern& pattern);
\endcode
Than install parser, using
\code
void pConsoleManager::addParser( AbstractCommandParser* p )
\endcode
\subsection MkSScript "MkS script"
Even easier, do not requires any C++ programming.
MkS scripting interface supports adding parsers by .mks scripts. Internaly, script creates CommandParser and fills it with Patterns.
You can create scripts dirrectly, but, I think, it's easier to use framework, writen in Python, for generate it. Framework allows to create patterns, test it, and generate resulting MkS script with parser. It requires only minimal Python knowlege (I think, it's possible to create parsers even without Python knowlege, bug with examples.)
Here is step-by-step instruction, which uses existing GNU Make parser as example. You can find full file in outputparsing/gnumake.py
- Creating Python file gnumake.py with next contents:
\code
import parsing
print '# It is a machine generated file. Do not edit it manualy!'
\endcode
First line imports framework functionality, last - prints text to standard output.
- Adding first pattern.
\code
# No rule for make target
noRule = parsing.Pattern(r"^((mingw32\-)?make: \*\*\* No rule to make target.*) Stop.",
type = 'error',
text = '%1')
noRule.setComment('No rule for make target')
\endcode
Line creates parsing.Pattern class instance. It requires one mandatory parameter - regular expression, and few named optional parameters, having default values:
\code
file = '',
line = '-1',
column = '-1',
type = 'error',
text = '%0',
hint = '%0'
\endcode
In this case, created pattern (CommandParser::Pattern), having next values:
\code
regExp = "^((mingw32\-)?make: \*\*\* No rule to make target.*) Stop."
FileName = "" // empty
col = "-1" // default value for the framework
row = "-1" // defaut value for the framework
Type = pConsoleManager::stError
Text = "%0"
FullText = "%0"
\endcode
- Set comment for this pattern
\code
noRule.setComment('No rule for make target')
\endcode
Comment used for better .mks script readability
- Let's test the pattern:
\code
noRule.test("mingw32-make: *** No rule to make target `release'. Stop.\n",
type = 'error',
text = "mingw32-make: *** No rule to make target `release'.",
hint = "mingw32-make: *** No rule to make target `release'. Stop.")
\endcode
Input string
\verbatim
"mingw32-make: *** No rule to make target `release'. Stop.\n"
\endverbatim
passed to the parser, and checked, that result (capture) is:
\verbatim
type = pConsoleManager::stError
text = "mingw32-make: *** No rule to make target `release'."
"mingw32-make: *** No rule to make target `release'. Stop."
\endverbatim
Every pattern can have as much tests, as needed
- Creating second parser:
\code
# Entering directory
entering = parsing.Pattern(r"^(mingw32\-)?make\[\d\]: Entering directory\s`([^\n]*)'",
type = 'compiling',
text = 'make: Building %2')
entering.setComment('Entering dirrectory')
\endcode
- Print created patterns to console:
\code
print noRule.generateMkSScript('GNU Make')
print entering.generateMkSScript('GNU Make')
\endcode
'GNU Make' is name for parser
- OK, now let's test created parser (console command):
\code
python gnumake.py
\endcode
Output:
\verbatim
# It is a machine generated file. Do not edit it manualy!
# No rule for make target
parser add "GNU Make" "^((mingw32\-)?make: \*\*\* No rule to make target.*) Stop." "" "-1" "-1" "error" "%1" "%0"
# Entering dirrectory
parser add "GNU Make" "^(mingw32\-)?make\[\d\]: Entering directory\s`([^\n]*)'" "" "-1" "-1" "compiling" "make: Building %2" "%0"
\endverbatim
- Now let's change first pattern, for make it incorrect:
\code
noRule = parsing.Pattern(r"^((mingw32\-)?make: \*\*\* No rule to make target.*) Stop",
type = 'error',
text = '%1')
(Removed "." from the end of the regular expression)
\endcode
- Trying to generate MkS script:
\code
a@a-laptop:~/code/mks/v2/branches/parsing/outputparsing$ python gnumake.py
hint <mingw32-make: *** No rule to make target `release'. Stop> != <mingw32-make: *** No rule to make target `release'. Stop.>
Traceback (most recent call last):
File "gnumake.py", line 13, in <module>
hint = "mingw32-make: *** No rule to make target `release'. Stop.")
File "/home/a/code/mks/v2/branches/parsing/outputparsing/parsing.py", line 82, in test
assert(not failed)
AssertionError
\endcode
At the second line we can see, that new regular expression gave invalid hint. Dot at the output hint is missing.
- When running script with "--debug" parameter, output contains more usefull info:
\verbatim
a@a-laptop:~/code/mks/v2/branches/parsing/outputparsing$ python gnumake.py --debug
For <No rule for make target>
Full match: < mingw32-make: *** No rule to make target `release'. Stop >
{'hint': "mingw32-make: *** No rule to make target `release'. Stop", 'column': '-1', 'text': "mingw32-make: *** No rule to make target `release'.", 'file': '', 'line': '-1', 'type': 'error'}
hint <mingw32-make: *** No rule to make target `release'. Stop> != <mingw32-make: *** No rule to make target `release'. Stop.>
Traceback (most recent call last):
File "gnumake.py", line 13, in <module>
hint = "mingw32-make: *** No rule to make target `release'. Stop.")
File "/home/a/code/mks/v2/branches/parsing/outputparsing/parsing.py", line 82, in test
assert(not failed)
AssertionError
\endverbatim
MkS script won't be printed, if any of unit tests for any of parsers failed.
- Restore valid regular expression, and let's generate all parsers:
\code
a@a-laptop:~/code/mks/v2/branches/parsing/outputparsing$ ./generate-parsers.sh
\endcode
This script will create in "datas/scripts" dirrectory files with parsers. For GNU Make output is:
\verbatim
a@a-laptop:~/code/mks/v2/branches/parsing/outputparsing$ cat ../datas/scripts/parser-gnumake.mks
# It is a machine generated file. Do not edit it manualy!
# No rule for make target
parser add "GNU Make" "^((mingw32\-)?make: \*\*\* No rule to make target.*) Stop." "" "-1" "-1" "error" "%1" "%0"
# Entering dirrectory
parser add "GNU Make" "^(mingw32\-)?make\[\d\]: Entering directory\s`([^\n]*)'" "" "-1" "-1" "compiling" "make: Building %2" "%0"
\endverbatim
- Remember, that MkS stores it's scripts in
\verbatim
{HOME DIRECTORY}.Monkey\ Studio/scripts-{VERSION}/
\endverbatim
and it copied only during first start. So, you might need copy it manualy, for make sure it is installed, and MkS will find it on first start.
Remember, that Python language allows you to make parsers generation easier, if you know it. Probably you can find few usable examples at outputparsing/gcc.py
*/
|
