1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
|
<html>
<head>
<title>OPARI: OpenMP Pragma And Region Instrumentor</title>
</head>
<body background="../images/back.jpg">
<div align="center">
<img src="opari-logo-100.gif">
</div>
<h1 align="center"><font color="#FF0000">O</font>penMP
<font color="#FF0000">P</font>ragma
<font color="#FF0000">A</font>nd
<font color="#FF0000">R</font>egion
<font color="#FF0000">I</font>nstrumentor - </h1>
<div align="center">
Version 1.1, October 2001
<p>
© 2001 Forschungszentrum Jülich, ZAM, Germany
<br>Bernd Mohr
</div>
<p>
<div align="justify">
<font color="#0000FF">OPARI</font> is a source-to-source translation tool
which automatically adds all necessary calls to the
<font color="#0000FF">pomp</font> runtime measurement library which
allows to collect runtime performance data of Fortran, C, or C++ OpenMP
applications. It is based on the idea of OpenMP pragma/directive
rewriting which is described in detail in a paper
(<a href="docs/lacsi01.ps.gz">PostScript</a>,
<a href="docs/lacsi01.pdf">PDF</a>)
for <a href="http://lacsi.lanl.gov/symposium/">LACSI'01</a>.
<p>
<font color="#0000FF">OPARI</font> was developed as part of the
<a href="http://www.fz-juelich.de/zam/kojak/">KOJAK</a> and
<a href="http://www.acl.lanl.gov/tau/">TAU</a> projects.
<h2><hr>DOWNLOAD<hr></h2>
This software is <b>free</B> but <b>copyright</B> © 2001 by
Forschungszentrum Juelich, ZAM, Germany. By downloading and using this
software you automatically agree to comply with the regulations as
described in the <a href="LICENSE">license agreement</a>.
<p>
<table border=1>
<tr>
<th colspan=3>Sources in gzipped tar format
</tr>
<tr>
<th>Version
<th>Date
<th>Description
</tr>
<tr>
<td><a href="opari-pomp-1.1.tar.gz">1.1</a>
<td>17-Oct-2001
<td><a href="CHANGES">Changes</a>
</tr>
<tr>
<td><a href="opari-pomp-1.0.tar.gz">1.0</a>
<td>28-Aug-2001
<td>Initial version
</tr>
</table>
<p>
<h2><hr>USAGE<hr></h2>
Before compiling the source files of an OpenMP application, each file
needs to be transformed by a call to the <font color="#0000FF">OPARI</font>
tool. In addition, the application has to be linked against the
<font color="#0000FF">pomp</font> runtime measurement library and the
<font color="#0000FF">OPARI</font> runtime table file. The latter has to
be generated by using the <tt><b>-table</b></tt> option to
<font color="#0000FF">OPARI</font> either together with the
transformation of the <b>last</b> input source file or with a separate call
to <font color="#0000FF">OPARI</font> after all transformations are done.
A call to <font color="#0000FF">OPARI</font> has the following syntax:
</div>
<pre><b>
opari [-f77|-f90|-c|-c++] [-nosrc] [-disable <i>constructs</i>]
[-rcfile <i>file</i>] [-table <i>tabfile</i>] <i>infile</i> [outfile]
</b>or:<b>
opari [-rcfile <i>file</i>] -table <i>tabfile</i>
</b></pre>
<div align="justify">
The options and parameters have the following meaning:
<table cellspacing=10>
<tr>
<td valign="top"><tt><b>[-f77|-f90|-c|-c++]</b></tt>
<td valign="top">[OPTIONAL] Specifies the programming language
of the input source file. This option is only
necessary if the automatic language detection
based on the input file suffix fails.
</tr>
<tr>
<td valign="top"><tt><b>[-nosrc]</b></tt>
<td valign="top">[OPTIONAL] If specified,
<font color="#0000FF">OPARI</font> does not generate
<tt><b>#line</b></tt> constructs in the transformation process
which allow to preserve the original source file
and line number information. This option might
be necessary if the OpenMP compiler does not
understand <tt><b>#line</b></tt> constructs. The default is to
generate <tt><b>#line</b></tt> constructs.
</tr>
<tr>
<td valign="top"><tt><b>[-rcfile <i>file</i>]</b></tt>
<td valign="top">[OPTIONAL] <font color="#0000FF">OPARI</font>
uses the file <tt><b>./opari.rc</b></tt> to preserve state information
between calls to <font color="#0000FF">OPARI</font> if the OpenMP
application consists of more than one source file. With the
<tt><b>-rcfile</b></tt> option the file <tt><b><i>file</i></b></tt> is
used instead. This can be useful if more than one application is
stored in the same directory or if the source files of an application
are stored in more than more directory.
</tr>
<tr>
<td valign="top"><tt><b>-table <i>tabfile</i></b></tt>
<td valign="top">Generate the <font color="#0000FF">OPARI</font> runtime
table in file <tt><b><i>tabfile</i></b></tt>.
This option has to be used either together with the
call to <font color="#0000FF">OPARI</font> for the transformation of
the <b>last</b> input source file or with a separate call to
<font color="#0000FF">OPARI</font> after all transformations are done.
</tr>
<tr>
<td valign="top"><tt><b>-disable <i>constructs</i></b></tt>
<td valign="top">[OPTIONAL] Disable the instrumentation of
the more fine-grained OpenMP constructs such as <tt><b>!$OMP</b></tt>
<tt><b>ATOMIC</b></tt>. <tt><b><i>constructs</i></b></tt>
is a comma separated list of the constructs for which the
instrumentation should be disabled. Accepted tokens are
<tt><b>atomic</b></tt>, <tt><b>critical</b></tt>, <tt><b>master</b></tt>,
<tt><b>flush</b></tt>, <tt><b>single</b></tt>, or <tt><b>locks</b></tt>
as well as <tt><b>sync</b></tt> to disable all of them.
</tr>
<tr>
<td valign="top"><tt><b><i>infile</i></b></tt>
<td valign="top">Input file name.
</tr>
<tr>
<td valign="top"><tt><b>[<i>outfile</i>]</b></tt>
<td valign="top">[OPTIONAL] Output file name. If not specified,
<font color="#0000FF">OPARI</font> uses the name
<tt><b><i>infile</i>.mod.<i>suffix</i></b></tt> if
the input file is called <tt><b><i>infile</i>.<i>suffix</i></b></tt>.
</tr>
</table>
In addition to the modified output file, <font color="#0000FF">OPARI</font>
also generates a file named <tt><b><i>infile</i>.opari.inc</b></tt>. It
contains OpenMP region descriptors, one for each OpenMP region found in the
input file. The meaning of the descriptor fields are described in the file
<a href="lib/pomp_lib.h"><tt><b>pomp_lib.h</b></tt></a>.
<p>
In summary, the typical usage of <font color="#0000FF">OPARI</font> consists of the following steps:
<ol>
<li>Reset <font color="#0000FF">OPARI</font> state information by removing
the state information file if it exists.
<pre><b>
% rm -f opari.rc
</b></pre>
<li>Call <font color="#0000FF">OPARI</font> for each input source file
<pre><b>
% opari file1.f90
...
% opari fileN.f90
</b></pre>
<li>Generate the <font color="#0000FF">OPARI</font> runtime table and
compile it using a ANSI C compiler
<pre><b>
% opari -tabfile opari.tab.c
% cc -c opari.tab.c
</b></pre>
<li>Compile all modified output files <tt><b>*.mod.*</b></tt> using the OpenMP
compiler
<p>
<li>Link the resulting object files against the
<font color="#0000FF">OPARI</font> runtime table
<tt><b>opari.tab.o</b></tt> and the <font color="#0000FF">pomp</font>
runtime measurement library.
</ol>
<h2><hr>LIMITATIONS<hr></h2>
<font color="#0000FF">OPARI</font> understands all OpenMP constructs by the
<a href="http://www.openmp.org/specs/">Fortran 77/90 OpenMP 2.0</a> and
<a href="http://www.openmp.org/specs/">C/C++ OpenMP 1.0</a> specifications
as well as the OpenMP extension <tt><b>INST</b></tt> directives/pragmas
and the alternative <tt><b>POMP</b></tt> sentinel proposed in
<a href="docs/lacsi01.ps.gz">lacsi01.ps.gz</a> /
<a href="docs/lacsi01.pdf">lacsi01.pdf</a>.
<h3>Limitations due to fuzzy parsing</h3>
Because <font color="#0000FF">OPARI</font> does not contain full parsers for
the supported programming languages the following restrictions apply:
<h4>Fortran 77/90:</h4>
<ol>
<li> The <tt><b>!$OMP</b></tt> <tt><b>END</b></tt> <tt><b>DO</b></tt> and
<tt><b>!$OMP</b></tt> <tt><b>END</b></tt> <tt><b>PARALLEL</b></tt>
<tt><b>DO</b></tt> directives
are required (and not optional as described in the OpenMP specification)
<p>
<li> The <i>atomic expression</i> controlled by a
<tt><b>!$OMP</b></tt> <tt><b>ATOMIC</b></tt>
directive has to be on a line all by itself.
<p>
<li> If the measurement environment does not support the automatic
recording of user function entries and exits, the
<font color="#0000FF">OPARI</font> runtime
measurement library has to be initialized by a
<tt><b>!$OMP</b></tt> <tt><b>INST</b></tt> <tt><b>INIT</b></tt>
directive prior to any other OpenMP directive.
</ol>
<h4>C/C++:</h4>
<ol>
<li> <i>structured blocks</i> describing the extend of an OpenMP pragma
need to be either compound statements <tt><b>{....}</b></tt>,
<tt><b>while</b></tt> loops, or simple statements. In addition,
<tt><b>for</b></tt> loops are supported after
<tt><b>omp</b></tt> <tt><b>for</b></tt> and <tt><b>omp</b></tt>
<tt><b>parallel</b></tt> <tt><b>for</b></tt> pragmas.
Complex statements like
<tt><b>if-then-else</b></tt> or <tt><b>do-while</b></tt> need to
be enclosed in a block ( <tt><b>{....}</b></tt> ).
<p>
<li> If the measurement environment does not support the automatic
recording of user function entries and exits, the
<font color="#0000FF">OPARI</font> runtime measurement library has to
be initialized by a <tt><b>omp</b></tt> <tt><b>inst</b></tt>
<tt><b>init</b></tt> pragma prior to any other OpenMP pragma.
</ol>
We did not find these limitations overly restrictive during our tests and
experiments. They rarely apply for well-written code. If they do, the
original source code can easily be fixed. Of course, it would be possible
to remove these limitations by enhancing <font color="#0000FF">OPARI</font>`s
parsing sophistication.
<h3>Limitations due to source-to-source translation</h3>
In addition, because of some subtleties in the OpenMP standard specifications,
the transformations performed by <font color="#0000FF">OPARI</font> on the
source code level can differ from the same instrumentation done by a real
OpenMP compiler. Here is the list of limitations we currently know about:
<ul>
<li><font color="#0000FF">OPARI</font> makes implicit barriers explicit.
Unfortunately, this method cannot be used for measuring the barrier waiting
time at the end of <tt><b>PARALLEL</tt></b> directives because they do not
allow a <tt><b>NOWAIT</tt></b> clause. Therefore, we add an explicit
barrier with corresponding performance interface calls here. For <font
color="#0000FF">OPARI</font>, this means that actually two barriers get
called. But the second (implicit) barrier should execute and succeed
immediately because the threads of the OpenMP team are already synchronized
by the first barrier.
<p>
<li>The OpenMP standard (unfortunately) allows compilers to ignore
<tt><b>NOWAIT</tt></b>s, which means that in this case <font
color="#0000FF">OPARI</font> inserts an extra barrier and the
<font color="#0000FF">pomp</font> functions get invoked on this extra
(and not the real) barrier.
<p>
<li><font color="#0000FF">OPARI</font> cannot instrument the (required)
internal synchronization inside <tt><b>!$OMP</tt></b>
<tt><b>WORKSHARE</tt></b>.
<p>
<li>Mark Bull's <a
href="http://www.epcc.ed.ac.uk/research/openmpbench/">Microbenchmarks</a>
show that some compiler use different implementations (with different
characteristics) for implicit and explicit barriers. If <font
color="#0000FF">OPARI</font> changes implicit to explicit barriers, we
measure the wrong behavior on these compilers.
</ul>
Of course, an OpenMP compiler can insert the <font color="#0000FF">pomp</font>
performance interface calls directly around the implicit barriers, thereby
avoiding the described overheads and discrepancies.
<h2><hr>COMPILING OPARI<hr></h2>
All source files needed for the compilation of
<font color="#0000FF">OPARI</font> can be found in the subdirectory
<tt><b>tool</b></tt>. <font color="#0000FF">OPARI</font> is implemented
using ISO C++. To compile, the variables <tt><b>CXX</b></tt> and
<tt><b>CCFLAGS</b></tt> at the top of the <tt><b>Makefile</b></tt> have to be
set to the name of the C++ compiler and the necessary compilation flags,
respectively. <font color="#0000FF">OPARI</font> was tested with
<a href="http://www.kai.com/C_plus_plus/">KAI's KCC</a> version 3.4 and
4.0, <a href="http://gcc.gnu.org">GNU's g++</a> version 2.95.3 and later,
IBM xlC 5.x, SUN CC 6.1, and SGI 7.3.1.
<table cellspacing=10>
<tr>
<td valign="top"><tt><b>make</b></tt>
<td valign="top">compiles and generates the standalone
executable <tt><b>opari</b></tt>.
</tr>
<tr>
<td valign="top"><tt><b>make test</b></tt>
<td valign="top">runs a set of tests. The <tt><b>diff</b></tt> commands
should not produce any output.
</table>
The directory <tt><b>lib</b></tt> contains simple stubs for all functions
of the <font color="#0000FF">pomp</font> runtime measurement library as
well as test cases which can serve as a basis for an implementation of
the <font color="#0000FF">pomp</font> API.
</div>
</body>
</html>
|
