1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
|
<html>
<head>
<!-- This file has been generated by unroff 1.0, 01/16/02 10:26:11. -->
<!-- Do not edit! -->
<STYLE TYPE="text/css">
<!--
A:link{text-decoration:none}
A:visited{text-decoration:none}
A:active{text-decoration:none}
-->
</STYLE>
<title>ploticus: proc getdata</title>
<body bgcolor=D0D0EE vlink=0000FF>
<br>
<br>
<center>
<table cellpadding=2 bgcolor=FFFFFF width=550 ><tr>
<td>
<table cellpadding=2 width=550><tr>
<td><br><h2>proc getdata</h2></td>
<td align=right>
<small>
<a href="../doc/Welcome.html"><img src="../doc/ploticus.gif" border=0></a><br>
<a href="../doc/Welcome.html">Welcome</a>
<a href="../gallery/index.html">Gallery</a>
<a href="../doc/Contents.html">Handbook</a>
<td></tr></table>
</td></tr>
<td>
<br>
<br>
<title>Manual page for proc_getdata(PL)</title>
</head>
<body>
<p>
<b>proc getdata</b> accesses or defines data for plotting.<tt> </tt>
<b>proc getdata</b> must be executed before any
data-based plotting can be done.<tt> </tt>
<br><br><br>
<h2>Features</h2>
Data may be specified literally within the script file, or it may be
located in an external file or be the result of an externally executed command.
<p>
Conditional selection of data records (<tt>select</tt>).<tt> </tt>
<p>
Data filtering (<tt>filter</tt>) for purposes such as combining or rearranging fields,
performing calculations, truncating outliers, etc.<tt> </tt>
<p>
Field names may be assigned explicitly (<tt>fieldnames</tt> or <tt>fieldnamerows</tt>), or via a data file
header line (<tt>fieldnameheader</tt>). These names can then be used anywhere a
data field reference is expected.<tt> </tt>
<p>
Data sets with variable number of fields per row can be accomodated by
setting a maximum <tt>nfields</tt> value.
<p>
Data may be located at the end of the script file, to get it out of the way (<tt>#intrailer</tt>).<tt> </tt>
<br><br><br>
<h2>Examples</h2>
Virtually all of the Gallery examples use #proc getdata.<tt> </tt>
<br><br><br>
<h2>Acceptable data formats</h2>
See
<a href="dataformat.html">
dataformat
</a>
.<tt> </tt>
<br><br><br>
<h2>Prerequisites</h2>
None.<tt> </tt>
<br><br><br>
<h2>Troubleshooting</h2>
Set <tt>showresults: yes</tt> in order to see the data after it is read and parsed.<tt> </tt>
Especially useful when working with <tt>filter</tt>.<tt> </tt>
<br><br><br>
<h2>Variables that are set</h2>
The variable NRECORDS will be set to the number of records read,
and the variable NFIELDS will be set to the number of fields per record.<tt> </tt>
<br><br><br>
<h2>Handling error conditions gracefully</h2>
If no data records were read, NRECORDS will be set to zero.<tt> </tt>
This may be tested using script code such as this, which generates
an image containing the words "No data found" :
<dl>
<dt><dd><p>
<pre>
#proc getdata
...
#proc endproc
#if @NRECORDS = 0
#proc annotate
location: 3 3
text: No data found.
#exit
#endif
</pre>
</dl>
<h2>Setting variables from within a data file</h2>
Data files may contain embedded <b>#set</b> commands to set ploticus variables.<tt> </tt>
This may be a convenient way for title, labels, etc. to be driven by the data file.<tt> </tt>
The syntax is the same as in ploticus scripts:
<dl>
<dt><dd><p>
<tt>#set </tt><i>varname</i><tt> = </tt><i>value</i>
</dl>
<p>
<i>varname</i> will be set to <i>value</i>. <i>value</i> may
contain embedded spaces or any other character. <i>value</i> may not
be a $function(), nor may it contain any other @variables.<tt> </tt>
None of the other script operators, such as <b>#if</b>, are supported in this context.<tt> </tt>
<br><br><br>
<h2>Mandatory attributes</h2>
A data source (either <tt>file</tt>, <tt>pathname</tt>, <tt>command</tt>, or <tt>data</tt>) must be specified.<tt> </tt>
Default field delimitation method is <tt>spacequoted</tt>; <tt>delim</tt> must be set for other
types such as tab-delimited or comma-quote delimited.<tt> </tt>
<br><br><br>
<h2>Attributes</h2>
<p>
<b>data</b>
<a href="attributetypes.html#text">
multiline-text
</a>
<dl>
<dt><dd><p>
Literal specification of plotting data. Terminates at
first blank (empty) line. Example:
<pre>
data: "Case 1" 0 4 4.1
"Case 2" 1 5 4.4
"Case 3" 2 2 4.0
"Case 4" 3 9 4.8
</pre>
<tt></tt></dl>
<p>
<b>file</b> <i>filename</i>
<dl>
<dt><dd><p>
Shell-expandable name of a file containing plotting data.<tt> </tt>
This name will be used along with cat(1) in a shell command, thus
exported shell variables and metacharacters may be part of the name.<tt> </tt>
A dash (-) may be used if data is to be read from the standard input,
(or the <tt>standardinput</tt> attribute may be used).<tt> </tt>
Example: <tt>filename: myfile.dat</tt>
</dl>
<p>
<b>pathname</b> <i>filename</i>
<dl>
<dt><dd><p>
Name of a file containing plotting data.<tt> </tt>
The file will be opened directly.<tt> </tt>
Shell variables and metacharacters may not be used.<tt> </tt>
</dl>
<p>
<b>command</b> <i>shell command line</i>
<dl>
<dt><dd><p>
An external shell command that will produce plot data on its standard output.<tt> </tt>
<br>
Example: <tt>command: cat mydat | uniq -c</tt>
</dl>
<p>
<b>delim</b> <tt>spacequoted</tt> | <tt>whitespace</tt> | <tt>comma</tt> | <tt>tab</tt>
<dl>
<dt><dd><p>
The type of delimiting method to be used when
parsing the data. <tt>spacequoted</tt> is the default (<tt>space</tt> is
equivalent to <tt>spacequoted</tt>). See
<a href="dataformat.html">
dataformat
</a>
for details.<tt> </tt>
<br>
Example: <tt>delim: comma</tt>
</dl>
<p>
<b>fieldnames</b> <i>namelist</i>
<dl>
<dt><dd><p>
If specified, the names given in <i>namelist</i> may be
used in any plotting proc to identify data fields. <i>namelist</i> is a
list delimited by spaces and/or commas.<tt> </tt>
Names may include any alphanumeric characters with a maximum length of 38, and are
case-insensitive.<tt> </tt>
Field names may not contain embedded spaces or commas.<tt> </tt>
<br>
Example: <tt>fieldnames: date group n</tt>
</dl>
<p>
<b>fieldnamerows</b>
<a href="attributetypes.html#text">
multiline text
</a>
<dl>
<dt><dd><p>
Same as <tt>fieldnames</tt> (see above), except that field names are given one per line.<tt> </tt>
Must be terminated by a blank line.<tt> </tt>
Example:
<br>
<pre>
fieldnamerows:
id
type
age
sex
</pre>
</dl>
<p>
<b>fieldnameheader</b> <tt>yes</tt> | <tt>no</tt>
<dl>
<dt><dd><p>
Allows field names to be embedded in the input data.<tt> </tt>
If <tt>yes</tt>, the first non-comment line in the data is expected to
hold a list of field names. See the <tt>fieldnames</tt> attribute above for more information
about field names.<tt> </tt>
If the <tt>data</tt> attribute is used, use of <tt>fieldnames</tt>
is more efficent than <tt>fieldnameheader</tt>.<tt> </tt>
</dl>
<p>
<b>pf_fieldnames</b> <i>namelist</i>
<dl>
<dt><dd><p>
Post-filter fieldnames, for use with <tt>filter</tt> when field names are being used,
and the filter result has a different logical record format than the input.<tt> </tt>
<br>
Example: <tt>pf_fieldnames: date z sum1 sum2</tt>
</dl>
<p>
<b>nfields</b> <i>n</i>
<dl>
<dt><dd><p>
If specified, this sets the expected number of fields per record.<tt> </tt>
If a data row has more than the expected number of fields, extra fields are silently ignored.
If a data row has less than the expected number of fields, blank fields are silently added.<tt> </tt>
This is applied after any <tt>filter</tt> processing.<tt> </tt>
</dl>
<p>
<b>standardinput</b> <tt>yes</tt> | <tt>no</tt>
<dl>
<dt><dd><p>
If <tt>yes</tt>, data is read from the standard input.<tt> </tt>
</dl>
<p>
<b>#intrailer</b>
<dl>
<dt><dd><p>
Indicates that a <tt>data</tt> attribute
will be given in a #proc trailer, at the end of the script file.<tt> </tt>
See EXAMPLES, below.<tt> </tt>
</dl>
<p>
<b>commentchar</b>
<a href="attributetypes.html#string">
string
</a>
<dl>
<dt><dd><p>
A character or group of characters that is used to signify a comment in the data file.<tt> </tt>
Commented lines will be skipped. Default is <tt>//</tt>.<tt> </tt>
Example: <tt>commentchar: #</tt>
</dl>
<p>
<b>showresults</b> <tt>yes</tt> | <tt>no</tt>
<dl>
<dt><dd><p>
If <tt>yes</tt>, the results, after any selecting and/or filtering, are
written to the diagnostic file,
which may be useful in debugging, etc.<tt> </tt>
</dl>
<p>
<b>rotate</b> <tt>yes</tt> | <tt>no</tt>
<dl>
<dt><dd><p>
Allows data to be given all in one row, even when plotting proc
expects one record per instance (which most do).<tt> </tt>
Only applicable if your data set has one row.<tt> </tt>
(There still must be a blank line following the data attribute.)
To rotate more than one row, use <b>proc processdata</b>.<tt> </tt>
<br>
Example: see
<a href="../gallery/bars1.htm">
bars1
</a>
</dl>
<p>
<b>select</b>
<a href="condex.html">
conditional expression
</a>
<dl>
<dt><dd><p>
This allows data records to be selected for inclusion based
upon a selection expression.
Incoming data fields are referenced by number,
using a double at-sign (<tt>@@</tt>) prefix.
<tt>select</tt> cannot be used along with the <tt>data</tt> attribute.<tt> </tt>
Hint: use the <tt>showresults</tt> attribute when debugging.<tt> </tt>
<br>
Example: <tt>select: @@3 = g</tt>
<br>
This would select all data records having 3rd field equal to <tt>g</tt>.<tt> </tt>
</dl>
<p>
<b>filter</b>
<a href="attributetypes.html#text">
multiline text
</a>
<dl>
<dt><dd><p>
An embedded script which allows flexible processing to be applied to
incoming data records one at a time.<tt> </tt>
Typical uses are for
concatenating or splitting fields, doing on-the-fly date conversions,
or generating derived fields such as the sum of several fields or the difference
between two fields.<tt> </tt>
<br>
<br>
<dt><dd><p>
The embedded script will be applied once to every incoming data record.<tt> </tt>
The script should produce some "output"; generally the last statement is a
<b>##print</b>.
The output must use the same delimitation method as the input.<tt> </tt>
The output may have a different logical record format than the input.<tt> </tt>
If you are using field names,
the <tt>pf_fieldnames</tt> attribute (see above) may be used to name the
filter result fields when result record format differs from that of the input.<tt> </tt>
<dt><dd><p>
The script uses the same syntax as the greater ploticus script, except that:
</dl>
<dl><dt><dd>
<ul>
<li>
directives must begin with two pound signs (<tt>##</tt>) instead of one
<li>
local variables begin with two at signs (<tt>@@</tt>) instead of one
<li>
fields on the incoming data record are accessed like this: <tt>@@1</tt> for
the first field, <tt>@@2</tt> for the second, etc. If you are using field
names, these may be used as well, eg: <tt>@@score</tt>.<tt> </tt>
<li>
the only directives that may be used are
<b>##set</b>, <b>##if</b>, <b>##elseif</b>, <b>##else</b>, <b>##print</b>, <b>##call</b>,
and <b>##exit</b>
</ul>
</dl>
<dl>
<dt><dd><p>
Other things worth noting:
</dl>
<dl><dt><dd>
<ul>
<li>
the filter script is terminated using a blank line.<tt> </tt>
<li>
use the <tt>showresults</tt> attribute when debugging.<tt> </tt>
<li>
if <tt>filter</tt> is used along with <b>select</b>, the <tt>select</tt> is applied first.
<li>
<tt>filter</tt> cannot be used along with the <tt>data</tt> attribute.<tt> </tt>
<li>
<tt>filter</tt> can only access fields from one data record at a time.<tt> </tt>
<li>
variables from the ploticus script may be referenced within
the <tt>filter</tt> script (use one at-sign <tt>@</tt>).<tt> </tt>
Evaluation occurs before the filter script executes.<tt> </tt>
<li>
ploticus script #if/#else statements (single pound sign)
may be used to selectively execute portions of the filter script.<tt> </tt>
Interpretation occurs before the filter script executes.<tt> </tt>
<li>
Example: This filters out data records
having field 2 or field 3 equal to M. It then
calulates the difference in days between two dates
and puts this difference in the variable DIFF.<tt> </tt>
Finally it "prints" incoming field 1 along with DIFF.<tt> </tt>
Thus the result of this #proc getdata
will have be data records having two fields.<tt> </tt>
<pre>
filter:
##if @@2 = M || @@3 = M
##exit
##endif
##set DIFF = $daysdiff(@@3,@@2)
##print @@1 @@DIFF
</pre>
<tt></tt><li>
There are several more <tt>filter</tt> examples in the
<a href="Faq.html#filter">
FAQ
</a>
<br><br><br>
</ul>
</dl>
<h2>More examples</h2>
<p>
Data specification may be located at the end of the script file
by using <b>#intrailer</b> and <b>#proc trailer</b>. This may be
useful in "getting the data out of the way", or with automated building
of script files.<tt> </tt>
Here is how this is done:
<pre>
#proc getdata
#intrailer
<i>other #procs, etc.</i>
#proc trailer
Data: 0.3 0.5 2.3
3.5 9.4 1.4
<i>..etc..</i>
<i>end of file</i>
<br>
<br>
</td></tr>
<td align=right>
<a href="Welcome.html">
<img src="../doc/ploticus.gif" border=0></a><br><small>data display engine <br>
<a href="../doc/Copyright.html">Copyright Steve Grubb</a>
<br>
<br>
<center>
<img src="../gallery/all.gif">
</center>
</td></tr>
</table>
</pre>
<p><hr>
Markup created by <em>unroff</em> 1.0, <tt> </tt> <tt> </tt>January 16, 2002.
</body>
</html>
|
