1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
|
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>Reading data — Veusz 3.6.2 documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Veusz command line and embedding interface (API)" href="api.html" />
<link rel="prev" title="Introduction" href="introduction.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="reading-data">
<h1>Reading data<a class="headerlink" href="#reading-data" title="Permalink to this headline">¶</a></h1>
<p>Currently Veusz supports reading data from files with text, CSV, HDF5,
FITS, 2D text or CSV, QDP, binary and NPY/NPZ formats. Use the
<span class="menuselection">Data ‣ Import</span> dialog to read data, or the importing
commands in the API can be used. In addition, the user can load or
write import plugins in Python which load data into Veusz in an
arbitrary format. At the moment QDP, binary and NPY/NPZ files are
supported with this method. The HDF5 file format is the most
sophisticated, and is recommended for complex datasets.</p>
<p>By default, data are “linked” to the file imported from. This means
that the data are not stored in the Veusz saved file and are reloaded
from the original data file when opening. In addition, the user can
use the <span class="menuselection">Data ‣ Reload</span> menu option to reload data
from linked files. Unselect the linked option when importing to remove
the association with the data file and to store the data in the Veusz
saved document.</p>
<p>Note that a prefix and suffix can be given when importing. These are
added to the front or back of each dataset name imported. They are
convenient for grouping data together.</p>
<img alt="_images/importdialog.png" src="_images/importdialog.png" />
<p>We list the various types of import below.</p>
<section id="standard-text-import">
<h2>Standard text import<a class="headerlink" href="#standard-text-import" title="Permalink to this headline">¶</a></h2>
<p>The default text import operates on simple text files. The data are
assumed to be in columns separated by whitespace. Each column
corresponds to dataset (or its error bars). Each row is an entry in
the dataset.</p>
<p>The way the data are read is goverened by a simple “descriptor”. This
can simply be a list of dataset names separated by spaces. If no
descriptor is given, the columns are treated as separate datasets and
are given names <cite>col1</cite>, <cite>col2</cite>, etc. Veusz attempts to automatically
determine the type of the data.</p>
<p>When reading in data, Veusz treats any whitespace as separating
columns. The columns do not actually need to be aligned. Furthermore a
<cite>\</cite> symbol can be placed at the end of a line to mark a
continuation. Veusz will read the next line as if it were placed at
the end of the current line. In addition comments and blank lines are
ignored (unless in block mode). Comments start with a <cite>#</cite>, <cite>;</cite>, <cite>!</cite> or
<cite>%</cite>, and continue until the end of the line. The special value <cite>nan</cite>
can be used to specify a break in a dataset.</p>
<p>If the option to read data in blocks is enabled, Veusz treats blank
lines (or lines starting with the word <cite>no</cite>) as block separators. For
each dataset in the descriptor, separate datasets are created for each
block, using a numeric suffix giving the block number, e.g. <cite>_1</cite>, <cite>_2</cite>.</p>
<section id="data-types-in-text-import">
<h3>Data types in text import<a class="headerlink" href="#data-types-in-text-import" title="Permalink to this headline">¶</a></h3>
<p>Veusz supports reading in several types of data. The type of data can
be added in round brackets after the name in the descriptor. Veusz
will try to guess the type of data based on the first value, so you
should specify it if there is any form of ambiguity (e.g. is 3 text or
a number). Supported types are numbers (use numeric in brackets) and
text (use text in brackets). An example descriptor would be
<cite>x(numeric) +- y(numeric) + - label(text)</cite> for an x dataset followed
by its symmetric errors, a y dataset followed by two columns of
asymmetric errors, and a final column of text for the label dataset.</p>
<p>A text column does not need quotation unless it contains space
characters or escape characters. However make sure you deselect the
“ignore text” option in the import dialog. This ignores lines of text
to ease the import of data from other applications. Quotation marks
are recommended around text if you wish to avoid ambiguity. Text is
quoted according to the Python rules for text. Double or single
quotation marks can be used, e.g. <cite>“A ‘test’”</cite>, <cite>‘A second
“test”’</cite>. Quotes can be escaped by prefixing them with a backslash,
e.g. <cite>“A new \”test\””</cite>. If the data are generated from a Python
script, the repr function provides the text in a suitable form.</p>
<p>Dates and times are also supported with the syntax
<cite>dataset(date)</cite>. Dates must be in ISO format <cite>YYYY-MM-DD</cite>. Times are
in 24 hour format hh:mm:ss.ss. Dates with times are written
YYYY-MM-DDThh:mm:ss.ss (this is a standard ISO format, see
<a class="reference external" href="http://www.w3.org/TR/NOTE-datetime">http://www.w3.org/TR/NOTE-datetime</a>). Dates are stored within
Veusz as a number which is the number of seconds since the start of
January 1st 2009. Veusz also supports dates and times in the local
format, though take note that the same file and data may not work on a
system in a different location.</p>
</section>
<section id="descriptors">
<h3>Descriptors<a class="headerlink" href="#descriptors" title="Permalink to this headline">¶</a></h3>
<p id="id1">A list of datasets, or a “Descriptor”, is given in the
Import dialog to describe how the data are formatted in the
import file. The descriptor at its simplest is a space or
comma-separated list of the names of the datasets to import.
These are columns in the file.</p>
<p>Following a dataset name the text <cite>+</cite>, <cite>-</cite>, or <cite>+-</cite> can be
given to say that the following column is a positive error
bar, negative error bar or symmetric error bar for the
previous (non error bar) dataset. These symbols should be
separated from the dataset name or previous symbol with a
space or a comma symbol.</p>
<p>In addition, if multiple numbered columns should be
imported, the dataset name can be followed by square
brackets containing a range in the form <cite>[a:b]</cite> to number
columns a to b, or <cite>[:]</cite> to number remaining columns. See
below for examples of this use.</p>
<p>Dataset names can contain virtually any character, even
unicode characters. If the name contains non alpha-numeric
characters (characters outside of A-Z, a-z and 0-9), then
the dataset name should be contained within back-tick
characters. An example descriptor is <strong class="command">`length data
(m)`,+- `speed (mps)`,+,-</strong>, for two datasets with
spaces and brackets in their names.</p>
<p>Instead of specifying the descriptor in the Import dialog,
the descriptor can be placed in the data file using a
descriptor statement on a separate line, consisting of
“descriptor” followed by the descriptor. Multiple
descriptors can be placed in a single file, for example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># here is one section</span>
<span class="n">descriptor</span> <span class="n">x</span><span class="p">,</span><span class="o">+-</span> <span class="n">y</span><span class="p">,</span><span class="o">+</span><span class="p">,</span><span class="o">-</span>
<span class="mi">1</span> <span class="mf">0.5</span> <span class="mi">2</span> <span class="mf">0.1</span> <span class="o">-</span><span class="mf">0.1</span>
<span class="mi">2</span> <span class="mf">0.3</span> <span class="mi">4</span> <span class="mf">0.2</span> <span class="o">-</span><span class="mf">0.1</span>
<span class="c1"># my next block</span>
<span class="n">descriptor</span> <span class="n">alpha</span> <span class="n">beta</span> <span class="n">gamma</span>
<span class="mi">1</span> <span class="mi">2</span> <span class="mi">3</span>
<span class="mi">4</span> <span class="mi">5</span> <span class="mi">6</span>
<span class="mi">7</span> <span class="mi">8</span> <span class="mi">9</span>
<span class="c1"># etc...</span>
</pre></div>
</div>
</section>
<section id="descriptor-examples">
<h3>Descriptor examples<a class="headerlink" href="#descriptor-examples" title="Permalink to this headline">¶</a></h3>
<ol class="arabic">
<li><p><strong class="command">x y</strong> two columns are present in the file, they will be
read in as datasets <cite>x</cite> and <cite>y</cite>.</p></li>
<li><p><strong class="command">x,+- y,+,-</strong> or <strong class="command">x +- y + -</strong> two datasets are in
the file. Dataset “x” consists of the first two columns. The first
column are the values and the second are the symmetric errors. “y”
consists of three columns (note the comma between + and -). The
first column are the values, the second positive asymmetric errors,
and the third negative asymmetric errors. Suppose the input file
contains:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="mf">1.0</span> <span class="mf">0.3</span> <span class="mi">2</span> <span class="mf">0.1</span> <span class="o">-</span><span class="mf">0.2</span>
<span class="mf">1.5</span> <span class="mf">0.2</span> <span class="mf">2.3</span> <span class="mf">2e-2</span> <span class="o">-</span><span class="mf">0.3E0</span>
<span class="mf">2.19</span> <span class="mf">0.02</span> <span class="mi">5</span> <span class="mf">0.1</span> <span class="o">-</span><span class="mf">0.1</span>
</pre></div>
</div>
<p>Then x will contain <cite>1+-0.3</cite>, <cite>1.5+-0.2</cite>, <cite>2.19+-0.02</cite>. y will
contain <cite>2 +0.1 -0.2</cite>, <cite>2.3 +0.02 -0.3</cite>, <cite>5 +0.1 -0.1</cite>.</p>
</li>
<li><p><strong class="command">x[1:2] y[:]</strong> the first column is the data <cite>x_1</cite>, the
second <cite>x_2</cite>. Subsequent columns are read as <cite>y[1]</cite> to <cite>y[n]</cite>.</p></li>
<li><p><strong class="command">y[:]+-</strong> read each pair of columns as a dataset and its
symmetric error, calling them <cite>y[1]</cite> to <cite>y[n]</cite>.</p></li>
<li><p><strong class="command">foo,,+-</strong> read the first column as the foo dataset, skip a
column, and read the third column as its symmetric error.</p></li>
</ol>
</section>
</section>
<section id="csv-files">
<h2>CSV files<a class="headerlink" href="#csv-files" title="Permalink to this headline">¶</a></h2>
<p>CVS (comma separated variable) files are often written from other
programs, such as spreadsheets, including Excel and Gnumeric. Veusz
supports reading from these files.</p>
<p>In the import dialog choose “CSV”, then choose a filename to import
from. In the CSV file the user should place the data in either rows or
columns. Veusz will use a name above a column or to the left of a row
to specify what the dataset name should be. The user can use new names
further down in columns or right in rows to specify a different
dataset name. Names do not have to be used, and Veusz will assign
default <cite>col</cite> and <cite>row</cite> names if not given. You can also specify a
prefix which is prepended to each dataset name read from the file.</p>
<p>To specify symmetric errors for a column, put <cite>+-</cite> as the dataset name
in the next column or row. Asymmetric errors can be stated with <cite>+</cite>
and <cite>-</cite> in the columns.</p>
<p>The data type in CSV files are automatically detected unless
specified. The data type can be given in brackets after the column
name, e.g. <cite>name (text)</cite>, where the data type is <cite>date</cite>, <cite>numeric</cite> or
<cite>text</cite>. Explicit data types are needed if the data look like a
different data type (e.g. a text item of <cite>1.23</cite>). The date format in
CSV files can be specified in the import dialog box - see the examples
given. In addition CSV files support numbers in European format
(e.g. 2,34 rather than 2.34), depending on the setting in the dialog
box.</p>
</section>
<section id="hdf5-files">
<h2>HDF5 files<a class="headerlink" href="#hdf5-files" title="Permalink to this headline">¶</a></h2>
<p>HDF5 is a flexible data format. Datasets and tables can be stored in a
hierarchical arrangements of groups within a file. Veusz supports
reading 1D numeric, text, date-time, 2D numeric or n-dimensional
numeric data from HDF files. The <strong class="command">h5py</strong> Python module must be
installed to use HDF5 files (included in binary releases).</p>
<p>In the import dialog box, choose which individual datasets to import,
or selecting a group to import all the datasets within the group. If
selecting a group, datasets in the group incompatible with Veusz are
ignored.</p>
<p>A name can be provided for each dataset imported by entering one under
“Import as”. If one is not given, the dataset or column name is
used. The name can also be specified by setting the HDF5 dataset
attribute <code class="docutils literal notranslate"><span class="pre">vsz_name</span></code> to the name. Note that for compound datasets
(tables), <code class="docutils literal notranslate"><span class="pre">vsz_</span></code> attributes for columns are given by appending the
suffix <code class="docutils literal notranslate"><span class="pre">_columnname</span></code> to the attribute.</p>
<section id="error-bars">
<h3>Error bars<a class="headerlink" href="#error-bars" title="Permalink to this headline">¶</a></h3>
<p>Error bars are supported in two ways. The first way is to combine 1D
datasets. For the datasets which are error bars, use a name which is
the same as the main dataset but with the suffix <cite>(+-)</cite>, <cite>(+)</cite> or
<cite>(-)</cite>, for symmetric, postive or negative error bars,
respectively. The second method is to use a 2D dataset with two or
three columns, for symmetric or asymmetric error bars,
respectively. Click on the dataset in the dialog and choose the option
to import as a 1D dataset. This second method can also be enabled by
adding an HDF5 attribute <code class="docutils literal notranslate"><span class="pre">vsz_twod_as_oned</span></code> set to a non-zero value
for the dataset.</p>
</section>
<section id="slices">
<h3>Slices<a class="headerlink" href="#slices" title="Permalink to this headline">¶</a></h3>
<p>You may wish to reduce the dimensions of a dataset before importing by
slicing. You can also give a slice to import a subset of a
dataset. When importing, in the slice column you can give a slice
expression. This should have the same number of entries as the dataset
has dimensions, separated by commas. An entry can be a single number,
to select a particular row or column. Alternatively it could be an
expression like <code class="docutils literal notranslate"><span class="pre">a:b:c</span></code> or <code class="docutils literal notranslate"><span class="pre">a:b</span></code>, where <code class="docutils literal notranslate"><span class="pre">a</span></code> is the starting
index, <code class="docutils literal notranslate"><span class="pre">b</span></code> is one beyond the stopping index and optionally <code class="docutils literal notranslate"><span class="pre">c</span></code> is
the step size. A slice can also be specified by providing an HDF5
attribute <code class="docutils literal notranslate"><span class="pre">vsz_slice</span></code> for the dataset.</p>
</section>
<section id="d-data-ranges">
<h3>2D data ranges<a class="headerlink" href="#d-data-ranges" title="Permalink to this headline">¶</a></h3>
<p>2D data have an associated X and Y range. By default the number of
pixels of the image are used to give this range. A range can be
specified by clicking on the dataset and entering a minimum and
maximum X and Y coordinates. Alternatively, provide the HDF5 attribute
for the dataset <code class="docutils literal notranslate"><span class="pre">vsz_range</span></code>, which should be set to an array of four
values (minimum x, minimum y, maximum x, maximum y).</p>
</section>
<section id="dates">
<h3>Dates<a class="headerlink" href="#dates" title="Permalink to this headline">¶</a></h3>
<p>Date/time datasets can be made from a 1D numeric dataset or from a
text dataset. For the 1D dataset, use the number of seconds relative
to the start of the year 2009 (this is Veusz format) or the year 1970
(this is Unix format). In the import dialog, click on the name of the
dataset and choose the date option. To specify a date format in the
HDF5 file, set the attribute <code class="docutils literal notranslate"><span class="pre">vsz_convert_datetime</span></code> to either
<code class="docutils literal notranslate"><span class="pre">veusz</span></code> or <code class="docutils literal notranslate"><span class="pre">unix</span></code>.</p>
<p>For text datasets, dates must be given in the right format, selected
in the import dialog after clicking on the dataset name. As in other
file formats, by default Veusz uses ISO 8601 format, which looks like
<cite>2013-12-22T21:08:07</cite>, where the date and time parts are optional. The
T is also optional. You can also provide your own format when
importing by giving a date expression using YYYY, MM, DD, hh, mm and
ss (e.g. <cite>YYYY-MM-DD|T|hh:mm:ss</cite>), where vertical bars mark optional
parts of the expression. To automate this, set the attribute
<code class="docutils literal notranslate"><span class="pre">vsz_convert_datetime</span></code> to the format expression or <code class="docutils literal notranslate"><span class="pre">iso</span></code> to
specify ISO format.</p>
</section>
</section>
<section id="d-text-or-csv-format">
<h2>2D text or CSV format<a class="headerlink" href="#d-text-or-csv-format" title="Permalink to this headline">¶</a></h2>
<p>Veusz can import 2D data from standard text or CSV files. In this case
the data should consist of a matrix of data values, with the columns
separated by one or more spaces or tabs and the rows on different
lines.</p>
<p>In addition to the data the file can contain lines at the top which
affect the import. Such specifiers are used, for example, to change
the coordinates of the pixels in the file. By default the first pixels
coordinates is between 0 and 1, with the centre at 0.5. Subsequent
pixels are 1 greater. Note that the lowest coordinate pixel is the
bottom-left value in the table of imported values. When using
specifiers in CSV files, put the different parts (separated by spaces)
in separate columns. Below are listed the specifiers:</p>
<ol class="arabic simple">
<li><p><strong class="command">xrange A B</strong> - make the 2D dataset span the coordinate
range A to B in the x-axis (where A and B are numbers). Note that
the range is inclusive, so a 1 pixel wide image with A=0 and B=1
would have the pixel centre at 0.5. The pixels are assumed to have
the same spacing. Do not use this as the same time as the
<strong class="command">xedge</strong> or <strong class="command">xcent</strong> options.</p></li>
<li><p><strong class="command">yrange A B</strong> - make the 2D dataset span the coordinate
range A to B in the y-axis (where A and B are numbers).</p></li>
<li><p><strong class="command">xedge A B C...</strong> - rather than assume the pixels have the
same spacing, give the coordinates of the edges of the pixels in
the x-axis. The numbers should be space-separated and there should
be one more number than pixels. Do not give <strong class="command">xrange</strong> or
<strong class="command">xcent</strong> if this is given. If the values are increasing,
the lowest coordinate value is at the left of the dataset,
otherwise if they are decreasing, it is on the right (unless the
rows/columns are inverted or transposed).</p></li>
<li><p><strong class="command">yedge A B C...</strong> - rather than assume the pixels have the
same spacing, give the coordinates of the edges of the pixels in
the y-axis. If the values are increasing, the lowest coordinate
value is at the bottom row. If they instead decrease, it is at the
top.</p></li>
<li><p><strong class="command">xcent A B C...</strong> - rather than give a total range or pixel
edges, give the centres of the pixels. There should be the same
number of values as pixels in the image. Do not give
<strong class="command">xrange</strong> or <strong class="command">xedge</strong> if this is given. The order
of the values specify whether the pixels are left to right or right
to left.</p></li>
<li><p><strong class="command">ycent A B C...</strong> - rather than give a total range or pixel
edges, give the centres of the pixels. The value order specifies
whether the pixels are bottom to top, or top to bottom.</p></li>
<li><p><strong class="command">invertrows</strong> - invert the rows after reading the data.</p></li>
<li><p><strong class="command">invertcols</strong> - invert the columns after reading the data.</p></li>
<li><p><strong class="command">transpose</strong> - swap rows and columns after importing data.</p></li>
<li><p><strong class="command">gridatedge</strong> - the first row and leftmost column give the
positions of the centres of the pixels. This is also an option in
the import dialog. The values should be increasing or decreasing.</p></li>
</ol>
</section>
<section id="fits-files">
<h2>FITS files<a class="headerlink" href="#fits-files" title="Permalink to this headline">¶</a></h2>
<p>1D, 2D or n-dimensional data can be read from FITS files. 1D or 2D
data can be read from image, primary or table HDUs. nD data can be
read from from image or primary extensions. Note that pyfits or
astropy must be installed to get FITS support.</p>
<p>The import dialog box uses a tree to show the structure of the FITS
file. The user can choose to import the whole file, by clicking the
check box at the top. They can import data from a particular HDU by
selecting that, or individual table columns can be selected.</p>
<p>In the dialog box, a dataset can be given a name for the
dataset. Otherwise the HDU or table column name is used. Note that a
prefix and/or suffix can be specified to be added to all dataset
names.</p>
<p>If dataset y should have an error bar specified by column yerr, then
in the name for yerr, enter ‘y (+-)’. Asymmetric error bars can be
specified using (+) and (-) on inidividual columns.</p>
<p>The slice column can be used to only import a subset of the dataset
imported. This uses Python slicing syntax, which is comma-separated
list of ranges and steps. A range is specified like 10:20, which
selects the 11th to 20th items (the indices are numbered from 0, and
the final index is one past the index you actually want). A stepped
range can look like 10:20:2, which selects every other item in that
range. Each of these numbers are optional, so : selects all items on
that dimension. For example the slice :,10:14:2 selects all values on
the first dimension, but only the 11th and 13th items on the next
axis.</p>
<p>When importing 2D data the user can specify whether to treat this as
1D plus error bars (dimensions should have 2 or 3 columns), or specify
a range in 2D space the data covers. Veusz will also attempt to use
WCS information in the file for the 2D range if not specified. The
standard mode is to use the CDELT, CRVAL and CRPIX keywords to specify
a linear range for the data. Alternatively the user can specify pixel
numbering (numbering from 0 to N-1). There is a fraction option for
using a range of 0 to 1. Finally there is a pixel numbering scheme
which numbers in pixels from the CRPIX keyword items.</p>
<p>Some of these options can be specified in the FITS file using the
‘VEUSZ’ header keyword. This header keyword can be added with the
value ‘KEY=VALUE’ (applying to the whole HDU) or ‘COLUMN: KEY=VALUE’
(applying to a particular column in a table). Supported options for
KEY are:</p>
<dl class="simple">
<dt>name</dt><dd><p>provide name for dataset in VALUE</p>
</dd>
<dt>slice</dt><dd><p>VALUE is slice to apply when importing dataset</p>
</dd>
<dt>range</dt><dd><p>range of data for 2D dataset in form <cite>[minx, miny, maxx, maxy]</cite></p>
</dd>
<dt>xrange/yrange</dt><dd><p>range of dataset individually in x or y</p>
</dd>
<dt>xcent/ycent</dt><dd><p>set to list of values giving centers of pixels</p>
</dd>
<dt>xedge/yedge</dt><dd><p>set to list of values giving edges of pixels</p>
</dd>
<dt>twod_as_oned</dt><dd><p>treat as 1D data with error bars if VALUE=1</p>
</dd>
<dt>wcsmode</dt><dd><p>use specific WCS mode for 2D dataset (should be
pixel/pixel_wcs/linear_wcs/fraction)</p>
</dd>
</dl>
</section>
<section id="reading-other-data-formats">
<h2>Reading other data formats<a class="headerlink" href="#reading-other-data-formats" title="Permalink to this headline">¶</a></h2>
<p>As mentioned above, a user may write some Python code to read a data
file or set of data files. To write a plugin which is incorportated
into Veusz, see <a class="reference external" href="https://github.com/veusz/veusz/wiki/ImportPlugins">https://github.com/veusz/veusz/wiki/ImportPlugins</a></p>
<p>You can also include Python code in an input file to read data, which
we describe here. Suppose an input file “in.dat” contains the
following data:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="mi">1</span> <span class="mi">2</span>
<span class="mi">2</span> <span class="mi">4</span>
<span class="mi">3</span> <span class="mi">9</span>
<span class="mi">4</span> <span class="mi">16</span>
</pre></div>
</div>
<p>Of course this data could be read using the <a class="reference internal" href="api.html#command-importfile"><span class="std std-ref">ImportFile</span></a> command. However, you could also read it with
the following Veusz script (which could be saved to a file and loaded
with <strong class="command">execfile</strong> or <a class="reference internal" href="api.html#command-load"><span class="std std-ref">Load</span></a>. The script
also places symmetric errors of 0.1 on the x dataset.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">y</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"in.dat"</span><span class="p">):</span>
<span class="n">parts</span> <span class="o">=</span> <span class="p">[</span><span class="nb">float</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">()]</span>
<span class="n">x</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="n">y</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">parts</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="n">SetData</span><span class="p">(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">symerr</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
<span class="n">SetData</span><span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
</pre></div>
</div>
</section>
</section>
<section id="manipulating-datasets">
<h1>Manipulating datasets<a class="headerlink" href="#manipulating-datasets" title="Permalink to this headline">¶</a></h1>
<p>Imported datasets can easily be modified in the Data Editor dialog
box. This dialog box can also be used to create new datasets from
scratch by typing them in. The Data Create dialog box is used to new
datasets as a numerical sequence, parametrically or based on other
datasets given expressions. If you want to plot a function of a
dataset, you often do not have to create a new dataset. Veusz allows
to enter expressions directly in many places.</p>
<section id="using-dataset-plugins">
<h2>Using dataset plugins<a class="headerlink" href="#using-dataset-plugins" title="Permalink to this headline">¶</a></h2>
<p>Dataset plugins can be used to perform arbitrary manipulation of
datasets. Veusz includes several plugins for mathematical operation of
data and other dataset manipulations, such as concatenation or
splitting. If you wish to write your own plugins look at
<a class="reference external" href="https://github.com/veusz/veusz/wiki/DatasetPlugins">https://github.com/veusz/veusz/wiki/DatasetPlugins</a>.</p>
</section>
<section id="using-expressions-to-create-new-datasets">
<h2>Using expressions to create new datasets<a class="headerlink" href="#using-expressions-to-create-new-datasets" title="Permalink to this headline">¶</a></h2>
<p>For instance, if the user has already imported dataset d, then they
can create d2 which consists of d**2. Expressions are in Python numpy
syntax and can include the usual mathematical functions.</p>
<img alt="_images/createdataset.png" src="_images/createdataset.png" />
<p>Expressions for error bars can also be given. By appending
<strong class="command">_data</strong>, <strong class="command">_serr</strong>, <strong class="command">_perr</strong> or
<strong class="command">_nerr</strong> to the name of the dataset in the expression, the
user can base their expression on particular parts of the given
dataset (the main data, symmetric errors, positive errors or negative
errors). Otherwise the program uses the same parts as is currently
being specified.</p>
<p>If a dataset name contains non alphanumeric characters, its name
should be quoted in the expression in back-tick characters,
e.g. <strong class="command">`length (cm)`*2</strong>.</p>
<p>The numpy functionality is particularly useful for doing more
complicated expressions. For instance, a conditional expression can be
written as <strong class="command">where(x<y,x,y)</strong> or
<strong class="command">where(isfinite(x),a,b))</strong>.</p>
<p>You often do not need to create a new dataset when. For example, with
the xy point plotter widget, you can directly enter an expression as
the X and Y dataset settings. When you give a direct dataset
expression, you can define error bar expressions by separating them by
commas, and optionally surrounding them by brackets. For example
<strong class="command">(a,0.1)</strong> plots dataset a as the data, with symmetric errors
bars of 0.1. Asymmetric bars are given as <strong class="command">(a,a*0.1,-a*0.1)</strong>.</p>
<p>Other useful functions in evaluation include those already mentioned
in the LaTeX expansion description. <strong class="command">DATA(name, [part])</strong>
returns the dataset with name given. The optional part, which can be
‘data’, ‘serr’, ‘perr’ or ‘nerr’, allows error bars to be returned for
numerical data. <strong class="command">SETTING(path)</strong> returns the value of the
Veusz setting, which can include, for example, the best fitting
parameters of a fit. <strong class="command">ENVIRON</strong> is the Python environment
variable dictionary, allowing values to be passed from the
environment, e.g. <strong class="command">float(ENVIRON['myvar'])</strong>.</p>
</section>
<section id="linking-datasets-to-expressions">
<h2>Linking datasets to expressions<a class="headerlink" href="#linking-datasets-to-expressions" title="Permalink to this headline">¶</a></h2>
<p>A particularly useful feature is to be able to link a dataset to an
expression, so if the expression changes the dataset changes with it,
like in a spreadsheet.</p>
</section>
<section id="splitting-data">
<h2>Splitting data<a class="headerlink" href="#splitting-data" title="Permalink to this headline">¶</a></h2>
<p>Data can also be chopped in this method, for example using the
expression <strong class="command">x[10:20]</strong>, which makes a dataset based on the
11th to 20th item in the x dataset (the ranges are Python syntax, and
are zero-based). Negative indices count backwards from the end of the
dataset. Data can be skipped using expressions such as
<strong class="command">data[::2]</strong>, which skips every other element</p>
</section>
<section id="defining-new-constants-or-functions">
<h2>Defining new constants or functions<a class="headerlink" href="#defining-new-constants-or-functions" title="Permalink to this headline">¶</a></h2>
<p>User defined constants or functions can be defined in the “Custom
definitions” dialog box under the edit menu. Functions can also be
imported from external python modules.</p>
<img alt="_images/customdefinition.png" src="_images/customdefinition.png" />
<p>Custom definitions are defined on a per-document basis, but can be
saved or loaded into a file. A default custom definitions file can be
set in the preferences dialog box.</p>
</section>
<section id="dataset-plugins">
<h2>Dataset plugins<a class="headerlink" href="#dataset-plugins" title="Permalink to this headline">¶</a></h2>
<p>In addition to creating datasets based on expressions, a variety of
dataset plugins exist, which make certain operations on datasets much
more convenient. See the Data, Operations menu for a list of the
default plugins. The user can easily create new plugins. See
<a class="reference external" href="https://github.com/veusz/veusz/wiki/DatasetPlugins">https://github.com/veusz/veusz/wiki/DatasetPlugins</a> for details.</p>
</section>
</section>
<section id="capturing-data">
<h1>Capturing data<a class="headerlink" href="#capturing-data" title="Permalink to this headline">¶</a></h1>
<p>In addition to the standard data import, data can be captured as it is
created from an external program, a network socket or a file or named
pipe. When capturing from a file, the behaviour is like the Unix
<strong class="command">tail -f</strong> command, where new lines written to the file are
captured. To use the capturing facility, the data must be written in
the simple line based standard Veusz text format. Data are whitespace
separated, with one value per dataset given on a single line.</p>
<p>To capture data, use the dialog box <span class="menuselection">Data ‣
Capture</span>. A list of datasets should be given. This is the
<a class="reference internal" href="#id1"><span class="std std-ref">standard descriptor format</span></a>. Choose the source of
the data, which is either a a filename or named pipe, a network socket
to connect to, or a command line for an external program. Capturing
ends if the source of the data runs out (for external programs or
network sockets) or the finish button is clicked. It can optionally
end after a certain number of data lines or when a time period has
expired. Normally the data are updated in Veusz when the capturing is
finished. There is an option to update the document at intervals,
which is useful for monitoring. A plot using the variables will
update when the data are updated.</p>
<p>Click the <code class="docutils literal notranslate"><span class="pre">Capture</span></code> button to start the capture. Click <code class="docutils literal notranslate"><span class="pre">Finish</span></code> or
<code class="docutils literal notranslate"><span class="pre">Cancel</span></code> to stop. Cancelling destroys captured data.</p>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">Veusz</a></h1>
<h3>Navigation</h3>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Reading data</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#standard-text-import">Standard text import</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#data-types-in-text-import">Data types in text import</a></li>
<li class="toctree-l3"><a class="reference internal" href="#descriptors">Descriptors</a></li>
<li class="toctree-l3"><a class="reference internal" href="#descriptor-examples">Descriptor examples</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#csv-files">CSV files</a></li>
<li class="toctree-l2"><a class="reference internal" href="#hdf5-files">HDF5 files</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#error-bars">Error bars</a></li>
<li class="toctree-l3"><a class="reference internal" href="#slices">Slices</a></li>
<li class="toctree-l3"><a class="reference internal" href="#d-data-ranges">2D data ranges</a></li>
<li class="toctree-l3"><a class="reference internal" href="#dates">Dates</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#d-text-or-csv-format">2D text or CSV format</a></li>
<li class="toctree-l2"><a class="reference internal" href="#fits-files">FITS files</a></li>
<li class="toctree-l2"><a class="reference internal" href="#reading-other-data-formats">Reading other data formats</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="#manipulating-datasets">Manipulating datasets</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#using-dataset-plugins">Using dataset plugins</a></li>
<li class="toctree-l2"><a class="reference internal" href="#using-expressions-to-create-new-datasets">Using expressions to create new datasets</a></li>
<li class="toctree-l2"><a class="reference internal" href="#linking-datasets-to-expressions">Linking datasets to expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="#splitting-data">Splitting data</a></li>
<li class="toctree-l2"><a class="reference internal" href="#defining-new-constants-or-functions">Defining new constants or functions</a></li>
<li class="toctree-l2"><a class="reference internal" href="#dataset-plugins">Dataset plugins</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="#capturing-data">Capturing data</a></li>
<li class="toctree-l1"><a class="reference internal" href="api.html">Veusz command line and embedding interface (API)</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="introduction.html" title="previous chapter">Introduction</a></li>
<li>Next: <a href="api.html" title="next chapter">Veusz command line and embedding interface (API)</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2003-2021, Jeremy Sanders.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 4.3.2</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
<a href="_sources/datasets.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
|
