1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
|
<html>
<head>
<meta name="keywords" content="eso, FITS format, C library">
<link href="doxygen.css" rel="stylesheet" type="text/css">
<title>QFITS Reference Manual 6.2.0</title>
</head>
<body text="#000000" bgcolor="#ffffff">
<!-- Generated by Doxygen 1.4.1 -->
<h1>qfits Reference manual </h1>
<p>
<h3 align="center">6.2.0 </h3><hr>
<h2><a class="anchor" name="welcome">
Introduction</a></h2>
<b>qfits</b> is a stand-alone library written in C to interact with files complying with the FITS format. It is fast and portable over any kind of POSIX-compliant platform.<p>
Rather than going through the FITS format specification and trying to implement some support for everything that is described there, this library was built day after day upon request. This guarantees that all the functions you will find in the library have been written for some purpose in the VLT pipeline context, and all of them are used daily in a production environment.<p>
Functionalities offered by this library are:<p>
<ul>
<li>Header queries (get keywords, values).</li><li>Header manipulation (load/modify/save).</li><li>Header/data offset queries.</li><li>Pixel loading/saving to memory.</li><li>Support for files of any dimension (NAXIS).</li><li>Support for FITS extensions, including ASCII and binary tables.</li></ul>
<p>
<hr>
<h2><a class="anchor" name="history">
History</a></h2>
<b>qfits</b> was born from the need to have a simple but powerful C library to handle FITS file access. There are already many libraries on the Net to handle FITS files, but they tend to be bloated with far too many features and may raise portability and maintenance issues.<p>
<b>qfits</b> was written to take care of all low-level aspects of the FITS format, and only these. You will find there a wealth of optimized functions to access everything inside FITS files, but nothing about what you could do with the data: this is left to other (e.g. image processing) libraries. There is no suggested image or cube type, and the table object only loads the data without trying to interpret them.<p>
The idea is that people wanting to work with FITS might have different requirements: somebody writing an image viewer might just want to load a pixel buffer, somebody handling headers only might want to have easy access to header information without touching pixel data. <b>qfits</b> allows you to get your hands on what is contained in a FITS file but does not force you to use any specific high-level data type to describe your data.<p>
<hr>
<h2><a class="anchor" name="header">
Header handling</a></h2>
This section gives you an overview of the functionalities offered to work on FITS headers with <b>qfits</b>. For a complete reference, check out the reference manual: qfits.h.<p>
FITS headers are simply formatted as 80-char lines (<em>cards</em>) containing ancillary data represented in ASCII format like:<p>
keyword = value / comment<p>
If you want to retrieve data from a FITS file header, you will find various useful query routines to get a keyword value from main or extension headers. Values are returned as strings and not coerced to any particular (e.g. numerical) type. See e.g.<p>
<ul>
<li><a class="el" href="group__qfits__tools.html#ga0">qfits_query_hdr()</a> to query the main header.</li><li><a class="el" href="group__qfits__tools.html#ga1">qfits_query_ext()</a> to query an extension header.</li><li><a class="el" href="group__qfits__tools.html#ga2">qfits_query_n_ext()</a> to get the number of extensions.</li></ul>
<p>
Since all FITS values are returned as strings, you may need to identify types. See e.g.<p>
<ul>
<li><a class="el" href="group__qfits__tools.html#ga10">qfits_get_type()</a> to identify a value type</li><li><a class="el" href="group__qfits__tools.html#ga5">qfits_is_boolean()</a></li><li><a class="el" href="group__qfits__tools.html#ga6">qfits_is_int()</a></li><li><a class="el" href="group__qfits__tools.html#ga7">qfits_is_float()</a></li><li><a class="el" href="group__qfits__tools.html#ga8">qfits_is_complex()</a></li><li><a class="el" href="group__qfits__tools.html#ga9">qfits_is_string()</a></li></ul>
<p>
Of course, you can use the usual <code>atof()</code>, <code>atoi()</code> and <code>scanf()</code> functions to convert a string to a numerical type.<p>
You may also want to perform more complex operations on FITS headers, like loading one, modifying it and saving it back to a new file. The qfits_header data structure is offered for that purpose. It comes with utilities like:<p>
<ul>
<li><a class="el" href="group__qfits__rw.html#ga0">qfits_header_read()</a> to load a header.</li><li><a class="el" href="group__qfits__rw.html#ga3">qfits_header_readext()</a> same from an extension.</li><li><a class="el" href="group__qfits__header.html#ga16">qfits_header_dump()</a> to save a header to a file.</li><li>...</li></ul>
<p>
And of course all manipulation tools you can think of:<p>
<ul>
<li><a class="el" href="group__qfits__header.html#ga2">qfits_header_add()</a></li><li><a class="el" href="group__qfits__header.html#ga3">qfits_header_add_after()</a></li><li><a class="el" href="group__qfits__header.html#ga4">qfits_header_append()</a></li><li><a class="el" href="group__qfits__header.html#ga6">qfits_header_mod()</a></li><li><a class="el" href="group__qfits__header.html#ga5">qfits_header_del()</a></li><li>...</li></ul>
<p>
There are other functions to create new FITS cards, dump a header to screen for debugging purposes, etc. See the reference manual for an exhaustive description.<p>
An important feature of the qfits_header object when loaded from a file, is that it keeps the memory of the initial FITS card. The data structure that is carried around for each FITS card contains:<p>
<ul>
<li>Identified keyword</li><li>Identified value</li><li>Possible comment</li><li>Line as read from initial file when applicable.</li></ul>
<p>
This feature is fairly useful if you want to perform header manipulation without getting too intrusive. You can use this to load a header, modify a couple of cards and dump all back to disk. The dumped header will only be modified for the cards you have touched, all others will be forwarded verbatim as they appeared in the initial file. This least-modification policy ensures that you will not modify the lines you do not need to touch.<p>
Notice that ESO's <code>HIERARCH</code> keywords and DICB ordering are natively supported by <b>qfits</b>, i.e. when you work with qfits_header objects you can be sure that they will properly recognize and sort keywords following ESO conventions.<p>
<hr>
<h2><a class="anchor" name="data">
Data handling</a></h2>
This section gives you an overview of the functionalities offered to work on FITS data segments with <b>qfits</b>. For a complete reference, check out the reference manual.<p>
Data segments are quite simply stored: they always contain data contiguously in uncompressed format, so reading them is mostly a matter of applying an <code>fread()</code> statement at the right place in the file, with the right size. To find out about offsets to data and header parts in a FITS file (possibly containing several extensions), you can make use of:<p>
<ul>
<li><a class="el" href="group__qfits__rw.html#ga6">qfits_get_hdrinfo()</a> to return information about a header position and size (in bytes) in a file.</li><li><a class="el" href="group__qfits__rw.html#ga7">qfits_get_datinfo()</a> to return information about a data segment position and size (in bytes) in a file.</li></ul>
<p>
<b>qfits</b> includes a caching mechanism to speed up accesses to large files to an optimal access time (only one parsing of the file is needed, even with multiple queries on the same file). This mechanism is internal to this module and invisible to the programmer using <b>qfits</b>. This means that you can loop on all sections on a huge file to retrieve file offsets for each extension, and pay the price of file parsing only at the first call.<p>
If you want to dive further into data loading, you may want to have a look at table and image handling functionalities.<h3><a class="anchor" name="table">
Table handling</a></h3>
A table is stored in a FITS file in an extension. The extension header contains all needed informations to read the data (number of columns, the column types, the columns labels, the columns unit, etc...).<p>
A FITS table is composed by columns. Within one column there can only be data of one defined type, but you can store several values by field.<p>
In BINARY tables, you can find 11 different data types, in ASCII tables, there are 5 different types. All these types are supported by <b>qfits:</b> <ul>
<li>ASCII tables:<ul>
<li>TFITS_ASCII_TYPE_A : Characters (1 byte)</li><li>TFITS_ASCII_TYPE_D : Double (8 bytes)</li><li>TFITS_ASCII_TYPE_E : Float (4 bytes)</li><li>TFITS_ASCII_TYPE_F : Float (4 bytes)</li><li>TFITS_ASCII_TYPE_I : Integer (4 bytes)</li></ul>
</li><li>BIN tables:<ul>
<li>TFITS_BIN_TYPE_A : Characters (1 byte)</li><li>TFITS_BIN_TYPE_B : Unsigned byte (1 byte)</li><li>TFITS_BIN_TYPE_C : Float complex (2 * 4 bytes)</li><li>TFITS_BIN_TYPE_D : Double (8 bytes)</li><li>TFITS_BIN_TYPE_E : Float (4 bytes)</li><li>TFITS_BIN_TYPE_I : Short (2 bytes)</li><li>TFITS_BIN_TYPE_J : Integer (4 bytes)</li><li>TFITS_BIN_TYPE_L : Logical (1 byte)</li><li>TFITS_BIN_TYPE_M : Double complex (2 * 8 bytes)</li><li>TFITS_BIN_TYPE_P : Array descriptor (2 * 4 bytes)</li><li>TFITS_BIN_TYPE_X : Bit (1 bit)</li></ul>
</li></ul>
<p>
The qfits_table object provided by <b>qfits</b> contains all necessary informations to define how the data are stored in the file: the name of the file it comes from, the table type (QFITS_BINTABLE or QFITS_ASCIITABLE), the number of columns and for each column a pointer to a qfits_col object.<p>
Each qfits_col object contains informations defining the associated column: the type (e.g. TFITS_BIN_TYPE_L), the number of rows, the number of atoms per field, the label, the unit, etc...).<p>
This qfits_table object is returned by <a class="el" href="group__qfits__table.html#ga5">qfits_table_open()</a> and should only be destroyed with <a class="el" href="group__qfits__table.html#ga6">qfits_table_close()</a>.<p>
This qfits_table object does not contain the table data, but has to be used to load the data with functions like:<p>
<ul>
<li><a class="el" href="group__qfits__table.html#ga7">qfits_query_column()</a></li><li><a class="el" href="group__qfits__table.html#ga9">qfits_query_column_data()</a></li><li><a class="el" href="group__qfits__table.html#ga11">qfits_query_column_nulls()</a></li></ul>
<p>
<a class="el" href="group__qfits__table.html#ga7">qfits_query_column()</a> returns you simply the data (byte swapped if necessary) as they were read in the file as a byte array. It is up to the user to interpret the data (e.g. to try to find out where the NULL values are).<p>
<a class="el" href="group__qfits__table.html#ga9">qfits_query_column_data()</a> does the same, but identifies the NULL values and replaces them by a user-provided value. In this case, the returned array is an array of data of the right type. It is returned as a void*, the user just has to cast it to a double*, a short* or what is needed.<p>
For example, if you require in a table that contains 15 rows a column where you can find 3 values of type TFITS_BIN_TYPE_M (double complex), <a class="el" href="group__qfits__table.html#ga9">qfits_query_column_data()</a> will return an array of 15 * 3 * 2 double values.<p>
<b>qfits</b> also provides functions to write a table to a FITS file (<a class="el" href="group__qfits__table.html#ga12">qfits_save_table_hdrdump()</a>), or to print out a table value or a complete table on the screen or in a file (<a class="el" href="group__qfits__table.html#ga15">qfits_table_field_to_string()</a>).<h3><a class="anchor" name="image">
Image handling</a></h3>
The qfitsloader and qfitsdumper objects are offered to simplify the action of reading/writing image data. The corresponding operators take care of retrieving all necessary ancillary information (image size, pixel type, buffer position), perform a memory allocation or mapping and read the file into memory. See the following functions:<p>
<ul>
<li><a class="el" href="group__qfits__image.html#ga0">qfitsloader_init()</a> to initialize a loader object.</li><li><a class="el" href="group__qfits__image.html#ga1">qfits_loadpix()</a> to load pixel data into memory.</li><li><a class="el" href="group__qfits__image.html#ga3">qfits_pixdump()</a> to save pixel data to disk.</li></ul>
<p>
As for tables, the idea is that you first launch an analysis of the file to get back a number of informations about what is present there. In the case of images, you use <a class="el" href="group__qfits__image.html#ga0">qfitsloader_init()</a> to see if pixel loading could be done, then use <a class="el" href="group__qfits__image.html#ga1">qfits_loadpix()</a> to perform the actual load.<p>
Loading pixels as floats means that whichever pixel format (BITPIX) is used in your input FITS file, all pixels will be converted to your local representation of float upon loading. The 3 offered types basically allow you to work with 3 kinds of pixels internally, choose the one that best suits your needs. 'int' is not recommended for loss of precision, and 'double' for memory and performance issues.<p>
Notice that as soon as you start using pixel loading functions, you must comply with the <b>qfits</b> memory model.<p>
<hr>
<h2><a class="anchor" name="features">
Features</a></h2>
This section describes various unique features of <b>qfits</b>.<h3><a class="anchor" name="portability">
Portability</a></h3>
This library has been ported and tested on the following platforms:<p>
<ul>
<li>Linux x86 and alpha</li><li>Solaris 2.5, 2.6 and 2.8</li><li>HPUX 8, 9, 10, 11</li><li>AIX</li><li>BSD compatible, including Darwin (Mac OS X)</li><li>OSF/1 or Tru64</li></ul>
<h3><a class="anchor" name="speed">
Speed</a></h3>
The offered routines make use of caching mechanisms and memory-mapping system calls to enhance FITS file parsing and speed up the code.<h3><a class="anchor" name="precision">
Numerical precision</a></h3>
Since FITS headers are stored as strings, numerical precision is limited by the number of digits used to write a number in a FITS card, which is in theory larger than what a 32-bit floating-point can store. Using <b>qfits</b>, these values are available to a C programmer as a string, making sure that precision has at least not be lost in the reading process.<h3><a class="anchor" name="conservative">
Conservative headers</a></h3>
As mentioned above, if you only need to load a header, modify a card and save it back, you will find out that <em>only</em> the card you touched has been modified, the rest was verbatim transferred. This is useful to ensure that the library formatting does not disturb your format.<h3><a class="anchor" name="hierarch">
Native HIERARCH support</a></h3>
Native support for ESO's <code>HIERARCH</code> keywords, as well as keywords of any length (up to 80 chars per line).<h3><a class="anchor" name="dicb">
ESO/DICB keyword ordering</a></h3>
Native support for DICB (ESO only) ordering of keywords in the headers. FITS files created with this library will be DICB compliant with respect to keyword ordering.<h3><a class="anchor" name="qfits_memory">
Memory model</a></h3>
<b>qfits</b> does not only offer pixel loading mechanisms, it also comes bundled with a memory module based on a model optimized for the handling of very large data segments. Calls to memory allocators will yield valid data pointers past the hardware limits of your machine, up to 2 or 4 Gb on a 32-bit workstation and insanely high values on 64-bit processors.<p>
<hr>
<h2><a class="anchor" name="install">
Installation instructions</a></h2>
In the main <b>qfits</b> directory, type:<p>
<div class="fragment"><pre class="fragment">
./configure --prefix=install_dir ; make ; make install
</pre></div><p>
To use the library in your programs, add the following line on top of your module:<p>
<div class="fragment"><pre class="fragment"><span class="preprocessor">#include "qfits.h"</span>
</pre></div><p>
And link your program with the <b>qfits</b> library by adding <code>-lqfits</code> to the compile line.<p>
<hr>
<h2><a class="anchor" name="faq">
Frequently Asked Questions</a></h2>
<h3><a class="anchor" name="faq1">
Where should I start to use qfits?</a></h3>
Try to build the library on your system first. You should then have a new library file called libqfits.a. To use the library functionalities in your programs, you must add the following include in your list of includes:<p>
<div class="fragment"><pre class="fragment">
#include "qfits.h"
</pre></div><p>
And then compile your program adding the correct flags to indicate where the <b>qfits.h</b> header file is, where the libqfits.a file is, and that you want to link against libqfits. Example: if the header file is in /usr/local/include and the library in /usr/local/lib, you would use:<p>
<div class="fragment"><pre class="fragment">
% cc -o myprog myprog.c -I/usr/local/include -L/usr/local/lib -lqfits
</pre></div><p>
Now you are all set. Refer to this documentation for more information about the offered data structures and associated methods.<h3><a class="anchor" name="faq2">
What should I know about the cache mechanism?</a></h3>
Parsing FITS files to find extensions is a lengthy process, because the FITS format does not declare in the main header where the following extensions are located in the file. Any access to a FITS file with this library will cache a number of offset pointers in the file to allow fast access to extensions. This means that the file is parsed completely only once, the first time it is accessed through any of the <b>qfits</b> functions.<p>
The cache is implemented (since version 4.2) as a rotating buffer of modest size: 128 FITS file information structures can be held simultaneously. If you do need to perform accesses on a list of more than 128 files and want to use the caching mechanism at its best, it is recommended to isolate all your FITS requests to a given file in the same code area.<p>
As a programmer, you do not need to initialize or shutdown the cache, this is done automatically.<p>
The rotating buffer is a global data structure private to the cache module. The immediate side-effect is that the library is thread-unsafe. If you are concerned about writing multi-threaded applications, you should use a mutex upon accessing this library. Making a thread-safe compliant version of this library should not be too hard, but the need has not been felt yet.
</body>
</html>
|
