1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>LibRaw: General Notes on API</title>
</head>
<body>
<a href=index-eng.html>[back to Index]</a>
<h1>LibRaw: General Notes on API</h1>
<h2>Contents</h2>
<ol>
<li><a href="#versions">LibRaw editions</a></li>
<li><a href="#errors">Error Code Conventions and Error Handling</a></li>
<li><a href="#warnings">Nonstandard Situations That Are Not Errors</a></li>
<li><a href="#io">Input Layer Abstraction</a></li>
<li><a href="threads">Thread Safety</a></li>
<li><a href="#CXX">The Use of C++</a></li>
<li><a href="#imgdata_params">Parameters of the LibRaw::imgdata.params Structure Affecting the Behavior of
open_file/unpack/unpack_thumb</a></li>
<li><a href="#memory">Memory Usage</a>
<ol>
<li><a href="#stack">Stack Usage</a></li>
<li><a href="#memmgr">Dynamic Memory Management</a></li>
<li><a href="#memuse">Dynamic Memory Usage</a>
<ol>
<li><a href="#memraw">Memory Buffer for the RAW Image</a></li>
<li><a href="#memimage">Memory for the Postprocessed Image</a></li>
<li><a href="#memthumb">Memory for the Decoded Thumbnail</a></li>
<li><a href="#memprofile">Memory for the Decoded ICC Profile</a></li>
<li><a name="#memraw">Memory for RAW Unpacking</a></li>
<li><a href="#mempostproces">Memory for Postprocessing</a></li>
<li><a href="#memwrite">Memory for File Writing</a></li>
<li><a href="#memunpack">Unpacking into memory buffer</a></li>
</ol>
</li>
</ol>
</li>
<li><a href="#incompat">Incompatibilities with dcraw</a>
<ol>
<li><a href="#incompat_kodak">Processing of Thumbnails from Kodak cameras</a></li>
</ol>
</li>
</ol>
<a name=versions></a>
<h2>LibRaw Versions</h2>
<p>Since version 0.9, there is only one LibRaw variants. Older versions have three separate editions (normal,
-Lite and -Commercial versions).
</p>
<a name=errors></a>
<h2>Error Code Conventions and Error Handling</h2>
<p>
The following conventions concern the returned errors:
</p>
<ol>
<li>All functions that can return an error code have integer type of return data.</li>
<li>If there is no error, the return value is 0 (LIBRAW_SUCCESS).</li>
<li>If an error has happened in a system call, the return value is errno
(a positive number), which can be analyzed using strerror() or similar means.</li>
<li>All LibRaw's own error codes are negative; each of these errors belongs to one of two types:
<dl>
<dt><b>Non-fatal errors</b></dt>
<dd>
Non-fatal errors do not forbid execution of other functions in the processing succession
(e.g., <a href="API-CXX-eng.html#unpack_thumb">unpack_thumb()</a> can easily return the code corresponding to "preview is absent"
but this does not prevent further call of <a href="API-CXX-eng.html#unpack">unpack()</a>.
</dd>
<dt><b>Fatal errors</b></dt>
<dd>
In the case of fatal errors (memory shortage, input data error, data unpacking failure), the current stage of processing
is terminated and all allocated resouces are freed.<br/>
If an attempt to continue processing is made, all subsequent API calls will return the LIBRAW_OUT_OF_ORDER_CALL error.<br/>
At the same time, the LibRaw instance in which a fatal error has occurred can process the next RAW
files in the usual way (by calling <a href="API-CXX-eng.html#open_file">open_file()</a> (or other input methods), then <a href="API-CXX-eng.html#unpack">unpack()</a>,
etc.).
</dd>
</dl>
<li>The macro LIBRAW_FATAL_ERROR(error code) checks if an error is fatal or not.</li>
<li>The error codes are <a href="API-datastruct-eng.html#errors">listed and deciphered here</a>.</li>
</ol>
<a name=warnings></a>
<h2>Nonstandard Situations That Are Not Errors</h2>
<p>If the program has encountered a nonstandard situation that does not prevent retrieval of some data
from a file, it sends a signal by setting the corresponding bit in <a href="API-datastruct-eng.html#libraw_data_t">imgdata.process_warnings</a>.
The possible types of warnings are <a href="API-datastruct-eng.html#warnings">listed and deciphered here</a>.
</p>
<a name="io"></a>
<h2>Input Layer Abstraction</h2>
<p>
LibRaw uses objects derived from
<a href="API-CXX-eng.html#datastream">LibRaw_abstract_datastream</a> for data input.
Semantics of these objects is similar to 'file with arbitrary seek' object: both read and seek
operations are used.
</p>
<p>
Some RAW formats requires temporary switch to another data stream created on top on memory buffer for metadata
read. Methods for doing so are implemented in base class
<a href="API-CXX-eng.html#datastream">LibRaw_abstract_datastream</a> by internal data field <b>substream</b>.
Look into source code of <a href="API-CXX-eng.html#file_datastream">LibRaw_file_datastream</a> class
in <b>libraw/libraw_datastream.h</b> file for more details.
<br/>
When implementing own datastream classes, you need to take <b>substream</b> into account and pass control to
methods of this field if it is active (not NULL).
</p>
<p>
If datastream implementaton knows name of input file, it should provide fname() call. This name will be used
in <a href="API-CXX-eng.html#callbacks">error callbacks</a> and in guessing name of JPEG file with metadata
(for RAW files with external metadata).
</p>
<p>
For external metadata support input class should implement
<b>subfile_open()/subfile_close()</b> methods.
.
<br/>
Sample of these methods implementation may be found in
<a href="API-CXX-eng.html#file_datastream">LibRaw_file_datastream</a> class (look into
<b>libraw/libraw_datastream.h</b> file for details).
</p>
<a name="threads"></a>
<h2>Thread safety</h2>
<p>
Thread safety is ensured if a LibRaw object is created and used within one thread. At the same time, the number
of threads (each with its own LibRaw object) is not limited in any way (except by memory requirements).
</p>
<p>
If a LibRaw object is created in one execution thread and used in another, external synchronization is
necessary.
</p>
<p>
There is two libraries under Unix enviroment (Linux/FreeBSD/MacOS): libraw_r.a (thread-safe) and libraw.a
(single-threaded, slightly faster).
</p>
<p>
Thread-safe library version stores intermediate unpacker data into LibRaw class data. So, several copies of
LibRaw, working in parallel, is possible.
</p>
<p>
Not thread-safe library uses global variable for intermediate data store which is faster but not reenterant.
This library may be used in multi-threaded apps, but only if exactly one LibRaw class copy exists in program.
</p>
<p>
Windows version is similar to multi-threaded Unix one.
</p>
<a name="CXX"></a>
<h2>The Use of C++</h2>
<p>
Exception situations within LibRaw are handled using the C++ exception mechanism. All exceptions are caught inside
the library functions and should not penetrate outside.
</p>
<p>
Memory is allocated/freed using functions malloc(calloc)/free rather than new/delete.
</p>
<p>
No specific libraries (STL, Boost, smart pointers) are used. </p>
<p>
If C API is used, references to C++ calls new/delete still remain, and so linking with libstdc++(Unix)/....(Windows) is necessary.
</p>
<a name="imgdata_params"></a>
<h2>Parameters of the LibRaw::imgdata.params Structure Affecting the Behavior of open_file/unpack/unpack_thumb</h2>
<p>
Most data fields of structure LibRaw::imgdata.params affect only <a href="API-CXX-eng.html#dcrawemu">data
postprocessing</a>, but there are some exceptions, which have been inherited by the current version of LibRaw
from/ dcraw source texts (these dependences will be gradually removed).
<dl>
<dt><b>imgdata.params.use_camera_matrix and imgdata.params.use_camera_wb</b></dt>
<dd>
These fields affect loading of RAW data for cameras with a color matrix.<br/>
<b>Attention!</b> If parameter <b>imgdata.params.use_camera_matrix</b> is not set by the user, it is copied from
<b>imgdata.params.use_camera_wb</b> at the stage of file opening.
</dd>
<dt><b>imgdata.params.user_flip</b></dt>
<dd>
If this parameter is greater than or equal to zero, assignment <code>imgdata.sizes.flip = imgdata.params.user_flip</code> is
performed at the <a href="API-CXX-eng.html#open_file">open_file()</a> stage.
</dd>
<dt><b>imgdata.params.shot_select</b></dt>
<dd>
This parameter makes it possible to select the number of the extracted image for data formats in which storage
of several RAW images in one data file is possible.
</dd>
<dt><b>imgdata.params.half_size</b></dt>
<dd>
Affects RAW data loading for Phase One and Sinar backs. Also, it this parameter is set then image bitmap
will be reduced by half in each dimension. In later case, all 4 components of bitmap will be filled during
data extraction phase.
</dd>
<dt><b>imgdata.params.threshold, imgdata.params.aber</b></dt>
<dd>
If these parameters used, then half-sized bitmap will be used for data unpacking. See above for details.
</dd>
<dt><b>imgdata.params.use_camera_wb</b></dt>
<dd>
Affects loading of white balance matrix for Leaf backs.
</dd>
<dt><b>imgdata.params.document_mode</b></dt>
<dd>
<b>Does not affect data reading in the current version of LibRaw</b>. In dcraw, this parameter affects the thumbnail data for certain Kodak cameras.
</dd>
</dl>
<a name="memory"></a>
<h2>Memory Usage</h2>
<a name="stack"></a>
<h3>Stack Usage</h3>
<p>
An instance of the LibRaw class has its own size about <b>100 Kb</b>; if constructions like <code>LibRaw
imageProcessor;</code> are used, this memory is stack-allocated.
</p>
<p>
Methods of class LibRaw (and C API calls) may allocate up to 130-140 Kb of data on the stack (to place auto
variables) during their work.
</p>
<p>Thus, the work of one LibRaw instance may require about 250 Kb of stack memory. This is not a problem for most
contemporary architectures. However, when working in a multithreaded environment, one should not forget to
allocate a sufficient amount of memory for the thread stack.
</p>
<p>In the case of dynamic allocation (<code>LibRaw *iProcessor = new LibRaw;</code>), the requirements to stack
memory will decrease by 100 Kb, which is the size of a class instance). If <a href="API-C-eng.html">C API</a> is
used, the LibRaw instance is allocated dynamically.
</p>
<a name="memmgr"></a>
<h3>Dynamic Memory Management</h3>
<p>LibRaw keeps record of all allocated dynamic memory blocks; in the case of an exceptional situation (fatal
error), they are all freed. The code for keeping this record is fairly primitive and not designed to consider
allocation of many blocks (in the normal situation, allocation takes place from 2 to 6 times during file
processing); this fact should be taken into account by developers trying to add new methods to LibRaw.
</p>
<a name="memuse"></a>
<h3>Dynamic Memory Usage</h3>
<p>LibRaw uses dynamic memory</p>
<ul>
<li>for the decoded image;</li>
<li>for the decoded thumbnail;</li>
<li>for the postprocessed image;</li>
<li>for the ICC profile retrieved from the RAW file (if available);</li>
<li>for temporary data at the stage of RAW file unpacking;</li>
<li>for temporary data at the stage of postprocessing and result output;</li>
<li>for reading of the RAW source file (only under Win32).</li>
</ul>
<a name="memraw"></a>
<h4>Memory buffer for the RAW image</h4>
<p>
Decoded RAW data are stored:
</p>
<ul>
<li>one 16-bit value per pixel for "bayer" images. The masked pixels (black or dark or masked frame) are
stored with image data.
</li>
<li>Four 16-bit values for full-color images (Foveon, Linear DNG, Canon sRAW etc.).
</li>
</ul>
<p>The buffer for RAW data is allocated by
href="API-CXX-eng.html#unpack">unpack()</a> call and freeed upon calling
<a href="API-CXX-eng.html#recycle">recycle()</a>.
</p>
<a name="memimage"></a>
<h4>Memory for the Postprocessed Image</h4>
<p>
On postprocessing stage each pixel contains four 16-bit values, one for each possible color channel (some
sensors are actually 4-color).
<p>The buffer for the decoded image is allocated upon calling
<a href="API-CXX-eng.html#raw2image">raw2image()</a>,
<a href="API-CXX-eng.html#dcraw_process">dcraw_process()</a> or
<a href="API-CXX-eng.html#dcraw_document_mode_processing">dcraw_mode_processing()</a> calls.
</p>
<p>The buffer freed upon calling <a href="API-CXX-eng.html#recycle">recycle()</a>
or
<a href="API-CXX-eng.html#free_image">free_image()</a> calls.
</p>
<a name="memthumb"></a>
<h4>Memory for the Decoded Thumbnail</h4>
<p>
Memory for the thumbmail is allocated upon calling <a href="API-CXX-eng.html#unpack_thumb">unpack_thumb()</a> and freed upon
calling <a href="API-CXX-eng.html#recycle">recycle()</a>. The size of the allocated buffer is precisely adjusted to the thumbnail size,
i.e., up to several Mb.
</p>
<a name="memprofile"></a>
<h4>Memory for the Decoded ICC Profile</h4>
<p>
Memory for the ICC profile is allocated upon calling <a href="API-CXX-eng.html#unpack_profile">unpack_profile()</a> and freed upon
calling <a href="API-CXX-eng.html#recycle">recycle()</a>. The size of the allocated buffer is precisely adjusted to the ICC profile size,
i.e., up to several hundred Kb.
</p>
<a name="memraw"></a>
<h4>Memory for RAW Unpacking</h4>
<p>
Memory for temporary buffer needed during RAW data unpacking may be allocated during the work of <a
href="API-CXX-eng.html#unpack">unpack()</a> and freed before completion of this function. The sizes of the allocated
buffers are small, up to tens of Kb.
</p>
<a name="mempostproces"></a>
<h4>Memory for Postprocessing</h4>
<p>During image postprocessing (inherited from dcraw), memory for the histogram (128 Kb) is allocated.
This memory is allocated upon calling <a
href="API-CXX-eng.html#dcraw_document_mode_processing">dcraw_document_mode_processing()</a> and
<a href="API-CXX-eng.html#dcraw_process">dcraw_process()</a> and freed upon calling
<a href="API-CXX-eng.html#recycle">recycle()</a>.
</p>
<p>
In addition, during the work of <a href="API-CXX-eng.html#dcraw_process">dcraw_process()</a> and during the
usage of some available possibilities, like
</p>
<ul>
<li>rotation of images from FUJI cameras;</li>
<li>correction of chromatic aberrations;</li>
<li>image size changes (including correction of non-square pixels);</li>
<li>highlight recovery;</li>
</ul>
<p>
a temporary buffer with the size equal to the size of the resultant image (6-8 bytes per pixel for various processing stages)
will be allocated. As soon as the intermediate substage of processing is completed, the buffer with the previous copy
of the image will be freed.<br/>
If postprocessing is not used, then temporary buffers are not allocated.
</p>
<a name="memwrite"></a>
<h4>Memory for File Writing</h4>
<p>
Upon calling <a href="API-CXX-eng.html#dcraw_ppm_tiff_writer">dcraw_ppm_tiff_writer()</a>, memory for a single row of the
output image is allocated. The allocated memory is freed before the end of this call.
</p>
<a name="memunpack"></a>
<h4>Unpacking into memory buffer</h4>
<p>
Functons <a href="API-CXX-eng.html#dcraw_make_mem_image">dcraw_make_mem_image()</a>
<a href="API-CXX-eng.html#dcraw_make_mem_thumb">dcraw_make_mem_thumb()</a> (and complementary calls in C-API)
allocates memory for entire output datasets (full RGB bitmap and thumbnail, respectively).
<b>Calling function should free() this memory themself.</b>
</p>
<a name="incompat"></a>
<h2>Incompatibilities with dcraw</h2>
<a name="incompat_kodak"></a>
<h3>Processing of Thumbnails from Kodak cameras</h3>
<p>In some Kodak cameras, the preview (thumbnail) is stored in the form of uncorrected image. During its extraction using
<b>dcraw -e</b>, the white balance, color conversion, and other settings are the same as those used for extraction of the main RAW data
(including defect removal and dark frame subtraction, which is erroneous, since the image size is different).
<br/>
In LibRaw::unpack_thumb() calls, the white balance taken from the camera ("as shot") is used and no settings from
imgdata.params are considered.
</p>
<p>
For all other cameras, thumbnails are extracted "as is," without any color conversions, both in dcraw and in LibRaw.
</p>
<a href=index-eng.html>[back to Index]</a>
<hr>
<address><a href="mailto:info@libraw.org">LibRaw Team</a></address>
<!-- Created: Sun Mar 16 10:05:08 MSK 2008 -->
<!-- hhmts start -->
Last modified: Mon Jul 11 20:33:38 MSD 2011
<!-- hhmts end -->
</body>
</html>
|
