1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
|
.. index::
single: Serialization
.. _serialization:
Serialization
-------------
.. cpp:namespace:: zfp
|zfp|'s read-write compressed arrays can be serialized to sequential,
contiguous storage and later recovered back into an object, e.g., to support
I/O of compressed-array objects. Two pieces of information are needed
to describe a |zfp| array: the raw compressed data, obtained via
:cpp:func:`array::compressed_data` and :cpp:func:`array::compressed_size`,
and a :ref:`header <header>` that describes the array scalar type,
dimensions, and rate.
The user may concatenate the header and compressed data to form a
fixed-rate byte stream that can be read by the |zfp|
:ref:`command-line tool <zfpcmd>`. When serializing the array,
the user should first call :cpp:func:`array::flush_cache` before
accessing the raw compressed data.
There are two primary ways to construct a compressed-array object from
compressed data: via array-specific :ref:`constructors <array_ctor_header>`
and via a generic :ref:`factory function <array_factory>`:
- When the array scalar type (i.e., :code:`float` or :code:`double`) and
dimensionality (i.e., 1D, 2D, 3D, or 4D) are already known, the corresponding
array :ref:`constructor <array_ctor_header>` may be used. If the
scalar type and dimensionality stored in the header do not match
the array class, then an :ref:`exception <exception>` is thrown.
- |zfp| provides a :ref:`factory function <array_factory>` that can be used
when the serialized array type is unknown but described in the header.
This function returns a pointer to the abstract base class,
:cpp:class:`array`, which the caller should dynamically cast to the
corresponding derived array, e.g., by examining
:cpp:func:`array::scalar_type` and :cpp:func:`array::dimensionality`.
The (static) factory function is made available by including
:file:`zfp/factory.hpp`. This header must be included *after* first
including the header files associated with the compressed arrays, i.e.,
:file:`zfp/array1.hpp`, :file:`zfp/array2.hpp`, :file:`zfp/array3.hpp`, and
:file:`zfp/array4.hpp`. Only those arrays whose header files are included
can be constructed by the factory function. This design decouples the
array classes so that they may be included independently, for example,
to reduce compilation time.
Both types of deserialization functions accept an :cpp:class:`array::header`,
an optional buffer holding compressed data, and an optional buffer size.
If this buffer is provided, then a separate copy of the compressed data it
holds is made, which is used to initialize the array. If the optional buffer
size is also provided, then these functions throw an
:ref:`exception <exception>` if the size is not at least as large as is
expected from the metadata stored in the header. This safeguard is
implemented to avoid accessing memory beyond the end of the buffer. If no
buffer is provided, then all array elements are default initialized to zero.
The array may later be initialized by directly reading/copying data into the
space pointed to by :cpp:func:`array::compressed_data` and calling
:cpp:func:`array::clear_cache` (in either order).
Below is a simple example of serialization of a 3D compressed array of doubles
(error checking has been omitted for clarity)::
zfp::array3d a(nx, ny, nz, rate);
...
a.flush_cache();
zfp::array::header h(a);
fwrite(h.data(), h.size_bytes(), 1, file);
fwrite(a.compressed_data(), a.compressed_size(), 1, file);
We may then deserialize this array using the factory function. The following
example reads the compressed data directly into the array without making a
copy::
zfp::array::header h;
fread(h.data(), h.size_bytes(), 1, file);
zfp::array* p = zfp::array::construct(h);
fread(p->compressed_data(), p->compressed_size(), 1, file);
assert(p->dimensionality() == 3 && p->scalar_type() == zfp_type_double);
zfp::array3d& a = *dynamic_cast<zfp::array3d*>(p);
When the array is no longer in use, call :code:`delete p;` to deallocate it.
.. note::
The array serialization API changed significantly in |zfp| |crpirelease|.
The :cpp:func:`array::get_header` function is now deprecated and has been
replaced with a :ref:`header constructor <header_ctor>` that takes an
array as parameter. Exceptions are now part of the main :code:`zfp`
namespace rather than nested within the array header. The header is no
longer a simple POD data structure but should be queried for its data
pointer and size.
.. index::
single: Header
.. _header:
Header
^^^^^^
.. cpp:namespace:: zfp
Short 12-byte headers are used to describe array metadata and compression
parameters when serializing a compressed array. This header is the same as
supported by the :c:func:`zfp_read_header` and :c:func:`zfp_write_header`
functions, using :c:macro:`ZFP_HEADER_FULL` to indicate that complete metadata
is to be stored in the header. The header is also compatible with the |zfp|
:ref:`command-line tool <zfpcmd>`. Processing of the header may result in an
:ref:`exception <exception>` being thrown.
.. note::
Compressed-array headers use |zfp|'s most concise representation of only
96 bits. Such short headers support compressed blocks up to 2048 bits long.
This implies that the highest rate for 3D arrays is 2048/4\ :sup:`3` = 32
bits/value; the highest rate for 4D arrays is only 2048/4\ :sup:`4` = 8
bits/value. 3D and 4D arrays whose rate exceeds these limits cannot be
serialized and result in an exception being thrown. 1D and 2D arrays
support rates up to 512 and 128 bits/value, respectively, which both
are large enough to represent all usable rates.
.. cpp:class:: array::header
The header stores information such as scalar type, array dimensions, and
compression parameters such as rate. Compressed-array headers are always
96 bits long.
.. cpp:namespace:: zfp::array
----
.. cpp:function:: header::header()
Default constructor for header.
----
.. _header_ctor:
.. cpp:function:: header::header(const array& a)
Construct header for compressed-array *a*. Throws an
:ref:`exception <exception>` upon failure.
----
.. _header_ctor_buffer:
.. cpp:function:: header::header(const void* buffer, size_t bytes = 0)
Deserialize header from memory buffer given by *buffer* of optional
size *bytes*. This memory buffer is obtained from an existing
header during serialization via :cpp:func:`header::data` and
:cpp:func:`header::size_bytes`. The constructor throws an
:ref:`exception <exception>` upon failure.
----
.. cpp:function:: zfp_type header::scalar_type() const
Scalar type associated with array (see :cpp:func:`array::scalar_type`).
----
.. cpp:function:: uint header::dimensionality() const
Dimensionality associated with array (see :cpp:func:`array::dimensionality`).
----
.. _header_dims:
.. cpp:function:: size_t header::size_x() const
.. cpp:function:: size_t header::size_y() const
.. cpp:function:: size_t header::size_z() const
.. cpp:function:: size_t header::size_w() const
:ref:`Array dimensions <array_dims>`. Unused dimensions have a size of zero.
----
.. cpp:function:: double header::rate() const
Rate in bits per value (see :cpp:func:`array::rate`);
----
.. cpp:function:: virtual const void* header::data() const = 0
Return pointer to header data.
----
.. cpp:function:: virtual size_t header::size_bytes(uint mask = ZFP_DATA_HEADER) const = 0
When *mask* = :c:macro:`ZFP_DATA_HEADER`, return header payload size in
bytes pointed to by :cpp:func:`header::data`. Only those bytes are needed
to (de)serialize a header. The header object stores additional (redundant)
metadata whose size can be queried via :c:macro:`ZFP_DATA_META`.
.. index::
single: Exceptions
.. _exception:
Exceptions
^^^^^^^^^^
.. cpp:class:: exception : public std::runtime_error
Compressed arrays may throw this exception upon serialization, when
constructing a header via its :ref:`constructor <header_ctor>`, or
deserialization, when constructing a compressed array via its
:ref:`constructor <array_ctor_header>` or
:ref:`factory function <array_factory>`.
The :cpp:func:`exception::what` method returns a :code:`std::string`
error message that indicates the cause of the exception.
Most error messages changed in |zfp| |4darrrelease|.
|
