1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
|
<!--
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the files COPYING and Copyright.html. COPYING can be found at the root *
* of the source code distribution tree; Copyright.html can be found at the *
* root level of an installed copy of the electronic HDF5 document set and *
* is linked from the top-level documents page. It can also be found at *
* http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
* access to either file, you may request a copy from help@hdfgroup.org. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
-->
<!-- NEW PAGE -->
<!-- HEADER RIGHT "h5repack" -->
<hr>
<dl>
<dt><div align=right><font color=999999 size=-1><i>
Last modified: 10 September 2013
</i></font></div>
<dt><strong>Tool Name:</strong> <a name="Tools-Repack">h5repack</a>
<dt><strong>Syntax:</strong>
<dd><code>h5repack </code>
[<em>OPTIONS</em>]
<em>in_file</em>
<em>out_file</em>
<p>
<dd><code>h5repack </code>
<code>-i</code> <em>in_file</em>
<code>-o</code> <em>out_file</em>
[<em>OPTIONS</em>]
<p>
<dt><strong>Purpose:</strong>
<dd>Copies an HDF5 file to a new file with or without compression
and/or chunking.
<p>
<dt><strong>Description:</strong>
<dd><code>h5repack</code> is a command line tool that
applies HDF5 filters to an input file <em>in_file</em>,
saving the output in a new output file, <em>out_file</em>.
<p>
<dt><strong>Options and Parameters:</strong>
<dd>
<dl>
<dt><code>-i</code> <em>in_file</em>
<dd>Input HDF5 file
<p>
<dt><code>-o</code> <em>out_file</em>
<dd>Output HDF5 file
<p>
<dt><code>-h</code>
or
<code>--help</code>
<dd>Print help message.
<p>
<dt><code>-v</code>
or
<code>--verbose</code>
<dd>Print verbose output.
<p>
<dt><code>-V</code>
or
<code>--version</code>
<dd>Print version number.
<p>
<dt><code>-n</code>
or
<code>--native</code>
<dd>Use native HDF5 datatypes when repacking.
<br>
(Default behavior is to use original file datatypes.)
<br>
<em>Note that this is a change in default behavior;
prior to Release 1.6.6, <code>h5repack</code> generated
files only with native datatypes.</em>
<p>
<dt><code>-L</code>
or
<code>--latest</code>
<dd>Use latest version of the HDF5 file format.
<p>
<dt><code>-c</code> <em>max_compact_links</em>
or
<code>--compact</code>=<em>max_compact_links</em>
<dd>Set the maximum number of links, <em>max_compact_links</em>,
that can be stored in a group header message (compact format).
<p>
<dt><code>-d</code> <em>min_indexed_links</em>
or
<code>--indexed</code>=<em>min_indexed_links</em>
<dd>Set the minimum number of links, <em>min_indexed_links</em>,
in the indexed format.
<p>
<em>max_compact_links</em> and <em>min_indexed_links</em>
are closely related and the first must be equal to or
greater than the second.
In the general case, however, performance will suffer,
possibly dramatically, if they are equal;
performance can be improved by tuning the gap between
the two values to minimize unnecessary thrashing
between the compact storage and indexed storage modes
as group size waxes and wanes.
The relationship between <em>max_compact_links</em> and
<em>min_indexed_links</em> is most important when
group sizes are highly dynamic; that relationship is much
less important in files with a stable structure.
Compact mode is space and performance-efficient when groups
have small numbers of members;
indexed mode requires slightly more storage space,
but provides increasingly better performance as the
number of members in each group increases.
<p>
<dt><code>-m</code> <em>size</em>
or
<code>--minimum</code>=<em>size</em>
<dd>Apply filter(s) only to objects whose size in bytes is
equal to or greater than <em>size</em>.
<br>
<em>size</em> must be an integer greater than one
( <code>1</code> ).
<p>
<i>Default:</i>
If no size is specified, a threshold of 1024 bytes is assumed.
<p>
<dt><code>-u</code> <em>file</em>
or
<code>--ublock</code>=<em>file</em>
<dd>Specify name of file containing user block data to be added.
<p>
<dt><code>-b</code> <em>user_block_size</em>
or
<code>--block</code>=<em>user_block_size</em>
<dd>Set size in bytes of user block to be added.
<br>
<em>user_block_size</em> must be <code>512</code> or greater
and a power of <code>2</code>.
<p>
<i>Default:</i> <code>1024</code>
<p>
<dt><code>-M</code> <em>size</em>
or
<code>--metadata_block_size</code>=<em>size</em>
<dd>Metadata block size to be used when <code>h5repack</code>
calls <a href="RM_H5P.html#Property-SetMetaBlockSize">
<code>H5Pset_meta_block_size</code></a>.
<br>
<em>size</em> must be a non-negative integer.
<p>
<dt><code>-t</code> <em>alignment_threshold</em>
or
<code>--threshold</code>=<em>alignment_threshold</em>
<dd>Set threshold value for <code>H5Pset_alignment</code> call.
<br>
<em>alignment_threshold</em> must be an integer.
<p>
<dt><code>-a</code> <em>alignment</em>
or
<code>--alignment</code>=<em>alignment</em>
<dd>Set alignment value for <code>H5Pset_alignment</code> call.
<br>
<em>alignment</em> must be a positive integer.
<p>
<!-- NEW PAGE -->
<dt><code>-s</code> <em>min_size</em>[:<em>header_type</em>]
or
<code>--ssize</code>=<em>min_size</em>[:<em>header_type</em>]
<dd>Set the minimum size of optionally specified types
of shared object header messages.
<p>
<em>min_size</em> is the minimum size, in bytes, of a
shared object header message. Header messages smaller than the
specified size will not be shared.
<p>
<em>header_type</em> specifies the type(s) of header message that
this minimum size is to be applied to.
Valid values of <em>header_type</em> are any of the following:
<br><code> dspace </code>
for dataspace header messages
<br><code> dtype </code>
for datatype header messages
<br><code> fill </code>
for fill values
<br><code> pline </code>
for property list header messages
<br><code> attr </code>
for attribute header messages
<br>If <em>header_type</em> is not specified,
<em>min_size</em> will be applied to all header messages.
<p>
<dt><code>-f</code> <em>filter</em>
or
<code>--filter</code>=<em>filter</em>
<dd>Filter type
<p>
<em>filter</em> is a string of the following format:
<dir>
<em>list_of_objects</em> : <em>name_of_filter</em>[=<em>filter_parameters</em>]
<p>
<em>list_of_objects</em> is a comma separated list of object
names meaning apply the filter(s) only to those objects.
If no object names are specified, the filter is applied to
all objects.
<p>
<em>name_of_filter</em> can be one of the following:
<br>
<code>GZIP</code>, to apply the HDF5 GZIP filter
(GZIP compression)
<br>
<code>SZIP</code>, to apply the HDF5 SZIP filter
(SZIP compression)
<br>
<code>SHUF</code>, to apply the HDF5 shuffle filter
<br>
<code>FLET</code>, to apply the HDF5 checksum filter
<br>
<code>NBIT</code>, to apply the HDF5 N-bit filter
<br>
<code>SOFF</code>, to apply the HDF5 scale/offset filter
<br>
<code>UD</code>, to apply a user-defined filter
<br>
<code>NONE</code>, to remove any filter(s)
<p>
<em>filter_parameters</em> conveys optional compression
information:
<br>
<code>GZIP=</code><em>deflation_level</em> from 1-9
<br>
<code>SZIP=</code><em>pixels_per_block,coding_method</em>
<br>
<em>pixels_per_block</em> is a even number
in the range 2-32.
<br>
<em>coding_method</em> is
<code>EC</code> or <code>NN</code>.
<br>
<code>SHUF</code> (no parameter)
<br>
<code>FLET</code> (no parameter)
<br>
<code>NBIT</code> (no parameter)
<br>
<code>SOFF=</code><em>scale_factor,scale_type</em>
<br>
<em>scale_factor</em> is an integer.
<br>
<em>scale_type</em> is either <code>IN</code> or
<code>DS</code>.
<br>
<code>UD=</code><em>filter_id,nfilter_params,value_1[,value_2,....,value_n]</em>
<br>
<em>filter_id</em> is the filter identifier.
<br>
<em>nfilter_params</em> is the number of filter parameters.
<br>
<em>value_1</em> through <em>value_n</em> are the values
of each filter parameter.
<br>
Number of values must match the value of
<em>nfilter_params</em>.
<br>
<code>NONE</code> (no parameter)
</dir>
<p>
<!-- NEW PAGE -->
<dt><code>-l</code> <em>layout</em>
or
<code>--layout</code>=<em>layout</em>
<dd>Layout type
<p>
<em>layout</em> is a string of the following format:<br>
<dir>
<em>list_of_objects</em> : <em>layout_type</em>[=<em>layout_parameters</em>]
<p>
<em>list_of_objects</em> is a comma separated list of object
names, meaning that layout information is supplied for those
objects. If no object names are specified, the layout
is applied to all objects.
<p>
<em>layout_type</em> can be one of the following:
<br>
<code>CHUNK</code>, to apply chunking layout
<br>
<code>COMPA</code>, to apply compact layout
<br>
<code>CONTI</code>, to apply contiguous layout
<p>
<em>layout_parameters</em> is present only in the
<code>CHUNK</code> case and specifies the chunk size of
each dimension in the following format with no intervening
spaces:
<br>
<code><em>dim_1</em> × <em>dim_2</em> × ...
<em>dim_n</em></code>
</dd>
</dir>
<p>
<dt><code>-e</code> <em>file</em>
or
<code>--file</code>=<em>file</em>
<dd>File containing values to be passed in for the
<code>-f</code> (or <code>--filter</code>) and
<code>-l</code> (or <code>--layout</code>) options.
<br>
This file contains only the filter and layout flags.
<p>
<dt><em>in_file</em>
<dd>Input HDF5 file
<p>
<dt><em>out_file</em>
<dd>Output HDF5 file
</dl>
<p>
<dt><strong>Exit Status:</strong></dt>
<dd><table border=0>
<tr valign=top align=left>
<td>0</td>
<td>Succeeded.</td>
</tr>
<tr valign=top align=left>
<td>>0 </td>
<td>An error occurred.</td>
</tr>
</table>
<dt><strong>Examples:</strong>
<dd>
<ol>
<li><code>h5repack -f GZIP=1 -v file1 file2 </code>
<br>
Applies GZIP compression to all objects in <code>file1</code>
and saves the output in <code>file2</code>.
Prints verbose output.
<br>
<li><code>h5repack -f dset1:SZIP=8,NN file1 file2 </code>
<br>
Applies SZIP compression only to object <code>dset1</code>.
<br>
<li><code>h5repack -l dset1,dset2:CHUNK=20x10 file1 file2</code>
<br>
Applies chunked layout to objects <code>dset1</code>
and <code>dset2</code>.
<br>
<li><code>h5repack -f UD=307,1,9 file1 file2</code>
<br>
Adds <code>bzip2</code> filter to all datasets.
<br>
</ol>
<p>
<dt><strong>History:</strong>
<dd><table width="90%">
<tr>
<td valign="top" align="left" width="10%">
<strong>Release</strong> </td>
<td valign="top" align="left">
<strong>Change</strong></td></tr>
<tr>
<td valign="top">1.8.12</td>
<td valign="top">
Added user-defined filter parameter (<code>UD</code>) to
<code>-f <i>filter</i></code>, <code>--filter=<i>filter</i></code>
option for use in read and write operations.</td></tr>
<tr>
<td valign="top">1.8.9</td>
<td valign="top">
<code>-M <i>number</i>, --medata_block_size=<i>number</i></code>
option introduced in this release.</td></tr>
<tr>
<td valign="top">1.8.1</td>
<td valign="top">
Original syntax restored; both the new and the original syntax
are now supported.</td></tr>
<tr>
<td valign="top">1.8.0</td>
<td valign="top">
<code>h5repack</code> command line syntax changed in this
release.</td></tr>
<tr>
<td valign="top">1.6.2</td>
<td valign="top">
<code>h5repack</code> introduced in this release.</td></tr>
</table></dd>
</dl>
|
