1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
|
<!-- Generated by Harlequin WebMaker 2.2.3 (24-Apr-1996)
LispWorks 3.2.2 -->
<HTML> <HEAD>
<TITLE>Appendix B File Format Specification</TITLE>
</HEAD>
<BODY bgcolor="#ffffff">
<A NAME=HEADING18></A>
<A HREF="guidef-19.html">[Next] </A><A HREF="guidef-17.html">[Previous] </A><A HREF="guidef-1.html">[Top] </A><A HREF="guidef-3.html">[Contents] </A><A HREF="guidef-21.html">[Index] </A><A HREF="http://www.unidata.ucar.edu/packages/netcdf/">[netCDF Home Page]</A><A HREF="http://www.unidata.ucar.edu/">[Unidata Home Page]</A><P>
NetCDF User's Guide for Fortran<P>
<A NAME=HEADING18-0></A>
<H1>Appendix B <A NAME=MARKER-2-3316></A><A NAME=MARKER-2-3317></A><A NAME=MARKER-2-3318></A><A NAME=MARKER-2-3319></A><A NAME=MARKER-9-3320></A>File Format Specification</H1>
<HR>
This appendix specifies the netCDF <A NAME=MARKER-2-3321></A><A NAME=MARKER-2-3322></A><A NAME=MARKER-2-3323></A><A NAME=MARKER-2-3324></A>file format version 1. This format will be in use at least through netCDF library version 3.0. <P>
The format is first presented formally, using a BNF <A NAME=MARKER-2-3325></A><A NAME=MARKER-2-3326></A><A NAME=MARKER-2-3327></A>grammar notation. In the grammar, optional components are enclosed between braces ('<CODE>[</CODE>' and '<CODE>]</CODE>'). Comments follow '<CODE>//</CODE>' characters. Nonterminals are in lower case, and terminals are in upper case. A sequence of zero or more occurrences of an entity are denoted by '<CODE>[entity ...]</CODE>'. <P>
<A NAME=HEADING18-3></A>
<H4> The Format in Detail</H4>
<PRE>
netcdf_file := header data
header := magic numrecs dim_array gatt_array var_array
magic := 'C' 'D' 'F' VERSION_BYTE
VERSION_BYTE := '\001' // the file format version number
numrecs := NON_NEG
dim_array := ABSENT | NC_DIMENSION nelems [dim ...]
gatt_array := att_array // global attributes
att_array := ABSENT | NC_ATTRIBUTE nelems [attr ...]
var_array := ABSENT | NC_VARIABLE nelems [var ...]
ABSENT := ZERO ZERO // Means array not present (equivalent to
// nelems == 0).
nelems := NON_NEG // number of elements in following sequence
dim := name dim_length
name := string
dim_length := NON_NEG // If zero, this is the record dimension.
// There can be at most one record dimension.
attr := name nc_type nelems [values]
nc_type := NC_BYTE | NC_CHAR | NC_SHORT | NC_INT | NC_FLOAT | NC_DOUBLE
var := name nelems [dimid ...] vatt_array nc_type vsize begin
// nelems is the rank (dimensionality) of the
// variable; 0 for scalar, 1 for vector, 2 for
// matrix, ...
vatt_array := att_array // variable-specific attributes
dimid := NON_NEG // Dimension ID (index into dim_array) for
// variable shape. We say this is a "record
// variable" if and only if the first
// dimension is the record dimension.
vsize := NON_NEG // Variable size. If not a record variable,
// the amount of space, in bytes, allocated to
// that variable's data. This number is the
// product of the dimension lengths times the
// size of the type, padded to a four byte
// boundary. If a record variable, it is the
// amount of space per record. The netCDF
// "record size" is calculated as the sum of
// the vsize's of the record variables.
begin := NON_NEG // Variable start location. The offset in
// bytes (seek index) in the file of the
// beginning of data for this variable.
data := non_recs recs
non_recs := [values ...] // Data for first non-record var, second
// non-record var, ...
recs := [rec ...] // First record, second record, ...
rec := [values ...] // Data for first record variable for record
// n, second record variable for record n, ...
// See the note below for a special case.
values := [bytes] | [chars] | [shorts] | [ints] | [floats] | [doubles]
string := nelems [chars]
bytes := [BYTE ...] padding
chars := [CHAR ...] padding
shorts := [SHORT ...] padding
ints := [INT ...]
floats := [FLOAT ...]
doubles := [DOUBLE ...]
padding := <0, 1, 2, or 3 bytes to next 4-byte boundary>
// In header, padding is with 0 bytes. In
// data, padding is with variable's fill-value.
NON_NEG := <INT with non-negative value>
ZERO := <INT with zero value>
BYTE := <8-bit byte>
CHAR := <8-bit ACSII/ISO encoded character>
SHORT := <16-bit signed integer, Bigendian, two's complement>
INT := <32-bit signed integer, Bigendian, two's complement>
FLOAT := <32-bit IEEE single-precision float, Bigendian>
DOUBLE := <64-bit IEEE double-precision float, Bigendian>
// tags are 32-bit INTs
NC_BYTE := 1 // data is array of 8 bit signed integer
NC_CHAR := 2 // data is array of characters, i.e., text
NC_SHORT := 3 // data is array of 16 bit signed integer
NC_INT := 4 // data is array of 32 bit signed integer
NC_FLOAT := 5 // data is array of IEEE single precision float
NC_DOUBLE := 6 // data is array of IEEE double precision float
NC_DIMENSION := 10
NC_VARIABLE := 11
NC_ATTRIBUTE := 12
</PRE>
<A NAME=HEADING18-121></A>
<H4> <A NAME=MARKER-2-3328></A>Computing File Offsets</H4>
To calculate the <A NAME=MARKER-2-3329></A><A NAME=MARKER-2-3331></A><A NAME=MARKER-2-3332></A>offset (position within the file) of a specified data value, let <I>external_sizeof</I> be the external size in bytes of one data value of the appropriate type for the specified variable, <I>nc_type</I>: <P>
<PRE>
NC_BYTE 1
NC_CHAR 1
NC_SHORT 2
NC_INT 4
NC_FLOAT 4
NC_DOUBLE 8
</PRE>
On a call to <A NAME=MARKER-10-3333></A>NF_OPEN (or <A NAME=MARKER-10-3334></A>NF_ENDDEF), scan through the array of variables, denoted <I>var_array</I> above, and sum the <I>vsize</I> fields of "record" variables to compute <I>recsize</I>. <P>
Form the products of the dimension lengths for the variable from right to left, skipping the leftmost (record) dimension for record variables, and storing the results in a <I>product</I> array for each variable. For example: <P>
<PRE>
Non-record variable:
dimension lengths: [ 5 3 2 7]
product: [210 42 14 7]
Record variable:
dimension lengths: [0 2 9 4]
product: [0 72 36 4]
</PRE>
At this point, the leftmost product, when rounded up to the next multiple of 4, is the variable size, <I>vsize</I>, in the grammar above. For example, in the non-record variable above, the value of the <I>vsize</I> field is 212 (210 rounded up to a multiple of 4). For the record variable, the value of <I>vsize</I> is just 72, since this is already a multiple of 4. <P>
Let <I>coord</I> be an array of the coordinates of the desired data value, and <I>offset</I> be the desired result. Then <I>offset</I> is just the file offset of the first data value of the desired variable (its <I>begin</I> field) added to the inner product of the <I>coord </I>and <I>product</I> vectors times the size, in bytes, of each datum for the variable. Finally, if the variable is a record variable, the product of the record number, '<CODE>coord[0]</CODE>', and the record size, <I>recsize</I> is added to yield the final <I>offset</I> value. <P>
In pseudo-C code, here's the calculation of <I>offset</I>:<P>
<PRE>
for (innerProduct = i = 0; i < var.rank; i++)
innerProduct += product[i] * coord[i]
offset = var.begin;
offset += external_sizeof * innerProduct
if(IS_RECVAR(var))
offset += coord[0] * recsize;
</PRE>
So, to get the data value (in external representation):<P>
<PRE>
lseek(fd, offset, SEEK_SET);
read(fd, buf, external_sizeof);
</PRE>
<B>A special case:</B> Where there is exactly one record variable, we drop the restriction that each record be four-byte aligned, so in this case there is no record padding. <P>
<A NAME=HEADING18-153></A>
<H4> <A NAME=MARKER-2-3335></A><A NAME=MARKER-2-3336></A><A NAME=MARKER-2-3337></A>Examples</H4>
By using the grammar above, we can derive the <A NAME=MARKER-2-3338></A><A NAME=MARKER-2-3339></A><A NAME=MARKER-2-3340></A><A NAME=MARKER-2-3341></A>smallest valid netCDF file, having no dimensions, no variables, no attributes, and hence, no data. A CDL representation of the empty netCDF file is <P>
<PRE>
netcdf empty { }
</PRE>
This <A NAME=MARKER-2-3342></A><A NAME=MARKER-2-3343></A><A NAME=MARKER-2-3344></A>empty netCDF file has 32 bytes, as you may verify by using '<CODE>ncgen -b empty.cdl</CODE>' to generate it from the CDL representation. It begins with the four-byte "magic number" that identifies it as a netCDF version 1 file: 'C', 'D', 'F', '\001'. Following are seven 32-bit integer zeros representing the number of records, an empty array of dimensions, an empty array of global attributes, and an empty array of variables. <P>
Below is an (edited) dump of the file produced on a big-endian machine using the Unix command<P>
<PRE>
od -xcs empty.nc
</PRE>
Each 16-byte portion of the file is displayed with 4 lines. The first line displays the bytes in hexadecimal. The second line displays the bytes as characters. The third line displays each group of two bytes interpreted as a signed 16-bit integer. The fourth line (added by human) presents the interpretation of the bytes in terms of netCDF components and values. <P>
<PRE>
4344 4601 0000 0000 0000 0000 0000 0000
C D F 001 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
17220 17921 00000 00000 00000 00000 00000 00000
[magic number ] [ 0 records ] [ 0 dimensions (ABSENT) ]
0000 0000 0000 0000 0000 0000 0000 0000
\0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
00000 00000 00000 00000 00000 00000 00000 00000
[ 0 global atts (ABSENT) ] [ 0 variables (ABSENT) ]
</PRE>
As a slightly less trivial example, consider the CDL<P>
<PRE>
netcdf tiny {
dimensions:
dim = 5;
variables:
short vx(dim);
data:
vx = 3, 1, 4, 1, 5 ;
}
</PRE>
which corresponds to a 92-byte netCDF file. The following is an edited dump of this file:<P>
<PRE>
4344 4601 0000 0000 0000 000a 0000 0001
C D F 001 \0 \0 \0 \0 \0 \0 \0 \n \0 \0 \0 001
17220 17921 00000 00000 00000 00010 00000 00001
[magic number ] [ 0 records ] [NC_DIMENSION ] [ 1 dimension ]
0000 0003 6469 6d00 0000 0005 0000 0000
\0 \0 \0 003 d i m \0 \0 \0 \0 005 \0 \0 \0 \0
00000 00003 25705 27904 00000 00005 00000 00000
[ 3 char name = "dim" ] [ size = 5 ] [ 0 global atts
0000 0000 0000 000b 0000 0001 0000 0002
\0 \0 \0 \0 \0 \0 \0 013 \0 \0 \0 001 \0 \0 \0 002
00000 00000 00000 00011 00000 00001 00000 00002
(ABSENT) ] [NC_VARIABLE ] [ 1 variable ] [ 2 char name =
7678 0000 0000 0001 0000 0000 0000 0000
v x \0 \0 \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 \0
30328 00000 00000 00001 00000 00000 00000 00000
"vx" ] [1 dimension ] [ with ID 0 ] [ 0 attributes
0000 0000 0000 0003 0000 000c 0000 0050
\0 \0 \0 \0 \0 \0 \0 003 \0 \0 \0 \f \0 \0 \0 P
00000 00000 00000 00003 00000 00012 00000 00080
(ABSENT) ] [type NC_SHORT] [size 12 bytes] [offset: 80]
0003 0001 0004 0001 0005 8001
\0 003 \0 001 \0 004 \0 001 \0 005 200 001
00003 00001 00004 00001 00005 -32767
[ 3] [ 1] [ 4] [ 1] [ 5] [fill ]
</PRE>
<!-- TOC --><DL>
<DT><A HREF="guidef-18.html#HEADING18-3"><B></B>The Format in Detail</A>
<DD>
<DT><A HREF="guidef-18.html#HEADING18-121"><B></B>Computing File Offsets</A>
<DD>
<DT><A HREF="guidef-18.html#HEADING18-153"><B></B>Examples</A>
<DD>
</DL>
<HR>
<ADDRESS>NetCDF User's Guide for Fortran - 5 JUN 1997</ADDRESS>
<A HREF="guidef-19.html">[Next] </A><A HREF="guidef-17.html">[Previous] </A><A HREF="guidef-1.html">[Top] </A><A HREF="guidef-3.html">[Contents] </A><A HREF="guidef-21.html">[Index] </A><A HREF="http://www.unidata.ucar.edu/packages/netcdf/">[netCDF Home Page]</A><A HREF="http://www.unidata.ucar.edu/">[Unidata Home Page]</A><P>
</BODY>
|
