1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
|
<!doctype HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!--
(C) Copyright 2002-4 Robert Ramey - http://www.rrsd.com .
Use, modification and distribution is subject to the Boost Software
License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
http://www.boost.org/LICENSE_1_0.txt)
-->
<head>
<meta http-equiv="Content-Type" content="text/html; UTF-8">
<link rel="stylesheet" type="text/css" href="../../../boost.css">
<link rel="stylesheet" type="text/css" href="style.css">
<title>Serialization - More on Archives</title>
</head>
<body link="#0000ff" vlink="#800080">
<table border="0" cellpadding="7" cellspacing="0" width="100%" summary="header">
<tr>
<td valign="top" width="300">
<h3><a href="../../../index.htm"><img height="86" width="277" alt="C++ Boost" src="../../../boost.png" border="0"></a></h3>
</td>
<td valign="top">
<h1 align="center">Serialization</h1>
<h2 align="center">Archive Class Reference</h2>
</td>
</tr>
</table>
<hr>
<dl class="page-index">
<dt><a href="#implementation">Implementation</a>
<dt><a href="#usage">Usage</a>
<dt><a href="#testing">Testing</a>
<dt><a href="#polymorphic">Polymorphic Archives</a>
</dl>
<h3><a name="implementation">Implementation</a></h3>
The <a href="archives.html"><strong>Archive</strong></a> concept specifies the functions that a
class must implement to in order to be used to serialize
<a href="serialization.html"><strong>Serializable</strong></a> types.
The library implements a family of archives appropriate for different purposes.
This section describes how they have been implemented and how one can implement his own archive class.
Our discussion will focus on archives used for loading as the hierarchy is exactly analogous
for archives used for saving data.
<p>
Our archives have been factored in to a tree of classes in order to minimize
repetition of code. This is shown in the accompanying
<a target="class_diagram" href="class_diagram.html">class diagram</a>.
All input archives should be derived from the following template:
<pre><code>
template<class Archive>
detail::common_iarchive;
</code></pre>
<h4>Minimum Requirments</h4>
An instance of the this template handles all the "bookkeeping" associated
with serialization. In order to be a functional only the following additional
functions <strong>must</strong> be defined:
<dl>
<dt><h4><code>void load(T &t);</code></h4></dt>
<dd>
This function must be implemented for all primitive data types. This can be
accomplished through the use of a member template or explict declarations
for all prmititive types.
</dd>
<dt><h4><code>void load_binary(void *address, std::size_t size);</code></h4></dt>
<dd>
This function should <code style="white-space: normal">size</code> bytes from the archive, and
copy them in to memory starting at address <code style="white-space: normal">address</code>.
</dd>
<dt><h4><code>friend class boost::archive::load_access;</code></h4></dt>
<dd>
In addition, such a class <strong>must</strong> provide a the following
friend declaration grant access to the core serialization library to the functions
of this class.
</dd>
</dl>
So, the most trivial implementation of an input archive would look like this:
<pre><code>
<a href="../../../boost/archive/detail/common_iarchive.hpp" target="common_iarchive_hpp">
#include <boost/archive/detail/common_iarchive.hpp>
</a>
/////////////////////////////////////////////////////////////////////////
// class trivial_iarchive - read serialized objects from a input text stream
class trivial_iarchive :
public boost::archive::detail::common_iarchive<trivial_iarchive>
{
// permit serialization system priviledged access to permit
// implementation of inline templates for maximum speed.
friend class boost::archive::load_access;
// member template for loading primitive types.
// Override for any types/templates that special treatment
template<class T>
void load(T & t);
public:
//////////////////////////////////////////////////////////
// public interface used by programs that use the
// serialization library
// archives are expected to support this function
void load_binary(void *address, std::size_t count);
};
</code></pre>
The simplest possible output archive class is exactly analogous to the above.
In the following discussion, only input archives will be addressed.
Output archives are exactly symmetrical to input archives.
<p>
Given a suitable definitions of <code style="white-space: normal">load</code>
and <code style="white-space: normal">load_binary</code>,
any program using serialization with a conforming C++ compiler should compile
and run with this archive class.
<h4>Optional Overrides</h4>
The <code style="white-space: normal">detail::common_iarchive</code> class contains
a number of functions that are used by various parts of the serialization library
to help render the archive in a particular form.
<dl>
<dt><h4><code>void load_start()</code></h4></dt>
<dd>
<strong>Default</strong>:Does nothing.<br>
<strong>Purpose</strong>:To inject/retrieve an object name into the archive. Used
by XML archive to inject "<name " before data.
</dd>
<p>
<dt><h4><code>void load_end()</code></h4></dt>
<dd>
<strong>Default</strong>:Does nothing.<br>
<strong>Purpose</strong>:To inject/retrieve an object name into the archive. Used
by XML archive to inject "</name>" after data.
<dd>
</dd>
<p>
<dt><h4><code>void end_preamble()</code></h4></dt>
<dd>
<strong>Default</strong>:Does nothing.<br>
<strong>Purpose</strong>:Called <strong>each time</strong> user data is saved.
Its not called when archive book keeping data is saved. This is used by XML archives
to determine when to inject a ">" character at end of XML header. XML output archives
keep their own internal flag indicating that data being written is header data. This
internal flag is reset when an object start tag is written. When
<code style="white-space: normal">void end_preamble()</code> is invoked and this internal flag is set
a ">" character is appended to the output and the internal flag is reset. The default
implementation for <code style="white-space: normal">void end_preamble()</code> is a no-op there by permitting it
to be optimised away for archive classes that don't use it.
</dd>
<p>
<dt><h4><code>
template<class T>
void load_override(T & t, int);
</code></h4></dt>
<dd>
<strong>Default</strong>:Invokes <code style="white-space: normal">archive::load(Archive & ar, t)</code><br>
This is the main entry into the serialization library.<br>
<strong>Purpose</strong>:This can be overridden in cases where the data is to be written
to the archive in some special way. For example, XML archives implement special handling for
name-value pairs by overriding this function template for name-value pairs.
This replaces the default name-value pair handling, which is just to throw away the name,
with one appropriate for XML which writes out the start of an XML tag with the correct object name.
<p>
The second argument must be part of the function signature even this it is not used.
Its purpose is to be sure that code is portable to compilers which fail to correctly
implement partial function template ordering. For more information see
<a href="implementation.html#functiontemplateordering">this</a>.
</dd>
</dl>
<h4>Types used by the serialization library</h4>
The serialization library injects bookkeeping data into the serialization archive.
This data includes things like object ids, version numbers, class names etc. Each
of these objects is included in a wrapper so that the archive class can override the
implementation of <code style="white-space: normal">void load_override(T & t, int);</code>.
For example, in the XML archive, the override for this type renders an object_id equal to 23 as
"object_id=_23". The following table lists the types defined in the
<code style="white-space: normal">boost::archive namespace</code>
used internally by the serialization library:
<p>
<table border>
<tr><th align=left>type</th><th align=left><code style="white-space: normal">default<br>serialed as</code></th>
<tr><td><code style="white-space: normal">version_type</code></td><td><code style="white-space: normal">unsigned int</code></td>
<tr><td><code style="white-space: normal">object_id_type</code></td><td><code style="white-space: normal">unsigned int</code></td>
<tr><td><code style="white-space: normal">object_id_reference_type</code></td><td><code style="white-space: normal">unsigned int</code></td>
<tr><td><code style="white-space: normal">class_id_type</code></td><td><code style="white-space: normal">int</code></td>
<tr><td><code style="white-space: normal">class_id_optional_type</code></td><td><code style="white-space: normal">nothing</code></td>
<tr><td><code style="white-space: normal">class_id_reference_type</code></td><td><code style="white-space: normal">int</code></td>
<tr><td><code style="white-space: normal">tracking_type</code></td><td><code style="white-space: normal">bool</code></td>
<tr><td><code style="white-space: normal">classname_type</code></td><td><code style="white-space: normal">string</code></td>
</table>
<p>
All of these are associated with a default serialization defined in terms of primitive types
so it isn't a requirement to define <code style="white-space: normal">load_override</code>
for these types.
<p>
These are defined in
<a href="../../../boost/archive/basic_archive.hpp" target="basic_archive_hpp"><code style="white-space: normal">basic_archive.hpp</code></a>.
All of these types have been assigned an
<a target="detail" href="traits.html#level">implementation level</a> of
<code style="white-space: normal">primitive</code> and are convertible to types such as int, unsigned int, etc.
so that they have default implementations. This is illustrated by
<a href="../../../boost/archive/basic_text_iarchive.hpp" target="basic_text_iarchive_hpp"><code style="white-space: normal">basic_text_iarchive.hpp</code></a>.
which relies upon the default. However, in some cases, overrides will have to be
explicitly provided for these types. For an example see
<a href="../../../boost/archive/basic_xml_iarchive.hpp" target="basic_xml_iarchive_hpp"><code style="white-space: normal">basic_xml_iarchive.hpp</code></a>.
<p>
In real practice, we probably won't be quite done.
One or more of the following issues may need to be addressed:
<ul>
<li>Many compilers fail to implement correct partial ordering of
function templates. The archives included with this library work around
this using argument overloading. This technique is described in
<a target="detail" href="implementation.html#functiontemplateordering">
another section of this manual</a>
<li>Even if we are using a conforming compiler, we might want our new archive class
to be portable to non-conforming compilers.
<li>Our archive format might require extra information inserted into it. For
example, XML archives need <name ... >...</name> surrounding
all data objects.
<li>Addressing any of the above may generate more issues to be addressed.
<li>The archives included with library are all templates which use a
<code style="white-space: normal">stream</code> as a template parameter rather than simple classes.
Combined with the above, even more issues arise with non-conforming compilers.
</ul>
The attached <a target="class_diagram" href="class_diagram.html">class diagram</a>
shows the relationships between classes used to implement the serialization library.
<p>
A close examination of the archives included with the library illustrate
what it takes to make a portable archive that covers all data types.
<h3><a name="usage">Usage</a></h3>
The newly created archive will usually be stored in its own header module. All
that is necessary is to include the header and construct an instance of the new archive.
EXCEPT for one special case.
<ul>
<li>Instances of a derived class are serialized through a base class pointer.
<li>Such instances are not "registered" neither implicitly nor explicitly. That
is, the macro <code style="white-space: normal">BOOT_CLASS_EXPORT</code> is used to instantiate the serialization
code for the included archives.
</ul>
The problem here is that BOOT_CLASS_EXPORT only generates code for those archives
included with the library - not those added subsequently. To generate code for
newly created archive classes, the following should be used.
<pre><code>
#define BOOST_ARCHIVE_CUSTOM_OARCHIVE_TYPES trivial_oarchive
#define BOOST_ARCHIVE_CUSTOM_IARCHIVE_TYPES trivial_iarchive
</code></pre>
before <code style="white-space: normal">BOOST_CLASS_EXPORT</code> is invoked for any serializable class.
Failure to do this will not inhibit the program from compiling, linking
and executing properly - except in one case. If an instance of a derived
class is serialized through a pointer to its base class, the program
will throw an
<a href="exceptions.html#unregistered_class"><code style="white-space: normal">unregistered_class</code></a>
exception.
<p>
Only one of the above statements is permitted, However, any number of new archive
classes can be specified as list separated by commas.
<h4><a name="testing">Testing</h4>
Exhaustive testing of the library requires testing the different aspects of object
serialization with each archive. There are 36 different tests that can run with any archive. There are
5 "standard archives" included with the system. (3 in systems which don't support wide
charactor i/o).
<p>
In addition, there are 22 other tests which aren't related to any particular archive
class.
<p>
The default <code style="white-space: normal">bjam</code> testing setup will run all
the above described tests. This will result in as many as 39 archive tests * 5
standard archives + 25 general tests = 220 tests. Note that a complete test of the
library would include DLL vs static library, release vs debug so the actual total
would be closer to 880 tests.
<p>
For each archive there is a header file in the test directory similar to the one below.
The name of this archive is passed to the test program by setting the
environmental variable <code style="white-space: normal">BOOST_ARCHIVE_TEST</code>
to the name of the header. Here is the header file
<code style="white-space: normal">test_archive.hpp</code> . Test header files for
other archives are similar.
<pre><code>
// text_archive test header
// include output archive header
#include <boost/archive/text_oarchive.hpp>
// set name of test output archive
typedef boost::archive::text_oarchive test_oarchive;
// set name of test output stream
typedef std::ofstream test_ostream;
// repeat the above for input archive
#include <boost/archive/text_iarchive.hpp>
typedef boost::archive::text_iarchive test_iarchive;
typedef std::ifstream test_istream;
// define open mode for streams
// binary archives should use std::ios_base::binary
#define TEST_STREAM_FLAGS (std::ios_base::openmode)0
</code></pre>
To test a new archive, for example, portable binary archives, make a
header file <code style="white-space: normal">portable_binary_archive.hpp</code>
and invoke <code style="white-space: normal">bjam</code> with
<pre><code>
-sBOOST_ARCHIVE_LIST=portable_binary_archive.hpp
</code></pre>
This process in encapsulated in the shell script
<code style="white-space: normal">run_archive_test</code> whose command line is
<pre><code>
run_archive_test <test header file> <toolset> [<boost root>] [<target directory>]
</code></pre>
<h3><a name="polymorphic">Polymorphic Archives</a></h3>
<h4>Motivation</h4>
All archives described so far are implemented as templates. Code to save and load
data to archives is regenerated for each combination of archive class and data type.
Under these cirumstances, a good optimizing compiler that can expand
<code>inline</code> functions to enough depth will generate fast code.
However:
<ul>
<li>Much inline code may be replicated.
<li>If there are several archive classes, code will be regenerated for each archive class.
<li>If serialization code is placed in a library, that library must be rebuilt
each time a new archive class is created.
<li>If serialization code is placed in a DLL,
<ul>
<li>The DLL will contain versions of code for each known archive type.
This would result in loading of DLLs which contain
much code that is not used - basically defeating one of the main motivations
for choosing to use a DLL in the first place.
<li>If a new archive is created and an application shipped, all DLLs have to be
rebuilt, and reshipped along with the application which uses the new archive. Thus
the other main motivation for using a DLL is defeated.
</ul>
</ul>
<h4>Implementation</h4>
The solution is the the pair <code>polymorphic_oarchive</code>
and <code>polymorphic_iarchive</code>. They present a common interface of virtual
functions - no templates - that is equivalent to the standard templated one.
This is shown in the accompanying
<a target="class_diagram" href="class_diagram.html">class diagram</a>
<p>
The accompanying demo program in files
<a target=demo_polymorphic_cp href="../example/demo_polymorphic.cpp"><code style="white-space: normal">demo_polymorphic.cpp</code></a>,
<a target=demo_polymorphic_A_hpp href="../example/demo_polymorphic_A.hpp"><code style="white-space: normal">demo_polymorphic_A.hpp</code></a>, and
<a target=demo_polymorphic_A_cpp href="../example/demo_polymorphic_A.cpp"><code style="white-space: normal">demo_polymorphic_A</code></a>
show how polymorphic archives are to be used. Note the following:
<ul>
<li><a target=demo_polymorphic_A_hpp href="../example/demo_polymorphic_A.hpp"><code style="white-space: normal">demo_polymorphic_A.hpp</code></a> and
<a target=demo_polymorphic_A_cpp href="../example/demo_polymorphic_A.cpp"><code style="white-space: normal">demo_polymorphic_A.cpp</code></a>
contain no templates and no reference to any specific archive implementation. That is, they will
only have to be compiled once for all archive implementations. The even applies to archives classes
created in the future.
<li>The main program <a target=demo_polymorphic_cp href="../example/demo_polymorphic.cpp"><code style="white-space: normal">demo_polymorphic.cpp</code></a>
specifies a specific archive implementation.
</ul>
As can be seen in the
<a target="class_diagram" href="class_diagram.html">class diagram</a>
and the header files, this implementation is just a composition of the polymorphic
interface and the standard template driven implementation. This composition is
accomplished by the templates
<a target=polymorphic_iarchive_dispatch_hpp href="../../../boost/archive/detail/polymorphic_iarchive_dispatch.hpp"><code style="white-space: normal">polymorphic_iarchive_dispatch.hpp</code></a>
and
<a target=polymorphic_oarchive_dispatch_hpp href="../../../boost/archive/detail/polymorphic_oarchive_dispatch.hpp"><code style="white-space: normal">polymorphic_oarchive_dispatch.hpp</code></a>.
As these contain no code specific to the particular implementation archive, they can be used to create
a polymorphic archive implementation from any functioning templated archive implementation.
<p>
As a convenience, small header files have been included which contain
<code style="white-space: normal">typedef</code> for polymorphic implementation for each corresponding
templated one. For example, the headers
<a target=polymorphic_text_iarchive_hpp href="../../../boost/archive/polymorphic_text_iarchive.hpp"><code style="white-space: normal">polymorphic_text_iarchive.hpp</code></a>
and
<a target=polymorphic_text_oarchive_hpp href="../../../boost/archive/polymorphic_text_oarchive.hpp"><code style="white-space: normal">polymorphic_text_oarchive.hpp</code></a>.
contain the <code style="white-space: normal">typedef</code> for the polymorphic implementation
of the standard text archive classes
<a target=text_iarchive_hpp href="../../../boost/archive/text_iarchive.hpp"><code style="white-space: normal">text_iarchive.hpp</code></a>
and
<a target=text_oarchive_hpp href="../../../boost/archive/text_oarchive.hpp"><code style="white-space: normal">text_oarchive.hpp</code></a>
respectively. All included polymorphic archives use the same naming scheme.
<h4>Usage</h4>
Polymorphic archives address the issues raised above regarding templated implementation.
That is, there is no replicated code, and no recompilation for new archives. This will
result in smaller executables for program which use more than one type of archive, and
smaller DLLS. There is a
penalty for calling archive functions through a virtual function dispatch table and there
is no possibility for a compiler to <code style="white-space: normal">inline</code> archive functions. This will result
in a detectable degradation in performance for saving and loading archives.
<p>
The main utility of polymorphic archives will be to permit the buiding of class DLLs that will
include serialization code for all present and future archives with no redundant code.
<hr>
<p><i>© Copyright <a href="http://www.rrsd.com">Robert Ramey</a> 2002-2004.
Distributed under the Boost Software License, Version 1.0. (See
accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
</i></p>
</body>
</html>
|
