1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Database and log file archival</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="up" href="transapp.html" title="Chapter 11. Berkeley DB Transactional Data Store Applications" />
<link rel="prev" href="transapp_checkpoint.html" title="Checkpoints" />
<link rel="next" href="transapp_logfile.html" title="Log file removal" />
</head>
<body>
<div class="navheader">
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Database and log file archival</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="transapp_checkpoint.html">Prev</a> </td>
<th width="60%" align="center">Chapter 11.
Berkeley DB Transactional Data Store Applications
</th>
<td width="20%" align="right"> <a accesskey="n" href="transapp_logfile.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="transapp_archival"></a>Database and log file archival</h2>
</div>
</div>
</div>
<p>The third component of the administrative infrastructure, archival for
catastrophic recovery, concerns the recoverability of the database in
the face of catastrophic failure. Recovery after catastrophic failure
is intended to minimize data loss when physical hardware has been
destroyed -- for example, loss of a disk that contains databases or log
files. Although the application may still experience data loss in this
case, it is possible to minimize it.</p>
<p>First, you may want to periodically create snapshots (that is, backups)
of your databases to make it possible to recover from catastrophic
failure. These snapshots are either a standard backup, which creates a
consistent picture of the databases as of a single instant in time; or
an on-line backup (also known as a <span class="emphasis"><em>hot</em></span> backup), which creates
a consistent picture of the databases as of an unspecified instant
during the period of time when the snapshot was made. The advantage of
a hot backup is that applications may continue to read and write the
databases while the snapshot is being taken. The disadvantage of a hot
backup is that more information must be archived, and recovery based on
a hot backup is to an unspecified time between the start of the backup
and when the backup is completed.</p>
<p>Second, after taking a snapshot, you should periodically archive the
log files being created in the environment. It is often helpful to
think of database archival in terms of full and incremental filesystem
backups. A snapshot is a full backup, whereas the periodic archival of
the current log files is an incremental backup. For example, it might
be reasonable to take a full snapshot of a database environment weekly
or monthly, and archive additional log files daily. Using both the
snapshot and the log files, a catastrophic crash at any time can be
recovered to the time of the most recent log archival; a time long after
the original snapshot.</p>
<p>To create a standard backup of your database that can be used to recover
from catastrophic failure, take the following steps:</p>
<div class="orderedlist">
<ol type="1">
<li>Commit or abort all ongoing transactions.</li>
<li>Stop writing your databases until the backup has completed. Read-only
operations are permitted, but no write operations and no filesystem
operations may be performed (for example, the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> and
<a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> methods may not be called).</li>
<li>Force an environment checkpoint (see the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint utility</a> for more
information).</li>
<li>
<p>
Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> with option <span class="bold"><strong>-s</strong></span> to identify all the database data
files, and copy them to a backup device such as CD-ROM, alternate
disk, or tape.
</p>
<p>
If the database files are stored in a separate directory from the
other Berkeley DB files, it may be simpler to archive the directory
itself instead of the individual files (see <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a> for
additional information). <span class="bold"><strong>Note: if any of the
database files did not have an open <a href="../api_reference/C/db.html" class="olink">DB</a> handle during
the lifetime of the current log files, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> will
not list them in its output!</strong></span> This is another reason
it may be simpler to use a separate database file directory and
archive the entire directory instead of archiving only the files
listed by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a>.
</p>
</li>
<li>
Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> with option <span class="bold"><strong>-l</strong></span> to
identify all the log files, and copy the last one (that is, the one
with the highest number) to a backup device such as CD-ROM, alternate
disk, or tape.
</li>
</ol>
</div>
<p>
To create a <span class="emphasis"><em>hot</em></span> backup of your
database that can be used to recover from catastrophic failure, take
the following steps:
</p>
<div class="orderedlist">
<ol type="1">
<li>
Archive your databases, as described in the previous step #4. You
do not have to halt ongoing transactions or force a checkpoint. As
this is a hot backup, and the databases may be modified during the
copy, the utility you use to copy the databases must read database
pages atomically (as described by <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a>).
</li>
<li>
<p>
Archive <span class="bold"><strong>all</strong></span> of the log files.
The order of these two operations is required, and the database
files must be archived <span class="bold"><strong>before</strong></span>
the log files. This means that if the database files and log
files are in the same directory, you cannot simply archive the
directory; you must make sure that the correct order of
archival is maintained.
</p>
<p>
To archive your log files, run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> using the
<span class="bold"><strong>-l</strong></span> option to identify all the
database log files, and copy them to your backup media. If the
database log files are stored in a separate directory from the
other database files, it may be simpler to archive the
directory itself instead of the individual files (see the
<a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method for more information).
</p>
</li>
</ol>
</div>
<p>To minimize the archival space needed for log files when doing a hot
backup, run db_archive to identify those log files which are not in use.
Log files which are not in use do not need to be included when creating
a hot backup, and you can discard them or move them aside for use with
previous backups (whichever is appropriate), before beginning the hot
backup.</p>
<p>After completing one of these two sets of steps, the database
environment can be recovered from catastrophic failure (see
<a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a> for more information).</p>
<p>For an example of a hot backup implementation in the Berkeley DB distribution,
see the source code for the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup utility</a>.</p>
<p>To update either a hot or cold backup so that recovery from catastrophic
failure is possible to a new point in time, repeat step #2 under the
hot backup instructions and archive <span class="bold"><strong>all</strong></span> of the log files in the
database environment. Each time both the database and log files are
copied to backup media, you may discard all previous database snapshots
and saved log files. Archiving additional log files does not allow you
to discard either previous database snapshots or log files. Generally,
updating a backup must be integrated with the application's log file
removal procedures.</p>
<p>The time to restore from catastrophic failure is a function of the
number of log records that have been written since the snapshot was
originally created. Perhaps more importantly, the more separate pieces
of backup media you use, the more likely it is that you will have a
problem reading from one of them. For these reasons, it is often best
to make snapshots on a regular basis.</p>
<p>
<span class="bold">
<strong>Obviously, the reliability of your archive media will affect the safety
of your data. For archival safety, ensure that you have multiple copies
of your database backups, verify that your archival media is error-free
and readable, and that copies of your backups are stored offsite!</strong>
</span>
</p>
<p>The functionality provided by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive utility</a> is also
available directly from the Berkeley DB library. The following code fragment
prints out a list of log and database files that need to be archived:</p>
<pre class="programlisting">void
log_archlist(DB_ENV *dbenv)
{
int ret;
char **begin, **list;
/* Get the list of database files. */
if ((ret = dbenv->log_archive(dbenv,
&list, DB_ARCH_ABS | DB_ARCH_DATA)) != 0) {
dbenv->err(dbenv, ret, "DB_ENV->log_archive: DB_ARCH_DATA");
exit (1);
}
if (list != NULL) {
for (begin = list; *list != NULL; ++list)
printf("database file: %s\n", *list);
free (begin);
}
/* Get the list of log files. */
if ((ret = dbenv->log_archive(dbenv,
&list, DB_ARCH_ABS | DB_ARCH_LOG)) != 0) {
dbenv->err(dbenv, ret, "DB_ENV->log_archive: DB_ARCH_LOG");
exit (1);
}
if (list != NULL) {
for (begin = list; *list != NULL; ++list)
printf("log file: %s\n", *list);
free (begin);
}
}</pre>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="transapp_checkpoint.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="transapp.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="transapp_logfile.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Checkpoints </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Log file removal</td>
</tr>
</table>
</div>
</body>
</html>
|
