1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Configuring the Logging Subsystem</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" />
<link rel="up" href="filemanagement.html" title="Chapter 5. Managing DB Files" />
<link rel="prev" href="logfileremoval.html" title="Removing Log Files" />
<link rel="next" href="wrapup.html" title="Chapter 6. Summary and Examples" />
</head>
<body>
<div xmlns="" class="navheader">
<div class="libver">
<p>Library Version 11.2.5.3</p>
</div>
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Configuring the Logging Subsystem</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a> </td>
<th width="60%" align="center">Chapter 5. Managing DB Files</th>
<td width="20%" align="right"> <a accesskey="n" href="wrapup.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="logconfig"></a>Configuring the Logging Subsystem</h2>
</div>
</div>
</div>
<div class="toc">
<dl>
<dt>
<span class="sect2">
<a href="logconfig.html#logfilesize">Setting the Log File Size</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a>
</span>
</dt>
</dl>
</div>
<p>
You can configure the following aspects of the logging
subsystem:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
Size of the log files.
</p>
</li>
<li>
<p>
Size of the logging subsystem's region.
See <a class="xref" href="logconfig.html#logregionsize" title="Configuring the Logging Region Size">Configuring the Logging Region Size</a>.
</p>
</li>
<li>
<p>
Maintain logs entirely in-memory.
See <a class="xref" href="logconfig.html#inmemorylogging" title="Configuring In-Memory Logging">Configuring In-Memory Logging</a>
for more information.
</p>
</li>
<li>
<p>
Size of the log buffer in memory.
See <a class="xref" href="logconfig.html#logbuffer" title="Setting the In-Memory Log Buffer Size">Setting the In-Memory Log Buffer Size</a>.
</p>
</li>
<li>
<p>
On-disk location of your log files. See
<a class="xref" href="enabletxn.html#splittingdata" title="Identifying Specific File Locations">Identifying Specific File Locations</a>.
</p>
</li>
</ul>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="logfilesize"></a>Setting the Log File Size</h3>
</div>
</div>
</div>
<p>
Whenever a pre-defined amount of data is written to a
log file (10 MB by default), DB stops using the
current log file and starts writing to a new file.
You can change the maximum amount of data contained in each
log file by using the
<span>
<code class="methodname">EnvironmentConfig.setMaxLogFileSize()</code>
method.
</span>
Note that this method can be used at any time
during an application's lifetime.
</p>
<p>
Setting the log file size to something larger than its
default value is largely a matter of
convenience and a reflection of the application's
preference in backup media and frequency. However, if
you set the log file size too low relative to your
application's traffic patterns, you can cause
yourself trouble.
</p>
<p>
From a performance perspective, setting the log file
size to a low value can cause your active transactions to
pause their writing activities more frequently than would
occur with larger log file sizes. Whenever a
transaction completes the log buffer is flushed to
disk. Normally other transactions can continue to
write to the log buffer while this flush is in
progress. However, when one log file is being closed
and another created, all transactions must cease
writing to the log buffer until the switch over is
completed.
</p>
<p>
Beyond performance concerns, using smaller log files
can cause you to use more physical files on disk.
As a result, your application could run out
of log sequence numbers, depending on how busy your
application is.
</p>
<p>
Every log file is identified with a 10 digit number.
Moreover, the maximum number of log files that your application
is allowed to create in its lifetime is 2,000,000,000.
</p>
<p>
For example, if your application performs 6,000 transactions per
second for 24 hours a day, and you are logging 500 bytes of
data per transaction into 10 MB log files, then you
will run out of log files in around 221 years:
</p>
<pre class="programlisting"> (10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 *60 * 24) = 221 </pre>
<p>
However, if you were writing 2000 bytes of data per
transaction, and using 1 MB log files, then the same
formula shows you running out of log files in 5 years time.
</p>
<p>
All of these time frames are quite long, to be sure,
but if you do run out of log files after, say, 5 years
of continuous operations, then you must reset your log
sequence numbers. To do so:
</p>
<div class="orderedlist">
<ol type="1">
<li>
<p>
Backup your databases as if to prepare for
catastrophic failure. See
<a class="xref" href="backuprestore.html" title="Backup Procedures">Backup Procedures</a>
for more information.
</p>
</li>
<li>
<p>
Reset the log file's sequence number using the
<span class="command"><strong>db_load</strong></span> utility's
<code class="literal">-r</code> option.
</p>
</li>
<li>
<p>
Remove all of the log files from your
environment. Note that this is the only
situation in which all of the log files are
removed from an environment; in all other
cases, at least a single log file is retained.
</p>
</li>
<li>
<p>
Restart your application.
</p>
</li>
</ol>
</div>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="logregionsize"></a>Configuring the Logging Region Size</h3>
</div>
</div>
</div>
<p>
The logging subsystem's default region size is 60 KB. The
logging region is used to store filenames, and so you may
need to increase its size if a large number of files (that
is, if you have a very large number of databases) will
be opened and registered with DB's log manager.
</p>
<p>
You can set the size of your logging region by using the
<span>
<code class="methodname">EnvironmentConfig.setLogRegionSize()</code>
</span>
method. Note that this method can only be called before the
first environment handle for your application is opened.
</p>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="inmemorylogging"></a>Configuring In-Memory Logging</h3>
</div>
</div>
</div>
<p>
It is possible to configure your logging subsystem such
that logs are maintained entirely in memory. When
you do this, you give up your transactional durability
guarantee. Without log files, you have no way to run
recovery so any system or software failures that you might
experience can corrupt your databases.
</p>
<p>
However, by giving up your durability guarantees, you can
greatly improve your application's throughput by avoiding
the disk I/O necessary to write logging information to
disk. In this case, you still retain your transactional
atomicity, consistency, and isolation guarantees.
</p>
<p>
To configure your logging subsystem to maintain your logs
entirely in-memory:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
Make sure your log buffer is capable of holding all
log information that can accumulate during the longest
running transaction. See <a class="xref" href="logconfig.html#logbuffer" title="Setting the In-Memory Log Buffer Size">Setting the In-Memory Log Buffer Size</a> for details.
</p>
</li>
<li>
<p>
Do not run normal recovery when you open your environment. In this configuration, there are no
log files available against which you can run recovery. As a result, if you specify recovery
when you open your environment, it is ignored.
</p>
</li>
<li>
<p>
Specify
<span>
<code class="literal">true</code>
to the
<code class="methodname">EnvironmentConfig.setLogInMemory()</code>
</span>
method. Note that you must specify this before
your application opens its first environment
handle.
</p>
</li>
</ul>
</div>
<p>
For example:
</p>
<pre class="programlisting">package db.txn;
import com.sleepycat.db.Database;
import com.sleepycat.db.DatabaseConfig;
import com.sleepycat.db.DatabaseEntry;
import com.sleepycat.db.DatabaseException;
import com.sleepycat.db.Environment;
import com.sleepycat.db.EnvironmentConfig;
import java.io.File;
...
Database myDatabase = null;
Environment myEnv = null;
try {
EnvironmentConfig myEnvConfig = new EnvironmentConfig();
myEnvConfig.setInitializeCache(true);
myEnvConfig.setInitializeLocking(true);
myEnvConfig.setInitializeLogging(true);
myEnvConfig.setTransactional(true);
// Specify in-memory logging
myEnvConfig.setLogInMemory(true);
// Specify the in-memory log buffer size.
myEnvConfig.setLogBufferSize(10 * 1024 * 1024);
myEnv = new Environment(new File("/my/env/home"),
myEnvConfig);
// From here, you open databases, create transactions and
// perform database operations exactly as you would if you
// were logging to disk. This part is omitted for brevity. </pre>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="logbuffer"></a>Setting the In-Memory Log Buffer Size</h3>
</div>
</div>
</div>
<p>
When your application is configured for on-disk logging
(the default behavior for transactional applications), log
information is stored in-memory until the storage space
fills up, or a transaction commit forces the log
information to be flushed to disk.
</p>
<p>
It is possible to increase the amount of memory available
to your file log buffer. Doing so improves throughput for
long-running transactions, or for transactions that produce
a large amount of data.
</p>
<p>
When you have your logging subsystem configured to maintain
your log entirely in memory (see
<a class="xref" href="logconfig.html#inmemorylogging" title="Configuring In-Memory Logging">Configuring In-Memory Logging</a>), it is very important
to configure your log buffer size because the log buffer
must be capable of holding all log information that can
accumulate during the longest running transaction.
You must make sure that the in-memory log buffer size is
large enough that no transaction will ever span the entire
buffer. You must also avoid a state where the in-memory
buffer is full and no space can be freed because a
transaction that started the first log "file" is still
active.
</p>
<p>
When your logging subsystem is configured for on-disk
logging, the default log buffer space is 32 KB. When
in-memory logging is configured, the default log buffer
space is 1 MB.
</p>
<p>
You can increase your log buffer space using the
<span>
<code class="methodname">EnvironmentConfig.setLogBufferSize()</code>
</span>
method. Note that this method can only be called before the
first environment handle for your application is opened.
</p>
</div>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="filemanagement.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="wrapup.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Removing Log Files </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Chapter 6. Summary and Examples</td>
</tr>
</table>
</div>
</body>
</html>
|
