1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Memory-only or Flash configurations</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="up" href="program.html" title="Chapter 14. Programmer Notes" />
<link rel="prev" href="program_namespace.html" title="Name spaces" />
<link rel="next" href="program_cache.html" title="Disk drive caches" />
</head>
<body>
<div class="navheader">
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Memory-only or Flash configurations</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="program_namespace.html">Prev</a> </td>
<th width="60%" align="center">Chapter 14.
Programmer Notes
</th>
<td width="20%" align="right"> <a accesskey="n" href="program_cache.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="program_ram"></a>Memory-only or Flash configurations</h2>
</div>
</div>
</div>
<p>Berkeley DB supports a variety of memory-based configurations for systems
where filesystem space is either limited in availability or entirely
replaced by some combination of memory and Flash. In addition, Berkeley DB
can be configured to minimize writes to the filesystem when the
filesystem is backed by Flash memory.</p>
<p>There are four parts of the Berkeley DB database environment normally written
to the filesystem: the database environment region files, the database
files, the database environment log files and the replication internal
files. Each of these items can
be configured to live in memory rather than in the filesystem:</p>
<div class="variablelist">
<dl>
<dt>
<span class="term">The database environment region files:</span>
</dt>
<dd>
<p>
Each of the Berkeley DB subsystems in a database environment is
described by one or more regions, or chunks of memory. The regions
contain all of the per-process and per-thread shared information
(including mutexes), that comprise a Berkeley DB environment. By
default, these regions are backed by the filesystem. In situations
where filesystem backed regions aren't optimal, applications can
create memory-only database environments in two different types of
memory: either in the application's heap memory or in system shared
memory.
</p>
<p>
To create the database environment in heap memory, specify the
<a href="../api_reference/C/envopen.html#open_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. Note that database
environments created in heap memory are only accessible to the
threads of a single process, however.
</p>
<p>
To create the database environment in system shared memory, specify
the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. Database
environments created in system memory are accessible to multiple
processes, but note that database environments created in system
shared memory do create a small (roughly 8 byte) file in the
filesystem, read by the processes to identify which system shared
memory segments to use.
</p>
<p>
For more information, see <a class="xref" href="env_region.html" title="Shared memory regions">Shared memory regions</a>.
</p>
</dd>
<dt>
<span class="term">The database files:</span>
</dt>
<dd>
<p>
By default, databases are periodically flushed from the
Berkeley DB memory cache to backing physical files in the
filesystem. To keep databases from being written to backing
physical files, pass the <a href="../api_reference/C/mempset_flags.html#mpool_set_flags_DB_MPOOL_NOFILE" class="olink">DB_MPOOL_NOFILE</a> flag to the
<a href="../api_reference/C/mempset_flags.html" class="olink">DB_MPOOLFILE->set_flags()</a> method. This flag
implies the application's databases must fit entirely in the
Berkeley DB cache, of course. To avoid a database file growing
to consume the entire cache, applications can limit the size of
individual databases in the cache by calling the
<a href="../api_reference/C/mempset_maxsize.html" class="olink">DB_MPOOLFILE->set_maxsize()</a> method.
</p>
</dd>
<dt>
<span class="term">The database environment log files:</span>
</dt>
<dd>
<p>
If a database environment is not intended to be transactionally
recoverable after application or system failure (that is, if it
will not exhibit the transactional attribute of "durability"),
applications should not configure the database environment for
logging or transactions, in which case no log files will be
created. If the database environment is intended to be
durable, log files must either be written to stable storage and
recovered after application or system failure, or they must be
replicated to other systems.
</p>
<p>
In applications running on systems without any form of stable
storage, durability must be accomplished through replication.
In this case, database environments should be configured to
maintain database logs in memory, rather than in the
filesystem, by specifying the <a href="../api_reference/C/envlog_get_config.html#log_set_config_DB_LOG_IN_MEMORY" class="olink">DB_LOG_IN_MEMORY</a> flag to the
<a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> method.
</p>
</dd>
<dt>
<span class="term">The replication internal files:</span>
</dt>
<dd>
<p>
By default, Berkeley DB replication stores a small amount of
internal data in the filesystem. To store this replication
internal data in memory only and not in the filesystem, specify
the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag to the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method before
opening the database environment.
</p>
</dd>
</dl>
</div>
<p>In systems where the filesystem is backed by Flash memory, the number
of times the Flash memory is written may be a concern. Each of the
four parts of the Berkeley DB database environment normally written to the
filesystem can be configured to minimize the number of times the
filesystem is written:</p>
<div class="variablelist">
<dl>
<dt>
<span class="term">The database environment region files:</span>
</dt>
<dd>
<p>
On a Flash-based filesystem, the database environment should be placed
in heap or system memory, as described previously.
</p>
</dd>
<dt>
<span class="term">The database files:</span>
</dt>
<dd>
<p>
The Berkeley DB library maintains a cache of database pages.
The database pages are only written to backing physical files
when the application checkpoints the database environment with
the <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> method, when database handles are closed
with the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method, or when the application explicitly
flushes the cache with the <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> methods.
</p>
<p>
To avoid unnecessary writes of Flash memory due to checkpoints,
applications should decrease the frequency of their
checkpoints. This is especially important in applications
which repeatedly modify a specific database page, as repeatedly
writing a database page to the backing physical file will
repeatedly update the same blocks of the filesystem.
</p>
<p>
To avoid unnecessary writes of the filesystem due to closing a
database handle, applications should specify the <a href="../api_reference/C/dbclose.html#dbclose_DB_NOSYNC" class="olink">DB_NOSYNC</a> flag
to the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method.
</p>
<p>
To avoid unnecessary writes of the filesystem due to flushing
the cache, applications should not explicitly flush the cache
under normal conditions – flushing the cache is rarely if
ever needed in a normally-running application.
</p>
</dd>
<dt>
<span class="term">The database environment log files:</span>
</dt>
<dd>
<p>
The Berkeley DB log files do not repeatedly overwrite the same
blocks of the filesystem as the Berkeley DB log files are not
implemented as a circular buffer and log files are not re-used.
For this reason, the Berkeley DB log files should not cause any
difficulties for Flash memory configurations.
</p>
<p>
However, as Berkeley DB does not write log records in
filesystem block sized pieces, it is probable that sequential
transaction commits (each of which flush the log file to the
backing filesystem), will write a block of Flash memory twice,
as the last log record from the first commit will write the
same block of Flash memory as the first log record from the
second commit. Applications not requiring absolute durability
should specify the <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> or
<a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags to the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a>
method to avoid this overwrite of a block of Flash memory.
</p>
</dd>
<dt>
<span class="term">The replication internal files:</span>
</dt>
<dd>
<p>
On a Flash-based filesystem, the replication internal data
should be stored in memory only, as described previously.
</p>
</dd>
</dl>
</div>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="program_namespace.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="program.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="program_cache.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Name spaces </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Disk drive caches</td>
</tr>
</table>
</div>
</body>
</html>
|
