1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Chapter 17. The Memory Pool Subsystem</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="up" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="prev" href="log_limits.html" title="Log file limits" />
<link rel="next" href="mp_config.html" title="Configuring the memory pool" />
</head>
<body>
<div class="navheader">
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Chapter 17.
The Memory Pool Subsystem
</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="log_limits.html">Prev</a> </td>
<th width="60%" align="center"> </th>
<td width="20%" align="right"> <a accesskey="n" href="mp_config.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="chapter" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title"><a id="mp"></a>Chapter 17.
The Memory Pool Subsystem
</h2>
</div>
</div>
</div>
<div class="toc">
<p>
<b>Table of Contents</b>
</p>
<dl>
<dt>
<span class="sect1">
<a href="mp.html#mp_intro">Introduction to the memory pool subsystem</a>
</span>
</dt>
<dt>
<span class="sect1">
<a href="mp_config.html">Configuring the memory pool</a>
</span>
</dt>
</dl>
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="mp_intro"></a>Introduction to the memory pool subsystem</h2>
</div>
</div>
</div>
<p>The Memory Pool subsystem is the general-purpose shared memory buffer
pool used by Berkeley DB. This module is useful outside of the Berkeley DB package
for processes that require page-oriented, shared and cached file
access. (However, such "use outside of Berkeley DB" is not supported
in replicated environments.)</p>
<p>A <span class="emphasis"><em>memory pool</em></span> is a memory cache shared among any number of
threads of control. The <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> flag to the
<a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method opens and optionally creates a memory pool. When
that pool is no longer in use, it should be closed using the
<a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> method.</p>
<p>The <a href="../api_reference/C/mempfcreate.html" class="olink">DB_ENV->memp_fcreate()</a> method returns a <a href="../api_reference/C/memp.html" class="olink">DB_MPOOLFILE</a> handle on an
underlying file within the memory pool. The file may be opened using
the <a href="../api_reference/C/mempfopen.html" class="olink">DB_MPOOLFILE->open()</a> method. The <a href="../api_reference/C/mempfget.html" class="olink">DB_MPOOLFILE->get()</a> method is used to retrieve
pages from files in the pool. All retrieved pages must be subsequently
returned using the <a href="../api_reference/C/mempput.html" class="olink">DB_MPOOLFILE->put()</a> method. At the time pages are returned,
they may be marked <span class="bold"><strong>dirty</strong></span>, which causes them to be written to
the underlying file before being discarded from the pool. If there is
insufficient room to bring a new page in the pool, a page is selected
to be discarded from the pool using a least-recently-used algorithm.
All dirty pages in the pool from the file may be flushed using the
<a href="../api_reference/C/mempfsync.html" class="olink">DB_MPOOLFILE->sync()</a> method. When the file handle is no longer in use, it
should be closed using the <a href="../api_reference/C/mempfclose.html" class="olink">DB_MPOOLFILE->close()</a> method.</p>
<p>There are additional configuration interfaces that apply when opening
a new file in the memory pool:</p>
<div class="itemizedlist">
<ul type="disc">
<li>The <a href="../api_reference/C/mempset_clear_len.html" class="olink">DB_MPOOLFILE->set_clear_len()</a> method specifies the number of bytes to clear
when creating a new page in the memory pool.</li>
<li>The <a href="../api_reference/C/mempset_fileid.html" class="olink">DB_MPOOLFILE->set_fileid()</a> method specifies a unique ID associated with the file.</li>
<li>The <a href="../api_reference/C/mempset_ftype.html" class="olink">DB_MPOOLFILE->set_ftype()</a> method specifies the type of file for the purposes of
page input and output processing.</li>
<li>The <a href="../api_reference/C/mempset_lsn_offset.html" class="olink">DB_MPOOLFILE->set_lsn_offset()</a> method specifies the byte offset of each page's
log sequence number (<a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>) for the purposes of transaction
checkpoints.</li>
<li>The <a href="../api_reference/C/mempset_pgcookie.html" class="olink">DB_MPOOLFILE->set_pgcookie()</a> method specifies an application provided argument
for the purposes of page input and output processing.</li>
</ul>
</div>
<p>There are additional interfaces for the memory pool as a whole:</p>
<div class="itemizedlist">
<ul type="disc">
<li>It is possible to gradually flush buffers from the pool in order to
maintain a consistent percentage of clean buffers in the pool using
the <a href="../api_reference/C/memptrickle.html" class="olink">DB_ENV->memp_trickle()</a> method.</li>
<li>Because special-purpose processing may be necessary when pages are read
or written (for example, endian conversion, or page checksums), the
<a href="../api_reference/C/mempregister.html" class="olink">DB_ENV->memp_register()</a> function allows applications to specify automatic
input and output processing in these cases.</li>
<li>The <a href="../api_reference/C/db_stat.html" class="olink">db_stat utility</a> uses the <a href="../api_reference/C/mempstat.html" class="olink">DB_ENV->memp_stat()</a> method to display
statistics about the efficiency of the pool.</li>
<li>All dirty pages in the pool may be flushed using the <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> method.
In addition, <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> takes an argument that is specific to
database systems, and which allows the memory pool to be flushed up to
a specified log sequence number (<a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>).</li>
<li>The entire pool may be discarded using the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method.</li>
</ul>
</div>
<div class="informaltable">
<table border="1" width="80%">
<colgroup>
<col />
<col />
</colgroup>
<thead>
<tr>
<th>Memory Pools and Related Methods</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="../api_reference/C/dbget_mpf.html" class="olink">DB->get_mpf()</a> handle</td>
<td>Return <a href="../api_reference/C/db.html" class="olink">DB</a>'s underlying <a href="../api_reference/C/memp.html" class="olink">DB_MPOOLFILE</a> handle</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempstat.html" class="olink">DB_ENV->memp_stat()</a>
</td>
<td>Return memory pool statistics</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a>
</td>
<td>Flush pages from a memory pool</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/memptrickle.html" class="olink">DB_ENV->memp_trickle()</a>
</td>
<td>Trickle flush pages from a memory pool</td>
</tr>
<tr>
<td>
<span class="emphasis">
<em>Memory Pool Configuration</em>
</span>
</td>
<td> </td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempregister.html" class="olink">DB_ENV->memp_register()</a>
</td>
<td>Register input/output functions for a file in a memory pool</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/envset_cachesize.html" class="olink">DB_ENV->set_cachesize()</a>
</td>
<td>Set the environment cache size</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/envset_cache_max.html" class="olink">DB_ENV->set_cache_max()</a>
</td>
<td>Set the maximum cache size</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_mp_max_openfd.html" class="olink">DB_ENV->set_mp_max_openfd()</a>
</td>
<td>Set the maximum number of open file descriptors</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_mp_max_write.html" class="olink">DB_ENV->set_mp_max_write()</a>
</td>
<td>Set the maximum number of sequential disk writes</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/envset_mp_mmapsize.html" class="olink">DB_ENV->set_mp_mmapsize()</a>
</td>
<td>Set maximum mapped-in database file size</td>
</tr>
<tr>
<td>
<span class="emphasis">
<em>Memory Pool Files</em>
</span>
</td>
<td> </td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempfcreate.html" class="olink">DB_ENV->memp_fcreate()</a>
</td>
<td>Create a memory pool file handle</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempfclose.html" class="olink">DB_MPOOLFILE->close()</a>
</td>
<td>Close a file in a memory pool</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempfget.html" class="olink">DB_MPOOLFILE->get()</a>
</td>
<td>Get page from a file in a memory pool</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempfopen.html" class="olink">DB_MPOOLFILE->open()</a>
</td>
<td>Open a file in a memory pool</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempput.html" class="olink">DB_MPOOLFILE->put()</a>
</td>
<td>Return a page to a memory pool</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempfsync.html" class="olink">DB_MPOOLFILE->sync()</a>
</td>
<td>Flush pages from a file in a memory pool</td>
</tr>
<tr>
<td>
<span class="emphasis">
<em>Memory Pool File Configuration</em>
</span>
</td>
<td> </td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_clear_len.html" class="olink">DB_MPOOLFILE->set_clear_len()</a>
</td>
<td>Set file page bytes to be cleared</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_fileid.html" class="olink">DB_MPOOLFILE->set_fileid()</a>
</td>
<td>Set file unique identifier</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_flags.html" class="olink">DB_MPOOLFILE->set_flags()</a>
</td>
<td>General memory pool file configuration</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_ftype.html" class="olink">DB_MPOOLFILE->set_ftype()</a>
</td>
<td>Set file type</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_lsn_offset.html" class="olink">DB_MPOOLFILE->set_lsn_offset()</a>
</td>
<td>Set file log-sequence-number offset</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_maxsize.html" class="olink">DB_MPOOLFILE->set_maxsize()</a>
</td>
<td>Set maximum file size</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_pgcookie.html" class="olink">DB_MPOOLFILE->set_pgcookie()</a>
</td>
<td>Set file cookie for pgin/pgout</td>
</tr>
<tr>
<td>
<a href="../api_reference/C/mempset_priority.html" class="olink">mempset_priority()</a>
</td>
<td>Set memory pool file priority</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="log_limits.html">Prev</a> </td>
<td width="20%" align="center"> </td>
<td width="40%" align="right"> <a accesskey="n" href="mp_config.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Log file limits </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Configuring the memory pool</td>
</tr>
</table>
</div>
</body>
</html>
|
