1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Configuring locking: sizing the system</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="up" href="lock.html" title="Chapter 16. The Locking Subsystem" />
<link rel="prev" href="lock_config.html" title="Configuring locking" />
<link rel="next" href="lock_stdmode.html" title="Standard lock modes" />
</head>
<body>
<div xmlns="" class="navheader">
<div class="libver">
<p>Library Version 11.2.5.3</p>
</div>
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Configuring locking: sizing the system</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="lock_config.html">Prev</a> </td>
<th width="60%" align="center">Chapter 16.
The Locking Subsystem
</th>
<td width="20%" align="right"> <a accesskey="n" href="lock_stdmode.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="lock_max"></a>Configuring locking: sizing the system</h2>
</div>
</div>
</div>
<p>
The amount of memory available to the locking system is specified
using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method. Sizing of the enviroment
using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method is discussed in
<a class="xref" href="env_size.html" title="Sizing a database environment">Sizing a database environment</a>. Here we will
discuss how to estimate the number of objects your application is
likely to lock. Since running out of memory for locking structures
is a fatal error requiring reconfiguration and restarting the
environment it is best to overestimate the numbers.
</p>
<p>
When configuring a Berkeley DB Concurrent Data Store application,
the number of lock objects needed is two per open database (one for
the database lock, and one for the cursor lock when the
<a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> option is not specified). The number of locks
needed is one per open database handle plus one per simultaneous
cursor or non-cursor operation.
</p>
<p>
Configuring a Berkeley DB Transactional Data Store application is
more complicated. The recommended algorithm for selecting the
number of locks, lockers, and lock objects is to run the
application under stressful conditions and then review the lock
system's statistics to determine the number of locks,
lockers, and lock objects that were used. Then, double these
values for safety. However, in some large applications, finer
granularity of control is necessary in order to minimize the size
of the Lock subsystem.
</p>
<p>
The number of lockers can be estimated as follows:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
If the database environment is using transactions, the
number of lockers can be estimated by adding the number of
simultaneously active non-transactional cursors and open database
handles to the number of simultaneously active transactions and
child transactions (where a child transaction is active until
it commits or aborts, not until its parent commits or aborts).
</li>
<li>
If the database environment is not using transactions, the
number of lockers can be estimated by adding the number
of simultaneously active non-transactional cursors and open
database handles to the number of simultaneous non-cursor
operations.
</li>
</ul>
</div>
<p>
The number of lock objects needed for a transaction
can be estimated as follows:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
For each access to a non-Queue database, one lock object is
needed for each page that is read or updated.
</p>
</li>
<li>
<p>
For the Queue access method you will need one lock object per
record that is read or updated. Deleted records skipped by a
DB_NEXT or DB_PREV operation do not require a separate lock
object.
</p>
</li>
<li>
<p>
For Btree and Recno databases additional lock objects may be
needed for each node in the btree that has to be split due to
an update.
</p>
</li>
<li>
<p>
For Hash and Queue databases, every access must obtain a lock on
the metadata page for the duration of the access. This is not
held to the end of the transaction.
</p>
</li>
<li>
<p>
If the transaction performs an update that needs to allocate a page
to the database then a lock object for the metadata page will
be needed to the end of the transaction.
</p>
</li>
</ul>
</div>
<p>
Note that transactions accumulate locks over the transaction lifetime,
and the lock objects required by a single transaction is the total lock
objects required by all of the database operations in the transaction.
However, a database page (or record, in the case of the Queue access
method), that is accessed multiple times within a transaction only
requires a single lock object for the entire transaction. So if a
transaction in your application typically accesses 10 records, that
transaction will require about 10 lock objects (it may be a few more if
it splits btree nodes). If you have up to 10 concurrent threads in your
application, then you need to configure your system to have about 100
lock objects. It is always better to configure more than you need so
that you don't run out of lock objects. The memory overhead of
over-allocating lock objects is minimal as they are small structures.
</p>
<p>
The number of locks required by an application cannot be easily
estimated. It is possible to calculate a number of locks by
multiplying the number of lockers, times the number of
lock objects, times two (two for the two possible lock modes for each
object, read and write). However, this is a pessimal value, and real
applications are unlikely to actually need that many locks. Reviewing
the Lock subsystem statistics is the best way to determine this value.
</p>
<p>
By default a minimum number of locking objects are allocated at
startup. To avoid contention due to allocation the application may
use the <a href="../api_reference/C/envset_memory_init.html" class="olink">DB_ENV->set_memory_init()</a> method to preallocate and initialize
the following lock structures:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
<code class="literal">DB_MEM_LOCK</code>
</p>
<p>
Specifies the number of locks that can be
simultaneously requested in the system.
</p>
</li>
<li>
<p>
<code class="literal">DB_MEM_LOCKER</code>
</p>
<p>
Specifies the number of lockers that can
simultaneously request locks in the system.
</p>
</li>
<li>
<p>
<code class="literal">DB_MEM_LOCKOBJECTS</code>
</p>
<p>
Specifies the number of objects that can
simultaneously be locked in the system.
</p>
</li>
</ul>
</div>
<p>
In addition to the above structures, sizing your locking subsystem
also requires specifying the number of lock table partitions. You
do this using the <a href="../api_reference/C/envset_lk_partitions.html" class="olink">DB_ENV->set_lk_partitions()</a> method. Each partition
may be accessed independently by a thread. More partitions can lead
to higher levels of concurrency. The default is to set the number
of partitions to be 10 times the number of cpus that the operating
system reports at the time the environment is created. Having more
than one partition when there is only one cpu is not beneficial
because the locking system is more efficient when there is a single
partition. Some operating systems (Linux, Solaris) may report
thread contexts as cpus, and so it may be necessary to override the
default to force a single partition on a single hyperthreaded cpu
system. Objects and locks are divided among the partitions so it is
best to allocate several locks and objects per partition. The
system will force there to be at least one per partition. If a
partition runs out of locks or objects it will steal what is needed
from the other partitions. This operation could impact performance
if it occurs too often. The final values specified for the locks
and lock objects should be more than or equal to the number of lock
table partitions.
</p>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="lock_config.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="lock.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="lock_stdmode.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Configuring locking </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Standard lock modes</td>
</tr>
</table>
</div>
</body>
</html>
|
