1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Retrieving records in bulk</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="up" href="am_misc.html" title="Chapter 4. Access Method Wrapup" />
<link rel="prev" href="am_misc.html" title="Chapter 4. Access Method Wrapup" />
<link rel="next" href="am_misc_partial.html" title="Partial record storage and retrieval" />
</head>
<body>
<div class="navheader">
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Retrieving records in bulk</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="am_misc.html">Prev</a> </td>
<th width="60%" align="center">Chapter 4.
Access Method Wrapup
</th>
<td width="20%" align="right"> <a accesskey="n" href="am_misc_partial.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="am_misc_get_bulk"></a>Retrieving records in bulk</h2>
</div>
</div>
</div>
<p>When retrieving large numbers of records from the database, the number
of method calls can often dominate performance. Berkeley DB offers bulk get
interfaces which can significantly increase performance for some
applications. To retrieve records in bulk, an application buffer must
be specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods. This is done
in the C API by setting the <span class="bold"><strong>data</strong></span> and <span class="bold"><strong>ulen</strong></span> fields of the
<span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> to reference an application buffer, and the
<span class="bold"><strong>flags</strong></span> field of that structure to <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a>. In
the Berkeley DB C++ and Java APIs, the actions are similar, although there
are API-specific methods to set the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> values. Then, the
<a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> or <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flags are specified to
the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods, which cause multiple records
to be returned in the specified buffer.</p>
<p>The difference between <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> and <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a>
is as follows: <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> returns multiple data items for a
single key. For example, the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag would be used to
retrieve all of the duplicate data items for a single key in a single
call. The <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag is used to retrieve multiple
key/data pairs, where each returned key may or may not have duplicate
data items.</p>
<p>Once the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method has returned, the
application will walk through the buffer handling the returned records.
This is implemented for the C and C++ APIs using four macros:
<a href="../api_reference/C/DB_MULTIPLE_INIT.html" class="olink">DB_MULTIPLE_INIT</a>, <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a>.
For the Java API, this is implemented as three iterator classes:
<a class="ulink" href="../../java/com/sleepycat/db/MultipleDataEntry.html" target="_top">MultipleDataEntry</a>,
<a class="ulink" href="../../java/com/sleepycat/db/MultipleKeyDataEntry.html" target="_top">MultipleKeyDataEntry</a>, and
<a class="ulink" href="../../java/com/sleepycat/db/MultipleRecnoDataEntry.html" target="_top">MultipleRecnoDataEntry</a>.</p>
<p>The <a href="../api_reference/C/DB_MULTIPLE_INIT.html" class="olink">DB_MULTIPLE_INIT</a> macro is always called first. It
initializes a local application variable and the <span class="bold"><strong>data</strong></span>
<a href="../api_reference/C/dbt.html" class="olink">DBT</a> for stepping through the set of returned records. Then,
the application calls one of the remaining three macros:
<a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a>.
</p>
<p>If the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>
method, the application will always call the <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a> macro. If
the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method, and the
underlying database is a Btree or Hash database, the application will
always call the <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a> macro. If the
<a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or
<a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method, and the underlying database is a Queue or Recno
database, the application will always call the
<a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a> macro. The <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>,
and <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a> macros
are called repeatedly, until the end of the returned records is reached.
The end of the returned records is detected by the application's local
pointer variable being set to NULL.</p>
<p>The following is an example of a routine that displays the contents of
a Btree database using the bulk return interfaces.</p>
<pre class="programlisting">int
rec_display(dbp)
DB *dbp;
{
DBC *dbcp;
DBT key, data;
size_t retklen, retdlen;
char *retkey, *retdata;
int ret, t_ret;
void *p;
memset(&key, 0, sizeof(key));
memset(&data, 0, sizeof(data));
/* Review the database in 5MB chunks. */
#define BUFFER_LENGTH (5 * 1024 * 1024)
if ((data.data = malloc(BUFFER_LENGTH)) == NULL)
return (errno);
data.ulen = BUFFER_LENGTH;
data.flags = DB_DBT_USERMEM;
/* Acquire a cursor for the database. */
if ((ret = dbp->cursor(dbp, NULL, &dbcp, 0)) != 0) {
dbp->err(dbp, ret, "DB->cursor");
free(data.data);
return (ret);
}
for (;;) {
/*
* Acquire the next set of key/data pairs. This code does
* not handle single key/data pairs that won't fit in a
* BUFFER_LENGTH size buffer, instead returning DB_BUFFER_SMALL
* to our caller.
*/
if ((ret = dbcp->c_get(dbcp,
&key, &data, DB_MULTIPLE_KEY | DB_NEXT)) != 0) {
if (ret != DB_NOTFOUND)
dbp->err(dbp, ret, "DBcursor->c_get");
break;
}
for (DB_MULTIPLE_INIT(p, &data);;) {
DB_MULTIPLE_KEY_NEXT(p,
&data, retkey, retklen, retdata, retdlen);
if (p == NULL)
break;
printf("key: %.*s, data: %.*s\n",
(int)retklen, retkey, (int)retdlen, retdata);
}
}
if ((t_ret = dbcp->c_close(dbcp)) != 0) {
dbp->err(dbp, ret, "DBcursor->close");
if (ret == 0)
ret = t_ret;
}
free(data.data);
return (ret);
}</pre>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="am_misc.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="am_misc.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="am_misc_partial.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Chapter 4.
Access Method Wrapup
</td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Partial record storage and retrieval</td>
</tr>
</table>
</div>
</body>
</html>
|
