1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
|
<!DOCTYPE html>
<html><head>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<link href="sqlite.css" rel="stylesheet">
<title>An Asynchronous I/O Module For SQLite</title>
<!-- path= -->
</head>
<body>
<div class=nosearch>
<a href="index.html">
<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite" border="0">
</a>
<div><!-- IE hack to prevent disappearing logo --></div>
<div class="tagline desktoponly">
Small. Fast. Reliable.<br>Choose any three.
</div>
<div class="menu mainmenu">
<ul>
<li><a href="index.html">Home</a>
<li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a>
<li class='wideonly'><a href='about.html'>About</a>
<li class='desktoponly'><a href="docs.html">Documentation</a>
<li class='desktoponly'><a href="download.html">Download</a>
<li class='wideonly'><a href='copyright.html'>License</a>
<li class='desktoponly'><a href="support.html">Support</a>
<li class='desktoponly'><a href="prosupport.html">Purchase</a>
<li class='search' id='search_menubutton'>
<a href="javascript:void(0)" onclick='toggle_search()'>Search</a>
</ul>
</div>
<div class="menu submenu" id="submenu">
<ul>
<li><a href='about.html'>About</a>
<li><a href='docs.html'>Documentation</a>
<li><a href='download.html'>Download</a>
<li><a href='support.html'>Support</a>
<li><a href='prosupport.html'>Purchase</a>
</ul>
</div>
<div class="searchmenu" id="searchmenu">
<form method="GET" action="search">
<select name="s" id="searchtype">
<option value="d">Search Documentation</option>
<option value="c">Search Changelog</option>
</select>
<input type="text" name="q" id="searchbox" value="">
<input type="submit" value="Go">
</form>
</div>
</div>
<script>
function toggle_div(nm) {
var w = document.getElementById(nm);
if( w.style.display=="block" ){
w.style.display = "none";
}else{
w.style.display = "block";
}
}
function toggle_search() {
var w = document.getElementById("searchmenu");
if( w.style.display=="block" ){
w.style.display = "none";
} else {
w.style.display = "block";
setTimeout(function(){
document.getElementById("searchbox").focus()
}, 30);
}
}
function div_off(nm){document.getElementById(nm).style.display="none";}
window.onbeforeunload = function(e){div_off("submenu");}
/* Disable the Search feature if we are not operating from CGI, since */
/* Search is accomplished using CGI and will not work without it. */
if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){
document.getElementById("search_menubutton").style.display = "none";
}
/* Used by the Hide/Show button beside syntax diagrams, to toggle the */
function hideorshow(btn,obj){
var x = document.getElementById(obj);
var b = document.getElementById(btn);
if( x.style.display!='none' ){
x.style.display = 'none';
b.innerHTML='show';
}else{
x.style.display = '';
b.innerHTML='hide';
}
return false;
}
</script>
</div>
<h1 align="center">An Asynchronous I/O Module For SQLite</h1>
<hr>
<p><font size=+1><b>NOTE:</b>
<a href="wal.html">WAL mode</a> with <a href="pragma.html#pragma_synchronous">PRAGMA synchronous</a> set to NORMAL avoids calls to
fsync() during transaction commit and only invokes fsync() during
a <a href="wal.html#ckpt">checkpoint</a> operation. The use of <a href="wal.html">WAL mode</a> largely obviates the
need for this asynchronous I/O module. Hence, this module is no longer
supported. The source code continues to exist in the SQLite source tree,
but it is not a part of any standard build and is no longer maintained.
This documentation is retained for historical reference.</font></p><hr>
<p>Normally, when SQLite writes to a database file, it waits until the write
operation is finished before returning control to the calling application.
Since writing to the file-system is usually very slow compared with CPU
bound operations, this can be a performance bottleneck. The asynchronous I/O
backend is an extension that causes SQLite to perform all write requests
using a separate thread running in the background. Although this does not
reduce the overall system resources (CPU, disk bandwidth etc.), it does
allow SQLite to return control to the caller quickly even when writing to
the database.
<h2>1.0 FUNCTIONALITY</h2>
<p>With asynchronous I/O, write requests are handled by a separate thread
running in the background. This means that the thread that initiates
a database write does not have to wait for (sometimes slow) disk I/O
to occur. The write seems to happen very quickly, though in reality
it is happening at its usual slow pace in the background.
<p>Asynchronous I/O appears to give better responsiveness, but at a price.
You lose the Durable property. With the default I/O backend of SQLite,
once a write completes, you know that the information you wrote is
safely on disk. With the asynchronous I/O, this is not the case. If
your program crashes or if a power loss occurs after the database
write but before the asynchronous write thread has completed, then the
database change might never make it to disk and the next user of the
database might not see your change.
<p>You lose Durability with asynchronous I/O, but you still retain the
other parts of ACID: Atomic, Consistent, and Isolated. Many
applications get along fine without the Durability.
<h3>1.1 How it Works</h3>
<p>Asynchronous I/O works by creating an SQLite <a href="c3ref/vfs.html">VFS object</a>
and registering it with <a href="c3ref/vfs_find.html">sqlite3_vfs_register()</a>.
When files opened via
this VFS are written to (using the vfs xWrite() method), the data is not
written directly to disk, but is placed in the "write-queue" to be
handled by the background thread.
<p>When files opened with the asynchronous VFS are read from
(using the vfs xRead() method), the data is read from the file on
disk and the write-queue, so that from the point of view of
the vfs reader the xWrite() appears to have already completed.
<p>The asynchronous I/O VFS is registered (and unregistered) by calls to the
API functions sqlite3async_initialize() and sqlite3async_shutdown().
See section "Compilation and Usage" below for details.
<h3>1.2 Limitations</h3>
<p>In order to gain experience with the main ideas surrounding asynchronous
IO, this implementation is deliberately kept simple. Additional
capabilities may be added in the future.
<p>For example, as currently implemented, if writes are happening at a
steady stream that exceeds the I/O capability of the background writer
thread, the queue of pending write operations will grow without bound.
If this goes on for long enough, the host system could run out of memory.
A more sophisticated module could to keep track of the quantity of
pending writes and stop accepting new write requests when the queue of
pending writes grows too large.
<h3>1.3 Locking and Concurrency</h3>
<p>Multiple connections from within a single process that use this
implementation of asynchronous IO may access a single database
file concurrently. From the point of view of the user, if all
connections are from within a single process, there is no difference
between the concurrency offered by "normal" SQLite and SQLite
using the asynchronous backend.
<p>If file-locking is enabled (it is enabled by default), then connections
from multiple processes may also read and write the database file.
However concurrency is reduced as follows:
<ul>
<li><p> When a connection using asynchronous IO begins a database
transaction, the database is locked immediately. However the
lock is not released until after all relevant operations
in the write-queue have been flushed to disk. This means
(for example) that the database may remain locked for some
time after a "<a href="lang_transaction.html">COMMIT</a>" or "<a href="lang_transaction.html">ROLLBACK</a>" is issued.
<li><p> If an application using asynchronous IO executes transactions
in quick succession, other database users may be effectively
locked out of the database. This is because when a <a href="lang_transaction.html">BEGIN</a>
is executed, a database lock is established immediately. But
when the corresponding COMMIT or ROLLBACK occurs, the lock
is not released until the relevant part of the write-queue
has been flushed through. As a result, if a COMMIT is followed
by a BEGIN before the write-queue is flushed through, the database
is never unlocked,preventing other processes from accessing
the database.
</ul>
<p>File-locking may be disabled at runtime using the sqlite3async_control()
API (see below). This may improve performance when an NFS or other
network file-system, as the synchronous round-trips to the server be
required to establish file locks are avoided. However, if multiple
connections attempt to access the same database file when file-locking
is disabled, application crashes and database corruption is a likely
outcome.
<h2>2.0 COMPILATION AND USAGE</h2>
<p>
The asynchronous IO extension consists of a single file of C code
(sqlite3async.c), and a header file (sqlite3async.h), located in the
<a href="http://www.sqlite.org/src/dir?name=ext/async">
<tt>ext/async/</tt> subfolder</a> of the SQLite source tree, that defines the
C API used by applications to activate and control the modules
functionality.
<p>
To use the asynchronous IO extension, compile sqlite3async.c as
part of the application that uses SQLite. Then use the APIs defined
in sqlite3async.h to initialize and configure the module.
<p>
The asynchronous IO VFS API is described in detail in comments in
sqlite3async.h. Using the API usually consists of the following steps:
<ol>
<li><p>Register the asynchronous IO VFS with SQLite by calling the
sqlite3async_initialize() function.
<li><p>Create a background thread to perform write operations and call
sqlite3async_run().
<li><p>Use the normal SQLite API to read and write to databases via
the asynchronous IO VFS.
</ol>
<p>Refer to comments in the
<a href="http://www.sqlite.org/src/finfo?name=ext/async/sqlite3async.h">
sqlite3async.h header file</a> for details.
<h2>3.0 PORTING</h2>
<p>Currently the asynchronous IO extension is compatible with win32 systems
and systems that support the pthreads interface, including Mac OS X, Linux,
and other varieties of Unix.
<p>To port the asynchronous IO extension to another platform, the user must
implement mutex and condition variable primitives for the new platform.
Currently there is no externally available interface to allow this, but
modifying the code within sqlite3async.c to include the new platforms
concurrency primitives is relatively easy. Search within sqlite3async.c
for the comment string "PORTING FUNCTIONS" for details. Then implement
new versions of each of the following:
<blockquote><pre>
static void async_mutex_enter(int eMutex);
static void async_mutex_leave(int eMutex);
static void async_cond_wait(int eCond, int eMutex);
static void async_cond_signal(int eCond);
static void async_sched_yield(void);
</pre></blockquote>
<p>The functionality required of each of the above functions is described
in comments in sqlite3async.c.
|
