1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>Architecture of SQLite</title>
<style type="text/css">
body {
margin: auto;
font-family: Verdana, sans-serif;
padding: 8px 1%;
}
a { color: #044a64 }
a:visited { color: #734559 }
.logo { position:absolute; margin:3px; }
.tagline {
float:right;
text-align:right;
font-style:italic;
width:300px;
margin:12px;
margin-top:58px;
}
.toolbar {
text-align: center;
line-height: 1.6em;
margin: 0;
padding: 0px 8px;
}
.toolbar a { color: white; text-decoration: none; padding: 6px 12px; }
.toolbar a:visited { color: white; }
.toolbar a:hover { color: #044a64; background: white; }
.content { margin: 5%; }
.content dt { font-weight:bold; }
.content dd { margin-bottom: 25px; margin-left:20%; }
.content ul { padding:0px; padding-left: 15px; margin:0px; }
/* rounded corners */
.se { background: url(images/se.gif) 100% 100% no-repeat #044a64}
.sw { background: url(images/sw.gif) 0% 100% no-repeat }
.ne { background: url(images/ne.gif) 100% 0% no-repeat }
.nw { background: url(images/nw.gif) 0% 0% no-repeat }
/* Things for "fancyformat" documents start here. */
.fancy img+p {font-style:italic}
.fancy .codeblock i { color: darkblue; }
.fancy h1,.fancy h2,.fancy h3,.fancy h4 {font-weight:normal;color:#044a64}
.fancy h2 { margin-left: 10px }
.fancy h3 { margin-left: 20px }
.fancy h4 { margin-left: 30px }
.fancy th {white-space:nowrap;text-align:left;border-bottom:solid 1px #444}
.fancy th, .fancy td {padding: 0.2em 1ex; vertical-align:top}
.fancy #toc a { color: darkblue ; text-decoration: none }
.fancy .todo { color: #AA3333 ; font-style : italic }
.fancy .todo:before { content: 'TODO:' }
.fancy p.todo { border: solid #AA3333 1px; padding: 1ex }
.fancy img { display:block; }
.fancy :link:hover, .fancy :visited:hover { background: wheat }
.fancy p,.fancy ul,.fancy ol { margin: 1em 5ex }
.fancy li p { margin: 1em 0 }
/* End of "fancyformat" specific rules. */
</style>
</head>
<body>
<div><!-- container div to satisfy validator -->
<a href="index.html">
<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite Logo"
border="0"></a>
<div><!-- IE hack to prevent disappearing logo--></div>
<div class="tagline">Small. Fast. Reliable.<br>Choose any three.</div>
<table width=100% style="clear:both"><tr><td>
<div class="se"><div class="sw"><div class="ne"><div class="nw">
<table width=100% style="padding:0;margin:0;cell-spacing:0"><tr>
<td width=100%>
<div class="toolbar">
<a href="about.html">About</a>
<a href="sitemap.html">Sitemap</a>
<a href="docs.html">Documentation</a>
<a href="download.html">Download</a>
<a href="copyright.html">License</a>
<a href="news.html">News</a>
<a href="support.html">Support</a>
</div>
<script>
gMsg = "Search SQLite Docs..."
function entersearch() {
var q = document.getElementById("q");
if( q.value == gMsg ) { q.value = "" }
q.style.color = "black"
q.style.fontStyle = "normal"
}
function leavesearch() {
var q = document.getElementById("q");
if( q.value == "" ) {
q.value = gMsg
q.style.color = "#044a64"
q.style.fontStyle = "italic"
}
}
</script>
<td>
<div style="padding:0 1em 0px 0;white-space:nowrap">
<form name=f method="GET" action="http://www.sqlite.org/search">
<input id=q name=q type=text
onfocus="entersearch()" onblur="leavesearch()" style="width:24ex;padding:1px 1ex; border:solid white 1px; font-size:0.9em ; font-style:italic;color:#044a64;" value="Search SQLite Docs...">
<input type=submit value="Go" style="border:solid white 1px;background-color:#044a64;color:white;font-size:0.9em;padding:0 1ex">
</form>
</div>
</table>
</div></div></div></div>
</td></tr></table>
<div class=startsearch></div>
<h2>The Architecture Of SQLite</h2>
<h3>Introduction</h3>
<table align="right" border="1" cellpadding="15" cellspacing="1">
<tr><th>Block Diagram Of SQLite</th></tr>
<tr><td><img src="images/arch2.gif"></td></tr>
</table>
<p>This document describes the architecture of the SQLite library.
The information here is useful to those who want to understand or
modify the inner workings of SQLite.
</p>
<p>
A block diagram showing the main components of SQLite
and how they interrelate is shown at the right. The text that
follows will provide a quick overview of each of these components.
</p>
<p>
This document describes SQLite version 3.0. Version 2.8 and
earlier are similar but the details differ.
</p>
<h3>Interface</h3>
<p>Much of the public interface to the SQLite library is implemented by
functions found in the <b>main.c</b>, <b>legacy.c</b>, and
<b>vdbeapi.c</b> source files
though some routines are
scattered about in other files where they can have access to data
structures with file scope. The
<b>sqlite3_get_table()</b> routine is implemented in <b>table.c</b>.
<b>sqlite3_mprintf()</b> is found in <b>printf.c</b>.
<b>sqlite3_complete()</b> is in <b>tokenize.c</b>.
The Tcl interface is implemented by <b>tclsqlite.c</b>. More
information on the C interface to SQLite is
<a href="capi3ref.html">available separately</a>.<p>
<p>To avoid name collisions with other software, all external
symbols in the SQLite library begin with the prefix <b>sqlite3</b>.
Those symbols that are intended for external use (in other words,
those symbols which form the API for SQLite) begin
with <b>sqlite3_</b>.</p>
<h3>Tokenizer</h3>
<p>When a string containing SQL statements is to be executed, the
interface passes that string to the tokenizer. The job of the tokenizer
is to break the original string up into tokens and pass those tokens
one by one to the parser. The tokenizer is hand-coded in C in
the file <b>tokenize.c</b>.
<p>Note that in this design, the tokenizer calls the parser. People
who are familiar with YACC and BISON may be used to doing things the
other way around -- having the parser call the tokenizer. The author
of SQLite
has done it both ways and finds things generally work out nicer for
the tokenizer to call the parser. YACC has it backwards.</p>
<h3>Parser</h3>
<p>The parser is the piece that assigns meaning to tokens based on
their context. The parser for SQLite is generated using the
<a href="http://www.sqlite.org/src/doc/trunk/doc/lemon.html">Lemon</a>
LALR(1) parser
generator. Lemon does the same job as YACC/BISON, but it uses
a different input syntax which is less error-prone.
Lemon also generates a parser which is reentrant and thread-safe.
And lemon defines the concept of a non-terminal destructor so
that it does not leak memory when syntax errors are encountered.
The source file that drives Lemon is found in <b>parse.y</b>.</p>
<p>Because
lemon is a program not normally found on development machines, the
complete source code to lemon (just one C file) is included in the
SQLite distribution in the "tool" subdirectory. Documentation on
lemon is found in the "doc" subdirectory of the distribution.
</p>
<h3>Code Generator</h3>
<p>After the parser assembles tokens into complete SQL statements,
it calls the code generator to produce virtual machine code that
will do the work that the SQL statements request. There are many
files in the code generator:
<b>attach.c</b>,
<b>auth.c</b>,
<b>build.c</b>,
<b>delete.c</b>,
<b>expr.c</b>,
<b>insert.c</b>,
<b>pragma.c</b>,
<b>select.c</b>,
<b>trigger.c</b>,
<b>update.c</b>,
<b>vacuum.c</b>
and <b>where.c</b>.
In these files is where most of the serious magic happens.
<b>expr.c</b> handles code generation for expressions.
<b>where.c</b> handles code generation for WHERE clauses on
SELECT, UPDATE and DELETE statements. The files <b>attach.c</b>,
<b>delete.c</b>, <b>insert.c</b>, <b>select.c</b>, <b>trigger.c</b>
<b>update.c</b>, and <b>vacuum.c</b> handle the code generation
for SQL statements with the same names. (Each of these files calls routines
in <b>expr.c</b> and <b>where.c</b> as necessary.) All other
SQL statements are coded out of <b>build.c</b>.
The <b>auth.c</b> file implements the functionality of
<b>sqlite3_set_authorizer()</b>.</p>
<h3>Virtual Machine</h3>
<p>The program generated by the code generator is executed by
the virtual machine. Additional information about the virtual
machine is <a href="opcode.html">available separately</a>.
To summarize, the virtual machine implements an abstract computing
engine specifically designed to manipulate database files. The
machine has a stack which is used for intermediate storage.
Each instruction contains an opcode and
up to three additional operands.</p>
<p>The virtual machine itself is entirely contained in a single
source file <b>vdbe.c</b>. The virtual machine also has
its own header files: <b>vdbe.h</b> that defines an interface
between the virtual machine and the rest of the SQLite library and
<b>vdbeInt.h</b> which defines structure private the virtual machine.
The <b>vdbeaux.c</b> file contains utilities used by the virtual
machine and interface modules used by the rest of the library to
construct VM programs. The <b>vdbeapi.c</b> file contains external
interfaces to the virtual machine such as the
<b>sqlite3_bind_...</b> family of functions. Individual values
(strings, integer, floating point numbers, and BLOBs) are stored
in an internal object named "Mem" which is implemented by
<b>vdbemem.c</b>.</p>
<p>
SQLite implements SQL functions using callbacks to C-language routines.
Even the built-in SQL functions are implemented this way. Most of
the built-in SQL functions (ex: <b>coalesce()</b>, <b>count()</b>,
<b>substr()</b>, and so forth) can be found in <b>func.c</b>.
Date and time conversion functions are found in <b>date.c</b>.
</p>
<h3>B-Tree</h3>
<p>An SQLite database is maintained on disk using a B-tree implementation
found in the <b>btree.c</b> source file. A separate B-tree is used for
each table and index in the database. All B-trees are stored in the
same disk file. Details of the file format are recorded in a large
comment at the beginning of <b>btree.c</b>.</p>
<p>The interface to the B-tree subsystem is defined by the header file
<b>btree.h</b>.
</p>
<h3>Page Cache</h3>
<p>The B-tree module requests information from the disk in fixed-size
chunks. The default chunk size is 1024 bytes but can vary between 512
and 65536 bytes.
The page cache is responsible for reading, writing, and
caching these chunks.
The page cache also provides the rollback and atomic commit abstraction
and takes care of locking of the database file. The
B-tree driver requests particular pages from the page cache and notifies
the page cache when it wants to modify pages or commit or rollback
changes. The page cache handles all the messy details of making sure
the requests are handled quickly, safely, and efficiently.</p>
<p>The code to implement the page cache is contained in the single C
source file <b>pager.c</b>. The interface to the page cache subsystem
is defined by the header file <b>pager.h</b>.
</p>
<h3>OS Interface</h3>
<p>
In order to provide portability between POSIX and Win32 operating systems,
SQLite uses an abstraction layer to interface with the operating system.
The interface to the OS abstraction layer is defined in
<b>os.h</b>. Each supported operating system has its own implementation:
<b>os_unix.c</b> for Unix, <b>os_win.c</b> for Windows, and so forth.
Each of these operating-specific implements typically has its own
header file: <b>os_unix.h</b>, <b>os_win.h</b>, etc.
</p>
<h3>Utilities</h3>
<p>
Memory allocation and caseless string comparison routines are located
in <b>util.c</b>.
Symbol tables used by the parser are maintained by hash tables found
in <b>hash.c</b>. The <b>utf.c</b> source file contains Unicode
conversion subroutines.
SQLite has its own private implementation of <b>printf()</b> (with
some extensions) in <b>printf.c</b> and its own random number generator
in <b>random.c</b>.
</p>
<h3>Test Code</h3>
<p>
If you count regression test scripts,
more than half the total code base of SQLite is devoted to testing.
There are many <b>assert()</b> statements in the main code files.
In additional, the source files <b>test1.c</b> through <b>test5.c</b>
together with <b>md5.c</b> implement extensions used for testing
purposes only. The <b>os_test.c</b> backend interface is used to
simulate power failures to verify the crash-recovery mechanism in
the pager.
</p>
|
