1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
|
<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title>Rivet Internals</title><link rel="stylesheet" type="text/css" href="rivet.css"><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot"><link rel="home" href="index.html" title="Apache Rivet 3.2"><link rel="up" href="index.html" title="Apache Rivet 3.2"><link rel="prev" href="help.html" title="Resources - How to Get Help"><link rel="next" href="lazybridge.html" title="Example: the “Lazy” bridge"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Rivet Internals</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="help.html"><img src="images/prev.png" alt="Prev"></a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="lazybridge.html"><img src="images/next.png" alt="Next"></a></td></tr></table></div><div class="section"><div class="titlepage"><div><div><hr><h2 class="title" style="clear: both"><a name="internals"></a>Rivet Internals</h2></div></div></div><p style="width:90%">
This section easily falls out of date, as new code is added, old
code is removed, and changes are made. The best place to look
is the source code itself. If you are interested in the changes
themselves, the Subversion revision control system
(<span style="font-family:monospace"><span class="command"><strong>svn</strong></span></span>) can provide you with information about
what has been happening with the code.
</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4424"></a>Rivet approach to Apache Multiprocessing Models</h3></div></div></div><p style="width:90%">
The Apache HTTP web server has an extremely modular architecture
that made it very popular among web developers. Most of the server
features can be implemented in external modules, including some of
the way the server interfaces to the operative system. The multiprocessing
modules are meant to provide different models for distributing the
server workload but also to cope with different operative systems
having their specific architectures and services.
</p><p style="width:90%">
From the very beginning mod_rivet was designed to work with
the <a class="ulink" href="https://httpd.apache.org/docs/2.4/mod/prefork.html" target="_top">prefork MPM</a>
MPM (Multi Processing Module) which assumes the OS to have 'fork' capabilities.
This prerequisite basically restricted mod_rivet to work only with
Unix-like operative systems. Starting with version 3.0 we reorganized
mod_rivet to offer a design that could work together with more MPM and
hopefully pave the way to support different OS that have no 'fork'
call. At the same time we tried to preserve some of the basic
features of mod_rivet when working with the prefork MPM, chiefly
the feature of the Unix fork system call of 'cloning' a parent process
memory into its child, thus allowing fast initialization of interpreters.
</p><p style="width:90%">
The central design of mod_rivet now relies on the idea of <span class="quote">“<span class="quote">MPM bridges</span>”</span>,
loadable modules that are responsible to adapt the module procedural design to
a given class of Apache MPMs. This design is open to the development of more
MPM bridges coping with different multi-processing models but also to the development of
different approaches to resource consumption and workload balance. Currently we have 3 bridges:
</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">rivet_prefork_mpm.c: a bridge for the prefork MPM</li><li class="listitem">rivet_worker_mpm.c: a threaded bridge creating a pool of threads
each running Tcl interpreters and communicating with the worker MPM threads
through a thread safe queue. This bridge is needed by the worker MPM and by
any the winnt MPM on Windows.</li><li class="listitem">rivet_lazy_mpm.c: a threaded bridge where Tcl threads are
started <span class="quote">“<span class="quote">on demand</span>”</span>. The bridge creates no threads and Tcl interpreters
at start up, only when requests come in Tcl execution threads are created.
This bridge is explained in detail in the <a class="xref" href="lazybridge.html" title="Example: the “Lazy” bridge">the section called “Example: the <span class="quote">“<span class="quote">Lazy</span>”</span> bridge”</a>.
Since the resource demand at startup is minimal this bridge should suit
development machines that go through frequent web server restarts.</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4437"></a>mod_rivet MPM Bridge callbacks</h3></div></div></div><p style="width:90%">
A bridge is a loadable library implementing different ways to handle
specific features needed to mod_rivet. It was originally meant as a way
to handle the prefork/worker/event MPM specificities, at the same time
avoiding the need to stuff the code with conditional statements that would
have implied useless complexity (an instance of the Apache web server can
run only an MPM at a time), error prone programming and performance costs.
New bridges could be imagined also to implement different models of workload
and resource management (like the resources demanded by the Tcl interpreters).
We designed an interface between the core of mod_rivet and its MPM bridges
based on a set of functions defined in the rivet_bridge_table structure.
</p><pre class="programlisting">typedef struct _mpm_bridge_table {
RivetBridge_ServerInit *mpm_server_init;
RivetBridge_ChildInit *mpm_child_init;
RivetBridge_Request *mpm_request;
RivetBridge_Finalize *mpm_finalize;
RivetBridge_Exit_Handler *mpm_exit_handler;
RivetBridge_Thread_Interp *mpm_thread_interp;
} rivet_bridge_table;</pre><p style="width:90%">
</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="emphasis"><em>mpm_server_init</em></span>: pointer to any
specific server inititalization function. This field can be NULL
if no bridge specific initialization is needed. The core of
mod_rivet runs the <span style="font-family:monospace"><span class="command"><strong>ServerInitScript</strong></span></span> before
calling this function.
</li><li class="listitem"><span class="emphasis"><em>mpm_child_init</em></span>: Bridge specific
child process initialization. If the pointer is assigned with
a non-NULL value the function is called by Rivet_ChildInit.
</li><li class="listitem"><span class="emphasis"><em>mpm_request</em></span>: This pointer must
be a valid function pointer to the content generator
implemented by the bridge. If the pointer is not defined the Apache
web server will stop at start up. This condition is motivated by
the need of avoiding useless testing of the pointer. The fundamental
purpose of a content generator module (like mod_rivet) is to respond
to requests creating content, thus whatever it is
a content generating function must exist (during the early stages of
development you can create a simple test function for that). In a
threaded MPM this function typically prepares the request processing
stuffing somewhere the pointer to the request_rec structure
passed by the web server and then it calls some method to communicate
these data to the Tcl execution thread waiting for result to be
returned. The <span class="quote">“<span class="quote">prefork</span>”</span> bridge is an exception since there
are no threads and the bridge calls directly Rivet_SendContent
</li><li class="listitem"><span class="emphasis"><em>mpm_finalize</em></span>: pointer to a finalization
function called during a child process exit. This function is registered
as child process memory pool cleanup function. If the pointer is NULL
the pool is given a default cleanup function (apr_pool_cleanup_null)
defined in src/mod_rivet/mod_rivet.c. For instance the finalize function
in the <span class="emphasis"><em>worker</em></span> MPM bridge notifies
a supervisor thread demanding the whole pool of threads running Tcl
interpreters to orderly exit. This pointer can be NULL if the bridge
has no special need when a child process must exit (unlikely if you have
multiple threads running)
</li><li class="listitem"><span class="emphasis"><em>mpm_exit_handler</em></span>: mod_rivet replaces
the core <span style="font-family:monospace"><span class="command"><strong>exit</strong></span></span> command with a new one
(<span style="font-family:monospace"><span class="command"><strong>::rivet::exit</strong></span></span>). This command must handle
the process exit in the best possible way for the bridge and the
threading model it implements (for the 2 current threaded bridges this implies
signaling the threads to exit). The <span style="font-family:monospace"><span class="command"><strong>::rivet::exit</strong></span></span>
actually doesn't terminate the process, but interrupts execution
returning a specific error code commands <span style="font-family:monospace"><span class="command"><strong>::rivet::catch</strong></span></span>
and <span style="font-family:monospace"><span class="command"><strong>::rivet::try</strong></span></span> can detect. Before the process is terminated
the <span style="font-family:monospace"><span class="command"><strong>AbortScript</strong></span></span> script is fired and <span style="font-family:monospace"><span class="command"><strong>::rivet::abort_code</strong></span></span>
returns a message describing the exit condition. For instance
the <span class="emphasis"><em>worker</em></span> MPM bridge the finalize function
is called after the current thread itself is set up for termination.
See function Rivet_ExitCmd in
<a class="ulink" href="https://svn.apache.org/repos/asf/tcl/rivet/trunk/src/mod_rivet_ng/rivetCore.c" target="_top">rivetCore.c</a>
to have details on how and at what stage this callback is invoked.
</li><li class="listitem"><span class="emphasis"><em>mpm_thread_interp</em></span> must be a function returning
the interpreter object (a pointer to record of type
<span style="font-family:monospace"><span class="command"><strong>rivet_thread_interp</strong></span></span>) associated
to a given configuration as stored in a <span style="font-family:monospace"><span class="command"><strong>rivet_server_conf*</strong></span></span>
object. This element was temporarily introduced in the
<span style="font-family:monospace"><span class="command"><strong>mpm_bridge_table</strong></span></span> table and should be accessed
through the macro RIVET_PEEK_INTERP.
<pre class="programlisting">interp_obj = RIVET_PEEK_INTERP(private,private->conf);</pre>
Every bridge implementation should have its own way to store interpreter data and manage their
status. So this macro (and associated function) should hide from the module core function
the specific approach followed in a particular bridge
</li></ul></div><p style="width:90%">
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4471"></a>Server Initialization and MPM Bridge</h3></div></div></div><p style="width:90%">
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4474"></a>RivetChan</h3></div></div></div><p style="width:90%">
The <span class="structname">RivetChan</span> system was created in
order to have an actual Tcl channel that we could redirect
standard output to. This enables us use, for instance, the
regular <span style="font-family:monospace"><span class="command"><strong>puts</strong></span></span> command in .rvt pages. It
works by creating a channel that buffers output, and, at
predetermined times, passes it on to Apache's I/O system.
Tcl's regular standard output is replaced with an instance of
this channel type, so that, by default, output will go to the
web page.
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4479"></a>The <span style="font-family:monospace"><span class="command"><strong>global</strong></span></span> Command</h3></div></div></div><p style="width:90%">
Rivet aims to run standard Tcl code with as few surprises as
possible. At times this involves some compromises - in this
case regarding the <span style="font-family:monospace"><span class="command"><strong>global</strong></span></span> command. The
problem is that the command will create truly global
variables. If the user is just cut'n'pasting some Tcl code
into Rivet, they most likely just want to be able to share the
variable in question with other procs, and don't really care
if the variable is actually persistant between pages. The
solution we have created is to create a proc
<span style="font-family:monospace"><span class="command"><strong>::request::global</strong></span></span> that takes the place of
the <span style="font-family:monospace"><span class="command"><strong>global</strong></span></span> command in Rivet templates. If
you really need a true global variable, use either
<span style="font-family:monospace"><span class="command"><strong>::global</strong></span></span> or add the :: namespace qualifier
to variables you wish to make global.
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4487"></a>Page Parsing, Execution and Caching</h3></div></div></div><p style="width:90%">
When a Rivet page is requested, it is transformed into an
ordinary Tcl script by parsing the file for the <? ?>
processing instruction tags. Everything outside these tags
becomes a large <span style="font-family:monospace"><span class="command"><strong>puts</strong></span></span> statement, and
everything inside them remains Tcl code.
</p><p style="width:90%">
Each .rvt file is evaluated in its own
<code class="constant">::request</code> namespace, so that it is not
necessary to create and tear down interpreters after each
page. By running in its own namespace, though, each page will
not run afoul of local variables created by other scripts,
because they will be deleted automatically when the namespace
goes away after Apache finishes handling the request.
</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top">
One current problem with this system is that while
variables are garbage collected, file handles are not, so
that it is very important that Rivet script authors make
sure to close all the files they open.
</td></tr></table></div><p style="width:90%">
</p><p style="width:90%">
After a script has been loaded and parsed into it's "pure Tcl"
form, it is also cached, so that it may be used in the future
without having to reload it (and re-parse it) from the disk.
The number of scripts stored in memory is configurable. This
feature can significantly improve performance.
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4495"></a>Extending Rivet by developing C code procedures</h3></div></div></div><p style="width:90%">
Rivet endows the Tcl interpreter with new commands
serving as interface between the application layer and the
Apache web server. Many of these commands
are meaningful only when a HTTP request is under way and
therefore a request_rec object allocated by the framework
is existing and was passed to mod_rivet as argument of a callback.
In case commands have to gain access to a valid request_rec
object the C procedure must check if such
a pointer exists and it's initialized
with valid data. For this purpose the procedure handling requests
(Rivet_SendContent) makes a copy of such pointer and keeps it
in an internal structure. The copy is set to NULL just before
returning to the framework, right after mod_rivet's has
carried out its request processing. When the pointer copy is NULL
the module is outside any request processing and this
condition invalidates the execution of
many of the Rivet commands. In case they are called
(for example in a ChildInitScript, GlobalInitScript,
ServerInitScript or ChildExitScript) they fail with a Tcl error
you can handle with a <span style="font-family:monospace"><span class="command"><strong>catch</strong></span></span> command.
</p><p style="width:90%">
For this purpose in src/rivet.h the macro
CHECK_REQUEST_REC was defined accepting two arguments: the thread
private data object and the command name. If the pointer is NULL
the macro calls Tcl_NoRequestRec and returns TCL_ERROR
causing the command to fail. These are the steps to follow
in order to write a new C language command for mod_rivet
</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">
Define the command and associated C language procedure
in src/mod_rivet_ng/rivetCore.c using the macro
RIVET_OBJ_CMD<pre class="programlisting">RIVET_OBJ_CMD("mycmd",Rivet_MyCmd,private)</pre>
This macro ensures the command is defined as <span style="font-family:monospace"><span class="command"><strong>::rivet::mycmd</strong></span></span>
and its ClientData pointer is defined with the thread private data
</li><li class="listitem">
Add the code of Rivet_MyCmd to src/mod_rivet_ng/rivetCore.c (in case
the code resides in a different file also src/Makefile.am should be
changed to tell the build system how to compile the code and
link it into mod_rivet.so)
</li><li class="listitem">
If the code must have access to the request record in <span style="font-family:monospace"><span class="command"><strong>private->r</strong></span></span>
use the macro THREAD_PRIVATE_DATA in order to claim the thread private data, then
check for the validity of the pointer using the macro
CHECK_REQUEST_REC(private,"::rivet::<cmd_name>")
<pre class="programlisting">TCL_CMD_HEADER(Rivet_MyCmd)
{
/* we have to get the thread private data */
THREAD_PRIVATE_DATA(private)
/* if ::rivet::mycmd works within a request processing we have
* to check if 'private' is carrying a non null request_rec pointer
*/
CHECK_REQUEST_REC(private,"::rivet::mycmd");
....
return TCL_OK;
}</pre></li><li class="listitem">
Add a test for this command in tests/checkfails.tcl. For
instance
<pre class="programlisting">...
check_fail no_body
check_fail virtual_filename unkn
check_fail my_cmd <arg1> <arg2>
....</pre>
Where <arg1> <arg2> are optional
arguments in case the command has different forms depending on
the arguments. Then, if <span style="font-family:monospace"><span class="command"><strong>::rivet::mycmd</strong></span></span> must fail also
tests/failtest.tcl should modified as
<pre class="programlisting">virtual_filename->1
mycmd->1</pre>
The value associated to the test must be 0 in case the
command doesn't need to test the <span style="font-family:monospace"><span class="command"><strong>private->r</strong></span></span> pointer.
</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idm4519"></a>Debugging Rivet and Apache</h3></div></div></div><p style="width:90%">
If you are interested in hacking on Rivet, you're welcome to
contribute! Invariably, when working with code, things go
wrong, and it's necessary to do some debugging. In a server
environment like Apache, it can be a bit more difficult to
find the right way to do this. Here are some techniques to
try.
</p><p style="width:90%">
The first thing you should know is that Apache can be launched
as a <span class="emphasis"><em>single process</em></span> with the
-X argument:
</p><pre class="programlisting">httpd -X</pre>.
<p style="width:90%">
On Linux, one of the first things to try is the system call
tracer, <span style="font-family:monospace"><span class="command"><strong>strace</strong></span></span>. You don't even have to
recompile Rivet or Apache for this to work.
</p><pre class="programlisting">strace -o /tmp/outputfile -S 1000 httpd -X</pre><p style="width:90%">
This command will run httpd in the system call tracer,
which leaves its output (there is potentially a lot of it) in
<code class="filename">/tmp/outputfile</code>. The -S
option tells <span style="font-family:monospace"><span class="command"><strong></strong></span></span>strace to only record the
first 1000 bytes of a syscall. Some calls such as
<code class="function">write</code> can potentially be much longer than
this, so you may want to increase this number. The results
are a list of all the system calls made by the program. You
want to look at the end, where the failure presumably occured,
to see if you can find anything that looks like an error. If
you're not sure what to make of the results, you can always
ask on the Rivet development mailing list.
</p><p style="width:90%">
If <span style="font-family:monospace"><span class="command"><strong>strace</strong></span></span> (or its equivalent on your
operating system) doesn't answer your question, it may be time
to debug Apache and Rivet. To do this, you will need to rebuild mod_rivet.
First of all you have to configure the build by running the
<span style="font-family:monospace"><span class="command"><strong>./configure</strong></span></span> script with the
-enable-symbols option and after you have
set the CFLAGS and LDFLAGS environment variables
</p><pre class="programlisting">export CFLAGS="-g -O0"
export LDFLAGS="-g"
./configure --enable-symbols ......
make
make install</pre><p style="width:90%">
Arguments to <span style="font-family:monospace"><span class="command"><strong>./configure</strong></span></span> must fit your Apache HTTP
web server installation. See the output produced by
</p><pre class="programlisting">./configure --help</pre><p style="width:90%">
And check the <a class="xref" href="installation.html" title="Apache Rivet 3.2 Installation">the section called “Apache Rivet 3.2 Installation”</a> page to
have further information.
Since it's easier to debug a single process, we'll still run
Apache in single process mode with -X:
</p><pre class="programlisting">@ashland [~] $ gdb /usr/sbin/apache.dbg
GNU gdb 5.3-debian
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-linux"...
(gdb) run -X
Starting program: /usr/sbin/apache.dbg -X
[New Thread 16384 (LWP 13598)]
.
.
.</pre><p style="width:90%">
When your apache session is up and running, you can request a
web page with the browser, and see where things go wrong (if
you are dealing with a crash, for instance).
</p></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="help.html"><img src="images/prev.png" alt="Prev"></a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="lazybridge.html"><img src="images/next.png" alt="Next"></a></td></tr><tr><td width="40%" align="left" valign="top">Resources - How to Get Help </td><td width="20%" align="center"><a accesskey="h" href="index.html"><img src="images/home.png" alt="Home"></a></td><td width="40%" align="right" valign="top"> Example: the <span class="quote">“<span class="quote">Lazy</span>”</span> bridge</td></tr></table></div></body></html>
|
