1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Author" content="Jean-Luc Rochat">
<meta name="GENERATOR" content="vi">
<title>How to : Scalability - Load-Balancing - Fault-tolerance with Apache JServ 1.1</title>
</head>
<body bgcolor="#FFFFFF">
<p align="center"><a href="http://java.apache.org/" target="_top"><img SRC="images/java-apache-project.gif" BORDER="0" WIDTH="609"
HEIGHT="100" ALT="The Java Apache Project"></a></p>
<center>
<h1>
How to : Scalability - Load-Balancing - Fault tolerance</h1></center>
<center>
<h1>
with <a href="http://java.apache.org">Apache JServ</a> 1.1</h1></center>
<p><br>
<h4>
document version: 1.2.1 (03 Feb 2000)</h4>
<h3>
Summary</h3>
<ol>
<li>
<a href="#Introduction">Introduction</a></li>
<li>
<a href="#Changes from previous versions">Changes from previous versions</a></li>
<li>
<a href="#Features">Features</a></li>
<ol>
<li>
<a href="#Scalability">Scalability</a></li>
<ul>
<li>
<a href="#Apache">Apache Scalability</a></li>
<li>
<a href="#JServ">JServ Scalability</a></li>
</ul>
<li>
<a href="#Openess">Openess</a></li>
<li>
<a href="#Security">Security</a></li>
<li>
<a href="#balancing">Load-balancing</a></li>
<li>
<a href="#Session">Session handling</a></li>
<ul>
<li>
<a href="#Howsessionswork">How sessions work</a></li>
</ul>
<li>
<a href="#Faulttolerance">Fault tolerance</a></li>
<ul>
<li>
<a href="#noSPF">No Single point of failure</a></li>
<li>
<a href="#Scenario1">Apache fault tolerance</a></li>
<li>
<a href="#Scenario2">JServ fault tolerance</a></li>
</ul>
<li>
<a href="#Manageability">Manageability</a></li>
</ol>
<li>
<a href="#Configuration">Configuration</a></li>
<li>
<a href="#Internal">Internals</a></li>
<ol>
<li>
<a href="#Internal">Load-balancing algorithm</a></li>
<li>
<a href="#Watchdogprocess">Watchdog process</a></li>
<li>
<a href="#Sharedmemory">Shared memory</a></li>
</ol>
<li>
<a href="#Internalstate - Admistration tasks">Internal state - Admistration
tasks</a></li>
<li>
<a href="#Largesites">Large sites</a></li>
<li>
<a href="#Knownproblems">Known problems</a></li>
<li>
<a href="#FAQ">FAQ</a></li>
<li>
<a href="#Tips">Tips</a></li>
<li>
<a href="#Authors">Authors</a></li>
<br></NL>
<br>
<br>
<br>
<br>
<br>
<br>
<h2>
<a NAME="Introduction"></a>1. Introduction</h2>
Scalability, load-balancing, fault-tolerance are key issues for internet
content providers : As the market becomes more mature, the need for static
pages websites decreases, and the dynamic content generation, personnalized,
and transactionnal becomes a must.
<br>We strongly believe that Java is a key technology on the server
side, because of its rapid prototyping and validation phases, which give
a time-to-market bonus to those that are using it.
<p>However, as the Java Virtual Machines become faster over years, people
tend to ask for always more personnalized and dynamic contents, and this
type of application requires always more and more CPU power.
<a href="http://java.apache.org">Apache
JServ</a> addresses this requirement and lets you distribute your application
load over as many hosts as needed.
<p>At the same time, applications and user transactions are based on HTTP
sessions, and <a href="http://java.apache.org">Apache JServ</a> ensures
that the system will take care of them. Of course the whole system has
to be fault-tolerant.
<p>And of course all for free ... ;-)
<p>This document describes how to use <a href="http://java.apache.org">Apache
JServ</a> load-balancing and fault tolerance features.
<p>Thank you for using <a href="http://java.apache.org">Apache JServ</a>.
<br>
<br>
<h2>
<a NAME="Changes from previous versions"></a>2. Changes from previous versions</h2>
<br>This is just a partial list, within our current scope.
<br>1.1 Fixes security bugs present in all 1.0: a potential Denial of Service when using load-balancing !
<br>1.1 Fixes numerous bugs present in all 1.0: a potential file descriptor leak when using load-balancing !
<br>1.1 Offers enhanced configuration feedback on /jserv/ status page with load-balancing.
<br>1.1 Contributed sh and perl scripts to allow immediate or graceful shutdown of JServs in shared memory (still experimental).
<br>1.1 Load-balancing can work above ajpv11 AND the new ajpv12 at the same time.
<br>1.1 Load-balancing can also work with URL rewritten Sessions Ids.
<br>1.1 JServ is now bound to 1 IP address : Syntax: bindaddress=[ipaddress] or [localhost] (Default: localhost).
This is possibly a cause of non working installations.
<br>1.1 JServ's default protocol is now ajpv12. (update your config files).
<br>1.0, 1.1 Unix : Full Load balancing & fail-over support.
<br>1.0, 1.1 Unix : Fail-over optimized support based on shared memory & internal watchdog process (using the appropriate <a href="#Configuration">ApJServShmFile</a> directive).
<br>1.0, 1.1 Win32 : Full Load balancing & basic fail-over support
<br>1.0, 1.1 Win32 : Fail-over optimized support not supported.
<br>1.0b3 and above use new <a href="#Configuration">configuration</a>
parameters for load-balancing.
<br>
<h2>
<a NAME="Features"></a>3. Features</h2>
<ul>
<h3>
<a NAME="Scalability"></a>3.1. Scalability</h3>
As the server side application are growing, it becomes more and more difficult
to use the 2 Tier model (Web Server + ServerAPI), or worse the CGI model.
<a href="http://java.apache.org">Apache
JServ</a> is a 100%Java multithreaded servlet engine which uses a n tier
model, and scales of course better. It is scalable from very little sites
(my home site is running on a Pentium 120 MHz w/48MB RAM, Apache + JServ
running /Linux), to very large sites running lots of powerful machines.
<br>Between these 2 opposites, the <a href="http://java.apache.org">Apache
JServ</a> architecture lets you also choose between one 64-CPU machine,
and 64 low-cost PCs, with it's built-in load-balancing capabilities : It
can distribute the incoming load between lots of servlet engines.
<br>The key is the use of AJP protocol, which is a socket based protocol,
and permits several hosts running the 2 parts : Apache+mod_jserv on one
side, the servlet engine on the other side.
<p>Just as example : scalable from :
<ul>
<li>
1 PC, running 1 Apache + 1 JServ (from 1 to z zones) (my home PC)</li>
</ul>
<p><br>To (OK, let's take one realistic example) :
<br>
<ul>
<li>
10 hosts running Apache. (load-balancing with round-robin DNS / or dedicated
hardware)</li>
<li>
33+ JServs running on top of :</li>
<ul>
<li>
15 PC running Linux JDK1.2</li>
<li>
5 Sun E450</li>
<li>
10 PC NT JDK1.1.6</li>
<li>
3 AS400</li>
<li>
+ ... (What fits your needs)</li>
</ul>
</ul>
<h4>
<a NAME="Apache"></a>3.1.1. Scaling Apache HTTP/HTTPS servers :</h4>
Every standard Apache server can start 1..256 httpd processes (This limit
can be increased at compilation time).
<br>This means that one Apache server can serve at the same time a maximum
of 256 client requests.
<br>For websites that have to serve thousands of concurrent request, there
is a need for running multiple occurences of the same server, and to use
a load-balancing mechanism to distribute the incoming load over the different
Apache instances.
<br>The different instances of Apache server can be running on the same
host, each of them using a different IPaddr/TCP port couple, or on different
hosts.
<br>The load-balancing mechanism can be either a simple round robin DNS,
either a dedicated hardware. This is a deployment decision, and doesn't
have impacts on Apache and JServ. We do not rely on this load-balancing
level for keeping care of our applicative sessions.
<br>Some of these Apache servers can be SSL enabled, and SSL sessions have
of course to be maintained through the load-balancing system. Dedicated
hardwares usually take care of this.
<br>As we can see our Apache servers as gateways for our JServ servlet
engines, we can easily imagine that all these apache servers will be seen
by clients as a "cluster" of hosts, with a unique hostname, and so will
always receive session cookies set for this "cluster" hostname. This will
allow keeping HTTP sessions over SSL/not SSL URLs.
<br>We will see later that these different Apache servers have to share
a common knowledge about JServ identification (routing & TCP infos).
<br>
<h4>
<a NAME="JServ"></a>3.1.2. Scaling JServ servers :</h4>
Jserv is a 100% pure Java program, where every servlet is running in it's
own thread. With a very large number of concurrent requests, it seems reasonnable
to run multiple instances of JServ for the following reasons :
<ul>
<li>
No JVM is today able of running thousands of (doing something really useful)
threads at the same time.</li>
<br>AFAIK.
<li>
Some Java classes have synchronized methods. Increasing the number of JServs
increases the parallelism.</li>
<li>
Redundancy increases system safety. If one JVM crashes, or if one servlet
produces an System.exit(), or anything bad happens, others Apache JServ
will be still alive.</li>
</ul>
<p><br>Servlet zones can be distributed on different JServs. This increases
security, as these Jservs can be started with different userid.
<ul>
<li>
every JServ can be started with it's own CLASSPATH.</li>
<li>
every JServ can be started with it's own JDK version, or JRE option (-nojit,
- verbosegc, ...).</li>
</ul>
One JServ is able to run on one 64*CPU machine, and any number (64, 128,
other) of JServs can run on 64 low-cost PCs.
<br>And yes, you can run more than one JServ on one machine, as each of
them needs only a IPaddr/TCP port couple. Can be interesting for above
reasons.
<br>We currently have one limit in the Apache plug-in (mod_jserv) for the
max number of JServs it can address. This limit is fixed at compile time
(can be increased), and is just here for reserving space in share memory.
(limit = 25 rigth now).
<br>
<h3>
<a NAME="Openess"></a>3.2. Openess</h3>
While Jserv is a 100% Java code, it can run on any system that has
a JVM. So you can have at the same time some of your JServs running on
top of OS/2, and others on Linux, Solaris, NT, ...
<br>Apache itself can be running on a different hardware/Operating system,
and just sending request to JServs using Apache Java Protocol (AJP) over
TCP/IP.
<br>Apache is known for running efficiently over freeware OSes like *BSD
or Linux.
<br>
<h3>
<a NAME="Security"></a>3.3. Security</h3>
<ul>
<li>
You can run Apache & JServ on same/different hosts with different userids
and rights.</li>
<li>
You can separate different zones on different JServs.</li>
<li>
JServ can be run behind a firewall (AJP protocol uses only one TCP port)</li>
<li>
JServ uses an ACL and allows AJP requests only from ACL'ed hosts.</li>
<li>
The communication can be authenticated with a secret key between Apache
& JServ.</li>
<li>
Apache+SSLeay allows 128bits SSL encryption with https URLs.(between the
browser and the http server).</li>
<li>
Server redundancy (duplicating both Apache & JServ) increases the service
availability.</li>
<li>
Apache + JServ does not rely on any external component. There is no single
point of failure (SPF).</li>
</ul>
<br>
<br>
<h3>
<a NAME="balancing"></a>3.4 Load-balancing</h3>
<ul>
<li>
Level 0 :</li>
<br>1 host with Apache, 1 host with JServ. The JServ is hosting all servlet
zones.
<br>This is the current default mode for Apache JServ (no load-balancing).
JServ can be started in <b>automatic</b> mode.
<li>
Level 1 :</li>
<br>1 host with Apache, n hosts with JServ. Every JServ is hosting its
own servlet zone.This is a possible mode in 1.0 versions. You have to specify
one JServ/zone. Apache cannot start the different JServ in automatic mode
: you have to set the Manual mode to true.
<li>
Level 2 :</li>
<br>1 host with Apache, m*n hosts with JServ.
<br>Every zone can have the incoming load distributed on several JServs.
You have to use the new "<b>balance</b>" parameters. (see <a href="#Configuration">Configuration</a>).
<br>For every zone you have to define a set of JServ hosts that are identical
(same servlet classes), and affect a logical weight for every host in that
set.
<br>The traffic will be balanced between the JServs in the set, using a
simple but efficient algorithm based on host weight + random.
<br>Every httpd process (inside one Apache server) will randomly(modulo
weight) choose a default target for each zone, and send new requests (sessions
not yet created) to this target. The underlying Operating system guaranties
(at least on U**xes) that equivalent processes gently share the CPU ressources,
and get elected at their turn.
<li>
Level 3 :</li>
<br>p*hosts with Apache, m*n hosts with JServ. Same as Level 2, but all
our p Apache servers have to use same configuration files to keep sessions
alive across all of them (really mandatory: use the same <a href="#Configuration">routing</a>
parameters).</ul>
<br>
<h3>
<a NAME="Session"></a>3.5. Session handling</h3>
Once established, a session is bound to one particular JServ. It is possible
to ensure that the next request (in the same http session) will be sent
to the right JServ, wherever the request came from (means : any Apache
used). Cookies are used to identify sessions, and cookies contains the
identifier of the JServ who did set it.
<br>This ensures that sessions are fault-tolerant to Apache failure.
<h4>
<a NAME="Howsessionswork"></a>3.5.1 How sessions work</h4>
Sessions are based on session cookie set by servlets.
<br>Every time a request comes for a balanced ServletMountPoint, mod_jserv
chooses (see algorithm above) which JServ will be used.
<br>mod_jserv adds the cookie trailer (the ApJServRoute = eg: JS3 for PC3)
to the environment variables sent to JServ.
<br>JServ appends this cookie trailer to the end of session cookie, before
sending it back to the client browser.
<br>Next hit, some httpd (same host as precedent or not) will get the request,
examine headers, find the session cookie, determine which JServ owns
the cookie trailer, and then sendthe request to thez correct servlet engine.
<br>If more than one server runs Apache for a given domain (round-robin
DNS), all the Apache MUST have the same routing parameters.
<br>
<h3>
<a NAME="Faulttolerance"></a><b>3.6 Fault tolerance</b></h3>
<h4>
<a NAME="noSPF"></a>3.6.1 No single point of failure</h4>
If you have more than one Apache host, one of them can stop, and the system
will still work (modulo SSL sessions for https servers).
<br>It is up to you to provide a load-balancing/fail-over mechanism (round-robin
DNS or dedicated hardware) to be able to use fail-over on Apache. (But
we ensure that user sessions will survive a Apache crash).
<p>If you have more than one JServ host, one of them can stop, and the
system will still work. (modulo broken sessions).
<p>Any Apache is able to route a request to any JServ. (session are
maintained, does not rely on any of the following elements : load-balancer
hardware, or Apache server).
<br>As long as you have one Apache and one JServ running, the system can
work.
<p>Fault-tolerance is implicit if load-balancing is enabled for this zone.
<br>All our httpd processed share a memory zone, which contains the last
known status for every JServ.
<br>As soon as a JServ is marked "unavailable" by one httpd process, the
information is accessible by other processes on the same host. This prevents
repetitive connect failures or even TCP connect timeouts.
<br>The request & following requests will be redirected to others JServ
in the same "set" of JServs.
<br>A watchdog program will run silently, and try to connect to this JServ
until success. Once succeeded, the JServ will be accessed again by httpd
processes.
<br>The session is broken if a request is redirected to another JServ.
The existing one is considered invalid, as sessions can not travel across
the network (not in Servlet API specs).
<br>
<h4>
<a NAME="Scenario1"></a>3.6.2. Scenario 1: Apache fault-tolerance</h4>
<p><br>We have 2 Apache servers on 2 different hosts.
<br><b>1 - first step</b>
<br>a. Web client requests a page from www.jserv.com, port 80
<br>b. www.jserv.com is a load-balancing system which actually resolves
the request to go to server Httpd server 111.222.333.10, port 3000.
<br>c. The Apache Httpd server listening on 111.222.333.10, port 3000 chooses
(at random) a Jserv machine, 192.168.0.51, port 8885, to handle the request.
<br>d. The Jserv machine at 192.168.0.51, port 8885 responds to the request
with the content of the page along with a cookie with name JServSessionID
and value "xxxx-JS1".
<p><b>2 - second step</b>
<br>a. Web client requests another page from www.jserv.com, port 80
<br>b. www.jserv.com resolves to 111.222.333.20, port 3000 (a different
machine from last hit time).
<br>c. The Apache server recognizes the JServSessionID cookie and looks
at the Jserv identifier, "JS1" at the end of the cookie.
<br>d. The Apache server sees that JS1 is the identifier for 192.168.0.51,
port 8885, and passes the request to that same server.
<br>
<br> </ul>
<img SRC="images/lbprod001.gif" height=480 width=640>
<ul>
<br>
<br>
<br>
<h4>
<a NAME="Scenario2"></a>3.6.3. Scenario 2 : JServ load-balancing &
fault tolerance</h4>
<b>1 - first step : JServ</b> <b>load-balancing</b>
<br>a. client A requests a servlet (A1)
<br>b. the httpd server chooses a target JServ1 (A'1)
<br>c. result: a cookie is set for JS1.
<br>d. client B requests a servlet (B1)
<br>e. the httpd server chooses a target JServ2 (B1')
<br>f. result: a cookie is set for JS2.</ul>
<img SRC="images/lbfail001.gif" height=480 width=640>
<ul><b>2 - second step : session handling</b>
<br>a. client B requests a servlet (and send the previously set cookie
JS2) (B2)
<br>b. the httpd server recognizes the cookie JS2
<br>c. the request is passed to JServ2 (B2')</ul>
<p><br><img SRC="images/lbfail002.gif" height=480 width=640>
<ul>JServ1 now dies in a horrible death ...
<p><b>3 - third step : JServ fault tolerance</b>
<br>a. client A requests a servlet (and send the previously set cookie
JS1) (A2).
<br>b. the httpd server recognizes the cookie JS1.
<br>c. the request is passed to JServ1, resulting in a failure (A2')
<br> (the httpd process marks JServ1 "dead" in shared memory).
<br>d. the http server finds a backup and sends the request to JServ3 (A2'')
<br>e. result of A2'': If a session is needed, a new one is created and a cookie JS3 is set. (JS1 cookie erased).
<br>
<br> </ul>
<img SRC="images/lbfail003.gif" height=480 width=640>
<ul>
<br>
<h3>
<a NAME="Manageability"></a>3.7. Manageability</h3>
All information about JServ state is accessible and updatable in the shared
<br>You can stop one of your JServs for maintenance reasons, without denying
access to your customers, as the requests will be sent to the other JServs
inside the set.
<br>An administrator can update the status of one particular JServ in shared
memory, and mark it SHUTDOWN, so our httpd processes will notice it, and
redirect the incoming requests to other JServs.
<br>Monitoring tasks can read the JServ status in shared memory, and send
reports. (can even be protected CGI programs).
<br>
<br>
</ul>
<h3>
<a NAME="Configuration"></a>4. Configuration
</h3>
<p><b>Nothing special to do on the Java side. NO new parameter in properties
file.</b>
<br><b>Administrator installs JServ and then chooses to use load-balancing
or not from the Apache side.</b>
<br>
<h4>
1 - defining hosts and routing parameters (<font color="#FF0000">in red</font>)</h4>
Requests with cookies containing .JS2 will be routed to PC2 using protocol ajpv11
on port 8888 host 192.168.0.52.<br> NB: PC2 and PC3 are using ajpv11. PC1 and SPARK are using ajpv12.
<br>Ajpv12 is the new (since v1.1) default protocol and using 2 protocols at the same time is maybe source of errors.
<br>
<h4>
2 - defining hosts weight (<font color="#FF6600">in orange</font>)</h4>
<p><br><i><font size=-1><font color="#000000">ApJServBalance set1
SPARK </font><font color="#FF6600">4</font></font></i>
<p>Default weight is = 1. This one (SPARK) is a 4*CPU engine.
<br>
<h4>
3 - defining set (here called set1) of equivalent JServs (<font color="#006600">in
green</font>)</h4>
Set "set1" contains PC1 + PC2 + PC3 + SPARK
<br>
<h4>
4 - defining load-balanced servlet mount point (<font color="#3333FF">in
blue</font>)</h4>
Requests for /servlet will be load-balanced on set "set1".
<br>SPARK should receive 4 times(average) more requests than others.
<p>If one of the JServs fails; requests will be redirected to other members
in set "set1".
<br>
<h4>
5 - defining the shared memory file (<font color="#CC33CC">in pink</font><font color="#000000">)</font></h4>
Defines the path of the file that will be used as shared memory between
processes (needs read+write+create rights for the JServ's userid).
<br>This can be an absolute or relative file path.
<p><i><font color="#CC33CC"><font size=-1>ApJServShmFile log/jserv_shm</font></font></i>
<br>
<p><i><font size=-1><IfModule mod_jserv.c></font></i>
<br><i><font size=-1>####################################################</font></i>
<br><i><font size=-1>#
<a href="http://java.apache.org">Apache JServ</a> Configuration File
#</font></i>
<br><i><font size=-1>####################################################</font></i>
<br><i><font size=-1># Note: this file should be appended to httpd.conf</font></i>
<br><i><font size=-1>ApJServManual on</font></i>
<p><i><font size=-1>ApJServMount /oldservlet ajpv12://192.168.0.1:7777/zone2</font></i>
// old style config
<p><i><font color="#3366FF"><font size=-1>ApJServMount /servlet balance://set1/zone1</font></font></i>
<p><i><font color="#006600"><font size=-1>ApJServBalance set1 PC1</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 PC2</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 PC3</font></font></i>
<br><i><font size=-1><font color="#006600">ApJServBalance set1 SPARK
</font><font color="#FF6600">4</font></font></i>
<p><i><font color="#CC0000"><font size=-1>ApJServHost PC1 ajpv12://192.168.0.51:7777</font></font></i>
<br><i><font color="#CC0000"><font size=-1>ApJServHost PC2 ajpv11://192.168.0.52:8888</font></font></i>
<br><i><font color="#CC0000"><font size=-1>ApJServHost PC3 ajpv11://192.168.0.53:9999</font></font></i>
<br><i><font color="#CC0000"><font size=-1>ApJServHost SPARK ajpv12://192.168.0.54:7777</font></font></i>
<p><i><font color="#CC0000"><font size=-1>ApJServRoute JS1 PC1</font></font></i>
<br><i><font color="#CC0000"><font size=-1>ApJServRoute JS2 PC2</font></font></i>
<br><i><font color="#CC0000"><font size=-1>ApJServRoute JS3 PC3</font></font></i>
<br><i><font color="#CC0000"><font size=-1>ApJServRoute sp1 SPARK</font></font></i>
<p><i><font color="#CC33CC"><font size=-1>ApJServShmFile log/jserv_shm</font></font></i>
<p><i><font size=-1>ApJServSecretKey DISABLED</font></i>
<br><i><font size=-1></IfModule></font></i>
<br>
<h3>
<a NAME="Internal"></a>5. Internal</h3>
<h4>
<a NAME="Sharedmemory"></a>5.1 Load-balancing algorithm</h4>
All the job is done by mod_jserv.
<p>When Apache (re)starts, the configuration file (httpd.conf) is parsed.
For each set (of servlet engines), a circular list of engines is created.
Every engine is inserted exactly n times, where n is the weight as described
above.
<br>In our example, the circular list contains :
<br>PC1 - PC2 - PC3 - SPARK - SPARK - SPARK - SPARK
<br>
<pre>for each HTTP request coming for a balanced Servlet mount point:
do
if is the first HTTP request for a balanced Servlet mount point
then
process_mount_default_target = randomly chosen in set
endif
if a session cookie is found
then
the process finds the JServ which owns the session (using ApJServRoute parameters)
if the JServ is not dead/stopped (in shared memory)
then
the process sends the request to this JServ.
if the JServ replies
then
return
else
mark the JServ dead (in shared memory)
endif
endif
endif
# here we have either got a request without session cookie
# or a broken session. In this last case, we send te request
# to another target in the list without saying it to the client.
# (the application will have to notice the broken session anyway)
if process_target != process_mount_default_target AND
our process_mount_default_target is alive (in shared memory)
then
process_target = process_mount_default_target
endif
while (process_target exists)
do
if process_target is alive (in shared memory)
then
the process sends the request to this JServ.
if the JServ replies
then
return
else
mark the JServ dead (in shared memory)
endif
endif
process_target = next in set
done
done</pre>
<h4>
<a NAME="Watchdogprocess"></a>5.2. Watchdog process</h4>
At Apache startup a process is created. This process is a httpd process,
which doesn't listen to HTTP requests. This process wakes up every 10
sec, reads the shared memory, and tries to connect to dead JServs. When
a dead JServ replies, it is marked running in shared memory.
<pre>while true
do
sleep(10)
if this process is not the default watchdog (in shared memory )
break
endif
for each JServ present in shared memory list
do
if JServ.state = down
connect JServ
if failed
continue
else
mark it alive in shared memory
endif
endif
done
done
</pre>
<h4>
<a NAME="Sharedmemory"></a>5.3. Shared memory</h4>
It is used to communicate between httpd processes.
<br>Every httpd process opens a regular file on the disk (open read+write),
and then mmaps it. This makes the file appear as regular memory.
<br>As many processes can do the same at the same moment, modifications
in this memory zone are immediately seen by other processes.
<br>This is a very convenient way to share information about server state
between httpd processes, and avoids a lot of infructuous connexions when
a JServ is down.
<br>This shared memory can be read/written by any process (according to
file permissions), and we can just imagine CGI programs, SNMP proxies,
inetd started programs than can run and access this shared memory, mainly
for admistration/monitoring tasks.
<br>The information stored in this shared memory is NOT MT-safe, not MP-safe
: accesses are NOT synchronized.
<br>Why ? Well, the fault-tolerance part doesn't need it (really not).
This would be just annoying if we wanted to share counters (nb of hits
by example : the LB algorithm could use it to distribute the load, but
this would work only for 1 Apache/N JServs. Well, random+weight is not
a so bad choice, and easier to implement ;-)
<br>This file is not human readable/writable. use a C/Perl program to access
it. (structure is in jserv.h).
<p><img SRC="images/lbshm001.gif" height=480 width=640>
<ul>
<h3>
<a NAME="Internalstate - Admistration tasks"></a>6. Internal state - Admistration
tasks</h3>
Every JServ is described in shared memory. One parameter is the server
state. This state can be :
<ul>
<li>
UP (internally '+') : means this JServ is running.</li>
<li>
DOWN (internally '-') : means JServ doesn't answer. (Watchdog is allowed
to change the state back to UP)</li>
<li>
SHUTDOWN_IMMEDIATE (internally 'X') : JServ is stopped by administator,
no traffic allowed.</li>
<li>
SHUTDOWN_GRACEFUL (internally '/') : JServ is stopped by administrator,
no new traffic allowed, but existing sessions still allowed. (example:
will be stopped in 10 minutes).</li>
</ul>
<p>Before an non urgent administration task, the administrator changes
the state from UP to SHUTDOWN_GRACEFUL, and then wait some minutes, before changing again the state to SHUTDOWN_IMMEDIATE and perform the task. How long should he wait is depending of your application.
<p>Once set to SHUTDOWN*, the watchdog doen't attempt to reconnect and set it up again.
<p>Before an urgent administration task, the administrator changes
the state from UP to SHUTDOWN_IMMEDIATE, and then performs the task. This ensures that no request
will be routed to this JServ.
<p>If the web site has 10 Apache, he has to modify the shared memory file on each of them.
<p><br>After performing an administration task, the administrator changes
the state from SHUTDOWN_IMMEDIATE to UP back.
<h4>How to read/modify the shared memory file ?</h4>
This file contains binary and readable datas. One needs a home-made program (write it in C, or Perl), maybe we'll provide one example later. emacs or vi, or even Win32 Notepad are not well suited for this task.
<br>You have to write your own tool, if you want to access/modify this shared memory. It is not more complicated than reading a regular file, and you can use the src/modules/jserv/jserv_mmap.c as an example.
<br><b>New !</b> scripts have been contributed (experimental, do it at your own risks) on the web site : search the FAQ (search key= "shared memory").
Thanks to Mark Cox.
<br><b>New !</b> The internal state of every JServ is also displayed in the /jserv/ status page on your website : somewhere like <A href=http://localhost/jserv/>http://localhost/jserv/</A> .
The status is present & updated in shared memory only after opening a connection on the servlet engine !<br>
<h3>
<a NAME="Largesites"></a>7. Large sites</h3>
It is possible to create very large sites with Apache+JServ without problems
and still no single point of failure. The basic idea is to create different sets
of JServs per Apache, rather than having n (number of JServs) possible choices for every Apache. This is just a limitation to the built-in load-balancing capabilities.
<br>Let's say that Apache1 will have 5 JServs in its set : <i>JServ1, JServ2,
JServ3, JServ4, JServ5</i>.
<br>Apache2 will have 5 other JServs in its own set (for the same zone):
<i>JServ11, JServ12, JServ13, JServ14, JServ15</i>.
<br>We assume that you have a Round Robin DNS that distributes statically
the load for hostname www.jserv.org on 2 IP addresses PLUS a fail-over
equipment (some exist that can do both).
<ul>
<li>
A client connects and obtains IP1 from the DNS for www.jserv.com.</li>
<li>
The sequest is sent to Apache1.</li>
<li>
Apache1 uses its load-balancing algorithm and chooses a JServ from its
set. (Let's say JServ1)</li>
<li>
JServ1 sets a (JServSessionId=xxxx.JS1) cookie to the client's browser.</li>
<li>
Following requests from this client, (session establihed) coming to Apache1
are routed to JServ1.</li>
<li>
Apache1 dies.</li>
<li>
Hopefully (thanks to our load-balancing equipment) the request comes on
Apache2.</li>
<li>
The cookie is recognized (JServSessionId=xxxx.JS1) and Apache2 sends the
request to JServ1.</li>
<li>
Every Apache never sends requests outside its own set, except for established
sessions.</li>
</ul>
<img SRC="images/lbprod002.gif" height=480 width=640>
<br><i><font size=-1>####################################################</font></i>
<br><i><font size=-1>#
<a href="http://java.apache.org">Apache JServ</a> Configuration File
for Apache A
#</font></i>
<br><i><font size=-1>####################################################</font></i>
<br><i><font size=-1># Note: this file should be appended to httpd.conf</font></i>
<br><i><font size=-1>ApJServManual on</font></i>
<p><i><font color="#000000"><font size=-1>ApJServMount /servlet balance://set1/zone1</font></font></i><i><font color="#3366FF"><font size=-1></font></font></i>
<p># my own set of JServs for servlet zone "zone1"
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ1</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ2</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ3</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ4</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ5</font></font></i>
<p><i><font color="#000000"><font size=-1>ApJServHost JServ1 ajpv12://192.168.0.51:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ2 ajpv12://192.168.0.52:8888</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ3 ajpv12://192.168.0.53:9999</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ4 ajpv12://192.168.0.54:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ5 ajpv12://192.168.0.55:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ11 ajpv12://192.168.0.61:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ12 ajpv12://192.168.0.62:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ13 ajpv12://192.168.0.63:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ14 ajpv12://192.168.0.64:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ15 ajpv12://192.168.0.65:7777</font></font></i><font color="#000000"></font>
<p><i><font color="#000000"><font size=-1>ApJServRoute JS1 JServ1</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS2 JServ2</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS3 JServ3</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS4 JServ4</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS5 JServ5</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ11</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ12</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ13</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ14</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ15</font></font></i><font color="#000000"></font>
<p><i><font color="#000000"><font size=-1>ApJServShmFile log/jserv_shm</font></font></i><font color="#000000"></font>
<p><i><font color="#000000"><font size=-1>ApJServSecretKey DISABLED</font></font></i>
<br><i><font size=-1></IfModule></font></i><i><font size=-1></font></i>
<p><i><font size=-1>####################################################</font></i>
<br><i><font size=-1>#
<a href="http://java.apache.org">Apache JServ</a> Configuration File
for Apache B
#</font></i>
<br><i><font size=-1>####################################################</font></i>
<br><i><font size=-1># Note: this file should be appended to httpd.conf</font></i>
<br><i><font size=-1>ApJServManual on</font></i>
<p><i><font color="#000000"><font size=-1>ApJServMount /servlet balance://set1/zone1</font></font></i><i><font color="#3366FF"><font size=-1></font></font></i>
<p># my own set of JServs for servlet zone "zone1"
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ11</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ12</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ13</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ14</font></font></i>
<br><i><font color="#006600"><font size=-1>ApJServBalance set1 JServ15</font></font></i>
<p><i><font color="#000000"><font size=-1>ApJServHost JServ1 ajpv12://192.168.0.51:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ2 ajpv12://192.168.0.52:8888</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ3 ajpv12://192.168.0.53:9999</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ4 ajpv12://192.168.0.54:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ5 ajpv12://192.168.0.55:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ11 ajpv12://192.168.0.61:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ12 ajpv12://192.168.0.62:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ13 ajpv12://192.168.0.63:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ14 ajpv12://192.168.0.64:7777</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServHost JServ15 ajpv12://192.168.0.65:7777</font></font></i><font color="#000000"></font>
<p><i><font color="#000000"><font size=-1>ApJServRoute JS1 JServ1</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS2 JServ2</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS3 JServ3</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS4 JServ4</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS5 JServ5</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ11</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ12</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ13</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ14</font></font></i>
<br><i><font color="#000000"><font size=-1>ApJServRoute JS11 JServ15</font></font></i><font color="#000000"></font>
<p><i><font color="#000000"><font size=-1>ApJServShmFile log/jserv_shm</font></font></i><font color="#000000"></font>
<p><i><font color="#000000"><font size=-1>ApJServSecretKey DISABLED</font></font></i>
<br><i><font size=-1></IfModule></font></i>
<h3>
<a NAME="Knownproblems"></a>8. Known problems</h3>
Avoid using big weights for servlet engines, as they are inserted exactly
this number of times in the set list. (weight=800 for a 2*400MHz CPU is
not a good idea).
<p>The maximum number of active JServs is limited to 25. To increase it,
just increase the NB_MAX_JSERVS in jserv.h and recompile.
<p>Oops ! forgot this one : The full functionnality is working on U**xes
(tested on Solaris/Linux). But the NT port is only partial : no watchdog
for the moment. Volunteers to work on this ?
<br>Today we have load-balancing + but NO shared memory on NT.
<p>The Watchdog process is created (forked) at server startup by the main
program (Unix only), so it inherits (too much IMHO) its characteristics : runs with
Apache's main process userid.
<br>(must be root for TCP ports < 1024). This is potentially dangerous,
and even if exploits can be difficult to create (as this process does'nt
listen HTTP requests), it has to be fixed ASAP.
<h3>
<a NAME="FAQ"></a>9. FAQ</h3>
Q: I can't understand how the load is distributed on different JServs.
<br>A: How are the request dispatched to different httpd processes by the
system ? Every process has a preferred target, chosen randomly, and the
operating system dispatches requests to our processes. So, one can understand
this, but not predict where the next request will go !
<p>Q: One of the JServs get twice the load of others
<br>A: Yes, due to the randomly chosen target, some of the JServs can be
used as default more often.
<br>In order to get a correct distribution, you need to run lots of httpd
processes. This is designed for big sites, so ...
<p>Q: My ApJServShmFile is corrupted
<br>A: Remove it. Start Apache.
<br>
<br>
<h3>
<a NAME="Tips"></a>10. Tips</h3>
<h4>
Desactivating load-balancing</h4>
You can choose not to use load-balancing for part or all your system.
<br>In the above <a href="#Configuration">configuration</a> example we
had :
<p><i><font size=-1>ApJServMount /svlet1 ajpv12://192.168.0.1:7777/sv
<<<<< this one is NOT load balanced</font></i>
<br><i><font color="#3366FF"><font size=-1>ApJServMount /servlet balance://set1/zone1
<<<<< this one IS.</font></font></i>
<h4>
using load-balancing without shared memory nor watchdog</h4>
It is possible to use load-balancing without shared memory & watchdog.
<br>Just comment the <i><font color="#CC33CC"><font size=-1>ApJServShmFile
</font></font></i>line
(from httpd.conf):
<p><i><font color="#CC33CC"><font size=-1>######ApJServShmFile log/jserv_shm</font></font></i>
<p><i><font color="#000000"><font size=-1>Still using the load-balancing
syntax :</font></font></i>
<p><i><font color="#3366FF"><font size=-1>ApJServMount /servlet balance://set1/zone1</font></font></i>
<br><i><font color="#3366FF"><font size=-1>.../...</font></font></i>
<p>and the shared memory will NOT be used between httpd processes, and
you'll get NO watchdog to detect JServs that get alive again.
<br>
<h4>
Removing all load-balancing code</h4>
Just comment the line :
<br>#define LOAD_BALANCE in src/modules/jserv/jserv.h
<br>and recompile JServ+Apache :
<br>make clean; make; make install
<h3>
<a NAME="Authors"></a>11. Authors</h3>
Bernard Bernstein <bernard@corp.talkcity.com>
<br>Jean-Luc Rochat <jlrochat@jnix.com>
<center>
<p><font size=-1>Copyright (c) 2000 <a href="http://java.apache.org/" target="_top">The
Java Apache Project</a>.</font>
<br><font size=-1>$Id: howto.load-balancing.html,v 1.0 1999/04/20 16:05:12
<a href="mailto://jlrochat@jnix.com">Jean-Luc
Rochat</a> Exp $</font>
<br><font size=-1>All rights reserved.</font></center>
</ul>
</ol>
</body>
</html>
|
