1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<HTML>
<HEAD>
<META NAME="GENERATOR" CONTENT="LinuxDoc-Tools 0.9.21">
<TITLE>SQUID Frequently Asked Questions: Operational issues</TITLE>
<LINK HREF="FAQ-8.html" REL=next>
<LINK HREF="FAQ-6.html" REL=previous>
<LINK HREF="FAQ.html#toc7" REL=contents>
</HEAD>
<BODY>
<A HREF="FAQ-8.html">Next</A>
<A HREF="FAQ-6.html">Previous</A>
<A HREF="FAQ.html#toc7">Contents</A>
<HR>
<H2><A NAME="s7">7.</A> <A HREF="FAQ.html#toc7">Operational issues</A></H2>
<H2><A NAME="ss7.1">7.1</A> <A HREF="FAQ.html#toc7.1">How do I see system level Squid statistics?</A>
</H2>
<P>The Squid distribution includes a CGI utility called <EM>cachemgr.cgi</EM>
which can be used to view squid statistics with a web browser.
This document has a section devoted to <EM>cachemgr.cgi</EM> usage
which you should consult for more information.</P>
<H2><A NAME="ss7.2">7.2</A> <A HREF="FAQ.html#toc7.2">How can I find the biggest objects in my cache?</A>
</H2>
<P>
<PRE>
sort -r -n +4 -5 access.log | awk '{print $5, $7}' | head -25
</PRE>
</P>
<H2><A NAME="cleancache"></A> <A NAME="ss7.3">7.3</A> <A HREF="FAQ.html#toc7.3">I want to restart Squid with a clean cache</A>
</H2>
<P><EM>Note: The information here is current for version 2.2 and later.</EM></P>
<P>First of all, you must stop Squid of course. You can use
the command:
<PRE>
% squid -k shutdown
</PRE>
</P>
<P>The fastest way to restart with an entirely clean cache is
to over write the <EM>swap.state</EM> files for each <EM>cache_dir</EM>
in your config file. Note, you can not just remove the
<EM>swap.state</EM> file, or truncate it to zero size. Instead,
you should put just one byte of garbage there. For example:
<PRE>
% echo "" > /cache1/swap.state
</PRE>
Repeat that for every <EM>cache_dir</EM>, then restart Squid.
Be sure to leave the <EM>swap.state</EM> file with the same
owner and permissions that it had before!</P>
<P>Another way, which takes longer, is to have squid recreate all the
<EM>cache_dir</EM> directories. But first you must move the existing
directories out of the way. For example, you can try this:
<PRE>
% cd /cache1
% mkdir JUNK
% mv ?? swap.state* JUNK
% rm -rf JUNK &
</PRE>
Repeat this for your other <EM>cache_dir</EM>'s, then tell Squid
to create new directories:
<PRE>
% squid -z
</PRE>
</P>
<H2><A NAME="ss7.4">7.4</A> <A HREF="FAQ.html#toc7.4">How can I proxy/cache Real Audio?</A>
</H2>
<P>by
<A HREF="mailto:roever@nse.simac.nl">Rodney van den Oever</A>,
and
<A HREF="mailto:jrg@blodwen.demon.co.uk">James R Grinter</A></P>
<P>
<UL>
<LI>Point the RealPlayer at your Squid server's HTTP port (e.g. 3128).
</LI>
<LI>Using the Preferences->Transport tab, select <EM>Use specified transports</EM>
and with the <EM>Specified Transports</EM> button, select use <EM>HTTP Only</EM>.</LI>
</UL>
</P>
<P>The RealPlayer (and RealPlayer Plus) manual states:
<PRE>
Use HTTP Only
Select this option if you are behind a firewall and cannot
receive data through TCP. All data will be streamed through
HTTP.
Note: You may not be able to receive some content if you select
this option.
</PRE>
</P>
<P>Again, from the documentation:
<PRE>
RealPlayer 4.0 identifies itself to the firewall when making a
request for content to a RealServer. The following string is
attached to any URL that the Player requests using HTTP GET:
/SmpDsBhgRl
Thus, to identify an HTTP GET request from the RealPlayer, look
for:
http://[^/]+/SmpDsBhgRl
The Player can also be identified by the mime type in a POST to
the RealServer. The RealPlayer POST has the following mime
type:
"application/x-pncmd"
</PRE>
</P>
<P>Note that the first request is a POST, and the second has a '?' in the URL, so
standard Squid configurations would treat it as non-cachable. It also looks
rather ``magic.''</P>
<P>HTTP is an alternative delivery mechanism introduced with version 3 players,
and it allows a reasonable approximation to ``streaming'' data - that is playing
it as you receive it. </P>
<P>It isn't available in the general case: only if someone has made the realaudio
file available via an HTTP server, or they're using a version 4 server, they've
switched it on, and you're using a version 4 client. If someone has made the
file available via their HTTP server, then it'll be cachable. Otherwise, it
won't be (as far as we can tell.)</P>
<P>The more common RealAudio link connects via their own <EM>pnm:</EM> method and is
transferred using their proprietary protocol (via TCP or UDP) and not using
HTTP. It can't be cached nor proxied by Squid, and requires something such as
the simple proxy that Progressive Networks themselves have made available, if
you're in a firewall/no direct route situation. Their product does not cache
(and I don't know of any software available that does.)</P>
<P>Some confusion arises because there is also a configuration option to use an
HTTP proxy (such as Squid) with the Realaudio/RealVideo players. This is
because the players can fetch the ``<CODE>.ram</CODE>'' file that contains the <EM>pnm:</EM>
reference for the audio/video stream. They fetch that .ram file from an HTTP
server, using HTTP.</P>
<H2><A NAME="purging-objects"></A> <A NAME="ss7.5">7.5</A> <A HREF="FAQ.html#toc7.5">How can I purge an object from my cache?</A>
</H2>
<P>Squid does not allow
you to purge objects unless it is configured with access controls
in <EM>squid.conf</EM>. First you must add something like
<PRE>
acl PURGE method PURGE
acl localhost src 127.0.0.1
http_access allow PURGE localhost
http_access deny PURGE
</PRE>
The above only allows purge requests which come from the local host and
denies all other purge requests.</P>
<P>To purge an object, you can use the <EM>squidclient</EM> program:
<PRE>
squidclient -m PURGE http://www.miscreant.com/
</PRE>
If the purge was successful, you will see a ``200 OK'' response:
<PRE>
HTTP/1.0 200 OK
Date: Thu, 17 Jul 1997 16:03:32 GMT
Server: Squid/1.1.14
</PRE>
If the object was not found in the cache, you will see a ``404 Not Found''
response:
<PRE>
HTTP/1.0 404 Not Found
Date: Thu, 17 Jul 1997 16:03:22 GMT
Server: Squid/1.1.14
</PRE>
</P>
<H2><A NAME="using-icmp"></A> <A NAME="ss7.6">7.6</A> <A HREF="FAQ.html#toc7.6">Using ICMP to Measure the Network</A>
</H2>
<P>As of version 1.1.9, Squid is able to utilize ICMP Round-Trip-Time (RTT)
measurements to select the optimal location to forward a cache miss.
Previously, cache misses would be forwarded to the parent cache
which returned the first ICP reply message. These were logged
with FIRST_PARENT_MISS in the access.log file. Now we can
select the parent which is closest (RTT-wise) to the origin
server.</P>
<H3>Supporting ICMP in your Squid cache</H3>
<P>It is more important that your parent caches enable the ICMP
features. If you are acting as a parent, then you may want
to enable ICMP on your cache. Also, if your cache makes
RTT measurements, it will fetch objects directly if your
cache is closer than any of the parents.</P>
<P>If you want your Squid cache to measure RTT's to origin servers,
Squid must be compiled with the USE_ICMP option. This is easily
accomplished by uncommenting "-DUSE_ICMP=1" in <EM>src/Makefile</EM> and/or
<EM>src/Makefile.in</EM>.</P>
<P>An external program called <EM>pinger</EM> is responsible for sending and
receiving ICMP packets. It must run with root privileges. After
Squid has been compiled, the pinger program must be installed
separately. A special Makefile target will install <EM>pinger</EM> with
appropriate permissions.
<PRE>
% make install
% su
# make install-pinger
</PRE>
There are three configuration file options for tuning the
measurement database on your cache. <EM>netdb_low</EM> and <EM>netdb_high</EM>
specify high and low water marks for keeping the database to a
certain size (e.g. just like with the IP cache). The <EM>netdb_ttl</EM>
option specifies the minimum rate for pinging a site. If
<EM>netdb_ttl</EM> is set to 300 seconds (5 minutes) then an ICMP packet
will not be sent to the same site more than once every five
minutes. Note that a site is only pinged when an HTTP request for
the site is received.</P>
<P>Another option, <EM>minimum_direct_hops</EM> can be used to try finding
servers which are close to your cache. If the measured hop count
to the origin server is less than or equal to <EM>minimum_direct_hops</EM>,
the request will be forwarded directly to the origin server.</P>
<H3>Utilizing your parents database</H3>
<P>Your parent caches can be asked to include the RTT measurements
in their ICP replies. To do this, you must enable <EM>query_icmp</EM>
in your config file:
<PRE>
query_icmp on
</PRE>
This causes a flag to be set in your outgoing ICP queries.</P>
<P>If your parent caches return ICMP RTT measurements then
the eighth column of your access.log will have lines
similar to:
<PRE>
CLOSEST_PARENT_MISS/it.cache.nlanr.net
</PRE>
In this case, it means that <EM>it.cache.nlanr.net</EM> returned
the lowest RTT to the origin server. If your cache measured
a lower RTT than any of the parents, the request will
be logged with
<PRE>
CLOSEST_DIRECT/www.sample.com
</PRE>
</P>
<H3>Inspecting the database</H3>
<P>The measurement database can be viewed from the cachemgr by
selecting "Network Probe Database." Hostnames are aggregated
into /24 networks. All measurements made are averaged over
time. Measurements are made to specific hosts, taken from
the URLs of HTTP requests. The recv and sent fields are the
number of ICMP packets sent and received. At this time they
are only informational.</P>
<P>A typical database entry looks something like this:
<PRE>
Network recv/sent RTT Hops Hostnames
192.41.10.0 20/ 21 82.3 6.0 www.jisedu.org www.dozo.com
bo.cache.nlanr.net 42.0 7.0
uc.cache.nlanr.net 48.0 10.0
pb.cache.nlanr.net 55.0 10.0
it.cache.nlanr.net 185.0 13.0
</PRE>
This means we have sent 21 pings to both www.jisedu.org and
www.dozo.com. The average RTT is 82.3 milliseconds. The
next four lines show the measured values from our parent
caches. Since <EM>bo.cache.nlanr.net</EM> has the lowest RTT,
it would be selected as the location to forward a request
for a www.jisedu.org or www.dozo.com URL.</P>
<H2><A NAME="ss7.7">7.7</A> <A HREF="FAQ.html#toc7.7">Why are so few requests logged as TCP_IMS_MISS?</A>
</H2>
<P>When Squid receives an <EM>If-Modified-Since</EM> request, it will
not forward the request unless the object needs to be refreshed
according to the <EM>refresh_pattern</EM> rules. If the request
does need to be refreshed, then it will be logged as TCP_REFRESH_HIT
or TCP_REFRESH_MISS.</P>
<P>If the request is not forwarded, Squid replies to the IMS request
according to the object in its cache. If the modification times are the
same, then Squid returns TCP_IMS_HIT. If the modification times are
different, then Squid returns TCP_IMS_MISS. In most cases, the cached
object will not have changed, so the result is TCP_IMS_HIT. Squid will
only return TCP_IMS_MISS if some other client causes a newer version of
the object to be pulled into the cache.</P>
<H2><A NAME="ss7.8">7.8</A> <A HREF="FAQ.html#toc7.8">How can I make Squid NOT cache some servers or URLs?</A>
</H2>
<P>In Squid-2, you use the <EM>no_cache</EM> option to specify uncachable
requests. For example, this makes all responses from origin servers
in the 10.0.1.0/24 network uncachable:
<PRE>
acl Local dst 10.0.1.0/24
no_cache deny Local
</PRE>
</P>
<P>This example makes all URL's with '.html' uncachable:
<PRE>
acl HTML url_regex .html$
no_cache deny HTML
</PRE>
</P>
<P>This example makes a specific URL uncachable:
<PRE>
acl XYZZY url_regex ^http://www.i.suck.com/foo.html$
no_cache deny XYZZY
</PRE>
</P>
<P>This example caches nothing between the hours of 8AM to 11AM:
<PRE>
acl Morning time 08:00-11:00
no_cache deny Morning
</PRE>
</P>
<P>In Squid-1.1,
whether or not an object gets cached is controlled by the
<EM>cache_stoplist</EM>, and <EM>cache_stoplist_pattern</EM> options. So, you may add:
<PRE>
cache_stoplist my.domain.com
</PRE>
Specifying uncachable objects by IP address is harder. The
<A HREF="../1.1/patches.html">1.1 patch page</A> includes a patch called <EM>no-cache-local.patch</EM> which
changes the behaviour of the <EM>local_ip</EM> and <EM>local_domain</EM> so
that matching requests are NOT CACHED, in addition to being fetched directly.</P>
<H2><A NAME="ss7.9">7.9</A> <A HREF="FAQ.html#toc7.9">How can I delete and recreate a cache directory?</A>
</H2>
<P>Deleting an existing cache directory is not too difficult. Unfortunately,
you can't simply change squid.conf and then reconfigure. You can't
stop using a <EM>cache_dir</EM> while Squid is running. Also note
that Squid requires at least one <EM>cache_dir</EM> to run.</P>
<P>
<OL>
<LI>Edit your <EM>squid.conf</EM> file and comment out, or delete
the <EM>cache_dir</EM> line for the cache directory that you want to
remove.</LI>
<LI>If you don't have any <EM>cache_dir</EM> lines in your squid.conf,
then Squid was using the default. You'll need to add a new
<EM>cache_dir</EM> line because Squid will continue to use
the default otherwise. You can add a small, temporary directory, fo
example:
<PRE>
/usr/local/squid/cachetmp ....
</PRE>
If you add a new <EM>cache_dir</EM> you have to run <EM>squid -z</EM>
to initialize that directory.
</LI>
<LI>Remeber that
you can not delete a cache directory from a running Squid process;
you can not simply reconfigure squid. You must
shutdown Squid:
<PRE>
squid -k shutdown
</PRE>
</LI>
<LI>Once Squid exits, you may immediately start it up again. Since you
deleted the old <EM>cache_dir</EM> from squid.conf, Squid won't
try to access that directory.
If you
use the RunCache script, Squid should start up again automatically.</LI>
<LI>Now Squid is no longer using the cache directory that you removed
from the config file. You can verify this by checking "Store Directory"
information with the cache manager. From the command line, type:
<PRE>
squidclient mgr:storedir
</PRE>
</LI>
<LI>Now that Squid is not using the cache directory, you can <EM>rm -rf</EM> it,
format the disk, build a new filesystem, or whatever.
</LI>
</OL>
</P>
<P>The procedure is similar to recreate the directory.
<OL>
<LI>Edit <EM>squid.conf</EM> and add a new <EM>cache_dir</EM> line.</LI>
<LI>Shutdown Squid (squid -k shutdown)</LI>
<LI>Initialize the new directory by running
<PRE>
% squid -z
</PRE>
</LI>
<LI>Start Squid again</LI>
</OL>
</P>
<H2><A NAME="ss7.10">7.10</A> <A HREF="FAQ.html#toc7.10">Why can't I run Squid as root?</A>
</H2>
<P>by Dave J Woolley</P>
<P>If someone were to discover a buffer overrun bug in Squid and it runs as
a user other than root, they can only corrupt the files writeable to
that user, but if it runs a root, they can take over the whole machine.
This applies to all programs that don't absolutely need root status, not
just squid.</P>
<H2><A NAME="ss7.11">7.11</A> <A HREF="FAQ.html#toc7.11">Can you tell me a good way to upgrade Squid with minimal downtime?</A>
</H2>
<P>Here is a technique that was described by
<A HREF="mailto:radu@netsoft.ro">Radu Greab</A>.</P>
<P>Start a second Squid server on an unused HTTP port (say 4128). This
instance of Squid probably doesn't need a large disk cache. When this
second server has finished reloading the disk store, swap the
<EM>http_port</EM> values in the two <EM>squid.conf</EM> files. Set the
original Squid to use port 5128, and the second one to use 3128. Next,
run ``squid -k reconfigure'' for both Squids. New requests will go to
the second Squid, now on port 3128 and the first Squid will finish
handling its current requests. After a few minutes, it should be safe
to fully shut down the first Squid and upgrade it. Later you can simply
repeat this process in reverse.</P>
<H2><A NAME="ss7.12">7.12</A> <A HREF="FAQ.html#toc7.12">Can Squid listen on more than one HTTP port?</A>
</H2>
<P><EM>Note: The information here is current for version 2.3.</EM></P>
<P>Yes, you can specify multiple <EM>http_port</EM> lines in your <EM>squid.conf</EM>
file. Squid attempts to bind() to each port that you specify. Sometimes
Squid may not be able to bind to a port, either because of permissions
or because the port is already in use. If Squid can bind to at least
one port, then it will continue running. If it can not bind to
any of the ports, then Squid stops.</P>
<P>With version 2.3 and later you can specify IP addresses
and port numbers together (see the squid.conf comments).</P>
<H2><A NAME="ss7.13">7.13</A> <A HREF="FAQ.html#toc7.13">Can I make origin servers see the client's IP address when going through Squid?</A>
</H2>
<P>Normally you cannot. Most TCP/IP stacks do not allow applications to
create sockets with the local endpoint assigned to a foreign IP address.
However, some folks have some
<A HREF="http://www.balabit.hu/en/downloads/tproxy/">patches to Linux</A> that allow exactly that.</P>
<P>In this situation, you must ensure that all HTTP packets destined for
the client IP addresses are routed to the Squid box. If the packets
take another path, the real clients will send TCP resets to the
origin servers, thereby breaking the connections.</P>
<HR>
<A HREF="FAQ-8.html">Next</A>
<A HREF="FAQ-6.html">Previous</A>
<A HREF="FAQ.html#toc7">Contents</A>
</BODY>
</HTML>
|
