File: README

package info (click to toggle)
calamaris 2.8-4
  • links: PTS
  • area: main
  • in suites: slink
  • size: 152 kB
  • ctags: 16
  • sloc: perl: 1,555; sh: 47; makefile: 39
file content (271 lines) | stat: -rw-r--r-- 10,900 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
Calamaris
Version 2

What is it?
-----------

Calamaris is a Perl script, which was first intended as demo for a statistical
software for Squid.  I started it at 13 January 1997 (Version 1.1) as a
rewrite of my old Squid-Analysis-Script weekly.pl (which was in German
language). I announced it (Version 1.16) to the public at 28 Feb 1997.  (see
http://squid.nlanr.net/Mail-Archive/squid-users/9702/archive/0555.html for the
Original-Announcement) Since then it is used by people all around the world,
and i decided to build a new improved version of it.  Calamaris V2 is a nearly
complete rewrite, with changed and more reports.

It works with Native-Log-files from Squid V1.1.x and later.

There is also the possibility to parse Log-files from NetCache. (but see 'Bugs
and Shortcomings' on this.)


Where to get it?
----------------

The Calamaris-Home-page is located at
http://www.Cord.de/~cord/tools/squid/calamaris/

There is also an Announcement-Mailing-list. To subscribe send mail with
'subscribe' in the Mail-Body to Calamaris-announce-request@Cord.de.
Subscribers will get a mail on every new release, including a list of the
changes. --> Low-Traffic.


How to use it?
--------------

* You'll need Perl Version 5 (see http://www.Perl.com/) Calamaris is reported
  to work with Perl 5.001 (maybe you have to remove the '-w' from the first
  line and comment out the 'use vars'-line), but it is highly recommended
  (especially for security of your computer) that you use an actual version
  (>=5.005_02) of it.

* You'll also need Squid (see http://squid.nlanr.net/) Version 1.1.x or later
  (or Log-files derived from it). It has to produce the 'Native
  Log-file-Format' (which is the default since Version 1.1.x). If Calamaris
  can't parse the access.log, check your Log-file-format.  It is explained in
  the Squid FAQ which can be reached via the Address above.

* Put Calamaris itself into a warm, dry place on your computer (i.e. into the
  Squid-bin-directory, or /usr/local/bin/). Maybe (if your Perl isn't located
  at /usr/bin/perl) you'll have to change the first line of Calamaris to point
  to your copy of Perl.

* Use it! 

  'cat access.log.1 access.log.0 | calamaris.pl'

  Calamaris generates by default a brief ASCII report of incoming and outgoing
  requests.

  NOTE: If you pipe more than one log-file into Calamaris, make sure that they
    are chronological ordered (oldest file first), else some reports can
    return wrong values.

  You can alter Calamaris' behaviour with switches.  There are four groups of
  switches:

  Reports-switches:

    without these switches you get a brief report of incoming and outgoing
    requests.

    -a	    all. switch all available reports on. It equals 
	    -d 20 -p -P 60 -r -1 -s -t 20

    -d n    domain-report. It switches the Top-level- and the
	    second-level-report on. The Data is derived from the URL. The
	    output is limited by the number n. (-1 is unlimited)

    -p	    peak-report.  Measure the peaks of the Squid usage in requests per
	    second, minute and hour. It reports peaks for TCP, UDP and ALL
	    requests.

    -P n    performance-report.  It shows the throughput of TCP-Requests for
	    every n minutes.

    -r n    requester-report. It switches the UDP- and TCP-Requester-Reports
	    on. The output is limited by the number n. (-1 is unlimited)

    -s	    status-report. This switch alters the default behaviour of
	    Calamaris and makes the status-reports more verbose.

    -t n    type-report. It switches the content-type- and the
	    file-extension-report on. The output is limited by the number n.
	    (-1 is unlimited)

  Output-switches:

    without these switches you'll get a plain ASCII output with 80 chars
    width.

    -m	    mail-output. It adds a Subject-Header to the beginning of the
	    report, so you can pipe the output into a simple mail-program.

    -w      web-report. All output is given in HTML-Format with HTML-Tables.
	    Combined with the -m-option you can also send HTML-Mails *urgs*

  Caching-switches:

    -i file input-file. You can reuse cache-file generated with the -o-option,
	    to add old data to a new report. You can also add many cache-files
	    by separating them with a ':'.

	    NOTE: If you use more than one cache-file, make sure that they are
	      chronological ordered (oldest file first), else some reports can
	      return wrong values.

    -o file output-file. The file defined contains the processed data. You can
	    reuse this file in a later run with the -i-option. The output-file
	    can be the same as a input-file. It will be overwritten with the
	    new data. The cached data is altered by the report-switches above.

	    NOTE: It is not recommended to change the switches between runs,
	      if you use cache-files, else you can get some strange
	      measurements.

  Misc-switches:

    -b n    benchmark. a switch for the impatient. ;-) It prints a # for 
	    every n parsed lines.

    -H name Host-name. This names the output on the Title/Subject of the
	    output. If set to 'lookup' Calamaris looks up the host-name of the
	    current machine and puts that name in.

    -n	    no lookup. Switch the IP-number-lookup for the requesters off. 

    -u	    user. Switches the usage of eventually available ident-information
	    for requester-reports on. WARNING: This breaks the privacy of your
	    users!

    -z	    zero. Switch reading of the Standard-Input off. You can use this
	    to merge many cache-files to one (see -i and -o) or to generate a
	    report out of a cache-file.

    -c	    copyright. Prints out the copyright of Calamaris.

    -h	    help. Prints a brief description of command-line-options.


  You should also take a look at the EXAMPLES-File, for
  'Real-Life'-usage-examples of Calamaris.


Are there known bugs or other problems?
---------------------------------------

* I fixed a bug regarding caching of Performance-Data. This brakes old
  Calamaris-Cache-Files... I a workaround in, which allows to parse old and
  new cachefiles. This sets 'Cache-Hits' in the Performance-Report to '-'.

* It is possible that you encounter problems if you use Squid V1.2.x/V2.x.
  Calamaris isn't well tested with the new release. (I don't have a production
  cache of it yet.) If you find problems, please report them to
  Calamaris-bug@Cord.de. I'll try to fix them ASAP.  I'm also very interested
  how Calamaris works with CACHE_DIGEST and CARP-Support. So if you use it,
  please send me a mail to Calamaris@Cord.de, even if it works without
  problem. Thank You.

* If you parse NetCache Log-files with Calamaris, it seems that you can run
  into trouble. I've looked for information on the Log-file-format from the
  vendor, but it seems that they don't give info to people who don't want to
  tell them their name and address.  So, if someone can provide me with that
  info, I'll try to put it into Calamaris, if not... (I've seen that NetCache
  produces broken Log-files with different counts of fields, is that a buggy
  version?)

* If you want to parse more than one Log-file (i.e. from the 'logfilerotate')
  or want to use more than one input-cache-file you have to put them in
  chronological sorted order (oldest first), else you get wrong peak values.
  (Is this something that i should fix? Don't think so...)

* However: If you use the caching function the peak-values can be wrong. 

* Squid doesn't log outgoing UDP-Requests, so i can't put them into the
  statistics without parsing squid.conf and the cache.log-file. (Javier Puche
  (Javier.Puche@rediris.es) asked for this), but i don't think that i should
  put this into Calamaris...  (Check last point of 'Bugs and shortcomings'.)

* It is written in Perl. Yea, Perl is a great language for something like this
  (also it is the only one I'm able to write something like this in).
  Calamaris was first intended as demo for what i wanted from a statistical
  software. (OK, it is fun to write it, and it is even more fun to recognize
  that many people use the script). For my Caches with about 150MB-Log-file
  per week it is OK, but for those people on a heavy loaded Parent-cache it is
  simply to slow.

* Hmmm, while looking through those many different reports i generate, i think
  that i generate more than anybody ever wants to now about squid :-) So i
  added switches, so everybody can switch on or off the reports wanted. But
  this is also a speed disadvantage because of the many checks if set or
  not...


What will happen next?
----------------------

I think that Calamaris v2 is now finished. (except for bugs, that maybe were
not found yet.)

But if you have an idea what is still missing in a software for parsing
Proxy-Log-files, let me know. --> Calamaris@Cord.de. I'll will build it in, or
add it to the wish-list below :-)

* add switch for sorting by requests and sorting by bytes. (Gerd Michael
  Hoffmann <Hoffmann@dvgw.de> asked for this)

* rewrite it so that the Main-loop is constructed first and than run via eval.
  This could make Calamaris significantly faster. (suggested by Jost Krieger
  <Jost.Krieger@ruhr-uni-bochum.de>)

* add report for byte-peak (inspirated by Andreas Strotmann
  <A.Strotmann@Uni-Koeln.DE>) (I have no idea how to calculate that)

* build graphics (hope i remember who suggested this first, the mail must be
  somewhere in my work-mailbox ;-) (This is a thing for Calamaris v3, if i
  ever going to write it. there are nice gd-libs in Perl ;-)

* add real NetCache-Support. The NetCache-Support is rudimentary working, but
  is only in some kind of alpha-state. see above.

* make Calamaris faster. see above. If someone wants to rewrite Calamaris in a
  faster language: Feel Free! (But respect the GNU-License) It would be nice
  if you drop me a line about it, I'll mention it below. And please please
  please don't use the name 'Calamaris' for it without asking me!
  

Is there anything else?
-----------------------

Ernst Heiri builds a spin-off of my Calamaris V1, which can be found *where?*

There is also now a C++-port of Ernst Heiri's Calamaris available which is
(according to the author Jens-S. Voeckler (voeckler@rvs.uni-hannover.de)) five
times faster than the Perl-variant.  check
http://statistics.www-cache.dfn.de/Projects/seafood/ for this.

more Squid-Log-file-Analysers can be found via the Squid-Home-page at
http://squid.nlanr.net/


Thank You!
----------

* The developers and contributors of Squid.
* The developers and contributors of Perl.
* The contributors, feature requesters and bug-reporters of Calamaris.
* Gerold 'Nimm Perl' Meerkoetter.


Not happy yet?
--------------

Drop me a line to Calamaris@Cord.de and tell me what is missing or wrong or
not clear or whatever. You are welcome :-)


Version of the README
---------------------

$Id: README,v 2.6 1998/10/12 20:03:53 cord Exp $