1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<meta name="description" content="AWStats Documentation - Other tools">
<meta name="keywords" content="awstats, awstat, tools, utilities, awstats_buildstaticpages, logresolvemerge, common2combined">
<meta name="robots" content="index,follow">
<meta name="title" content="AWStats Documentation - Other tools">
<title>AWStats Documentation - Other tools</title>
<link rel="stylesheet" href="styles.css" type="text/css">
<!-- $Revision: 1.39 $ - $Author: eldy $ - $Date: 2005/02/18 19:39:59 $ -->
</head>
<body topmargin=10 leftmargin=5>
<table style="font: 10pt arial,helvetica,verdana" cellpadding=0 cellspacing=0 border=0 bgcolor=#FFFFFF width=100%>
<!-- Large -->
<tr style="font: 10pt arial,helvetica,verdana">
<td bgcolor=#9999cc align=center><a href="/"><img src="images/awstats_logo4.png" border=0></a></td>
<td bgcolor=#9999cc align=center>
<br>
<font style="font: 16pt arial,helvetica,sans-serif" color=#EEEEFF><b>AWStats logfile analyzer 6.4 Documentation</b></font><br>
<br>
</td>
<td bgcolor=#9999cc align=center>
</td>
</tr>
</table>
<br><br><H1 style="font: 26px arial,helvetica,sans-serif">Other utilities</H1>
<br>
This is a list of other tools provided with AWStats.<br>
All those tools are available in <b>tools</b> directory of AWStats distribution.<br>
<br><br>
<!--
<br>
<br><a name="awstats_configure"><H2 style="font: 18px arial,helvetica,sans-serif color: #606060"><u>awstats_configure.pl</u></H2></a>
<br>This script creates one config file for each web servers provided by Apache.
<br>After running this tool, AWStats can immediatly be used.<br>
<br>
<br> This tool is not yet available...
<br>
<br>
-->
<br>
<br><a name="awstats_updateall"><H2 style="font: 18px arial,helvetica,sans-serif color: #606060"><u>awstats_updateall.pl</u></H2></a>
<br>awstats_updateall launches update process for all AWStats config files (except
<br>awstats.model.conf) found in a particular directory, so you can easily setup a
<br>cron/scheduler job. The scanned directory is by default /etc/awstats.
<br>
<br>Usage: awstats_updateall.pl now [options]
<br>
<br>Where options are:
<br> -awstatsprog=pathtoawstatspl
<br> -configdir=confdirtoscan
<br>
<br>
<br>
<br><a name="awstats_buildstaticpages"><H2 style="font: 18px arial,helvetica,sans-serif color: #606060"><u>awstats_buildstaticpages.pl</u></H2></a>
<br>awstats_buildstaticpages allows you to launch AWStats with -staticlinks option
<br>to build all possible pages allowed by AWStats -output option.
<br>
<br>Usage:
<br>awstats_buildstaticpages.pl (awstats_options) [awstatsbuildstaticpages_options]
<br>
<br> where awstats_options are any option known by AWStats
<br> -config=configvalue is value for -config parameter (REQUIRED)
<br> -update option used to update statistics before to generate pages
<br> -lang=LL to output a HTML report in language LL (en,de,es,fr,...)
<br> -month=MM to output a HTML report for an old month=MM
<br> -year=YYYY to output a HTML report for an old year=YYYY
<br>
<br> and awstatsbuildstaticpages_options can be
<br> -awstatsprog=pathtoawstatspl gives AWStats software (awstats.pl) path
<br> -dir=outputdir to set output directory for generated pages
<br> -builddate=%YY%MM%DD Used to add build date in built pages filenames
<br> -staticlinksext=xxx For pages with .xxx extension instead of .html
<br> -buildpdf[=pathtohtmldoc] Build a PDF file after building HTML pages.
<br> Output directory must contains icon directory
<br> when this option is used (need 'htmldoc').
<br>
<br>New versions and FAQ at http://awstats.sourceforge.net
<br>
<br>
<br>
<br><a name="logresolvemerge"><H2 style="font: 18px arial,helvetica,sans-serif color: #606060"><u>logresolvemerge.pl</u></H2></a>
<br>logresolvemerge allows you to merge several log files into one output,
<br>sorted on date. It also makes a fast reverse DNS lookup to replace
<br>all IP addresses into host names in resulting log file.
<br>logresolvemerge comes with ABSOLUTELY NO WARRANTY. It's a free software
<br>distributed with a GNU General Public License (See COPYING.txt file).
<br>logresolvemerge is part of AWStats but can be used alone as a log merger
<br>or resolver before using any other log analyzer.
<br>
<br>Usage:
<br> logresolvemerge.pl [options] file
<br> logresolvemerge.pl [options] file1 ... filen
<br> logresolvemerge.pl [options] *.*
<br>Options:
<br> -dnslookup make a reverse DNS lookup on IP adresses
<br> -dnscache=file make DNS lookup from cache file first before network lookup
<br> -showsteps print on stderr benchmark information every 8192 lines
<br> -addfilenum if used with several files, file number can be added in first
<br> field of output file.
<br>
<br>This runs logresolvemerge in command line to open one or several web
<br>server log files to merge them (sorted on date) and/or to make a reverse
<br>DNS lookup (if wanted). The result log file is sent on standard output.
<br>Note: logresolvemerge is not a 'sort' tool to sort one file. It's a
<br>software able to output sorted log records (with a reverse DNS lookup
<br>included or not) even if log records are dispatched in several files.
<br>Each of thoose files must be already independently sorted itself
<br>(but that is the case in all web server log files). So you can use it
<br>for load balanced log files or to process several old log files.
<br>
<br>Don't forget that the main goal of logresolvemerge is to send log records to
<br>a log analyzer in a sorted order without merging files on disk (NO NEED
<br>OF DISK SPACE AT ALL) and without loading files into memory (NO NEED
<br>OF MORE MEMORY). Choose of output records is done on the fly.
<br>
<br>So logresolvemerge is particularly usefull when you want to output several
<br>and/or large log files in a fast process, with no use of disk or
<br>more memory and in a chronological order through a pipe (to be used by a log
<br>analyzer.
<br>
<br>Note: If input records are not 'exactly' sorted but 'nearly' sorted (this
<br>occurs with heavy servers), this is not a problem, the output will also
<br>be 'nearly' sorted but a few log analyzers (like AWStats) knowns how to deal
<br>with such logs.
<br>
<br>WARNING: If log files are old MAC text files (lines ended with CR char), you
<br>can't run this tool on Win or Unix platforms.
<br>
<br>WARNING: Because of important memory holes in ActiveState Perl version, use
<br>another Perl interpreter if you need to process large lof files.
<br>
<br>Now supports/detects:
<br> Automatic detection of log format
<br> Files can be .gz/.bz2 files if zcat/bzcat tools are available in PATH.
<br>New versions and FAQ at http://awstats.sourceforge.net
<br>
<br>
<br>
<br><a name="maillogconvert"><H2 style="font: 18px arial,helvetica,sans-serif color: #606060"><u>maillogconvert.pl</u></H2></a>
<br>maillogconvert is mail log preprocessor that convert a mail log file (from
<br>postfix, sendmail or qmail servers) into a human readable format.
<br>The output format is also ready to be used by a log analyzer, like AWStats.
<br>
<br>Usage:
<br> perl maillogconvert.pl [standard|vadmin] [year] < logfile > output
<br>
<br>The first parameter specifies what format the mail logfile is :
<br> standard - logfile is standard postfix,sendmail,qmail or mdaemon log format
<br> vadmin - logfile is qmail log format with vadmin multi-host support
<br>
<br>The second parameter specifies what year to timestamp logfile with, if current
<br>year is not the correct one (ie. 2002). Always use 4 digits. If not specified,
<br>current year is used.
<br>
<br>If no output is specified, it goes to the console (stdout).
<br>For example, the following sample from postfix mail log server:
<br>
<i>
<br># 1 Mail fromuser@aol.com -> touser@toserver.com, forward touser@toserver.com -> touser@mainserver.com
<br>Jan 01 07:27:31 apollon postfix/smtpd[1684]: connect from remt30.cluster1.charter.net[209.225.8.40]
<br>Jan 01 07:27:32 apollon postfix/smtpd[1684]: 2BC793B8A4: client=remt30.cluster1.charter.net[209.225.8.40]
<br>Jan 01 07:27:32 apollon postfix/cleanup[1687]: 2BC793B8A4: message-id=<36027278@vneka>
<br>Jan 01 07:27:32 apollon postfix/qmgr[13860]: 2BC793B8A4: from=<fromuser@aol.com>, size=2130, nrcpt=1 (queue active)
<br>Jan 01 07:27:32 apollon postfix/smtpd[1684]: disconnect from remt30.cluster1.charter.net[209.225.8.40]
<br>Jan 01 07:27:38 apollon postfix/local[1689]: 2BC793B8A4: to=<touser@mainserver.com>, orig_to=<touser@toserver.com>, relay=local, delay=6, status=sent ("|/usr/bin/procmail")
<br># 2 Reject: 450
<br>Jan 01 14:05:44 apollon postfix/smtpd[2114]: connect from baby.mainframe.nl[81.29.4.2]
<br>Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: client=baby.mainframe.nl[81.29.4.2]
<br>Jan 01 14:05:44 apollon postfix/smtpd[2114]: E0C9D3BD9A: reject: RCPT from baby.mainframe.nl[81.29.4.2]: 450 <touser2@toserver.com>: User unknown in local recipient table; from=<> to=<touser2@toserver.com> proto=ESMTP helo=<baby.mainframe.nl>
<br>Jan 01 14:10:16 juni postfix/smtpd[2568]: C34ED1432B: reject: RCPT from relay2.tp2rc.edu.tw[163.28.32.177]: 450 <unknownuser@unknownserver.com>: User unknown in local recipient table; from=<> proto=ESMTP helo=<rmail.nccu.edu.tw>
<br># 1 From unknown
<br>Jan 01 15:17:05 apollon postfix/smtpd[29866]: connect from tomts12.bellnexxia.net[209.226.175.56]
<br>Jan 01 15:17:05 apollon postfix/smtpd[29866]: 578093B8B5: client=tomts12.bellnexxia.net[209.226.175.56]
<br>Jan 01 15:17:05 apollon postfix/cleanup[28931]: 578093B8B5: message-id=<20030905131913.EOVH11393.tomts12-srv.bellnexxia.net@tomts12-srv>
<br>Jan 01 15:17:06 apollon postfix/qmgr[965]: 578093B8B5: from=<>, size=109367, nrcpt=1 (queue active)
<br>Jan 01 15:17:06 apollon postfix/local[32432]: 578093B8B5: to=<touser@mainserver.com>, orig_to=<touser@toserver.com>, relay=local, delay=1, status=sent ("|/usr/bin/procmail")
<br>Jan 01 15:17:06 apollon postfix/smtpd[29866]: disconnect from tomts12.bellnexxia.net[209.226.175.56]
</i>
<br>
<br>will give a file that looks like this:
<br>
<i>
<br>2004-01-01 07:27:38 fromuser@aol.com touser@toserver.com remt30.cluster1.charter.net localhost SMTP - 1 2130
<br>2004-01-01 14:05:44 <> touser2@toserver.com baby.mainframe.nl - SMTP - 450 0
<br>2004-01-01 14:10:16 <> unknownuser@unknownserver.com relay2.tp2rc.edu.tw - SMTP - 450 0
<br>2004-01-01 15:17:06 <> touser@toserver.com tomts12.bellnexxia.net localhost SMTP - 1 109367
</i>
<br>
<br>See <a href="awstats_faq.html#MAIL">FAQ-COM100</a> to see how to use maillogconvert.pl with AWStats to analyze mail log files.
<br>
<br>
<br>
<br>
<br><a name="urlaliasbuilder"><H2 style="font: 18px arial,helvetica,sans-serif color: #606060"><u>urlaliasbuilder.pl</u></H2></a>
<br>Urlaliasbuilder generates an 'urlalias' file from an input file.
<br>The input file must contain a list of URLs (It can be an AWStats history file).
<br>For each of thoose URLs, the script get the corresponding HTML page and catch the
<br>header information (title), then it writes an output file that contains one line
<br>for each URLs and several fields:
<br>- The first field is the URL,
<br>- The second is title caught from web page.
<br>This resulting file can be used by AWStats urlalias plugin.
<br>
<br>Usage: urlaliasbuilder.pl -site=www.myserver.com [options]
<br>
<br>The site parameter contains the web server to get the page from.
<br>Where options are:
<br> -urllistfile=Input urllist file
<br> If this file is an AWStats history file then urlaliasbuilder will use the
<br> SIDER section of this file as its input URL's list.
<br> -urlaliasfile=Output urlalias file to build
<br> -overwrite Overwrite output file if exists
<br> -secure Use https protocol
<br>
<br>Example: urlaliasbuilder.pl -site=www.someotherhost.com
<br>
<br>New versions and FAQ at http://awstats.sourceforge.net
<br>
<br>This script was written from Simon Waight original works title-grabber.pl.
<br>
<br>
<br>
<hr>
<script language=javascript>
var date='$Date: 2005/02/18 19:39:59 $';
document.writeln("Last revision: "+date);
</script>
</body>
</html>
|
