1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
|
$Id: BUGS,v 1.263 2008/03/09 20:22:16 vanbaal Exp $
Lire BUGS list.
(
severities are as used by the debian bugtracking system:
critical
makes unrelated software on the system (or the whole system) break, or
causes serious data loss, or introduces a security hole on systems
where you install Lire.
grave
makes Lire unuseable or mostly so, or causes data loss, or introduces a
security hole allowing access to the accounts of users who use the
package.
serious
makes the package unsuitable for release.
important
a bug which has a major effect on the usability of Lire, without
rendering it completely unusable to everyone.
normal
the default value, applicable to most bugs.
minor
a problem which doesn't affect Lire's usefulness, and is presumably
trivial to fix.
wishlist
for any feature request, and also for any bugs that are very difficult to
fix due to major design considerations.
)
- serious: announce pkgsrc port on announcement list, once stable release is
available.
- lire >= 2:2.0.2.99.2-1 is ready for upload to Debian unstable
- serious: Lire requires GNU Make. Specifically all/po/Makefile.am does.
This is tedious for Solaris users (and BSD users, as well). See
Date: Sat, 30 Oct 2004 23:13:33 +0200
From: Joost van Baal
To: "Francis J. Lacoste", Wolfgang Sourdeau
Cc: LogReport Questions List
Subject: Lire requires GNU Make (was: Re: the workaround ...)
Message-ID: <20041030211333.GO9648@nagy.mdcc.cx>
It was reported too in
Date: Thu, 3 Aug 2006 10:35:34 +0200
From: Joost van Baal
To: lhc
Cc: LogReport Development List
Subject: Re: Unable to compile lire-2.0.2 under SunOS 5.8
Message-ID: <20060803083534.GJ3428@nagy.mdcc.cx>
Either update http://download.logreport.org/pub/current/doc/user-manual/ch02s02.html#sect:standalone-install-req
or fix the Makefile.am.
- important: Lire fails to build with a non-GNU Make. See
http://logreport.org/contact/lists/questions/msg00559.php . Either fix this
or document GNU Make in INSTALL as a requirement. This especially hurts the
Lire on Solaris community.
JvB, 30 October 2004
- important: It seems DBD::SQLite2 v 0.33 (released 09-Aug-2004), which uses
SQLite 2.8.12 is unusable on Solaris: bus error. See
http://logreport.org/contact/lists/questions/msg00561.php . This seems to
make Lire unusable on Solaris. Should we drop Solaris support?
JvB, 30 October 2004
- normal: get rid of Debian-centric .3pm extension for manpages: make it
configurable. Thanks to Peter Bex for reporting this.
JvB, 9 March 2008
- normal: Add support for "Message forwarded from hostname:" tag in
Lire::Syslog. Reported by Edward Quick.
FLJ, 20 December 2002
- normal: dubious bug in lr_anonymize : Malformed domainnames do
not get anonymized. (See also one of the wishlist items below).
It can be argued that this really is not a bug, because the domain name
is not conform the RFC. However, lr_anonymize already deals with
other non-conformant domain names. (where to draw the line...)
- normal: sendmail2dlf chokes on some wacky sendmail logs. see the BUGS
section in the sendmail2dlf manpage. Reported by Edward Eldred in
<Pine.A41.4.21L1.0205241026120.28780-100000@login5.isis.unc.edu>
JvB, 26 July 2002
- normal: Table shouldn't size themselves according to the
longest possible entry in the table. It should try to fit *most*
entries, long one should be cropped in PDF.
FLJ, Jul 8 2003
- normal: all DLF converters should get migrated to the Perl module style
setup. Migrated ones are the ones in superservice/lib/fooDlfConverter.pm,
they have a supporting /etc/lire/plugins/*_init file. Unmigrated ones
live as a script in lib/lire/convertors/*2dlf.
JvB, 21 March 2004
- normal: On OpenBSD, running
lr_log2report -o html ipfilter < /etc/motd
(Lire 2.0) fails with:
Formatting report as html in -...
tar: Failed open to write on /dev/rst0: Permission denied
This is likely due to differences in BSD tar and GNU tar; it might fail
with Solaris tar too.
JvB, 4 Sept 2004
- normal: Fix bug in lr_log2report's html commandline, see bugs section
in manpage.
JvB, 4 Oct 2006
- minor: contents of TODO should get merged in this file.
JvB, 22 June 2002
- minor: when specifying `0' as parameter in a top-report, the title shows
up as `Foo by Bar, Top 0', while it actually no longer is a top-report,
but shows all values.
JvB/FLJ, 22 June 2002
- minor: documentation needs to get reviewed and completed:
- The list of supported applications/log files in the user manual is
obsolete/incomplete (though it does mention how to generate an authoritive
list) JvB, 7 may 2003
- minor: Handle warning: line in qmail log files nicely instead of rejecting
them:
Oct 7 10:52:37 hibou lire: email qmail lr_tag-20021007105232-19195 qmail2dlf warning skipping line '1033731266.302474 warning: trouble opening remote/15/1079428; will try again later': first field should be one of 'new', 'info', 'starting', 'delivery', 'end', 'bounce' or 'status:', not 'warning:'
FLJ, 10 Oct 2002
- minor: subst-configvars should be replaced by calls to AC_DEFINE_DIR from
the Autoconf Macro Archive ( http://www.gnu.org/software/ac-archive/ ):
the problem we're trying to solve has been solved by others before!
JvB, 27 Apr 2003
- wishlist: drop lr_scale_ configuration variable and make the unit to
use fixed so that numbers are comparable in the table. As suggested
by Edwin Groothuis make that unit overridable in the report
specifications (a unit attribute).
FLJ, Nov 12 2002
- wishlist: the fw1_lea2dlf DLF converter requires the Date::Manip module
which isn't a standard component. It also requires the use of a
helper program available from
http://www.fellhauer-web.de/projects/fw1-loggrabber.html
FLJ, May 17 2003
- wishlist: the spamassassin2dlf convertor should document exact expected
log file format. Bug found during discussion with Tom Northeast.
JvB, Aug 24 2005
- wishlist: get reports generated via merging display from which data they
where generated. E.g. "Weekly top-m report, generated from daily top-n
reports". This in order to give the reader a clue on accuracy.
JvB, 23 June 2002
- wishlist: make it possible to specify default output format in configure:
./configure --with-default-output=XXX
Make configure fails if the needed requirements can't be found. Asked by
Rob Dinoff.
- wishlist: make it possible to build Lire on a system which lacks
XML::Parser. This is useful for package builders. (The Debian package
would no longer have libxml-parser-perl in its Build-Depends-Indep.)
configure.in would have to check for some environment variable before
failing.
JvB, 21 Jul 2002
- normal: Get the Firewall-1 firewall DLF converter properly integrated
and tested. Solve the LEA issues. Requested by Oscar Castaneda.
JvB, 10 Apr 2003
- wishlist: Adapt WELF firewall convertor to fully support Netasq (
www.netasq.com ). See
Date: Sat, 6 Apr 2002 13:01:29 -0500
From: "Francis J. Lacoste" <flacoste@logreport.org>
To: Van-Hau TRAN
Subject: Re: hibou Online Responder Report for 04/05/02
Message-ID: <20020406180129.GD613@Contre.COM>
JvB, May 10 2002
- wishlist: Implement a Watchguard Soho (
http://www.watchguard.com/products/fireboxsoho.asp ) firewall DLF convertor.
See
From: Ben el Cherb
To: logreport@logreport.org
Cc: flacoste@logreport.org
Subject: Re: hibou Online Responder Report for 04/05/02
Date: Sun, 07 Apr 2002 04:01:06
Message-ID: <F54HyM14MCYMPSO3ZZ8000055e0@hotmail.com>
JvB, May 10 2002
- wishlist: add a convertor for Netscreen ScreenOS 3.0.1's native syslog logs.
(We now only support the somewhat broken WELF-style logs from this box.)
suggested in a discussion between Francis and Mark D. Nagel, see
<03fe01c22ed4$e2b98a30$cc6fa8c0@BLEU> (not publicly archived.)
JvB, July 30 2002
- wishlist: Add a "sort" attribute to the records group operation.
- wishlist: Improve email reports :
- Better error reports (Longer error messages, maybe with failed email
split errors by to-relay and from-relay). See also
Subject: Re: email reports
From: "Francis J. Lacoste"
To: development
Message-Id: <1028124202.32487.29.camel@Arendt>
- Add a summary (with received, sent, forwarded subcategories).
- SMTPD reports (connections by hour).
- Add total (to_domain and from_domain) domain statistics
(requested by Matthias Jung)
- Add domain-by-period reports (requested by Matthias Jung).
- wishlist: handle postfix/pipe log messages. This is encountered, for example,
on hosts that have postfix with the Amavis virus scanner installed, as
well as hosts using the cyrus message transport. See also
Message-ID: <3B2BE13C.7040101@tbcinc.net>
Date: Sat, 16 Jun 2001 17:44:12 -0500
From: Justin Mecham <XXXXXXX>
To: LogReport <logreport@logreport.org>
Subject: Re: your log file to log@postfix.logreport.org
(Inform Justin once we've fixed this.)
- wishlist: handle 'reject:' log line in the postfix converter:
Dec 30 07:12:21 smtp postfix/smtpd[680]: reject: RCPT from \
unknown[212.96.116.2]: 554 Service unavailable; [212.96.116.2] blocked \
using relays.ordb.org, reason: This mail was handled by an open relay \
- please visit <http://ORDB.org/lookup/?host=212.96.116.2>; \
from=<oxygen9690@eudoramail.com> to=<xyz@mooreheadcomm.com>
Requested by Andy Thompson on Dec 29
FLJ, 30 Dec 2002
- wishlist: the email dlf format could better have a dlf record for _all_
message-queued events. We now only record the final action taken on the
message. Only when the logfile doesn't contain the last action taken on the
message at hand, a stat=queued events makes it into the dlf now. Queueing
events migth very well be interesting information, and might be needed for
some useful reports. Thanks Mark Huizer for reminding me about this.
JvB, 31 May 2002
Even better: we should probably redesign the email DLF format to be more in
line with what actual email logs contain, that is records for
receiving, queued, sending, forwarding, bouncing, etc. events. This could be
modelled after the exim logging model that is very clean (compared to all
the others
we support now). This would simplify the email DLF converters a lot,
since the flow analysis can be moved to generic modules instead of
having to be reimplemented in each of DLF converter. This would make all
sorts of reports possible (bounce, forwarding, anomalies).
It is possible to implement this in the current "single schema model"
by using something like we did in the firewall superservice where some
field are only relevant to IDS type of event and others only to packet
accounting type of events.
FLJ, 31 May 2002
- wishlist: we should map http requests with escaped url's to their unescaped
equivalent in some reports, eg map '%7E' to '~'. See
From: "E.L. Willighagen" <egonw@sci.kun.nl>
Date: Tue, 30 Oct 2001 17:20:43 +0100 (MET)
Message-Id: <200110301620.f9UGKhh16160@studs3.sci.kun.nl>
To: bugs@logreport.org, edwin@mavetju.org
Subject: re: hex-encoded pathnames
Define a new derived DLF format to support this. The destinction needs to
be made in the default dlf format, since apache will give a 404 when given
a request for a cgi script with an escaped '&' in the url. See also
Debian Bug #291063.
Related to this: when writing an XML report, we are not guarded against a
buggy report schema writing non-utf8 stuff to the XML.
Lire::Report::write_report() assumes all subreports are in UTF-8 format, but
does not check for this. It should. If a schema turns out to be buggy in this
way, Lire dies with a message like: "not well-formed (invalid token) at line
1004, column 46, byte 53763 at /usr/lib/perl5/XML/Parser.pm line 187".
updated by JvB, 16 jul 2006
- wishlist: Get rid of commands in the Subject: field of mails to a
responder (currently we support 'anon'). Use subemailaddresses for this,
e.g.: log-anon-html@<service>.logreport.org .
- wishlist: Get lr_log2mail support multipart/alternative emails, so that a
responder could send both ascii and html reports in one response.
JvB, 27 may 2002
- wishlist: define a DlfConverter that could parse log files based on field
specifications.
- wishlist: Make it possible to auto-detect log file types. This could
be done by trying all the DlfConverters on a small sample of lines and
compare errors/success statistics.
- wishlist: implement a mechanism to translate IP address to FQDN.
Two possibilities:
1) A lr_getaddrinfo filter which works on the DLF
2) A lr_xml2resolver filter which works on the XML report.
Useful for DNS, firewall, www superservice (at least). See
Date: Wed, 20 Oct 2004 12:53:19 +0200
From: Joost van Baal
To: Jeffeny Hoogervorst
Cc: LogReport Questions List
Subject: Re: Question about logreport (resolving hostnames)
Message-ID: <20041020105319.GM8544@nagy.mdcc.cx>
archived at http://logreport.org/contact/lists/questions/msg00545.php .
- wishlist: improve ftp reports:
1: Sort by file. It would display each time a particular file was accessed by
a particular person and it would list the time.
2: Sort by user and file. It would display each time a particular file was
accessed by a particular person.
3: Sort by user. It would show all files and times accessed by a particular
user.
reported by Brett Simpson on Thu, 15 Nov 2001 09:46:59 -0500, in
Message-Id: <sbf38f1b.032@GroupWise>, Subject: FTP improvement questions
Furthermore:
1: Show the number of failed logins.
2: Show who failed to login and at what time.
3: Show the number of successful logins.
4: Show who was successful to login and at what time.
5: Differentiate between downloads and uploads.
reported by Brett Simpson on Wed, 21 Nov 2001 09:59:08 -0500 in Message-Id:
<sbfb7aef.077@GroupWise>, Subject: Another idea for FTP reporting.
This might better be handled in a new 'login' super service, however. JvB.
- wishlist: improve www reports. See a comparison with our competitors,
mentioned in
Date: Tue, 11 Mar 2003 10:26:05 +0100
From: Wytze van der Raay
To: people
Cc: Stichting LogReport <bod>
Subject: FYI: comparison of web access log analyzers
Message-ID: <20030311102605.A24779@navarro.bebop.nlnet.nl>
.
JvB, 2003-11-20
- wishlist: document this:
- why do you advise to create a dedicated user account? why not 'nobody'
or 'daemon'?
- wishlist: Define a cross-superservice report for virtual hosts.
Requested by Jens Ott,
Lars Magerkohl. FLJ, 24 Mar 2002
4 Jun 2002
- wishlist: Make Lire conforms to the NIST standard and uses GiB, MiB
and KiB for size units and G, M and B for non-size fields. See
http://kerneltrap.com/article.php?sid=434&target=new. Egonw Dec 27 2001
- wishlist: New email service: Exchange 2000. Requested by
Andy Van den Heede. FLJ, Apr 06 2002
- wishlist: Support for IPv6 addresses in Lire. (E.g. the bind9 service and
the lr_anonymize script.)
Maybe we should put in a Lire::LogRegex module common regular expressions
that could be used across service to parse IP addresses and other common
regular expressions. Kimura Fuyuki's patch for Regexp::Common adds IPv6
support to this CPAN module. See <86elfmjy7r.wl@sz.homedns.org> .
Requested by Kimura Fuyuki. FLJ, Jun 04 2002
- wishlist: Better layout in Excel output format (uses border around
surreports, contrasting row colours, etc.)
- wishlist: Define a login and daemon schema that could
probably be used by lot of services.
See message <1028130934.32593.80.camel@Arendt> to the development
mailing list.
FLJ, July 31 2002
- wishlist: Define a new user feedback interface that could be used
to inform the users about the progress of the importation, analysis, report
generation etc. Backend for syslog, stderr and Lire::UI could be made.
FLJ, Aug 31 2004
- wishlist: Add menu items to import a log file, run an analysers and
generate a report in Lire::UI.
FLJ, Aug 31 2004
- wishlist: Add a store argument to the lr_log2mail, lr_log2report instead
of always using a temporary one.
FLJ, Aug 31 2004
- wishlist: Add a lr_import_log command that could import a log file
into an existing DlfStore.
FLJ, Aug 31 2004
- wishlist: rename the `dns' superservice to `dnsquery': more descriptive
name.
JvB, 16 Aug 2002
- wishlist: make bind8_query, bind9_query and qmail DLF converters
support more log formats. These log files may be encapsulated in
syslog log files. qmail can use other timestamp formats.
FLJ, 16 Aug 2002
- wishlist: get rid of the term `address': it is used at some places, to
refer to an online responders' address. That'd better be called `service',
since these belong to exactly the same namespace. The file address.cf could
better be named service_map.cf.
JvB, 18 Aug 2002
- wishlist: qmail2dlf should support more timestamp formats and syslog
encoded log files.
FLJ, Sep 01 2002
- wishlist: provide configuration hooks to normalize usernames (for
case-insensitive systems for example). Suggested by Brett Hales.
FLJ, Nov 12 2002
- wishlist: support for Microsoft Internet Connection Firewall log files.
Suggested by Diego Quintela.
FLJ, Nov 23 2002
- wishlist: support for Real Server logfile format. Spec is available from
http://service.real.com/help/library/guides/g270/htmfiles/report.htm#44459
Requested by Frank Elsner on questions mailing list.
FLJ, Nov 29 2002
- wishlist: support for bind query log files logged through syslog.
FLJ, Dec 03 2002
- wishlist: update argomail parser to support latest log files (version
1.8.1.9 log file) break because of different greeting message format and
use of EHLO.
FLJ, Jan 16 2003
- wishlist: add possibility to generate a cover sheets from a bunch of
reports. Suggested by Stewart James (see http://news.its.vu.edu.au/reports/
for a cover sheet sample).
FLJ, Feb 21 2003
- wishlist: Support the 'show-timezone' PIX option which adds a timezone
string to the log. An example can be found in the email sent
by Amit Sood to our bugs alias.
FLJ, Feb 28 2003
- wishlist: Add support for OpenBSD pf firewall logs. Requested by
Paul Weissmann.
FLJ, Feb 28 2003.
- wishlist: Add an option to display the timestamps in another timezone
than the one of the log file. Requested by Kevin Krumwiede.
This could be an additional propertiy on ImportJob and ReportJob.
FLJ, Apr 11 2003.
- wishlist: in Lire::UI, display the store manipulation utility through a
tabbed interface so as to reduce the amount of widgets displayed. This
would enable an easier handling of the widgets as well as a reduction
of contraints related to the screen estate.
WAS, Aug 31 2004.
- wishlist: if a integer range contains only one integer, it should be
listed as "n", not as ["n"-"n+1">. E.g. in
User Sessions by Their Recurrence
Visit # Sessions % Total
------------------------------------------------------- -------- -------
[1-2> 190 68.3
[2-3> 35 12.6
[3-4> 15 5.4
. Reported in
From: "E.L. Willighagen"
To: LogReport Support
Subject: Re: logreport.com on hibou 20041024
Date: Mon, 25 Oct 2004 08:47:20 +0200
Cc: people
Message-Id: <200410250847.21559.e.willighagen@science.ru.nl>
EW, Oct 25 2004
- wishlist: make Lire easier extensible. E.g., it'd be nice if DLF files
could be exported in their ascii representation from a DLF store. And
it'd be nice if we had a CSV report format: that's machine parsable for
people who don't know XML and/or Perl. Some people _are_ using old-style
Lire ways to do these kind of things, e.g. in situations where just a
part of Lire is used.
JvB, Oct 13, 2004
- wishlist: some suggestion items on what could be
part of a 2.1 release. (In no particular order)
a. Convert remaining pre-1.3 DLF converters to the new API.
b. Add reports for the lire_import_log and lire_import_stats
superservice.
c. Write new responder using new APIs.
d. Interactive performance improvement, Lire takes much time
to start because of all the XML files it has to parse. It would
be easy to add a persistent cache to speed that part up.
e. Add more information on where the data came from to the XML report
(DLF sources used, reports merged)
f. Report generation performance improvements.
FJL, Oct 12, 2004
|
