$Id: BUGS,v 1.104 2002/02/12 20:44:20 vanbaal Exp $

Lire BUGS list.

(

severities are as used by the debian bugtracking system:

critical 
     makes unrelated software on the system (or the whole system) break, or 
     causes serious data loss, or introduces a security hole on systems
     where you install Lire.
grave 
     makes Lire unuseable or mostly so, or causes data loss, or introduces a 
     security hole allowing access to the accounts of users who use the 
     package. 
serious 
     makes the package unsuitable for release. 
important 
     a bug which has a major effect on the usability of Lire, without 
     rendering it completely unusable to everyone. 
normal 
     the default value, applicable to most bugs. 
minor 
     a problem which doesn't affect Lire's usefulness, and is presumably 
     trivial to fix. 
wishlist 
     for any feature request, and also for any bugs that are very difficult to 
     fix due to major design considerations. 

)

- normal: The display of tables with lire:group should be improved in HTML 
	  format. (The group aren't indented under the main entry).

- normal: There is a bug in the DocBook transformation for lire:entry with 
          multiple lire:name in them : they are output as multiple entries in 
          the same row. Generate multiple para or row.

- normal: dubious bug in lr_anonimize : Malformed domainnames do 
  not get anonimized. (See also one of the wishlist items below).
  It can be argued that this really is not a bug, because the domain name
  is not conform the RFC. However, lr_anonimize already deals with
  other non-conformant domain names. (where to draw the line...)

- minor: make uninstall breaks on the full Lire tarball.  The lacking 
  XML::Parser .packlist blows up things.

- minor: lacking manpages:

   Lire/AsciiDlf/AsciiDlfFactory.pm
   Lire/AsciiDlf/DerivedRecordsCreator.pm
   Lire/AsciiDlf/ExtendedFieldsCreator.pm
   Lire/AsciiDlf/FilterExpr.pm
   Lire/AsciiDlf/FilterSpec.pm
   Lire/AsciiDlf/Group.pm
   Lire/AsciiDlf/GroupOp.pm
   Lire/AsciiDlf/ReportSpec.pm
   Lire/AsciiDlf/Timegroup.pm
   Lire/Config.pm
   Lire/DataTypes.pm
   Lire/DerivedSchema.pm
   Lire/ExtendedSchema.pm
   Lire/Extension.pm
   Lire/Extensions/Email/EmailSchema.pm
   Lire/Extensions/WWW/AttackSchema.pm
   Lire/Extensions/WWW/CountrySchema.pm
   Lire/Extensions/WWW/DirSchema.pm
   Lire/Extensions/WWW/RobotSchema.pm
   Lire/Extensions/WWW/UserAgentSchema.pm
   Lire/Extensions/WWW/UserSessionSchema.pm
   Lire/FilterExpr.pm
   Lire/FilterSpec.pm
   Lire/Group.pm
   Lire/GroupField.pm
   Lire/GroupOp.pm
   Lire/Param.pm
   Lire/ReportParser.pm
   Lire/ReportParser/AsciiWriter.pm
   Lire/ReportParser/ChartWriter.pm
   Lire/ReportSpec.pm
   Lire/ReportSpecFactory.pm
   Lire/Timegroup.pm

- minor: The Lire User Manual is not yet finished. E.g. the 'responder
  installation' section needs to get reviewed.

- minor: Apache.pm should use Lire::WWW::URL

- wishlist: define a 'policy' on performance.  What do we want,
  performancewise?

  Performance of lire-20020211-cvs.tar.gz (I used top(1)'s batch mode for
  this; processaccounting tools didn't give me the information I wanted.)

   We processed a 128MB apache `combined' log file, 660,000 records on the
   LogReport server (Pentium III 662 MHz with 128 MB RAM).  This took about one
   hour wallclock time.  combined2dlf consumed about 4MB SIZE/RSS, spent 6

   minutes on cpu.  lr_dlf2xml, which converts the DLF file to a report in the
   Lire XML Lire Report Markup Language, consumed about 55MB SIZE/RSS.  It
   spent 44 minutes on cpu.  Generating an ascii report from the XML consumes
   very little resources: too little for top(1) to measure.

   Processing a 36M cisco log, 240,000 records, took about 40 minutes wallclock
   time.  acl_cisco2dlf consumed about 4MB SIZE/RSS, spent 2:13 on cpu.
   lr_dlf2xml consumed about 100MB RSS.  It spent 6 minutes on cpu.

  This is with the default LR_MAX_MEMORY setting: 40Megs.  Figures are likely
  better for dns logs, since the reports of these superservices are more
  straightforward.

  Are we happy with this?  Should we state: use big fat machines for big fat
  log files?  Should we state: Lire is generic, therefore it's not as fast
  as tools, targeted at one specific log file format?  Should we reimplement
  bottlenecks in C?

                                                     JvB, 12 feb 2002

- wishlist: makes it possible to specify default output format in configure:
  ./configure --with-default-output=XXX 
  Make configure fails if the needed requirements can't be found. Asked by 
  Rob Dinoff.

- wishlist: Generate catalog.xml at runtime (instead of at configure time) 
  based on the values in defaults. (This wasn't done because we need a
  catalog.xml in the source tree to build documentation, but we could
  still use a dynamically generated catalog.xml at runtime). Asked by
  Rob Dinoff.

- wishlist: use stuff from service/doc/images/ in the user- and/or developers
  manual.

- wishlist: Add a "record" aggregator which could be used to display records 
  according to certain sorting and selection criteria. Fields of the record 
  to display should also be selectable. (Could be use to show the file and
  time of download by user. Requested by Brett Simpson 2001/11/21).

- wishlist: Make it possible to use timegroup in group operation. Maybe 
  we should consider all aggregators (group, timegroup, rangegroup, timeslot, 
  summary) as candidate for nesting in groups? summary/other aggregator isn't
  a particularly good idea, but group/summary may be. (Requested by
  Cedric Gross 2002/01/30)

- wishlist: Improve email reports :
    - Better error reports (Longer error messages, maybe with failed email
      split errors by to-relay and from-relay).
    - Add a summary (with received, sent, forwarded subcategories).
    - SMTPD reports (connections by hour).

- wishlist: Improvements to lr_run_tests
    - Add tests for anonimizer and deanonimizer process. 
    - Add automatic verification.
    - Integrate in build process.

- wishlist: handle postfix/pipe log messages. This is encountered, for example,
    on hosts that have postfix with the Amavis virus scanner installed, as
    well as hosts using  the cyrus message transport.  See also

     Message-ID: <3B2BE13C.7040101@tbcinc.net>
     Date: Sat, 16 Jun 2001 17:44:12 -0500
     From: Justin Mecham <XXXXXXX>
     To: LogReport <logreport@logreport.org>
     Subject: Re: your log file to log@postfix.logreport.org

    (Inform Justin once we've fixed this.)

- wishlist: we should map http requests with escaped url's to their unescaped
  equivalent in some reports, eg map '%7E' to '~'.  See

   From: "E.L. Willighagen" <egonw@sci.kun.nl>
   Date: Tue, 30 Oct 2001 17:20:43 +0100 (MET)
   Message-Id: <200110301620.f9UGKhh16160@studs3.sci.kun.nl>
   To: bugs@logreport.org, edwin@mavetju.org 
   Subject: re: hex-encoded pathnames

  Define a new derived DLF format to support this.  The destinction needs to
  be made in the default dlf format, since apache will give a 404 when given
  a request for a cgi script with an escaped '&' in the url.

- wishlist: get rid of lr_xslt.  We are relying on libxslt and libxml2 anyway.
  Since we only support xsltproc (not xalan or sablotron), there's no need
  to make a wrapper.  Clean up configure.in 's setting of XSLT_PROCESSOR and
  XSLTPROC .

- wishlist: get munpack found during configure.  Expand an automake munpack
  macro in lr_getbody.in.  Get lr_getbody generate a sane error when there's
  no munpack on the system.  Or get rid of mpack altogether, and use some CPAN
  perl module to parse MIME messages.

- wishlist: Add a lr_xml2report command which has a similar command line
  footprint then lr_log2report ([-i] -o format). That command could be used 
  both by lr_log2report and in a client-only installation.
  Move back lr_xml2ascii to libexec.

- wishlist: pdfjadetex should not generate any errors or warnings when
   typesetting our docs or reports.  Currently, we redirect warnings
   to /dev/null.

- wishlist: Have lr_log2mail send PDF in an attachment, get rid of commands
  in the Subject: field of mails to a responder (currently we support
  'anon').  Use subemailaddresses for this, e.g.: 
  log-anon-html@<service>.logreport.org .

- wishlist: Change all used environment variables to LR_something, to avoid
  clashes with env vars used by other apps.

- wishlist: Sanely handle non ascii encoding in xml.  Do some I18N-ing.  Deal
  with iso-8859-1.

- wishlist: lr_config should be modified to configure the reports from their
  XML specification and write ~/.lire/etc/www.cfg and friends.

- wishlist: have Xlog2xml scripts use field content descriptors like in the
  superservice.xml specs file for the DLF files. This would make it easier
  to autodetect the format of the log (no fields + field format
  match) and detecting of bogus lines, e.g. one half log line (due to
  crash) + one full line...

- wishlist: implement a mechanism to translate IP address to FQDN.
  Two possibilities:
    1) A lr_gethostbyaddr filter which works on the DLF
    2) A lr_xml2resolver filter which works on the XML report.
  Useful for DNS, firewall, www superservice (at least).

- wishlist: improve ftp reports:
  1: Sort by file. It would display each time a particular file was accessed by
     a particular person and it would list the time.
  2: Sort by user and file. It would display each time a particular file was
     accessed by a particular person.
  3: Sort by user. It would show all files and times accessed by a particular
     user.
  reported by Brett Simpson on Thu, 15 Nov 2001 09:46:59 -0500, in
  Message-Id: <sbf38f1b.032@GroupWise>, Subject: FTP improvement questions

  Furthermore:
  1: Show the number of failed logins.
  2: Show who failed to login and at what time.
  3: Show the number of successful logins.
  4: Show who was successful to login and at what time.
  5: Differentiate between downloads and uploads.
  reported by Brett Simpson on Wed, 21 Nov 2001 09:59:08 -0500 in Message-Id: 
  <sbfb7aef.077@GroupWise>, Subject: Another idea for FTP reporting.

  This might better be handled in a new 'login' super service, however.  JvB.

- wishlist: sablotron 0.60 doesn't generate proper indentation with ascii.xsl
    comment by Egon: indentation with xsltproc is fixed with <xsl:strip-space>
             but sablotron does not respond to it... xalan-c also works
             correctly
    comment by joostvb: severity lowered from normal to wishlist: we're not
     planning to support sablotron or xalan-c.  We strongly recommend libxslt.

- wishlist: Xalan-C does not handle logml.xsl correctly, which results in bad
    LogML output for edges in www superservice reports.
    comment by joostvb: severity lowered from normal to wishlist: we're not
     planning to support sablotron or xalan-c.  We strongly recommend libxslt.

- wishlist: Make Lire more userfriendly:
  - Make sure Lire's error messages are understandable for newbies.
  - write a faq: - why is your lr_log2report commandline so bloated?
   - why do the errormessages look so strange?
   - why do you advise to create a dedicated user account? why not 'nobody'
    or 'daemon'?

- wishlist: support web page traversals (two and three page are common), a top X
   would be nice

- wishlist: better PDF look for documentation through the use of
   cvs-hibou/hibou/doc/foundation/print.dsl