INSTALLATION: Please read the entire file!!

Operation
---------

Logcheck contains several files:

logcheck.sh -- The main script. This file controls all processing and looks 
at log files with simple grep commands. This file is executed on a timed 
basis from cron and reports findings to the sysadmin.

logtail -- A custom executable that remembers the last position of a text 
file. This program is used by logcheck to parse out information from the 
last time the log was opened, this prevents reviewing old material twice. 
All log files will be processed with this program and will have a file named 
"########.offset" put in the same directory, where ####### is the name of 
the log file checked. This file contains the decimal offset information for 
logtail to work. If you delete it, logtail will parse the file from the 
beginning again. Logcheck tracks the size and inode of log files to 
enable it to tell when a log file has been rotated. If the inode of the 
log changes, or the file size is smaller than the last run, logtail will 
reset the counter and parse the entire file. 

logcheck.hacking -- This file contains keywords that are certifiable attacks 
on your system. I leave this file sparse, unless I know what a certain 
pattern of attack looks like (The default keywords are almost always 
generated by Internet Security Scanner attacks, or sendmail(8) if it is 
being fed illegal syntax in address lines). Any keyword in a log file that 
matches here will generate a report with a more obnoxious header to grab 
your attention faster: "ACTIVE SYSTEM ATTACK"

logcheck.violations -- This file contains keywords of system events that 
are usually seen as negative. Words such as "denied", "refused", etc. 
Positive words such as 'su' successes are also put in here. This file is of 
course not all inclusive and is heavily biased towards FWTK messages and 
BSDish messages with TCP wrappers installed. Violations here are reported 
under the heading "Security Violations" in the reports. 

logcheck.violations.ignore -- This file contains words that are reverse 
searched against the logcheck.violations file. If these words are found, 
that entry is not reported. An example of this are the following log entries:

Feb 28 21:00:08 nemesis sendmail[5475]: VAA05473: to=crowland, ctladdr=root 
(0/0), delay=00:00:02, xdelay=00:00:01, mailer=local, stat=refused

Feb 28 22:13:53 nemesis rshd: refused connect from hacker@evil.com:1490
 
The top entry is from sendmail and is a fairly common error, the stat line 
indicates that the remote host refused connections (stat=refused). This can 
happen for a variety of reasons and generally is not a problem.

The bottom line however indicates that a person (hacker@evil.com) has tried 
unsuccessfully to start an rsh session on my machine, this is bad (of 
course you shouldn't be running rshd to begin with).

The logcheck.violations file will find the word 'refused' and will flag it 
to be logged, however this will report both instances as being bad and you 
will get false alarms from sendmail (both had the word refused). By putting 
the following in the logcheck.violations.ignore file you tell logcheck to 
ignore the sendmail problem and it will only report to you the bad rsh 
connection:

(in logcheck.violations.ignore)

mailer=local, stat=refused

This will prevent reports from any line that contains "refused" but has the 
rest of the keywords "mailer=local, stat=refused." This is of course pretty 
basic, and not very intelligent, however you must remember that by forcing 
you to be specific in what you ignore, you will not overlook something 
important. A word of caution though, if you don't pick a long enough string 
to put in the logcheck.violations.ignore file then you could ignore 
significant events. Be very very careful what you put in here. The default 
file has only one entry in it to allow grep to run. Tune it to your system 
carefully! If the above did not make sense at all, leave the file as it is.

logcheck.ignore -- This file is the catch-all file for words to look for in 
the logs and to NOT REPORT. Again be specific with what you want to ignore 
and go easy on the wildcards. Anything that does not match what is in this 
file is reported (so you don't risk missing anything) as "Unusual System 
Activity." The default is again BSDish and biased towards FWTK and TCP 
Wrappers.

To preserve integrity of the scans the following search order and rules are 
kept:

1) Active System Attacks are reported first.
2) Security Violations are reported second.
3) Unusual System Events are reported last.

Keyword searches on the logcheck.hacking and logcheck.violations file are 
CASE INSENSITIVE to ensure we don't miss anything. Keyword searches on the 
logcheck.violations.ignore and logcheck.ignore files are CASE SENSITIVE 
to ensure again that we don't miss anything. The *.ignore files REQUIRE you 
to put in the exact text as part of the contents. The more sensitive 
logcheck.violations and logcheck.hacking files will report on any word,
regardless of case, that is found as a match.

The whole process follows the following structure:

logcheck.sh executes hourly ----> 
logcheck.sh executes logtail on log files ----> 
logtail parses off any text from the last time it was run ---> 
logcheck greps text for system attack messages ---> 
logcheck greps text for security violations ---> 
logcheck greps text for security violations to ignore ---> 
logcheck greps text for all messages to ignore. ---> 
any messages found are mailed to system admin.

Overall it's a very simple process and is surprisingly good at telling you 
information about your system you were never aware of, but probably should 
have been.


Installation
------------

If you know what a syslog.conf file is, know you have it set up to log 
as much information as possible, AND HAVE SECURED THE LOGS, go to step TWO, 
otherwise you should read step ONE.

Step ONE: Configuring syslog daemon and SECURING your log files.

Before setting up logcheck, you should ensure that your system is not only 
running syslog, but that you have it configured for maximum logging. On 
most all systems I recommend that you send all syslog messages to ONE file 
for logcheck to parse through. This configuration ensures that messages 
will not be missed. On BSDish systems this involves editing the file 
syslog.conf located in /etc. This file contains parameters for syslogd and 
if you don't understand them, PLEASE check:

man syslog.conf

- OR -

A book.

Many syslog.conf files are sensitive to using tabs instead of spaces for 
your entries and you will mysteriously hose syslogd daemon if you put in 
spaces so be careful.

In the syslog.conf file you should put in an entry like this:

*.info						/var/log/messages

Which will log EVERYTHING to the file "messages" located in /var/log. 
Obviously you should substitute /var/log to the directory typically found 
on your system. For BSDI and most variants this is /var/log for Linux this 
is /var/adm. Your syslog.conf file for your site will have the default in 
it. Remember this will log everything, if you have a very high volume 
server (for instance: mail) you may want to cut back on the logging to prevent 
over running of disk space. You can do this in the following way:

*.info;mail.none				/var/log/messages
mail.notice					/var/log/messages

This will only log non-standard mail messages, I don't recommend this 
however as it will make it hard to track mail into and out of your system 
if someone attempts, or succeeds, to gain entry. Many systems have separate
log files for different system services, configuring syslog to do this could
look like the following

*.info;mail,ftp,daemon,authpriv.none		/var/log/messages
mail.info					/var/log/mail.log
ftp.info					/var/log/ftp.log
daemon.info					/var/log/daemon.log
authpriv.*					/var/log/secure.log

This configuration will have separate logs for the general system 
messages, mail, ftp, daemon and security messages. Logcheck can be setup to 
check for all of them. Again please see your syslog.conf man 
page for more information.

Now that you have edited your syslog.conf file you need to re-start syslogd 
by sending the HUP signal to it.

IMPORTANT: You must now go to your log directory (/var/log in the example 
above) and change the log file to owner root, group wheel and mode 600 on 
file permissions. First check if the file exists if it doesn't, you should 
make it. For example if your log files is simply called 'messages' you 
would do the following:

touch /var/log/messages

Now you must ensure that you change the permissions in the following way:

chown root.wheel /var/log/messages
chmod 600 /var/log/messages

I also recommend that any other log files have their permissions set in a 
similar way (at least to mode 600 if you can't change the owner/group). Log 
files contain very sensitive data about system operations and could 
contain passwords, system errors, and other data that can reveal 
vulnerabilites if you are not careful. I personally feel that these files 
should never be readable by any person other than root. 

BSD and FreeBSD: You should go to the /etc directory and edit 
the /etc/daily, /etc/weekly, and /etc/monthly scripts and change the 
'rotate()' script function to change the log permissions on rotation. 
Simply change the line:

cp /dev/null "$file"; chmod 644 "$file"

To:

cp /dev/null "$file"; chmod 600 "$file"

(The above is for BSDI 2.x, BSDI 3.x uses an external rotate
function now, just change the mode sent to it from 644 to 600
and you'll be OK. FreeBSD will be similiar to the BSDI 2.x script)

When logs are auto-rotated they will have the permissions set automatically.

Once these steps have been completed you can move onto step TWO:

Step TWO: Logcheck and logtail installation.

Logcheck requires the following files to run:

logcheck.sh
logtail.c
logcheck.hacking
logcheck.violations
logcheck.violations.ignore
logcheck.ignore

Pull logcheck.sh into your favorite editor and find the section entitled: 
CONFIGURATION SECTION.

Change the name of the SYSADMIN variable to one of your liking. You can use
local names (default is root), or e-mail addresses for remote logging.

Go to the section entitled: LOG FILE CONFIGURATION SECTION and either 
uncomment the log files that apply to you, or add your own. Be sure 
you know the difference between the > and >> operators before you do this. 

If you have one of the default system types (Linux, BSDI, FreeBSD,
SunOS, Digital) you can simply type "make <systype>" and it will
install for you at this point.

If you are using an alternate path for the files (i.e NOT in
/usr/local/whatever), you need to change the path entries for
logcheck.hacking, logcheck.violations, logcheck.ignore, 
logcheck.violations.ignore, and logtail in the main logcheck.sh script. I 
don't recommend you do this unless you have to.  

If you changed the default paths /usr/local/etc and /usr/local/bin in 
the logcheck.sh file you need to edit the Makefile and change INSTALLDIR 
and INSTALLDIR_BIN to point to the same directories. 

Note that the Makefile will create a directory called /usr/local/etc/tmp by 
default. This is the scratch area for logcheck to handle it's files. I do 
NOT recommend that you use /tmp for any reason as it is publically 
accessible and may pose a danger if a user creates symbolic links to trick 
the logcheck script into overwriting an important system file. I would
also change all automated system scripts to use this directory instead of
/tmp which is notoriously unsafe. 


Editing Cron
------------

After installing logcheck, you should edit your local crontab file for root 
and set logcheck to run once per hour (recommended, although you can do it 
more frequently, or less frequently, although the absolute minimum in my 
opinion is once every few hours or so). Examples are the following:

Hourly check (BSD Systems and Redhat /etc/crontab):

00 * * * * root /bin/sh /usr/local/etc/logcheck.sh

15 Minute check (Linux Slackware Systems /var/spool/cron/crontabs/root):

00,15,30,45 * * * * /usr/local/etc/logcheck.sh


The 15 minute check I would recommend for firewalls that generally don't 
produce messages unless they are in trouble.

Remember, logcheck does not report anything if it has nothing useful to say 
(only if the rest of USENET could do this). So running it every 15 
minutes will have no impact on your mailbox if the system being watched is 
quiet. Busier systems can be addressed by less frequent reporting, however 
longer reports mean you must spend more time analyzing them, and you may 
not like this either. Again I recommend hourly.

After you have edited the crontab, you must send the crond daemon a HUP to 
reset it.


Final Check and Troubleshooting
-------------------------------

You are almost done. I recommend that you run the logcheck.sh script by 
hand to ensure that it reads the log files without errors and sends mail to 
the right account. You should check that syslog is in fact logging events 
and that you generate an event or two (just su to root) for logcheck to 
report. You should have a report mailed to you. Run logcheck multiple times 
to ensure that you don't get repeat messages. 

If you get repeat messages then something is wrong with the logtail program 
(not marking the file correctly) and you should check your log directory 
for files that end in *.offset. If you do not see these files there you have 
a problem with the logtail binary. Run logtail manually against these logs 
to see if it correctly marks the file offsets. If it still refuses to do it 
you should check your file permissions and make sure that you are 
running all this as root. 

Default file permissions for logcheck files:

logcheck* -- 600 -- Read/Write for root ONLY. Owner root. Group Wheel.
logtail*  -- 700 -- Read/Write/Execute for root ONLY. Owner root. Group Wheel.

*** NOTE: None of the files should ever be SUID root for any reason. 

For the more cautious, you may want to create a special group/user that owns 
the logfiles and have the logfile scanner run as that user. Configuration 
to do this should be rather straight forward, just adjust the file
permissions on the logcheck files accordingly and ensure they can read 
and write to the directory where the logs are stored. 

If you have any questions you can send e-mail for some limited help...
unfortunately my schedule keeps me rather busy so be patient with the 
response...

Thanks,

Craig Rowland

crowland@psionic.com