1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330
|
ULOGD - the Userspace Logging Daemon
Harald Welte <laforge@gnumonks.org>
Revision 1.5, 2001/05/20 15:07:45
This is the documentation for ulogd, the Userspace logging daemon.
ulogd makes use of the Linux 2.4 firewalling subsystem (netfilter) and
the ULOG target for netfilter.
______________________________________________________________________
Table of Contents
1. DESIGN
1.1 CONECEPT
1.2 DETAILS
2. INSTALLATION
2.1 Linux kernel
2.2 netfilter / iptables
2.3 ulogd
2.3.1 Recompiling the source
2.3.2 Using a precompiled package
3. Configuration
3.1 netfilter
3.2 ulogd
4. Available plugins
5. QUESTIONS / COMMENTS
______________________________________________________________________
1. DESIGN
1.1. CONECEPT
I want to provide a flexible, almost universal logging daemon for my
netfilter ULOG target. It is not optimized in any way, the goal is to
keep as simple as possible. These are my thoughts about how the
architecture which is most capable of doing that:
Interpreter lugins
It should be possible to add plugins / runtime modules for new
protocols, etc. For example the standard logging daemon
provides source-ip, dest-ip, source-port, dest-port, etc.
Logging for variuos other protocols (GRE, IPsec, ...) may be
implemented as modules.
Output plugins
... describe how and where to put the information gained by
logging plugins. The easiest way is to build a line per packet
and fprint it to a file. Some people might want to log into a
SQL database or want an output conforming to the intrusion
detection systems communication draft from the IETF.
1.2. DETAILS
The major clue is providing a framework which is as flexible as
possible. Nobody knows what strange network protocols are out there
:) Flexibility depends on the communication between the output of the
logging plugins and input of the output plugins.
Rusty advised me to use some kind of type-key-value triples, but I
think this is the total overkill and is too complicated for me to
implement it in a reasonable short period of time. (3 hours later)
Hmm... Rusty finally convinced me to use linked lists of type-key-
value triples - and it wasn't that difficult.
Another issue is, of course, performance. Up to ulogd 0.3, ulogd did
several linked list iterations and about 30 malloc() calls _per
packet_. This changed with the new 0.9 revision:
o Not a single dynamic allocation in the core during runtime.
Everything is pre-allocated at start of ulogd to provide the
highest possible throughput.
o Hash tables in addition to the linked lists. Linked lists are only
traversed if we really want to access each element of the list.
2. INSTALLATION
2.1. Linux kernel
First you will need a recent 2.4.x kernel. At the time this document
was written, 2.4.0-test11-pre5 was the latest development version.
Ulogd should work with all kernels >= 2.4.0-test4.
2.2. netfilter / iptables
In addition you need the latest iptables package, or even better: the
latest CVS snapshot. A description how to obtain this is provided on
the netfilter homepage <http://netfilter.kernelnotes.org>.
ulogd is based on a special netfilter extension, called the netfilter
ULOG target module. You have to patch this extension into your kernel,
as it has not been integrated into the main kernel yet. To make this
as easy as possible, netfilter provides the 'patch-o-matic' subsystem.
To run patch-o-matic, just type
make patch-o-matic
in the userspace directory of netfilter CVS.
2.3. ulogd
2.3.1. Recompiling the source
Download the ulogd package from
<http://www.gnumonks.org/ftp/pub/netfilter/> and untar it.
If you want to build ulogd with MySQL support, type './configure
--with-mysql'. You may also have to specify the path of the mysql
libraries using '--with-mysql=path'. To build ulogd without MySQL
support, just use './configure'.
To compile and install the program, call 'make install'.
2.3.2. Using a precompiled package
I also provide a SRPM, which should compile on almost any rpm-based
distribution. It is available at
<http://www.gnumonks.org/ftp/pub/netfilter/>
Just download the package and do the usual 'rpm --rebuild <file>'.
3. Configuration
3.1. netfilter
Just add rules using the ULOG target to your firewalling chain. A very
basic example:
iptables -A FORWARD -j ULOG --ulog-nlgroup 32 --prefix foo
To increase logging performance, try to use the
--ulog-qthreshold N
option (where 1 < N <= 50). The number you specify is the amout of
packets batched together in one multipart netlink message. If you set
this to 20, the kernel schedules ulogd only once every 20 packets. All
20 packets are then processed by ulogd. This reduces the number of
context switches between kernel and userspace.
Of course you can combine the ULOG target with the different netfilter
match modules. For a more detailed description, have a look at the
netfilter HOWTO's, available on the netfilter homepage.
3.2. ulogd
All configurable parameters of ulogd are in the configfile
'/etc/ulogd.conf'
The following configuration parameters are available:
nlgroup
The netlink multicast group, which ulgogd should bind to. This
is the same as given with the '--ulog-nlgroup' option to
iptables.
logfile
The main logfile, where ulogd reports any errors, warnings and
other unexpected conditions.
loglevel
This specifies, how verbose the logging to logfile is. Currently
defined loglevels are: 1=debug information, 3=informational
messages, 5=noticable exceptional conditions, 7=error
conditions, 8=fatal errors, program abort.
plugin
This option is followed by a filename of a ulogd plugin, which
ulogd shold load upon initialization. This option may appear
more than once.
4. Available plugins
ulogd comes with the following plugins:
ulogd_BASE.so
Basic interpreter plugin for nfmark, timestamp, mac address, ip
header, tcp header, udp header, icmp header, ah/esp header.
ulogd_PWSNIFF.so
Example interpreter plugin to log plaintext passwords as used
with FTP and POP3. Don't blame me for writing this plugin! The
protocols are inherently insecure, and there are a lot of other
tools for sniffing passwords... it's just an example.
ulogd_OPRINT.so
A very simple output module, dumping all packets in the format
===>PACKET BOUNDARY
key=value
key=value
...
===>PACKET BOUNDARY
...
to a file.
The module defines the following configuration directives:
dumpfile
The filename where it should log to. The default is
/var/log/ulogd.pktlog
ulogd_LOGEMU.so
An output module which tries to emulate the old syslog-based LOG
targed as far as possible. Logging is done to a textfile instead
of syslog, though.
The module defines the following configuration directives:
syslogfile
The filename where it should log to. The default is
/var/log/ulogd.syslogemu
syslogsync
Set this to 1 if you want to have your logfile written
synchronously. This may reduce performance, but makes your
log-lines appear immediately. The default is 0
ulogd_MYSQL.so
An output plugin for logging into a mysql database. This is only
compiled if you have the mysql libraries installed, and the
configure script was able to detect them. (that is: --with-mysql
was specified for ./configure)
The plugin automagically inserts the data into the configured
table; It connects to mysql during the startup phase of ulogd
and obtains a list of the columns in the table. Then it tries to
resolve the column names against keys of interpreter plugins.
This way you can easly select which information you want to log
- just by the layout of the table.
If, for example, your table contains a field called 'ip_saddr',
ulogd will resolve this against the key 'ip.saddr' and put the
ip address as 32bit unsigned integer into the table.
You may want to have a look at the file 'doc/mysql.table' as an
example table including fields to log all keys from
ulogd_BASE.so. Just delete the fields you are not interested in,
and create the table.
The module defines the following configuration directives:
mysqltable
Name of the table to which ulogd should log
mysqldb
Name of the mysql database
mysqlhost
Name of the mysql database host
mysqluser
Name of the mysql user
mysqlpass
Password for mysql
5. QUESTIONS / COMMENTS
All comments / questions / ... are appreciated.
Just drop me a note to laforge@gnumonks.org
Please note also that there is now a mailinglist,
ulogd@lists.gnumonks.org. You can subscribe at
<http://lists.gnumonks.org/mailman/listinfo/ulogd/>
|