1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<article>
<!--$Id$-->
<articleinfo>
<title>Shorewall Traffic Accounting</title>
<authorgroup>
<author>
<firstname>Tom</firstname>
<surname>Eastep</surname>
</author>
</authorgroup>
<pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
<copyright>
<year>2003-2009</year>
<holder>Thomas M. Eastep</holder>
</copyright>
<legalnotice>
<para>Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version
1.2 or any later version published by the Free Software Foundation; with
no Invariant Sections, with no Front-Cover, and with no Back-Cover
Texts. A copy of the license is included in the section entitled
<quote><ulink url="GnuCopyright.htm">GNU Free Documentation
License</ulink></quote>.</para>
</legalnotice>
</articleinfo>
<caution>
<para><emphasis role="bold">This article applies to Shorewall 4.0 and
later. If you are running a version of Shorewall earlier than Shorewall
4.0.0 then please see the documentation for that
release</emphasis>.</para>
</caution>
<section id="Basics">
<title>Accounting Basics</title>
<para>Shorewall accounting rules are described in the file
<filename><filename>/etc/shorewall/accounting</filename></filename>. By
default, the accounting rules are placed in a chain called
<quote>accounting</quote> and can thus be displayed using
<quote>shorewall[-lite] show -x accounting</quote>. All traffic passing
into, out of, or through the firewall traverses the accounting chain
including traffic that will later be rejected by interface options such as
<quote>tcpflags</quote> and <quote>maclist</quote>.</para>
<para>The columns in the accounting file are described in <ulink
url="manpages/shorewall-accounting.html">shorewall-accounting</ulink> (5)
and <ulink
url="manpages6/shorewall6-accounting.html">shorewall6-accounting</ulink>
(5).</para>
<para>In all columns except ACTION and CHAIN, the values <quote>-</quote>,
<quote>any</quote> and <quote>all</quote> are treated as
wild-cards.</para>
<para>The accounting rules are evaluated in the Netfilter
<quote>filter</quote> table. This is the same environment where the
<quote>rules</quote> file rules are evaluated and in this environment,
DNAT has already occurred in inbound packets and SNAT has not yet occurred
on outbound packets.</para>
<para>Accounting rules are not stateful -- each rule only handles traffic
in one direction. For example, if eth0 is your Internet interface, and you
have a web server in your DMZ connected to eth1, then to count HTTP
traffic in both directions requires two rules:</para>
<programlisting> #ACTION CHAIN SOURCE DESTINATION PROTOCOL DEST SOURCE
# PORT PORT
DONE - eth0 eth1 tcp 80
DONE - eth1 eth0 tcp - 80</programlisting>
<para>Associating a counter with a chain allows for nice reporting. For
example:</para>
<programlisting> #ACTION CHAIN SOURCE DESTINATION PROTOCOL DEST SOURCE
# PORT PORT
web:COUNT - eth0 eth1 tcp 80
web:COUNT - eth1 eth0 tcp - 80
web:COUNT - eth0 eth1 tcp 443
web:COUNT - eth1 eth0 tcp - 443
DONE web</programlisting>
<para>Now <command>shorewall show web</command> (or
<command>shorewall-lite show web</command> for Shorewall Lite users) will
give you a breakdown of your web traffic:</para>
<programlisting> [root@gateway shorewall]# shorewall show web
Shorewall-1.4.6-20030821 Chain web at gateway.shorewall.net - Wed Aug 20 09:48:56 PDT 2003
Counters reset Wed Aug 20 09:48:00 PDT 2003
Chain web (4 references)
pkts bytes target prot opt in out source destination
11 1335 tcp -- eth0 eth1 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
18 1962 tcp -- eth1 eth0 0.0.0.0/0 0.0.0.0/0 tcp spt:80
0 0 tcp -- eth0 eth1 0.0.0.0/0 0.0.0.0/0 tcp dpt:443
0 0 tcp -- eth1 eth0 0.0.0.0/0 0.0.0.0/0 tcp spt:443
29 3297 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
[root@gateway shorewall]#</programlisting>
<para>Here is a slightly different example:</para>
<programlisting> #ACTION CHAIN SOURCE DESTINATION PROTOCOL DEST SOURCE
# PORT PORT
web - eth0 eth1 tcp 80
web - eth1 eth0 tcp - 80
web - eth0 eth1 tcp 443
web - eth1 eth0 tcp - 443
COUNT web eth0 eth1
COUNT web eth1 eth0</programlisting>
<para>Now <command>shorewall show web</command> (or
<command>shorewall-lite show web</command> for Shorewall Lite users)
simply gives you a breakdown by input and output:</para>
<programlisting> [root@gateway shorewall]# shorewall show accounting web
Shorewall-1.4.6-20030821 Chains accounting web at gateway.shorewall.net - Wed Aug 20 10:27:21 PDT 2003
Counters reset Wed Aug 20 10:24:33 PDT 2003
Chain accounting (3 references)
pkts bytes target prot opt in out source destination
8767 727K web tcp -- eth0 eth1 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
0 0 web tcp -- eth0 eth1 0.0.0.0/0 0.0.0.0/0 tcp dpt:443</programlisting>
<programlisting>
11506 13M web tcp -- eth1 eth0 0.0.0.0/0 0.0.0.0/0 tcp spt:80
0 0 web tcp -- eth1 eth0 0.0.0.0/0 0.0.0.0/0 tcp spt:443
Chain web (4 references)
pkts bytes target prot opt in out source destination
8767 727K all -- eth0 eth1 0.0.0.0/0 0.0.0.0/0
11506 13M all -- eth1 eth0 0.0.0.0/0 0.0.0.0/0
[root@gateway shorewall]#</programlisting>
<para>Here's how the same example would be constructed on an HTTP server
with only one interface (eth0).</para>
<caution>
<para>READ THE ABOVE CAREFULLY -- IT SAYS <emphasis
role="bold">SERVER</emphasis>. If you want to account for web browsing,
you have to reverse the rules below.</para>
</caution>
<programlisting> #ACTION CHAIN SOURCE DESTINATION PROTOCOL DEST SOURCE
# PORT PORT
web - eth0 - tcp 80
web - - eth0 tcp - 80
web - eth0 - tcp 443
web - - eth0 tcp - 443
COUNT web eth0
COUNT web - eth0</programlisting>
<para>Note that with only one interface, only the SOURCE (for input rules)
or the DESTINATION (for output rules) is specified in each rule.</para>
<para>Here's the output:</para>
<programlisting> [root@mail shorewall]# shorewall show accounting web Shorewall-1.4.7
Chains accounting web at mail.shorewall.net - Sun Oct 12 10:27:21 PDT 2003
Counters reset Sat Oct 11 08:12:57 PDT 2003
Chain accounting (3 references)
pkts bytes target prot opt in out source destination
8767 727K web tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
11506 13M web tcp -- * eth0 0.0.0.0/0 0.0.0.0/0 tcp spt:80
0 0 web tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443
0 0 web tcp -- * eth0 0.0.0.0/0 0.0.0.0/0 tcp spt:443
Chain web (4 references)
pkts bytes target prot opt in out source destination
8767 727K all -- eth0 * 0.0.0.0/0 0.0.0.0/0
11506 13M all -- * eth0 0.0.0.0/0 0.0.0.0/0
[root@mail shorewall]#</programlisting>
<para>For an example of integrating Shorewall Accounting with MRTG, see
<ulink
url="http://www.nightbrawler.com/code/shorewall-stats/">http://www.nightbrawler.com/code/shorewall-stats/</ulink>.</para>
</section>
<section id="Bridge">
<title>Accounting with Bridges</title>
<para>The structure of the accounting rules changes slightly when there
are <ulink url="bridge-Shorewall-perl.html">bridges</ulink> defined in the
Shorewall configuration. Because of the restrictions imposed by Netfilter
in kernel 2.6.21 and later, output accounting rules must be segregated
from forwarding and input rules. To accomplish this separation,
Shorewall-perl creates two accounting chains:</para>
<itemizedlist>
<listitem>
<para><emphasis role="bold">accounting</emphasis> - for input and
forwarded traffic.</para>
</listitem>
<listitem>
<para><emphasis role="bold">accountout</emphasis> - for output
traffic.</para>
</listitem>
</itemizedlist>
<para>If the CHAIN column contains <quote>-</quote>, then:</para>
<itemizedlist>
<listitem>
<para>If the SOURCE column in a rule includes the name of the firewall
zone (e.g., $FW), then the default chain to insert the rule into is
<emphasis role="bold">accountout</emphasis> only.</para>
</listitem>
<listitem>
<para>Otherwise, if the DEST in the rule is <emphasis
role="bold">any</emphasis> or <emphasis role="bold">all</emphasis> or
0.0.0.0/0, then the rule is added to both <emphasis
role="bold">accounting</emphasis> and <emphasis
role="bold">accountout</emphasis>.</para>
</listitem>
<listitem>
<para>Otherwise, the rule is added to <emphasis
role="bold">accounting</emphasis> only.</para>
</listitem>
</itemizedlist>
</section>
<section>
<title>Sectioned Accounting Rules</title>
<para>Traditionally, the root of the Shorewall accounting rules has been
the <emphasis role="bold">accounting</emphasis> chain. Having a single
root chain has drawbacks:</para>
<itemizedlist>
<listitem>
<para>Many rules are traversed needlessly (they could not possibly
match traffic).</para>
</listitem>
<listitem>
<para>At any time, the Netfilter team could begin generating errors
when loading those same rules.</para>
</listitem>
<listitem>
<para>MAC addresses may not be used in the accounting rules.</para>
</listitem>
<listitem>
<para>The <emphasis role="bold">accounting</emphasis> chain cannot be
optimized when OPTIMIZE_ACCOUNTING=Yes.</para>
</listitem>
<listitem>
<para>The rules may be defined in any order so the rules compiler must
post-process the ruleset to ensure that there are no loops and to
alert the user to unreferenced chains.</para>
</listitem>
</itemizedlist>
<para>Beginning with Shorewall 4.4.18, the accounting structure can be
created with three root chains:</para>
<itemizedlist>
<listitem>
<para><emphasis role="bold">accountin</emphasis>: Rules that are valid
in the <emphasis role="bold">INPUT</emphasis> chain (may not specify
an output interface).</para>
</listitem>
<listitem>
<para><emphasis role="bold">accountout</emphasis>: Rules that are
valid in the OUTPUT chain (may not specify an input interface or a MAC
address).</para>
</listitem>
<listitem>
<para><emphasis role="bold">accounting</emphasis>: Other rules.</para>
</listitem>
</itemizedlist>
<para>The new structure is enabled by sectioning the accounting file in a
manner similar to the <ulink url="manpages/shorewall-rules.html">rules
file</ulink>. The sections are <emphasis role="bold">INPUT</emphasis>,
<emphasis role="bold">OUTPUT</emphasis> and <emphasis
role="bold">FORWARD</emphasis> and must appear in that order (although any
of them may be omitted). The first non-commentary record in the accounting
file must be a section header when sectioning is used. </para>
<para>Beginning with Shorewall 4.4.20, the ACCOUNTING_TABLE setting was
added to shorewall.conf and shorewall6.conf. That setting determines the
Netfilter table (filter or mangle) where the accounting rules are added.
When ACCOUNTING_TABLE=mangle is specified, the available sections are
<emphasis role="bold">PREROUTING</emphasis>, <emphasis
role="bold">INPUT</emphasis>, <emphasis role="bold">OUTPUT</emphasis>,
<emphasis role="bold">FORWARD</emphasis> and <emphasis
role="bold">POSTROUTING</emphasis>.</para>
<para>Section headers have the form:</para>
<para><option>SECTION</option>
<replaceable>section-name</replaceable></para>
<para>When sections are enabled:</para>
<itemizedlist>
<listitem>
<para>You must jump to a user-defined accounting chain before you can
add rules to that chain.</para>
</listitem>
<listitem>
<para>This eliminates loops and unreferenced chains.</para>
</listitem>
<listitem>
<para>You may not specify an output interface in the <emphasis
role="bold">PREROUTING</emphasis> and <emphasis
role="bold">INPUT</emphasis> sections.</para>
</listitem>
<listitem>
<para>In the <emphasis role="bold">OUTPUT</emphasis> and <emphasis
role="bold">POSTROUTING</emphasis> sections:</para>
<itemizedlist>
<listitem>
<para>You may not specify an input interface</para>
</listitem>
<listitem>
<para>You may not jump to a chain defined in the <emphasis
role="bold">INPUT</emphasis> or <emphasis
role="bold">PREROUTING</emphasis> sections that specifies an input
interface</para>
</listitem>
<listitem>
<para>You may not specify a MAC address</para>
</listitem>
<listitem>
<para>You may not jump to a chain defined in the <emphasis
role="bold">INPUT</emphasis> or <emphasis
role="bold">PREROUTING</emphasis> section that specifies a MAC
address.</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>The default value of the CHAIN column is:</para>
<itemizedlist>
<listitem>
<para><emphasis role="bold">accountin</emphasis> in the <emphasis
role="bold">INPUT</emphasis> section</para>
</listitem>
<listitem>
<para><emphasis role="bold">accounout</emphasis> in the <emphasis
role="bold">OUTPUT</emphasis> section</para>
</listitem>
<listitem>
<para><emphasis role="bold">accountfwd</emphasis> in the <emphasis
role="bold">FORWARD</emphasis> section</para>
</listitem>
<listitem>
<para><emphasis role="bold">accountpre</emphasis> in the <emphasis
role="bold">PREROUTING</emphasis> section</para>
</listitem>
<listitem>
<para><emphasis role="bold">accountpost</emphasis> in the
<emphasis role="bold">POSTROUTING</emphasis> section</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>Traffic addressed to the firewall goes through the rules defined
in the INPUT section.</para>
</listitem>
<listitem>
<para>Traffic originating on the firewall goes through the rules
defined in the OUTPUT section.</para>
</listitem>
<listitem>
<para>Traffic being forwarded through the firewall goes through the
rules from the FORWARD sections.</para>
</listitem>
</itemizedlist>
<para>Here is a sample sectioned file that used <link
linkend="perIP">Per-IP Accounting</link>.</para>
<caution>
<para>In this example, the dmz net corresponds to a vserver zone so
lives on the firewall itself.</para>
</caution>
<programlisting>#ACTION CHAIN SOURCE DESTINATION PROTO DEST SOURCE USER/ MARK IPSEC
# PORT(S) PORT(S) GROUP
SECTION INPUT
ACCOUNT(fw-net,$FW_NET) - COM_IF
ACCOUNT(dmz-net,$DMZ_NET) - COM_IF
SECTION OUTPUT
ACCOUNT(fw-net,$FW_NET) - - COM_IF
ACCOUNT(dmz-net,$DMZ_NET) - - COM_IF
SECTION FORWARD
ACCOUNT(loc-net,$INT_NET) - COM_IF INT_IF
ACCOUNT(loc-net,$INT_NET) - INT_IF COM_IF
</programlisting>
</section>
<section id="Collectd">
<title>Integrating Shorewall Accounting with Collectd</title>
<para>Sergiusz Pawlowicz has written a nice article that shows how to
integrate Shorewall Accounting with collectd to produce nice graphs of
traffic activity. The article may be found at <ulink
url="http://collectd.org/wiki/index.php/Plugin:IPTables">http://collectd.org/wiki/index.php/Plugin:IPTables</ulink>.</para>
</section>
<section id="perIP">
<title>Per-IP Accounting</title>
<para>Shorewall 4.4.17 added support for per-IP accounting using the
ACCOUNT target. That target is only available when xtables-addons is
installed. This support has been successfully tested with xtables-addons
1.32 on:</para>
<itemizedlist>
<listitem>
<para>Fedora 14</para>
</listitem>
<listitem>
<para>Debian Squeeze</para>
</listitem>
<listitem>
<para>OpenSuSE 11.3</para>
</listitem>
</itemizedlist>
<para>and xtables-addons Version 1.21 on:</para>
<itemizedlist>
<listitem>
<para>Debian Lenny</para>
</listitem>
</itemizedlist>
<para>Information about xtables-addons installation may be found at <ulink
url="Dynamic.html#xtables-addons">here</ulink>.</para>
<para>Per-IP accounting is configured in <ulink
url="manpages/shorewall-accounting.html">shorewall-accounting</ulink> (5)
(it is currently not supported in IPv6). In the ACTION column,
enter:</para>
<simplelist>
<member><emphasis
role="bold">ACCOUNT(</emphasis><replaceable>table</replaceable>,<replaceable>network</replaceable><emphasis
role="bold">)</emphasis></member>
</simplelist>
<para>where</para>
<simplelist>
<member><replaceable>table</replaceable> is the name of an accounting
table (you choose the name). All rules specifying the same table will
have their per-IP counters accumulated in that table.</member>
<member><replaceable>network</replaceable> is an IPv4 network in CIDR
notation. The network can be as large as a /8 (class A).</member>
</simplelist>
<para>One nice feature of per-IP accounting is that the counters survive
<command>shorewall restart</command>. This has a downside, however. If you
change the network associated with an accounting table, then you must
<command>shorewall stop; shorewall start</command> to have a successful
restart (counters will be cleared).</para>
<para>Example: Suppose your WAN interface is eth0 and your LAN interface
is eth1 with network 172.20.1.0/24. To account for all traffic between the
WAN and LAN interfaces:</para>
<programlisting>#ACTION CHAIN SOURCE DEST ...
ACCOUNT(net-loc,172.20.1.0/24) - eth0 eth1
ACCOUNT(net-loc,172.20.1.0/24) - eth1 eth0</programlisting>
<para>This will create a <emphasis role="bold">net-loc</emphasis> table
for counting packets and bytes for traffic between the two
interfaces.</para>
<para>The table is dumped using the <command>iptaccount</command> utility
(part of xtables-addons):</para>
<programlisting><command>iptaccount [-f] -l net-loc</command></programlisting>
<para>Example:</para>
<programlisting>gateway:~# <emphasis role="bold">iptaccount -l net-loc</emphasis>
libxt_ACCOUNT_cl userspace accounting tool v1.3
Showing table: net-loc
Run #0 - 3 items found
IP: 172.20.1.105 SRC packets: 115 bytes: 131107 DST packets: 68 bytes: 20045
IP: 172.20.1.131 SRC packets: 47 bytes: 12729 DST packets: 38 bytes: 25304
IP: 172.20.1.145 SRC packets: 20747 bytes: 2779676 DST packets: 27050 bytes: 32286071
Finished.
gateway:~#</programlisting>
<para>For each local IP address with non-zero counters, the packet and
byte count for both incoming traffic (IP is DST) and outgoing traffic (IP
is SRC) are listed. The -f option causes the table to be flushed (reset
all counters to zero) after printing.</para>
<para>For a command synopsis:</para>
<programlisting><command>iptaccount --help</command></programlisting>
<para><filename>/sbin/shorewall</filename> also supports a <command>show
ipa</command> command (from my own gateway just after I flushed the
counters using <command>iptaccount -f -l</command>.:</para>
<programlisting>gateway:~# <command>shorewall show ipa</command>
Shorewall 4.4.18-Beta1 per-IP Accounting at gateway - Thu Feb 10 13:28:37 PST 2011
Showing table: loc-net
IP: 172.20.1.146 SRC packets: 9 bytes: 574 DST packets: 9 bytes: 770
Showing table: dmz-net
IP: 70.90.191.124 SRC packets: 243 bytes: 23726 DST packets: 248 bytes: 39036
IP: 70.90.191.125 SRC packets: 73 bytes: 10640 DST packets: 73 bytes: 4846
Showing table: fw-net
IP: 70.90.191.121 SRC packets: 0 bytes: 0 DST packets: 4 bytes: 243
IP: 70.90.191.122 SRC packets: 11 bytes: 1338 DST packets: 8 bytes: 5465
IP: 70.90.191.123 SRC packets: 42 bytes: 4604 DST packets: 44 bytes: 10662
gateway:~#
</programlisting>
</section>
</article>
|
