1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
|
<HTML
><HEAD
><TITLE
>flow-tools</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.73
"></HEAD
><BODY
CLASS="REFENTRY"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><H1
><A
NAME="AEN1"
><SPAN
CLASS="APPLICATION"
>flow-tools</SPAN
></A
></H1
><DIV
CLASS="REFNAMEDIV"
><A
NAME="AEN6"
></A
><H2
>Name</H2
><SPAN
CLASS="APPLICATION"
>flow-tools</SPAN
> -- Tool set for working with NetFlow data.</DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN10"
></A
><H2
>DESCRIPTION</H2
><P
>Flow-tools is library and a collection of programs used to collect,
send, process, and generate reports from NetFlow data. The tools
can be used together on a single server or distributed to multiple
servers for large deployments. The flow-toools library provides an
API for development of custom applications for NetFlow export versions
1,5,6 and the 14 currently defined version 8 subversions. A Perl and
Python interface have been contributed and are included in the distribution.</P
><P
>Flow data is collected and stored by default in host byte order, yet
the files are portable across big and little endian architectures.</P
><P
>Commands that utilize the network use a localip/remoteip/port designation
for communication. "localip" is the IP address the host will use as a
source for sending or bind to when receiving NetFlow PDU's (ie the destination
address of the exporter. Configuring the "localip" to 0 will force the kernel
to decide what IP address to use for sending and listen on all IP addresses
for receiving. "remoteip" is the destination IP address used for sending or
the expected address of the source when receiving. If the "remoteip" is
0 then the application will accept flows from any source address. The "port"
is the UDP port number used for sending or receiving. When using multicast
addresses the localip/remoteip/port is used to represent the source, group,
and port respectively.</P
><P
>Flows are exported from a router in a number of different configurable
versions. A flow is a collection of key fields and additional data.
The flow key is {srcaddr, dstaddr, input, output, srcport, dstport, prot,
ToS}. Flow-tools supports one export version per file.</P
><P
>Export versions 1, 5, 6, and 7 all maintain {nexthop, dPkts, dOctets,
First, Last, flags}, ie the next-hop IP address, number of packets, number
of octets (bytes), start time, end time, and flags such as the TCP header
bits. Version 5 adds the additional fields {src_as, dst_as, src_mask,
dst_mask}, ie source AS, destination AS, source network mask, and
destination network mask. Version 7 which is specific to the Catalyst
switches adds in addition to the version 5 fields {router_sc}, which is
the Router IP address which populates the flow cache shortcut in the
Supervisor. Version 6 which is not officially supported by Cisco adds
in addition to the version 5 fields {in_encaps, out_encaps, peer_nexthop},
ie the input and output interface encapsulation size, and the IP address
of the next hop within the peer. Version 1 exports do not contain a
sequence number and therefore should be avoided, although it is safe
to store the data as version 1 if the additional fields are not used.</P
><P
>Version 8 IOS NetFlow is a second level flow cache that reduces the
data exported from the router. There are currently 11 formats, all
of which provide {dFlows, dOctets, dPkts, First, Last} for the key
fields.</P
><P
><P
CLASS="LITERALLAYOUT"
> 8.1 - Source and Destination AS, Input and Output interface<br>
8.2 - Protocol and Port<br>
8.3 - Source Prefix and Input interface<br>
8.4 - Destination Prefix and Output interface<br>
8.5 - Source/Destination Prefix and Input/Output interface<br>
8.9 - 8.1 + ToS<br>
8.10 - 8.2 + ToS<br>
8.11 - 8.3 + ToS<br>
8.12 - 8.5 + ToS<br>
8.13 - 8.2 + ToS<br>
8.14 - 8.3 + ports + ToS</P
></P
><P
>Version 8 CatIOS NetFlow appears to be a less fine grained first level
flow cache.</P
><P
><P
CLASS="LITERALLAYOUT"
> 8.6 - Destination IP, ToS, Marked ToS, <br>
8.7 - Source/Destination IP, Input/Output interface, ToS, Marked ToS, <br>
8.8 - Source/Destination IP, Source/Destination Port,<br>
Input/Output interface, ToS, Marked ToS, </P
></P
><P
></P
><P
>The following programs are included in the flow-tools distribution.</P
><P
><B
CLASS="COMMAND"
>flow-capture</B
> - Collect, compress, store, and
manage disk space for exported flows from a router.</P
><P
><B
CLASS="COMMAND"
>flow-cat</B
> - Concatenate flow files. Typically flow files
will contain a small window of 5 or 15 minutes of exports. Flow-cat
can be used to append files for generating reports that span longer time
periods.</P
><P
><B
CLASS="COMMAND"
>flow-fanout</B
> - Replicate NetFlow datagrams to unicast or
multicast destinations. Flow-fanout is used to facilitate
multiple collectors attached to a single router.</P
><P
><B
CLASS="COMMAND"
>flow-report</B
> - Generate reports for NetFlow data sets.
Reports include source/destination IP pairs, source/destination AS,
and top talkers. Over 50 reports are currently supported.</P
><P
><B
CLASS="COMMAND"
>flow-tag</B
> - Tag flows based on IP address or AS #.
Flow-tag is used to group flows by customer network. The tags
can later be used with flow-fanout or flow-report
to generate customer based traffic reports.</P
><P
><B
CLASS="COMMAND"
>flow-filter</B
> - Filter flows based on any of the export
fields. Flow-filter is used in-line with other programs
to generate reports based on flows matching filter expressions.</P
><P
><B
CLASS="COMMAND"
>flow-import</B
> - Import data from ASCII or cflowd format.</P
><P
><B
CLASS="COMMAND"
>flow-export</B
> - Export data to ASCII or cflowd format.</P
><P
><B
CLASS="COMMAND"
>flow-send</B
> - Send data over the network using the NetFlow
protocol.</P
><P
><B
CLASS="COMMAND"
>flow-receive</B
> - Receive exports using the NetFlow protocol
without storing to disk like flow-capture.</P
><P
><B
CLASS="COMMAND"
>flow-gen</B
> - Generate test data.</P
><P
><B
CLASS="COMMAND"
>flow-dscan</B
> - Simple tool for detecting some types of network
scanning and Denial of Service attacks.</P
><P
><B
CLASS="COMMAND"
>flow-merge</B
> - Merge flow files in chronoligical order.</P
><P
><B
CLASS="COMMAND"
>flow-xlate</B
> - Perform translations on some flow fields.</P
><P
><B
CLASS="COMMAND"
>flow-expire</B
> - Expire flows using the same policy of
flow-capture.</P
><P
><B
CLASS="COMMAND"
>flow-header</B
> - Display meta information in flow file.</P
><P
><B
CLASS="COMMAND"
>flow-split</B
> - Split flow files into smaller files based on
size, time, or tags.</P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN59"
></A
><H2
>AUTHOR</H2
><P
>Mark Fullmer
<TT
CLASS="EMAIL"
><<A
HREF="mailto:maf@splintered.net"
>maf@splintered.net</A
>></TT
></P
><P
><B
CLASS="COMMAND"
>flow-merge</B
> by
Larry Lidz
<TT
CLASS="EMAIL"
><<A
HREF="mailto:ellidz@eridu.uchicago.edu"
>ellidz@eridu.uchicago.edu</A
>></TT
></P
><P
>Patches and other contribitions by a list too long to mention here.</P
><P
><B
CLASS="COMMAND"
>flow-tools</B
> is avalable at
<A
HREF="http://www.splintered.net/sw/flow-tools"
TARGET="_top"
>http://www.splintered.net/sw/flow-tools</A
>.</P
><P
>A mailing list is maintained at <TT
CLASS="EMAIL"
><<A
HREF="mailto:flow-tools@splintered.net"
>flow-tools@splintered.net</A
>></TT
></P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN78"
></A
><H2
>SEE ALSO</H2
><P
><SPAN
CLASS="APPLICATION"
>flow-capture</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-cat</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-dscan</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-expire</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-export</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-fanout</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-filter</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-nfilter</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-gen</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-header</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-import</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-merge</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-print</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-receive</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-report</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-send</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-split</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-stat</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-tag</SPAN
>(1)
<SPAN
CLASS="APPLICATION"
>flow-xlate</SPAN
>(1)</P
></DIV
></BODY
></HTML
>
|
