File: flow-capture.html.in

package info (click to toggle)
flow-tools 1%3A0.68-12.1
links: PTS
area: main
in suites: jessie, jessie-kfreebsd, wheezy
size: 5,136 kB
ctags: 5,259
sloc: ansic: 43,197; sh: 1,674; perl: 661; python: 629; yacc: 303; makefile: 208; lex: 49
file content (729 lines) | stat: -rw-r--r-- 12,085 bytes
parent folder | download | duplicates (4)
<HTML
><HEAD
><TITLE
>flow-capture</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.73
"></HEAD
><BODY
CLASS="REFENTRY"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><H1
><A
NAME="AEN1"
><SPAN
CLASS="APPLICATION"
>flow-capture</SPAN
></A
></H1
><DIV
CLASS="REFNAMEDIV"
><A
NAME="AEN6"
></A
><H2
>Name</H2
><SPAN
CLASS="APPLICATION"
>flow-capture</SPAN
>&nbsp;--&nbsp;Manage storage of flow file archives by expiring old data.</DIV
><DIV
CLASS="REFSYNOPSISDIV"
><A
NAME="AEN10"
></A
><H2
>Synopsis</H2
><P
><B
CLASS="COMMAND"
>flow-capture</B
>  [-hu] [-b<TT
CLASS="REPLACEABLE"
><I
> big|little</I
></TT
>] [-C<TT
CLASS="REPLACEABLE"
><I
> comment</I
></TT
>] [-c<TT
CLASS="REPLACEABLE"
><I
> flow_clients</I
></TT
>] [-d<TT
CLASS="REPLACEABLE"
><I
> debug_level</I
></TT
>] [-D<TT
CLASS="REPLACEABLE"
><I
> daemonize</I
></TT
>] [-e<TT
CLASS="REPLACEABLE"
><I
> expire_count</I
></TT
>] [-f<TT
CLASS="REPLACEABLE"
><I
> filter_fname</I
></TT
>] [-F<TT
CLASS="REPLACEABLE"
><I
> filter_definition</I
></TT
>] [-E<TT
CLASS="REPLACEABLE"
><I
> expire_size</I
></TT
>] [-n<TT
CLASS="REPLACEABLE"
><I
> rotations</I
></TT
>] [-N<TT
CLASS="REPLACEABLE"
><I
> nesting_level</I
></TT
>] [-p<TT
CLASS="REPLACEABLE"
><I
> pidfile</I
></TT
>] [-R<TT
CLASS="REPLACEABLE"
><I
> rotate_program</I
></TT
>] [-S<TT
CLASS="REPLACEABLE"
><I
> stat_interval</I
></TT
>] [-t<TT
CLASS="REPLACEABLE"
><I
> tag_fname</I
></TT
>] [-T<TT
CLASS="REPLACEABLE"
><I
> active_def</I
></TT
>|<TT
CLASS="REPLACEABLE"
><I
>active_def,active_def</I
></TT
>...] [-V<TT
CLASS="REPLACEABLE"
><I
> pdu_version</I
></TT
>] [-z<TT
CLASS="REPLACEABLE"
><I
> z_level</I
></TT
>] {-w<TT
CLASS="REPLACEABLE"
><I
> workdir</I
></TT
>} [-x<TT
CLASS="REPLACEABLE"
><I
> xlate_fname</I
></TT
>] [-X<TT
CLASS="REPLACEABLE"
><I
> xlate_definition</I
></TT
>] {<TT
CLASS="REPLACEABLE"
><I
>localip/remoteip/port</I
></TT
>}</P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN59"
></A
><H2
>DESCRIPTION</H2
><P
>The <B
CLASS="COMMAND"
>flow-capture</B
> utility will receive and store
NetFlow exports to disk.  The flow files are rotated <TT
CLASS="REPLACEABLE"
><I
>rotations</I
></TT
>times per day
and expiration of old flow files can be configured by number of files
or total space utilization.  Files are stored in <TT
CLASS="FILENAME"
>workdir</TT
>
and can optionally be stored in additional levels of directories.  Active
files created by <B
CLASS="COMMAND"
>flow-capture</B
> begin
with 'tmp'.  Files that are complete begin with 'ft'.</P
><P
>When the <TT
CLASS="REPLACEABLE"
><I
>remoteip</I
></TT
> is configured only flows
from that exporter will be processed, this is the most secure and recommended
configuration.  When the <TT
CLASS="REPLACEABLE"
><I
>localip</I
></TT
> is configured
<B
CLASS="COMMAND"
>flow-capture</B
> will only process flows
sent to the <TT
CLASS="REPLACEABLE"
><I
> localip</I
></TT
> IP address.  If
<TT
CLASS="REPLACEABLE"
><I
>remoteip</I
></TT
> is 0 (not configured) flows from any
source IP address are accepted.  Multiple non aggregated PDU versions may
be accepted at once to support Cisco's Catalyst 6500 NetFlow
implementation which exports from both the supervisor and MSFC with the
same IP address and same port but different export versions.  In this case
the exports will be stored in the format specified by <TT
CLASS="REPLACEABLE"
><I
>pdu_version</I
></TT
> or whichever export type is received first.</P
><P
>NetFlow exports are UDP and do not employ congestion control or a
retransmission mechanism.  If the server flow-capture is configured
on is too busy, or the network is congested or lossy NetFlow exports will
be lost.  An estimate of lost flows is recorded in the flow files, and
logged via syslog.  Most servers will provide a count of dropped packets
due to full socket buffers via the <B
CLASS="COMMAND"
>netstat</B
> utility.
For example <B
CLASS="COMMAND"
>netstat -s | grep full</B
> will provide a count
of UDP packets dropped due to full socket buffers.  If this is a persistent
occurrence either <B
CLASS="COMMAND"
>flow-capture</B
> will need a larger server
or the compression level should be decreased with -z.</P
><P
>A SIGHUP signal will cause <B
CLASS="COMMAND"
>flow-capture</B
> to close
the current file and create a new one.</P
><P
>A SIGQUIT or SIGTERM signal will cause <B
CLASS="COMMAND"
>flow-capture</B
> to close
the current file and exit.</P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN81"
></A
><H2
>OPTIONS</H2
><P
></P
><DIV
CLASS="VARIABLELIST"
><DL
><DT
>-b<TT
CLASS="REPLACEABLE"
><I
> big</I
></TT
>|<TT
CLASS="REPLACEABLE"
><I
>little</I
></TT
></DT
><DD
><P
>Byte order of output.</P
></DD
><DT
>-c<TT
CLASS="REPLACEABLE"
><I
> flow_clients</I
></TT
></DT
><DD
><P
>Enable <TT
CLASS="REPLACEABLE"
><I
>flow_clients</I
></TT
> TCP clients.  When libwrap
is available the client must be in a permit list for the service
flow-capture-client.</P
></DD
><DT
>-C<TT
CLASS="REPLACEABLE"
><I
> Comment</I
></TT
></DT
><DD
><P
>Add a comment.</P
></DD
><DT
>-d<TT
CLASS="REPLACEABLE"
><I
> debug_level</I
></TT
></DT
><DD
><P
>Enable debugging.</P
></DD
><DT
>-e<TT
CLASS="REPLACEABLE"
><I
> expire_count</I
></TT
></DT
><DD
><P
>Retain the maximum number of files so that the total file count is
less than <TT
CLASS="REPLACEABLE"
><I
>expire_count</I
></TT
>.  Defaults to
0 (do not expire).</P
></DD
><DT
>-E<TT
CLASS="REPLACEABLE"
><I
> expire_size</I
></TT
></DT
><DD
><P
>Retain the maximum number of files so that the total storage is less
than <TT
CLASS="REPLACEABLE"
><I
>expire_size</I
></TT
>.  The letters b,K,M,G can
be used as multipliers, ie 16 Megabytes is 16M.  Default to 0 (do not expire).</P
></DD
><DT
>-f<TT
CLASS="REPLACEABLE"
><I
> filter_fname</I
></TT
></DT
><DD
><P
>Filter list filename.  Defaults to <TT
CLASS="FILENAME"
>@localstatedir@/cfg/filter</TT
>.</P
></DD
><DT
>-F<TT
CLASS="REPLACEABLE"
><I
> filter_definition</I
></TT
></DT
><DD
><P
>Select the active definition.  Defaults to default.</P
></DD
><DT
>-h</DT
><DD
><P
>Display help.</P
></DD
><DT
>-n<TT
CLASS="REPLACEABLE"
><I
> rotations</I
></TT
></DT
><DD
><P
>Configure the number of times flow-capture will create a new file per day.
The default is 95, or every 15 minutes.</P
></DD
><DT
>-N<TT
CLASS="REPLACEABLE"
><I
> nesting_level</I
></TT
></DT
><DD
><P
>Configure the nesting level for storing flow files.  The default is 0.
   -3    YYYY/YYYY-MM/YYYY-MM-DD/flow-file
   -2    YYYY-MM/YYYY-MM-DD/flow-file
   -1    YYYY-MM-DD/flow-file
    0    flow-file
    1    YYYY/flow-file
    2    YYYY/YYYY-MM/flow-file
    3    YYYY/YYYY-MM/YYYY-MM-DD/flow-file</P
></DD
><DT
>-p<TT
CLASS="REPLACEABLE"
><I
> pidfile</I
></TT
></DT
><DD
><P
>Configure the process ID file.  Use - to disable pid file creation.</P
></DD
><DT
>-R<TT
CLASS="REPLACEABLE"
><I
> rotate_program</I
></TT
></DT
><DD
><P
>Execute <TT
CLASS="REPLACEABLE"
><I
>rotate_program</I
></TT
> with the first argument
as the flow file name after rotating it.</P
></DD
><DT
>-S<TT
CLASS="REPLACEABLE"
><I
> stat_interval</I
></TT
></DT
><DD
><P
>When configured <B
CLASS="COMMAND"
>flow-capture</B
> will log a timestamped
message every <TT
CLASS="REPLACEABLE"
><I
>stat_interval</I
></TT
> minutes
indicating counters such as the number of flows received, packets processed,
and lost flows.</P
></DD
><DT
>-t<TT
CLASS="REPLACEABLE"
><I
> tag_fname</I
></TT
></DT
><DD
><P
>Load tags from <TT
CLASS="FILENAME"
>tag_name</TT
></P
></DD
><DT
>-T<TT
CLASS="REPLACEABLE"
><I
> active_def</I
></TT
>|<TT
CLASS="REPLACEABLE"
><I
>active_def,active_def...</I
></TT
></DT
><DD
><P
>Use <TT
CLASS="REPLACEABLE"
><I
>active_def</I
></TT
> as the active tag definition(s).</P
></DD
><DT
>-u</DT
><DD
><P
>Preserve inherited umask.  By default the umask will be set to 0022.</P
></DD
><DT
>-V<TT
CLASS="REPLACEABLE"
><I
> pdu_version</I
></TT
></DT
><DD
><P
>Use <TT
CLASS="REPLACEABLE"
><I
>pdu_version</I
></TT
> format output.
<P
CLASS="LITERALLAYOUT"
>&nbsp;&nbsp;&nbsp;&nbsp;1&nbsp;&nbsp;&nbsp;&nbsp;NetFlow&nbsp;version&nbsp;1&nbsp;(No&nbsp;sequence&nbsp;numbers,&nbsp;AS,&nbsp;or&nbsp;mask)<br>
&nbsp;&nbsp;&nbsp;&nbsp;5&nbsp;&nbsp;&nbsp;&nbsp;NetFlow&nbsp;version&nbsp;5<br>
&nbsp;&nbsp;&nbsp;&nbsp;6&nbsp;&nbsp;&nbsp;&nbsp;NetFlow&nbsp;version&nbsp;6&nbsp;(5+&nbsp;Encapsulation&nbsp;size)<br>
&nbsp;&nbsp;&nbsp;&nbsp;7&nbsp;&nbsp;&nbsp;&nbsp;NetFlow&nbsp;version&nbsp;7&nbsp;(Catalyst&nbsp;switches)<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.1&nbsp;&nbsp;NetFlow&nbsp;AS&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.2&nbsp;&nbsp;NetFlow&nbsp;Proto&nbsp;Port&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.3&nbsp;&nbsp;NetFlow&nbsp;Source&nbsp;Prefix&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.4&nbsp;&nbsp;NetFlow&nbsp;Destination&nbsp;Prefix&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.5&nbsp;&nbsp;NetFlow&nbsp;Prefix&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.6&nbsp;&nbsp;NetFlow&nbsp;Destination&nbsp;(Catalyst&nbsp;switches)<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.7&nbsp;&nbsp;NetFlow&nbsp;Source&nbsp;Destination&nbsp;(Catalyst&nbsp;switches)<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.8&nbsp;&nbsp;NetFlow&nbsp;Full&nbsp;Flow&nbsp;(Catalyst&nbsp;switches)<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.9&nbsp;&nbsp;NetFlow&nbsp;ToS&nbsp;AS&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.10&nbsp;NetFlow&nbsp;ToS&nbsp;Proto&nbsp;Port&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.11&nbsp;NetFlow&nbsp;ToS&nbsp;Source&nbsp;Prefix&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.12&nbsp;NetFlow&nbsp;ToS&nbsp;Destination&nbsp;Prefix&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.13&nbsp;NetFlow&nbsp;ToS&nbsp;Prefix&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;8.14&nbsp;NetFlow&nbsp;ToS&nbsp;Prefix&nbsp;Port&nbsp;Aggregation<br>
&nbsp;&nbsp;&nbsp;&nbsp;1005&nbsp;Flow-Tools&nbsp;tagged&nbsp;version&nbsp;5</P
></P
></DD
><DT
>-w<TT
CLASS="REPLACEABLE"
><I
> workdir</I
></TT
></DT
><DD
><P
>Work in <TT
CLASS="FILENAME"
>workdir</TT
>.</P
></DD
><DT
>-x<TT
CLASS="REPLACEABLE"
><I
> xlate_fname</I
></TT
></DT
><DD
><P
>Translation config file name.  Defaults to <TT
CLASS="FILENAME"
>@localstatedir@/cfg/xlate.c
fg</TT
></P
></DD
><DT
>-X<TT
CLASS="REPLACEABLE"
><I
> xlate_definition</I
></TT
></DT
><DD
><P
>Translation definition.  Defaults to default.</P
></DD
><DT
>-z<TT
CLASS="REPLACEABLE"
><I
> z_level</I
></TT
></DT
><DD
><P
>Configure compression level to <TT
CLASS="REPLACEABLE"
><I
> z_level</I
></TT
>.  0 is
disabled (no compression), 9 is highest compression.</P
></DD
></DL
></DIV
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN208"
></A
><H2
>EXAMPLES</H2
><DIV
CLASS="INFORMALEXAMPLE"
><A
NAME="AEN210"
></A
><P
></P
><P
>Receive flows from the exporter at 10.0.0.1 port 9800.  Maintain 5 Gigabytes
of flow files in /flows/krc4.  Mask the source and destination IP addresses
contained in the flow exports with 255.255.248.0.</P
><P
>  <B
CLASS="COMMAND"
>flow-capture -w /flows/krc4 -m 255.255.248.0 -E5G 0/10.0.0.1/9800</B
></P
><P
></P
></DIV
><DIV
CLASS="INFORMALEXAMPLE"
><A
NAME="AEN214"
></A
><P
></P
><P
>Receive flows from any exporter on port 9800.  Do not perform any flow
file space management.  Store the exports in /flows/krc4.  Emit a stat
log message every 5 minutes.</P
><P
>  <B
CLASS="COMMAND"
>flow-capture -w /flows/krc4 0/0/9800 -S5</B
></P
><P
></P
></DIV
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN218"
></A
><H2
>BUGS</H2
><P
>Empty directories are not removed.</P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN221"
></A
><H2
>FILES</H2
><P
>  Configuration files:
    Tag - <TT
CLASS="FILENAME"
>@localstatedir@/cfg/tag.cfg</TT
>.
    Filter - <TT
CLASS="FILENAME"
>@localstatedir@/cfg/filter.cfg</TT
>.
    Xlate - <TT
CLASS="FILENAME"
>@localstatedir@/cfg/xlate.cfg</TT
>.</P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN227"
></A
><H2
>AUTHOR</H2
><P
>Mark Fullmer
<TT
CLASS="EMAIL"
>&#60;<A
HREF="mailto:maf@splintered.net"
>maf@splintered.net</A
>&#62;</TT
></P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN234"
></A
><H2
>SEE ALSO</H2
><P
><SPAN
CLASS="APPLICATION"
>flow-tools</SPAN
>(1)</P
></DIV
></BODY
></HTML
>