File: CampusIO.README

package info (click to toggle)
flowscan 1.006-13.2
  • links: PTS
  • area: main
  • in suites: buster, jessie, jessie-kfreebsd, stretch, wheezy
  • size: 732 kB
  • ctags: 185
  • sloc: sh: 1,670; perl: 1,518; makefile: 148
file content (384 lines) | stat: -rw-r--r-- 15,387 bytes parent folder | download | duplicates (5)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
NAME
    CampusIO - a FlowScan module for reporting on campus traffic I/O

SYNOPSIS
       $ flowscan CampusIO

    or in flowscan.cf:

       ReportClasses CampusIO

DESCRIPTION
    CampusIO is a general flowscan report for reporting on flows of
    traffic in and out of a site or campus. It does this by
    processing flows reported by one or more routers at the network
    border. The site or campus may be an Autonomous System (AS), as
    is often the case for large universities, but this is not
    necessary. CampusIO can be used by smaller institutions and
    other enterprises as well.

    `flowscan' will run the CampusIO report if you configure this in
    your flowscan.cf:

       ReportClasses CampusIO

CONFIGURATION
    CampusIO's configuration file is CampusIO.cf. This configuration
    file is located in the directory in which the flowscan script
    resides.

    The CampusIO configuration directives include:

    NextHops
        This directive is suggested if `OutputIfIndexes' is not
        defined. Defining `NextHops' causes `flowscan' to identify
        outbound flows by their nexthop value. `NextHops' is a
        comma-seperated list of IP addresses or resolvable
        hostnames, e.g.:

           # NextHops
           NextHops gateway.provider.net, gateway.other.net

        If neither `NextHops' nor `OutputIfIdexes' is defined,
        `CampusIO' will use the flows' destination addresses to
        determine whether or not they are outbound. This is a less
        reliable and more CPU intensive method than `NextHops' or
        `OutputIfIdexes'.

    OutputIfIndexes
        This directive is suggested if `NextHops' is not defined.
        Defining `OutputIfIndexes' causes `flowscan' to identify
        outbound flows by their output interface value.
        `OutputIfIndexes' is a comma-seperated list of ifIndexes as
        determined using SNMP, e.g.:

           $ snmpwalk router.our.domain public interfaces.ifTable.ifEntry.ifDescr

        or by looking at the raw flows from Cflowd to determine the
        `$output_if'. e.g.:

           # OutputIfIndexes
           OutputIfIndexes 1, 2, 3

        If neither `NextHops' nor `OutputIfIdexes' is defined,
        `CampusIO' will use the flows' destination addresses to
        determine whether or not they are outbound. This is a less
        reliable and more CPU intensive method than `NextHops' or
        `OutputIfIdexes'.

    LocalSubnetFiles
        This directive is required. It is a a comma-seperated list
        of files containing the definitions of "local" subnets.
        E.g.:

           # LocalSubnetFiles local_nets.boulder
           LocalSubnetFiles bin/local_nets.boulder

    OutputDir
        This directive is required. It is the directory in which RRD
        files will be written. E.g.:

           # OutputDir /var/local/flows/graphs
           OutputDir graphs

    LocalNextHops
        This is an "advanced" option which is only required if you
        are exporting and collecting flows from multiple routers to
        the same FlowScan. It is a comma-seperated list of IP
        addresses or resolvable hostnames.

        Specify all the local routers for which you have configured
        cflowd to collecting flows on this FlowScan host. This will
        ensure that the same traffic isn't counted twice by ignoring
        flows destined for these next-hops, which otherwise might
        look as if they're inbound flows. FlowScan will only count
        flows that represent traffic forwarded outside this set of
        local routers.

        E.g.:

           # LocalNextHops other-router.our.domain

    TCPServices
        This directive is optional, but is required if you wish to
        produce the CampusIO service graphs. It is a comma-seperated
        list of TCP services by name or number. E.g., it is
        recommended that it contain at least the services shown
        here:

           # TCPServices ftp-data, ftp, smtp, nntp, www, 7070, 554
           TCPServices ftp-data, ftp, smtp, nntp, www, 7070, 554

    UDPServices
        This directive is optional. It is a comma-seperated list of
        UDP services by name or number. E.g.:

           # UDPServices domain, snmp, snmp-trap

    Protocols
        This directive is optional, but is required if you wish to
        produce the CampusIO protocol graphs. It is a comma-
        seperated list of IP protocols by name. E.g.:

           # Protocols icmp, tcp, udp
           Protocols icmp, tcp, udp

    ASPairs
        This directive is optional, but is required if you wish to
        build any custom AS graphs. It is a list of source and
        destination AS pairs. E.g.:

           # source_AS:destination_AS, e.g.:
           # ASPairs 0:0
           ASPairs 0:0

        Note that the effect of setting ASPairs will be different
        based on whether you specified "peer-as" or "origin-as" when
        you configured your Cisco. This option was intended to be
        used when "peer-as" is configured.

        See the `BGPDumpFile' directive for other AS-related
        features.

    Verbose
        This directive is optional. If non-zero, it makes `flowscan'
        more verbose with respect to messages and warnings.
        Currently the values `1' and `2' are understood, the higher
        value causing more messages to be produced. E.g.:

           # Verbose (OPTIONAL, non-zero = true)
           Verbose 1

    NapsterSubnetFiles
        This directive is optional, but is required if you wish to
        produce the CampusIO service graphs. It is a comma-seperated
        list of files containing the definitions of "Napster"
        subnets. E.g.:

           # NapsterSubnetFiles (OPTIONAL)
           NapsterSubnetFiles bin/Napster_subnets.boulder

    NapsterSeconds
        This directive is optional. It is the number of seconds
        after which a given campus host has communicated with a host
        within the "Napster" subnet(s) will no longer be considered
        to be using the Napster application. E.g. Half an hour:

           # NapsterSeconds (OPTIONAL)
           NapsterSeconds 1800

    NapsterPorts
        This directive is optional. It a comma-seperated list of
        default TCP ports used by Napster. These will be used to
        determine the confidence level of whether or not it's really
        Napster traffic. If confidence is low, it will be reported
        as "NapsterMaybe" rather than "NapUser" traffic. E.g.,
        reasonable values are:

           # NapsterPorts (OPTIONAL)
           NapsterPorts 8875, 4444, 5555, 6666, 6697, 6688, 6699, 7777, 8888

    TopN
        This directive is optional. It's use requires the
        `HTML::Table' perl module. `TopN' is the number of entries
        to show in the tables that will be generated in HTML top
        reports. E.g.:

           # TopN (OPTIONAL)
           TopN 10

        If you'd prefer to see hostnames rather than IP addresses in
        your top reports, use the ip2hostname script. E.g.:

           $ ip2hostname -I *.*.*.*_*.html

    ReportPrefixFormat
        This directive is optional. It is used to specify the file
        name prefix for the HTML or text reports such as the
        "originAS", "pathAS", and "Top Talkers" reports. You should
        use strftime(3) format specifiers in the value, and it may
        also specify sub-directories. If not set, the prefix
        defaults to the null string, which means that, every five
        minutes, subsequent reports will overwrite the previous.
        E.g.:

           # Preserve one day of HTML reports using the time of day as the dir name:
           ReportPrefixFormat html/CampusIO/%H:%M/

        or:

           # Preserve one month by using the day of month in the dir name (like sar(1)):
           ReportPrefixFormat html/CampusIO/%d/%H:%M_

    BGPDumpFile
        This directive is optional and is experimental. In
        combination with `TopN' and `ASNFile' it causes FlowScan to
        produce "Top ASN" reports which show the "top" Autonomous
        Systems with which your site exchanges traffic.

        `BGPDumpFile' requires the `ParseBGPDump' perl module by
        Sean McCreary, which is supplied with CAIDA's CoralReef
        Package:

           http://www.caida.org/tools/measurement/coralreef/status.xml

        Unfortunately, CoralReef is governed by a different license
        than FlowScan itself. The Copyright file says this:

           Permission to use, copy, modify and distribute any part of this
           CoralReef software package for educational, research and non-profit
           purposes, without fee, and without a written agreement is hereby
           granted, provided that the above copyright notice, this paragraph
           and the following paragraphs appear in all copies.
           [...]

           The CoralReef software package is developed by the CoralReef
           development team at the University of California, San Diego under
           the Cooperative Association for Internet Data Analysis (CAIDA)
           Program. Support for this effort is provided by the CAIDA grant
           NCR-9711092, and by CAIDA members.

        After fetching the `coral' release from:

           http://www.caida.org/tools/measurement/coralreef/dists/coral-3.4.1-public.tar.gz

        install `ParseBGPDump.pm' in FlowScan's perl include path,
        such as in the `bin' sub-directory:

           $ cd /tmp
           $ gunzip -c coral-3.4.1-public.tar.gz |tar x coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm
           $ mv coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm $PREFIX/bin/ParseBGPDump.pm

        Also you must specify `TopN' to be greater than zero, e.g.
        10, and the `HTML::Table' perl module is required if you do
        so.

        The `BGPDumpFile' value is the name of a file containing the
        output of `show ip bgp' from a Cisco router, ideally from
        the router that is exporting flows. If this option is used,
        and the specified file exists, it will cause the "originAS"
        and "pathAS" reports to be generated. E.g.:

           TopN 10
           BGPDumpFile etc/router.our.domain.bgp

        One way to create the file itself, is to set up rsh access
        to your Cisco, e.g.:

           ip rcmd rsh-enable
           ip rcmd remote-host username 10.10.42.69 username

        Then do something like this:

           $ cd $PREFIX
           $ mkdir etc
           $ echo show ip bgp >etc/router.our.domain.bgp # required by ParseBGPDump.pm
           $ time rsh router.our.domain "show ip bgp" >>etc/router.our.domain.bgp
              65.65s real     0.01s user     0.05s system
           $ wc -l /tmp/router.our.domain.bgp
            197883 /tmp/router.our.domain.bgp

        Once `flowscan' is up and running with `BGPDumpFile'
        configured, it will reload that file if its timestamp
        indicates that it has been modified. This allows you to
        "freshen" the image of the routing table without having to
        restart `flowscan' itself.

        Using the `BGPDumpFile' option causes `FlowScan' to use much
        more memory than usual. This memory is used to store a
        `Net::Patricia' trie containing a node for every prefix in
        the BGP routing table. For instance, on my system it caused
        the `FlowScan' process to grow to over 50MB, compared to
        less than 10MB without `BGPDumpFile' configured.

    ASNFile
        This directive is optional and is only useful in conjunction
        with `BGPDumpFile'. If specified, this directive will cause
        the AS names rather than just their numbers to appear in the
        Top ASN HTML reports. Its value should be the path to a file
        having the format of the file downloaded from this URL:

           ftp://ftp.arin.net/netinfo/asn.txt

        E.g.:

           TopN 10
           BGPDumpFile etc/router.our.domain.bgp
           ASNfile etc/asn.txt

        Once `flowscan' is up and running with `ASNFile' configured,
        it will reload the file if its timestamp indicates that it
        has been modified.

METHODS
    This module provides no public methods. It is a report module
    meant only for use by `flowscan'. Please see the `FlowScan'
    module documentation for information on how to write a FlowScan
    report module.

SEE ALSO
    perl(1), FlowScan, SubNetIO, flowscan(1), Net::Patricia.

BUGS
    When using the `BGPDumpFile' directive, `ParseBGPDump' issues a
    bunch of warnings which can safely be ignored:

       Failed to parse table version from: show ip bgp
        at (eval 4) line 1
       Failed to parse router IP address from: show ip bgp
        at (eval 4) line 1
       Nexthop not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Metric not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Local Preference not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Weight not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Origin code not found:    Network          Next Hop            Metric LocPrf Weight Path
       $ at (eval 4) line 1
       Possible truncated file, end-of-dump prompt not found
        at (eval 4) line 1

    I'm not keen on patching `ParseBGPDump' to fix this since its
    license isn't compatible with the GPL. We probably just need to
    hack up a complete replacement for `ParseBGPDump'.

    When using the `BGPDumpFile' directive, `ParseBGPDump' sometimes
    mistakes the `Weight' for the first ASN in the path. This has
    the totally undesirable effect of producing a "Top Path ASNs"
    report that erroneously reports the weight as one of the Top
    ASNs! I assume this is an indication of the difficulty of
    parsing the output of `show ip bgp', which apparently was meant
    for human consumption.

    When using the `ASPairs' directive, CampusIO will create RRD
    files that have a `:' character in the file name. While RRDTool
    is able to create RRD files with those names, it is not able to
    graph from them. To work around this problem, create symbolic
    links in your `OutputDir' before attempting to graph from these
    files. For example:

       $ ln -s 0:n.rrd Us2Them.rrd
       $ ln -s n:0.rrd Them2Us.rrd

AUTHOR
    Dave Plonka <plonka@doit.wisc.edu>

    Copyright (C) 1998-2001 Dave Plonka. This program is free
    software; you can redistribute it and/or modify it under the
    terms of the GNU General Public License as published by the Free
    Software Foundation; either version 2 of the License, or (at
    your option) any later version.

VERSION
    The version number is the module file RCS revision number
    ($Revision: 1.63 $) with the minor number printed right
    justified with leading zeroes to 3 decimal places. For instance,
    RCS revision 1.1 would yield a package version number of 1.001.

    This is so that revision 1.10 (which is version 1.010), for
    example, will test greater than revision 1.2 (which is version
    1.002) when you want to require a minimum version of this
    module.