File: CampusIO.html

package info (click to toggle)
flowscan 1.006-13.2
  • links: PTS
  • area: main
  • in suites: buster, jessie, jessie-kfreebsd, stretch, wheezy
  • size: 732 kB
  • ctags: 185
  • sloc: sh: 1,670; perl: 1,518; makefile: 148
file content (695 lines) | stat: -rw-r--r-- 17,855 bytes parent folder | download | duplicates (5)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
    <HTML> 
	<HEAD> 
	    <TITLE>CampusIO - a FlowScan module for reporting on campus traffic I/O

</TITLE> 
	</HEAD>

	<BODY>

<!-- INDEX BEGIN -->

<UL>

	<LI><A HREF="#NAME">NAME</A>
	<LI><A HREF="#SYNOPSIS">SYNOPSIS</A>
	<LI><A HREF="#DESCRIPTION">DESCRIPTION</A>
	<LI><A HREF="#CONFIGURATION">CONFIGURATION</A>
	<LI><A HREF="#METHODS">METHODS</A>
	<LI><A HREF="#SEE_ALSO">SEE ALSO</A>
	<LI><A HREF="#BUGS">BUGS</A>
	<LI><A HREF="#AUTHOR">AUTHOR</A>
	<LI><A HREF="#VERSION">VERSION</A>
</UL>
<!-- INDEX END -->

<HR>
<P>
<H1><A NAME="NAME">NAME

</A></H1>
CampusIO - a FlowScan module for reporting on campus traffic I/O


<P>

<P>
<HR>
<H1><A NAME="SYNOPSIS">SYNOPSIS

</A></H1>
<PRE>   $ flowscan CampusIO
</PRE>

<P>

or in <EM>flowscan.cf</EM>:


<P>

<PRE>   ReportClasses CampusIO
</PRE>

<P>

<P>
<HR>
<H1><A NAME="DESCRIPTION">DESCRIPTION

</A></H1>
CampusIO is a general flowscan report for reporting on flows of traffic in
and out of a site or campus. It does this by processing flows reported by
one or more routers at the network border. The site or campus may be an
Autonomous System (AS), as is often the case for large universities, but
this is not necessary. CampusIO can be used by smaller institutions and
other enterprises as well.


<P>

<CODE>flowscan</CODE> will run the CampusIO report if you configure this in your <EM>flowscan.cf</EM>:


<P>

<PRE>   ReportClasses CampusIO
</PRE>

<P>

<P>
<HR>
<H1><A NAME="CONFIGURATION">CONFIGURATION

</A></H1>
CampusIO's configuration file is <EM>CampusIO.cf</EM>. This configuration file is located in the directory in which the <EM>flowscan</EM> script resides.


<P>

The CampusIO configuration directives include:


<P>

<DL>
<DT><STRONG><A NAME="item_NextHops">NextHops

</A></STRONG><DD>
This directive is suggested if <A HREF="#item_OutputIfIndexes">OutputIfIndexes</A> is not defined. Defining <A HREF="#item_NextHops">NextHops</A> causes <CODE>flowscan</CODE> to identify outbound flows by their nexthop value.  <A HREF="#item_NextHops">NextHops</A> is a comma-seperated list of IP addresses or resolvable hostnames, e.g.:


<P>

<PRE>   # NextHops
   NextHops gateway.provider.net, gateway.other.net
</PRE>

<P>

If neither <A HREF="#item_NextHops">NextHops</A> nor <CODE>OutputIfIdexes</CODE> is defined, <CODE>CampusIO</CODE>
will use the flows' destination addresses to determine whether or not they
are outbound. This is a less reliable and more CPU intensive method than <A HREF="#item_NextHops">NextHops</A> or <CODE>OutputIfIdexes</CODE>.


<P>

<DT><STRONG><A NAME="item_OutputIfIndexes">OutputIfIndexes

</A></STRONG><DD>
This directive is suggested if <A HREF="#item_NextHops">NextHops</A> is not defined. Defining
<A HREF="#item_OutputIfIndexes">OutputIfIndexes</A> causes <CODE>flowscan</CODE> to identify outbound flows by their output interface value.  <A HREF="#item_OutputIfIndexes">OutputIfIndexes</A> is a comma-seperated list of ifIndexes as determined using SNMP, e.g.:


<P>

<PRE>   $ snmpwalk router.our.domain public interfaces.ifTable.ifEntry.ifDescr
</PRE>

<P>

or by looking at the raw flows from Cflowd to determine the <CODE>$output_if</CODE>. e.g.:


<P>

<PRE>   # OutputIfIndexes
   OutputIfIndexes 1, 2, 3
</PRE>

<P>

If neither <A HREF="#item_NextHops">NextHops</A> nor <CODE>OutputIfIdexes</CODE> is defined, <CODE>CampusIO</CODE>
will use the flows' destination addresses to determine whether or not they
are outbound. This is a less reliable and more CPU intensive method than <A HREF="#item_NextHops">NextHops</A> or <CODE>OutputIfIdexes</CODE>.


<P>

<DT><STRONG><A NAME="item_LocalSubnetFiles">LocalSubnetFiles

</A></STRONG><DD>
This directive is required. It is a a comma-seperated list of files
containing the definitions of ``local'' subnets. E.g.:


<P>

<PRE>   # LocalSubnetFiles local_nets.boulder
   LocalSubnetFiles bin/local_nets.boulder
</PRE>

<P>

<DT><STRONG><A NAME="item_OutputDir">OutputDir

</A></STRONG><DD>
This directive is required. It is the directory in which RRD files will be
written. E.g.:


<P>

<PRE>   # OutputDir /var/local/flows/graphs
   OutputDir graphs
</PRE>

<P>

<DT><STRONG><A NAME="item_LocalNextHops">LocalNextHops

</A></STRONG><DD>
This is an ``advanced'' option which is only required if you are exporting
and collecting flows from multiple routers to the same FlowScan. It is a
comma-seperated list of IP addresses or resolvable hostnames.


<P>

Specify all the local routers for which you have configured cflowd to
collecting flows on this FlowScan host. This will ensure that the same
traffic isn't counted twice by ignoring flows destined for these next-hops,
which otherwise might look as if they're inbound flows. FlowScan will only
count flows that represent traffic forwarded outside this set of local
routers.


<P>

E.g.:


<P>

<PRE>   # LocalNextHops other-router.our.domain
</PRE>

<P>

<DT><STRONG><A NAME="item_TCPServices">TCPServices

</A></STRONG><DD>
This directive is optional, but is required if you wish to produce the
CampusIO service graphs. It is a comma-seperated list of TCP services by
name or number. E.g., it is recommended that it contain at least the
services shown here:


<P>

<PRE>   # TCPServices ftp-data, ftp, smtp, nntp, www, 7070, 554
   TCPServices ftp-data, ftp, smtp, nntp, www, 7070, 554
</PRE>

<P>

<DT><STRONG><A NAME="item_UDPServices">UDPServices

</A></STRONG><DD>
This directive is optional. It is a comma-seperated list of UDP services by
name or number. E.g.:


<P>

<PRE>   # UDPServices domain, snmp, snmp-trap
</PRE>

<P>

<DT><STRONG><A NAME="item_Protocols">Protocols

</A></STRONG><DD>
This directive is optional, but is required if you wish to produce the
CampusIO protocol graphs. It is a comma-seperated list of IP protocols by
name. E.g.:


<P>

<PRE>   # Protocols icmp, tcp, udp
   Protocols icmp, tcp, udp
</PRE>

<P>

<DT><STRONG><A NAME="item_ASPairs">ASPairs

</A></STRONG><DD>
This directive is optional, but is required if you wish to build any custom
AS graphs. It is a list of source and destination AS pairs. E.g.:


<P>

<PRE>   # source_AS:destination_AS, e.g.:
   # ASPairs 0:0
   ASPairs 0:0
</PRE>

<P>

Note that the effect of setting ASPairs will be different based on whether
you specified ``peer-as'' or ``origin-as'' when you configured your Cisco.
This option was intended to be used when ``peer-as'' is configured.


<P>

See the <A HREF="#item_BGPDumpFile">BGPDumpFile</A> directive for other AS-related features.


<P>

<DT><STRONG><A NAME="item_Verbose">Verbose

</A></STRONG><DD>
This directive is optional. If non-zero, it makes <CODE>flowscan</CODE> more verbose with respect to messages and warnings. Currently the values <CODE>1</CODE> and <CODE>2</CODE> are understood, the higher value causing more messages to be produced.
E.g.:


<P>

<PRE>   # Verbose (OPTIONAL, non-zero = true)
   Verbose 1
</PRE>

<P>

<DT><STRONG><A NAME="item_NapsterSubnetFiles">NapsterSubnetFiles

</A></STRONG><DD>
This directive is optional, but is required if you wish to produce the
CampusIO service graphs. It is a comma-seperated list of files containing
the definitions of ``Napster'' subnets. E.g.:


<P>

<PRE>   # NapsterSubnetFiles (OPTIONAL)
   NapsterSubnetFiles bin/Napster_subnets.boulder
</PRE>

<P>

<DT><STRONG><A NAME="item_NapsterSeconds">NapsterSeconds

</A></STRONG><DD>
This directive is optional. It is the number of seconds after which a given
campus host has communicated with a host within the ``Napster''
<CODE>subnet(s)</CODE> will no longer be considered to be using the Napster
application. E.g. Half an hour:


<P>

<PRE>   # NapsterSeconds (OPTIONAL)
   NapsterSeconds 1800
</PRE>

<P>

<DT><STRONG><A NAME="item_NapsterPorts">NapsterPorts

</A></STRONG><DD>
This directive is optional. It a comma-seperated list of default TCP ports
used by Napster. These will be used to determine the confidence level of
whether or not it's really Napster traffic. If confidence is low, it will
be reported as ``NapsterMaybe'' rather than ``NapUser'' traffic. E.g.,
reasonable values are:


<P>

<PRE>   # NapsterPorts (OPTIONAL)
   NapsterPorts 8875, 4444, 5555, 6666, 6697, 6688, 6699, 7777, 8888
</PRE>

<P>

<DT><STRONG><A NAME="item_TopN">TopN

</A></STRONG><DD>
This directive is optional. It's use requires the <CODE>HTML::Table</CODE> perl module.  <A HREF="#item_TopN">TopN</A> is the number of entries to show in the tables that will be generated in
HTML top reports. E.g.:


<P>

<PRE>   # TopN (OPTIONAL)
   TopN 10
</PRE>

<P>

If you'd prefer to see hostnames rather than IP addresses in your top
reports, use the <EM>ip2hostname</EM> script. E.g.:


<P>

<PRE>   $ ip2hostname -I *.*.*.*_*.html
</PRE>

<P>

<DT><STRONG><A NAME="item_ReportPrefixFormat">ReportPrefixFormat

</A></STRONG><DD>
This directive is optional. It is used to specify the file name prefix for
the HTML or text reports such as the ``originAS'', ``pathAS'', and ``Top
Talkers'' reports. You should use <CODE>strftime(3)</CODE> format
specifiers in the value, and it may also specify sub-directories. If not
set, the prefix defaults to the null string, which means that, every five
minutes, subsequent reports will overwrite the previous. E.g.:


<P>

<PRE>   # Preserve one day of HTML reports using the time of day as the dir name:
   ReportPrefixFormat html/CampusIO/%H:%M/
</PRE>

<P>

or:


<P>

<PRE>   # Preserve one month by using the day of month in the dir name (like sar(1)):
   ReportPrefixFormat html/CampusIO/%d/%H:%M_
</PRE>

<P>

<DT><STRONG><A NAME="item_BGPDumpFile">BGPDumpFile

</A></STRONG><DD>
This directive is optional and is <STRONG>experimental</STRONG>. In combination with
<A HREF="#item_TopN">TopN</A> and <A HREF="#item_ASNFile">ASNFile</A> it causes FlowScan to produce ``Top ASN'' reports which show the ``top''
Autonomous Systems with which your site exchanges traffic.


<P>

<A HREF="#item_BGPDumpFile">BGPDumpFile</A> requires the <CODE>ParseBGPDump</CODE> perl module by Sean McCreary, which is supplied with CAIDA's CoralReef
Package:


<P>

<PRE>   <A HREF="http://www.caida.org/tools/measurement/coralreef/status.xml">http://www.caida.org/tools/measurement/coralreef/status.xml</A>
</PRE>

<P>

Unfortunately, CoralReef is governed by a different license than FlowScan
itself. The <EM>Copyright</EM> file says this:


<P>

<PRE>   Permission to use, copy, modify and distribute any part of this
   CoralReef software package for educational, research and non-profit
   purposes, without fee, and without a written agreement is hereby
   granted, provided that the above copyright notice, this paragraph
   and the following paragraphs appear in all copies.
   [...]
</PRE>

<P>

<PRE>   The CoralReef software package is developed by the CoralReef
   development team at the University of California, San Diego under
   the Cooperative Association for Internet Data Analysis (CAIDA)
   Program. Support for this effort is provided by the CAIDA grant
   NCR-9711092, and by CAIDA members.
</PRE>

<P>

After fetching the <CODE>coral</CODE> release from:


<P>

<PRE>   <A HREF="http://www.caida.org/tools/measurement/coralreef/dists/coral-3.4.1-public.tar.gz">http://www.caida.org/tools/measurement/coralreef/dists/coral-3.4.1-public.tar.gz</A>
</PRE>

<P>

install <CODE>ParseBGPDump.pm</CODE> in FlowScan's perl include path, such as in the <CODE>bin</CODE> sub-directory:


<P>

<PRE>   $ cd /tmp
   $ gunzip -c coral-3.4.1-public.tar.gz |tar x coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm
   $ mv coral-3.4.1-public/./libsrc/misc-perl/ParseBGPDump.pm $PREFIX/bin/ParseBGPDump.pm
</PRE>

<P>

Also you must specify <A HREF="#item_TopN">TopN</A> to be greater than zero, e.g. 10, and the
<CODE>HTML::Table</CODE> perl module is required if you do so.


<P>

The <A HREF="#item_BGPDumpFile">BGPDumpFile</A> value is the name of a file containing the output of
<CODE>show ip bgp</CODE> from a Cisco router, ideally from the router that is exporting flows. If
this option is used, and the specified file exists, it will cause the
``originAS'' and ``pathAS'' reports to be generated. E.g.:


<P>

<PRE>   TopN 10
   BGPDumpFile etc/router.our.domain.bgp
</PRE>

<P>

One way to create the file itself, is to set up rsh access to your Cisco,
e.g.:


<P>

<PRE>   ip rcmd rsh-enable
   ip rcmd remote-host username 10.10.42.69 username
</PRE>

<P>

Then do something like this:


<P>

<PRE>   $ cd $PREFIX
   $ mkdir etc
   $ echo show ip bgp &gt;etc/router.our.domain.bgp # required by ParseBGPDump.pm
   $ time rsh router.our.domain &quot;show ip bgp&quot; &gt;&gt;etc/router.our.domain.bgp
      65.65s real     0.01s user     0.05s system
   $ wc -l /tmp/router.our.domain.bgp
    197883 /tmp/router.our.domain.bgp
</PRE>

<P>

Once <CODE>flowscan</CODE> is up and running with <A HREF="#item_BGPDumpFile">BGPDumpFile</A> configured, it will reload that file if its timestamp indicates that it has
been modified. This allows you to ``freshen'' the image of the routing
table without having to restart <CODE>flowscan</CODE> itself.


<P>

Using the <A HREF="#item_BGPDumpFile">BGPDumpFile</A> option causes <CODE>FlowScan</CODE> to use much more memory than usual. This memory is used to store a <CODE>Net::Patricia</CODE>
trie containing a node for every prefix in the BGP routing table. For
instance, on my system it caused the <CODE>FlowScan</CODE> process to grow to over 50MB, compared to less than 10MB without <A HREF="#item_BGPDumpFile">BGPDumpFile</A>
configured.


<P>

<DT><STRONG><A NAME="item_ASNFile">ASNFile

</A></STRONG><DD>
This directive is optional and is only useful in conjunction with
<A HREF="#item_BGPDumpFile">BGPDumpFile</A>. If specified, this directive will cause the AS names rather than just
their numbers to appear in the Top ASN HTML reports. Its value should be
the path to a file having the format of the file downloaded from this URL:


<P>

<PRE>   <A HREF="ftp://ftp.arin.net/netinfo/asn.txt">ftp://ftp.arin.net/netinfo/asn.txt</A>
</PRE>

<P>

E.g.:


<P>

<PRE>   TopN 10
   BGPDumpFile etc/router.our.domain.bgp
   ASNfile etc/asn.txt
</PRE>

<P>

Once <CODE>flowscan</CODE> is up and running with <A HREF="#item_ASNFile">ASNFile</A> configured, it will reload the file if its timestamp indicates that it has
been modified.


<P>

</DL>
<P>
<HR>
<H1><A NAME="METHODS">METHODS

</A></H1>
This module provides no public methods. It is a report module meant only
for use by <CODE>flowscan</CODE>. Please see the <CODE>FlowScan</CODE> module documentation for information on how to write a FlowScan report
module.


<P>

<P>
<HR>
<H1><A NAME="SEE_ALSO">SEE ALSO

</A></H1>
<CODE>perl(1),</CODE> FlowScan, SubNetIO, <CODE>flowscan(1),</CODE>
Net::Patricia.


<P>

<P>
<HR>
<H1><A NAME="BUGS">BUGS

</A></H1>
When using the <A HREF="#item_BGPDumpFile">BGPDumpFile</A> directive, <CODE>ParseBGPDump</CODE> issues a bunch of warnings which can safely be ignored:


<P>

<PRE>   Failed to parse table version from: show ip bgp
    at (eval 4) line 1
   Failed to parse router IP address from: show ip bgp
    at (eval 4) line 1
   Nexthop not found:    Network          Next Hop            Metric LocPrf Weight Path
   $ at (eval 4) line 1
   Metric not found:    Network          Next Hop            Metric LocPrf Weight Path
   $ at (eval 4) line 1
   Local Preference not found:    Network          Next Hop            Metric LocPrf Weight Path
   $ at (eval 4) line 1
   Weight not found:    Network          Next Hop            Metric LocPrf Weight Path
   $ at (eval 4) line 1
   Origin code not found:    Network          Next Hop            Metric LocPrf Weight Path
   $ at (eval 4) line 1
   Possible truncated file, end-of-dump prompt not found
    at (eval 4) line 1
</PRE>

<P>

I'm not keen on patching <CODE>ParseBGPDump</CODE> to fix this since its license isn't compatible with the GPL. We probably
just need to hack up a complete replacement for <CODE>ParseBGPDump</CODE>.


<P>

When using the <A HREF="#item_BGPDumpFile">BGPDumpFile</A> directive, <CODE>ParseBGPDump</CODE> sometimes mistakes the <CODE>Weight</CODE> for the first ASN in the path. This has the totally undesirable effect of
producing a ``Top Path ASNs'' report that erroneously reports the weight as
one of the Top ASNs! I assume this is an indication of the difficulty of
parsing the output of <CODE>show ip
bgp</CODE>, which apparently was meant for human consumption.


<P>

When using the <A HREF="#item_ASPairs">ASPairs</A> directive, CampusIO will create RRD files that have a <CODE>:</CODE> character in the file name. While RRDTool is able to create RRD files with
those names, it is not able to graph from them. To work around this
problem, create symbolic links in your <A HREF="#item_OutputDir">OutputDir</A>
before attempting to graph from these files. For example:


<P>

<PRE>   $ ln -s 0:n.rrd Us2Them.rrd
   $ ln -s n:0.rrd Them2Us.rrd
</PRE>

<P>

<P>
<HR>
<H1><A NAME="AUTHOR">AUTHOR

</A></H1>
Dave Plonka <A
HREF="MAILTO:<plonka@doit.wisc.edu>"><plonka@doit.wisc.edu></A>


<P>

Copyright (C) 1998-2001 Dave Plonka. This program is free software; you can
redistribute it and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation; either version 2 of
the License, or (at your option) any later version.


<P>

<P>
<HR>
<H1><A NAME="VERSION">VERSION

</A></H1>
The version number is the module file RCS revision number (<STRONG>$Revision: 1.63 $</STRONG>) with the minor number printed right justified with leading zeroes to 3
decimal places. For instance, RCS revision 1.1 would yield a package
version number of 1.001.


<P>

This is so that revision 1.10 (which is version 1.010), for example, will
test greater than revision 1.2 (which is version 1.002) when you want to <STRONG>require</STRONG> a minimum version of this module.


<P>

</DL>
    </BODY>

    </HTML>