1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
|
<HTML>
<HEAD>
<TITLE>NDiff_Quickstart - tutorial for ndiff and related tools.</TITLE>
<LINK REV="made" HREF="mailto:root@porky.devel.redhat.com">
</HEAD>
<BODY>
<!-- INDEX BEGIN -->
<UL>
<LI><A HREF="#NAME">NAME</A>
<LI><A HREF="#DESCRIPTION">DESCRIPTION</A>
<LI><A HREF="#1_ndiff_calculate_the_differe">1. ndiff - calculate the differences between two nmap scans</A>
<LI><A HREF="#2_ngen_synthetically_create_b">2. ngen - synthetically create baseline nmap results </A>
<UL>
<LI><A HREF="#Example_a_single_host_range">Example: a single host range</A>
<LI><A HREF="#Example_several_host_ranges_and">Example: several host ranges and port specifications</A>
<LI><A HREF="#Example_negation">Example - negation</A>
<LI><A HREF="#Important_nmap_port_ranges">Important: nmap port ranges</A>
</UL>
<LI><A HREF="#3_nrun_execute_nmap_and_ndiff">3. nrun - execute nmap and ndiff, doing some housekeeping</A>
<UL>
<LI><A HREF="#Additional_functionality_Data_">Additional functionality - Data Stores</A>
</UL>
</UL>
<!-- INDEX END -->
<HR>
<P>
<H1><A NAME="NAME">NAME</A></H1>
<P>
NDiff_Quickstart - tutorial for ndiff and related tools.
<P>
<HR>
<H1><A NAME="DESCRIPTION">DESCRIPTION</A></H1>
<P>
This document provides a fast introduction to each of the ndiff tools:
<DL>
<DT><STRONG><A NAME="item_ndiff">ndiff - calculate the differences between two nmap scans</A></STRONG><DD>
<DT><STRONG><A NAME="item_ngen">ngen - synthetically create baseline nmap results</A></STRONG><DD>
<DT><STRONG><A NAME="item_nrun">nrun - execute nmap and ndiff, doing some housekeeping</A></STRONG><DD>
</DL>
<P>
<HR>
<H1><A NAME="1_ndiff_calculate_the_differe">1. ndiff - calculate the differences between two nmap scans</A></H1>
<P>
The simplest invocation of ndiff is to using the machine-parseable output
of two previous nmap runs on the same net, for example:
<P>
<PRE> # nmap -m first_scan.nm 10.0.0.0/24 ...
</PRE>
<P>
<PRE> ... later ...
</PRE>
<P>
<PRE> # nmap -m second_scan.nm 10.0.0.0/24 ...
</PRE>
<P>
OK, now we have two scans of the same net at different moments in time. Now
to see the changes:
<P>
<PRE> # ndiff -baseline first_scan.nm -observed second_scan.nm
</PRE>
<P>
<PRE> ... ndiff outputs: ...
</PRE>
<P>
<PRE> missing hosts:
< hosts present in first_scan, but missing in second_scan >
</PRE>
<P>
<PRE> new hosts:
< hosts present in second_scan, but missing from first_scan >
</PRE>
<P>
<PRE> changed hosts:
< hosts present in both scans, but whose port states have changed>
[ for each host, a list of changes in port states]
</PRE>
<P>
Note the options passed to ndiff. We designate first_scan as the
``baseline'' for comparison. Changes are reported as differences from
first_scan.
<P>
Ndiff has additional options and features for controlling output detail and
format. See <EM>ndiff</EM> for more information.
<P>
Most parameters to the ndiff tools have both short and long versions. For
example <STRONG>-baseline</STRONG> above is synonymous with <STRONG>-b</STRONG>. They are used interchangeably throughout this document. See the relevant
man pages for full information.
<P>
<HR>
<H1><A NAME="2_ngen_synthetically_create_b">2. ngen - synthetically create baseline nmap results</A></H1>
<P>
Using the results of a previous scan as your baseline for comparison is
fine for many purposes, but if you never knew or liked the state of the
scanned net, the previous scan probably didn't yield a satisfactory
baseline.
<P>
In that case, what you really want as your baseline is a description of
your ideal net - one which reflects your firewall rules and/or security
policy.
<P>
That is where ngen comes in. Ngen accepts host and port specifications, and
outputs an equivalent nmap scan result which can be used as an ndiff
baseline. Comparisons with this output then will show how your net varies
from your ideal net.
<P>
<HR>
<H2><A NAME="Example_a_single_host_range">Example: a single host range</A></H2>
<P>
Suppose we have a net 192.168.2.0/24. All hosts on this net are web
servers, so port 80 is open, the rest should be closed. There is no
firewall or filter device on the net. To generate the appropriate baseline:
<P>
<PRE> # ngen -o baseline.nm -h 192.168.2.0/24:80o
</PRE>
<P>
The ``<STRONG>-o</STRONG> baseline.nm'' option specifies the output filename. The <STRONG>-h</STRONG> switch specifies a host specification. The host specification has two
parts. The part before the colon gives the host IP range, in this case
using CIDR notation. The part after the colon is the port specification,
``80'', followed by ``o''. The ``o'' says the port's state is open. There
are several accepted flags- o=open(default), c=closed, f=filtered,
x=unfiltered, t=tcp (default), and u=udp.
<P>
<HR>
<H2><A NAME="Example_several_host_ranges_and">Example: several host ranges and port specifications</A></H2>
<P>
Now suppose we actually have two nets to scan. Most should have only port
80 open. But a few hosts should also have port 22 (ssh) and port 53 tcp,udp
(domain) open. The command-line would appear as follows:
<P>
<PRE> # ngen -o baseline_2.nm -h 192.168.2.0/24:80 \
-h 10.0.2.128/25:80 \
-h 10.0.2.144-150:22,53,53u
</PRE>
<P>
A few things are worth noticing in this example. First, the three host
specifications. They are applied in order of their appearance, and are
cumulative. So the 10.0.2.128/25 subnet net will end up with ports 80 open
on every host, and ports 22/tcp, 53/tcp, and 53/udp open on hosts .144
through .150.
<P>
Second, host ranges can be specified with dash notation, that is 144-150
means every number in that range. This notation works for any quad in an IP
address. Dash notation works for ports as well, so 10-12ux means udp ports
10, 11, and 12, each unfiltered (that's the ``x'' flag).
<P>
Third -- and this was just demonstrated -- you can specify both a protocol
and a port state at the same time for a port, that is ``53cu'' means port
53/udp, in a closed state. You never need to explicitly use the t and o
flags, since they are the default.
<P>
<HR>
<H2><A NAME="Example_negation">Example - negation</A></H2>
<P>
You can also punch out parts of a host specification with the negation
operator (!). For example, on a net with all web servers except for a few
hosts which should never appear:
<P>
<PRE> # ngen -o baseline.nm -h 10.0.1.0/24:80 \
-h '!10.0.1.0' \
-h '!10.0.1.45-53' \
-h '!10.0.1.255'
</PRE>
<P>
Again, the hosts specifications are applied in order of their appearance,
so first the full host range is created, then the negations are applied to
remove the specific ports and/or hosts. Also note that the negations are
quoted to avoid interpretation by the shell.
<P>
<HR>
<H2><A NAME="Important_nmap_port_ranges">Important: nmap port ranges</A></H2>
<P>
Nmap shows ports it deems ``interesting'', but this is somewhat ambiguous.
What about the ports which were ``uninteresting'', and what about those
which were never scanned by nmap?
<P>
For each host, nmap reports an ``Ignored state'', which is a shorthand.
Every scanned port which wasn't explicitly reported is actually in that
Ignored state. To know which ports these are, we need the set of all ports
scanned. When running nmap, this is generated with the <STRONG>-v</STRONG> and <STRONG>-m</STRONG> switches together. On the other hand, ngen needs to be told explicitly
which ports to include in the set-
<P>
<PRE> # ngen -o baseline.nm -p 1-1024,6000,53u -h 10.0.1.0/25:80
</PRE>
<P>
The <STRONG>-p</STRONG> switch above tells ngen that tcp ports 1-1024, tcp port 6000, and udp port
53 are the set of scanned ports. If no <STRONG>-p</STRONG> switch is specified, ngen will generate a list of ports from /etc/services
and /usr/local/lib/nmap/nmap-services. However this may not be what you
really want to happen.
<P>
By default, the ``Ignored state'' is closed, so all ports in the <STRONG>-p</STRONG> set above except for tcp/80 will be in the closed state. If you want to
specify a non-closed default state for a given host, you must append it on
to the host specification-
<P>
<PRE> # ngen -o baseline.nm -p 1-1024,6000,53u -h 10.0.1.0/25:80:filtered
</PRE>
<P>
Now the default state for the 10.0.1.0/25 subnet is filtered. All scanned
ports will be filtered, except for tcp/80 which will be open.
<P>
Acceptable Ignore states are: open, closed, filtered, and unfiltered. Ndiff
will also report the psuedo-state ``unknown'' when a port was present in
one scan but not in the other.
<P>
<HR>
<H1><A NAME="3_nrun_execute_nmap_and_ndiff">3. nrun - execute nmap and ndiff, doing some housekeeping</A></H1>
<P>
Ndiff and ngen give you a way to create an idealized picture of how your
net should look, then compare the actual state of your net with that
picture. Nrun goes the next step by taking care of bread-and-butter
housekeeping tasks necessary to use ndiff in an automated fashion, such as
with cron.
<P>
Specifically, nrun will-
<DL>
<DT><STRONG><A NAME="item_execute">execute nmap</A></STRONG><DD>
<DT><STRONG><A NAME="item_name">name the results something reasonable</A></STRONG><DD>
<DT><STRONG><A NAME="item_store">store the results somewhere reasonable</A></STRONG><DD>
<DT><STRONG><A NAME="item_optionally">optionally run ndiff against those results to generate a report</A></STRONG><DD>
</DL>
<P>
A simple nrun session is as follows-
<P>
<PRE> # nrun -o my_net_scan.nm -- -sS 192.168.2.0/26
</PRE>
<P>
<PRE> ... executes "nmap -sS 192.168.2.0/26 -v -m nmapXXX" ...
... "-v -m nmap$$" added by nrun ...
... nmap output follows ...
</PRE>
<P>
All parameters after the ``--'' string are passed directly to nmap. Nrun
adds the <STRONG>-m</STRONG> and <STRONG>-v</STRONG> nmap switches to get output in a form suitable for ndiff.
<P>
Notice ``<STRONG>-o</STRONG> my_net_scan.nm''. This specifies the output file for the nmap scan. A
literal name for the output file is fine for a one-off scan. But for
monitoring your network you will probably want scans to run regularly, and
results stored from each scan. A naming convention is necessary.
<P>
Nrun and the other tools allow %-style substitions for inserting time,
date, and/or the local hostname dynamically. So if we were to have cron run
a scan once a day, we might want a naming convention incorporating the
date. For example:
<P>
<PRE> # nrun -o "192.168.2.0.26-%Y%m%D-syn-scan.nm" -- -sS 192.168.2.0/26
</PRE>
<P>
If today's date is Dec 31, 1999, the name will expand to
``192.168.2.0.26-19991231-syn-scan.nm''.
<P>
The % codes are consistent with the gnu date command, with the addition of
%F, which expands to the local hostname. See <EM>nrun</EM> for more information.
<P>
<HR>
<H2><A NAME="Additional_functionality_Data_">Additional functionality - Data Stores</A></H2>
<P>
Nrun, ndiff, and ngen use nmap-format files on the local filesystem by
default. This allows you the most flexibility in managing the nmap results
data.
<P>
For the sake of future expansion, the ndiff tools have a mechanism to use
other methods of storage and retrieval, for example pgp-encrypted nmap
files, or a database engine.
<P>
Anywhere you specify a results file name in any of the ndiff tools, you can
instead specify a string starting with a colon (:). When you do this, the
string (sans colon) is regarded as a unique key into a data store. For
example:
<P>
<PRE> # ngen -o :baseline-syn-192.168.2.0-net
</PRE>
<P>
<PRE> ...
</PRE>
<P>
<PRE> # nrun -o :daily-scan-for-192.168.2.0-net-%Y%m%D
</PRE>
<P>
<PRE> ...
</PRE>
<P>
<PRE> # ndiff -b :baseline-syn-192.168.2.0-net \
-o :daily-scan-for-192.168.2.0-net-20000401
</PRE>
<P>
In all of the above cases, the tools will store/retrieve the results data
from the configured data store. This can be very convenient, allowing you
to think about the scan rather than managing nmap files.
<P>
Currently the only implemented data store type is an nmap file stored in
special directory. That special directory is set in a global configuration
file, <STRONG>ndiff.conf</STRONG>, which is read at runtime.
<P>
A sample file <STRONG>ndiff.conf.sample</STRONG> is installed into the same directory as the NDiff perl modules (PortScan
subdirectory under perl extensions).
<P>
Copy and edit this file per the comments to configure data stores. The only
thing you need to change is ``root'' attribute to point to the desired
results storage directory.
<P>
The Ndiff tools will look for ndiff.conf in the following places, in order:
<P>
<PRE> ~/.ndiff/ndiff.conf
the directory pointed to by the environment variable NDIFF_ROOT
/usr/local/lib/ndiff/ndiff.conf
./ndiff.conf
</PRE>
<P>
A sample ndiff.conf is included below, but it is not definitive -- use the
sample file stored with the installed NDiff modules.
<P>
NDiff.conf:
<P>
<PRE> #!/usr/bin/perl
#
# $Id: ndiff.conf,v 1.5 2000/05/04 23:15:57 levine Exp $
#
# Copyright (C) 2000 James D. Levine (jdl@vinecorp.com)
#
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
# 02111-1307, USA.
#
####################################################################
</PRE>
<P>
<PRE> package PortScan::ScanContext;
</PRE>
<P>
<PRE> #
# Site-specific settings go here. Currently there are only
# two that need setting.
#
# The ndiff tools try to find ndiff.conf in order of the following:
#
# $HOME/.ndiff/ndiff.conf
# the directory pointed to by the environment variable NDIFF_ROOT
# /usr/local/lib/ndiff/ndiff.conf
# ./ndiff.conf
#
# This order of precedence is not necessarily rational.
#
</PRE>
<P>
<PRE> sub get_config
{
return {
</PRE>
<P>
<PRE> #
# The configured DataStore type - currently only
# NmapFile is implemented
#
</PRE>
<P>
<PRE> default_data_store_type => "PortScan::NmapFile",
</PRE>
<P>
<PRE> #
# Properties to set in the default DataStore instance.
# this is a comma-separated list of key=value pairs.
#
# For the NmapFile object, only "root" is used, which
# should point to a directory where results and baseline
# files will be stored.
#
# This directory must be initially created by hand
# and have appropriate permissions.
#
</PRE>
<P>
<PRE> default_data_store_props => "root=/tmp/ndiff_data",
</PRE>
<P>
<PRE> }
};
</PRE>
<P>
1;
</BODY>
</HTML>
|
