=head1 NAME

NDiff_Quickstart - tutorial for ndiff and related tools.  

=head1 DESCRIPTION

This document provides a fast introduction to each of the ndiff tools:

=over 4

=item ndiff - calculate the differences between two nmap scans

=item ngen - synthetically create baseline nmap results 

=item nrun - execute nmap and ndiff, doing some housekeeping

=back 


=head1 1. ndiff - calculate the differences between two nmap scans

The simplest invocation of ndiff is to using the machine-parseable output
of two previous nmap runs on the same net, for example:


  # nmap -m first_scan.nm  10.0.0.0/24 ...   

  ... later ...

  # nmap -m second_scan.nm 10.0.0.0/24 ...



OK, now we have two scans of the same net at different moments in time. 
Now to see the changes:



  # ndiff -baseline first_scan.nm -observed second_scan.nm

  ... ndiff outputs: ...

  missing hosts:
   < hosts present in first_scan, but missing in second_scan >

  new hosts:
   < hosts present in second_scan, but missing from first_scan >

  changed hosts:
   < hosts present in both scans, but whose port states have changed>
    [ for each host, a list of changes in port states]




Note the options passed to ndiff.   We designate first_scan as the "baseline"
for comparison.  Changes are reported as differences from first_scan. 

Ndiff has additional options and features for controlling output detail and format. 
See L<ndiff> for more information.

Most parameters to the ndiff tools have both short and long versions.
For example B<-baseline> above is synonymous with B<-b>.  They are used interchangeably
throughout this document.  See the relevant man pages for full information.

=head1 2. ngen - synthetically create baseline nmap results 

Using the results of a previous scan as your baseline for comparison is fine
for many purposes, but if you never knew  or liked the state of 
the scanned net, the previous scan probably didn't yield a
satisfactory baseline.

In that case, what you really want as your baseline is a description
of your ideal net - one which reflects your firewall rules and/or security policy.

That is where ngen comes in.  Ngen accepts host and port specifications, and
outputs an equivalent nmap scan result which can be used as an ndiff baseline.
Comparisons with this output then will show how your net varies from your
ideal net.

=head2 Example: a single host range

Suppose we have a net 192.168.2.0/24.  All hosts on this net are web servers, so port
80 is open, the rest should be closed.  There is no firewall or filter device
on the net.  To generate the appropriate baseline:

  # ngen -o baseline.nm  -h 192.168.2.0/24:80o  


The "B<-o> baseline.nm" option specifies the output filename.  The B<-h> switch specifies
a host specification.  The host specification has two parts.  The part before the colon
gives the host IP range, in this case using CIDR notation.  The part after
the colon is the port specification, "80", followed by "o".  The "o" says the
port's state is open.  There are several accepted flags- o=open(default), c=closed,
f=filtered, x=unfiltered, t=tcp (default), and u=udp.

=head2 Example: several host ranges and port specifications

Now suppose we actually have two nets to scan. Most should have only port 80 open. 
But a few hosts should also have port 22 (ssh) and port 53 tcp,udp  (domain) open.  The
command-line would appear as follows:

  # ngen -o baseline_2.nm -h 192.168.2.0/24:80 \
        -h 10.0.2.128/25:80   \
        -h 10.0.2.144-150:22,53,53u

A few things are worth noticing in this example.  First, the three host specifications.
They are applied in order of their appearance, and are cumulative.  So the 10.0.2.128/25
subnet net will end up with ports 80 open on every host, and ports 22/tcp, 53/tcp, and 53/udp
open on hosts .144 through .150.

Second, host ranges can be specified with dash notation, that is 144-150 means every
number in that range.  This notation works for any quad in an IP address.
Dash notation works for ports as well, so 10-12ux means udp ports 10, 11, and 12, each
unfiltered (that's the "x" flag).

Third -- and this was just demonstrated -- you can specify both a protocol and a port
state at the same time for a port, that is "53cu" means port 53/udp, in a closed state.
You never need to explicitly use the t and o flags, since they are the default.


=head2 Example - negation

You can also punch out parts of a host specification with the negation operator (!).
For example, on a net with all web servers except for a few hosts which should
never appear:

  # ngen -o baseline.nm -h 10.0.1.0/24:80 \
              -h '!10.0.1.0'       \
              -h '!10.0.1.45-53'   \
              -h '!10.0.1.255'

Again, the hosts specifications are applied in order of their appearance, so first
the full host range is created, then the negations are applied to remove the
specific ports and/or hosts.  Also note that the negations are quoted to
avoid interpretation by the shell.

=head2 Important: nmap port ranges

Nmap shows ports it deems "interesting", but this is somewhat ambiguous.   What
about the ports which were "uninteresting", and what about those which were never
scanned by nmap?

For each host, nmap reports an "Ignored state", which is a shorthand.  Every
scanned port which wasn't explicitly reported is actually in that Ignored state.
To know which ports these are, we need the set of all ports scanned.  When
running nmap, this is generated with the B<-v> and B<-m> switches together.  On the other
hand, ngen needs to be told explicitly which ports to include in the set-

 # ngen -o baseline.nm -p 1-1024,6000,53u -h 10.0.1.0/25:80

The B<-p> switch above tells ngen that tcp ports 1-1024, tcp port 6000, and udp port 53
are the set of scanned ports.   If no B<-p> switch is specified, ngen will generate
a list of ports from /etc/services and /usr/local/lib/nmap/nmap-services.  However
this may not be what you really want to happen.

By default, the "Ignored state" is closed, so all ports in the B<-p> set above except
for tcp/80 will be in the closed state.  If you want to specify a non-closed
default state for a given host, you must append it on to the host specification-

 # ngen -o baseline.nm  -p 1-1024,6000,53u -h 10.0.1.0/25:80:filtered

Now the default state for the 10.0.1.0/25 subnet is filtered. All scanned ports
will be filtered, except for tcp/80 which will be open.  

Acceptable Ignore states are: open, closed, filtered, and unfiltered.  Ndiff
will also report the psuedo-state "unknown" when a port was present in one
scan but not in the other.



=head1 3. nrun - execute nmap and ndiff, doing some housekeeping

Ndiff and ngen give you a way to create an idealized picture of how your net should
look, then compare the actual state of your net with that picture.  Nrun goes
the next step by taking care of bread-and-butter housekeeping tasks necessary
to use ndiff in an automated fashion, such as with cron.

Specifically, nrun will-

=over 4

=item execute nmap

=item name the results something reasonable

=item store the results somewhere reasonable

=item optionally run ndiff against those results to generate a report

=back

A simple nrun session is as follows-

  # nrun -o my_net_scan.nm  -- -sS 192.168.2.0/26

  ... executes "nmap -sS 192.168.2.0/26   -v -m nmapXXX" ... 
  ...     "-v -m nmap$$" added by nrun ...
  ...     nmap output follows ...


All parameters after the "--" string are passed directly to nmap.  Nrun
adds the B<-m> and B<-v> nmap switches to get output in a form suitable for ndiff.

Notice "B<-o> my_net_scan.nm". This specifies the output file for the
nmap scan.  A literal name for the output file is fine for a one-off scan.
But for monitoring your network you will probably want scans to run
regularly, and results stored from each scan.  A naming convention is
necessary.

Nrun and the other tools allow %-style substitions for inserting time, date,
and/or the local hostname dynamically.  So if we were to have cron run a
scan once a day, we might want a naming convention incorporating the date. 
For example:

  # nrun -o "192.168.2.0.26-%Y%m%D-syn-scan.nm"  -- -sS 192.168.2.0/26

If today's date is Dec 31, 1999, the name will expand to
"192.168.2.0.26-19991231-syn-scan.nm".  

The % codes are consistent with the gnu date command, with the addition of %F,
which expands to the local hostname.  See L<nrun> for more information.


=head2 Additional functionality - Data Stores

Nrun, ndiff, and ngen use nmap-format files on the local filesystem by default.
This allows you the most flexibility in managing the nmap results data.  

For the sake of future expansion, the ndiff tools have a mechanism to use
other methods of storage and retrieval, for example pgp-encrypted nmap
files, or a database engine.

Anywhere you specify a results file name in any of the ndiff tools, you can
instead specify a string starting with a colon (:).  When you do this, the
string (sans colon) is regarded as a unique key into a data store.
For example:

 # ngen -o :baseline-syn-192.168.2.0-net

 ...

 # nrun -o :daily-scan-for-192.168.2.0-net-%Y%m%D

 ...

 # ndiff -b :baseline-syn-192.168.2.0-net \
        -o :daily-scan-for-192.168.2.0-net-20000401


In all of the above cases, the tools will store/retrieve the results data
from the configured data store.  This can be very convenient, allowing
you to think about the scan rather than managing nmap files.

Currently the only implemented data store type is an nmap file stored in
special directory.  That special directory is set in a global 
configuration file, B<ndiff.conf>, which is read at runtime.

A sample file B<ndiff.conf.sample> is installed into the same directory
as the NDiff perl modules (PortScan subdirectory under perl extensions).

Copy and edit this file per the comments to configure data stores.  The only
thing you need to change is "root" attribute to point to the desired
results storage directory.

The Ndiff tools will look for ndiff.conf in the following places, in order:

  ~/.ndiff/ndiff.conf
  the directory pointed to by the environment variable NDIFF_ROOT
  /usr/local/lib/ndiff/ndiff.conf
  ./ndiff.conf

A sample ndiff.conf is included below, but it is not definitive -- use the
sample file stored with the installed NDiff modules.

NDiff.conf:


 #!/usr/bin/perl
 #
 # $Id: NDiff_Quickstart.pod,v 1.8 2000/12/17 09:40:26 levine Exp $
 #
 # Copyright (C) 2000  James D. Levine (jdl@vinecorp.com)
 #
 #
 #   This program is free software; you can redistribute it and/or
 #   modify it under the terms of the GNU General Public License
 #   as published by the Free Software Foundation; either version 2
 #   of the License, or (at your option) any later version.
 # 
 #   This program is distributed in the hope that it will be useful,
 #   but WITHOUT ANY WARRANTY; without even the implied warranty of
 #   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 #   GNU General Public License for more details.
 # 
 #   You should have received a copy of the GNU General Public License
 #   along with this program; if not, write to the Free Software
 #   Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 
 #   02111-1307, USA.
 #
 ####################################################################

 package PortScan::ScanContext;

 #
 # Site-specific settings go here.  Currently there are only
 # two that need setting.
 #
 # The ndiff tools try to find ndiff.conf in order of the following:  
 # 
 #  $HOME/.ndiff/ndiff.conf
 #  the directory pointed to by the environment variable NDIFF_ROOT
 #  /usr/local/lib/ndiff/ndiff.conf
 #  ./ndiff.conf
 #
 # This order of precedence is not necessarily rational.
 #

 sub get_config
 {
     return {

	 #
	 # The configured DataStore type - currently only 
	 # NmapFile is implemented
	 #

	 default_data_store_type => "PortScan::NmapFile",


	 #
	 # Properties to set in the default DataStore instance.
	 # this is a comma-separated list of key=value pairs.
	 #
	 # For the NmapFile object, only "root" is used, which
	 # should point to a directory where results and baseline
	 # files will be stored.
         #
         # This directory must be initially created by hand
	 # and have appropriate permissions.
         #

	 default_data_store_props => "root=/tmp/ndiff_data",

     }
 };


1;




=cut