------------------------------------------------------------
Ipgrab Developer How To

$Id: DEVELOPER-HOWTO,v 1.2 2007/07/10 13:28:00 farooq-i-azam Exp $
------------------------------------------------------------

Introduction
------------

This document is meant for those who would like to learn
about the development of ipgrab, and then to contribute and
write some modules themselves if they would wish so. This is 
also meant to be used for educational purposes when ipgrab 
is used to teach network protocols, issues related to 
protocol implementation, etc. 

Though there are many open source packet sniffers available,
ipgrab is unique beacuse of its light weight and verbose
output. It is not packed with too many not-needed options.

Also, ipgrab provides for a unique learning opportunity due 
to excellent documentation available and simple design. The
code also contains lot of comments. 

Ipgrab has been developed using C language for the Unix and
Linux environments. It also runs under Microsoft Windows
if installed under cygwin. 

Ipgrab Modules
--------------

Ipgrab consists of many source code files each of which may
be named as an independent module. For example, ip.c is a 
module which dissects and displays IP header information in
a packet. Similary, udp.c is another module which dissects
and displays UPD header information in a packet. As you see,
module names are descriptive. 

Architecture
------------

Before you can write a module, it is also important that you
should understand the current architecture of ipgrab.

In module ipgrab.c, each time a packet is captured, the 
function datalink_pcap() is called. This function itself is
defined in module datalink.c

Function datalink_pcap(), in turn, calls function
datalink() defined in module datalink.c. Depending upon the
type of datalink layer, datalink() further calls the 
appropriate module. Ipgrab, at present, supports the 
following datalink layer types:

Loopback - function dump_loopback() in module loopback.c
Ethernet - function dump_ethernet() in module ethernet.c
SLIP - function dump_slip() in module slip.c
PPP - function dump_ppp() in module ppp.c
RAWIP - function dump_rawip() in module rawip.c

For each datalink layer type, appropriate function provided
in respective module is called by datalink().

As you are aware, for the most part we would be interested
in Ethernet which is the available datalink type under most
of the circumstances.

In function dump_ethernet() in module ethernet.c, a function
ethertype2func() is called upon to determine the network 
layer protocol encapsulated in the ethernet frame and to 
call appropriate function to dissect and display network
layer header information. The function ethertype2func() is 
itself defined in module ethertypes.c. This function may 
call any of the following functions depending upon the type 
of network layer protocol:

IP - dunction dump_ip() in module ip.c
ARP, RARP - function dump_arp() in module arp.c
IPX - function dump_ipx() in module ipx.c
IPv6 - function dump_ipv6() in module ipv6.c
PPP - function dump_ppp() in module ppp.c
PPPHDLC - function dump_ppp_hdlc() in module ppp.c
PPPoE Discovery - function dump_pppoed() in module pppoe.c
PPPoE Session - function dump_pppoes() in module pppoe.c

Most widely used protocol on the network layer is the 
Internet Protocol i.e. IP. Hence, we would be further
interested in IP.

In module ip.c, the function ip_proto_func() determines the
next header on top of IP header and calls the appropriate
function. The function ip_proto_func() is itself defined in
module ip_protocols.c. The following protocols on top of IP
are supported:

ICMP - function dump_icmp() from module icmp.c
IGMP - function dump_igmp() from module igmp.c
TCP - function dump_tcp() from module tcp.c
UDP - function dump_udp() from module udp.c
GRE - function dump_gre() from module gre.c
RSVP - function dump_rsvp() from module rsvp.c
ESP - function dump_esp() from module esp.c
AH - function dump_ah() from module ah.c
OSPF - function dump_ospf() from module ospf.c
ICMPv6 - function dump_icmpv6() from module icmpv6.c

Of all these, only transport layer protocols i.e. TCP and 
UDP carry most commonly used application layer protocols
above them. In both tcp.c and udp.c, the function 
prot2func() is called to determine the application layer
protocol above the transport layer and to call appropriate
function to dissect and display the application layer 
protocol. This functin itself is defined in ip_services.c.

It should be obvious by now that if you would like ipgrab to
dissect and display a new protocol, you should write a new
module. In addition, if the protocol is on the datalink 
layer, you would need to add a couple of lines i.e. register
the module in module datalink.c. If the new protocol falls
in network layer, you should register the new module in 
ethertypes.c. Similarly, if the new protocol is a transport
layer protocol, you should register it in ip_protocols.c.
If the new protocol is an application layer protocol, this
should be registered in ip_services.c.

For example, if I would like to dissect an application layer
protocol NEWPROTO which runs on port, say 1234, I would 
perform the following steps:

1- Write a module newproto.c accompanied by a header file
newproto.h which should include function prototypes, 
header definition, other include files, etc. The module
newproto.c would contain a function e.g. dump_newproto()
to dissect the protocol.

2- Add following lines in ip_services.h

#define PORT_NEWPROTO 1234
#include "newproto.h" 

3- Add following lines in ip_services.c in function 
port2func():
  
   case PORT_NEWPROTO:
      f = dump_newproto();
      break;

This would enable ipgrab to call your module whenever the
protcol NEWPROTO is encountered in a packet. 
 
Writing A Module
----------------

In general, ipgrab modules can be divided into two 
categories:

1- Protocol Modules
2- Support Modules

Protocol modules are the ones which dissect and display
protocols like the examples of ip.c and upd.c given above.
Support modules are auxiliary modules and which make up the
architecture of ipgrab. These modules provide for various
functions which are called by protocol modules. For example,
almost each protocol module calls a function named
get_packet_bytes() which is defined in support module named
packet_manip.c. This function copies a specified number of
packets from the packet to the specified destination. In 
short, protocol modules use functions defined in support
modules to dissect and display packet information.

Generally, we would be interested in writing and adding more
protocol modules to ipgrab. To write a moduel for a 
particular protocol, you should first be familiar with the
protocol itself. Best source of information to learn about
the protocol is the document where it has been defined. 
Under most of the circumstances, this would be an RFC or a
collection of RFCs published by IETF. Appendix A at the end
of this document also contains a list of resources and RFCs
along with existing module names. This list should be 
helpful to those who would like to learn and read various 
protocols and their associated modules. 

As would be clear from above description, a knowledge of 
functions provided in source modules is required if you 
would like to write protocol modules. Here is a list of all
the important support modules:

display.c
error.c
hexbuffer.c
layers.c
packet_manip.c
utilities.c

Anyone interested to contribute a protocol module for ipgrab
should familiarize himself or herself with the functions 
defined in above modules, particularly those in display.c, 
layers.c and packet_manip.c.

Generally, you start your module with declaration of 
necessary variables and then the following line:

set_layer(LAYER_APPLICATION);

Replace LAYER_APPLICATION with whatever layer your protocol
belongs to. Definitions of layers are provided in layers.h.

The module you would write should have a function with the
following prototype:

void dump_newproto (packet_t * pkt); 

packet_t is a structure defined in packet_manip.c.

A copy of the entire contents of the packet would be passed
to your module. A pointer in the form of p->current in 
packet_manip.c keeps track of which parts of the packet have
already been read and dissected.

You can simply copy NEWPROTO header from the packet to your
own structure for the NEWPROTO defined probably in your 
header file for the new module. This can be accomplished by
the function call below:

if (get_packet_bytes((u_int8_t *) &newproto, pkt, 
    sizeof (newproto)) == 0)

   return;

This would simply return if it fails to get packet bytes. 
Remember you do not want to clutter the display during the
packet display. If you would like to report any errors, 
please use functions provided in error.c. But as a rule of
thumb, no error reporting should be done in a protocol 
module. 

After the protocol header has been copied to your structure,
next important task is to perform byte order conversions.
If a protocol field is stored in a structure member which is
two or more bytes long, its byte order should be converted
to host byte order from the network byte order. You use 
functions ntohs() and ntohl() to convert 2 and 4 byte 
variables respectively to host byte order.

After this, you would proceed to display the header fields.
Various functions in display.c are used for this purpose.
Ipgrab supports minimal mode and verbose mode. In minimal
mode, only small amount of information is displayed and
in verbose mode almost all header information is displayed.
The minimal mode is chosen using -m at the command line. We
would like to display information for both modes and proceed
somewhat like below:

if ( my_args->m)
   {
      /* display information for minimal mode */
   }
else
   {
      /* display information for verbose mode */
   }
 
To display items in the minimal mode, we normally use 
function display_minimal() and to display fields in verbose
mode, we normally use function display(). Both these 
functions are defined in display.c. It would be a good idea
to read these functions and understand how they work.

After all the fields have been displayed, the function 
hexbuffer_flush() should be called to display the protocol
fields in hex. This function is defined in hexbuffer.c.
This function would get executed only if -x is specified on 
the command line and ipgrab in not running in the minimal
mode. However, you do not need to check for these 
conditions. These checks are performed by the function 
itself.

If this is an application layer protocol and there is no
further header encapsulated, you are done. However, if there
is any other header encapsulated, you should call the next
appropriate function. 

There are two template files named template.h and template.c
available in src directory that you can use as templates for
your module.

-----------------------------------------------------------

Mike Borella <mike@borella.net>
Muhammad Farooq-i-Azam <farooq@chase.org.pk> 

------------------------------------------------------------

APPENDIX A
----------

ah.c		RFC 2402	IP Authentication Header

arp.c		RFC 0826	Ethernet Address Resolution Protocol
				TCP/IP Tutorial and Technical Overview

ccp.c		RFC 1962	The PPP Compression Control Protocol (CCP)
		RFC 1332	PPP Internet Protocol Control Protocl (IPCP)
				iana/ppp-numbers

chap.c		RFC 1994	PPP Challenge Handshake Authentication Protocol
				iana/ppp-numbers

dhcp.c		RFC 2131	Dynamic Host configuration Protocol
		RFC 2132	DHCP Options and BOOTP Vendor Extensions
				iana/bootp-dhcp-parameters

dns.c		RFC 1034	Domain Names - Concepts And Facilities
		RFC 1035	Domain Names - Implementation And Specification
		RFC 0882	Domain Names - Concepts And Facilities
		RFC 0883	Domain Names - Implementation And Specification
				iana/dns-header-flags
				iana/dns-key-rr
				iana/dns-parameters

esp.c		RFC 2406	IP Encapsulating Security Payload

ethernet.c			TCP/IP Tutorial And Technical Overview

				Ethernet Frame Types: Don Provan's Definitive
				Answer
				http://netlab1/usu.edu/novell.faq/nvfaq-1.htm

				iana/ethernet-numbers
				iana/ieee-802-numbers

ethertypes.c			iana/ethernet-numbers
				iana/ieee-802-numbers

file.c		RFC 1761	Snoop Version 2 Packet Capture File Format

ftpctrl.c	RFC 0959	File Transfer Protocol
				RFC Handbook - http://www.networksorcery.com

gre.c		RFC 1701	Generic Routing Encapsulation
		RFC 2637	Point-to-Point Tunneling Protocol
		RFC 2784	Generic Routing Encapsulation

http.c		RFC 1945	Hypertext Transfer Protocol -- HTTP/1.0

iana.c				iana/enterprise-numbers

icmp.c		RFC 0792	Internet Control Message Protocol
		RFC 1256	ICMP Router Discovery Messages
				iana/icmp-parameters

icmpv6.c	RFC 2463	Internet Control Message Protocol (ICMPv6)
				for the Internet Protocol Version 6 (IPv6)
				Specification
		RFC 2461	Neighbor Discovery for IP Version 6 (IPV6)
		RFC 2460	Internet Protocol, Version 6 (IPv6)
				Specifications
				iana/icmpv6-parameters

igmp.c		RFC 2236	Internet Group Management Protocol, Version 2
				iana/igmp-type-numbers

ip.c		RFC 0791	Internet Protocol
				iana/ip-parameters

ipcp.c		RFC 1332	PPP Internet Protocol Control Protocol (IPCP)
				iana/ppp-numbers

ip_protocols.c	RFC 0791	Internet Protocol			
				iana/protocol-numbers

ip_services.c			iana/port-numbers

ipv6.c		RFC 2460	Internet Protocol, Version 6 (IPv6)
				Specification

ipx.c				RFC Source Book
				http://www.networksorcery.com

				Protocols Directory
				http://www.protocols.com

ipxrip.c			Protocols Directory
				http://www.protocols.com

isakmp.c	RFC 2408	Internet Security Association And Key 
				Management Protocol (ISAKMP)
		RFC 2407	The Internet IP Security Domain of 
				Interpretation for ISAKMP
		RFC 2409	The Internet Key Exchange (IKE)
				iana/isakmp-registry

l2tp.c		RFC 2661	Layer Two Tunneling Protocol "L2TP"
				iana/l2tp-parameters

lcp.c		RFC 1661	The Point-to-Point Protocol (PPP)
				iana/ppp-numbers

llc.c				TCP/IP Tutorial and Technical Overview
				http://www.auggy.mlnet.com/ibm/3376c28.html	
				IEEE 802.2 LLC
				http://ckp.made-it.com/ieee8022.html

mgcp.c		RFC 3435	Media Gateway Control Protocol (MGCP)
				Version 1.0
		RFC 2705	Media Gateway Control Protocol (MGCP) 
				Version 1.0
		RFC 2805	Media Gateway Control Protocol 
				Architecture and Requirements

mobileip.c	RFC 3344	IP Mobility Support for IPv4
				iana/mobileip-numbers

mppc.c		RFC 2118	Microsoft Point-to-Point Compression (MPPC)
				Protocol

netbios_ns.c	RFC 1001	Protocol Standard For a NetBIOS Service
				on a TCP/UDP Transport: Concepts and Methods
		RFC 1002	Protocol Standard For a NetBIOS Service
				on a TCP/UDP Transport: Detailed
				Specifications

nntp.c		RFC 0977	Network News Transfer Protocol

ns_labels.c	RFC 0883	Domain Names: Implementation Specification
		RFC 1001	Protocol Standard For a NetBIOS Service
				on a TCP/UDP Transport: Concepts and Methods
		RFC 1002	Protocol Standard For a NetBIOS Service
				on a TCP/UDP Transport: Detailed
				Specifications	
		RFC 1035	Domain Names - Implementation And
				Specification

open_pcap.c			pcap man pages
				pcap.h

				tcpdump
				http://www.tcpdump.org

ospf.c		RFC 2328	OSPF Version 2
				iana/ospf-authentication-codes
				iana/ospf-apaque-types
				iana/ospf-sig-alg

padding.c	RFC 0791	Internet Protocol
		RFC 0826	Ethernet Address Resolution Protocol

ppp.c		RFC 1661	The Point-to-Point Protocol (PPP)
		RFC 1662	PPP in HDLC-like Framing
				iana/ppp-numbers

pppoe.c		RFC 2516	A Method for Transmitting PPP Over Ethernet
				(PPPoE)

pptp.c		RFC 2637	Point-to-Point Tunneling Protocol (PPTP)

radius_3gpp2.c			3rd Generation Partnership Project 2
				http://www.3gpp2.com

radius.c	RFC 2865	Remote Authentication Dial In User Service
				(RADIUS)
		RFC 2866	RADIUS Accounting
				iana/radius-types

rip.c		RFC 1058	Routing Information Protocol
		RFC 2453	RIP Version 2

ripng.c		RFC 2080	RIPng for IPv6

rsvp.c		RFC 2205	Resource ReSerVation Protocol (RSVP) --
				Version 1 Functional Specification
				iana/rsvp-parameters

rtcp.c		RFC 1889	RTP: A Transport Protocol for Real-Time
				Applications
				iana/rtp-parameters

rtp.c		RFC 1889	RTP: A Transport Protocol for Real-Time
				Applications
				iana/rtp-parameters

sdp.c		RFC 2327	SDP: Session Description Protocol

sip.c		RFC 3261	SIP: Session Initiation Protocol
		RFC 2543	SIP: Session Initiation Protocol (Obsolete)
		RFC 2616	Hypertext Transfer Protocol HTTP/1.1

slip.c		RFC 0914	Thinwire protocol for connecting personal
				computers to the Internet
	
				Zvon - RFC914 - Appendix D -- Serial Line
				Interface Protocol (SLIP)
				http://www.zvon.org/tmRFC/RFC914/output/
				chapter8.html	

slp.c		RFC 2165	Service Location Protocol
		RFC 2608	Service Location Protocol, Version 2

snmp.c		RFC 1157	Simple Network Management Protocol (SNMP)

spx.c				RFC Source Book
				http://www.networksorcery.com
			
				Protocols Directory
				http://www.protocols.com

tcp.c		RFC 0793	Transmission Control Protocol
				iana/tcp-parameters

tftp.c		RFC 1350	The TFTP Protocol (Revision 2)

udp.c		RFC 0768	User Datagram Protocol


Note:
In addition to the above documents for specific topics, the following
resources have been helpful in general for all the topics:

RFC Handbook 		-	http://www.networksorcery.com/
Protocols Directory	-	http://www.protocols.com/