\documentstyle[12pt,twoside]{article}
\def\TITLE{IP Command Reference}
\input preamble
\begin{center}
\Large\bf IP Command Reference.
\end{center}


\begin{center}
{ \large Alexey~N.~Kuznetsov } \\
\em Institute for Nuclear Research, Moscow \\
\verb|kuznet@ms2.inr.ac.ru| \\
\rm April 14, 1999
\end{center}

\vspace{5mm}

\tableofcontents

\newpage

\section{About this document.}

This document presents comprehensive description of \verb|ip| utility
from \verb|iproute2| package. It is not tutorial and even not user guide,
it is rather {\em dictionary\/} not explaining terms,
but translating them to another ones, which also may be unknown to reader.
However, the document is self-contained and provided the reader has 
basic networking background, he will find enough information
and examples to understand and to configure Linux-2.2 IP and IPv6
networking.

The document is split to the sections explaining \verb|ip| commands
and options, decrypting the \verb|ip| output and containing a few of examples.
More voluminous examples and some topics, which require more elaborated
discussion, are moved to appendix.

The paragraphs beginning with sign NB contain side notes, warnings about
bugs and design drawbacks. They may be skipped at the first reading.

\section{{\tt ip} --- command syntax.}

Generic form of \verb|ip| command is:
\begin{verbatim}
ip [ OPTIONS ] OBJECT [ COMMAND [ ARGUMENTS ]]
\end{verbatim}
where \verb|OPTIONS| is set of optional modifiers affecting
general behaviour of the \verb|ip| utility or changing its output. All the options
begin with character \verb|'-'| and may be used both in long and abbreviated 
form. Currently the following options are available:

\begin{itemize}
\item \verb|-V|, \verb|-Version|

--- print version of the \verb|ip| utility and exit.


\item \verb|-s|, \verb|-stats|, \verb|-statistics|

--- output more information. If the option
is repeated twice or more, amount of the information increases.
As rule the information is statistics or some time values.


\item \verb|-f|, \verb|-family| followed by protocol family
identifier: \verb|inet|, \verb|inet6| or \verb|link|.

--- enforce protocol family to use. If the option is not present,
protocol family is guessed from another arguments. If the rest  of command
line does not give enough information to guess family, \verb|ip| falls back to default
one, usually to \verb|inet| or to \verb|any|. \verb|link| is special family
identifier meaning that no networking protocol is involved.

\item \verb|-4|

--- shortcut for \verb|-family inet|.

\item \verb|-6|

--- shortcut for \verb|-family inet6|.

\item \verb|-0|

--- shortcut for \verb|-family link|.


\item \verb|-o|, \verb|-oneline|

--- output each record in single line, replacing line feeds
with character \verb|'\'|. It is convenient when you want to
count records with \verb|wc| or to \verb|grep| the output. The trivial
script \verb|rtpr| converts the output back to readable form.

\item \verb|-r|, \verb|-resolve|

--- allow to use system name resolver to print DNS names instead of
host addresses.

\begin{NB}
 Do not use this option, when reporting bugs or querying for an advice.
\end{NB}
\begin{NB}
 \verb|ip| never tries to use DNS to resolve names to addresses.
\end{NB}

\end{itemize}

\verb|OBJECT| is object to manage or to get information about.
The object types understood by current \verb|ip| are:

\begin{itemize}
\item \verb|link| --- network device.
\item \verb|address| --- protocol (IP or IPv6) address on a device.
\item \verb|neighbour| --- ARP or NDISC cache entry.
\item \verb|route| --- routing table entry.
\item \verb|rule| --- rule in routing policy database.
\item \verb|maddress| --- multicast address.
\item \verb|mroute| --- multicast routing cache entry.
\item \verb|tunnel| --- tunnel over IP.
\end{itemize}

Again, names of all the objects may be written both in full
and abbreviated form, f.e.\ \verb|address| is abbreviated as \verb|addr|
or just \verb|a|.

\verb|COMMAND| specifies action to perform on the object.
The set of possible actions depends on object type,
as rule it is possible to \verb|add|, to \verb|delete| and
to \verb|show| (or to \verb|list|) the object(s), but some objects
allow not all these operations or have some additional commands.
Command \verb|help| is available for all the objects, it prints
out list of available commands and argument syntax conventions.

If no command is given, some default command is assumed.
Usually it is \verb|list| or, if the objects of this class
cannot be listed, \verb|help|.

\verb|ARGUMENTS| is list of arguments to the command.
The arguments depend on command and object. There are two types of arguments:
{\em flags\/}, abbreviated with single keyword, and {\em parameters\/},
consisting of a keyword followed by a value. For convenience
each command has some {\em default parameter\/},
which may be omitted. F.e.\ parameter \verb|dev| is default
for {\tt ip link} command, so that {\tt ip link ls eth0} is equivalent
to {\tt ip link ls dev eth0}.
In command descriptions following below such parameters
are distinguished with marker ``(default)''.

Almost all the keywords may be abbreviated with several first (or even single)
letters. The shortcuts are convenient when \verb|ip| is used interactively,
but they are not recommended in scripts or when reporting bugs
or asking for an advice. ``Officially'' allowed abbreviations are listed
in the document body.



\section{{\tt ip} --- error messages.}

\verb|ip| may fail by one of the following reasons:

\begin{itemize}
\item
Wrong syntax of command line: an unknown keyword, wrongly formatted
IP address {\em et al\/}. In this case \verb|ip| exits not doing any actions,
but printing some error message, as rule containing an information
about the reason of the failure. Sometimes it prints also help page.

\item
The arguments did not pass verification for self-consistency.

\item
\verb|ip| failed to compile kernel request from the arguments,
because user gave not enough information.

\item
Kernel returned an error to some syscall. In this case \verb|ip|
prints the error message, as it is output with \verb|perror(3)|,
prefixed with a comment and syscall identifier.

\item
Kernel returned an error to some RTNETLINK request.
In this case \verb|ip| prints the error message, as it is output
with \verb|perror(3)| prefixed with ``RTNETLINK answers:''.

\end{itemize}

All the operations are atomic, i.e.\ 
if the \verb|ip| utility fails, it does not change anything
in the system. One harmful exception is \verb|ip link| command
(Sec.\ref{IP-LINK}, p.\pageref{IP-LINK}),
which may change only part of device parameters ordered
by command line.

It is difficult to list all the error messages (especially,
about syntax errors), however as rule their meaning is clear
in the context of the command.

The most common mistakes are:

\begin{enumerate}
\item Netlink is not configured in the kernel. The message is:
\begin{verbatim}
Cannot open netlink socket: Invalid value
\end{verbatim}

\item RTNETLINK is not configured in the kernel. In this case
one of the following messages may be printed, depending on the command:
\begin{verbatim}
Cannot talk to rtnetlink: Connection refused
Cannot send dump request: Connection refused
\end{verbatim}

\item Option \verb|CONFIG_IP_MULTIPLE_TABLES| was not selected
when configuring kernel. In this case any attempt to use command
\verb|ip| \verb|rule| will fail, f.e.
\begin{verbatim}
kuznet@kaiser $ ip rule list
RTNETLINK error: Invalid argument
dump terminated
\end{verbatim}

\end{enumerate}


\section{{\tt ip link} --- network device configuration.}
\label{IP-LINK}

\paragraph{Object:} \verb|link| is a network device and the corresponding
commands allow to look at the state of devices and to change it.

\paragraph{Commands:} \verb|set| and \verb|show| (or \verb|list|).

\subsection{{\tt ip link set} --- change device attributes.}

\paragraph{Abbreviations:} \verb|set|, \verb|s|.

\paragraph{Arguments:}

\begin{itemize}
\item \verb|dev NAME| (default)

--- \verb|NAME| specifies network device to operate on.

\item \verb|up| and \verb|down|

--- change state of the device to \verb|UP| or to \verb|DOWN|.

\item \verb|arp on| or \verb|arp off|

--- change \verb|NOARP| flag on the device.

\begin{NB}
This operation is {\em not allowed\/} if the device is in state \verb|UP|.
Though neither \verb|ip| utility nor kernel check for this condition,
you can get unpredictable results changing the flag while the
device is running.
\end{NB}

\item \verb|multicast on| or \verb|multicast off|

--- change \verb|MULTICAST| flag on the device.

\item \verb|dynamic on| or \verb|dynamic off|

--- change \verb|DYNAMIC| flag on the device.

\item \verb|name NAME|

--- change name of the device. This operation is not
recommended if the device is running or has some addresses
already configured.

\item \verb|txqueuelen NUMBER| or \verb|txqlen NUMBER|

--- change transmit queue length of the device.

\item \verb|mtu NUMBER|

--- change MTU of the device.

\item \verb|address LLADDRESS|

--- change station address of the interface.

\item \verb|broadcast LLADDRESS|, \verb|brd LLADDRESS| or \verb|peer LLADDRESS|

--- change link layer broadcast address or peer address in the case
if the interface is \verb|POINTOPOINT|.

\vskip 1mm
\begin{NB}
For the most of devices (f.e.\ for Ethernet) changing link layer
broadcast address will break networking.
Do not use it, if you do not understand what this operation really does.
\end{NB}

\end{itemize}

\vskip 1mm
\begin{NB}
{\tt ip} utility does not allow to change \verb|PROMISC| 
and \verb|ALLMULTI| flags, these flags are considered as
obsolete and should not be changed administratively.
\end{NB}

\paragraph{Warning:} If multiple parameter changes are requested,
\verb|ip| aborts immediately after the change of at least one of them
has failed.
It is the only case, when \verb|ip| can move the system to
an unpredictable state. The solution is to avoid to change
several parameters by one {\tt ip link set} call.

\paragraph{Examples:}
\begin{itemize}
\item \verb|ip link set dummy address 00:00:00:00:00:01|

--- change station address of the interface \verb|dummy|.

\item \verb|ip link set dummy up|

--- start the interface \verb|dummy|.

\end{itemize}


\subsection{{\tt ip link show} --- look at device attributes.}
\label{IP-LINK-SHOW}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|lst|, \verb|sh|, \verb|ls|,
\verb|l|.

\paragraph{Arguments:}
\begin{itemize}
\item \verb|dev NAME| (default)

--- \verb|NAME| specifies network device to show.
If this argument is omitted, the command lists all the devices.

\item \verb|up|

--- display only running interfaces.

\end{itemize}


\paragraph{Output format:}

\begin{verbatim}
kuznet@alisa:~ $ ip link ls eth0
3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc cbq qlen 100
    link/ether 00:a0:cc:66:18:78 brd ff:ff:ff:ff:ff:ff
kuznet@alisa:~ $ ip link ls sit0
5: sit0@NONE: <NOARP,UP> mtu 1480 qdisc noqueue
    link/sit 0.0.0.0 brd 0.0.0.0
kuznet@alisa:~ $ ip link ls dummy
2: dummy: <BROADCAST,NOARP> mtu 1500 qdisc noop
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
kuznet@alisa:~ $ 
\end{verbatim}


The number followed by colon is {\em interface index\/} or {\em ifindex\/}.
This number uniquely identifies the interface. Then {\em interface name\/}
follows (\verb|eth0|, \verb|sit0| etc.). Interface name is also
unique at every given moment, however interface may disappear from the
list (f.e.\ when corresponding driver module is unloaded) and another
one with the same name will be recreated later. Besides that,
administrator may change name of any device with
\verb|ip| \verb|link| \verb|set| \verb|name|
to make it more intelligible.

The interface name may have another name or \verb|NONE| appended 
after sign \verb|@|. It means that this device is bound to some another
device,
i.e.\ packets send through it are encapsulated and sent via the ``master''
device. If the name is \verb|NONE|, the master is unknown.

Then we see interface {\em mtu\/} (``maximal transfer unit''), it determines
maximal size of data, which can be sent as single packet over this interface.

{\em qdisc\/} (``queuing discipline'') shows queuing algorithm used
on the interface. Particularly, \verb|noqueue| means that this interface
does not queue anything and \verb|noop| means that the interface is in blackhole
mode i.e.\ all the packets sent to it are immediately discarded.
{\em qlen\/} is default transmit queue length of the device measured
in packets.

In angle brackets interface flags are summarized.

\begin{itemize}
\item \verb|UP| --- this device is turned on, it is ready to accept
packets for transmission and it may inject to kernel packets received
from another nodes on the network.

\item \verb|LOOPBACK| --- the interface does not communicate to another
hosts, all the packets, which are sent through it, will be returned
back and nothing but bounced back packets can be received.

\item \verb|BROADCAST| --- this device has facility to send packets
to all the hosts sharing the same link. Typical example is Ethernet link.

\item \verb|POINTOPOINT| --- the link has only two ends and two nodes
attached to it. All the packets sent to the link will reach the peer
and all the packets received by us are origined by this single peer.

If neither \verb|LOOPBACK| nor \verb|BROADCAST| nor \verb|POINTOPOINT|
are set, the interface is supposed to be NMBA (Non-Broadcast Multi-Access).
It is the most generic type of devices and the most complicated one, because
the host attached to a NBMA link has no means to send to anyone
without additionally configured information.

\item \verb|MULTICAST| --- is advisory flag marking that the interface
is aware of multicasting i.e.\ sending packets to some subset of neighbouring
nodes. Broadcasting is particular case of multicasting, when multicast
group consists of all the nodes on the link. It is important to emphasize
that software {\em must not\/} interpret absence of this flag as impossibility
to use multicasting on the interface. Any \verb|POINTOPOINT| and
\verb|BROADCAST| link is multicasting by definition, because we have
direct access to all the neighbours and, hence, to any part of them.
Certainly, use of high bandwidth multicast transfers is not recommended
on broadcast-only links because of high expenses, but it is not strictly
prohibited.

\item \verb|PROMISC| --- the device listens and feeds to kernel all the
traffic on the link even if it is not destined to us, not broadcasted
and not destined to a multicast group, which we are member of. Usually
this mode exists only on broadcast links and used by bridges and for network
monitoring.

\item \verb|ALLMULTI| --- the device receives all multicast packets
wandering on the link. This mode is used by multicast routers.

\item \verb|NOARP| --- this flag is different of another ones. It has
no invariant value and its interpretation depends on network protocols
involved. As rule it indicates that the device need not any address
resolution and software or hardware know, how to deliver packets
without any help from protocol stacks.

\item \verb|DYNAMIC| --- is advisory flag marking this interface as
dynamically created and destroyed.

\item \verb|SLAVE| --- this interface is bonded to some another interfaces
to share link capacities.

\end{itemize}

\vskip 1mm
\begin{NB}
There exist another flags, but they are either obsolete (\verb|NOTRAILERS|)
or not implemented (\verb|DEBUG|) or even specific to some devices
(\verb|MASTER|, \verb|AUTOMEDIA| and \verb|PORTSEL|). We do not discuss
them here.
\end{NB}
\begin{NB}
The values of \verb|PROMISC| and \verb|ALLMULTI| flags
shown by \verb|ifconfig| utility and by \verb|ip| utility
are {\em different\/}. \verb|ip link ls| shows true device state,
while \verb|ifconfig| showing virtual state which was set with
\verb|ifconfig| itself.
\end{NB}


The second line contains information on link layer addresses,
associated with the device. The first word (\verb|ether|, \verb|sit|)
defines interface hardware type. This type determines format and semantics
of the addresses and logically it is part of the address.
Default format of station address and broadcast address
(or peer address for pointopoint links) is
sequence of hexadecimal bytes separated by colons, but some link
types may have their natural address format, f.e.\ addresses
of tunnels over IP are printed as dotted-quad IP addresses.

\vskip 1mm
\begin{NB}
  NBMA links have no well-defined broadcast or peer address,
  however this field may contain useful information, f.e.\
  about address of broadcast relay or about address of ARP server.
\end{NB}
\begin{NB}
Multicast addresses are not shown by this command, see
\verb|ip maddr ls| in~Sec.\ref{IP-MADDR} (p.\pageref{IP-MADDR} of this
document).
\end{NB}


\paragraph{Statistics:} With the option \verb|-statistics| \verb|ip| also
prints interface statistics:

\begin{verbatim}
kuznet@alisa:~ $ ip -s link ls eth0
3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc cbq qlen 100
    link/ether 00:a0:cc:66:18:78 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast   
    2449949362 2786187  0       0       0       0      
    TX: bytes  packets  errors  dropped carrier collsns 
    178558497  1783945  332     0       332     35172  
kuznet@alisa:~ $
\end{verbatim}
\verb|RX:| and \verb|TX:| lines summarize receiver and transmitter
statistics. They contain:
\begin{itemize}
\item \verb|bytes| --- total number of bytes received or transmitted
on the interface. This number wraps, when maximal length of data type,
natural for the architecture, is exceeded, so that continuous monitoring requires
an user level daemon snapping it periodically.
\item \verb|packets| --- total number of packets received or transmitted
on the interface.
\item \verb|errors| --- total number of receiver or transmitter errors.
\item \verb|dropped| --- total number of packets dropped because of lack
of resources.
\item \verb|overrun| --- total number of receiver overruns resulting
in packet drops. As rule, if the interface is overrun, it means
serious problem in the kernel or that your machine is too slow
for this interface.
\item \verb|mcast| --- total number of received multicast packets. This option
is supported only by several devices.
\item \verb|carrier| --- total number of link media failures f.e.\ because
of lost carrier.
\item \verb|collsns| --- total number of collision events
on Ethernet-like media. This number may have different sense on another
link types.
\item \verb|compressed| --- total number of compressed packets. It is
available only for links using VJ header compression.
\end{itemize}


If the option \verb|-s| is entered twice or more,
\verb|ip| prints more detailed statistics on receiver
and transmitter errors.

\begin{verbatim}
kuznet@alisa:~ $ ip -s -s link ls eth0
3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc cbq qlen 100
    link/ether 00:a0:cc:66:18:78 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast   
    2449949362 2786187  0       0       0       0      
    RX errors: length   crc     frame   fifo    missed
               0        0       0       0       0      
    TX: bytes  packets  errors  dropped carrier collsns 
    178558497  1783945  332     0       332     35172  
    TX errors: aborted  fifo    window  heartbeat
               0        0       0       332    
kuznet@alisa:~ $
\end{verbatim}
These error names are pure Ethernetisms, another devices
may have these fields not zero, but they may have different
interpretation.


\section{{\tt ip address} --- protocol address management.}

\paragraph{Abbreviations:} \verb|address|, \verb|addr|, \verb|a|.

\paragraph{Object:} \verb|address| is a protocol (IP or IPv6) address attached
to a network device. Each device must have at least one address
to use the corresponding protocol. It is possible to have several
different addresses attached to one device. These addresses are not
discriminated, so that the term {\em alias\/} is not quite appropriate
for them and we do not use it in this document.

The \verb|ip addr| command allows to look at addresses and their properties,
to add new addresses and to delete old ones.

\paragraph{Commands:} \verb|add|, \verb|delete|, \verb|flush| and \verb|show|
(or \verb|list|).


\subsection{{\tt ip address add} --- add new protocol address.}
\label{IP-ADDR-ADD}

\paragraph{Abbreviations:} \verb|add|, \verb|a|.

\paragraph{Arguments:}

\begin{itemize}
\item \verb|dev NAME|

\noindent--- name of the device to add the address.

\item \verb|local ADDRESS| (default)

--- address of the interface. The format of the address depends
on the protocol, it is dotted quad for IP and sequence of hexadecimal halfwords
separated by colons for IPv6. The \verb|ADDRESS| may be followed by
slash and decimal number, which encodes network prefix length.


\item \verb|peer ADDRESS|

--- address of remote endpoint for pointopoint interfaces.
Again, the \verb|ADDRESS| may be followed by slash and decimal number,
encoding network prefix length. If a peer address is specified,
local address {\em cannot\/} have prefix length, network prefix is associated
with peer rather than with local address.


\item \verb|broadcast ADDRESS|

--- broadcast address on the interface.

It is possible to use special symbols \verb|'+'| and \verb|'-'|
instead of broadcast address. In this case broadcast address
is derived by setting/resetting host bits of interface prefix.

\vskip 1mm
\begin{NB}
Unlike \verb|ifconfig| \verb|ip| utility {\em does not\/} set any broadcast
address, if it was not requested explicitly.
\end{NB}


\item \verb|label NAME|

--- Each address may be tagged with label string.
In order to preserve compatibility with Linux-2.0 net aliases,
this string must coincide with the name of the device or must be prefixed
with device name followed by colon.


\item \verb|scope SCOPE_VALUE|

--- scope of the area, where this address is valid.
The available scopes are listed in file \verb|/etc/iproute2/rt_scopes|.
Predefined scope values are:

 \begin{itemize}
	\item \verb|global| --- the address is globally valid.
	\item \verb|site| --- (IPv6 only) the address is site local,
	i.e.\ it is valid inside this site.
	\item \verb|link| --- the address is link local, i.e.\ 
	it is valid only on this device.
	\item \verb|host| --- the address is valid only inside this host.
 \end{itemize}

Appendix~\ref{ADDR-SEL} (p.\pageref{ADDR-SEL} of this document)
contains more details on address scopes.

\end{itemize}

\paragraph{Examples:}
\begin{itemize}
\item \verb|ip addr add 127.0.0.1/8 dev lo brd + scope host|

--- add usual loopback address to loopback device.

\item \verb|ip addr add 10.0.0.1/24 brd + dev eth0 label eth0:Alias|

--- add address 10.0.0.1 with prefix length 24 (i.e.\ netmask
\verb|255.255.255.0|, standard broadcast and label \verb|eth0:Alias|
to the interface \verb|eth0|.
\end{itemize}


\subsection{{\tt ip address delete} --- delete protocol address.}

\paragraph{Abbreviations:} \verb|delete|, \verb|del|, \verb|d|.

\paragraph{Arguments:} coincide with arguments of \verb|ip addr add|.
The device name is required argument, the rest are optional.
If no arguments are given, the first address is deleted.

\paragraph{Examples:}
\begin{itemize}
\item \verb|ip addr del 127.0.0.1/8 dev lo|

--- deletes loopback address from loopback device.
It would be better not to try to repeat this experiment.

\item Disable IP on the interface \verb|eth0|:
\begin{verbatim}
  while ip -f inet addr del dev eth0; do
    : nothing
  done
\end{verbatim}
Another method to disable IP on an interface using {\tt ip addr flush}
may be found in sec.\ref{IP-ADDR-FLUSH}, p.\pageref{IP-ADDR-FLUSH}.

\end{itemize}


\subsection{{\tt ip address show} --- look at protocol addresses.}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|lst|, \verb|sh|, \verb|ls|,
\verb|l|.

\paragraph{Arguments:}

\begin{itemize}
\item \verb|dev NAME| (default)

--- name of the device.

\item \verb|scope SCOPE_VAL|

--- list only addresses with this scope.

\item \verb|to PREFIX|

--- list only addresses matching this prefix.

\item \verb|label PATTERN|

--- list only addresses with labels matching the \verb|PATTERN|.
\verb|PATTERN| is usual shell style pattern.


\item \verb|dynamic| and \verb|permanent|

--- (IPv6 only) list only addresses installed due to stateless
address configuration or list only permanent (not dynamic) addresses.

\item \verb|tentative|

--- (IPv6 only) list only addresses, which did not pass duplicate
address detection.

\item \verb|deprecated|

--- (IPv6 only) list only deprecated addresses.


\item  \verb|primary| and \verb|secondary|

--- list only primary (or secondary) addresses.

\end{itemize}


\paragraph{Output format:}

\begin{verbatim}
kuznet@alisa:~ $ ip addr ls eth0
3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc cbq qlen 100
    link/ether 00:a0:cc:66:18:78 brd ff:ff:ff:ff:ff:ff
    inet 193.233.7.90/24 brd 193.233.7.255 scope global eth0
    inet6 3ffe:2400:0:1:2a0:ccff:fe66:1878/64 scope global dynamic 
       valid_lft forever preferred_lft 604746sec
    inet6 fe80::2a0:ccff:fe66:1878/10 scope link 
kuznet@alisa:~ $ 
\end{verbatim}

The first two lines coincide with output of \verb|ip link ls|,
it is natural to interpret link layer addresses
as addresses of protocol family \verb|AF_PACKET|.

Then the list of IP and IPv6 addresses follows, accompanied with
additional address attributes: scope value (see Sec.\ref{IP-ADDR-ADD},
p.\pageref{IP-ADDR-ADD} above), flags and address label.

Address flags are set by kernel and cannot be changed
administratively. Currently the following flags are defined:

\begin{enumerate}
\item \verb|secondary|

--- the address is not used, when selecting default source address
of outgoing packets. (Cf.\ Appendix~\ref{ADDR-SEL}, p.\pageref{ADDR-SEL}.)
An IP address becomes secondary, if another address with the same
prefix bits already exists. The first address is primary,
it is leader of group of all the secondary addresses. When the leader
is deleted, all the secondaries are purged too.


\item \verb|dynamic|

--- the address was created due to stateless autoconfiguration~\cite{RFC-ADDRCONF}.
In this case output contains also information on times, when
the address is still valid. After \verb|preferred_lft| expires the address is
moved to deprecated state and after \verb|valid_lft| expires the address
is finally invalidated.

\item \verb|deprecated|

--- the address is deprecated, i.e.\ it is still valid, but cannot
be used by newly created connections.

\item \verb|tentative|

--- the address is not used because duplicate address detection~\cite{RFC-ADDRCONF}
is still not complete or failed.

\end{enumerate}


\subsection{{\tt ip address flush} --- flush protocol addresses.}
\label{IP-ADDR-FLUSH}

\paragraph{Abbreviations:} \verb|flush|, \verb|f|.

\paragraph{Description:}This commands flushes protocol addresses
selected by some criteria.

\paragraph{Arguments:} This command has the same arguments as \verb|show|.
The differences are that it does not run, when no arguments are given.

\paragraph{Warning:} This command (and another \verb|flush| commands
described below) is pretty dangerous. If you did a mistake, it will
not forgive it, but really will purge all the addresses cruelly.

\paragraph{Statistics:} With the option \verb|-statistics| the command
becomes verbose, it prints out number of deleted addresses and number
of rounds made to flush the address list. If the option is given
twice, \verb|ip addr flush| also dumps all the deleted addresses
in the format described in the previous subsection.

\paragraph{Example:} Delete all the addresses from private network
10.0.0.0/8:
\begin{verbatim}
netadm@amber:~ # ip -s -s a f to 10/8
2: dummy    inet 10.7.7.7/16 brd 10.7.255.255 scope global dummy
3: eth0    inet 10.10.7.7/16 brd 10.10.255.255 scope global eth0
4: eth1    inet 10.8.7.7/16 brd 10.8.255.255 scope global eth1

*** Round 1, deleting 3 addresses ***
*** Flush is complete after 1 round ***
netadm@amber:~ # 
\end{verbatim}
Another instructive example is disabling IP on all the Ethernets:
\begin{verbatim}
netadm@amber:~ # ip -4 addr flush label "eth*"
\end{verbatim}
And the last example shows how to flush all the IPv6 addresses,
acquired by the host from stateless address autoconfiguration
after you enabled forwarding or disabled autoconfiguration.
\begin{verbatim}
netadm@amber:~ # ip -6 addr flush dynamic
\end{verbatim}



\section{{\tt ip neighbour} --- neighbour/arp tables management.}

\paragraph{Abbreviations:} \verb|neighbour|, \verb|neighbor|, \verb|neigh|,
\verb|n|.

\paragraph{Object:} \verb|neighbour| objects establish bindings between protocol
addresses and link layer addresses of hosts sharing the same link.
Neighbour entries are organized to tables, IPv4 neighbour table
known under other name --- ARP table.

The corresponding commands allow to look at neighbour bindings
and their properties, to add new neighbour entries and to delete old ones.

\paragraph{Commands:} \verb|add|, \verb|change|, \verb|replace|,
\verb|delete|, \verb|flush| and \verb|show| (or \verb|list|).

\paragraph{See also:} Appendix~\ref{PROXY-NEIGH}, p.\pageref{PROXY-NEIGH}
describes how to manage proxy ARP/NDISC with \verb|ip| utility.


\subsection{{\tt ip neighbour add} --- add new neighbour entry\\
	{\tt ip neighbour change} --- change existing entry\\
	{\tt ip neighbour replace} --- add new or change existing one.}

\paragraph{Abbreviations:} \verb|add|, \verb|a|; \verb|change|, \verb|chg|;
\verb|replace|,	\verb|repl|.

\paragraph{Description:} These commands create new neighbour records
or update existing ones.

\paragraph{Arguments:}

\begin{itemize}
\item \verb|to ADDRESS| (default)

--- protocol address of the neighbour. It is either IPv4 or IPv6 address.

\item \verb|dev NAME|

--- the interface which this neighbour is attached to.


\item \verb|lladdr LLADDRESS|

--- link layer address of the neighbour. \verb|LLADDRESS| can be also
\verb|null|. 

\item \verb|nud NUD_STATE|

--- state of the neighbour entry. \verb|nud| is abbreviation for ``Neighbour
Unreachability Detection''. The state can take one of the following values:

\begin{enumerate}
\item \verb|permanent| --- the neighbour entry is valid forever and can be removed
only administratively.
\item \verb|noarp| --- the neighbour entry is valid, no attempts to validate
this entry will be made, but it can be removed, when its lifetime expires.
\item \verb|reachable| --- the neighbour entry is valid until reachability
timeout expires.
\item \verb|stale| --- the neighbour entry is valid, but suspicious.
This option to \verb|ip neigh| does not change neighbour state, if
it was valid and the address is not changed by this command.
\end{enumerate}

\end{itemize}

\paragraph{Examples:}
\begin{itemize}
\item \verb|ip neigh add 10.0.0.3 lladdr 0:0:0:0:0:1 dev eth0 nud perm|

--- add permanent ARP entry for neighbour 10.0.0.3 on the device \verb|eth0|.

\item \verb|ip neigh chg 10.0.0.3 dev eth0 nud reachable|

--- change its state to \verb|reachable|.
\end{itemize}


\subsection{{\tt ip neighbour delete} --- delete neighbour entry.}

\paragraph{Abbreviations:} \verb|delete|, \verb|del|, \verb|d|.

\paragraph{Description:} This command invalidates a neighbour entry.

\paragraph{Arguments:} The arguments are the same as with \verb|ip neigh add|,
only \verb|lladdr| and \verb|nud| are ignored.


\paragraph{Example:}
\begin{itemize}
\item \verb|ip neigh del 10.0.0.3 dev eth0|

--- invalidate ARP entry for neighbour 10.0.0.3 on the device \verb|eth0|.

\end{itemize}

\begin{NB}
 Deleted neighbour entry will not disappear from the tables
 immediately; if it is in use it cannot be deleted until the last
 client will release it, otherwise it will be destroyed during
 the next garbage collection.
\end{NB}


\paragraph{Warning:} attempts to delete or to change manually
a \verb|noarp| entry created by kernel may result in unpredictable behaviour.
Particularly, kernel may start to try to resolve this address even
on \verb|NOARP| interface or if the address is multicast or broadcast.


\subsection{{\tt ip neighbour show} --- list neighbour entries.}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|sh|, \verb|ls|.

\paragraph{Description:}This commands displays neighbour tables.

\paragraph{Arguments:}

\begin{itemize}

\item \verb|to ADDRESS| (default)

--- prefix selecting neighbours to list.

\item \verb|dev NAME|

--- list only neighbours attached to this device.

\item \verb|unused|

--- list only neighbours, which are not in use now.

\item \verb|nud NUD_STATE|

--- list only neighbour entries in this state. \verb|NUD_STATE| takes
values listed below or special value \verb|all|, which means all the states.
This option may occur more than once. If this option is absent, \verb|ip|
lists all the entries except for \verb|none| and \verb|noarp|.

\end{itemize}


\paragraph{Output format:}

\begin{verbatim}
kuznet@alisa:~ $ ip neigh ls
:: dev lo lladdr 00:00:00:00:00:00 nud noarp
fe80::200:cff:fe76:3f85 dev eth0 lladdr 00:00:0c:76:3f:85 router \
    nud stale
0.0.0.0 dev lo lladdr 00:00:00:00:00:00 nud noarp
193.233.7.254 dev eth0 lladdr 00:00:0c:76:3f:85 nud reachable
193.233.7.85 dev eth0 lladdr 00:e0:1e:63:39:00 nud stale
kuznet@alisa:~ $ 
\end{verbatim}

The first word of each line is protocol address of the neighbour,
then device name follows. The rest of the line describes contents of
neighbour entry identified by the pair (device, address).

\verb|lladdr| is link layer address of the neighbour.

\verb|nud| is the state of ``neighbour unreachability detection'' machine
for this entry. The detailed description of neighbour
state machine can be found in~\cite{RFC-NDISC}. The full list 
of the states with short descriptions:

\begin{enumerate}
\item\verb|none| --- state of the neighbour is void.
\item\verb|incomplete| --- the neighbour is in process of resolution.
\item\verb|reachable| --- the neighbour is valid and apparently reachable.
\item\verb|stale| --- the neighbour is valid, but probably it is already
unreachable, so that kernel will try to check it at the first transmission.
\item\verb|delay| --- a packet has been sent to the stale neighbour, kernel waits
for confirmation.
\item\verb|probe| --- delay timer expired, but no confirmation was received.
Kernel has started to probe neighbour with ARP/NDISC messages.
\item\verb|failed| --- resolution has failed.
\item\verb|noarp| --- the neighbour is valid, no attempts to check the entry
will be made.
\item\verb|permanent| --- it is \verb|noarp| entry, but only administrator
may remove the entry from neighbour table.
\end{enumerate}

Link layer address is valid in all the states except for \verb|none|,
\verb|failed| and \verb|incomplete|.

IPv6 neighbours can be marked with additional flag \verb|router|,
which means that the neighbour introduced itself as IPv6 router~\cite{RFC-NDISC}.

\paragraph{Statistics:} Option \verb|-statistics| allows to look at some usage
statistics, f.e.\

\begin{verbatim}
kuznet@alisa:~ $ ip -s n ls 193.233.7.254
193.233.7.254 dev eth0 lladdr 00:00:0c:76:3f:85 ref 5 used 12/13/20 \
    nud reachable
kuznet@alisa:~ $ 
\end{verbatim}

Here \verb|ref| is number of users of this entry,
and \verb|used| is triplet of time intervals in seconds
separated by slashes. In this case they show that:

\begin{enumerate}
\item The entry was used 12 seconds ago.
\item The entry was confirmed 13 seconds ago.
\item The entry was updated 20 seconds ago.
\end{enumerate}

\subsection{{\tt ip neighbour flush} --- flush neighbour entries.}

\paragraph{Abbreviations:} \verb|flush|, \verb|f|.

\paragraph{Description:}This commands flushes neighbour tables selecting
entries to flush by some criteria.

\paragraph{Arguments:} This command has the same arguments as \verb|show|.
The differences are that it does not run, when no arguments are given,
and that default neighbour states to be flushed do not include
\verb|permanent| and \verb|noarp|.


\paragraph{Statistics:} With the option \verb|-statistics| the command
becomes verbose, it prints out number of deleted neighbours and number
of rounds made to flush neighbour table. If the option is given
twice, \verb|ip neigh flush| also dumps all the deleted neighbours
in the format described in the previous subsection.

\paragraph{Example:}
\begin{verbatim}
netadm@alisa:~ # ip -s -s n f 193.233.7.254
193.233.7.254 dev eth0 lladdr 00:00:0c:76:3f:85 ref 5 used 12/13/20 \
    nud reachable

*** Round 1, deleting 1 entries ***
*** Flush is complete after 1 round ***
netadm@alisa:~ # 
\end{verbatim}


\section{{\tt ip route} --- routing table management.}
\label{IP-ROUTE}

\paragraph{Abbreviations:} \verb|route|, \verb|ro|, \verb|r|.

\paragraph{Object:} route entries in kernel routing tables, which keep
the information about paths to another networked nodes.

Each route entry has a {\em key\/}, consisting of {\em prefix\/},
i.e.\ pair of network address and length of its mask, and,
optionally, TOS value. IP packet matches to the route, if the highest
bits of its destination address are equal to route prefix at least
up to prefix length and if TOS of the route is zero or equal to
TOS of the packet.
 
If several routes match to the packet, the following pruning rules
are used to select the best one (see~\cite{RFC1812}):
\begin{enumerate}
\item The longest matching prefix is selected, all shorter ones
are dropped.

\item If TOS of some route with the longest prefix is equal to TOS
of the packet, the routes with different TOS are dropped.

If no exact TOS match was found and routes with TOS=0 exist,
the rest of routes are pruned.

Otherwise, route lookup fails.

\item If several routes remained after previous steps, then
routes with the best preference value are selected.

\item If we still have several routes, then the {\em first\/} of them
is selected.

\begin{NB}
 Note the ambiguity of the last bullet. Unfortunately, Linux
 historically allows such bizarre situation. The sense of the
word ``the first'' depends on order of route additions and it is practically
impossible to maintain bundle of such routes in this order.
\end{NB}

For simplicity we will limit ourselves to the case, when such situation
is impossible and routes are uniquely identified by triplet
\{prefix, tos, preference\}. Actually, it is impossible to create
not-unique routes with \verb|ip| commands described in this section.

One useful exception to this rule is default route on non-forwarding
hosts. It is ``officially'' allowed to have several fallback routes,
when several routers are present on directly connected networks.
In this case, Linux-2.2 makes ``dead gateway detection''~\cite{RFC1122}
controlled by neighbour unreachability detection and by advices
from transport protocols to select working router, so that order
of the routes is not essential. However, in this case it is not recommended
to fiddle with default routes manually, but to use Router Discovery
protocol (see Appendix~\ref{EXAMPLE-SETUP}, p.\pageref{EXAMPLE-SETUP})
instead. Actually, Linux-2.2 IPv6 even does not give user level applications
any access to default routes.
\end{enumerate}

Certainly, the steps above are not performed exactly
in this sequence. Instead, routing table in the kernel is kept
in some data structure, which allows to achieve the final result
with minimal cost. However, not depending on particular
routing algorithm implemented in the kernel, we can summarize
the statements above as: route is identified by triplet
\{prefix, tos, preference\}, this {\em key\/} allows to locate
the route in routing table.

\paragraph{Route attributes:} Each route key refers to a routing
information record, containing
data required to deliver IP packets (f.e.\ output device and
next hop router) and some optional attributes (f.e. path MTU or
source address, preferred when communicating to this destination).
These attributes are described in the following subsection.

\paragraph{Route types:} \label{IP-ROUTE-TYPES}
It is important that the set
of required and optional attributes depend on route {\em type\/}.
The most important route type
is \verb|unicast| route; it describes real paths to another hosts.
As rule, common routing tables contain only such routes. However,
there exist another types of routes with different semantics. The
full list of types understood by Linux-2.2 is:
\begin{itemize}
\item \verb|unicast| --- the route entry describes real paths to the
destinations covered by route prefix.
\item \verb|unreachable| --- these destinations are unreachable; packets
are discarded and ICMP message {\em host unreachable\/} is generated.
The local senders get error \verb|EHOSTUNREACH|.
\item \verb|blackhole| --- these destinations are unreachable; packets
are discarded silently. The local senders get error \verb|EINVAL|.
\item \verb|prohibit| --- these destinations are unreachable; packets
are discarded and ICMP message {\em communication administratively
prohibited\/} is generated. The local senders get error \verb|EACCES|.
\item \verb|local| --- the destinations are assigned to this
host, the packets are looped back and delivered locally.
\item \verb|broadcast| --- the destinations are broadcast addresses,
the packets are sent as link broadcasts.
\item \verb|throw| --- special control route used together with policy
rules (see sec.\ref{IP-RULE}, p.\pageref{IP-RULE}). If such route is selected, lookup
in this table is terminated pretending that no route was found.
Without policy routing it is equivalent to absence of the route in routing
table, the packets are dropped and ICMP message {\em net unreachable\/}
is generated. The local senders get error \verb|ENETUNREACH|.
\item \verb|nat| --- special NAT route. Destinations covered by the prefix
are considered as dummy (or external) addresses, which require translation
to real (or internal) ones before forwarding. The addresses to translate to
are selected with the attribute \verb|via|. More about NAT is
in Appendix~\ref{ROUTE-NAT}, p.\pageref{ROUTE-NAT}.
\item \verb|anycast| --- ({\em not implemented\/}) the destinations are
{\em anycast\/} addresses, assigned to this host. They are mainly equivalent
to \verb|local| with one difference: such addresses are invalid to be used
as source address of any packet.
\item \verb|multicast| --- special type, used for multicast routing.
It does not present in normal routing tables.
\end{itemize}

\paragraph{Route tables:} Linux-2.2 can pack routes to several routing
tables identified by number in the range from 1 to 255 or by
name from the file \verb|/etc/iproute2/rt_tables|. By default all normal
routes are inserted to the table \verb|main| (ID 254) and kernel uses
only this table, when calculating routes.

Actually, one another table always exists, which is invisible but
even more important. It is \verb|local| table (ID 255). This table
consists of routes for local and broadcast addresses. Kernel maintains
this table automatically and usually administrator need not to modify it
and even to look at it.

The multiple routing tables enter to the game, when {\em policy routing\/}
is used, see sec.\ref{IP-RULE}, p.\pageref{IP-RULE}.
In this case table identifier becomes
effectively one more parameter, which should be added to triplet
\{prefix, tos, preference\} to identify route uniquely.


\subsection{{\tt ip route add} --- add new route\\
	{\tt ip route change} --- change route\\
	{\tt ip route replace} --- change route or add new one.}
\label{IP-ROUTE-ADD}

\paragraph{Abbreviations:} \verb|add|, \verb|a|; \verb|change|, \verb|chg|;
	\verb|replace|, \verb|repl|.


\paragraph{Arguments:}
\begin{itemize}
\item \verb|to PREFIX| or \verb|to TYPE PREFIX| (default)

--- destination prefix of the route. If \verb|TYPE| is omitted,
\verb|ip| assumes type \verb|unicast|. Another values of \verb|TYPE|
are listed above. \verb|PREFIX| is IP or IPv6 address optionally followed
by slash and prefix length. If the length of the prefix is missing,
\verb|ip| assumes full-length host route. Also there is one special
\verb|PREFIX| --- \verb|default| --- which is equivalent to IP \verb|0/0| or
to IPv6 \verb|::/0|.

\item \verb|tos TOS| or \verb|dsfield TOS|

--- Type Of Service (TOS) key. This key has no mask associated and
the longest match is understood as: first, compare TOS
of the route and of the packet, if they are not equal, then the packet
still may match to a route with zero TOS. \verb|TOS| is either 8bit hexadecimal
number or an identifier from {\tt /etc/iproute2/rt\_dsfield}.


\item \verb|metric NUMBER| or \verb|preference NUMBER|

--- preference value of the route. \verb|NUMBER| is an arbitrary 32bit number.

\item \verb|table TABLEID|

--- table to add this route.
\verb|TABLEID| may be a number or a string from the file
\verb|/etc/iproute2/rt_tables|. If this parameter is omitted,
\verb|ip| assumes table \verb|main|, with exception of
\verb|local|, \verb|broadcast| and \verb|nat| routes, which are
put to table \verb|local| by default.

\item \verb|dev NAME|

--- the output device name.

\item \verb|via ADDRESS|

--- the address of nexthop router. Actually, the sense of this field depends
on route type. For normal \verb|unicast| routes it is either true nexthop
router or, if it is direct route installed in BSD compatibility mode,
it can be a local address of the interface.
For NAT routes it is the first address of block of translated IP destinations.

\item \verb|src ADDRESS|

--- the source address to prefer, when sending to the destinations
covered by route prefix.

\item \verb|realm REALMID|

--- the realm which this route is assigned to. 
\verb|REALMID| may be a number or a string from the file
\verb|/etc/iproute2/rt_realms|. Sec.\ref{RT-REALMS} (p.\pageref{RT-REALMS})
contains more information on realms.

\item \verb|mtu MTU| or \verb|mtu lock MTU|

--- the MTU along the path to destination. If modifier \verb|lock| is
not used, MTU may be updated by the kernel due to Path MTU Discovery.
If the modifier \verb|lock| is used, no path MTU discovery will be tried,
all the packets will be sent without DF bit in IPv4 case
or fragmented to MTU for IPv6.

\item \verb|window NUMBER|

--- the maximal window for TCP to advertise to these destinations,
measured in bytes. It limits maximal data bursts, which our TCP
peers are allowed to send to us.

\item \verb|rtt NUMBER|

--- the initial RTT (``Round Trip Time'') estimate.


\item \verb|rttvar NUMBER|

--- \threeonly initial RTT variance estimate.


\item \verb|ssthresh NUMBER|

--- \threeonly estimate for inititial slow start threshould.


\item \verb|cwnd NUMBER|

--- \threeonly clamp for congestion window. It is ignored, if \verb|lock|
    flag is not used.


\item \verb|advmss NUMBER|

--- \threeonly MSS (``Maximal Segment Size'') to advertise to these
    destinations when establishing TCP connections. If it is not given,
    Linux use default value calculated from first hop device MTU.

\begin{NB}
  If path to these destination is asymmetric, this guess may be wrong.
\end{NB}
    


\item \verb|nexthop NEXTHOP|

--- nexthop of multipath route. \verb|NEXTHOP| is complex value
with its own syntax similar to one of top level argument list:
\begin{itemize}
\item \verb|via ADDRESS| is nexthop router.
\item \verb|dev NAME| is output device.
\item \verb|weight NUMBER| is weight of this element of multipath
route reflecting its relative bandwidth or quality.
\end{itemize}

\item \verb|scope SCOPE_VAL|

--- scope of the destinations covered by the route prefix.
\verb|SCOPE_VAL| may be a number or a string from the file
\verb|/etc/iproute2/rt_scopes|.
If this parameter is omitted,
\verb|ip| assumes scope \verb|global| for all gatewayed \verb|unicast|
routes, scope \verb|link| for direct unicast routes and broadcasts
and scope \verb|host| for \verb|local| routes.

\item \verb|protocol RTPROTO|

--- routing protocol identifier of this route.
\verb|RTPROTO| may be a number or a string from the file
\verb|/etc/iproute2/rt_protos|. If the routing protocol ID is
not given, \verb|ip| assumes protocol \verb|boot| (i.e.\
it considers the route is added by someone, who does not
understand what he does). Several protocol values have a fixed interpretation.
Namely:
\begin{itemize}
\item \verb|redirect| --- route was installed due to ICMP redirect.
\item \verb|kernel| --- route was installed by the kernel during
autoconfiguration.
\item \verb|boot| --- route was installed during bootup sequence.
If a routing daemon will start, it will purge all of them.
\item \verb|static| --- route was installed by administrator
to override dynamic routing. Routing daemon will respect them
and, probably, even to advertise to its peers.
\item \verb|ra| --- route was installed by Router Discovery protocol.
\end{itemize}
The rest of values are not reserved and administrator is free
to assign (or not to assign) protocol tags. At least, routing
daemons should take care of setting some unique protocol values,
f.e.\ as they are assigned in \verb|rtnetlink.h| or in \verb|rt_protos|
database.


\item \verb|onlink|

--- pretend that the nexthop is directly attached to this link,
even if it does match any interface prefix. One application of this
option may be found in~\cite{IP-TUNNELS}.

\item \verb|equalize|

--- allow packet by packet randomization on multipath routes.
Without this modifier route will be frozen to one selected
nexthop, so that load splitting will occur only on per-flow base.
\verb|equalize| works only if the kernel is patched.


\end{itemize}


\begin{NB}
  Actually there exist more commands: \verb|prepend| does the same
  thing as classic \verb|route add|, i.e.\ adds route, even if another
  route to the same destination exists. Its opposite case is \verb|append|,
  which adds the route to the end of the list. Avoid to use these
  features.
\end{NB}
\begin{NB}
  More sad news, IPv6 understands only \verb|append| command correctly,
  all the rest of the set translating to \verb|append|. Certainly,
  it will change in the future.
\end{NB}

\paragraph{Examples:}
\begin{itemize}
\item add plain route to network 10.0.0/24 via gateway 193.233.7.65
\begin{verbatim}
  ip route add 10.0.0/24 via 193.233.7.65
\end{verbatim}
\item change it to direct route via device \verb|dummy|
\begin{verbatim}
  ip ro chg 10.0.0/24 dev dummy
\end{verbatim}
\item add default multipath route splitting load between \verb|ppp0|
and \verb|ppp1|
\begin{verbatim}
  ip route add default scope global nexthop dev ppp0 \
                                    nexthop dev ppp1
\end{verbatim}
Note scope value, it is not necessary, but it prompts kernel
that this route is gatewayed rather than direct. Actually, if you
know addresses of remote endpoints it would be better to use parameter
\verb|via|.
\item announce that address 192.203.80.144 is not real one, but
should be translated to 193.233.7.83 before forwarding
\begin{verbatim}
  ip route add nat 192.203.80.142 via 193.233.7.83
\end{verbatim}
Backward translation is setup with policy rules described
in the following section (sec.\ref{IP-RULE}, p.\pageref{IP-RULE}).
\end{itemize}

\subsection{{\tt ip route delete} --- delete route.}

\paragraph{Abbreviations:} \verb|delete|, \verb|del|, \verb|d|.

\paragraph{Arguments:} \verb|ip route del| has the same arguments as
\verb|ip route add|, but their semantics is a bit different.

Key values (\verb|to|, \verb|tos|, \verb|preference| and \verb|table|)
select route to delete. If optional attributes are present, \verb|ip|
verifies that they coincide with attributes of the route to delete.
If no route with given key and attributes was found \verb|ip route del|
fails.
\begin{NB}
Linux-2.0 had option to delete route selected only by prefix address
ignoring its length (i.e.\ netmask). This option does not exist more,
because it was ambiguous. However, look at {\tt ip route flush}
(sec.\ref{IP-ROUTE-FLUSH}, p.\pageref{IP-ROUTE-FLUSH}), it
provides similar and even more rich functionality.
\end{NB}

\paragraph{Example:}
\begin{itemize}
\item delete multipath route created by command in previous subsection
\begin{verbatim}
  ip route del default scope global nexthop dev ppp0 \
                                    nexthop dev ppp1
\end{verbatim}
\end{itemize}



\subsection{{\tt ip route show} --- list routes.}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|sh|, \verb|ls|, \verb|l|.

\paragraph{Description:} the command allows to view routing tables
contents or to look at the route(s) selected by some criteria.


\paragraph{Arguments:}
\begin{itemize}
\item \verb|to SELECTOR| (default)

--- select routes only from given range of destinations. \verb|SELECTOR|
consists of optional modifier (\verb|root|, \verb|match| or \verb|exact|)
and a prefix. \verb|root PREFIX| selects routes with prefixes not shorter
than \verb|PREFIX|. F.e.\ \verb|root 0/0| selects all the routing table.
\verb|match PREFIX| selects routes with prefixes not longer than
\verb|PREFIX|. F.e.\ \verb|match 10.0/16| selects \verb|10.0/16|,
\verb|10/8| and \verb|0/0|, but it does not select \verb|10.1/16| and
\verb|10.0.0/24|. And \verb|exact PREFIX| (or just \verb|PREFIX|)
selects routes exactly with this prefix. If neither of these options
are present, \verb|ip| assumes \verb|root 0/0| i.e.\ it lists all the table.


\item \verb|tos TOS| or \verb|dsfield TOS|

 --- Select only routes with given TOS.


\item \verb|table TABLEID|

 --- Show routes from this table(s). Default setting is to show
\verb|table| \verb|main|. \verb|TABLEID| may be either ID of a real table
or one of special values:
  \begin{itemize}
  \item \verb|all| --- list all the tables.
  \item \verb|cache| --- dump routing cache.
  \end{itemize}
\begin{NB}
  IPv6 has single table, however splitting to \verb|main|, \verb|local|
  and \verb|cache| is emulated by \verb|ip| utility.
\end{NB}

\item \verb|cloned| or \verb|cached|

--- list cloned routes i.e.\ routes, which were dynamically forked of
another routes because some route attribute (f.e.\ MTU) was updated.
Actually, it is equivalent to \verb|table cache|.

\item \verb|from SELECTOR|

--- the same syntax as for \verb|to|, but it bounds source address range
rather than destinations. Note, that \verb|from| option works only with
cloned routes.

\item \verb|protocol RTPROTO|

--- list only routes of this protocol.


\item \verb|scope SCOPE_VAL|

--- list only routes with this scope.

\item \verb|type TYPE|

--- list only routes of this type.

\item \verb|dev NAME|

--- list only routes going via this device.

\item \verb|via PREFIX|

--- list only routes going via selected by \verb|PREFIX| nexthop routers.

\item \verb|src PREFIX|

--- list only routes with preferred source addresses selected
by \verb|PREFIX|.

\item \verb|realm REALMID| or \verb|realms FROMREALM/TOREALM|

--- list only routes with these realms.

\end{itemize}

\paragraph{Examples:} Let us count routes of protocol \verb|gated/bgp|
on a router:
\begin{verbatim}
kuznet@amber:~ $ ip ro ls proto gated/bgp | wc
   1413    9891    79010
kuznet@amber:~ $
\end{verbatim}
To count size of routing cache we have to use option \verb|-o|,
because cached attributes can take more than one line of the output:
\begin{verbatim}
kuznet@amber:~ $ ip -o ro ls cloned | wc
   159    2543    18707
kuznet@amber:~ $
\end{verbatim}


\paragraph{Output format:} The output of this command consists
of per route records separated by line feeds.
However, some records may consist
of more than one line: particularly, it is the case when the route
is cloned or you requested additional statistics. If the option
\verb|-o| was given, then line feeds separating lines inside
records are replaced with backslash sign.

The output has the same syntax as arguments given to {\tt ip route add},
so that it can be understood easily. F.e.\
\begin{verbatim}
kuznet@amber:~ $ ip ro ls 193.233.7/24
193.233.7.0/24 dev eth0  proto gated/conn  scope link \
    src 193.233.7.65 realms inr.ac 
kuznet@amber:~ $
\end{verbatim}

If you list cloned entries the output contains another attributes, which
are evaluated during route calculation and updated during route
lifetime. The example of the output is:
\begin{verbatim}
kuznet@amber:~ $ ip ro ls 193.233.7.82 tab cache
193.233.7.82 from 193.233.7.82 dev eth0  src 193.233.7.65 \
  realms inr.ac/inr.ac 
    cache <src-direct,redirect>  mtu 1500 rtt 300 iif eth0
193.233.7.82 dev eth0  src 193.233.7.65 realms inr.ac 
    cache  mtu 1500 rtt 300
kuznet@amber:~ $
\end{verbatim}
\begin{NB}
  \label{NB-strange-route}
  The route looks a bit strange, does not it? Did you notice, that
  it is path from 193.233.7.82 back to 193.233.82? Well, you will
  see in the section on \verb|ip route get| (p.\pageref{NB-nature-of-strangeness})
  how it appeared.
\end{NB}
The second line, started with the word \verb|cache|, shows
additional attributes, which normal routes do not possess.
In angle brackets cached flags are summarized:
\begin{itemize}
\item \verb|local| --- packets are delivered locally.
It stands for loopback unicast routes, for broadcast routes
and for multicast routes, if this host is member of the corresponding
group.

\item \verb|reject| --- the path is bad. Any attempt to use it results
in error. See attribute \verb|error| below (p.\pageref{IP-ROUTE-GET-error}).

\item \verb|mc| --- the destination is multicast.

\item \verb|brd| --- the destination is broadcast.

\item \verb|src-direct| --- the source is on a directly connected
interface.

\item \verb|redirected| --- the route was created by an ICMP Redirect.

\item \verb|redirect| --- packets going via this route will 
trigger ICMP redirect.

\item \verb|fastroute| --- route is eligible to be used for fastroute.

\item \verb|equalize| --- make packet by packet randomization
along this path.

\item \verb|dst-nat| --- destination address requires translation.

\item \verb|src-nat| --- source address requires translation.

\item \verb|masq| --- source address requires masquerading.

\item \verb|notify| --- ({\em not implemented}) change/deletion
of this route will trigger RTNETLINK notification.
\end{itemize}

Then some optional attributes follow:
\begin{itemize}
\item \verb|error| --- on \verb|reject| routes it is error code,
returned to local senders, when they try to use this route.
These error codes are translated to ICMP error codes, sent to remote
senders, according to the rules described above in the subsection
devoted to route types (p.\pageref{IP-ROUTE-TYPES}).
\label{IP-ROUTE-GET-error}

\item \verb|expires| --- this entry will expire after this timeout.

\item \verb|iif| --- the packets for this path are expected to arrive
on this interface.
\end{itemize}

\paragraph{Statistics:} With the option \verb|-statistics| more
information about this route is shown:
\begin{itemize}
\item \verb|users| --- number of users of this entry.
\item \verb|age| --- shows when this route was used last time.
\item \verb|used| --- number of lookups of this route since its creation.
\end{itemize}


\subsection{{\tt ip route flush} --- flush routing tables.}
\label{IP-ROUTE-FLUSH}

\paragraph{Abbreviations:} \verb|flush|, \verb|f|.

\paragraph{Description:} the command allows to flush routes selected
by some criteria.

\paragraph{Arguments:} the arguments have the same syntax and semantics
as the arguments of \verb|ip route show|, but routing tables are not
listed but purged. The only different thing is default action: if \verb|show|
dumps all the IP main routing table, \verb|flush| prints helper page.
The reason of this difference does not require any explanation, does it?


\paragraph{Statistics:} With the option \verb|-statistics| the commands
becomes verbose, it prints out number of deleted routes and number
of rounds made to flush routing table. If the option is given
twice, \verb|ip route flush| also dumps all the deleted routes
in the format described in the previous subsection.

\paragraph{Examples:} The first example flushes all the
gatewayed routes from main table (f.e.\ after routing daemon crash).
\begin{verbatim}
netadm@amber:~ # ip -4 ro flush scope global type unicast
\end{verbatim}
This option deserved to be put into scriptlet \verb|routef|.
\begin{NB}
This option was described in \verb|route(8)| man page borrowed
from BSD, but never was implemented in Linux.
\end{NB}

The second example is flushing all IPv6 cloned routes:
\begin{verbatim}
netadm@amber:~ # ip -6 -s -s ro flush cache
3ffe:2400::220:afff:fef4:c5d1 via 3ffe:2400::220:afff:fef4:c5d1 \
  dev eth0  metric 0 
    cache  used 2 age 12sec mtu 1500 rtt 300
3ffe:2400::280:adff:feb7:8034 via 3ffe:2400::280:adff:feb7:8034 \
  dev eth0  metric 0 
    cache  used 2 age 15sec mtu 1500 rtt 300
3ffe:2400::280:c8ff:fe59:5bcc via 3ffe:2400::280:c8ff:fe59:5bcc \
  dev eth0  metric 0 
    cache  users 1 used 1 age 23sec mtu 1500 rtt 300
3ffe:2400:0:1:2a0:ccff:fe66:1878 via 3ffe:2400:0:1:2a0:ccff:fe66:1878 \
  dev eth1  metric 0 
    cache  used 2 age 20sec mtu 1500 rtt 300
3ffe:2400:0:1:a00:20ff:fe71:fb30 via 3ffe:2400:0:1:a00:20ff:fe71:fb30 \
  dev eth1  metric 0 
    cache  used 2 age 33sec mtu 1500 rtt 300
ff02::1 via ff02::1 dev eth1  metric 0 
    cache  users 1 used 1 age 45sec mtu 1500 rtt 300

*** Round 1, deleting 6 entries ***
*** Flush is complete after 1 round ***
netadm@amber:~ # ip -6 -s -s ro flush cache
Nothing to flush.
netadm@amber:~ #
\end{verbatim}

The third example is flushing BGP routing tables after \verb|gated|
death.
\begin{verbatim}
netadm@amber:~ # ip ro ls proto gated/bgp | wc
   1408    9856    78730
netadm@amber:~ # ip -s ro f proto gated/bgp

*** Round 1, deleting 1408 entries ***
*** Flush is complete after 1 round ***
netadm@amber:~ # ip ro f proto gated/bgp
Nothing to flush.
netadm@amber:~ # ip ro ls proto gated/bgp
netadm@amber:~ #
\end{verbatim}


\subsection{{\tt ip route get} --- get single route.}
\label{IP-ROUTE-GET}

\paragraph{Abbreviations:} \verb|get|, \verb|g|.

\paragraph{Description:} the command gets single route to a destination
and prints its contents exactly as kernel sees it.

\paragraph{Arguments:} 
\begin{itemize}
\item \verb|to ADDRESS| (default)

--- destination address.

\item \verb|from ADDRESS|

--- source address.

\item \verb|tos TOS| or \verb|dsfield TOS|

--- Type Of Service.

\item \verb|iif NAME|

--- device, which this packet is expected to arrive from.

\item \verb|oif NAME|

--- enforce output device, which this packet will be routed out.

\item \verb|connected|

--- if no source address (option \verb|from|) was given, relookup
route with source set to preferred address, received from the first lookup.
If policy routing is used, it may be different route.

\end{itemize}

Note that this operation is not equivalent to \verb|ip route show|.
\verb|show| shows existing routes, \verb|get| resolves them and
creates new clones, if it is necessary. Essentially, \verb|get|
is equivalent to sending a packet along this path.
If argument \verb|iif| is not given the kernel creates route
to output packets towards requested destination.
This is equivalent to pinging this destination
with subsequent {\tt ip route ls cache}, however no packets are
sent really. With the argument \verb|iif| kernel pretends
that a packet arrived from this interface and searches for
path to forward the packet.

\paragraph{Output format:} This command outputs routes in the same
format as \verb|ip route ls|.

\paragraph{Examples:} 
\begin{itemize}
\item Find route to output packets to 193.233.7.82:
\begin{verbatim}
kuznet@amber:~ $ ip route get 193.233.7.82
193.233.7.82 dev eth0  src 193.233.7.65 realms inr.ac
    cache  mtu 1500 rtt 300
kuznet@amber:~ $
\end{verbatim}

\item Find route to forward packets arriving on \verb|eth0|
from 193.233.7.82 and destined to 193.233.7.82:
\begin{verbatim}
kuznet@amber:~ $ ip r g 193.233.7.82 from 193.233.7.82 iif eth0
193.233.7.82 from 193.233.7.82 dev eth0  src 193.233.7.65 \
  realms inr.ac/inr.ac 
    cache <src-direct,redirect>  mtu 1500 rtt 300 iif eth0
kuznet@amber:~ $
\end{verbatim}
\begin{NB}
  \label{NB-nature-of-strangeness}
  It is this operation that created funny route to 193.233.7.82
  looped back to 193.233.7.82 (cf.\ NB on~p.\pageref{NB-strange-route}).
  Note \verb|redirect| flag on it.
\end{NB}

\item Find multicast route for packets arriving on \verb|eth0|
from host 193.233.7.82 and destined to multicast group 224.2.127.254
(it is supposed, that a multicast routing daemon is running;
in this case it is \verb|pimd|)
\begin{verbatim}
kuznet@amber:~ $ ip r g 224.2.127.254 from 193.233.7.82 iif eth0
multicast 224.2.127.254 from 193.233.7.82 dev lo  \
  src 193.233.7.65 realms inr.ac/cosmos 
    cache <mc> iif eth0 Oifs: eth1 pimreg
kuznet@amber:~ $
\end{verbatim}
This route differs of ones seen before. It contains ``normal'' part
and ``multicast'' part. Normal part is used to deliver (or not to
deliver the packet) to local IP listeners. In this case the router
is not member
of this group, so that route has no \verb|local| flag and only
forwards packets. The output device for such entries is always loopback.
Multicast part consists of additional \verb|Oifs:| list showing
output interfaces.
\end{itemize}


It is time for more complicated example. Let us add invalid
gatewayed route for a destination, which really is directly connected:
\begin{verbatim}
netadm@alisa:~ # ip route add 193.233.7.98 via 193.233.7.254
netadm@alisa:~ # ip route get 193.233.7.98
193.233.7.98 via 193.233.7.254 dev eth0  src 193.233.7.90
    cache  mtu 1500 rtt 3072
netadm@alisa:~ #
\end{verbatim}
and probe it with ping:
\begin{verbatim}
netadm@alisa:~ # ping -n 193.233.7.98
PING 193.233.7.98 (193.233.7.98) from 193.233.7.90 : 56 data bytes
From 193.233.7.254: Redirect Host(New nexthop: 193.233.7.98)
64 bytes from 193.233.7.98: icmp_seq=0 ttl=255 time=3.5 ms
From 193.233.7.254: Redirect Host(New nexthop: 193.233.7.98)
64 bytes from 193.233.7.98: icmp_seq=1 ttl=255 time=2.2 ms
64 bytes from 193.233.7.98: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 193.233.7.98: icmp_seq=3 ttl=255 time=0.4 ms
64 bytes from 193.233.7.98: icmp_seq=4 ttl=255 time=0.4 ms
^C
--- 193.233.7.98 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.4/1.3/3.5 ms
netadm@alisa:~ #
\end{verbatim}
What did occur? Router 193.233.7.254 understood that we have much
better path to the destination and sent to us ICMP redirect message.
We may retry \verb|ip route get| to look, what we have in routing
tables now:
\begin{verbatim}
netadm@alisa:~ # ip route get 193.233.7.98
193.233.7.98 dev eth0  src 193.233.7.90 
    cache <redirected>  mtu 1500 rtt 3072
netadm@alisa:~ #
\end{verbatim}



\section{{\tt ip rule} --- routing policy database management.}
\label{IP-RULE}

\paragraph{Abbreviations:} \verb|rule|, \verb|ru|.

\paragraph{Object:} Rules in routing policy database controlling
route selection algorithm.

Classic routing algorithms used in the Internet make routing decisions
based only on the destination address of packets (and in theory,
but not in practice, on TOS field). Seminal review of classic
routing algorithms and their modifications can be found in~\cite{RFC1812}.

In some circumstances we want to route packets differently depending not only
on the destination addresses, but also on another packet fields: source address,
IP protocol, transport protocol ports or even packet payload.
This task is called ``policy routing''.

\begin{NB}
  ``policy routing'' $\neq$ ``routing policy''.

\noindent	``policy routing'' $=$ ``cunning routing''.

\noindent	``routing policy'' $=$ ``routing tactics'' or ``routing plan''.
\end{NB}

To solve this task conventional destination based routing table, ordered
according to the longest match rule, is replaced with ``routing policy
database'' (or RPDB), which selects route
executing some set of rules. The rules may have lots of keys of different
nature and therefore they have no natural ordering, but imposed
by administrator. Linux-2.2 RPDB is linear list of rules
ordered by numeric priority value.
RPDB explicitly allows to match  a few of packet fields:

\begin{itemize}
\item packet source address.
\item packet destination address.
\item TOS.
\item incoming interface (which is packet metadata, rather than packet field).
\end{itemize}

Matching IP protocols and transport ports is also possible, but
it is indirected via \verb|ipchains|, exploiting their capability
to mark some classes of packets with \verb|fwmark|. Therefore,
\verb|fwmark| is also included to the set of keys checked by rules.

Each routing policy rule consists of a {\em selector\/} and an {\em action\/}
predicate. The RPDB is scanned in the order of increasing priority, the selector
of each rule is applied to \{source address, destination address, incoming
interface, tos, fwmark\} and, if the selector matches to the packet,
the action is performed.  The action predicate may return with success,
in this case it will give either route or failure indication
and RPDB lookup is terminated. Otherwise, RPDB program
continues on the next rule.

What is the action semantically? Natural action is to select
nexthop and output device. It is the way that is selected by
Cisco IOS~\cite{IOS}, let us call it ``match \& set''.
Linux-2.2 approach is more flexible, the action includes
lookups in destination-based routing tables and selecting
route from these tables according to classic longest match algorithm.
``match \& set'' approach is the simplest case of Linux one, it is realized
when second level routing table contains single default route.
It is time to remind, that Linux-2.2 supports multiple tables
managed with \verb|ip route| command, described in the previous section.

At startup time kernel configures default RPDB consisting of three
rules:

\begin{enumerate}
\item Priority: 0, Selector: match anything, Action: lookup routing
table \verb|local| (ID 255).
The table \verb|local| is special routing table, containing
high priority control routes for local and broadcast addresses.

Rule 0 is special, it cannot be deleted or overridden.


\item Priority: 32766, Selector: match anything, Action: lookup routing
table \verb|main| (ID 254).
The table \verb|main| is normal routing table, containing all not-policy
routes. This rule may be deleted and/or overridden with another
ones by administrator.

\item Priority: 32767, Selector: match anything, Action: lookup routing
table \verb|default| (ID 253).
The table \verb|default| is empty, it is reserved for some
post-processing, if previous default rules did not select the packet.
This rule also may be deleted.

\end{enumerate}

Do not mix routing tables and rules: rules point to routing tables,
several rules may refer to one routing table and some routing tables
may have no rules pointing to them. If administrator deletes all the rules
referring to a table, the table is not used, but it still exists
and will disappear only after all the routes contained in it are deleted.


\paragraph{Rule attributes:} Each RPDB entry has additional
attributes attached. F.e.\ each rule has pointer to some routing
table. NAT and masquerading rules have attribute to select new IP
address to translate/masquearade. Besides that, rules have some
of optional attributes, which routes have, namely \verb|realms|.
These values do not override those contained in routing tables, they
are used only if the route did not select any attributes.


\paragraph{Rule types:} RPDB may contain rules of the following
types:
\begin{itemize}
\item \verb|unicast| --- the rule prescribes to return the route found
in the routing table, referenced by the rule.
\item \verb|blackhole| --- the rule prescribes to drop packet silently.
\item \verb|unreachable| --- the rule prescribes to generate error ``Network
is unreachable''.
\item \verb|prohibit| --- the rule prescribes to generate error
``Communication is administratively prohibited''.
\item \verb|nat| --- the rule prescribes to translate source address
of the IP packet to some another value. More about NAT is
in Appendix~\ref{ROUTE-NAT}, p.\pageref{ROUTE-NAT}.
\end{itemize}


\paragraph{Commands:} \verb|add|, \verb|delete| and \verb|show|
(or \verb|list|).

\subsection{{\tt ip rule add} --- insert new rule\\
	{\tt ip rule delete} --- delete rule.}
\label{IP-RULE-ADD}

\paragraph{Abbreviations:} \verb|add|, \verb|a|; \verb|delete|, \verb|del|,
	\verb|d|.

\paragraph{Arguments:}

\begin{itemize}
\item \verb|type TYPE| (default)

--- type of this rule. The list of valid types was given in the previous
subsection.

\item \verb|from PREFIX|

--- select source prefix to match.

\item \verb|to PREFIX|

--- select destination prefix to match.

\item \verb|iif NAME|

--- select incoming device to match. If the interface is loopback,
the rule matches only packets originated by this host. It means that you
may create separate routing tables for forwarded and local packets and,
hence, completely segregate them.

\item \verb|tos TOS| or \verb|dsfield TOS|

--- select TOS value to match.

\item \verb|fwmark MARK|

--- select value of \verb|fwmark| to match.

\item \verb|priority PREFERENCE|

--- priority of this rule. Each rule should have an explicitly
set {\em unique\/} priority value.
\begin{NB}
  Really, by historical reasons \verb|ip rule add| does not require any
  priority value and allows it to be not unique.
  If user did not supplied any priority, it is selected by kernel.
  If user requested to create rule with a priority value, which
  already exists, kernel does not reject the request and adds
  new rule before all old rules of the same priority.

  It is mistake in design, not more. And it will be fixed one day,
  so that do not rely on this feature, use explicit priorities.
\end{NB}


\item \verb|table TABLEID|

--- routing table identifier to lookup, if the rule selector matches.

\item \verb|realms FROM/TO|

--- Realms to select if the rule matched and routing table lookup
succeeded. Realm \verb|TO| is used only if route did not select
any realm.

\item \verb|nat ADDRESS|

--- The base of IP address block to translate source address. 
The \verb|ADDRESS| may be either start of block of NAT addresses
(selected by NAT routes) or a local host address (or even zero).
In the last case router does not translate the packets, but masquerades them
to this address. More about NAT is in Appendix~\ref{ROUTE-NAT},
p.\pageref{ROUTE-NAT}.

\end{itemize}

\paragraph{Warning:} Changes to RPDB made with these commands
do not become active immediately. It is supposed that after
script finishes batch of updates it flushes routing cache
with \verb|ip route flush cache|.

\paragraph{Examples:}
\begin{itemize}
\item Route packets with source addresses from 192.203.80/24
according to routing table \verb|inr.ruhep|:
\begin{verbatim}
ip ru add from 192.203.80.0/24 table inr.ruhep prio 220
\end{verbatim}

\item Translate packet source 193.233.7.83 to 192.203.80.144
and route it according to table \#1 (actually, it is \verb|inr.ruhep|):
\begin{verbatim}
ip ru add from 193.233.7.83 nat 192.203.80.144 table 1 prio 320
\end{verbatim}

\item Delete unused default rule:
\begin{verbatim}
ip ru del prio 32767
\end{verbatim}

\end{itemize}



\subsection{{\tt ip rule show} --- list rules.}
\label{IP-RULE-SHOW}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|sh|, \verb|ls|, \verb|l|.


\paragraph{Arguments:} Good news, it is the only command,
which has no arguments.

\paragraph{Output format:}

\begin{verbatim}
kuznet@amber:~ $ ip ru ls
0:	from all lookup local 
200:	from 192.203.80.0/24 to 193.233.7.0/24 lookup main
210:	from 192.203.80.0/24 to 192.203.80.0/24 lookup main
220:	from 192.203.80.0/24 lookup inr.ruhep realms inr.ruhep/radio-msu
300:	from 193.233.7.83 to 193.233.7.0/24 lookup main
310:	from 193.233.7.83 to 192.203.80.0/24 lookup main
320:	from 193.233.7.83 lookup inr.ruhep map-to 192.203.80.144
32766:	from all lookup main 
kuznet@amber:~ $
\end{verbatim}

In the first position rule priority value stands followed
by colon. Then the selectors follow, each key is prefixed
by the same keyword, which was used to create the rule.

The keyword \verb|lookup| is followed by routing table identifier,
as it is recorded in the file \verb|/etc/iproute2/rt_tables|.

If the rule does NAT (f.e.\ rule \#320), it is shown by keyword
\verb|map-to| followed by start of block of addresses to map.

The sense of this example is pretty simple, the prefixes
192.203.80.0/24 and 193.233.7.0/24 form internal network, but
they are routed differently, when the packets leave it.
Besides that, the host 193.233.7.83 is translated to
another prefix to look as 192.203.80.144, when talking
to outer world.



\section{{\tt ip maddress} --- multicast addresses management.}
\label{IP-MADDR}

\paragraph{Object:} The \verb|maddress| objects are multicast addresses.

\paragraph{Commands:} \verb|add|, \verb|delete|, \verb|show| (or \verb|list|).

\subsection{{\tt ip maddress show} --- list multicast addresses.}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|sh|, \verb|ls|, \verb|l|.

\paragraph{Arguments:}

\begin{itemize}

\item \verb|dev NAME| (default)

--- the device name.

\end{itemize}

\paragraph{Output format:}

\begin{verbatim}
kuznet@alisa:~ $ ip maddr ls dummy
2:  dummy
    link  33:33:00:00:00:01
    link  01:00:5e:00:00:01
    inet  224.0.0.1 users 2
    inet6 ff02::1
kuznet@alisa:~ $ 
\end{verbatim}

The first line of the output shows interface index and its name.
Then multicast address list follows, each line starts by
protocol identifier. The word \verb|link| denotes link layer
multicast addresses.

If a multicast address has more than one user, the number
of users is shown after keyword \verb|users|.

One additional feature being not present in the example above
is flag \verb|static|, which means that this address was joined
with \verb|ip maddr add|, see the following subsection.



\subsection{{\tt ip maddress add} --- add multicast address\\
	    {\tt ip maddress delete} --- delete multicast address.}

\paragraph{Abbreviations:} \verb|add|, \verb|a|; \verb|delete|, \verb|del|, \verb|d|.

\paragraph{Description:} these commands allow to attach/detach
a static link layer multicast address to listen on the interface.
Note that it is impossible to join protocol multicast groups
statically, this command manages only link layer addresses.


\paragraph{Arguments:}

\begin{itemize}
\item \verb|address LLADDRESS| (default)

--- link layer multicast address.

\item \verb|dev NAME|

--- the device to join/leave this multicast address.

\end{itemize}


\paragraph{Example:} Let us continue with the example of previous subsection.

\begin{verbatim}
netadm@alisa:~ # ip maddr add 33:33:00:00:00:01 dev dummy
netadm@alisa:~ # ip -0 maddr ls dummy
2:  dummy
    link  33:33:00:00:00:01 users 2 static
    link  01:00:5e:00:00:01
netadm@alisa:~ # ip maddr del 33:33:00:00:00:01 dev dummy
\end{verbatim}

\begin{NB}
 Neither \verb|ip| nor kernel check for multicast address validity.
 Particularly, it means that you can try to load unicast address
 instead of multicast. The most of drivers will ignore such addresses,
 but several ones (f.e.\ Tulip) really intern it to their on-board filter.
 The effect may be strange. Namely, the addresses become additional
 local link addresses and, if you loaded address of another host
 to router, wait for duplicated packets on the wire.
 It is not a bug, it is rather hole in API and intra-kernel interfaces.
 This feature really more useful for traffic monitoring, but using it
 with Linux-2.2 you {\em have to\/} be sure, that the host is not
 router and, especially, it is not transparent proxy or masquerading
 agent.
\end{NB}



\section{{\tt ip mroute} --- multicast routing cache management.}
\label{IP-MROUTE}

\paragraph{Abbreviations:} \verb|mroute|, \verb|mr|.

\paragraph{Object:} The \verb|mroute| objects are multicast routing cache
entries, created by an user level mrouting daemon
(f.e.\ \verb|pimd| or \verb|mrouted|).

Due to limitations of current interface to multicast routing
engine it is impossible to change \verb|mroute| objects administratively,
so that we may only look at them. This limitation will be removed
in the future.

\paragraph{Commands:} \verb|show| (or \verb|list|).


\subsection{{\tt ip mroute show} --- list mroute cache entries.}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|sh|, \verb|ls|, \verb|l|.

\paragraph{Arguments:}

\begin{itemize}
\item \verb|to PREFIX| (default)

--- prefix selecting destination multicast addresses to list.


\item \verb|iif NAME|

--- interface which multicast packets are received on.


\item \verb|from PREFIX|

--- prefix selecting IP source addresses of multicast route.


\end{itemize}

\paragraph{Output format:}

\begin{verbatim}
kuznet@amber:~ $ ip mroute ls
(193.232.127.6, 224.0.1.39)      Iif: unresolved 
(193.232.244.34, 224.0.1.40)     Iif: unresolved 
(193.233.7.65, 224.66.66.66)     Iif: eth0       Oifs: pimreg 
kuznet@amber:~ $ 
\end{verbatim}

Each line shows one (S,G) entry in multicast routing cache,
where S is source address and G is multicast group. \verb|Iif| is
interface which multicast packets are expected to arrive on.
If word \verb|unresolved| stands instead of interface name,
it means that routing daemon still did not resolve this entry.
Keyword \verb|oifs| is followed by list of output interfaces, separated
by spaces. If multicast routing entry is created with non-trivial
TTL scope, administrative distances are appended the device names
in \verb|oifs| list.

\paragraph{Statistics:} Option \verb|-statistics| also prints
number of packets and bytes forwarded along this route and
number of packets arrived on wrong interface, if this number is not zero.

\begin{verbatim}
kuznet@amber:~ $ ip -s mr ls 224.66/16
(193.233.7.65, 224.66.66.66)     Iif: eth0       Oifs: pimreg 
  9383 packets, 300256 bytes
kuznet@amber:~ $
\end{verbatim}


\section{{\tt ip tunnel} --- tunnel configuration.}
\label{IP-TUNNEL}

\paragraph{Abbreviations:} \verb|tunnel|, \verb|tunl|.

\paragraph{Object:} The \verb|tunnel| objects are tunnels, encapsulating
packets in IPv4 packets and sending them over IP infrastructure then.

\paragraph{Commands:} \verb|add|, \verb|delete|, \verb|change|, \verb|show|
(or \verb|list|).

\paragraph{See also:} More informal discussion of tunneling
over IP and \verb|ip tunnel| command can be found in~\cite{IP-TUNNELS}.

\subsection{{\tt ip tunnel add} --- add new tunnel\\
	{\tt ip tunnel change} --- change existing tunnel\\
	{\tt ip tunnel delete} --- destroy a tunnel.}

\paragraph{Abbreviations:} \verb|add|, \verb|a|; \verb|change|, \verb|chg|;
\verb|delete|, \verb|del|, \verb|d|.


\paragraph{Arguments:}

\begin{itemize}

\item \verb|name NAME| (default)

--- select tunnel device name.

\item \verb|mode MODE|

--- set tunnel mode. Three modes are available now
	\verb|ipip|, \verb|sit| and \verb|gre|.

\item \verb|remote ADDRESS|

--- set remote endpoint of the tunnel.

\item \verb|local ADDRESS|

--- set fixed local address for tunneled packets.
It must be an address on another interface of this host.

\item \verb|ttl N|

--- set fixed TTL \verb|N| on tunneled packets.
	\verb|N| is number in the range 1--255. 0 is special value,
	meaning that packets inherit TTL value. 
		Default value is: \verb|inherit|.

\item \verb|tos T| or \verb|dsfield T|

--- set fixed TOS \verb|T| on tunneled packets.
		Default value is: \verb|inherit|.



\item \verb|dev NAME| 

--- bind tunnel to device \verb|NAME|, so that
	tunneled packets will be routed only via this device and will
	not able to escape to another device, when route to endpoint changes.

\item \verb|nopmtudisc|

--- disable Path MTU Discovery on this tunnel.
	It is enabled by default. Note that fixed ttl is incompatible
	with this option: tunnel with fixed ttl always makes pmtu discovery.

\item \verb|key K|, \verb|ikey K|, \verb|okey K|

--- (only GRE tunnels) use keyed GRE with key \verb|K|. \verb|K| is
	either number or IP address-like dotted quad.
   The parameter \verb|key| sets key to use in both directions,
   \verb|ikey| and \verb|okey| allow to set different keys for input and output.
   

\item \verb|csum|, \verb|icsum|, \verb|ocsum|

--- (only GRE tunnels) checksum tunneled packets.
   The flag \verb|ocsum| orders to checksum outgoing packets,
   \verb|icsum| requires that all the input packets had correct
   checksum. \verb|csum| is equivalent to the combination
  ``\verb|icsum| \verb|ocsum|''.

\item \verb|seq|, \verb|iseq|, \verb|oseq|

--- (only GRE tunnels) serialize packets.
   The flag \verb|oseq| enables sequencing outgouing packets,
   \verb|iseq| requires that all the input packets were serialized.
   \verb|seq| is equivalent to the combination ``\verb|iseq| \verb|oseq|''.

\begin{NB}
 I think this option does not
	work. At least, I did not test it, did not debug it and
	even do not understand,	how it is supposed to work and for what
	purpose Cisco planned to use it. Do not use it.
\end{NB}


\end{itemize}

\paragraph{Example:} create pointopoint IPv6 tunnel with maximal TTL of 32.
\begin{verbatim}
netadm@amber:~ # ip tunl add Cisco mode sit remote 192.31.7.104 \
    local 192.203.80.142 ttl 32 
\end{verbatim}

\subsection{{\tt ip tunnel show} --- list tunnels.}

\paragraph{Abbreviations:} \verb|show|, \verb|list|, \verb|sh|, \verb|ls|, \verb|l|.


\paragraph{Arguments:}

\paragraph{Output format:}
\begin{verbatim}
kuznet@amber:~ $ ip tunl ls Cisco
Cisco: ipv6/ip  remote 192.31.7.104  local 192.203.80.142  ttl 32 
kuznet@amber:~ $ 
\end{verbatim}
The line starts with tunnel device name terminated by colon,
then tunnel mode follows. The parameters of the tunnel are listed
with the same keywords which were used at tunnel creation.

\paragraph{Statistics:}

\begin{verbatim}
kuznet@amber:~ $ ip -s tunl ls Cisco
Cisco: ipv6/ip  remote 192.31.7.104  local 192.203.80.142  ttl 32 
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    12566      1707516      0      0        0        0       
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    13445      1879677      0      0        0        0     
kuznet@amber:~ $ 
\end{verbatim}
Essentially, these numbers are the same numbers, which are
printed with {\tt ip -s link show}
(sec.\ref{IP-LINK-SHOW}, p.\pageref{IP-LINK-SHOW}), but tags are different
to reflect tunnel specific.
\begin{itemize}
\item \verb|CsumErrs| --- total number of packets dropped
because of checksum failures for GRE tunnel with enabled checksumming.
\item \verb|OutOfSeq| --- total number of packets dropped
because they arrived out of sequence for GRE tunnel with enabled
serialization.
\item \verb|Mcasts| --- total number of multicast packets,
received on broadcast GRE tunnel.
\item \verb|DeadLoop| --- total number of packets, which were not
transmitted because tunnel is looped back to itself.
\item \verb|NoRoute| --- total number of packets, which were not
transmitted because there is no IP route to remote endpoint.
\item \verb|NoBufs| --- total number of packets, which were not
transmitted because kernel failed to allocate buffer.
\end{itemize}


\section{{\tt ip monitor} and {\tt rtmon} --- state monitoring.}
\label{IP-MONITOR}

\verb|ip| utility allows to monitor state of devices, addresses
and routes continuously. This option has a bit different format.
Namely,
command \verb|monitor| is the first in command line and then
object list follows:
\begin{verbatim}
  ip monitor [ file FILE ] [ all | OBJECT-LIST ]
\end{verbatim}
\verb|OBJECT-LIST| is list of object types, which we want to monitor.
It may contain \verb|link|, \verb|address| and \verb|route|.
If no \verb|file| argument is given, \verb|ip| opens RTNETLINK,
listens it and dumps state changes in the format, described
in the previous sections.

If a file name is given, it does not listen RTNETLINK,
but opens the file containing RTNETLINK messages saved in binary format
and dumps them. Such history file can be generated with utility
\verb|rtmon|. This utility has command line syntax similar to one
of \verb|ip monitor|.
Ideally, \verb|rtmon| should be started before
the first network configuration command is issued. F.e.\ if
you insert:
\begin{verbatim}
  rtmon file /var/log/rtmon.log
\end{verbatim}
in a startup script, you will be able to view the full history
later.

Certainly, it is possible to start \verb|rtmon| at any time;
it prepends the history with the state snapshot dumped at the moment
of start.


\section{Route realms and policy propagation, {\tt rtacct}.}
\label{RT-REALMS}

On routers using OSPF ASE or, especially, BGP protocol routing
tables may be huge. If we want to classify or to account the packets
per route, we will have to keep lots of information. Even worse, if we
want to distinguish the packets not only by their destination, but
also by their source, the task gets quadratic complexity and its solution
is impossible physically.

One approach to propagate the policy from routing protocols
to forwarding engine has been proposed in~\cite{IOS-BGP-PP}.
Essentially, Cisco Policy Propagation via BGP is based on the fact,
that dedicated routers have all the RIB (Routing Information Base)
close to forwarding engine, so that policy routing rules can
check all the route attributes, including ASPATH information
and community strings.

Linux architecture splitting RIB, maintained by user level
daemon, and kernel based FIB (Forwarding Information Base)
does not allow such simplex approach.

It is to our fortune, because there exist another solution,
which allows even more flexible policy and more rich semantics.

Namely, routes can be clustered together at user space, based on their
attributes.  F.e.\ BGP router knows route ASPATH, its community;
OSPF router knows route tag or its area. Administrator, when adding
routes manually, also knows their nature. Provided number of such
aggregates (we call them {\em realms\/}) is low, the task of full
classification both by source and destination becomes quite manageable.

So, each route may be assigned to a realm. It is supposed, that
this identification is made by routing daemon, but static routes
also can be handled manually with \verb|ip route| (see sec.\ref{IP-ROUTE},
p.\pageref{IP-ROUTE}).
\begin{NB}
  There exists patch to \verb|gated|, allowing to classify routes
  to realms with all the set of policy rules, implemented in \verb|gated|:
  by prefix, by ASPATH, by origin, by tag etc.
\end{NB}

To facilitate the construction (f.e.\ in the case, when routing
daemon is not aware of realms), missing realms may be completed
with routing policy rules, see sec.~\ref{IP-RULE}, p.\pageref{IP-RULE}.

For each packet kernel calculates tuple of realms: source realm
and destination realm, using the following algorithm:

\begin{enumerate}
\item If route has a realm, destination realm of the packet is set to it.
\item If rule has a source realm, source realm of the packet is set to it.
If destination realm was not get from route and rule has destination realm,
it is also set.
\item If at least one of realms is still unknown, kernel finds
reversed route to the source of the packet.
\item If source realm is still unknown, get it from reversed route.
\item If one of realms is still unknown, swap realms of reversed
routes and apply step 2 again.
\end{enumerate}

After this procedure is completed, we know what realm the packet
arrived from and the realm where it is going to propagate to.
If some of realms is unknown, it is initialized to zero
(or realm \verb|unknown|).

Main application of realms is TC \verb|route| classifier~\cite{TC-CREF},
where they are used to help to assign packets to traffic classes,
to account, to police and to schedule them according to this
classification.

Much simpler, but still very useful application is ingres
accounting by realms. Kernel gathers packet statistics summary,
which can be viewed with utility \verb|rtacct|.
\begin{verbatim}
kuznet@amber:~ $ rtacct russia
Realm      BytesTo    PktsTo     BytesFrom  PktsFrom   
russia     20576778   169176     47080168   153805     
kuznet@amber:~ $
\end{verbatim}
It shows that this router received 153805 packets from
realm \verb|russia| and forwarded 169176 packets to \verb|russia|.
The realm \verb|russia| consists of routes with ASPATHs not leaving
Russia.

Note that locally originated packets are not accounted here,
\verb|rtacct| shows ingres packets only. Using \verb|route|
classifier (see~\cite{TC-CREF}) you can get even more detailed
accounting information about outgres packets, optionally
summarizing traffic not only by source or destination, but
by any pair of source and destination realms.


\begin{thebibliography}{99}
\addcontentsline{toc}{section}{References}
\bibitem{RFC-NDISC} T.~Narten, E.~Nordmark, W.~Simpson.
``Neighbor Discovery for IP Version 6 (IPv6)'', RFC-2461.

\bibitem{RFC-ADDRCONF} S.~Thomson, T.~Narten.
``IPv6 Stateless Address Autoconfiguration'', RFC-2462.

\bibitem{RFC1812} F.~Baker.
``Requirements for IP Version 4 Routers'', RFC-1812.

\bibitem{RFC1122} R.~T.~Braden.
``Requirements for Internet hosts --- communication layers'', RFC-1122.

\bibitem{IOS} ``Cisco IOS Release 12.0 Network Protocols
Command Reference, Part 1'' and
``Cisco IOS Release 12.0 Quality of Service Solutions
Configuration Guide: Configuring Policy-Based Routing'',\\
http://www.cisco.com/univercd/cc/td/doc/product/software/ios120.

\bibitem{IP-TUNNELS} A.~N.~Kuznetsov.
``Tunnels over IP in Linux-2.2'', \\
In: {\tt ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz}.

\bibitem{TC-CREF} A.~N.~Kuznetsov. ``TC Command Reference'',\\
In: {\tt ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz}.

\bibitem{IOS-BGP-PP} ``Cisco IOS Release 12.0 Quality of Service Solutions
Configuration Guide: Configuring QoS Policy Propagation via
Border Gateway Protocol'',\\
http://www.cisco.com/univercd/cc/td/doc/product/software/ios120.

\bibitem{RFC-DHCP} R.~Droms.
``Dynamic Host Configuration Protocol.'', RFC-2131

\end{thebibliography}




\appendix
\addcontentsline{toc}{section}{Appendix}

\section{Source address selection.}
\label{ADDR-SEL}

When a host originates an IP packet, it must select some source
address. Correct source address selection is critical procedure,
because it gives to the receiver the information, how to deliver
reply. If the source is selected incorrectly, in the best case
the backward path may appear different of forward one, which
is harmful for performance. In the worst case, when the addresses
are administratively scoped, the reply may be lost at all.

Linux-2.2 selects source addresses using the following algorithm:

\begin{itemize}
\item
The application may select source address explicitly with \verb|bind(2)|
syscall or supplying it to \verb|sendmsg(2)| via ancillary data object
\verb|IP_PKTINFO|. In this case the kernel only checks validity
of the address and never tries to ``improve'' incorrect user choice,
generating an error instead.
\begin{NB}
 Never say ``Never''. Sysctl option \verb|ip_dynaddr| breaks
 this axiom. It has been made deliberately with purpose
 to reselect automatically address on hosts with dynamic dial-out interfaces.
 However, this hack {\em must not\/} be used on multihomed hosts
 and especially on routers: it would break them.
\end{NB}


\item Otherwise, IP routing tables can contain explicit source
address hint for this destination. The hint is set with parameter \verb|src|
to \verb|ip route| command, sec.\ref{IP-ROUTE}, p.\pageref{IP-ROUTE}.


\item Otherwise, the kernel searches through list of addresses,
attached to the interface, which packets will be routed out.
The search strategies are different for IP and IPv6. Namely:

\begin{itemize}
\item IPv6 searches for the first valid, not deprecated address
with the same scope as destination.

\item IP searches for the first valid address with scope wider
than scope of the destination, but it prefers the addresses
which fall to the same subnet, as the nexthop of the route
to the destination. Unlike IPv6, the scopes of IPv4 destinations
are not encoded in their addresses and they are supplied
in routing tables instead (parameter \verb|scope| to \verb|ip route| command,
sec.\ref{IP-ROUTE}, p.\pageref{IP-ROUTE}).

\end{itemize}


\item Otherwise, if the scope of destination is \verb|link| or \verb|host|,
the algorithm fails and returns zero source address.

\item Otherwise, all the interfaces are scanned to search for an address
with appropriate scope. Loopback device \verb|lo| is always the first
in search list, so that if an address with global scope (not 127.0.0.1!)
is configured on loopback, it is always preferred.

\end{itemize}


\section{Proxy ARP/NDISC.}
\label{PROXY-NEIGH}

Routers may answer ARP/NDISC solicitations on behalf of another hosts.
In linux-2.2 proxy ARP on an interface may be enabled
by setting kernel \verb|sysctl| variable 
\verb|net/ipv4/conf/<dev>/proxy_arp| to 1. After this router
starts to answer ARP requests on the interface \verb|<dev>|, provided
the route to the requested destination goes {\em not\/} back via the same
device. The variable \verb|net/ipv4/conf/all/proxy_arp| enables proxy
ARP on all the IP devices.

However, this approach fails in the case of IPv6, because router
must join solicited node multicast address to listen for the corresponding
NDISC queries. It means, that proxy NDISC is possible only on per destination
base.

Logically, proxy ARP/NDISC is not kernel task, it can be easily implemented
in user space. However, similar functionality was present in BSD kernels
and in Linux-2.0, so that we have to preserve it at least in the extent, which
is standardized in BSD.
\begin{NB}
  Linux-2.0 ARP had a feature called {\em subnet\/} proxy ARP.
  It is replaced with sysctl flag in Linux-2.2.
\end{NB}


The \verb|ip| utility provides a way to manage proxy ARP/NDISC
with command \verb|ip neigh|, namely:
\begin{verbatim}
  ip neigh add proxy ADDRESS [ dev NAME ]
\end{verbatim}
adds new proxy ARP/NDISC record and
\begin{verbatim}
  ip neigh del proxy ADDRESS [ dev NAME ]
\end{verbatim}
deletes it.

If the name of the device is not given, router will answer solicitations
for address \verb|ADDRESS| on all the devices, otherwise it will serve
only the device \verb|NAME|. Even if the proxy entry is created with
\verb|ip neigh|, router {\em will not\/} answer query, if route
to destination goes back via the interface, which the solicitation
was received from.

It is important to emphasize that proxy entries have {\em no\/}
parameters different of these ones (IP/IPv6 address and optional device).
Particularly, the entry does not store any link layer address,
it always advertises its own station address of the interface,
where it sends advertisements.

\section{Route NAT status.}
\label{ROUTE-NAT}

NAT (or ``Network Address Translation'') allows to remap some parts
of IP address space to another ones. Linux-2.2 route NAT is supposed
to be used to facilitate policy routing by rewriting addresses
to another routing domains or to help while renumbering sites
to another prefix.

\paragraph{What it does not.}
It is necessary to emphasize that {\em it is not supposed\/}
to be used to compress address space or to split load.
It is not missing functionality but design principle.
Route NAT is {\em stateless\/}, it does not hold any state
about translated sessions. It means that it handles any number
of sessions flawlessly. But also it means that it is {\em static\/},
it cannot detect the moment, when the last TCP client stops
to use the address. By the same reason it will not help to split
load between several servers.
\begin{NB}
It is pretty common belief that it is useful to split load between
several servers with NAT. It is mistake. All that you get from this
is that router have to keep state of all the TCP connections going
via it. Well, if the router is so powerful, run apache on it. 8)
\end{NB}

The second feature: it does not touch packet payload,
does not try to ``improve'' broken protocols by looking
through its data and mangling it. It mangles IP addresses,
only IP addresses and nothing but IP addresses.
It is also not missing functionality.

To resume: if you need to compress address space or keep
happy active FTP clients, your choice is not route NAT but masquerading,
port forwarding, NAPT etc. 
\begin{NB}
By the way, you may look also at
http://www.csn.tu-chemnitz.de/HyperNews/get/linux-ip-nat.html
\end{NB}


\paragraph{How it works.}
Some part of address space is reserved for dummy addresses,
which will look for all the world as some hosts addresses
inside your network. No another hosts may use these addresses,
however another routers also may be configured to translate them.
\begin{NB}
It is great advantage of route NAT, it may be used not
only in stub networks, but in environments with arbitrarily complicated
structure. It does not firewall, it {\em forwards.}
\end{NB}
These addresses are selected by \verb|ip route| command
(sec.\ref{IP-ROUTE-ADD}, p.\pageref{IP-ROUTE-ADD}). F.e.\
\begin{verbatim}
  ip route add nat 192.203.80.144 via 193.233.7.83
\end{verbatim}
tells that single address 192.203.80.144 is dummy NAT address.
For all the world it looks as a host address inside our network,
for neighbouring hosts and routers it looks as local address
of translating router. The router answers ARP for it, it advertises
this address, as routed via it {\em et al\/}. When the router
receives a packet destined to 192.203.80.144, it replaces 
this address to 193.233.7.83, which is address of some really
existing host and forwards the packet. If you need to remap
blocks of addresses, you may use command sort of:
\begin{verbatim}
  ip route add nat 192.203.80.192/26 via 193.233.7.64
\end{verbatim}
This command will map block of 63 addresses 192...255 to 64...127.

When internal host (193.233.7.83 in the example above)
send something to the outer world and these packets are forwarded
by our router, it should translate source address 193.233.7.83
to 192.203.80.144. This task is solved by setting special
policy rule (sec.\ref{IP-RULE-ADD}, p.\pageref{IP-RULE-ADD}):
\begin{verbatim}
  ip rule add prio 320 from 193.233.7.83 nat 192.203.80.144
\end{verbatim}
This rule says that source address 193.233.7.83
should be translated to 192.203.80.144 before forwarding.
It is important that address standing after keyword \verb|nat|
was some NAT address, declared by {\tt ip route add nat}.
If it is just a random address the router will not map to it.
\begin{NB}
The exception is when the address is a local address of this
router (or 0.0.0.0) and masquearading is configured in the kernel.
In this case router will masquerade the packets as this address.
If 0.0.0.0 is selected, the result is equivalent to one,
obtained with firewalling rules. Otherwise, you have the way
to order Linux to masquerade to this fixed address.
\end{NB}

If the network has non-trivial internal structure, it is
useful and even necessary to add rules disbabling translation,
when packet does not leave this network. Let us return to the
example from sec.\ref{IP-RULE-SHOW} (p.\pageref{IP-RULE-SHOW}).
\begin{verbatim}
300:	from 193.233.7.83 to 193.233.7.0/24 lookup main
310:	from 193.233.7.83 to 192.203.80.0/24 lookup main
320:	from 193.233.7.83 lookup inr.ruhep map-to 192.203.80.144
\end{verbatim}
This block of rules orders to make normal forwarding, when
packets from 193.233.7.83 do not leave networks 193.233.7/24
and 192.203.80/24. If the table \verb|inr.ruhep| does not
contain route to the destination (which means that routing
domain owning addresses from 192.203.80/24 is dead), no translation
also will occur. Otherwise, the packets are translated.

\paragraph{How to translate only selected ports.}
If you want to translate only selected ports (f.e.\ http)
and leave all the rest intact, you may use \verb|ipchains|
to mark a class of packets with a \verb|fwmark|.
Suppose, you did it and all the packets from 193.233.7.83
destined to port 80 are marked with marker 0x1234 in input fwchain.
In this case you may replace rule \#320 with:
\begin{verbatim}
320:	from 193.233.7.83 fwmark 1234 lookup main map-to 192.203.80.144
\end{verbatim}
and translation will be enabled only for outgouing http requests.

\section{Example: minimal host setup.}
\label{EXAMPLE-SETUP}

The script following below gives an example of fault safe
setup of IP (and IPv6, if it is compiled into the kernel)
in the common case of node attached to a single broadcast
network. More advanced script, which may be used both on multihomed
hosts and on routers, is described in the following
section.

The utilities, used in the script, may be found in the
directory ftp://ftp.inr.ac.ru/ip-routing/:
\begin{enumerate}
\item \verb|ip| --- package \verb|iproute2|.
\item \verb|arping| --- package \verb|iputils|.
\item \verb|rdisc| --- package \verb|iputils|.
\end{enumerate}
\begin{NB}
It also refers to DHCP client \verb|dhcpcd|. I should refrain from
attempt to recommend any good DHCP client to use. All that I can
say is that ISC \verb|dhcp-2.0b1pl6| patched by patch, which
can be found in subdirectory \verb|dhcp.bootp.rarp| of
the same ftp site, {\em does\/} work
at least on Ethernet and Token Ring.
\end{NB}

\begin{verbatim}
#! /bin/bash
\end{verbatim}
\begin{flushleft}
\# {\bf Usage: \verb|ifone ADDRESS[/PREFIX-LENGTH] [DEVICE]|}\\
\# {\bf Parameters:}\\
\# \$1 --- Static IP address, optionally followed by prefix length.\\
\# \$2 --- Device name. If it is missing, \verb|eth0| is asssumed.\\
\# F.e. \verb|ifone 193.233.7.90|
\end{flushleft}
\begin{verbatim}
dev=$2
: ${dev:=eth0}
ipaddr=
\end{verbatim}
\# Parse IP address, splitting prefix length.
\begin{verbatim}
if [ "$1" != "" ]; then
  ipaddr=${1%/*}
  if [ "$1" != "$ipaddr" ]; then
    pfxlen=${1#*/}
  fi
  : ${pfxlen:=24}
fi
pfx="${ipaddr}/${pfxlen}"
\end{verbatim}

\begin{flushleft}
\# {\bf Step 0} --- enable loopback.\\
\#\\
\# This step is necessary on any networked box before attempt\\
\# to configure any other device.\\
\end{flushleft}
\begin{verbatim}
ip link set up dev lo
ip addr add 127.0.0.1/8 dev lo brd + scope host
\end{verbatim}
\begin{flushleft}
\# IPv6 autoconfigure themself on loopback.\\
\#\\
\# If user gave loopback as device, we add the address as alias and exit.
\end{flushleft}
\begin{verbatim}
if [ "$dev" = "lo" ]; then
  if [ "$ipaddr" != "" -a  "$ipaddr" != "127.0.0.1" ]; then
    ip address add $ipaddr dev $dev
    exit $?
  fi
  exit 0
fi
\end{verbatim}

\noindent\# {\bf Step 1} --- enable device \verb|$dev|

\begin{verbatim}
if ! ip link set up dev $dev ; then
  echo "Cannot enable interface $dev. Aborting." 1>&2
  exit 1
fi
\end{verbatim}
\begin{flushleft}
\# The interface is \verb|UP|. IPv6 started stateless autoconfiguration itself,\\
\# and its configuration finishes here. However,\\
\# IP still needs some static preconfigured address.
\end{flushleft}
\begin{verbatim}
if [ "$ipaddr" = "" ]; then
  echo "No address for $dev is configured, trying DHCP..." 1>&2
  dhcpcd
  exit $?
fi
\end{verbatim}

\begin{flushleft}
\# {\bf Step 2} --- IP Duplicate Address Detection~\cite{RFC-DHCP}.\\
\# Send two probes and wait for result for 3 seconds.\\
\# If the interface opens slower f.e.\ due to long media detection,\\
\# you want to increase the timeout.\\
\end{flushleft}
\begin{verbatim}
if ! arping -q -c 2 -w 3 -D -I $dev $ipaddr ; then
  echo "Address $ipaddr is busy, trying DHCP..." 1>&2
  dhcpcd
  exit $?
fi
\end{verbatim}
\begin{flushleft}
\# OK, the address is unique, we may add it on the interface.\\
\#\\
\# {\bf Step 3} --- Configure the address on the interface.
\end{flushleft}

\begin{verbatim}
if ! ip address add $pfx brd + dev $dev; then
  echo "Failed to add $pfx on $dev, trying DHCP..." 1>&2
  dhcpcd
  exit $?
fi
\end{verbatim}

\noindent\# {\bf Step 4} --- Announce our presense on the link.
\begin{verbatim}
arping -A -c 1 -I $dev $ipaddr
noarp=$?
( sleep 2;
  arping -U -c 1 -I $dev $ipaddr ) >& /dev/null </dev/null &
\end{verbatim}

\begin{flushleft}
\# {\bf Step 5} (optional) --- Add some control routes.\\
\#\\
\# 1. Prohibit link local multicast addresses.\\
\# 2. Prohibit link local (alias, limited) broadcast.\\
\# 3. Add default multicast route.
\end{flushleft}
\begin{verbatim}
ip route add unreachable 224.0.0.0/24 
ip route add unreachable 255.255.255.255
if [ `ip link ls $dev | grep -c MULTICAST` -ge 1 ]; then
  ip route add 224.0.0.0/4 dev $dev scope global
fi
\end{verbatim}

\begin{flushleft}
\# {\bf Step 6} --- Add fallback default route with huge metric.\\
\# If a proxy ARP server is present on the interface, we will be\\
\# able to talk to all the Internet without further configuration.\\
\# It is not so cheap though and we still hope that this route\\
\# will be overridden by more correct one by rdisc.\\
\# Do not make this step if the device is not ARPable,\\
\# because dead nexthop detection does not work on them.
\end{flushleft}
\begin{verbatim}
if [ "$noarp" = "0" ]; then
  ip ro add default dev $dev metric 30000 scope global
fi
\end{verbatim}

\begin{flushleft}
\# {\bf Step 7} --- Restart router discovery and exit.
\end{flushleft}
\begin{verbatim}
killall -HUP rdisc || rdisc -fs
exit 0
\end{verbatim}


\section{Example: {\protect\tt ifcfg} --- interface address management.}
\label{EXAMPLE-IFCFG}

It is simplistic script replacing one option of \verb|ifconfig|,
namely, IP addresses management. It not only adds
address, but also carries out Duplicate Address Detection~\cite{RFC-DHCP},
sends unsolicited ARP to update cache of another hosts sharing
the interface, adds some control routes and restarts Route Discovery
when it is necessary.

I strongly recommend to use it {\em instead\/} of \verb|ifconfig| both
on hosts and on routes.

\begin{verbatim}
#! /bin/bash
\end{verbatim}
\begin{flushleft}
\# {\bf Usage: \verb?ifcfg DEVICE[:ALIAS] [add|del] ADDRESS[/LENGTH] [PEER]?}\\
\# {\bf Parameters:}\\
\# ---Device name. It may have alias suffix, separted by colon.\\
\# ---Command: add, delete or stop.\\
\# ---IP address, optionally followed by prefix length.\\
\# ---Optional peer address for pointopoint interfaces.\\
\# F.e. \verb|ifcfg eth0 193.233.7.90/24|

\noindent\# This function determines, whether it is router or host.\\
\# It returns 0, if the host is apparently not router.
\end{flushleft}
\begin{verbatim}
CheckForwarding () {
  local sbase fwd
  sbase=/proc/sys/net/ipv4/conf
  fwd=0
  if [ -d $sbase ]; then
    for dir in $sbase/*/forwarding; do
      fwd=$[$fwd + `cat $dir`]
    done
  else
    fwd=2
  fi
  return $fwd
}
\end{verbatim}
\begin{flushleft}
\# This function restarts Router Discovery.\\
\end{flushleft}
\begin{verbatim}
RestartRDISC () {
  killall -HUP rdisc || rdisc -fs
}
\end{verbatim}
\begin{flushleft}
\# Calculate ABC "natural" mask length\\
\# Arg: \$1 = dotquad address
\end{flushleft}
\begin{verbatim}
ABCMaskLen () {
  local class;
  class=${1%%.*}
  if [ $class -eq 0 -o $class -ge 224 ]; then return 0
  elif [ $class -ge 192 ]; then return 24
  elif [ $class -ge 128 ]; then return 16
  else  return 8 ; fi
}
\end{verbatim}


\begin{flushleft}
\# {\bf MAIN()}\\
\#\\
\# Strip alias suffix separated by colon.
\end{flushleft}
\begin{verbatim}
label="label $1"
ldev=$1
dev=${1%:*}
if [ "$dev" = "" -o "$1" = "help" ]; then
  echo "Usage: ifcfg DEV [[add|del [ADDR[/LEN]] [PEER] | stop]" 1>&2
  echo "       add - add new address" 1>&2
  echo "       del - delete address" 1>&2
  echo "       stop - completely disable IP" 1>&2
  exit 1
fi
shift

CheckForwarding
fwd=$?
\end{verbatim}
\begin{flushleft}
\# Parse command. If it is ``stop'', flush and exit.
\end{flushleft}
\begin{verbatim}
deleting=0
case "$1" in
add) shift ;;
stop)
  if [ "$ldev" != "$dev" ]; then
    echo "Cannot stop alias $ldev" 1>&2
    exit 1;
  fi
  ip -4 addr flush dev $dev $label || exit 1
  if [ $fwd -eq 0 ]; then RestartRDISC; fi
  exit 0 ;;
del*)
  deleting=1; shift ;;
*)
esac
\end{verbatim}
\begin{flushleft}
\# Parse prefix, split prefix length, separated by slash.
\end{flushleft}
\begin{verbatim}
ipaddr=
pfxlen=
if [ "$1" != "" ]; then
  ipaddr=${1%/*}
  if [ "$1" != "$ipaddr" ]; then
    pfxlen=${1#*/}
  fi
  if [ "$ipaddr" = "" ]; then
    echo "$1 is bad IP address." 1>&2
    exit 1
  fi
fi
shift
\end{verbatim}
\begin{flushleft}
\# If peer address is present, prefix length is 32.\\
\# Otheriwise, if prefix length was not given, guess it.
\end{flushleft}
\begin{verbatim}
peer=$1
if [ "$peer" != "" ]; then
  if [ "$pfxlen" != "" -a "$pfxlen" != "32" ]; then
    echo "Peer address with non-trivial netmask." 1>&2
    exit 1
  fi
  pfx="$ipaddr peer $peer"
else
  if [ "$pfxlen" = "" ]; then
    ABCMaskLen $ipaddr
    pfxlen=$?
  fi
  pfx="$ipaddr/$pfxlen"
fi
if [ "$ldev" = "$dev" -a "$ipaddr" != "" ]; then
  label=
fi
\end{verbatim}
\begin{flushleft}
\# If deletion was requested, delete the address and restart RDISC
\end{flushleft}
\begin{verbatim}
if [ $deleting -ne 0 ]; then
  ip addr del $pfx dev $dev $label || exit 1
  if [ $fwd -eq 0 ]; then RestartRDISC; fi
  exit 0
fi
\end{verbatim}
\begin{flushleft}
\# Start interface initialization.\\
\#\\
\# {\bf Step 0} --- enable device \verb|$dev|
\end{flushleft}
\begin{verbatim}
if ! ip link set up dev $dev ; then
  echo "Error: cannot enable interface $dev." 1>&2
  exit 1
fi
if [ "$ipaddr" = "" ]; then exit 0; fi
\end{verbatim}
\begin{flushleft}
\# {\bf Step 1} --- IP Duplicate Address Detection~\cite{RFC-DHCP}.\\
\# Send two probes and wait for result for 3 seconds.\\
\# If the interface opens slower f.e.\ due to long media detection,\\
\# you want to increase the timeout.\\
\end{flushleft}
\begin{verbatim}
if ! arping -q -c 2 -w 3 -D -I $dev $ipaddr ; then
  echo "Error: some host already uses address $ipaddr on $dev." 1>&2
  exit 1
fi
\end{verbatim}
\begin{flushleft}
\# OK, the address is unique. We may add it to the interface.\\
\#\\
\# {\bf Step 2} --- Configure the address on the interface.
\end{flushleft}
\begin{verbatim}
if ! ip address add $pfx brd + dev $dev $label; then
  echo "Error: failed to add $pfx on $dev." 1>&2
  exit 1
fi
\end{verbatim}
\noindent\# {\bf Step 3} --- Announce our presense on the link
\begin{verbatim}
arping -q -A -c 1 -I $dev $ipaddr
noarp=$?
( sleep 2 ;
  arping -q -U -c 1 -I $dev $ipaddr ) >& /dev/null </dev/null &
\end{verbatim}
\begin{flushleft}
\# {\bf Step 4} (optional) --- Add some control routes.\\
\#\\
\# 1. Prohibit link local multicast addresses.\\
\# 2. Prohibit link local (alias, limited) broadcast.\\
\# 3. Add default multicast route.
\end{flushleft}
\begin{verbatim}
ip route add unreachable 224.0.0.0/24 >& /dev/null 
ip route add unreachable 255.255.255.255 >& /dev/null
if [ `ip link ls $dev | grep -c MULTICAST` -ge 1 ]; then
  ip route add 224.0.0.0/4 dev $dev scope global >& /dev/null
fi
\end{verbatim}
\begin{flushleft}
\# {\bf Step 5} --- Add fallback default route with huge metric.\\
\# If a proxy ARP server is present on the interface, we will be\\
\# able to talk to all the Internet without further configuration.\\
\# Do not make this step on router or if the device is not ARPable.\\
\# because dead nexthop detection does not work on them.
\end{flushleft}
\begin{verbatim}
if [ $fwd -eq 0 ]; then
  if [ $noarp -eq 0 ]; then
    ip ro append default dev $dev metric 30000 scope global
  elif [ "$peer" != "" ]; then
    if ping -q -c 2 -w 4 $peer ; then
      ip ro append default via $peer dev $dev metric 30001
    fi
  fi
  RestartRDISC
fi

exit 0
\end{verbatim}
\begin{flushleft}
\# End of {\bf MAIN()}
\end{flushleft}


\end{document}