File: spec_45.html

package info (click to toggle)
exim-html 3.20-1
links: PTS
area: main
in suites: etch, etch-m68k, sarge, woody
size: 2,868 kB
ctags: 4,188
sloc: makefile: 40; sh: 19
file content (456 lines) | stat: -rw-r--r-- 18,208 bytes
<HTML>
<HEAD>
<!-- This HTML file has been created by texi2html 1.52
     from spec on 25 November 2000 -->

<TITLE>Exim Specification - 45. Verification of incoming mail</TITLE>
</HEAD>
<body bgcolor="#FFFFFF" text="#00005A" link="#FF6600" alink="#FF9933" vlink="#990000">
Go to the <A HREF="spec_1.html">first</A>, <A HREF="spec_44.html">previous</A>, <A HREF="spec_46.html">next</A>, <A HREF="spec_59.html">last</A> section, <A HREF="spec_toc.html">table of contents</A>.
<P><HR><P>


<H1><A NAME="SEC802" HREF="spec_toc.html#TOC802">45. Verification of incoming mail</A></H1>
<P>
<A NAME="IDX1734"></A>
Exim always checks the syntax of SMTP commands, and rejects any that are
invalid. There are a number of options that cause Exim to verify the semantic
validity of the data in an incoming SMTP message. Verification failures can
cause the message to be rejected, or they can just be logged. Other types of
control over incoming mail are discussed in subsequent chapters. The -<EM>bh</EM>
command line option can be used to run fake SMTP sessions for the purpose of
testing verification options.

</P>

<P>



<H2><A NAME="SEC803" HREF="spec_toc.html#TOC803">45.1 Host verification</A></H2>

<P>
<A NAME="IDX1735"></A>
<A NAME="IDX1736"></A>
<A NAME="IDX1737"></A>
<A NAME="IDX1738"></A>
<A NAME="IDX1739"></A>
<A NAME="IDX1740"></A>
The name of the sending host is looked up using <EM>gethostbyaddr()</EM> if its IP
address matches
<EM>host_lookup</EM>
(which is unset in the Exim binary, but in the default configuration file is
set to match all hosts). In some environments this might involve an expensive
DNS lookup, so some sites may wish to disable it. However, an SMTP server for
local desktop systems (which are frequently misconfigured) can normally look up
their host names cheaply. This improves the contents of Exim's logs by
including the correct host names.

</P>
<P>
Even if its address doesn't match
<EM>host_lookup</EM>,
a sending host's real name is looked up from its IP address if the argument it
provides for the HELO or EHLO command is the local host's own name, or
the name of one of its local domains, which seems to be a fairly common
misconfiguration.

</P>
<P>
A host name that is obtained from looking up the sender's IP address is placed
in the $<EM>sender_host_name</EM> variable. If no lookup was done, or if the lookup
failed, that variable is left empty. Failure to look up the sending host's name
is not of itself an error, nor is it by default an error for the name given in
the HELO or EHLO command (which is placed in $<EM>sender_helo_name</EM>)
to be different.

</P>
<P>
The RFCs specifically state that mail should not be refused on the basis of the
content of the HELO or EHLO commands. However, there are installations
that do want to be strict in this area, and to support them, Exim has the
<EM>helo_verify</EM> option. Even when this is not set, Exim checks the syntax of the
commands, and rejects them if there are syntax errors. It can be made less
strict by unsetting <EM>helo_strict_syntax</EM> (which allows underscores to get
through) or by setting
<EM>helo_accept_junk_hosts</EM> (which permits certain hosts to send any old junk).

</P>
<P>
When <EM>helo_verify</EM> is set, a HELO or EHLO command must precede any
MAIL commands in an incoming SMTP connection. If there wasn't one, all
MAIL commands are rejected with a permanent error code. In addition, the
argument supplied by HELO or EHLO is verified. If it is in the form of
a literal IP address in square brackets, it must match the actual IP address of
the sending host. If it is a domain name, the sending host's name is
looked up from its IP address (whether or not it matches
<EM>host_lookup</EM>)
and compared against it. If the comparison fails, the IP addresses associated
with the HELO or EHLO name are looked up using <EM>gethostbyname()</EM> and
compared against the sending host's IP address. If none of them match, the
HELO or EHLO command is rejected with a permanent error code, and an
entry is written in the main and reject logs.

</P>



<H2><A NAME="SEC804" HREF="spec_toc.html#TOC804">45.2 Sender verification</A></H2>

<P>
<A NAME="IDX1741"></A>
<A NAME="IDX1742"></A>
<A NAME="IDX1743"></A>
When <EM>sender_verify</EM> is set, Exim checks the senders of incoming SMTP
messages, that is, the addresses given in the SMTP MAIL commands.
<A NAME="IDX1744"></A>
<A NAME="IDX1745"></A>
This does not apply to batch SMTP input by default, but <EM>sender_verify_batch</EM>
can be set true if it is required.
<font color=green>
The check is performed by running the same verification code as is used when
Exim is called with the -<EM>bvs</EM> option, that is, by running the directors and
routers in verify mode.

</P>
<P>
A dilemma arises when a local address is expanded by aliasing or forwarding:
should verification continue with the generated addresses, or should the
successful expansion of the original address be enough to verify it? Exim
(since release 3.20) takes the following pragmatic approach:

</P>

<UL>

<LI>

When an incoming address is aliased to just one child address, in an
<EM>aliasfile</EM> or a <EM>smartuser</EM> director (but <EM>not</EM> for <EM>forwardfile</EM>),
verification continues with the child address, and if that fails to verify, the
original verification also fails.
</UL>

<P>
This seems the most reasonable behaviour for the common use of aliasing as a
way of directing different local parts to the same mailbox. It means, for
example, that a pair of alias entries of the form

<PRE>
A.Wol:   aw123
aw123:   :fail: Gone away, no forwarding address
</PRE>

<P>
work as expected, with both local parts causing verification failure. On the
other hand, when an alias generates more than one address, the behaviour is
more like a mailing list, where the existence of the alias itself is sufficient
for verification.
</font>

</P>
<P>
The sender verification check is performed when the MAIL command is
received. If the address cannot immediately be verified (typically because of
DNS timeouts), a temporary failure error response (code 451) is given after the
data for the message has been received. It is delayed until this time so that
the message's headers can be logged. However, if <EM>sender_try_verify</EM> is set,
the sender is accepted with a warning message after a temporary verification
failure.

</P>
<P>
Exim remembers temporary sender verification errors in a hints database.
Subsequent temporary errors for the same address from the same host within 24
hours cause a 451 error after MAIL instead of after the data. This
reduces the data on the reject log and also the amount repeatedly transferred
over the net.

</P>
<P>
If <EM>sender_verify_max_retry_rate</EM> is set greater than zero, and the rate of
temporary rejection of a specific incoming sender address from a specific host,
in units of rejections per hour, exceeds it, the temporary error is converted
into a permanent verification error. This should help in stopping hosts
hammering too frequently with temporarily failing sender addresses. The default
value of the option is 12, which means that a sender address that has a
temporary verification error more than once every 5 minutes will soon get
permanently rejected. Once permanent rejection has been triggered, subsequent
temporary failures will all cause permanent errors, until there has been an
interval of at least 24 hours since the last failure. After 24 hours, the hint
expires.

</P>
<P>
What happens if verification fails with a permanent error depends on the
setting of the <EM>sender_verify_reject</EM> option. If it is set (the default) then
the message is rejected. Otherwise a warning message is logged, and processing
continues.

</P>
<P>
Because remote postmasters always want to see the message headers when there is
a problem, Exim does not give an error response immediately a sender address
fails, but instead it reads the data for the message first. The headers of
rejected messages are written to the reject log, for use in tracking down the
problem or tracing mail abusers. Up to three envelope recipients are also
logged with the headers.

</P>
<P>
Unfortunately, there are a number of mailers in use that treat any SMTP error
response given after the data has been transmitted as a temporary failure. Exim
sends code 550 when it rejects a message because of a bad sender, and RFC 821
is quite clear in stating that all codes starting with 5 are always `permanent
negative completion' replies. However, it does not give any guidance as to what
should be done on receiving such replies, and some mailers persist in trying to
send messages when they receive such a code at the end of the data.

</P>
<P>
To get round this, Exim keeps a database in which it remembers the bad sender
address and host name when it rejects a message. If the same host sends the
same bad sender address within 24 hours, Exim rejects the message at the
MAIL command, before it reads the data for the message. This should prevent
the sender from trying to send the message again, but there seem to be plenty
of broken mailers out there that do keep on trying, sometimes for days on end.

</P>
<P>
In an attempt to shut such MTAs up, if the same host sends the same bad sender
for a third time within 24 hours, MAIL is accepted, but all subsequent
RCPT commands are rejected with a 550 error code. This means `unknown
user' and if a remote mailer doesn't treat that as a hard error, it is very
seriously broken.

</P>
<P>
The <EM>sender_verify_hosts</EM> option can be used to restrict hosts and RFC 1413
idents for which sender verification is not applied. If a cluster of hosts all
check incoming external messages, there is no need to waste effort checking
mail sent between them. For example:

<PRE>
sender_verify_hosts = ! *.ref.book : ! exim@mailer.fict.book
</PRE>

<P>
<font color=green>


<H2><A NAME="SEC805" HREF="spec_toc.html#TOC805">45.3 Sender verification with callback</A></H2>

<P>
<A NAME="IDX1746"></A>
<A NAME="IDX1747"></A>
When Exim verifies a remote sender address by running it through the routers,
as described above, it verifies the domain, but is unable to do any checking of
the local part. There are situations where some means of verifying the local
part is desirable, and this can be setup by configuring Exim to use an SMTP
<EM>callback</EM>. If the domain in the remote address verifies successfully when
calling back is enabled, Exim makes an SMTP call to the hosts to which the
sender's domain resolves, and tests the address as if it were the recipient of
a bounce message. Specifically, it sends

<PRE>
HELO &#60;<EM>primary host name</EM>&#62;
MAIL FROM:&#60;&#62;
RCPT TO:&#60;<EM>the address to be tested</EM>&#62;
QUIT
</PRE>

<P>
If the response to the RCPT command is a 2<EM>xx</EM> code, the verification
succeeds. If it is 5<EM>xx</EM>, the verification fails. For anything else, and in
cases when Exim cannot contact any of the relevant hosts, verification fails
with a temporary error code.

</P>
<P>
Callback verification occurs only if the sending host matches
<EM>sender_verify_hosts_callback</EM> (in addition to <EM>sender_verify_hosts</EM>), and
the sender's domain matches <EM>sender_verify_callback_domains</EM>. Both of these
options default unset. There is also an option called
<EM>sender_verify_callback_timeout</EM> which sets a timeout for connecting and for
each command. It defaults to 30 seconds. Callback verification is expensive,
and not recommended for general use, especially on busy hosts. Two cases where
it might be useful are:

</P>

<UL>

<LI>

On a small host that handles only a few messages a day, to keep out junk with
valid domains but invalid local parts in the senders, something that is
commonly encountered in spam messages. For this you would set

<PRE>
sender_verify_hosts_callback = *
sender_verify_callback_domains = *
</PRE>

<LI>

On a corporate gateway, to check sender addresses in domain(s) that are `yours'
in some sense, but not local (in the Exim sense). This could be restricted to
messages received from your on-site hosts.
</UL>

<P>
</font>

</P>



<H2><A NAME="SEC806" HREF="spec_toc.html#TOC806">45.4 Fixing bad senders</A></H2>

<P>
<A NAME="IDX1748"></A>
<A NAME="IDX1749"></A>
It is unfortunately the case that lots of messages are sent out onto the
Internet with invalid senders. In some cases, the message itself contains a
valid return address in one of its headers. If the <EM>sender_verify_fixup</EM>
option is set as well as <EM>sender_verify</EM>, Exim does not reject a message if
the sender is invalid, provided it can find a <EM>Sender:</EM>, <EM>Reply-To:</EM>, or
<EM>From:</EM> header containing a valid address. Instead, it replaces the sender with
the valid address, and records the fact that it has done so by adding a header
of the form:

<PRE>
X-BadReturnPath: &#60;<EM>invalid address</EM>&#62; rewritten using &#60;<EM>name</EM>&#62; header
</PRE>

<P>
If there are several occurrences of any of the relevant headers, they are all
checked. If any <TT>`Resent-'</TT> headers exist, it is those headers that are checked
rather than the original ones.

</P>
<P>
The fixup happens for both permanent and temporary errors. This covers the case
when the bad addresses refer to some DNS zone whose nameservers are
unreachable. This approach is, of course, fixing the symptom and not the
disease. If <EM>sender_verify_fixup</EM> is set when <EM>sender_verify_reject</EM> is
false, Exim does not modify the message, but records in the log the fixup it
would have made.

</P>



<H2><A NAME="SEC807" HREF="spec_toc.html#TOC807">45.5 Header verification</A></H2>

<P>
<A NAME="IDX1750"></A>
<A NAME="IDX1751"></A>
Exim's sender verification options can be used to block messages with bad
envelope senders. However, if a message arrives with a null envelope sender,
that is, if the SMTP command was

<PRE>
MAIL FROM:&#60;&#62;
</PRE>

<P>
then Exim has nothing to check, and is forced to accept the message (unless it
fails another check, of course). If <EM>headers_sender_verify_errmsg</EM> is set,
then for messages that have null senders (purporting to be mail delivery error
messages), Exim does some checking of the RFC 822 headers. It looks for a valid
address in the <EM>Sender:</EM>, <EM>Reply-To:</EM>, and <EM>From:</EM> headers, and if one cannot
be found, the message is rejected, unless <EM>headers_checks_fail</EM> is false, in
which case it just makes a warning entry in the reject log.

</P>
<P>
If there are several occurrences of any of the relevant headers, they are all
checked. If any <TT>`Resent-'</TT> headers exist, it is those headers that are checked
rather than the original ones.

</P>
<P>
Unfortunately, because it has to read the message before doing this check, the
rejection happens after the end of the data, and it is known that some mailers
do not treat hard (5<EM>xx</EM>) errors correctly at this point -- they keep the
message on their spools and try again later, but that is their problem, though
it does waste some resources.

</P>
<P>
The option <EM>headers_sender_verify</EM> is also available. It insists on there
being a valid <EM>Sender:</EM>, <EM>Reply-To:</EM>, or <EM>From:</EM> header on all incoming SMTP
messages, not just those with null senders.

</P>
<P>
The <EM>sender_verify_hosts</EM> option applies to both of these header checking
options as well as to checks on envelope senders (<EM>sender_verify</EM>).

</P>
<P>
A common spamming ploy is to send syntactically invalid headers such as

<PRE>
To: @
</PRE>

<P>
The option <EM>headers_check_syntax</EM> causes Exim to check the syntax of all
headers that can contain lists of addresses (<EM>Sender:</EM>, <EM>From:</EM>, <EM>Reply-To:</EM>,
<EM>To:</EM>, <EM>Cc:</EM>, and <EM>Bcc:</EM>) on all incoming messages (both local and SMTP). This
is a syntax check only. Like the <EM>headers_sender_verify</EM> options, the
rejection happens after the end of the data, and it is also controlled by
<EM>headers_checks_fail</EM>; if that is false, a bad message is accepted, with a
warning in the reject log.

</P>



<H2><A NAME="SEC808" HREF="spec_toc.html#TOC808">45.6 Receiver verification</A></H2>

<P>
<A NAME="IDX1752"></A>
<A NAME="IDX1753"></A>
By default, Exim just checks the syntax of addresses given in the SMTP RCPT
command. This minimizes the time required for an SMTP message transfer, and
also makes it possible to provide special processing for unknown local parts in
local domains, by using a <EM>smartuser</EM> director to pass messages with unknown
local parts to a script or to another host.

</P>
<P>
Some installations prefer to check receiver addresses as they are received. If
the <EM>receiver_verify</EM> option is set, the same code that is used by the -<EM>bv</EM>
option is used to check incoming addresses from remote hosts that match
<EM>receiver_verify_hosts</EM>, whose default setting is to match all hosts.
<font color=green>
Verification consists of running the directors and routers in verify mode. As
in the case of sender verification, when an incoming address is aliased to just
one child address, in an <EM>aliasfile</EM> or a <EM>smartuser</EM> director (but <EM>not</EM>
for <EM>forwardfile</EM>), verification continues with the child address, and if that
fails to verify, the original verification also fails.
</font>

</P>
<P>
When verification fails, a permanent negative response is given to the RCPT
command. If there is a temporary failure, a temporary error is given, unless
<EM>receiver_try_verify</EM> is set, in which case the address is accepted.

</P>
<P>
It is possible to restrict the addresses that are verified to certain domains
by setting <EM>receiver_verify_addresses</EM>, and receiver verification can also be
made conditional on the sender address by setting <EM>receiver_verify_senders</EM>.
All of these options operate only when <EM>receiver_verify</EM> or
<EM>receiver_try_verify</EM> is set.

</P>

<P><HR><P>
Go to the <A HREF="spec_1.html">first</A>, <A HREF="spec_44.html">previous</A>, <A HREF="spec_46.html">next</A>, <A HREF="spec_59.html">last</A> section, <A HREF="spec_toc.html">table of contents</A>.
</BODY>
</HTML>