1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
|
<HTML>
<HEAD>
<!-- This HTML file has been created by texi2html 1.52
from spec on 25 November 2000 -->
<TITLE>Exim Specification - 49. Message processing</TITLE>
</HEAD>
<body bgcolor="#FFFFFF" text="#00005A" link="#FF6600" alink="#FF9933" vlink="#990000">
Go to the <A HREF="spec_1.html">first</A>, <A HREF="spec_48.html">previous</A>, <A HREF="spec_50.html">next</A>, <A HREF="spec_59.html">last</A> section, <A HREF="spec_toc.html">table of contents</A>.
<P><HR><P>
<H1><A NAME="SEC829" HREF="spec_toc.html#TOC829">49. Message processing</A></H1>
<P>
<A NAME="IDX1847"></A>
Exim performs various transformations on the sender and recipient addresses of
all messages that it handles, and also on the messages' header lines. Some of
these are optional and configurable, while others always take place. All of
this processing, except rewriting as a result of routing, happens when a
message is received, before it is first written to the spool.
</P>
<P>
RFC 822 makes provision for headers starting with the string <TT>`Resent-'</TT>. It
states that in general, the <TT>`Resent-'</TT> fields should be treated as containing a
set of information that is independent of the set of original fields, and that
information for one set should not automatically be taken from the other. If
Exim finds any <TT>`Resent-'</TT> headers in the message, it applies the header
transformations described below only to the <TT>`Resent-'</TT> header set, leaving the
unqualified set alone.
</P>
<P>
<H2><A NAME="SEC830" HREF="spec_toc.html#TOC830">49.1 Unqualified addresses</A></H2>
<P>
<A NAME="IDX1848"></A>
<A NAME="IDX1849"></A>
By default, Exim expects every address it receives from an external host to be
fully qualified. Unqualified addresses cause negative responses to SMTP
commands. However, because SMTP is used as a means of transporting messages
from MUAs running on personal workstations, there is sometimes a requirement to
accept unqualified addresses from specific hosts or IP networks.
</P>
<P>
Exim has two options that separately control which hosts may send unqualified
sender or receiver addresses in SMTP commands, namely
<EM>sender_unqualified_hosts</EM> and <EM>receiver_unqualified_hosts</EM>. In both cases,
if an unqualified address is accepted, it is qualified by adding the value of
<EM>qualify_domain</EM> or
<EM>qualify_recipient</EM>,
</font>
as appropriate.
</P>
<P>
<A NAME="IDX1850"></A>
<A NAME="IDX1851"></A>
Unqualified addresses are accepted only from local sources or from hosts that
match one of the <EM>receiver_unqualified</EM> or <EM>sender_unqualified</EM> options, as
appropriate.
</P>
<H2><A NAME="SEC831" HREF="spec_toc.html#TOC831">49.2 The UUCP From line</A></H2>
<P>
<A NAME="IDX1852"></A>
<A NAME="IDX1853"></A>
<A NAME="IDX1854"></A>
<A NAME="IDX1855"></A>
<A NAME="IDX1856"></A>
<A NAME="IDX1857"></A>
Messages that have come from UUCP (and some other applications) often begin
with a line containing the envelope sender and a timestamp, following the word
`From'. Examples of two common formats are:
<PRE>
From a.oakley@berlin.mus Fri Jan 5 12:35 GMT 1996
From f.butler@berlin.mus Fri, 7 Jan 97 14:00:00 GMT
</PRE>
<P>
This line precedes the RFC 822 header lines. For compatibility with Sendmail,
Exim recognizes such lines at the start of messages that are submitted to it
via the command line (that is, on the standard input). It does not recognize
such lines in incoming SMTP messages, unless the sending
host matches <EM>ignore_fromline_hosts</EM>
or the -<EM>bs</EM> option was used for a local message and <EM>ignore_fromline_local</EM>
is set. The recognition is controlled by a regular expression that is defined
by the <EM>uucp_from_pattern</EM> option, whose default value matches the two common
cases shown above and puts the address that follows `From' into $<EM>1</EM>.
</P>
<P>
When the caller of Exim for a non-SMTP message is a trusted user, the message's
sender address is constructed by expanding the contents of
<EM>uucp_sender_address</EM>, whose default value is `$1'. This is then parsed as
an RFC 822 address. If there is no domain, the local part is qualified with
<EM>qualify_domain</EM> unless it is the empty string. However, if the command line
-<EM>f</EM> option is used, it overrides the `From' line.
</P>
<P>
If the caller of Exim is not trusted, the `From' line is recognized, but the
sender address is not changed. This is also the case for incoming SMTP messages
that are permitted to contain `From' lines.
</P>
<P>
Only one `From' line is recognized. If there is more than one, the second is
treated as a data line that starts the body of the message, as it is not valid
as a header line. This also happens if a `From' line is present in an incoming
SMTP message from a source that is not permitted to send them.
</P>
<H2><A NAME="SEC832" HREF="spec_toc.html#TOC832">49.3 The Bcc header</A></H2>
<P>
<A NAME="IDX1858"></A>
If Exim is called with the -<EM>t</EM> option, to take recipient addresses from a
message's headers, it removes any <EM>Bcc:</EM> header that may exist (after extracting
its addresses), unless the message has no <EM>To:</EM> or <EM>Cc:</EM> header, in which case a
<EM>Bcc:</EM> header with no addresses is left in the message, in accordance with RFC
822. If -<EM>t</EM> is not present on the command line, any existing <EM>Bcc:</EM> header is
not removed.
</P>
<P>
If Exim is called to receive a message with the recipient addresses given on
the command line, and there is no <EM>Bcc:</EM>, <EM>To:</EM>, or <EM>Cc:</EM> header in the message,
it normally adds a <EM>To:</EM> header, listing the recipients. Some mailing list
software is known to submit messages in this way, and in this case the creation
of a <EM>To:</EM> header is not what is wanted. If the <EM>always_bcc</EM> option is set,
Exim adds an empty <EM>Bcc:</EM> header instead in this circumstance.
</P>
<H2><A NAME="SEC833" HREF="spec_toc.html#TOC833">49.4 The Date header</A></H2>
<P>
<A NAME="IDX1859"></A>
If a message has no <EM>Date:</EM> header, Exim adds one, giving the current date and
time.
</P>
<H2><A NAME="SEC834" HREF="spec_toc.html#TOC834">49.5 The Delivery-date header</A></H2>
<P>
<A NAME="IDX1860"></A>
<A NAME="IDX1861"></A>
<EM>Delivery-date:</EM> headers are not part of the standard RFC 822 header set. Exim
can be configured to add them to the final delivery of messages. (See the
generic <EM>delivery_date_add</EM> transport option.) They should not be present in
messages in transit. If the <EM>delivery_date_remove</EM> configuration option is
set (the default), Exim removes <EM>Delivery-date:</EM> headers from incoming
messages.
</P>
<H2><A NAME="SEC835" HREF="spec_toc.html#TOC835">49.6 The Envelope-to header</A></H2>
<P>
<A NAME="IDX1862"></A>
<A NAME="IDX1863"></A>
<EM>Envelope-to:</EM> headers are not part of the standard RFC 822 header set. Exim
can be configured to add them to the final delivery of messages. (See the
generic <EM>envelope_to_add</EM> transport option.) They should not be present in
messages in transit. If the <EM>envelope_to_remove</EM> configuration option is set
(the default), Exim removes <EM>Envelope-to:</EM> headers from incoming messages.
</P>
<H2><A NAME="SEC836" HREF="spec_toc.html#TOC836">49.7 The From header</A></H2>
<P>
<A NAME="IDX1864"></A>
If an incoming message does not contain a <EM>From:</EM> header, Exim adds one
containing the sender's address. This is obtained from the message's envelope
in the case of remote messages; for locally-generated messages the calling
user's login name and full name are used to construct an address, as described
in section 49.14. They are obtained from the password file entry by
calling <EM>getpwuid()</EM> (but see the <EM>unknown_login</EM> configuration option). The
address is qualified with <EM>qualify_domain</EM>.
</P>
<P>
For compatibility with Sendmail, if an incoming, non-SMTP message has a <EM>From:</EM>
header containing just the unqualified login name of the calling user, this is
replaced by an address containing the user's login name and full name as
described in section 49.14.
</P>
<H2><A NAME="SEC837" HREF="spec_toc.html#TOC837">49.8 The Message-id header</A></H2>
<P>
<A NAME="IDX1865"></A>
If an incoming message does not contain a <EM>Message-id:</EM> header, Exim constructs
one and adds it to the message. The id is constructed from Exim's internal
message id, preceded by the letter E to ensure it starts with a letter, and
followed by @ and the primary host name. Additional information can be
included in this header by setting the
<A NAME="IDX1866"></A>
<EM>message_id_header_text</EM> option.
</P>
<H2><A NAME="SEC838" HREF="spec_toc.html#TOC838">49.9 The Received header</A></H2>
<P>
<A NAME="IDX1867"></A>
A <EM>Received:</EM> header is added at the start of every message. The contents of
this header are defined by the <EM>received_header_text</EM> configuration option,
and Exim automatically adds a semicolon and a timestamp to the configured
string.
</P>
<H2><A NAME="SEC839" HREF="spec_toc.html#TOC839">49.10 The Return-path header</A></H2>
<P>
<A NAME="IDX1868"></A>
<A NAME="IDX1869"></A>
<EM>Return-path:</EM> headers are defined as something the MTA may insert when it does
the final delivery of messages. (See the generic <EM>return_path_add</EM> transport
option.) Therefore, they should not be present in messages in transit. If the
<EM>return_path_remove</EM> configuration option is set (the default), Exim removes
<EM>Return-path:</EM> headers from incoming messages.
</P>
<H2><A NAME="SEC840" HREF="spec_toc.html#TOC840">49.11 The Sender header</A></H2>
<P>
<A NAME="IDX1870"></A>
<font color=green>
For locally-originated messages, unless originated by a trusted user, any
existing <EM>Sender:</EM> header is removed. For non-trusted callers, unless
<EM>local_from_check</EM> is set false, a check is made to see if the address given
in the <EM>From:</EM> header is the correct (local) sender of the message (prefixes
and suffixes for the local part can be permitted via <EM>local_from_prefix</EM> and
<EM>local_from_suffix</EM>). If not, a <EM>Sender:</EM> header giving the true sender
address is added to the message. No processing of the <EM>Sender:</EM> header is done
for messages originating externally.
</font>
</P>
<H2><A NAME="SEC841" HREF="spec_toc.html#TOC841">49.12 The To header</A></H2>
<P>
<A NAME="IDX1871"></A>
If a message has no <EM>To:</EM>, <EM>Cc:</EM>, or <EM>Bcc:</EM> header, Exim adds an empty <EM>Bcc:</EM>
header, in accordance with RFC 822,
except when the message is being received locally with the recipients supplied
on the command line. In this case, a <EM>To:</EM> header listing the recipients is
normally added. Some mailing list software is known to submit messages in this
way, and in this case the creation of a <EM>To:</EM> header is not what is wanted. If
the <EM>always_bcc</EM> option is set, Exim adds an empty <EM>Bcc:</EM> header instead in
this circumstance. An <EM>Apparently-to:</EM> header is never added.
</P>
<H2><A NAME="SEC842" HREF="spec_toc.html#TOC842">49.13 Adding and removing headers</A></H2>
<P>
<A NAME="IDX1872"></A>
<A NAME="IDX1873"></A>
The addition and removal of headers can be specified on any of the drivers, and
also in system filter files. Changes specified in the system filter affect all
deliveries of a message.
</P>
<P>
Header changes specified on a director or router affect all addresses handled
by that driver, and also any new addresses it generates. If an address passes
through several directors and/or routers, the changes are cumulative. When a
message is processed by a transport, the message's original set of headers is
output, except for those named in any <EM>headers_remove</EM> options that the
address has encountered as it was processed, and any in the transport's own
<EM>headers_remove</EM> option. Then any new headers from any <EM>headers_add</EM> options
are output.
</P>
<H2><A NAME="SEC843" HREF="spec_toc.html#TOC843">49.14 Constructed addresses</A></H2>
<P>
<A NAME="IDX1874"></A>
<A NAME="IDX1875"></A>
When Exim constructs a sender address for a locally-generated message, it uses
the form
<PRE>
<<EM>user name</EM>> <<<EM>login</EM>>@<<EM>qualify_domain</EM>>>
</PRE>
<P>
For example:
<PRE>
Zaphod Beeblebrox <zaphod@end.univ>
</PRE>
<P>
The user name is obtained from the -<EM>F</EM> command line option if set, or
otherwise by looking up the calling user by <EM>getpwuid()</EM> and extracting the
`gecos' field from the password entry. If the `gecos' field contains an
ampersand character, this is replaced by the login name with the first letter
upper-cased, as is conventional in a number of operating systems. See the
<EM>gecos_name</EM> option for a way to tailor the handling of the `gecos' field. The
<EM>unknown_username</EM> option can be used to specify user names in cases when
there is no password file entry.
</P>
<P>
<font color=green>
In all cases, the user name is made to conform to RFC 822 by quoting all or
parts of it if necessary. In addition, if it contains any non-printing
characters, it is encoded as described in RFC 2047, which defines a way of
including non-ASCII characters in header lines. The setting of
<EM>print_topbitchars</EM> controls whether characters with the top bit set (that is,
with codes greater than 127) count as printing characters or not.
</font>
</P>
<H2><A NAME="SEC844" HREF="spec_toc.html#TOC844">49.15 Case of local parts</A></H2>
<P>
<A NAME="IDX1876"></A>
<A NAME="IDX1877"></A>
RFC 822 states that the case of letters in the local parts of addresses cannot
be assumed not to be significant. Exim preserves the case of local parts of
remote addresses. However, when it is processing an address in one of its local
domains, the case of letters in the local part is significant only when
<EM>locally_caseless</EM> is unset. This option is set by default, and this causes
Exim to lowercase local parts in local domains before processing them.
</P>
<P>
If you must have mixed-case user names in your password file, the best way to
proceed, assuming you want case-independent handling of incoming email, is to
unset <EM>locally_caseless</EM> and then set up an initial <EM>smartuser</EM> director to
convert incoming local parts to the correct case by a file lookup such as
<PRE>
new_address = ${lookup{${lc:$local_part}}cdb\
{/etc/usercased.cdb}{$value}fail}\
@$domain
</PRE>
<H2><A NAME="SEC845" HREF="spec_toc.html#TOC845">49.16 Dots in local parts</A></H2>
<P>
<A NAME="IDX1878"></A>
<A NAME="IDX1879"></A>
RFC 822 forbids empty components in local parts. That is, an unquoted local
part may not begin or end with a dot, nor have two consecutive dots in the
middle. However, it seems that many MTAs do not enforce this, so Exim permits
empty components for compatibility.
</P>
<H2><A NAME="SEC846" HREF="spec_toc.html#TOC846">49.17 Rewriting addresses</A></H2>
<P>
<A NAME="IDX1880"></A>
<A NAME="IDX1881"></A>
Rewriting of sender and recipient addresses, and addresses in headers, can
happen automatically, or as the result of configuration options, as described
in chapter 34.
The headers that may be affected by this are <EM>Bcc:</EM>, <EM>Cc:</EM>, <EM>From:</EM>,
<EM>Reply-To:</EM>, <EM>Sender:</EM>, and <EM>To:</EM>.
</P>
<P>
Automatic rewriting includes qualification, as mentioned above. The other case
in which it can happen is when an incomplete non-local domain is given. The
routing process may cause this to be expanded into the full domain name. For
example, a header such as
<PRE>
To: hare@teaparty
</PRE>
<P>
might get rewritten as
<PRE>
To: hare@teaparty.wonderland.fict.book
</PRE>
<P>
Rewriting as a result of routing is the one kind of message processing that
does not happen at input time, as it cannot be done until the address has
been routed.
</P>
<P>
Strictly, one should not do <EM>any</EM> deliveries of a message until all its
addresses have been routed, in case any of the headers get changed as a
result of routing. However, doing this in practice would hold up many
deliveries for unreasonable amounts of time, just because one address could not
immediately be routed. Exim therefore does not delay other deliveries when
routing of one or more addresses is deferred.
</P>
<P><HR><P>
Go to the <A HREF="spec_1.html">first</A>, <A HREF="spec_48.html">previous</A>, <A HREF="spec_50.html">next</A>, <A HREF="spec_59.html">last</A> section, <A HREF="spec_toc.html">table of contents</A>.
</BODY>
</HTML>
|
