1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
|
<!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
"https://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Postfix Built-in Content Inspection</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel='stylesheet' type='text/css' href='postfix-doc.css'>
</head>
<body>
<h1><img src="postfix-logo.jpg" width="203" height="98" alt="">
Postfix Built-in Content Inspection</h1>
<hr>
<h2>Built-in content inspection introduction </h2>
<p> Postfix supports a built-in filter mechanism that examines
message header and message body content, one line at a time, before
it is stored in the Postfix queue. The filter is usually implemented
with POSIX or PCRE regular expressions, as described in the
<a href="header_checks.5.html">header_checks(5)</a> manual page. </p>
<p> The original purpose of the built-in filter is to stop an
outbreak of specific email worms or viruses, and it does this job
well. The filter has also helped to block bounced junk email,
bounced email from worms or viruses, and notifications from virus
detection systems. Information about this secondary application
is given in the <a href="BACKSCATTER_README.html">BACKSCATTER_README</a> document. </p>
<p> Because the built-in filter is optimized for stopping specific
worms and virus outbreaks, it has <a href="#limitations">limitations</a>
that make it NOT suitable for general junk email and virus detection.
For that, you should use one of the external content inspection
methods that are described in the <a href="FILTER_README.html">FILTER_README</a>, <a href="SMTPD_PROXY_README.html">SMTPD_PROXY_README</a>
and <a href="MILTER_README.html">MILTER_README</a> documents. </p>
<p> The following diagram gives an over-all picture of how Postfix
built-in content inspection works: </p>
<blockquote>
<table>
<tr>
<td colspan="4"> <td bgcolor="#f0f0ff" align="center"
valign="middle"> Postmaster<br> notifications </td>
</tr>
<tr>
<td colspan="4"> <td align="center"> <tt> |<br>v </tt></td>
</tr>
<tr>
<td bgcolor="#f0f0ff" align="center" valign="middle">
Network or<br> local users </td>
<td align="center" valign="middle"> <tt> -> </tt> </td>
<td bgcolor="#f0f0ff" align="center" valign="middle">
<b> Built-in<br> filter</b> </td>
<td align="center" valign="middle"> <tt> -> </tt> </td>
<td bgcolor="#f0f0ff" align="center" valign="middle">
Postfix<br> queue </td>
<td align="center" valign="middle"> <tt> -> </tt> </td>
<td bgcolor="#f0f0ff" align="center" valign="middle">
Delivery<br> agents </td>
<td align="center" valign="middle"> <tt> -> </tt> </td>
<td bgcolor="#f0f0ff" align="center" valign="middle">
Network or<br> local mailbox </td>
</tr>
<tr>
<td colspan="4"> <td align="center"> ^<br> <tt> | </tt> </td>
<td> </td> <td align="center"> <tt> |<br>v </tt> </td>
</tr>
<tr>
<td colspan="4"> <td colspan="3" bgcolor="#f0f0ff" align="center"
valign="middle"> Undeliverable mail<br> Forwarded mail</td>
</tr>
</table>
</blockquote>
<p> The picture makes clear that the filter works while Postfix is
receiving new mail. This means that Postfix can reject mail from
the network without having to return undeliverable mail to the
originator address (which is often spoofed anyway). However, this
ability comes at a price: if mail inspection takes too much time,
then the remote client will time out, and the client may send the
same message repeatedly. </p>
<p>Topics covered by this document: </p>
<ul>
<li><a href="#what">What mail is subjected to header/body checks </a>
<li><a href="#limitations">Limitations of Postfix header/body checks </a>
<li><a href="#daily">Preventing daily mail status reports from being blocked </a>
<li><a href="#remote_only">Configuring header/body checks for mail from outside users only</a>
<li><a href="#mx_submission">Configuring different header/body checks for MX service and submission service</a>
<li><a href="#domain_except">Configuring header/body checks for mail to some domains only</a>
</ul>
<h2><a name="what">What mail is subjected to header/body checks </a></h2>
<p> Postfix header/body checks are implemented by the <a href="cleanup.8.html">cleanup(8)</a>
server before it injects mail into the <a href="QSHAPE_README.html#incoming_queue">incoming queue</a>. The diagram
below zooms in on the <a href="cleanup.8.html">cleanup(8)</a> server, and shows that this server
handles mail from many different sources. In order to keep the
diagram readable, the sources of postmaster notifications are not
shown, because they can be produced by many Postfix daemon processes.
</p>
<blockquote>
<table>
<tr> <td colspan="2"> </td> <td bgcolor="#f0f0ff" align="center"
valign="middle"> <a href="bounce.8.html">bounce(8)</a><br> (undeliverable) </td> </tr>
<tr> <td bgcolor="#f0f0ff" align="center" valign="middle"> <b>
<a href="smtpd.8.html">smtpd(8)</a><br> (network)</b> </td> <td align="left" valign="bottom">
<tt> \ </tt> </td> <td align="center" valign="middle"> <tt> |<br>v
</tt> </td> </tr>
<tr> <td> </td> <td> </td> </tr>
<tr> <td bgcolor="#f0f0ff" align="center" valign="middle"> <b>
<a href="qmqpd.8.html">qmqpd(8)</a><br> (network)</b> </td> <td align="center" valign="middle">
<tt> -\<br>-/ </tt> </td> <td bgcolor="#f0f0ff" align="center"
valign="middle"> <a href="cleanup.8.html">cleanup(8)</a> </td> <td align="center" valign="middle">
<tt> -> </tt> </td> <td bgcolor="#f0f0ff" align="center"
valign="middle"> <a href="QSHAPE_README.html#incoming_queue">
incoming<br> queue </a> </td> </tr>
<tr> <td bgcolor="#f0f0ff" align="center" valign="middle"> <b>
<a href="pickup.8.html">pickup(8)</a><br> (local)</b> </td> <td align="left" valign="top"> <tt>
/ </tt> </td> <td align="center" valign="middle"> ^<br> <tt> |
</tt> </td> </tr>
<tr> <td colspan="2"> </td> <td bgcolor="#f0f0ff" align="center"
valign="middle"> <a href="local.8.html">local(8)</a><br> (forwarded) </td> </tr>
</table>
</blockquote>
<p> For efficiency reasons, only mail that enters from outside of
Postfix is inspected with header/body checks. It would be inefficient
to filter already filtered mail again, and it would be undesirable
to block postmaster notifications. The table below summarizes what
mail is and is not subject to header/body checks. </p>
<blockquote>
<table border="1">
<tr> <th> Message type </th> <th> Source </th> <th> Header/body checks? </th> </tr>
<tr> <td> Undeliverable mail </td> <td> <a href="bounce.8.html">bounce(8)</a> </td> <td> No </td> </tr>
<tr> <td> Network mail </td> <td> <a href="smtpd.8.html">smtpd(8)</a> </td> <td> Configurable </td> </tr>
<tr> <td> Network mail </td> <td> <a href="qmqpd.8.html">qmqpd(8)</a> </td> <td> Configurable </td> </tr>
<tr> <td> Local submission </td> <td> <a href="pickup.8.html">pickup(8)</a> </td> <td> Configurable </td> </tr>
<tr> <td> Local forwarding </td> <td> <a href="local.8.html">local(8)</a> </td> <td> No </td> </tr>
<tr> <td> Postmaster notice </td> <td> many </td> <td> No </td> </tr>
</table>
</blockquote>
<p> How does Postfix decide what mail needs to be filtered? It
would be clumsy to make the decision in the <a href="cleanup.8.html">cleanup(8)</a> server, as
this program receives mail from so many different sources. Instead,
header/body checks are requested by the source. Examples of how
to turn off header/body checks for mail received with <a href="smtpd.8.html">smtpd(8)</a>,
<a href="qmqpd.8.html">qmqpd(8)</a> or <a href="pickup.8.html">pickup(8)</a> are given below under "<a
href="#remote_only">Configuring header/body checks for mail from
outside users only</a>", "<a href="#mx_submission">Configuring
different header/body checks for MX service and submission
service</a>", and "<a href="#domain_except">Configuring
header/body checks for mail to some domains only</a>". </p>
<h2><a name="limitations">Limitations of Postfix header/body checks </a></h2>
<ul>
<li> <p> Header/body checks do not decode message headers or message
body content. For example, if text in the message body is BASE64
encoded (<a href="https://tools.ietf.org/html/rfc2045">RFC 2045</a>) then your regular expressions will have to match
the BASE64 encoded form. Likewise, message headers with encoded
non-ASCII characters (<a href="https://tools.ietf.org/html/rfc2047">RFC 2047</a>) need to be matched in their encoded
form. </p>
<li> <p> Header/body checks cannot filter on a combination of
message headers or body lines. Header/body checks examine content
one message header at a time, or one message body line at a time,
and cannot carry a decision over to the next message header or body
line. </p>
<li> <p> Header/body checks cannot depend on the recipient of a
message. </p>
<ul>
<li> <p> One message can have multiple recipients, and all recipients
of a message receive the same treatment. Workarounds have been
proposed that involve selectively deferring some recipients of
multi-recipient mail, but that results in poor SMTP performance
and does not work for non-SMTP mail. </p>
<li> <p> Some sources of mail send the headers and content ahead
of the recipient information. It would be inefficient to buffer up
an entire message before deciding if it needs to be filtered, and
it would be clumsy to filter mail and to buffer up all the actions
until it is known whether those actions need to be executed. </p>
</ul>
<li> <p> Despite warnings, some people try to use the built-in
filter feature for general junk email and/or virus blocking, using
hundreds or even thousands of regular expressions. This can result
in catastrophic performance failure. The symptoms are as follows:
</p>
<ul>
<li> <p> The <a href="cleanup.8.html">cleanup(8)</a> processes use up all available CPU time in
order to process the regular expressions, and/or they use up all
available memory so that the system begins to swap. This slows down
all incoming mail deliveries. </p>
<li> <p> As Postfix needs more and more time to receive an email
message, the number of simultaneous SMTP sessions increases to the
point that the SMTP server process limit is reached. </p>
<li> <p> While all SMTP server processes are waiting for the
<a href="cleanup.8.html">cleanup(8)</a> servers to finish, new SMTP clients have to wait until
an SMTP server process becomes available. This causes mail deliveries
to time out before they have even begun. </p>
</ul>
<p> The remedy for this type of performance problem is simple:
don't use header/body checks for general junk email and/or virus
blocking, and don't filter mail before it is queued. When performance
is a concern, use an external content filter that runs after mail
is queued, as described in the <a href="FILTER_README.html">FILTER_README</a> document. </p>
</ul>
<h2><a name="daily">Preventing daily mail status reports from being blocked </a></h2>
<p>The following is quoted from Jim Seymour's Pflogsumm FAQ at
<a href="https://jimsun.linxnet.com/downloads/pflogsumm-faq.txt">https://jimsun.linxnet.com/downloads/pflogsumm-faq.txt</a>. Pflogsumm
is a program that analyzes Postfix logs, including the logging from
rejected mail. If these logs contain text that was rejected by
Postfix <a href="postconf.5.html#body_checks">body_checks</a> patterns, then the logging is also likely to
be rejected by those same <a href="postconf.5.html#body_checks">body_checks</a> patterns. This problem does
not exist with <a href="postconf.5.html#header_checks">header_checks</a> patterns, because those are not applied
to the text that is part of the mail status report. </p>
<blockquote>
<p>You configure Postfix to do body checks, Postfix does its thing,
Pflogsumm reports it and Postfix catches the same string in the
Pflogsumm report. There are several solutions to this. </p>
<p> Wolfgang Zeikat contributed this: </p>
<blockquote>
<pre>
#!/usr/bin/perl
use MIME::Lite;
### Create a new message:
$msg = MIME::Lite->new(
From => 'your@send.er',
To => 'your@recipie.nt',
# Cc => 'some@other.com, some@more.com',
Subject => 'pflogsumm',
Date => `date`,
Type => 'text/plain',
Encoding => 'base64',
Path => '/tmp/pflogg',
);
$msg->send;
</pre>
</blockquote>
<p> Where "/tmp/pflogg" is the output of Pflogsumm. This puts Pflogsumm's
output in a base64 MIME attachment. </p>
</blockquote>
<p> Note by Wietse: if you run this on a machine that is accessible
by untrusted users, it is safer to store the Pflogsumm report in
a directory that is not world writable. </p>
<blockquote>
<p> In a follow-up to a thread in the postfix-users mailing list, Ralf
Hildebrandt noted: </p>
<blockquote> <p> "mpack does the same thing." </p> </blockquote>
</blockquote>
<p> And it does. Which tool one should use is a matter of preference.
</p>
<p> Other solutions involve additional <a href="postconf.5.html#body_checks">body_checks</a> rules that make
exceptions for daily mail status reports, but this is not recommended.
Such rules slow down all mail and complicate Postfix maintenance.
</p>
<h2><a name="remote_only">Configuring header/body checks for mail from outside users only</a></h2>
<p> The following information applies to Postfix 2.1 and later.
Earlier
Postfix versions do not support the <a href="postconf.5.html#receive_override_options">receive_override_options</a> feature.
</p>
<p> The easiest approach is to configure ONE Postfix instance with
multiple SMTP server IP addresses in <a href="master.5.html">master.cf</a>: </p>
<ul>
<li> <p> Two SMTP server IP addresses for mail from inside users
only, with header/body filtering turned off, and a local mail pickup
service with header/body filtering turned off. </p>
<pre>
/etc/postfix.<a href="master.5.html">master.cf</a>:
# ==================================================================
# service type private unpriv chroot wakeup maxproc command
# (yes) (yes) (yes) (never) (100)
# ==================================================================
1.2.3.4:smtp inet n - n - - smtpd
-o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_header_body_checks">no_header_body_checks</a>
127.0.0.1:smtp inet n - n - - smtpd
-o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_header_body_checks">no_header_body_checks</a>
pickup unix n - n 60 1 pickup
-o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_header_body_checks">no_header_body_checks</a>
</pre>
<li> <p> Add some firewall rule to prevent access to 1.2.3.4:smtp
from the outside world. </p>
<li> <p> One SMTP server address for mail from outside users with
header/body filtering turned on via <a href="postconf.5.html">main.cf</a>. </p>
<pre>
/etc/postfix.<a href="master.5.html">master.cf</a>:
# =================================================================
# service type private unpriv chroot wakeup maxproc command
# (yes) (yes) (yes) (never) (100)
# =================================================================
1.2.3.5:smtp inet n - n - - smtpd
</pre>
</ul>
<h2><a name="mx_submission">Configuring different header/body checks for MX service and submission service</a></h2>
<p> If authorized user submissions require different header/body
checks than mail from remote MTAs, then this is possible as long
as you have separate mail streams for authorized users and for MX
service. </p>
<p> The example below assumes that authorized users connect to TCP
port 587 (submission) or 465 (smtps), and that remote MTAs connect
to TCP port 25 (smtp). </p>
<p> First, we define a few "user-defined" parameters that will
override settings for the submission and smtps services. </p>
<blockquote>
<pre>
/etc/postfix/<a href="postconf.5.html">main.cf</a>:
msa_cleanup_service_name = msa_cleanup
msa_header_checks = <a href="pcre_table.5.html">pcre</a>:/etc/postfix/msa_header_checks
msa_body_checks = <a href="pcre_table.5.html">pcre</a>:/etc/postfix/msa_body_checks
</pre>
</blockquote>
<p> Next, we define msa_cleanup as a dedicated cleanup service that
will be used only by the submission and smtps services. This service
uses the <a href="postconf.5.html#header_checks">header_checks</a> and <a href="postconf.5.html#body_checks">body_checks</a> overrides that were defined
above. </p>
<blockquote>
<pre>
/etc/postfix.<a href="master.5.html">master.cf</a>:
# =================================================================
# service type private unpriv chroot wakeup maxproc command
# (yes) (yes) (yes) (never) (100)
# =================================================================
smtp inet n - n - - smtpd
msa_cleanup unix n - n - 0 cleanup
-o <a href="postconf.5.html#header_checks">header_checks</a>=$msa_header_checks
-o <a href="postconf.5.html#body_checks">body_checks</a>=$msa_body_checks
submission inet n - n - - smtpd
-o <a href="postconf.5.html#cleanup_service_name">cleanup_service_name</a>=$msa_cleanup_service_name
-o <a href="postconf.5.html#syslog_name">syslog_name</a>=postfix/submission
<i>...[see sample <a href="master.5.html">master.cf</a> file for more]...</i>
smtps inet n - n - - smtpd
-o <a href="postconf.5.html#cleanup_service_name">cleanup_service_name</a>=$msa_cleanup_service_name
-o <a href="postconf.5.html#syslog_name">syslog_name</a>=postfix/smtps
-o <a href="postconf.5.html#smtpd_tls_wrappermode">smtpd_tls_wrappermode</a>=yes
<i>...[see sample <a href="master.5.html">master.cf</a> file for more]...</i>
</pre>
</blockquote>
<p> By keeping the "msa_xxx" parameter settings in <a href="postconf.5.html">main.cf</a>, you
keep your <a href="master.5.html">master.cf</a> file simple, and you minimize the amount
of duplication. </p>
<h2><a name="domain_except">Configuring header/body checks for mail to some domains only</a></h2>
<p> The following information applies to Postfix 2.1. Earlier
Postfix versions do not support the <a href="postconf.5.html#receive_override_options">receive_override_options</a> feature.
</p>
<p> If you are an MX service provider and want to enable header/body
checks only for some domains, you can configure ONE Postfix
instance with multiple SMTP server IP addresses in <a href="master.5.html">master.cf</a>. Each
address provides a different service. </p>
<blockquote>
<pre>
/etc/postfix.<a href="master.5.html">master.cf</a>:
# =================================================================
# service type private unpriv chroot wakeup maxproc command
# (yes) (yes) (yes) (never) (100)
# =================================================================
# SMTP service for domains with header/body checks turned on.
1.2.3.4:smtp inet n - n - - smtpd
# SMTP service for domains with header/body checks turned off.
1.2.3.5:smtp inet n - n - - smtpd
-o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_header_body_checks">no_header_body_checks</a>
</pre>
</blockquote>
<p> Once this is set up you can configure MX records in the DNS
that route each domain to the proper SMTP server instance. </p>
</body>
</html>
|
