.\" Copyright (c) 2005 by Rhyolite Software
.\"
.\" Permission to use, copy, modify, and distribute this software for any
.\" purpose with or without fee is hereby granted, provided that the above
.\" copyright notice and this permission notice appear in all copies.
.\"
.\" THE SOFTWARE IS PROVIDED "AS IS" AND RHYOLITE SOFTWARE DISCLAIMS ALL
.\" WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
.\" OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL RHYOLITE SOFTWARE
.\" BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES
.\" OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
.\" WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
.\" ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
.\" SOFTWARE.
.\"
.\" Rhyolite Software DCC 1.2.74-1.70 $Revision$
.\"
.Dd 2005/03/07 18:40:05
.ds volume-ds-DCC Distributed Checksum Clearinghouse
.Dt dccproc 8 DCC
.Sh NAME
.Nm dccproc
.Nd Distributed Checksum Clearinghouse Procmail Interface
.Sh SYNOPSIS
.Nm dccproc
.Bk -words
.Op Fl VdAQCHER
.Op Fl h Ar homedir
.Op Fl m Ar map
.Op Fl w Ar whiteclnt
.Op Fl T Ar tmpdir
.Op Fl a Ar IP-address
.Op Fl f Ar env_from
.Op Fl t Ar targets
.br
.Op Fl x Ar exitcode
.Oo
.Fl c Xo
.Sm off
.Ar type,
.Op Ar log-thold,
.Ar rej-thold
.Sm on
.Xc
.Oc
.Oo
.Fl g Xo
.Sm off
.Op Ar not-
.Ar type
.Sm on
.Xc
.Oc
.Op Fl S Ar header
.Op Fl i Ar infile
.Op Fl o Ar outfile
.Op Fl l Ar logdir
.Op Fl B Ar dnsbl-option
.Op Fl L Ar ltype,facility.level
.Ek
.Sh DESCRIPTION
.Nm Dccproc
copies a complete SMTP message from standard input or a file
to standard output or another file.
As it copies the message,
it computes the DCC checksums for the message,
reports them to a DCC server, and adds
a header line to the message.
Another program such as
.Xr procmail 1
can use the added header line to filter mail.
Dccproc does not support any thresholds of its own,
because equivalent effects can be achieved with regular expressions
and you can apply dccproc several times using different DCC servers
and then score mail based what all of the DCC servers say.
.Pp
Error messages are sent to stderr as well as the system log.
Connect stderr and stdout to the same file to see errors in context,
but direct stderr to /dev/null to keep DCC error messages out of the mail.
The
.Fl i
option can also be used to separate the error messages.
.Pp
.Nm Dccproc
sends reports of checksums related to mail received by DCC clients
and queries about the total number of reports of particular checksums.
A DCC server receives no
mail, address, headers, or other information,
but only cryptographically secure checksums of such information.
A DCC server cannot determine the text or other information that corresponds
to the checksums it receives.
It only acts as a clearinghouse of counts of checksums computed by clients.
.Pp
For the sake of privacy for even the checksums of private mail,
the checksums of senders of purely internal mail or other
mail that is known to not be unsolicited bulk can be listed in a whitelist
to not be reported to the DCC server.
.Pp
When
.Xr sendmail 8
is used,
.Xr dccm 8
is a better DCC interface.
.Xr Dccifd 8
is more efficient than
.Nm
because it is a daemon, but that has costs in complexity.
See
.Xr dccsight 8
for a way to use previously computed checksums.
.Ss OPTIONS
The following options are available:
.Bl -tag -width 3n
.It Fl V
displays the version of the DCC
.Xr procmail 1
interface.
.It Fl d
enables debugging output from the DCC client library.
Additional
.Fl d
options increase the number of messages.
One causes error messages to be sent to STDERR as well as the system log.
.It Fl A
adds to existing X-DCC headers (if any)
of the brand of the current server
instead of
replacing existing headers.
.It Fl Q
only queries the DCC server about the checksums of messages
instead of reporting and then querying.
This is useful when
.Nm
is used to filter mail that has already been reported to a DCC
server by another DCC client such as
.Xr dccm 8 .
This can also be useful when applying a private white or black list to
mail that has already been reported to a DCC server.
No single mail message should be reported to a DCC
server more than once per recipient, such as would happen if
.Nm
is not given
.Fl Q
when processing a stream of mail that has already been seen by a DCC client.
Additional reports of a message
increase its apparent "bulkness."
.It Fl C
outputs only the X-DCC header
and the checksums for the message.
.It Fl H
outputs only the X-DCC header.
.It Fl E
adds lines to the start of the log file turned on with
.Fl l
and
.Fl c
describing what might have been the envelope of the message.
All of the information for the envelope lines comes from arguments to
.Nm
including
.Fl a
and
.Fl R .
No lines are generated for which no information is available,
such as the envelope recipient.
.It Fl R
says the first Received line has a standard
"name (name [IP address])..."
format and that the address is the IP address of the SMTP client
that would otherwise be provided with
.Fl a .
If the local SMTP server adds a Received line with some other format
or does not add a Received line,
the
.Fl a
option should be used.
.It Fl h Ar homedir
overrides the default DCC home directory, which is often /var/dcc.
.It Fl m Ar map
specifies a name or path of the memory mapped parameter file instead
of the default
.Pa map
in the DCC home directory.
It should be created with the
.Ic new map
operation of the
.Xr cdcc 8
command.
.It Fl w Ar whiteclnt
specifies an optional file containing SMTP client IP addresses and
SMTP headers
of mail that do not need X-DCC headers and whose checksums should not
be reported to the DCC server.
It can also contain checksums of spam.
If the pathname is not absolute, it is relative to the DCC home directory.
Thus, individual users with private whitelists usually specify them
with absolute paths.
Common whitelists shared by users must be in the DCC home directory or
one of its subdirectories and owned by the set-UID user of
.Nm dccproc .
It is useful to
.Ar include
a common or system-wide whitelist in private lists.
.Pp
The format of the
.Nm
whiteclnt file is the same as the
.Pa whitelist
file required by
.Xr dbclean 8
and
.Xr dccm 8 .
Unlike dccm, the
.Nm
whitelist is optional.
Unless
.Fl w
is used,
.Nm
uses only the whitelist in the DCC server,
which is rarely sufficient.
.Pp
Because the contents of the
.Ar whiteclnt
file are used frequently, a companion file is automatically
created and maintained.
It has the same pathname but with an added suffix of
.Ar .dccw .
It contains a memory mapped hash table of the main file.
.Pp
A local whitelist entry ("OK) or two or more semi-whitelistings ("OK2")
for one of the message's checksums prevents all of
the message's checksums from being reported to the DCC server 
and the addition of a
.Em X-DCC
header line by
.Nm dccproc .
Because it is run by or on behalf of a single user,
.Nm
ignores
.Ar env_To
entries in the
.Ar whiteclnt
file.
Users who don't want to use
.Nm
shouldn't.
.It Fl T Ar tmpdir
changes the default directory for temporary files from the default.
The default is the directory specified with
.Fl l
or the system default if there
.Fl l
is not used.
The system default is often
.Pa /tmp .
.It Fl a Ar IP-address
specifies the IP address (not the host name) of
the immediately previous SMTP client.
It is often not available.
See also
.Fl R .
.It Fl f Ar env_from
specifies the RFC\ 821 envelope "Mail\ From" value with which the
message arrived.
It is often not available.
If
.Fl f
is not present, the contents of the first Return-Path: or UNIX style
From_ header is used.
The
.Ar env_from
string is often but need not be bracketed with "<>".
.It Fl t Ar targets
specifies the number of addressees of the message if other than 1.
The string
.Ar many
instead of a number asserts that there were too many addressees
and that the message is unsolicited bulk email.
.It Fl x Ar exitcode
specifies the code or status with which
.Nm
exits if the
.Fl c
thresholds are reached or the
.Fl w Ar whiteclnt
file blacklists the message,
unless the message is whitelisted.
.Pp
The default value is EX_NOUSER.
EX_NOUSER is 67 on many systems.
Use 0 to always exit successfully.
.It Fl c Xo
.Sm off
.Ar type,
.Op Ar log-thold,
.Ar rej-thold
.Sm on
.Xc
sets logging and "spam" thresholds for checksum
.Ar type .
Each logged message placed in a separate file in the directory
specified with
.Fl l .
The checksum types are
.Ar IP ,
.Ar env_From ,
.Ar From ,
.Ar Message-ID ,
.Ar Received ,
.Ar Body ,
.Ar Fuz1 ,
and
.Ar Fuz2 .
The string
.Ar ALL
sets thresholds for all types, but is unlikely to be useful except for
setting logging thresholds.
The string
.Ar CMN
specifies the commonly used checksums
.Ar Body ,
.Ar Fuz1 ,
and
.Ar Fuz2 .
.Ar Rej-thold
and
.Ar log-thold
must be numbers, the string
.Ar NEVER ,
or the string
.Ar MANY
indicating millions of targets.
Counts from the DCC server as large as the threshold for any single type
are taken as sufficient evidence
that the message should be logged or rejected.
.Pp
.Ar Log-thold
is the threshold at which messages are logged.
It can be handy to log messages at a lower threshold to find
solicited bulk mail sources such as mailing lists.
Messages that reach at least one of their rejection thresholds or
that have complicated combinations of white- and blacklisting are
logged regardless of logging thresholds.
.Pp
.Ar Rej-thold
is the threshold at which messages are considered "bulk,"
and so should cause the X-DCC header line to contain the string "bulk"
and
.Nm
to exit with the value set by
.Fl x .
.Pp
The checksums of locally white-listed messages are not checked with
the DCC server and so only the number of targets of the current instance of
a white-listed message are compared against the thresholds.
.Pp
The default is
.Fl c Ar ALL,NEVER ,
so that nothing is discarded or logged.
A common choice is
.Fl c Ar CMN,25,50
to reject or discard
mail with common bodies except as overridden by
the whitelist of the DCC server and
.Fl g
and
.Fl w .
.It Fl g Xo
.Sm off
.Op Ar not-
.Ar type
.Sm on
.Xc
indicates that white-listed,
.Ar OK
or
.Ar OK2 ,
counts from the DCC server for a type of checksum are to be believed.
They should be ignored if prefixed with
.Ar not- .
.Ar Type
is one of the same set of strings as for
.Fl c .
Only
.Ar IP ,
.Ar env_From ,
and
.Ar From
are likely choices.
By default all three are honored,
and hence the need for
.Ar not- .
.It Fl S Ar hdr
adds to the list of substitute or locally chosen headers that
are checked with the
.Fl w Ar whiteclnt
file and sent to the DCC server.
The checksum of the last header of type
.Ar hdr
found in the message is checked.
.\"see DCC_MAX_SUB_CKS
As many as 6 different substitute headers can be specified, but only
the checksum of the first of the 6 will be sent to the DCC server.
.It Fl i Ar infile
specifies an input file for the entire message
instead of standard input.
If not absolute, the pathname is interpreted relative to the
directory in which
.Nm
was started.
.It Fl o Ar outfile
specifies an output file for the entire message including headers
instead of standard output.
If not absolute, the pathname is interpreted relative to the
directory in which
.Nm
was started.
.It Fl l Ar logdir
specifies a directory for copies of messages whose
checksum target counts exceed
.Fl c
thresholds.
The format of each file is affected by
.Fl E .
If
.Ar logdir
is not an absolute path,
it is relative to the directory in which
.Nm
is started.
If
.Ar logdir
starts with
.Em D? ,
log files are put into subdirectories of the form
.Ar logdir/JJJ
where
.Ar JJJ
is the current julian day.
.Ar H?logdir
puts logs files into subdirectories of the form
.Ar logdir/JJJ/HH
where
.Ar HH
is the current hour.
.Ar M?logdir
puts log files into subdirectories of the form
.Ar logdir/JJJ/HH/MM
where
.Ar MM
is the current minute.
See the FILES section below concerning the contents of the files.
.Pp
The directory is relative to the DCC home directory if it is not absolute
.\"
.\"
.\"
.\" syncronize changes in dccm, dccproc and dccifd man pages
.\"     it is too bad groff .so doesn't seem to work
.It Fl B Ar dnsbl-option
enables DNS blacklist checks of the SMTP client IP address, SMTP envelope
Mail_From sender domain name, and of host names in URLs in the message body.
Body URL blacklisting has far too many false positives to use on
abuse mailboxes.
It is less effective than greylisting with
.Xr dccm 8
or
.Xr dccifd 8
but can be useful in situations where
greylisting cannot be used.
.Pp
.Ar Dnsbl-option
is either of the form
.Ar set:option
or of the form
.Ar domain Ns Op Ar ,IPaddr Ns Op Ar ,bltype .
.Ar Domain
is a DNS blacklist domain such as example.com
that will be searched.
.Ar IPaddr
is the IP address in the DNS blacklist that indicates that the mail message
is spam.  127.0.0.1 is assumed if
.Ar IPaddr
is absent.
IPv6 addresses can be specified with the usual colon (:) notation.
Names can be used instead of numeric addresses.
The type of DNS blacklist
is specified by
.Ar bltype
as
.Ar name ,
.Ar IPv4 ,
or
.Ar IPv6 .
Given an envelope sender domain name or a domain name in a URL of
spam.domain.org
and a blacklist of type
.Ar name ,
spam.domain.org.example.com will be tried.
Blacklist types of
.Ar IPv4
and
.Ar IPv6
require that the domain name in a URL be resolved into an IPv4 or IPv6
address.
The address is then written as a reversed string of decimal
octets to check the DNS blacklist, as in 2.0.0.127.example.com,
.Pp
More than one blacklist can be specified.
They are searched in order.
All searching is stopped at the first positive result.
Positive results are ignored after being logged unless an
.Ar option Ar DNSBL-on
line appears in the global or per-user
.Pa whiteclnt
file.
.Pp
.Fl B Ar set:debug
sends more messages about all DNS resolutions to the system log.
.Pp
.Fl B Ar set:msg-secs=S
limits
.Nm
to
.Ar S
seconds total for checking all DNS blacklists.
The default is 20.
.Pp
.Fl B Ar set:URL-secs=S
limits
.Nm
to at most
.Ar S
seconds resolving and checking any single URL.
The default is 5.
Some spam contains dozens of URLs and that
some "spamvertised" URLs contain host names that need minutes to
resolve.
Busy mail systems cannot afford to spend minutes checking each incoming
mail message.
In order to use typical single-threaded DNS resolver libraries,
.Xr dccm 8
and
.Xr dccifd 8
use fleets of helper processes.
.Pp
.Fl B Ar set:no-envelope
says that SMTP client IP addresses and sender Mail_From domain names should
not be checked in the following blacklists.
.Fl B Ar set:envelope
restores the default for subsequently named blacklists.
.Pp
.Fl B Ar set:no-body
says that URLs in the message body should not be checked in the
in the following blacklists.
.Fl B Ar set:body
restores the default for later blacklists.
.Pp
.Fl B Ar set:no-MX
says MX servers of sender Mail_From domain names and host names in URLs
should not be checked in the following blacklists.
.Fl B Ar set:MX
restores the default.
.\"
.\"
.\"
.It Fl L Ar ltype,facility.level
specifies how messages should be logged.
.Ar Ltype
must be
.Ar error
or
.Ar info
to indicate which of the two types of messages are being controlled.
.Ar Level
must be a
.Xr syslog 3
level among
.Ar EMERG ,
.Ar ALERT ,
.Ar CRIT ,
.Ar ERR ,
.Ar WARNING ,
.Ar NOTICE ,
.Ar INFO ,
and
.Ar DEBUG .
.Ar Facility
must be among
.Ar AUTH ,
.Ar AUTHPRIV ,
.Ar CRON ,
.Ar DAEMON ,
.Ar FTP ,
.Ar KERN ,
.Ar LPR ,
.Ar MAIL ,
.Ar NEWS ,
.Ar USER ,
.Ar UUCP ,
and
.Ar LOCAL0
through
.Ar LOCAL7 .
The default is equivalent to
.Dl Fl L Ar info,MAIL.NOTICE Fl L Ar error,MAIL.ERR
Something like this turns off the log messages:
.Dl Fl L Ar notice,MAIL.debug Fl L Ar error,MAIL.DEBUG
.El
.Pp
.Nm
exits 0 on success and with the
.Fl x
value if the
.Fl c
thresholds are reached or the
.Fl w Ar whiteclnt
file blacklists the message.
If at all possible,
the input mail message is output to standard output or the
.Fl o Ar outfile
despite errors.
If possible, error messages are put into the system log instead of
being mixed with the output mail message.
The exit status is zero for errors so that the mail message
will not be rejected.
.Sh FILES
.Bl -tag -width whiteclnt -compact
.It Pa /var/dcc
DCC home directory in which other files are found.
.It Pa map
memory mapped file in the DCC home directory
of information concerning DCC servers.
.It Pa whiteclnt
contains the client whitelist in
the format described in
.Xr dcc 8 .
.It Pa whiteclnt.dccw
is a memory mapped hash table corresponding to the
.Pa whiteclnt
file.
.It Pa tmpdir
contains temporary files created and deleted as
.Nm
processes the message.
.It Pa logdir
is an optional directory specified with
.Fl l
and containing marked mail.
Each file in the directory contains one message, at least one of whose
checksums reached one of its
.Fl c
thresholds.
The entire body of the SMTP message including its header
is followed by the checksums for the message.
.El
.Sh EXAMPLES
The following
.Xr procmailrc 5
rule adds an X-DCC header to passing mail
.Bd -literal -offset 4n
:0 f
| /usr/local/bin/dccproc -ERw whiteclnt
.Ed
.Pp
This
.Xr procmailrc 5
recipe rejects mail with total counts of 10 or larger for
the commonly used checksums:
.Bd -literal -offset 4n
:0 fW
| /usr/local/bin/dccproc -ERw whiteclnt -ccmn,10
:0 e
{
    EXITCODE=67
    :0
    /dev/null
}
.Ed
.Sh SEE ALSO
.Xr cdcc 8 ,
.Xr dcc 8 ,
.Xr dbclean 8 ,
.Xr dccd 8 ,
.Xr dblist 8 ,
.Xr dccifd 8 ,
.Xr dccm 8 ,
.Xr dccsight 8 ,
.Xr mail 1 ,
.Xr procmail 1 .
.Sh HISTORY
Implementation of
.Nm
was started at Rhyolite Software in 2000.
This describes version 1.2.74.
.Sh BUGS
.Nm
uses
.Fl c
where
.Xr dccm 8
uses
.Fl t .
.\"  LocalWords:  dccproc whiteclnt dblist dccd dbclean dcc cdcc homedir