.\" Copyright (c) 2005 by Rhyolite Software
.\"
.\" Permission to use, copy, modify, and distribute this software for any
.\" purpose with or without fee is hereby granted, provided that the above
.\" copyright notice and this permission notice appear in all copies.
.\"
.\" THE SOFTWARE IS PROVIDED "AS IS" AND RHYOLITE SOFTWARE DISCLAIMS ALL
.\" WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
.\" OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL RHYOLITE SOFTWARE
.\" BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES
.\" OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
.\" WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
.\" ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
.\" SOFTWARE.
.\"
.\" Rhyolite Software DCC 1.2.74-1.100 $Revision$
.\"
.Dd 2005/03/20 19:39:08
.ds volume-ds-DCC Distributed Checksum Clearinghouse
.Dt dccd SMM
.Sh NAME
.Nm dccd
.Nd Distributed Checksum Clearinghouse Daemon
.Sh SYNOPSIS
.Bk -words
.Nm dccd
.Op Fl 64dVbfFQ
.Fl i Ar server-ID
.Op Fl n Ar brand
.Op Fl h Ar homedir
.br
.Oo
.Fl a Xo
.Sm off
.Op Ar server-addr
.Op Ar ,server-port
.Xc
.Sm on
.Oc
.Op Fl I Ar host-ID
.Op Fl q Ar qsize
.br
.Oo
.Fl G Xo
.Sm off
.Op Ar on,
.Op Ar weak-body,
.Op Ar weak-IP,
.Op Ar embargo
.Op Ar ,wait
.Op Ar ,white
.Xc
.Sm on
.Oc
.br
.Oo
.Fl t Xo
.Sm off
.Op Ar type
.Xc Ar ,threshold
.Sm on
.Oc
.Oo
.Fl K Xo
.Sm off
.Op Ar no-
.Ar type
.Sm on
.Xc
.Oc
.Op Fl T Ar tracemode
.Op Fl u Ar anon-delay Ns Op Ar *inflate
.Op Fl C Ar dbclean
.Op Fl L Ar ltype,facility.level
.br
.Oo
.Fl R Xo
.Sm off
.Op Ar RL_SUB ,
.Op Ar RL_ANON ,
.Op Ar RL_ALL_ANON ,
.Op Ar RL_BUGS
.Xc
.Sm on
.Oc
.Ek
.Sh DESCRIPTION
.Nm Dccd
receives reports of checksums related to mail received by DCC clients
and queries about the total number of reports of particular checksums.
A DCC server never receives
mail, address, headers, or other information from clients, but only
cryptographically secure checksums of such information.
A DCC server cannot determine the text or other information that corresponds
to the checksums it receives.
It only acts as a clearinghouse of total counts of checksums
computed by clients.
.Pp
Each DCC server or close cluster of DCC servers is identified by a numeric
.Ar server-ID .
Each DCC client is identified by a
.Ar client-ID ,
either explicitly listed in the
.Pa ids
file or
the special anonymous client-ID.
Many computers are expected to share a single
.Ar client-ID .
A
.Ar server-ID
is less than 32768 while a
.Ar client-ID
is between 32768 and 16777215.
DCC server-IDs need be known only to DCC servers and the people running
them.
The passwords associated with DCC server-IDs should be protected,
because DCC servers listen to commands authenticated with server-IDs
and their associated passwords.
Each client that does not use the anonymous ID must know the client-ID
and password used by each of its servers.
A single client computer can use different passwords with different
server computers.
See the
.Pa ids
file.
.Pp
A whitelist of known good (or bad) sources of email prevents
legitimate mailing lists from being seen as unsolicited bulk email
by DCC clients.
The whitelist used by a DCC server is built into
the database when old entries are removed by
.Xr dbclean 8 .
Each DCC client has its own, local whitelist, and in general,
whitelists work better in DCC clients than servers.
.Pp
The effectiveness of a Distributed Checksum Clearinghouse
increases as the number of subscribers increases.
Flooding reports of checksums among DCC servers increases
the effective number of subscribers to each server.
Each
.Nm
daemon tries to maintain TCP/IP connections to the other servers
listed in the
.Pa flod
file, and send them reports containing checksums with total
counts exceeding thresholds.
Changes in the
.Pa flod
file are noticed automatically within minutes.
.Pp
Controls on report flooding are specified in the
.Pa flod
file.
Each line specifies a hostname and port number to which reports
should be flooded,
a server-ID to identify and authenticate the output stream,
a server-ID to identify and authenticate an input stream from the
same server,
and flags with each ID.
The ability to delete reports of checksums is handy, but could
be abused.
If
.Ar del
is not present among the
.Ar in-opts
options for the incoming ID,
incoming delete requests are logged and then ignored.
Floods from DCC "brands" that count only mail to
"spam traps" and whose servers use the
.Fl Q
option to count extremely "bulk" mail
should be marked with
.Ar traps .
They can be seen as counting millions of targets, so the
.Ar traps
flag on their
.Pa flod
file entry changes their incoming flooded reports counts to "many."
.Pp
.Nm Dccd
automatically checks its
.Pa flod
and
.Pa ids
files periodically.
.Xr Cdcc 8
has the server commands
.Ic new ids
and
.Ic flood check
to tell
.Nm
to check those two files immediately.
Both files are also checked for changes in response to the SIGHUP signal.
.Ss OPTIONS
The following options are available:
.Bl -tag -width 3n
.It Fl 6
enable IPv6.
The default is equivalent to
.Fl 4 .
See also the IPv4 and IPv6 options in the
.Pa flod
file.
.It Fl 4
disable IPv6.
See also
.Fl 6 .
.It Fl d
enables debugging output.
Additional
.Fl d
options increase the number of messages.
.It Fl V
displays the version of the DCC server daemon.
.It Fl b
causes the server to not detach itself from the controlling tty
or put itself into the background.
.It Fl F
uses read() and write() instead of mmap() in some cases to access the DCC
database.
It is never the default.
.It Fl f
turns off
.Fl F .
.It Fl Q
causes the server to treat reports of checksums as queries
except from DCC clients marked trusted in the
.Pa ids
file with
.Ar rpt-ok .
See
.Fl u
to turn off access by anonymous or unauthenticated clients
.It Fl i Ar server-ID
specifies the ID of this DCC server.
Each server identifies itself as responsible for checksums
that it forwards to other servers.
.It Fl n Ar brand
is an arbitrary string of letters and numbers that
identifies the organization running the DCC server.
The brand is required, and appears in the SMTP
.Em X-DCC
headers generated by the DCC.
.It Fl h Ar homedir
overrides the default DCC home directory, which is often
.Pa /var/dcc .
.It Fl a Xo
.Sm off
.Op Ar server-addr
.Op Ar ,server-port
.Sm on
.Xc
adds an hostname or IP address to the list of local IP addresses
that the server answers.
Multiple
.Fl a
options can be used to specify a subset of the available network
interfaces or to use more than one port number.
The default is to listen on all local IP addresses.
It can be useful to list some or all of the IP addresses of
multi-homed hosts to deal with local or remote firewalls.
By default
.Ar server-port
is 6277 for DCC servers and 6276 for Greylist servers.
It is the UDP port at which DCC
requests are received and the TCP port for incoming floods of reports.
.Pp
If
.Ar server-addr
is absent
and if the
.Xr getifaddrs 8
function is supported,
separate UDP sockets are bound to each configured network interface so
that each DCC clients receives replies from the
IP addresses to which corresponding request are sent.
If
.Nm
is started before all network interfaces are turned on or
there are interfaces that are turned on and off or change their addresses
such as PPP interfaces,
then the special string
.Ar @
should be used to tell
.Nm
to bind to an IN_ADDRANY UDP socket.
.Pp
Outgoing TCP connections to flood checksum reports to other DCC servers
used the IP address of a single
.Fl a
option,
but only if there is single option.
Note that this means that
.Fl a 127.0.0.1
breaks flooding, often with "Invalid argument" messages.
See also the
.Pa flod
file.
.It Fl I Ar host-ID
changes the server's globally unique identity from the default value 
consisting of the first 16 characters of the host name.
.Ar Host-ID
is a string of up to 16 characters to be used instead of the first
16 characters of the system's hostname.
.It Fl q Ar qsize
specifies the maximum size of the queue of requests from anonymous or
unauthenticated clients.
The default value is the maximum DCC RTT in seconds times 200 or 1000.
.It Fl t Xo
.Sm off
.Op Ar type
.Ar ,threshold
.Sm on
.Xc
sets the threshold below which checksum reports are not sent or flooded
to peer DCC servers.
Checksums whose total counts are less than to the number
.Ar threshold
are not flooded.
If
.Ar threshold
is the string "many," a value of millions is understood.
It must be at least 10.
.\" see BULK_THRESHOLD/2
If
.Ar type
is absent, only the thresholds for the body checksums are set.
The thresholds built into
.Nm
for the body checksums,
.Ar Body ,
.Ar Fuz1 ,
and
.Ar Fuz2
are 20.
.\" see BULK_THRESHOLD
The thresholds for the other checksums are so high by default that
by themselves they can never cause reports to be flooded.
The script commonly used to start
.Nm
sets the body thresholds to one third of
.Em DCCM_REJECT_AT
in the dcc_conf file but no less than 10 or more than 20.
That is the rejection threshold for
.Xr dccm 8 ,
.Pp
This threshold has no direct effect on which checksums are marked "bulk" by
DCC clients.
Instead, it allows cooperating DCC servers to share only the checksums
of bulk mail and reduce inter-server communications.
The thresholds should be larger than the number of addressees of
typical private email but not much larger,
because reports of checksums that
total less than their thresholds can be flooded as many extra times as there
are other thresholds.
.Pp
Reports containing any checksums marked "OK or "OK2" are
not sent to other servers.
This reduces the bandwidth needed for
the inter-server flooding, the sizes of DCC database files, and helps
protect the privacy of email of clients of a DCC server.
.Ig
.It Fl G Xo
.Sm off
.Op Ar on,
.Op Ar weak-body,
.Op Ar weak-IP,
.Op Ar embargo
.Op Ar ,wait
.Op Ar ,white
.Xc
.Sm on
changes
.Nm
to a Greylist server for
.Xr dccm 8
or
.Xr dccifd 8 .
Greylisting consists of temporarily rejecting or embargoing mail from
unfamiliar combinations of SMTP client IP address, SMTP envelope sender,
and SMTP envelope recipient.
If the SMTP client persists for
.Ar embargo seconds
and so is probably not an "open proxy," worm-infected personal computer,
or other transient source of spam, the triple of
.Em (IP\ address,sender,recipient)
is added to a database similar to the usual DCC database.
If the SMTP client does not try again after
.Ar embargo
seconds and before
.Ar wait
seconds after the first attempt,
the triple is forgotten.
If the SMTP client persists past the embargo,
the triple is added to the database and becomes familiar
and the message is accepted.
Familiar triples are remembered for
.Ar white
seconds after the last accepted mail message.
The triple is forgotten if it is ever associated with unsolicited bulk email.
.Pp
All three durations can be a number of minutes, hours, days, or
weeks followed by
.Ar MINUTES ,
.Ar M ,
.Ar HOURS ,
.Ar H ,
.Ar DAYS ,
.Ar D ,
.Ar WEEKS
or
.Ar W .
The default is
.Fl G Ar 270seconds,7days,63days .
The first duration or the
.Ar embargo
should be longer than open proxies can linger retransmitting.
The second
.Ar wait
time should be as long as legitimate mail servers persist in retransmitting
to recognize embargoed messages whose retransmissions were not
received because of network or other problems.
The
.Ar white
time should be long enough to recognize and not embargo messages from
regular senders.
.Pp
Usually the DCC greylist system requires that an almost
identical copy of the message be retransmitted during the
.Ar embargo .
If
.Ar weak-body
is present,
any message with the same triple of sender IP address, sender mail
address, and target mail address ends the embargo.
.Pp
If
.Ar weak-IP
is present,
all mail from an SMTP client at an IP address is accept
after any message from the same IP address has been accepted.
.Pp
Unlike DCC checksums, the contents of
greylist databases are private and do not benefit from broad sharing.
However, large installations can use more two or more greylist servers
flooding triples among themselves.
Flooding among greylist servers is controlled by the
.Pa grey_flod
file.
.Pp
.Em Note:
All greylist cooperating or flooding greylist servers must use
the same
.Fl G
values.
.Pp
Clients of greylist servers cannot be anonymous and must have
client-IDs and passwords assigned in the
.Pa ids
file.
.Pp
White- and blacklists are honored by the DCC clients.
White-listed messages are embargoed or checked with a greylist server.
The greylist triples of blacklisted messages, messages whose DCC counts make
them spam, and other messages known to be spam are sent to a greylist
server to be removed from the greylist database and cause an embargo
on the next messages with those triples.
.Pp
Messages whose checksums match greylist server whitelists
are not embargoed and the checksums of their triples are not
added to the greylist database.
.Pp
The target counts of embargoed messages are reported to the DCC network
to improve the detection of bulk mail.
.It Fl K Xo
.Sm off
.Op Ar no-
.Ar type
.Sm on
.Xc
marks checksums of
.Ar type
(not) be "kept"
or counted in the database unless they appear in the whitelist.
The default is equivalent to
.Fl K Ar no-all
.Fl K Ar Body
.Fl K Ar Fuz1
.Fl K Ar Fuz2
to count only the body checksums.
.It Fl T Ar tracemode
causes the server to trace or record some operations.
.Ar tracemode
must be one of the following:
.Bl -tag -width QUERY -offset 2n -compact
.It Ar ALL
all tracing
.It Ar ADMN
administrative requests from the control program,
.Xr cdcc 8
.It Ar ANON
errors by anonymous clients
.It Ar CLNT
errors by authenticated clients
.It Ar RLIM
rate-limited messages
.It Ar QUERY
all queries and reports
.It Ar RIDC
some messages concerning the report-ID cache that is used
to detect duplicate reports from clients
.It Ar FLOOD
messages about inter-server flooding
.It Ar IDS
unknown server-IDs in flooded reports
.It Ar BL
requests from clients with IP addresses in the
.Pa blacklist
file.
.El
The default is
.Ar ANON CLNT .
.It Fl u Ar anon-delay Ns Op Ar *inflate
changes the number of milliseconds anonymous or unauthenticated clients
must wait for answers to their queries and reports.
The purpose of this delay is to discourage anonymous clients..
The
.Ar anon-delay
is multiplied by 1 plus the number of recent anonymous requests from
an IP address divided by the
.Ar inflate
value.
.Pp
The string
.Ar FOREVER
turns off all anonymous or unauthenticated access not only
for checksum queries and reports but also
.Xr cdcc 8
.Ic stats
requests.
A missing value for
.Ar inflate
turns off inflation.
.Pp
The default value is
.Ar 50,none ,
except when
.Fl G
is used in which case
.Ar FOREVER
is assumed and required.
.It Fl C Ar dbclean
changes the default name or path of the program used to rebuild
the hash table when it becomes too full.
The default value is
.Pa libexec/dbclean
in the DCC home directory.
The value can include arguments as in
.Ar -C '$DCC_LIBEXEC/dbclean -F' .
.It Fl L Ar ltype,facility.level
specifies how messages should be logged.
.Ar Ltype
must be
.Ar error
or
.Ar info
to indicate which of the two types of messages are being controlled.
.Ar Level
must be a
.Xr syslog 3
level among
.Ar EMERG ,
.Ar ALERT ,
.Ar CRIT ,
.Ar ERR ,
.Ar WARNING ,
.Ar NOTICE ,
.Ar INFO ,
and
.Ar DEBUG .
.Ar Facility
must be among
.Ar AUTH ,
.Ar AUTHPRIV ,
.Ar CRON ,
.Ar DAEMON ,
.Ar FTP ,
.Ar KERN ,
.Ar LPR ,
.Ar MAIL ,
.Ar NEWS ,
.Ar USER ,
.Ar UUCP ,
and
.Ar LOCAL0
through
.Ar LOCAL7 .
The default is equivalent to
.Pp
.Dl -L info,MAIL.NOTICE  -L error,MAIL.ERR
.It Fl R Xo
.Sm off
.Op Ar RL_SUB ,
.Op Ar RL_ANON ,
.Op Ar RL_ALL_ANON ,
.Op Ar RL_BUGS
.Xc
.Sm on
sets the four categories of rate-limits.
.Ar RL_SUB
limits the number of DCC transactions per second from subscribers
or DCC clients with known client-IDs and passwords.
This limit applies to each IP address independently.
.Pp
.Ar RL_ANON
limits the number of DCC transactions per second from anonymous DCC clients.
This limit applies to each IP address independently.
It is better to use
.Fl u
than to change this value to exclude anonymous clients.
.Pp
.Ar RL_ALL_ANON
limits the number of DCC transactions per second from all anonymous DCC clients.
Its default value is set by the compile-time value of DCCD_RL_ALL_ANON.
This limit applies to all anonymous clients as a group, regardless of their
IP addresses.
.Pp
.Ar RL_BUGS
limits the number of complaints or error messages per second for all
anonymous DCC clients as a group as well as for each DCC client by IP
address.
.Pp
The default is equivalent to
.Fl R Ar 200,50,200,0.1
.El
.Sh FILES
.Bl -hang -width xvarxdcc -compact
.It Pa /var/dcc
is the DCC home directory containing data and control files.
.It Pa dcc_db
is the database of mail checksums.
.It Pa dcc_db.hash
is the mail checksum database hash table.
.It Pa grey_db
is the database of greylist checksums.
.It Pa grey_db.hash
is the greylist database hash table.
.It Pa flod
contains lines controlling DCC flooding of the form:
.br
.Bd -ragged -offset 2n -compact
.Ar host Ns Xo
.Sm off
.Op Ar ,port
.Op Ar ;src
.Sm on
.Xc
.Ar rem-ID
.Op Ar passwd-ID Op Ar o-opts Op Ar i-opts
.Ed
where absent optional values are signaled with "-" and
.Bl -hang -offset 1n -width 2n -compact
.It Ar host
is the IP address or name of a DCC server.
.It Ar port
is the name or number of the UDP port used by the server.
.It Ar src
is the IP address or host name from which the outgoing connection should come.
.It Ar rem-id
is the server-ID of the remote DCC server.
.It Ar passwd-ID
is a server-ID that is not assigned to a server,
but whose first password is used to sign
checksum reports sent to the remote system.
Either of its passwords are required with incoming reports.
If it is absent or "-", outgoing floods are signed with the first
password of the local server in the
.Pa ids
file and incoming floods must be signed with either password of
the remote server-ID.
.It Ar i-opts Li and Ar o-opts
are comma-separated lists of
.Bl -hang -offset 1n -width 2n -compact
.It Ar off
turns off flooding to the remote or local system.
.It Ar traps
indicates that
the remote sending or local receiving system has only "spam traps."
.It Ar no-del
says checksum delete requests are refused by the remote or local server
and so turns off sending or accepting delete requests, respectively.
By default, delete requests are not sent to remote servers and refused
in incoming floods.
.It Ar del
says delete requests are accepted by the remote or local server.
.It Ar no-log-del
turns off logging of incoming requests to delete checksums.
.It Ar passive
is used to tell a server outside a firewall to expect a peer
inside to create both of the pair
of input and output TCP connections used for flooding.
The peer inside the firewall should use
.Ar SOCKS
on its
.Pa flod
file entry for this system.
.It Ar SOCKS
is used to tell a server inside a firewall that it should create both
of the TCP connections used for flooding and that SOCKS protocol should
be used.
The peer outside the firewall should use
.Ar passive
on its
.Pa flod
file entry for this system.
.It Ar ID1->ID2
converts server-ID
.Ar ID1
in flooded reports to server-ID
.Ar ID2 .
Either
.Ar ID1
or
.Ar ID2
may be the string
.Sq self
to specify the server's own ID.
.Ar ID1
can be the string
.Sq all
to specify all server-IDs
or a pair of server-IDs separated by a dash to specify an inclusive range.
.Ar ID2
can be the string
.Sq ok
to send or receive reports without translation
or the string
.Sq reject
to not send outgoing or refuse incoming reports.
Only the first matching conversion is applied.
For example, when
.Sq self->ok,all->reject
is applied to a locally generated report,
the first conversion is applied and the second is ignored.
.It Ar leaf=path-len
does not send reports with paths longer than
.Ar path-len
server-IDs.
.It Ar IPv4
overrides a
.Fl 6
setting for this flooding peer.
.It Ar IPv6
overrides the
default or an explicit
.Fl 4
setting.
.It Ar vers
specifies the version of the DCC flooding protocol used by the remote
DCC server with a string such as
.Sq version2 .
.El
.El
.Pp
.It Pa grey_flod
is the equivalent of
.Pa flod used by
.Nm
when it is a greylist server.
.Pp
.It Pa flod.map
is an automatically generated file in which
.Nm
records its progress sending or flooding reports to DCC peers.
.Pp
.It Pa grey_flod.map
is the equivalent of
.Pa flod.map used by
.Nm
when it is a greylist server.
.Pp
.It Pa ids
contains the IDs and passwords known by the DCC server.
An
.Pa ids
file that can be read by others cannot be used.
It contains blank lines, comments starting
with "#" and lines of the form:
.Bd -ragged -compact -offset indent
.Sm off
.Ar id
.Op Ar ,rpt-ok
.Op Ar ,delay=ms  Ns Op Ar *inflate
.Sm on
.Ar passwd1 Op Ar passwd2
.Ed
where
.Bl -hang -offset 1n -width 2n -compact
.It Ar id
is a DCC
.Ar client-ID
or
.Ar server-ID .
.It Ar Rpt-ok
if present overrides
.Fl Q
by saying that this client is trusted
to report only checksums for unsolicited bulk mail.
.It Ar delay=ms  Ns Op Ar *inflate
delays answers to systems using the client
.Ar id .
The
.Ar delay
in milliseconds is multiplied by 1 plus the number of recent requests from
an IP address using
.Ar id
divided by the
.Ar inflate
value.
See
.Fl U .
.It Ar passwd1
is the password currently used by clients with identifier
.Ar id .
It is a 1 to 32 character string that does not contain
blank, tab, newline or carriage return characters.
.It Ar passwd2
is the optional next password that those clients will use.
A DCC server accepts either password if both are present in the file.
.El
Both passwords can be absent if the entry not used except to tell
.Nm
that server-IDs in the flooded reports are valid.
The string
.Em unknown
is equivalent to the null string.
.Pp
.It Pa whitelist
contains the DCC server whitelist.
It is not used directly but is loaded into the database when
.Xr dbclean 8
is run.
.Pp
.It Pa grey_whitelist
contains the greylist server whitelist.
It is not used directly but is loaded into the database when
.Xr dbclean 8
is run with
.Fl G .
.Pp
.It Pa blacklist
if present, contains a list of IP addresses and blocks of IP addresses
DCC clients that are ignored.
Each line in the file should be blank, a comment starting with '#',
an IP address,
or a block of IP addresses in the xxx.xxx.xxx.xxx/yy form.
Changes to the file are automatically noticed and acted upon within
a few minutes.
Addresses can be followed with comments starting with '#'.
This mechanism is intended for no more than a few dozen blocks of addresses.
.El
.Sh EXAMPLES
.Nm
is usually started with other system daemons with something like the
script misc/start-dccd.
It uses values in the file dcc_conf
in the DCC home directory to start the server.
.Pp
The following is useful for cleanly stopping the daemon:
.Pp
.Dl cdcc 'id 100; stop'
.Pp
Again, the ID of the local server must be used instead of "100."
.Pp
Unless old reports are removed from the database, it grows too large.
.Xr dbclean 8
should be run daily with script like /var/dcc/libexec/cron-dccd.
.Sh SEE ALSO
.Xr cdcc 8 ,
.Xr dcc 8 ,
.Xr dbclean 8 ,
.Xr dblist 8 ,
.Xr dccifd 8 ,
.Xr dccm 8 ,
.Xr dccproc 8 .
.Xr dccsight 8 ,
.Sh HISTORY
.Nm
is based on an idea from Paul Vixie.
It was designed and written at Rhyolite Software
starting in 2000.
This document describes version 1.2.74.
.\"  LocalWords:  cdcc DCC ids flod dbclean dblist dccm dccproc greylist
.\"  LocalWords:  whitelist