File: mailfromd.texi

package info (click to toggle)
mailfromd 9.1-1
links: PTS, VCS
area: main
in suites: forky, sid
size: 11,512 kB
sloc: ansic: 56,882; sh: 22,979; yacc: 4,130; lex: 1,428; makefile: 928; lisp: 488; awk: 393; perl: 319; sed: 25
file content (12487 lines) | stat: -rw-r--r-- 403,723 bytes
\input texinfo @c -*-texinfo-*-
@c %**start of header
@setfilename mailfromd.info
@settitle Mailfromd Manual
@c %**end of header
@setchapternewpage odd

@defcodeindex pr
@c mailfromd options
@defcodeindex op
@c mtasim options.
@defcodeindex mt
@defcodeindex kw
@defcodeindex fl

@syncodeindex fn cp
@syncodeindex vr cp
@syncodeindex ky cp
@syncodeindex mt cp
@syncodeindex pg cp
@syncodeindex tp cp
@syncodeindex op cp
@syncodeindex pr cp
@syncodeindex kw cp
@syncodeindex fl cp

@include version.texi
@include rendition.texi
@include macros.texi
@include global.texi

@ifinfo
@dircategory Email
@direntry
* Mailfromd: (mailfromd).          General-purpose mail-filtering software.
* mailfromd: (mailfromd) Invocation.  Mail Filtering and Real-time Modification daemon.
* calloutd: (mailfromd) calloutd.  A Stand-Alone Callout Daemon.
* mfdbtool: (mailfromd) mfdbtool.  Database Management Tool.
* mtasim: (mailfromd) mtasim.      MTA simulator.
* pmult: (mailfromd) pmult.        Pmilter multiplexer program.
@end direntry
@end ifinfo

@copying
Copyright @copyright{} 2005--2025 Sergey Poznyakoff

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts.  A copy of the license is included in the section entitled ``GNU
Free Documentation License''.
@end copying

@titlepage
@title Mailfromd mail filter
@subtitle version @value{VERSION}, @value{UPDATED}
@author Sergey Poznyakoff.
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@headings off
@ifnothtml
@page
@w{ }
@sp 9
@end ifnothtml
@quotation
@i{Dedico aquest treball a Lluis Llach, per obrir els nous horitzons.}
@end quotation
@ifnothtml
@w{ }
@page
@w{ }
@page
@end ifnothtml
@headings on

@ifnothtml
@page
@summarycontents
@page
@end ifnothtml
@contents

@ifnottex
@node Top
@top Mailfromd

This edition of the @cite{Mailfromd Manual}, last updated @value{UPDATED},
documents @command{mailfromd} Version @value{VERSION}.
@end ifnottex

@menu
* Preface::                 Short description of this manual; brief
                            history and acknowledgments.
* Intro::                   Introduction to Mailfromd.
* Building::                Building the Package.
* Tutorial::                Mailfromd Tutorial.
* MFL::                     The Mail Filtering Language.
* Library::                 The MFL Library Functions.
* Using MFL Mode::          Using the GNU Emacs MFL Mode.
* Mailfromd Configuration:: Configuring @command{mailfromd}.
* Invocation::              How to Start and Stop @command{mailfromd}.
* MTA Configuration::       Using @command{mailfromd} with Various @acronym{MTA}s
* calloutd::                A Stand-Alone Callout Daemon.
* mfdbtool::                A Database Management Tool.
* mtasim::                  An @acronym{MTA} simulator.
* pmult::                   Pmilter multiplexer program.
* Reporting Bugs::          How to Report a Bug.

Appendices

* Gacopyz::
* Time and Date Formats::
* Upgrading::

* Copying This Manual::  The GNU Free Documentation License.
* Concept Index::        Index of Concepts.
@ifset WEBDOC
* This Manual in Other Formats::
@end ifset

@detailmenu
 --- The Detailed Node Listing ---

Preface

* History::                 Short @command{mailfromd} history.
* Acknowledgments::         Acknowledgments.

Introduction to @command{mailfromd}

* Conventions::             Typographical conventions.
* Overview::                Mailfromd at a first glance
* SAV::                     Principles of Sender Address Verification.
* Rate Limit::              Controlling Mail Sending Rate.
* SPF::                     SPF, DKIM, and others.

Sender Address Verification.

* Limitations::

Tutorial

* Start Up::
* Simplest Configurations::
* Conditional Execution::
* Functions and Modules::
* Domain Name System::
* Checking Sender Address::
* SMTP Timeouts::
* Avoiding Verification Loops::
* HELO Domain::
* rset::
* Controlling Number of Recipients::
* Sending Rate::
* Greylisting::
* Local Account Verification::
* Databases::
* Testing Filter Scripts::
* Run Mode::
* Examining Defaults::
* Logging and Debugging::
* Runtime errors::
* Notes::

Databases

* Database Formats::
* Basic Database Operations::
* Database Maintenance::

Run Mode

* top-block::   The Top of a Script File.
* getopt::      Parsing Command Line Arguments.

Mail Filtering Language

* Comments::                    Comments.
* include::
* line::
* Generated warnings and errors::
* Pragmas::                     Pragmatic comments.
* Data Types::
* Numbers::
* Literals::
* Here Documents::
* Sendmail Macros::
* Constants::
* Variables::
* Back references::
* Handlers::
* Special handlers::            Initialization and cleanup handlers.
* Functions::                   Functions.
* Expressions::                 Expressions.
* Shadowing::                   Variable and Constant Shadowing.
* Statements::
* Conditionals::                Conditional Statements.
* Loops::                       Loop Statements.
* Exceptions::                  Exceptional Conditions and their Handling.
* Polling::                     Sender Verification Tests.
* Modules::                     Modules are Collections of Useful Functions.
* mfmod::                       Dynamically Loaded Modules.
* Preprocessor::                Input Text Is Preprocessed.
* Filter Script Example::       A Working Filter Script Explained.
* Reserved Words::              A Reference List of Reserved Words.

Pragmatic comments

* prereq::          Pragma prereq.
* stacksize::       Pragma stacksize.
* regex::           Pragma regex.
* dbprop::          Pragma dbprop.
* greylist::        Pragma greylist.
* miltermacros::    Pragma miltermacros.
* provide-callout:: Pragma provide-callout.

Constants

* Built-in constants::

Variables

* Predefined variables::

Functions

* Some Useful Functions::

Expressions

* Constant expressions::      String and Numeric Constants.
* Function calls::            A Function Call is an Expression.
* Concatenation::             String Concatenation.
* Arithmetic operations::     @samp{+}, @samp{-}, etc.
* Bitwise shifts::            @samp{<<} and @samp{>>}.
* Relational expressions::    @samp{=}, @samp{<}, etc.
* Special comparisons::       @code{matches}, @code{mx matches}, etc.
* Boolean expressions::       @code{and}, @code{or}, @code{not}.
* Precedence::                How various operators nest.
* Type casting::

Statements

* Actions::                     Actions control the handling of the mail.
* Assignments::
* Pass::
* Echo::
@c Return statement::
@c Conditionals
@c Exception handlers

Exceptional Conditions

* Built-in Exceptions::
* User-defined Exceptions::
* Catch and Throw::

Modules

* module structure::    Declaring Modules
* scope of visibility::
* import::              Require and Import

Dynamically Loaded Modules

* Loadable Library::
* Interface Module::
* mfmodnew::  Creating a Mfmod Structure

Creating a Mfmod Structure

* mfmodnew invocation::

The MFL Library Functions

* Macro access::
* Character translation::
* String transformation::
* String manipulation::
* String formatting::
* Character Type::
* I/O functions::
* Filtering functions::
* Email processing functions::
* Envelope modification functions::
* Header modification functions::
* Body Modification Functions::
* Message modification queue::
* Mail header functions::
* Mail body functions::
* EOM Functions::
* Current Message Functions::
* Mailbox functions::
* Message functions::
* Quarantine functions::
* SMTP Callout functions::
* Compatibility Callout functions::
* Internet address manipulation functions::
* DNS functions::
* Geolocation functions::
* Database functions::
* Control database::
* System functions::
* Passwd functions::
* Sieve Interface::
* Interfaces to Third-Party Programs::
* Rate limiting functions::
* Greylisting functions::
* Special test functions::
* Mail Sending Functions::
* Blacklisting Functions::
* SPF Functions::
* DKIM::
* Sockmaps::
* NLS Functions::
* Syslog Interface::
* Debugging Functions::
* Informative Functions::
* Mfmod Interface::

Message Functions

* Header functions::
* Message body functions::
* MIME functions::
* Message digest functions::

Interfaces to Third-Party Programs

* SpamAssassin::
* ClamAV::

DKIM

* Setting up a DKIM record::

Configuring @command{mailfromd}

* conf-types::        Special Configuration Data Types
* conf-base::         Base Mailfromd Configuration
* conf-preprocessor:: Preprocessor Configuration
* conf-resolver::     DNS Resolver Configuration
* conf-server::       Server Configuration
* conf-milter::       Milter Connection Configuration
* conf-debug::        Logging and Debugging configuration
* conf-timeout::      Timeout Configuration
* conf-callout::      Call-out Configuration
* conf-priv::         Privilege Configuration
* conf-database::     Database Configuration
* conf-runtime::      Runtime Constants
* conf-mailutils::    Standard Mailutils Statements

@command{Mailfromd} Command Line Syntax

* options::                     Command Line Options.
* Starting and Stopping::       How to Start and Shut Down the Daemon.

Command Line Options.

* Operation Modifiers::
* General Settings::
* Preprocessor Options::
* Timeout Control::
* Logging and Debugging Options::
* Informational Options::

Using @command{mailfromd} with Various @acronym{MTA}s

* Sendmail::
* MeTA1::
* Postfix::

@command{calloutd}

* config-calloutd::     Calloutd Configuration.
* invocation-calloutd:: Calloutd Command-Line Options.
* protocol-calloutd::   The Callout Protocol.

Calloutd Configuration

* conf-calloutd-setup:: @command{calloutd} General Setup.
* conf-calloutd-server:: The @code{server} Statement.
* conf-calloutd-log:: @command{calloutd} Logging.

@command{mfdbtool}

* Invoking mfdbtool::
* Configuring mfdbtool::

@command{mtasim} --- a testing tool

* interactive mode::
* expect commands::
* traces::
* daemon mode::
* command summary::
* option summary::

Pmilter multiplexer program.

* pmult configuration::
* pmult example::
* pmult invocation::

Pmult Configuration

* pmult-conf::     Multiplexer Configuration.
* pmult-macros::   Translating MeTA1 macros.
* pmult-client::   Pmult Client Configuration.
* pmult-debug::    Debugging Pmult.

Upgrading

* 8140-8150:: Upgrading from 8.14 to 8.15
* 8130-8140:: Upgrading from 8.13 to 8.14
* 870-880::  Upgrading from 8.7 to 8.8
* 850-860::  Upgrading from 8.5 to 8.6
* 820-830::  Upgrading from 8.2 to 8.3 (or 8.4)
* 700-800::  Upgrading from 7.0 to 8.0
* 600-700::  Upgrading from 6.0 to 7.0
* 5x0-600::  Upgrading from 5.x to 6.0
* 500-510::  Upgrading from 5.0 to 5.1
* 440-500::  Upgrading from 4.4 to 5.0
* 43x-440::  Upgrading from 4.3.x to 4.4
* 420-43x::  Upgrading from 4.2 to 4.3.x
* 410-420::  Upgrading from 4.1 to 4.2
* 400-410::  Upgrading from 4.0 to 4.1
* 31x-400::  Upgrading from 3.1.x to 4.0
* 30x-31x::  Upgrading from 3.0.x to 3.1
* 2x-30x::   Upgrading from 2.x to 3.0.x
* 1x-2x::    Upgrading from 1.x to 2.x

@end detailmenu
@end menu

@node Preface
@unnumbered Preface

  Simple Mail Transfer Protocol (@acronym{SMTP}) which is the standard
for email transmissions across the Internet was designed in the
good old days when nobody could even think of the
possibility of e-mail being abused to send tons of unsolicited
messages of dubious contents.  Therefore it lacks mechanisms that
could have prevented this abuse (@dfn{spamming}), or at least could
have made it difficult.  Attempts to introduce such mechanisms (such
as @uref{http://tools.ietf.org/html/rfc2554, @acronym{SMTP-AUTH}
extension}) are being made, but they are not in wide use yet and,
probably, their introduction will not be enough to stop the e-mail
abuse.  Spamming is today's grim reality and developers spend lots of
time and efforts designing new protection measures against it.
@command{Mailfromd} is one of such attempts.

  The package is designed to work with any @acronym{MTA} supporting
@samp{Milter} or @samp{Pmilter} protocol, such as @samp{Sendmail},
@samp{MeTA1} or @samp{Postfix}.  It allows you to:

@itemize @bullet
@item
Control whether messages come from trustworthy senders, using so
called @dfn{callout} or @dfn{Sender Address Verification} (@pxref{SAV})
mechanism.

@item
Prevent emails coming from forged addresses by use of @acronym{SPF}
mechanism (@pxref{SPF Functions}).

@item
Limit connection and/or sending rates (@pxref{Rate Limit}).

@item
Use @dfn{black-}, @dfn{white-} and @dfn{greylisting} techniques.

@item
Invoke external programs or other mail filters.
@end itemize

@menu
* History::                 Short @command{mailfromd} history.
* Acknowledgments::         Acknowledgments.
@end menu

@node History
@unnumberedsec Short history of @command{mailfromd}.

  The idea of the utility appeared in 2005, and its first version
appeared soon afterward.  Back then it was a simple implementation of
Sender Address Verification (@pxref{SAV}) for @samp{Sendmail} (hence
its name -- @command{mailfromd}) with rudimentary tuning
possibilities.

  After a short run on my mail servers, I discovered that the utility
was not flexible enough.  It took less than a month to implement a
configuration file that allowed the user to control program and data flow
during the @samp{envfrom} @acronym{SMTP} state.  The new version, 1.0,
appeared in June, 2005.

  Next major release, 1.2 (1.1 contained mostly bugfixes),
appeared two months later, and introduced @dfn{mail sending rate}
control (@pxref{Rate Limit}).

  The program evolved during the next year, and the version 2.0 was
released in September, 2006.  This version was a major change in the
main idea of the program.  Configuration file become a flexible filter
script allowing the operator to control almost all @acronym{SMTP} states.  The
program supplied in the script file was compiled into a pseudo-code at
startup, this code being subsequently evaluated each time the filter
was invoked.  This caused a considerable speed-up in comparison with
the previous versions, where the run-time evaluator was traversing the
parse tree.  This version also introduced (implicitly, at the time),
two separate data types for the entities declared in the script, which
also played its role in the speed improvement (in the previous
versions all data were considered strings).  Lots of improvements were
made in the filter language (@pxref{MFL}) itself, such
as user-defined functions, the @code{switch} statement, the @code{catch}
statement for handling run-time errors, etc.  The set of
built-in functions extended considerably.  A testsuite (using
@i{DejaGNU}) was introduced in this version.

  During this initial development period the limitations
imposed by @command{libmilter} implementation became obvious.  Finally,
I felt they were stopping further development, and decided
that @command{mailfromd} should use its own @samp{Milter}
implementation.  This new library, @command{libgacopyz} was the main
new feature of the 3.0 release, which was released in November, 2006.
Another major feature was the @option{--dump-macros} option and
the @option{macros} subcommand to @command{rc.mailfromd} script, that
were intended to facilitate configuration on the @samp{Sendmail} side.

  The development of 3.@i{x} (more properly, 3.1.@i{x}) series
concentrated mainly on bug-fixes, while the main development was done
on the next branch.

  The version 4.0 appeared on May 12, 2007.  A full
list of changes in this release is more than 500 lines long, so it is
impractical to list them here.  In particular, this version introduced
lots of new features in @acronym{MFL} syntax and the
library of useful @acronym{MFL} functions.  The runtime engine was
also improved, in particular, stack space become expandable which
eliminated many run-time errors.  This version also provided a
foundation for @acronym{MFL} module system.  The code generation
was re-implemented to facilitate introduction of object files in
future versions.  Another new features in this release include
@acronym{SPF} support and @command{mtasim} utility --- an @acronym{MTA}
simulator designed for testing @command{mailfromd} scripts
(@pxref{mtasim}).  The test suite in this version was made portable by
rewriting it in @i{Autotest}.

  Another big leap forward was the 5.0 release, which appeared on
December 26, 2008.  It largely enriched a set of available functions
(61 new functions were introduced, which amounts to 41% of all the
available functions in 5.0 release) and introduced several
improvements in the MFL itself.  Among others, function aliases and
optional arguments in user-defined functions were introduced in this
release.  The new ``run operation mode'' allowed to execute arbitrary
MFL functions from the command line.  This release also raised the
Mailutils version requirements to at least 2.0.

  Version 6.0, which was released in on 12 December, 2009, introduced
a full-fledged modular system, akin to that of Python, and quite a few
improvements to the language. such as explicit type casts,
concatenation operator, static variables, etc.

  Starting from version 7.0, the focus of further development of
@command{mailfromd} has shifted.  While previously it had been regarded
as a mail-filtering server, since then it was developed as a system for
extending @acronym{MTA} functionality in the broad sense, mail
filtering being only one of features it provides.

  Version 7.0 makes the @acronym{MFL} syntax more consistent and the
language itself more powerful.  For example, it is no longer necessary
to use prefixes before variables to dereference them.  The new
@samp{try--catch} construct allows for elegant handling of exceptions
and errors.  User-defined exceptions provide a way for programming
complex loops and recursions with non-local exits.

  This version introduces a concept of dedicated callout server.
This allows @command{mailfromd} to defer verifications for a later
time if the remote server does not response within a reasonably short
period of time (@pxref{SMTP Timeouts}).

  Six years later the version 8.0 was released.  This version was a
major rewrite of the mailfromd codebase.  It introduced a separate
callout daemon that made it possible to separate the mailfromd server
machine from machines performing callout checks.  The MFL language
was extended by a number of built-in functions.

  Since version 8.3 (2017-11-02) @command{mailfromd} uses
@samp{adns}@footnote{@uref{https://www.gnu.org/software/adns}} for
DNS queries.

  The version 8.7 released in July, 2020 introduced DKIM support.

  The version 8.15 (2022-12-11) introduced dynamically loaded MFL modules.
These modules use dynamically loaded libraries to extend the program
functionality without having to modify its code.  Several such modules
were provided as separate projects.  @xref{mfmod}.
  
  The version 9.0 released in January, 2024 introduced full IPv6
support, separated module and search paths and added several new
MFL functions.

@node Acknowledgments
@unnumberedsec Acknowledgments

  Many people need to be thanked for their assistance in developing
and debugging @command{mailfromd}.  After S.@: C.@: Johnson, I can say
that this program ``@i{owes much to a most stimulating collection of
users, who have goaded me beyond my inclination, and frequently beyond
my ability in their endless search for "one more feature".  Their
irritating unwillingness to learn how to do things my way has usually
led to my doing things their way; most of the time, they have been right.}''

  A real test for a program like @command{mailfromd} cannot be done
but in conditions of production environment.  A decision to try it
in these conditions is by no means an easy one, it requires courage
and good faith in the intentions and abilities of the author.  To
begin with, I would like to thank my contributors for these virtues.

@cindex Jan Rafaj
  Jan Rafaj has intrepidly been using @command{mailfromd} since its
early releases and invested lots of efforts in improving the program
and its documentation.  He is the author of many of the @acronym{MFL}
library functions, shipped with the package.  Some of his ideas are
still waiting in my implementation queue, while new ones are
consistently arriving.

@cindex Peter Markeloff
  Peter Markeloff patiently tested every @command{mailfromd}
release and helped discover and fix many bugs.

@cindex Zeus Panchenko
  Zeus Panchenko contributed many ideas and gave lots of helpful
comments.  He offered invaluable help in debugging and testing
@command{mailfromd} on @acronym{FreeBSD} platform.

@cindex Sergey Afonin
  Sergey Afonin proposed many improvements and new ideas.  He also
invested a lot of his time in finding bugs and testing bugfixes.

@cindex John McEleney
@cindex Ben McKeegan
  John McEleney and Ben McKeegan contributed the token bucket filter
implementation (@pxref{TBF}).

@cindex Con Tassios
  Con Tassios helped to find and fix various bugs and contributed the
new implementation of the @code{greylist} function (@pxref{greylisting
types}).

  The following people (in alphabetical order) provided bug reports
and helpful comments for various versions of the program:
@cindex Alan Dobkin
@cindex Brent Spencer
@cindex Jeff Ballard
@cindex Nacho Gonz@'alez L@'opez
@cindex Phil Miller
@cindex Simon Christian
@cindex Thomas Lynch
Alan Dobkin, Brent Spencer, Jeff Ballard, Nacho Gonz@'alez L@'opez,
Phil Miller, Simon Christian, Thomas Lynch.

@node Intro
@chapter Introduction to @command{mailfromd}

  @command{Mailfromd} is a general-purpose mail filtering daemon and a
suite of accompanying utilities for @command{Sendmail}@footnote{See
@uref{http://www.sendmail.org}}, @command{MeTA1}@footnote{See
@uref{http://www.meta1.org}}, @command{Postfix}@footnote{See
@uref{http://www.postfix.org}} or any other @acronym{MTA} that
supports @command{Milter} (or @command{Pmilter}) protocol.  It is able
to filter both incoming and outgoing messages using a filter program,
written in @dfn{mail filtering language} (@acronym{MFL}).  The daemon
interfaces with the @acronym{MTA} using @command{Milter} protocol.

  The name @command{mailfromd} can be thought of as an abbreviation for
@samp{@emph{Mail} @emph{F}iltering and @emph{R}untime
@emph{M}odification} @emph{D}aemon, with an @samp{o} for itself.
Historically, it stemmed from the fact that the original
implementation was a simple filter implementing the @dfn{sender
address verification} technique.  Since then the program has changed
dramatically, and now it is actually a language translator and
run-time evaluator providing a set of built-in and library functions
for filtering electronic mail.

  The first part of this manual is an overview, describing the features
@command{mailfromd} offers in general.

  The second part is a tutorial, which provides an introduction for
those who have not used @command{mailfromd} previously.  It moves from
topic to topic in a logical, progressive order, building on
information already explained.  It offers only the principal information
needed to master basic practical usage of @command{mailfromd}, while
omitting many subtleties.

  The other parts are meant to be used as a reference for those who
know @command{mailfromd} well enough, but need to look up some notions
from time to time.  Each chapter presents everything that needs to be
said about a specific topic.

  The manual assumes that the reader has a good knowledge of the
@acronym{SMTP} protocol and the mail transport system he uses
(@command{Sendmail} , @command{Postfix} or @command{MeTA1}).

@menu
* Conventions::             Typographical conventions.
* Overview::                Mailfromd at a first glance
* SAV::                     Principles of Sender Address Verification.
* Rate Limit::              Controlling Mail Sending Rate.
* SPF::                     SPF, DKIM, and others.
@end menu

@node Conventions
@section Typographical conventions

@cindex Texinfo
  This manual is written using Texinfo, the GNU documentation
formatting language.  The same set of Texinfo source files is used to
produce both the printed and online versions of the documentation.
@ifnotinfo
Because of this, the typographical conventions
may be slightly different than in other books you may have read.
@end ifnotinfo
@ifinfo
This section briefly documents the typographical conventions used in
this manual.
@end ifinfo

  Examples you would type at the command line are preceded by the common
shell primary prompt, @samp{$}.  The command itself is printed @kbd{in
this font}, and the output it produces @samp{in this font}, for
example:

@example
$ @kbd{mailfromd --version}
mailfromd (mailfromd @value{VERSION})
@end example

In the text, the command names are printed @command{like this},
command line options are displayed in @option{this font}.  Some
notions are emphasized @emph{like this}, and if a point needs to be made
strongly, it is done @strong{this way}.  The first occurrence of
a new term is usually its @dfn{definition} and appears in the same
font as the previous occurrence of ``definition'' in this sentence.
File names are indicated like this: @file{/path/to/ourfile}.

The variable names are represented @var{like this}, keywords and
fragments of program text are written in @code{this font}.

@node Overview
@section Overview of Mailfromd

  In contrast to the most existing milter filters,
@command{mailfromd} does not implement any default filtering
policies.  Instead, it depends entirely on a @dfn{filter script},
supplied to it by the administrator.  The script, written in a
specialized and simple to use language, called @acronym{MFL}
(@pxref{MFL}), is supposed to run a set of tests and to decide
whether the message should be accepted by the @acronym{MTA} or not.
To perform the tests, the script can examine the values of
@command{Sendmail} macros, use an extensive set of built-in
and library functions, and invoke user-defined functions.

@node SAV
@section Sender Address Verification.

@cindex sender address verification, described
@cindex callout, described
  @dfn{Sender address verification}, or @dfn{callout}, is one of the
basic mail verification techniques, implemented by
@command{mailfromd}.  It consists in probing each @acronym{MX} server
for the given address, until one of them gives a definite (positive or
negative) reply.  Using this technique you can block a sender address
if it is not deliverable, thereby cutting off a large amount of spam.
It can also be useful to block mail for undeliverable recipients, for
example on a mail relay host that does not have a list of all the
valid recipient addresses.  This prevents undeliverable junk mail from
entering the queue, so that your @acronym{MTA} doesn't have to waste
resources trying to send @samp{MAILER-DAEMON} messages back.

  Let's illustrate how it works on an example:

@anchor{standard verification}
@cindex standard address verification
@cindex probe message
@cindex account probing
  Suppose that the user @samp{<jsmith@@somedomain.net>} is trying to
send mail to one of your local users.  The remote machine connects to
your @acronym{MTA} and issues @code{MAIL FROM: <jsmith@@somedomain.net>}
command.  However, your @acronym{MTA} does not have to take its word for it, so
it uses @command{mailfromd} to verify the sender address
validity.  @command{Mailfromd} strips the domain name from the address
(@samp{somedomain.net}) and queries @acronym{DNS} about @samp{MX} records for that
domain.  Suppose, it receives the following list

@multitable @columnfractions 0.2 0.7
@item 10 @tab relay1.somedomain.net
@item 20 @tab relay2.somedomain.net
@end multitable

  It then connects to first @acronym{MX} server, using @acronym{SMTP}
protocol, as if it were going to send a message to
@samp{<jsmith@@somedomain.net>}.  This is called sending a
@dfn{probe message}.  If the server accepts the recipient address, the
@command{mailfromd} accepts the incoming mail.  Otherwise, if the
server rejects the address, the mail is rejected as well.  If the @acronym{MX}
server cannot be connected, @command{mailfromd} selects next server
from the list and continues this process until it finds the
answer or the list of servers is exhausted.

  The @dfn{probe message} is like a normal mail except that no data
are ever being sent.  The probe message transaction in our example
might look as follows (@samp{S:} meaning messages sent by remote
@acronym{MTA}, @samp{C:} meaning those sent by @command{mailfromd}):

@example
C: HELO mydomain.net
S: 220 OK, nice to meet you
C: MAIL FROM: <>
S: 220 <>: Sender OK
C: RCPT TO: <jsmith@@somedomain.net>
S: 220 <jsmith@@remote.net>: Recipient OK
C: QUIT
@end example

  Probe messages are never delivered, deferred or bounced; they are
always discarded.

@cindex strict address verification
  The described method of address verification is called
a @dfn{standard} method throughout this document.  @command{Mailfromd}
also implements a method we call @dfn{strict}.  When using strict
method, @command{mailfromd} first resolves @acronym{IP} address of sender
machine to a fully qualified domain name.  Then it obtains @samp{MX} records
for this machine, and then proceeds with probing as described above.

  So, the difference between the two methods is in the set of @samp{MX}
records that are being probed: standard method queries @samp{MX}s based on
the sender email domain, strict method works with @samp{MX}s for the sender
@acronym{IP} address.

  Strict method allows to cut off much larger amount of spam,
although it does have many drawbacks.  Returning to our example above,
consider the following situation: @samp{<jsmith@@somedomain.net>} is a
perfectly normal address, but it is being used by a spammer from some
other domain, say @samp{otherdomain.com}.  The standard method is not able
to cope with such cases, whereas the strict one is.

  An alert reader will ask: what happens if @command{mailfromd} is
not able to get a definite answer from any of @acronym{MX} servers?  Actually,
it depends entirely on how you will instruct it to act in this case,
but the general practice is to return temporary failure, which will
urge the remote party to retry sending their message later.

@cindex caching @acronym{DNS} requests
  After receiving a definite answer, @command{mailfromd} will
cache it in its database, so that next time your @acronym{MTA} receives a
message from that address (or from the sender @acronym{IP}/email address pair,
for strict method), it will not waste its time trying to reach @acronym{MX}
servers again.  The records remain in the cache database for a certain
time, after which they are discarded.

@menu
* Limitations::
@end menu

@node Limitations
@subsection Limitations of Sender Address Verification

@cindex sender address verification, limitations
  Before deciding whether and how to use sender address
verification, you should be aware of its limitations.

  Both standard and strict methods suffer from the following
limitations:

@itemize @bullet
@item The sender verification methods will perform poorly on highly
loaded sites.  The traffic and/or resource usage overhead may not be
feasible for you.  However, you may experiment with various
@command{mailfromd} options to find an optimal configuration.

@item Some sites may blacklist your @acronym{MTA} if it probes them too
often.  @command{Mailfromd} eliminates this drawback by using a
@dfn{cache database}, which keeps results of the recent callouts.

@item When verifying the remote address, no attempt to actually
deliver the message is made.  If @acronym{MTA} accepts the address,
@command{mailfromd} assumes it is OK.  However in reality, a mail for a
remote address can bounce @emph{after} the nearest @acronym{MTA} accepts the
recipient address.

  This drawback can often be avoided by combining sender address
verification with greylisting (@pxref{Greylisting}).

@item If the remote server rejects the address, no attempt is being
made to discern between various reasons for rejection (client
rejected, @samp{HELO rejected}, @samp{MAIL FROM} rejected, etc.)

@item Some major sites such as @indicateurl{yahoo.com} do not
reject unknown addresses in reply to the @samp{RCPT TO} command, but report a
delivery failure in response to end of @samp{DATA} after a message is
transferred.  Of course, sender address verification does not work with such
sites.  However, a combination of address verification and greylisting
(@pxref{Greylisting}) may be a good choice in such cases.

@end itemize

  In addition, strict verification breaks forward mail
delivery.  This is obvious, since mail forwarding is based on
delivering unmodified message to another location, so the sender
address domain will most probably not be the same as that of the @acronym{MTA}
doing the forwarding.

@node Rate Limit
@section Controlling Mail Sending Rate.

@cindex mail sending rate, explained
@cindex sending rate, explained
  @dfn{Mail Sending Rate} for a given identity is defined as the number of
messages with this identity received within a predefined interval of
time.

  @acronym{MFL} offers a set of functions for limiting mail sending
rate (@pxref{Rate limiting functions}), and for controlling broader
rate aspects, such as data transfer rates (@pxref{TBF}).

@node SPF
@section SPF, DKIM, and others
@cindex @acronym{SPF}
@cindex Sender Policy Framework
  @dfn{Sender Policy Framework}, or @acronym{SPF} for short, is an
extension to @acronym{SMTP} protocol that allows to identify forged
identities supplied with the @code{MAIL FROM} and @code{HELO}
commands.  The framework is explained in detail in @acronym{RFC} 4408
(@uref{http://tools.ietf.org/html/rfc4408}) and on the
@uref{http://www.openspf.org/, SPF Project Site}.

  Mailfromd provides a set of functions for using @acronym{SPF} to
control mail flow.  These are described in @ref{SPF Functions}.

@cindex DKIM
  @dfn{DomainKeys Identified Mail} (@acronym{DKIM}) is an email
authentication method designed to detect forged sender addresses in
emails.  Mailfromd supports both @acronym{DKIM} signing and
verification.  @xref{DKIM}, for a detailed description of these
features.

  Mailfromd also provides support for some third-party
spam-abatement programs, namely @command{SpamAssassin},
and @command{ClamAV}.  These are discussed in
@ref{Interfaces to Third-Party Programs}.

@node Building
@chapter Building the Package

@cindex building @command{mailfromd}
@cindex @command{mailfromd}, building
  This chapter contains a detailed list of steps you need to
undertake in order to configure and build the package.

@enumerate 1
@item Make sure you have the necessary software installed.

  To build @command{mailfromd} you will need to have following
packages on your machine:

@enumerate A
@item GNU mailutils version 3.3 or newer.
@cindex mailutils

GNU mailutils is a general-purpose library for handling electronic mail.
It is available from @uref{http://mailutils.org}.

@item GNU adns library, version 1.5.1 or newer.
@cindex adns

GNU adns is an advanced DNS client library.  The recent version can be
downloaded from @uref{http://www.chiark.greenend.org.uk/~ian/adns/adns.tar.gz}.
Visit @uref{http://www.gnu.org/software/adns}, for more information.

@item A @acronym{DBM} library.
@anchor{DBM}
@cindex @acronym{DBM}
@cindex Berkeley DB
@cindex @acronym{GDBM}
@cindex @option{--with-dbm}, @command{configure} option
@command{Mailfromd} is able to link with any flavor of
@acronym{DBM} supported by GNU mailutils.  As of version
@value{VERSION} it will refuse to build without @acronym{DBM}.
By default, @command{configure} will try to
find the best implementation installed on your machine (preference is
given to Berkeley DB) and will use it.  You can, however, explicitly
specify which implementation you want to use.  To do so, use the
@option{--with-dbm} configure option. Its argument specifies the
@dfn{type} of database to use. It must be one of the types supported
by GNU mailutils. At the time of this writing, these are:

@table @asis
@item bdb
Berkeley DB (versions 2 to 6).
@item gdbm
GNU DBM.
@item kc
Kyoto Cabinet
@item tc
Tokyo Cabinet
@item ndbm
NDBM
@end table

To check what database types are supported by your version of
mailutils, run the following command:

@example
$ @kbd{mailutils dbd gdbm kc tc ndbm}
@end example

For backward compatibility, @command{configure} accepts the following
two options:

@table @option
@item --with-gdbm
Same as @option{--with-dbm=gdbm}.
@item --with-berkeley-db
Same as @option{--with-dbm=bdb}.
@end table

@cindex confMAPDEF, @command{Sendmail} macro
  For @command{Sendmail} users, it often makes sense to configure
@command{mailfromd} to use the same database flavor as @command{sendmail}.
The following table will help you do that.  The column @samp{DB type} lists
types of @acronym{DBM} databases supported by @command{mailfromd}.
The column @samp{confMAPDEF} lists the value of @code{confMAPDEF} Sendmail
configuration macro corresponding to that database type. The column
@samp{configure option} contains the corresponding option to configure.

@multitable @columnfractions 0.25 .25 .50
@headitem DB type @tab confMAPDEF @tab configure option
@item NDBM        @tab @option{-NNDBM}     @tab @option{--with-dbm=ndbm}
@item Berkeley DB @tab @option{-NNEWDB}    @tab @option{--with-dbm=bdb}
@item GDBM        @tab N/A                 @tab @option{--with-dbm=gdbm}
@end multitable

@end enumerate

@anchor{default user privileges}
@cindex default user privileges
@cindex @code{DEFAULT_USER}, @command{configure} variable
@item Decide what user privileges will be used to run @command{mailfromd}

  After startup, the program drops root privileges.  By default,
it switches to the privileges of user @samp{mail}, group @samp{mail}.
If there is no such user on your system, or you wish to use another
user account for this purpose, override it using @var{DEFAULT_USER}
environment variable.  For example for @command{mailfromd} to run as
user @samp{nobody}, use

@example
./configure DEFAULT_USER=nobody
@end example

@noindent
The user name can also be changed at run-time (@pxref{--user}).

@cindex @option{--prefix}, @command{configure} option
@item Decide where to install @command{mailfromd} and where its
filter script and data files will be located.

  As usual, the default value for the installation prefix is
@file{/usr/local}.  If it does not suit you, specify another location
using @option{--prefix} option, e.g.: @samp{--prefix=/usr}.

  During installation phase, the build system will install several
files.  These files are:

@table @file
@item @var{prefix}/sbin/mailfromd
Main daemon.  @xref{Invocation,, mailfromd}.

@anchor{default script file}
@item @var{prefix}/etc/mailfromd.mfl
Default main filter script file.  It is installed only if it is
not already there.  Thus, if you are upgrading to a newer version of
@command{mailfromd}, your old script file will be preserved
with all your changes.

@xref{MFL}, for a description of the mail filtering language.

@item @var{prefix}/share/mailfromd/@value{VERSION}/*.mfl
@acronym{MFL} modules.  @xref{Modules}.

@item @var{prefix}/share/mailfromd/@value{VERSION}/include/pp-setup
Default preprocessor setup file.  @xref{Preprocessor}.

@item @var{prefix}/info/mailfromd.info*
Documentation files.

@item @var{prefix}/bin/mtasim
@acronym{MTA} simulator program for testing @command{mailfromd}
scripts.  @xref{mtasim}.

@item @var{prefix}/sbin/pmult
Pmilter multiplexor for @command{MeTA1}.  @xref{pmult}.
It is build only if @command{MeTA1} version @samp{PreAlpha29.0} or
newer is installed on the system.  You may disable it by using the
@option{--disable-pmilter} command line option.

When testing for @command{MeTA1} presence, @command{configure} assumes
its default location.  If it is not found there, inform
@command{configure} about its actual location by using the following
option:

@example
--enable-pmilter=@var{prefix}
@end example

@noindent
where @var{prefix} stands for the @command{MeTA1} installation prefix.
@end table

@cindex @option{--sysconfdir}, @command{configure} option
  It is advisable to use the same settings for file name prefixes
as those you used when configuring @command{mailutils}.  In particular,
try to use the same @option{--sysconfdir}, since it will facilitate
configuring the whole system.

@anchor{statedir}
@cindex local state directory
@cindex DEFAULT_STATE_DIR, @command{configure} variable
  Another important point is location of @dfn{local state
directory}, i.e. a directory where @command{mailfromd} keeps its
data files (e.g. communication socket, @acronym{PID}-file and database
files).  By default, its full name is
@file{@var{localstatedir}/mailfromd}.  You can change it by setting
@code{DEFAULT_STATE_DIR} configuration variable.  This value can be
changed at run-time using the @code{state-directory} configuration
statement (@pxref{conf-base, state-directory}).  To inspect the
actual value of the local state directory, run

@example
mailfromd --no-config --show-defaults | grep '^statedir:'
@end example

@xref{Examining Defaults}, for information about the
@option{--show-defaults} option.

@item Select default communication socket.
@cindex default communication socket
@cindex default communication port
@cindex DEFAULT_SOCKET, @command{configure} variable
  This is the socket used to communicate with @acronym{MTA}, in
the usual @command{Milter} port notation (@pxref{milter port specification}).
If the socket name does not begin with a protocol or directory
separator, it is assumed to be a @acronym{UNIX} socket, located in the
local state directory.  The default value is @file{mailfrom}, which is
equivalent to @file{unix:@var{localstatedir}/mailfromd/mailfrom}.

 To alter this, use @code{DEFAULT_SOCKET} environment variable, e.g.:

@example
./configure DEFAULT_SOCKET=inet:999@@localhost
@end example

@noindent
The communication socket can be changed at run time using
@option{--port} command line option (@pxref{--port}) or the
@code{listen} configuration statement (@pxref{conf-server, listen}).

@item Select default expiration interval.
@cindex default expiration interval
@cindex DEFAULT_EXPIRE_INTERVAL, @command{configure} variable
  @dfn{Expiration interval} defines the period of time during which a
record in the @command{mailfromd} database is considered valid.  It is
described in more detail in @ref{Databases}.  The default value is
86400 seconds, i.e. 24 hours.  It is @sc{ok} for most sites.  If, however,
you wish to change it, use @var{DEFAULT_EXPIRE_INTERVAL} environment
variable.

@cindex DEFAULT_EXPIRE_RATES_INTERVAL, @command{configure} variable
  The @code{DEFAULT_EXPIRE_RATES_INTERVAL} variable sets default expiration time for mail rate database (@pxref{Rate limiting functions}).

Expiration settings can be changed at run time using
@code{database} statement in the @command{mailfromd} configuration file
(@pxref{conf-database}).

@cindex enable-syslog-async, @option{--enable-syslog-async}, @command{configure} option
@cindex syslog, non-blocking
@item Select a @command{syslog} implementation to use.

@anchor{syslog-async}
  @command{Mailfromd} uses @code{syslog} for diagnostics output.  The
default @code{syslog} implementation on most systems (most notably, on
GNU/Linux) uses blocking @code{AF_UNIX SOCK_DGRAM} sockets.  As a
result, when an application calls @code{syslog()}, and
@command{syslogd} is not responding and the socket buffers get full,
the application will hang.

@cindex Simon Kelley
  For @command{mailfromd}, as for any daemon, it is more important
that it continue to run, than that it continue to log.  For this
purpose, @command{mailfromd} is shipped with a non-blocking
@code{syslog} implementation by Simon Kelley.  This implementation,
instead of blocking, buffers log lines in memory.  When the buffer log
overflows, some lines are lost, but the daemon continues to run.  When
lines are lost, this fact is logged with a message of the form:

@example
   async_syslog overflow: 5 log entries lost
@end example

  To enable this implementation, configure the package with
@option{--enable-syslog-async} option, e.g.:

@example
./configure --enable-syslog-async
@end example

@cindex @code{DEFAULT_SYSLOG_ASYNC}, @command{configure} variable
  Additionally, you can instruct @command{mailfromd} to use
asynchronous syslog by default.  To do so, set
@code{DEFAULT_SYSLOG_ASYNC} to 1, as shown in example below:

@example
./configure --enable-syslog-async DEFAULT_SYSLOG_ASYNC=1
@end example

  You will be able to override these defaults at run-time by using
the @option{--logger} command line option
(@pxref{Logging and Debugging}).

@item Run @command{configure} with all the desired options.

  For example, the following command:

@example
./configure DEFAULT_SOCKET=inet:999@@localhost --with-berkeley-db=3
@end example

@noindent
will configure the package to use Berkeley DB database, version 2,
and @samp{inet:999@@localhost} as the default communication socket.

  At the end of its run @command{configure} will print a concise
summary of its configuration settings.  It looks like that (with the
long lines being split for readability):

@example
@group
*******************************************************************
Mailfromd configured with the following settings:

Mailutils version......................... 3.15
External preprocessor..................... /usr/bin/m4 -s
DBM version............................... Berkeley DB v. 3
Default user.............................. mail
State directory...........................
           $(localstatedir)/$(PACKAGE)
Socket.................................... mailfrom
Expiration interval....................... 86400
Compile asynchronous syslog............... no
Readline (for mtasim)..................... yes
Documentation rendition type.............. PROOF
Enable pmilter support.................... no
Enable GeoIP2 support..................... no
Enable DKIM support....................... yes
IPv6 support.............................. yes
*******************************************************************
@end group
@end example

  Make sure these settings satisfy your needs.  If they do not,
reconfigure the package with the right options.

@item Run @command{make}.

@item Run @command{make} install.

@item Make sure @file{@var{localstatedir}/mailfromd} has the right owner
and mode.

@item Examine filter script file
(@file{@var{sysconfdir}/mailfromd.mfl}) and edit it, if necessary.

@item If you are upgrading from an earlier release of Mailfromd, refer to
@ref{Upgrading}, for detailed instructions.

@end enumerate

@node Tutorial
@chapter Tutorial

  This chapter contains a tutorial introduction, guiding you
through various @command{mailfromd} configurations, starting from the
simplest ones and proceeding up to more advanced forms.  It omits
most complicated details, concentrating mainly on the
common practical tasks.

  If you are familiar with @command{mailfromd}, you can skip this
chapter and go directly to the next one (@pxref{MFL}), which contains
detailed discussion of the mail filtering language and
@command{mailfromd} interaction with the Mail Transport Agent.

@menu
* Start Up::
* Simplest Configurations::
* Conditional Execution::
* Functions and Modules::
* Domain Name System::
* Checking Sender Address::
* SMTP Timeouts::
* Avoiding Verification Loops::
* HELO Domain::
* rset::
* Controlling Number of Recipients::
* Sending Rate::
* Greylisting::
* Local Account Verification::
* Databases::
* Testing Filter Scripts::
* Run Mode::
* Examining Defaults::
* Logging and Debugging::
* Runtime errors::
* Notes::
@end menu

@node Start Up
@section Start Up

@cindex @acronym{MTA}
@cindex Mail Transfer Agent (@acronym{MTA})
  The @command{mailfromd} utility runs as a standalone @dfn{daemon}
program and listens on a predefined communication channel for requests
from the @dfn{Mail Transfer Agent} (@acronym{MTA}, for short).  When
processing each message, the @acronym{MTA} installs communication with
@command{mailfromd}, and goes through several states, collecting the
necessary data from the sender.  At each state it sends the relevant
information to @command{mailfromd}, and waits for it to reply.  The
@command{mailfromd} filter receives the message data through
@dfn{Sendmail macros} and runs a @dfn{handler program}
defined for the given state.  The result of this run is a @dfn{response
code}, that it returns to the @acronym{MTA}.  The following response
codes are defined:

@table @code
@anchor{continue}
@item continue
@cindex continue action, introduced
Continue message processing from next milter state.

@anchor{accept}
@item accept
@cindex accept action, introduced
Accept this message for delivery.  After receiving this code the
@acronym{MTA} continues processing this message without
further consulting @command{mailfromd} filter.

@anchor{reject}
@item reject
@cindex reject action, introduced
Reject this message.  The message processing stops at this stage, and the
sender receives the reject reply (@samp{5@var{xx}} reply code).  No
further @command{mailfromd} handlers are called for this message.

@anchor{discard}
@item discard
@cindex discard action, introduced
Silently discard the message.  This means that @acronym{MTA} will
continue processing this message as if it were going to deliver it,
but will discard it after receiving.  No further interaction with
@command{mailfromd} occurs.

@anchor{tempfail}
@item tempfail
@cindex tempfail action, introduced
Temporarily reject the message.  The message processing stops at this
stage, and the sender receives the @samp{temporary failure} reply
(@samp{4@var{xx}} reply code).  No further @command{mailfromd}
handlers are called for this message.
@end table

@cindex filter script, described
  The instructions on how to process the message are supplied to
@command{mailfromd} in its @dfn{filter script file}.  It is normally
called @file{/usr/local/etc/mailfromd.mfl} (but can be located elsewhere,
@pxref{Invocation}) and contains a set of @dfn{milter state handlers},
or subroutines to be executed in various @acronym{SMTP} states.  Each
interaction state can be supplied its own handling procedure.  A
missing procedure implies @code{continue} response code.

@anchor{milter state}
@cindex milter state handler, described
@cindex handler, described
@cindex connect, handler
@cindex helo, handler
@cindex envfrom, handler
@cindex envrcpt, handler
@cindex data, handler
@cindex header, handler
@cindex eoh, handler
@cindex body, handler
@cindex eom, handler
@cindex begin, special handler
@cindex end, special handler
@anchor{handler names}
  The filter script can define up to nine @dfn{milter state handlers},
called after the names of milter states: @samp{connect}, @samp{helo},
@samp{envfrom}, @samp{envrcpt}, @samp{data}, @samp{header}, @samp{eoh},
@samp{body}, and @samp{eom}.  The @samp{data} handler is invoked only
if @acronym{MTA} uses Milter protocol version 3 or later.  Two special
handlers are available for initialization and clean-up purposes:
@samp{begin} is called before the processing starts, and @samp{end} is
called after it is finished.  The diagram below shows the control flow
when processing an @acronym{SMTP} transaction.  Lines marked with
@code{C:} show @acronym{SMTP} commands issued by the remote machine (the
@dfn{client}), those marked with @samp{@result{}} show called handlers
with their arguments.  An @samp{[R]} appearing at the start of a line
indicates that this part of the transaction can be repeated any number
of times:

@float Figure, milter-control-flow
@caption{Mailfromd Control Flow}
@example
@group
@result{} begin()
@result{} connect(@var{hostname}, @var{family}, @var{port}, @samp{IP address})
C: HELO @var{domain}
helo(@var{domain})
for each message transaction
do
        C: MAIL FROM @var{sender}
        @result{} envfrom(@var{sender})

[R]     C: RCPT TO @var{recipient}
        @result{} envrcpt(@var{recipient})

        C: DATA
        @result{} data()
[R]     C: @var{header}: @var{value}
        @result{} header(@var{header}, @var{value})

        C:
        @result{} eoh()

[R]     C: @var{body-line}
        @result{} /* @r{Collect lines into blocks @var{blk} of}
        @result{}  * @r{at most @var{len} bytes and for each}
        @result{}  * @r{such block call:}
        @result{}  */
        @result{} body(@var{blk}, @var{len})

        C: .
        @result{} eom()
done
@result{} end()
@end group
@end example
@end float

  This control flow is maintained for as long as each called handler
returns @code{continue} (@pxref{Actions}).  Otherwise, if
any handler returns @code{accept} or @code{discard}, the message
processing continues, but no other handler is called.  In the case
of @code{accept}, the @acronym{MTA} will accept the message for
delivery, in the case of @code{discard} it will silently discard it.

  If any of the handlers returns @code{reject} or @code{tempfail}, the
result depends on the handler.  If this code is returned by
@code{envrcpt} handler, it causes this particular recipient address to
be rejected.  When returned by any other handler,
it causes the whole message will be rejected.

  The @code{reject} and @code{tempfail} actions executed by
@code{helo} handler do not take effect immediately.  Instead, their
action is deferred until the next @acronym{SMTP} command from the
client, which is usually @code{MAIL FROM}.

@node Simplest Configurations
@section Simplest Configurations

@cindex handler declaration
@cindex milter state handler, declaring
@cindex state handler, declaring
@cindex declaring milter state handler
  The @command{mailfromd} script file contains a
series of @dfn{declarations} of the handler procedures.  Each
declaration has the form:

@example
@group
prog @var{name}
do
  @dots{}
done
@end group
@end example

@noindent
where @code{prog}, @code{do} and @code{done} are the @dfn{keywords},
and @var{name} is the state name for this handler.  The dots in the
above example represent the actual @dfn{code}, or a set of
commands, instructing @command{mailfromd} how to process the message.

  For example, the declaration:

@example
@group
prog envfrom
do
  accept
done
@end group
@end example

@noindent
installs a handler for @samp{envfrom} state, which always approves the
message for delivery, without any further interaction with
@command{mailfromd}.

@cindex actions, introduced
  The word @code{accept} in the above example is an @dfn{action}.
@dfn{Action} is a special language statement that instructs the
run-time engine to stop execution of the program and to return a
response code to the @command{Sendmail}.  There are five actions, one
for each response code: @code{continue}, @code{accept}, @code{reject},
@code{discard}, and @code{tempfail}.  Among these, @code{reject} and
@code{discard} can optionally take one to three arguments.  There are
two ways of supplying the arguments.

In the first form, called @dfn{literal} or @dfn{traditional} notation,
the arguments are supplied as additional words after the action name,
separated by whitespace.  The first argument is a three-digit
@acronym{RFC} 2821 reply code.  It must begin with @samp{5} for
@code{reject} and with @samp{4} for @code{tempfail}.  If two arguments
are supplied, the second argument must be either an @dfn{extended
reply code} (@acronym{RFC} 1893/2034) or a textual string to be
returned along with the @acronym{SMTP} reply.  Finally, if all three
arguments are supplied, then the second one must be an extended reply
code and the third one must supply the textual string.  The following
examples illustrate all possible ways of using the @code{reject}
statement in literal notation:

@example
@group
reject
reject 503
reject 503 5.0.0
reject 503 "Need HELO command"
reject 503 5.0.0 "Need HELO command"
@end group
@end example

@noindent
Please note the quotes around the textual string.

Another form for these action is called @dfn{functional} notation,
because it resembles the function syntax.  When used in this form, the
action word is followed by a parenthesized group of exactly three
arguments, separated by commas.  The meaning and ordering of the
argument is the same as in literal form.  Any of three arguments may
be absent, in which case it will be replaced by the default value.  To
illustrate this, here are the statements from the previous example,
written in functional notation:

@example
@group
reject(,,)
reject(503,,)
reject(503, 5.0.0)
reject(503,, "Need HELO command")
reject(503, 5.0.0, "Need HELO command")
@end group
@end example

@node Conditional Execution
@section Conditional Execution

  Programs consisting of a single action are rarely useful.  In most
cases you will want to do some checking and decide whether to process
the message depending on its result.  For example, if you do not want
to accept messages from the address @samp{<badguy@@some.net>}, you
could write the following program:

@example
@group
prog envfrom
do
  if $f = "badguy@@some.net"
    reject
  else
    accept
  fi
done
@end group
@end example

  This example illustrates several important concepts.  First or
all, @code{$f} in the third line is a @dfn{Sendmail macro
reference}.  Sendmail macros are referenced the same way as in
@file{sendmail.cf}, with the only difference that curly braces around
macro names are optional, even if the name consists of several
letters.  The value of a macro reference is always a string.

  The equality operator (@samp{=}) compares its left and right
arguments and evaluates to true if the two strings are exactly the
same, or to false otherwise.  Apart from equality, you can use the
regular relational operators: @samp{!=}, @samp{>}, @samp{>=}, @samp{<}
and @samp{<=}.  Notice that string comparison in @command{mailfromd}
is always case sensitive.  To do case-insensitive comparison,
translate both operands to upper or lower case (@xref{tolower}, and
@pxref{toupper}).

  The @code{if} statement decides what actions to execute depending
on the value its condition evaluates to.  Its usual form is:

@example
if @var{expression} @var{then-body} [else @var{else-body}] fi
@end example

  The @var{then-body} is executed if the @var{expression} evaluates to
@code{true} (i.e. to any non-zero value).  The optional
@var{else-body} is executed if the @var{expression} yields
@code{false} (i.e. zero).  Both @var{then-body} and @var{else-body} can contain
other @code{if} statements, their nesting depth is not limited.  To
facilitate writing complex conditional statements, the @code{elif}
keyword can be used to introduce alternative conditions, for example:

@example
@group
prog envfrom
do
  if $f = "badguy@@some.net"
    reject
  elif $f = "other@@domain.com"
    tempfail 470 "Please try again later"
  else
    accept
  fi
done
@end group
@end example

    @xref{switch}, for more elaborate forms of conditional branching.

@node Functions and Modules
@section Functions and Modules

@cindex function, defined
  As any programming language, @acronym{MFL} supports a concept of
@dfn{function}, i.e. a body of code that is assigned a unique name and
can be invoked elsewhere as many times as needed.

  All functions have a @dfn{definition} that introduces types and
names of the formal parameters and the result type, if the function is
to return a meaningful value (function definitions in @acronym{MFL}
are discussed in detail in @pxref{User-defined, User-Defined Functions}).

@anchor{funcall}
@cindex function calls
  A function is invoked using a special construct, a @dfn{function
call}:

@example
 @var{name} (@var{arg-list})
@end example

@noindent
  where @var{name} is the function name, and @var{arg-list} is a
comma-separated list of expressions.  Each expression in
@var{arg-list} is evaluated, and its type is compared with that of the
corresponding formal argument.  If the types differ, the expression is
converted to the formal argument type.  Finally, a copy of its value
is passed to the function as a corresponding argument.  The order in
which the expressions are evaluated is not defined.  The compiler
checks that the number of elements in @var{arg-list} match the
number of mandatory arguments for function @var{name}.

  If the function does not deliver a result, it should only be called
as a statement.

  Functions may be recursive, even mutually recursive.

@cindex built-in and library functions, introduced
@cindex library and built-in functions, introduced
@cindex module, defined
@kwindex require
  @command{Mailfromd} comes with a rich set of predefined functions
for various purposes.  There are two basic function classes:
@dfn{built-in} functions, that are implemented by the @acronym{MFL}
runtime environment in @command{mailfromd}, and @dfn{library}
functions, that are implemented in @acronym{MFL}.  The built-in
functions are always available and no preparatory work is needed before
calling them.  In contrast, the library functions are defined in
@dfn{modules}, special @acronym{MFL} source files that contain functions
designed for a particular task.  In order to access a library
function, you must first @dfn{require} a module it is defined in.
This is done using @code{require} statement.  For example, the
function @code{hostname} looks up in the @acronym{DNS} the name
corresponding to the @acronym{IP} address specified as its argument.  This
function is defined in module @file{dns.mfl}, so before calling it you
must require this module:

@example
require dns
@end example

@noindent
The @code{require} statement takes a single argument: the name of the
requested module (without the @samp{.mfl} suffix).  It looks up the
module on disk and loads it if it is available.

  For more information about the module system @xref{Modules}.

@node Domain Name System
@section Domain Name System

  Site administrators often do not wish to accept mail from hosts that
do not have a proper reverse delegation in the Domain Name System.
In the previous section we introduced the library function
@code{hostname}, that looks up in the @acronym{DNS} the name corresponding to
the @acronym{IP} address specified as its argument.  If there is no
corresponding name, the function returns its argument unchanged.  This
can be used to test if the @acronym{IP} was resolved, as illustrated in the
example below:

@example
@group
require 'dns'

prog envfrom
do
  if hostname($client_addr) = $client_addr
    reject
  fi
done
@end group
@end example

  The @code{#require dns} statement loads the module @file{dns.mfl},
after which the definition of @code{hostname} becomes available.

  A similar function, @code{resolve}, which resolves the symbolic
name to the corresponding @acronym{IP} address is provided in the same
@file{dns.mfl} module.

@node Checking Sender Address
@section Checking Sender Address

  A special language construct is provided for verification of
sender addresses (@dfn{callout}):

@example
@group
on poll $f do
when success:
  accept
when not_found or failure:
  reject 550 5.1.0 "Sender validity not confirmed"
when temp_failure:
  tempfail 450 4.1.0 "Try again later"
done
@end group
@end example

  The @code{on poll} construct runs standard verification
(@pxref{standard verification}) for the email address specified as its
argument (in the example above it is the value of the Sendmail macro
@samp{$f}).  The check can result in the following conditions:

@table @code
@item success
The address exists.

@item not_found
The address does not exist.

@item failure
Some error of permanent nature occurred during the check.  The
existence of the address cannot be verified.

@item temp_failure
Some temporary failure occurred during the check.  The
existence of the address cannot be verified at the moment.
@end table

  The @code{when} branches of the @code{on poll} statement
introduce statements, that are executed depending on the actual
return condition.  If any condition occurs that is not handled within
the @code{on} block, the run-time evaluator will signal an
@dfn{exception}@footnote{For more information about exceptions and
their handling, please refer to @ref{Exceptions}.} and return temporary
failure, therefore it is advisable to always handle all four
conditions.  In fact, the condition handling shown in the above
example is preferable for most normal configurations: the mail is
accepted if the sender address is proved to exist and rejected
otherwise.  If a temporary failure occurs, the remote party is urged
to retry the transaction some time later.

  The @code{poll} statement itself has a number of options that
control the type of the verification.  These are discussed in detail
in @ref{poll}.

  It is worth noticing that there is one special email
address which is always available on any host, it is the @dfn{null
address} @samp{<>} used in error reporting.  It is of no use verifying
its existence:

@example
@group
prog envfrom
do
  if $f == ""
    accept
  else
    on poll $f do
    when success:
      accept
    when not_found or failure:
      reject 550 5.1.0 "Sender validity not confirmed"
    when temp_failure:
      tempfail 450 4.1.0 "Try again later"
    done
  fi
done
@end group
@end example

@node SMTP Timeouts
@section @acronym{SMTP} Timeouts
  When using polling functions, it is important to take into account
possible delays, which can occur in @acronym{SMTP} transactions.  Such
delays may be due to low network bandwidth or high load on the remote
server.  Some sites impose them willingly, as a spam-fighting measure.

@cindex timeout escalation
  Ideally the callout verification should use the timeout values
defined in the RFC 2822, but this is impossible in practice, because
it would cause a @dfn{timeout escalation}, which consists in
propagating delays encountered in a callout @acronym{SMTP} session
back to the remote client whose session initiated the callout.

  Consider, for example, the following scenario.  An @acronym{MFL}
script performs a callout on @samp{envfrom} stage.  The remote server
is overloaded and delays heavily in responding, so that the initial response
arrives 3 minutes after establishing the connection, and processing
the @samp{EHLO} command takes another 3 minutes.  These delays are
OK according to the RFC, which imposes a 5 minute limit for each stage,
but while waiting for the remote reply our @acronym{SMTP} server
remains in the @samp{envfrom} state with the client waiting
for a response to its @samp{MAIL} command more than 6 minutes, which is
intolerable, because of the same 5 minute limit.  Thus, the client
will almost certainly break the session.

@anchor{callout server}
@cindex callout server
@cindex server, callout
@cindex soft SMTP timeout
@cindex hard STMP timeout
  To avoid this, @command{mailfromd} uses a special instance, called
@dfn{callout server}, which is responsible for running callout
@acronym{SMTP} sessions asynchronously.  The usual sender verification
is performed using so-called @dfn{soft} timeout values, which
are set to values short enough to not disturb the incoming session
(e.g. a timeout for @samp{HELO} response is 3 seconds, instead of 5
minutes).  If this verification yields a definite answer, that answer
is stored in the cache database and returned to the calling procedure
immediately.  If, however, the verification is aborted due to a timeout,
the caller procedure is returned an @samp{e_temp_failure} exception, and
the callout is scheduled for processing by a callout server.  This
exception normally causes the milter session to return a temporary
error to the sender, urging it to retry the connection later.

  In the meantime, the callout server runs the sender verification
again using another set of timeouts, called @dfn{hard} timeouts, which
are normally much longer than @samp{soft} ones (they default to the
values required by RFC 2822).  If it gets a definitive result (e.g.
@samp{email found} or @samp{email not found}), the server stores it
in the cache database.  If the callout ends due to a timeout, a
@samp{not_found} result is stored in the database.

  Some time later, the remote server retries the delivery, and the
@command{mailfromd} script is run again.  This time, the callout
function will immediately obtain the already cached result from the
database and proceed accordingly.  If the callout server has not
finished the request by the time the sender retries the connection,
the latter is again returned a temporary error, and the process
continues until the callout is finished.

  Usually, callout server is just another instance of
@command{mailfromd} itself, which is started automatically to
perform scheduled SMTP callouts.  It is also possible to set up
a separate callout server on another machine.  This is discussed in
@ref{calloutd}.

  For a detailed information about callout timeouts and their
configuration, see @ref{conf-timeout}.

  For a description of how to configure @command{mailfromd} to use callout
servers, see @ref{conf-server}.

@node Avoiding Verification Loops
@section Avoiding Verification Loops

  An @code{envfrom} program consisting only of the @code{on poll}
statement will work smoothly for incoming mails, but will create
infinite loops for outgoing mails.  This is because upon sending an outgoing
message @command{mailfromd} will start the verification procedure, which
will initiate an @acronym{SMTP} transaction with the same mail server
that runs it.  This transaction will in turn trigger execution of
@code{on poll} statement, etc.  @i{ad infinitum}.  To avoid this, any
properly written filter script should not run the verification
procedure on the email addresses in those domains that are relayed by
the server it runs on.  This can be achieved using @code{relayed}
function.  The function returns @code{true} if its argument is
contained in one of the predefined @dfn{domain list} files.  These
files correspond to @command{Sendmail} plain text files used in
@code{F} class definition forms (see @cite{Sendmail Installation and
Operation Guide}, chapter 5.3), i.e. they contain one domain name per
line, with empty lines and lines started with @samp{#} being ignored.
The domain files consulted by @code{relayed} function are defined
in the @code{relayed-domain-file} configuration file statement
(@pxref{conf-base, relayed-domain-file}):

@example
relayed-domain-file (/etc/mail/local-host-names,
                     /etc/mail/relay-domains);
@end example

@noindent
or:

@example
@group
relayed-domain-file /etc/mail/local-host-names;
relayed-domain-file /etc/mail/relay-domains;
@end group
@end example

  The above example declares two domain list files, most commonly
used in @command{Sendmail} installations to keep hostnames of the server
@footnote{class @samp{w}, see @cite{Sendmail Installation and Operation
Guide}, chapter 5.2.} and names of the domains, relayed by this
server@footnote{class @samp{R}}.

  Given all this, we can improve our filter program:

@example
@group
require 'dns'

prog envfrom
do
  if $f == ""
    accept
  elif relayed(hostname($@{client_addr@}))
    accept
  else
    on poll $f do
    when success:
      accept
    when not_found or failure:
      reject 550 5.1.0 "Sender validity not confirmed"
    when temp_failure:
      tempfail 450 4.1.0 "Try again later"
    done
  fi
done
@end group
@end example

  If you feel that your Sendmail's relayed domains are not restrictive
enough for @command{mailfromd} filters (for example you are relaying
mails from some third-party servers), you can use a database of
trusted mail server addresses.  If the number of such servers is small
enough, a single @samp{or} statement can be used, e.g.:

@example
  elif $@{client_addr@} = "10.10.10.1"
       or $@{client_addr@} = "192.168.11.7"
    accept
  @dots{}
@end example

@noindent
otherwise, if the servers' @acronym{IP} addresses fall within one or
several @acronym{CIDR}s, you can use the @code{match_cidr} function
(@pxref{Internet address manipulation functions}), e.g.:

@example
  elif match_cidr ($@{client_addr@}, "199.232.0.0/16")
    accept
  @dots{}
@end example

@noindent
or combine both methods.  Finally, you can keep a @acronym{DBM}
database of relayed addresses and use @code{dbmap} or @code{dbget}
function for checking (@pxref{Database functions}).

@example
  elif dbmap("%__statedir__/relay.db", $@{client_addr@})
    accept
  @dots{}
@end example

@node HELO Domain
@section HELO Domain

@cindex s, @command{Sendmail} macro
  Some of the mail filtering conditions may depend on the value of
@dfn{helo domain} name, i.e. the argument to the @acronym{SMTP} @code{EHLO} (or
@code{HELO}) command.  If you ever need such conditions, take into
account the following caveats.  Firstly, although @command{Sendmail}
passes the helo domain in @code{$s} macro, it does not do this
consistently.  In fact, the @code{$s} macro is available only to
the @code{helo} handler, all other handlers won't see it, no matter what
the value of the corresponding @code{Milter.macros.@var{handler}}
statement.  So, if you wish to access its value from any
handler, other than @code{helo}, you will have to store it in a
@dfn{variable} in the @code{helo} handler and then use this variable
value in the other handler.  This approach is also recommended for
another @acronym{MTA}s.  This brings us to the concept of
variables in @command{mailfromd} scripts.

@cindex variables, introduced
@cindex variable declaration
  A variable is declared using the following syntax:

@example
@var{type} @var{name}
@end example

@noindent
where @var{variable} is the variable name and @var{type} is
@samp{string}, if the variable is to hold a string value, and
@samp{number}, if it is supposed to have a numeric value.

@cindex variable assignment
@cindex assignment to variable
  A variable is assigned a value using the @code{set} statement:

@example
set @var{name} @var{expr}
@end example

@noindent
where @var{expr} is any valid @acronym{MFL} expression.

  The @code{set} statement can occur within handler or function
declarations as well as outside of them.

  There are two kinds of @command{Mailfromd} variables: @dfn{global
variables}, that are visible to all handlers and functions, and
@dfn{automatic variables}, that are available only within the handler
or function where they are declared.  For our purpose we need a global
variable (@xref{Variables, Variable classes}, for detailed descriptions
of both kinds of variables).

  The following example illustrates an approach that allows to use
the @code{HELO} domain name in any handler:

@example
@group
# @r{Declare the helohost variable}
string helohost

prog helo
do
  # @r{Save the host name for further use}
  set helohost $s
done

prog envfrom
do
  # @r{Reject hosts claiming to be @i{localhost}}
  if helohost = "localhost"
    reject 570 "Please specify real host name"
  fi
done
@end group
@end example

  Notice, that for this approach to work, your @acronym{MTA}
must export the @samp{s} macro (e.g., in case of Sendmail, the
@code{Milter.macros.helo} statement in the @file{sendmail.cf} file must contain
@samp{s}.  @pxref{Sendmail}).  This requirement can be removed by
using the @dfn{handler argument} of @code{helo}.  Each
@command{mailfromd} handler is given one or several arguments.  The
exact number of arguments and their meaning are handler-specific and are
described in @ref{Handlers}, and @ref{milter-control-flow}.
The arguments are referenced by their ordinal number, using the notation
@code{$@var{n}}.  The @code{helo} handler takes one argument, whose
value is the helo domain.  Using this information, the @code{helo}
handler from the example above can be rewritten as follows:

@example
@group
prog helo
do
  # @r{Save the host name for further use}
  set helohost $1
done
@end group
@end example

@node rset
@section SMTP RSET and Milter Abort Handling
@cindex RSET
@cindex milter abort
  In previous section we have used a global variable to hold certain
information and share it between handlers.  In the majority of cases,
such information is session specific, and becomes invalid if the
remote party issues the SMTP @code{RSET} command.  Therefore,
@command{mailfromd} clears all global variables when it receives a
Milter @samp{abort} request, which is normally generated by this
command.

@kwindex precious
@cindex precious variables
@cindex variable, precious
  However, you may need some variables that retain their values
even across SMTP session resets.  In @command{mailfromd} terminology
such variables are called @dfn{precious}.  Precious variables are
declared by prefixing their declaration with the keyword
@code{precious}.  Consider, for example, this snippet of code:

@example
precious number rcpt_counter

prog envrcpt
do
  set rcpt_counter rcpt_counter + 1
done
@end example

  Here, the variable @samp{rcpt_counter} is declared as precious and
its value is incremented each time the @samp{envrcpt} handler is
called.  This way, @samp{rcpt_counter} will keep the total number of
SMTP @code{RCPT} commands issued during the session, no matter how
many times it was restarted using the @code{RSET} command.

@node Controlling Number of Recipients
@section Controlling Number of Recipients

@cindex @code{MaxRecipientsPerMessage}, @command{sendmail} option
  Any @acronym{MTA} provides a way to limit the number of recipients
per message.  For example, in @command{Sendmail} you may use the
@code{MaxRecipientsPerMessage} option@footnote{@cite{Sendmail (tm)
Installation and Operation Guide}, chapter 5.6, @samp{O -- Set
Option}.}.  However, such methods are not flexible, so
you are often better off using @command{mailfromd} for this purpose.

  @command{Mailfromd} keeps the number of recipients collected so far
in variable @code{rcpt_count}, which can be controlled in
@code{envrcpt} handler as shown in the example below:

@example
@group
prog envrcpt
do
  if rcpt_count > 10
    reject 550 5.7.1 "Too many recipients"
  fi
done
@end group
@end example

  This filter will accept no more than 10 recipients per message.
You may achieve finer granularity by using additional conditions.  For
example, the following code will allow any number of recipients if the
mail is coming from a domain relayed by the server, while limiting it
to 10 for incoming mail from other domains:

@example
@group
prog envrcpt
do
  if not relayed(hostname($client_addr)) and rcpt_count > 10
    reject 550 5.7.1 "Too many recipients"
  fi
done
@end group
@end example

  There are three important features to notice in the above code.
First of all, it introduces two @dfn{boolean} operators:
@code{and}, which evaluates to @code{true} only if both
left-side and right-side expressions are @code{true}, and @code{not},
which reverses the value of its argument.

  Secondly, the scope of an operation is determined by its
@dfn{precedence}, or @dfn{binding strength}.  @code{Not} binds more
tightly than @code{and}, so its scope is limited by the next
expression between it and @code{and}.  Using parentheses to underline the
operator scoping, the above @code{if} condition can be rewritten as
follows:

@example
    if (not (relayed(hostname($client_addr)))) and (%rcpt_count > 10)
@end example

  Finally, it is important to notice that all boolean expressions
are computed using @dfn{shortcut evaluation}.  To understand what it
is, let's consider the following expression: @code{@var{x} and
@var{y}}.  Its value is @code{true} only if both @var{x} and @var{y}
are @code{true}.  Now suppose that we evaluate the expression from
left to right and we find that @var{x} is false.  This means that
no matter what the value of @var{y} is, the resulting expression will be
@code{false}, therefore there is no need to compute @var{y} at all.
So, the boolean shortcut evaluation works as follows:

@table @code
@item @var{x} and @var{y}
If @code{@var{x} @result{} @code{false}}, do not evaluate @var{y} and
return @code{false}.

@item @var{x} or @var{y}
If @code{@var{x} @result{} @code{true}}, do not evaluate @var{y} and
return @code{true}.
@end table

  Thus, in the expression @code{not relayed(hostname($client_addr)) and
rcpt_count > 10}, the value of the @code{rcpt_count} variable will be
compared with @samp{10} only if the @code{relayed} function yielded
@code{false}.

  To further enhance our sample filter, you may wish to make the
@code{reject} output more informative, to let the sender know what
the recipient limit is.  To do so, you can use the @dfn{concatenation
operator} @samp{.} (a dot):

@example
@group
set max_rcpt 10
prog envrcpt
do
  if not relayed(hostname($client_addr)) and rcpt_count > 10
    reject 550 5.7.1 "Too many recipients, max=" . max_rcpt
  fi
done
@end group
@end example

  When evaluating the third argument to @code{reject},
@command{mailfromd} will first convert @code{max_rcpt} to string and
then concatenate both strings together, producing string @samp{Too many
recipients, max=10}.

@node Sending Rate
@section Sending Rate

  We have introduced the notion of mail sending rate in @ref{Rate
Limit}.  @command{Mailfromd} keeps the computed rates in the special
@code{rate} database (@pxref{Databases}).  Each record in this
database consists of a @code{key}, for which the rate is computed, and
the rate value, in form of a double precision floating point number,
representing average number of messages per second sent by this
@code{key} within the last sampling interval.  In the simplest case,
the sender email address can be used as a @code{key}, however we recommend
to use a conjunction @var{email}-@var{sender_ip} instead, so the
actual @var{email} owner won't be blocked by actions of some spammer
abusing his/her address.

  Two functions are provided to control and update sending rates.  The
@code{rateok} function takes three mandatory arguments:

@example
  bool rateok(string @var{key}, number @var{interval}, number @var{threshold})
@end example

The @var{key} meaning is described above.  The @var{interval} is the
sampling interval, or the number of seconds to which the actual
sending rate value is converted.  Remember that it is stored
internally as a floating point number, and thus cannot be directly
used in @command{mailfromd} filters, which operate only on integer
numbers.  To use the rate value, it is first converted to messages per
given interval, which is an integer number.  For example, the rate
@code{0.138888} brought to 1-hour interval gives @code{500}
(messages per hour).

  When the @code{rateok} function is called, it recomputes
rate record for the given @var{key}.  If the new rate value converted
to messages per given @var{interval} is less than @var{threshold},
the function updates the database and returns @code{True}.  Otherwise it
returns @code{False} and does not update the database.

  This function must be @dfn{required} prior to use, by placing the
following statement somewhere at the beginning of your script:

@example
require rateok
@end example

For example, the following code limits the mail sending rate for each
@samp{email address}-@samp{@acronym{IP}} combination to 180 per hour.
If the actual rate value exceeds this limit, the sender is returned a
temporary failure response:

@example
@group
require rateok

prog envfrom
do
  if not rateok($f . "-" . $@{client_addr@}, 3600, 180)
    tempfail 450 4.7.0 "Mail sending rate exceeded.  Try again later"
  fi
done
@end group
@end example

@noindent
  Notice argument concatenation, used to produce the key.

  It is often inconvenient to specify intervals in seconds,
therefore a special @code{interval} function is provided.  It
converts its argument, which is a textual string representing time
interval in English, to the corresponding number of seconds.  Using
this function, the function invocation would be:

@example
     rateok($f . "-" . $@{client_addr@}, interval("1 hour"), 180)
@end example

  The @code{interval} function is described in @ref{interval}, and time
intervals are discussed in @ref{time interval specification}.

  The @code{rateok} function begins computing the rate
as soon as it has collected enough data.  By default, it needs at least
four mails.  Since this may lead to a big number of false positives
(i.e. overestimated rates) at the beginning of sampling interval,
there is a way to specify a minimum number of samples @code{rateok}
must collect before starting to actually compute rates.  This number of
samples is given as the optional fourth argument to the function.  For
example, the following call will always return @code{True} for the
first 10 mails, no matter what the actual rate:

@example
     rateok($f . "-" . $@{client_addr@}, interval("1 hour"), 180, 10)
@end example

@anchor{TBF}
  The @code{tbf_rate} function allows to exercise more control over
the mail rates.  This function implements a @dfn{token bucket filter}
(@acronym{TBF}) algorithm.

  The token bucket controls when the data can be transmitted based on
the presence of abstract entities called @dfn{tokens} in a container
called @dfn{bucket}.  Each token represents some amount of data.  The
algorithm works as follows:

@itemize @bullet
@item A token is added to the bucket at a constant rate of 1 token per
@var{t} microseconds.
@item A bucket can hold at most @var{m} tokens.  If a token arrives
when the bucket is full, that token is discarded.
@item When @var{n} items of data arrive (e.g.@: @var{n} mails), @var{n}
tokens are removed from the bucket and the data are accepted.
@item If fewer than @var{n} tokens are available, no tokens are
removed from the bucket and the data are not accepted.
@end itemize

  This algorithm allows to keep the data traffic at a constant rate
@var{t} with bursts of up to @var{m} data items.  Such bursts occur
when no data was being arrived for @var{m}*@var{t} or more
microseconds.

  @command{Mailfromd} keeps buckets in a database @samp{tbf}.  Each
bucket is identified by a unique @dfn{key}.  The @code{tbf_rate}
function is defined as follows:

@example
 bool tbf_rate(string @var{key}, number @var{n}, number @var{t}, number @var{m})
@end example

The @var{key} identifies the bucket to operate upon.  The rest of
arguments is described above.  The @code{tbf_rate} function returns
@samp{True} if the algorithm allows to accept the data and
@samp{False} otherwise.

Depending on how the actual arguments are selected the @code{tbf_rate}
function can be used to control various types of flow rates.  For
example, to control mail sending rate, assign the arguments as
follows: @var{n} to the number of mails and @var{t} to the control
interval in microseconds:

@example
@group
prog envfrom
do
  if not tbf_rate($f . "-" . $client_addr, 1, 10000000, 20)
    tempfail 450 4.7.0 "Mail sending rate exceeded.  Try again later"
  fi
done
@end group
@end example

The example above permits to send at most one mail each 10 seconds.
The burst size is set to 20.

Another use for the @code{tbf_rate} function is to limit the total
delivered mail size per given interval of time.  To do so, the
function must be used in @code{prog eom} handler, because it is the
only handler where the entire size of the message is known.   The
@var{n} argument must contain the number of bytes in the email (or
email bytes * number of recipients), and the @var{t} must be set to
the number of bytes per microsecond a given user is allowed to send.  The
@var{m} argument must be large enough to accommodate a couple of
large emails.  E.g.:

@example
@group
  prog eom
  do
    if not tbf_rate("$f-$client_addr",
                    message_size(current_message()),
                    10240*1000000,  # At most 10 kb/sec
                    10*1024*1024)
      tempfail 450 4.7.0 "Data sending rate exceeded.  Try again later"
    fi
  done
@end group
@end example

@xref{Rate limiting functions}, for more information about
@code{rateok} and @code{tbf_rate} functions.

@node Greylisting
@section Greylisting

  Greylisting is a simple method of defending against the spam
proposed by Evan Harris.  In few words, it consists in recording the
@samp{sender @acronym{IP}}-@samp{sender email}-@samp{recipient email} triplet of
mail transactions.  Each time the unknown triplet is seen, the
corresponding message is rejected with the @code{tempfail} code.  If the
mail is legitimate, this will make the originating server retry
the delivery later, until the destination eventually accepts it.  If,
however, the mail is a spam, it will probably never be retried, so
the users will not be bothered by it.  Even if the spammer will retry
the delivery, the @dfn{greylisting period} will give spam-detection
systems, such as @acronym{DNSBL}s, enough time to detect and blacklist it,
so by the time the destination host starts accepting emails from this
triplet, it will already be blocked by other means.

  You will find the detailed description of the method in
@uref{http://projects.puremagic.com/@/greylisting/@/whitepaper.html,
The Next Step in the Spam Control War: Greylisting}, the original
whitepaper by Evan Harris.

  The @command{mailfromd} implementation of greylisting is based on
@code{greylist} function.  The function takes two arguments:
the @code{key}, identifying the greylisting triplet, and the
@code{interval}.  The function looks up the key in the @dfn{greylisting
database}.  If such a key is not found, a new entry is created for it
and the function returns @code{true}.  If the key is
found, @code{greylist} returns @code{false}, if it was inserted to the
database more than @code{interval} seconds ago, and @code{true} otherwise.
In other words, from the point of view of the  greylisting algorithm, the
function returns @code{true} when the message delivery should be
blocked.  Thus, the simplest implementation of the algorithm would be:

@example
@group
prog envrcpt
do
 if greylist("$@{client_addr@}-$f-$@{rcpt_addr@}", interval("1 hour"))
   tempfail 451 4.7.1 "You are greylisted"
 fi
done
@end group
@end example

@cindex greylist_seconds_left, global variable, introduced
  However, the message returned by this example, is not informative
enough.  In particular, it does not tell when the message will be
accepted.  To help you produce more informative messages, @code{greylist}
function stores the number of seconds left to the end of the
greylisting period in the global variable
@code{greylist_seconds_left}, so the above example could be enhanced
as follows:

@example
@group
prog envrcpt
do
  set gltime interval("1 hour")
  if greylist("$@{client_addr@}-$f-$@{rcpt_addr@}", gltime)
    if greylist_seconds_left = gltime
      tempfail 451 4.7.1
         "You are greylisted for %gltime seconds"
    else
      tempfail 451 4.7.1
         "Still greylisted for %greylist_seconds_left seconds"
    fi
  fi
done
@end group
@end example

  In real life you will have to avoid greylisting some messages, in
particular those coming from the @samp{<>} address and from the @acronym{IP}
addresses in your relayed domain.  It can easily be done using the
techniques described in previous sections and is left as an exercise
to the reader.

@anchor{greylisting types}
@cindex greylisting types
@cindex greylisting, traditional
  @code{Mailfromd} provides two implementations of greylisting
primitives, which differ in the information stored in the database.
The one described above is called @dfn{traditional}.  It keeps in the
database the time when the greylisting was activated for the given
key, so the @code{greylisting} function uses its second argument
(@code{interval}) and the current timestamp to decide whether the key
is still greylisted.

@cindex greylisting, Con Tassios type
@cindex Con Tassios greylisting type
  The second implementation is called by the name of its inventor
@dfn{Con Tassios}.  This implementation stores in the database the
time when the greylisting period is set to expire, computed by the
@code{greylist} when it is first called for the given key, using the
formula @samp{current_timestamp + interval}.  Subsequent calls to
@code{greylist} compare the current timestamp with the one stored in
the database and ignore their second argument.  This implementation is
enabled by one of the following pragmas:

@example
#pragma greylist con-tassios
@end example
@noindent
or
@example
#pragma greylist ct
@end example

  When Con Tassios implementation is used, yet another function
becomes available.  The function @code{is_greylisted}
(@pxref{Greylisting functions,,is_greylisted}) returns
@samp{True} if its argument is greylisted and @samp{False} otherwise.
It can be used to check for the greylisting status without actually
updating the database:

@example
  if is_greylisted("$@{client_addr@}-$f-$@{rcpt_addr@}")
    @dots{}
  fi
@end example

@anchor{whitelisting}
@cindex whitelisting
  One special case is @dfn{whitelisting}, which is often used
together with greylisting.  To implement it, @command{mailfromd}
provides the function @code{dbmap}, which takes two mandatory arguments:
@code{dbmap(@var{file}, @var{key})} (it also allows an optional third
argument, see @ref{dbmap}, for more information on it).  The first argument is
the name of the @acronym{DBM} file where to search for the key, the second one
is the key to be searched.  Assuming you keep your whitelist database
in file @file{/var/run/whitelist.db}, a more practical example will be:

@example
@group
prog envrcpt
do
  set gltime interval("1 hour")

  if not ($f = "" or relayed(hostname($@{client_addr@}))
         or dbmap("/var/run/whitelist.db", $@{client_addr@}))
    if greylist("$@{client_addr@}-$f-$@{rcpt_addr@}", gltime)
      if greylist_seconds_left = gltime
        tempfail 451 4.7.1
           "You are greylisted for %gltime seconds"
      else
        tempfail 451 4.7.1
           "Still greylisted for %greylist_seconds_left seconds"
      fi
    fi
  fi
done
@end group
@end example


@node Local Account Verification
@section Local Account Verification

  In your filter script you may need to verify if the given
user name is served by your mail server, in other words, to verify if
it represents a @dfn{local account}.  Notice that in this context, the word
@dfn{local} does not necessarily mean that the account is local for
the server running @command{mailfromd}, it simply means any account
whose mailbox is served by the mail servers using @command{mailfromd}.

  The @code{validuser} function may be used for this purpose.  It
takes one argument, the user name, and returns @code{true} if
this name corresponds to a local account.  To verify this, the
function relies on @command{libmuauth}, a powerful authentication
library shipped with GNU @command{mailutils}.  More precisely, it
invokes a list of @dfn{authorization} functions.  Each function is
responsible for looking up the user name in a particular source of
information, such as system @file{passwd} database, an @acronym{SQL} database,
etc.  The search is terminated when one of the functions finds
the name in question or the list is exhausted.  In the former case, the
account is local, in the latter it is not.  This concept is
discussed in detail in @pxref{authentication, Authentication,
Authorization and Authentication Principles, mailutils, GNU Mailutils
Manual}).  Here we will give only some practical advices for
implementing it in @command{mailfromd} filters.

  The actual list of available authorization modules depends on your
@command{mailutils} installation.  Usually it includes, apart from
traditional @acronym{UNIX} @file{passwd} database, the functions for verifying
@acronym{PAM}, @acronym{RADIUS} and @acronym{SQL} database accounts.
Each of the authorization methods is configured using special
configuration file statements.  For the description of the Mailutils
configuration files, @xref{configuration, Mailutils Configuration
File, Mailutils Configuration File, mailutils, GNU Mailutils Manual}.
You can obtain the template for @command{mailfromd} configuration by
running @command{mailfromd --config-help}.

  For example, the following @file{mailfromd.conf} file:

@example
@group
auth @{
  authorization pam:system;
@}

pam @{
  service mailfromd;
@}
@end group
@end example

@noindent
sets up the authorization using @acronym{PAM} and system @file{passwd}
database.  The name of @acronym{PAM} service to use is @samp{mailfromd}.

@cindex aliases, looking up
  The function @code{validuser} is often used together with
@code{dbmap}, as in the example below:

@example
@group
#pragma dbprop /etc/mail/aliases.db null

if dbmap("/etc/mail/aliases.db", localpart($rcpt_addr))
   and validuser(localpart($rcpt_addr))
  @dots{}
fi
@end group
@end example

For more information about @code{dbmap} function, see @ref{dbmap}.
For a description of @code{dbprop} pragma, see @ref{Database functions}.

@node Databases
@section Databases
  Some @command{mailfromd} functions use @acronym{DBM} databases to save their
persistent state data.  Each database has a unique @dfn{identifier},
and is assigned several pieces of information for its maintenance: the
database @dfn{file name} and the @dfn{expiration period}, i.e. the
time after which a record is considered expired.

@xopindex{show-defaults, mailfromd, introduced}
  To obtain the list of available databases along with their
preconfigured settings, run @kbd{mailfromd --show-defaults}
(@pxref{Examining Defaults}).  You will see an output similar to
this:

@example
@group
version:             @value{VERSION}
script file:         /etc/mailfromd.mfl
preprocessor:        /usr/bin/m4 -s
user:                mail
statedir:            /var/run/mailfromd
socket:              unix:/var/run/mailfromd/mailfrom
pidfile:             /var/run/mailfromd/mailfromd.pid
default syslog:          blocking
supported databases:     gdbm, bdb
default database type:   bdb
optional features:   DKIM GeoIP2 STARTTLS
greylist database:      /var/run/mailfromd/greylist.db
greylist expiration:    86400
tbf database:        /var/run/mailfromd/tbf.db
tbf expiration:      86400
rate database:      /var/run/mailfromd/rates.db
rate expiration:    86400
cache database:      /var/run/mailfromd/mailfromd.db
cache positive expiration: 86400
cache negative expiration: 43200
@end group
@end example

  The text below @samp{optional features} line describes the available
built-in databases.  Notice that the @samp{cache} database, in
contrast to the rest of databases, has two expiration periods
associated with it.  This is explained in the next subsection.

@menu
* Database Formats::
* Basic Database Operations::
* Database Maintenance::
@end menu

@node Database Formats
@subsection Database Formats
@cindex database formats
  The version @value{VERSION} runs the following database types (or
@dfn{formats}):

@cindex databases used by @command{mailfromd}
@table @samp
@cindex cache database
@item cache
@anchor{cache database}
@dfn{Cache database} keeps the information about external emails,
obtained using sender verification functions (@pxref{Checking Sender
Address}).  The key entry to this database is an email address or
@var{email}:@var{sender-ip} string, for addresses checked using strict
verification.  The data its stores for each key are:

@enumerate 1
@item
Address validity.  This field can be either @code{success} or
@code{not_found}, meaning the address is confirmed to exists or it
is not.

@item
The time when the entry was entered into the database.  It is used to
check for expired entries.
@end enumerate

@anchor{cache expiration}
@cindex positive expiration period, defined
@cindex negative expiration period, defined
  The @samp{cache} database has two expiration periods: a
@dfn{positive expiration} period, that is applied to  entries with
the first field set to @code{success}, and a @dfn{negative expiration}
period, applied to entries marked as @code{not_found}.

@cindex rate database
@item rate
@anchor{rate database}
The mail sending rate data, maintained by @code{rate} function
(@pxref{Rate limiting functions}).  A record consists of the following fields:

@table @asis
@item timestamp
The time when the entry was entered into the database.

@item interval
Interval during which the rate was measured (seconds).

@item count
Number of mails sent during this interval.
@end table

@cindex tbf database
@item tbf
@anchor{tbf database}
This database is maintained by @code{tbf_rate} function (@pxref{TBF}).
Each record represents a single bucket and consists of the following
keys:

@table @asis
@item timestamp
Timestamp of most recent token, as a 64-bit unsigned integer
(microseconds resolution).

@item expirytime
Estimated time when this bucket expires (seconds since epoch).

@item tokens
Number of tokens in the bucket (@code{size_t}).
@end table

@cindex greylist database
@item greylist
@anchor{greylist database}
This database is maintained by @code{greylist} function
(@pxref{Greylisting}).  Each record holds only the timestamp.
Its semantics depends on the greylisting implementation in
use (@pxref{greylisting types}).  In traditional implementation, it
is the time when the entry was entered into the database.  In Con
Tassios implementation, it is the time when the greylisting period
expires.
@end table

@node Basic Database Operations
@subsection Basic Database Operations
@pindex mfdbtool
  The @command{mfdbtool} utility is provided for performing various
operations on the @command{mailfromd} database.

@cindex database, listing
@cindex listing a database contents
@xopindex{list, mfdbtool, described}
@anchor{--list option}
  To list the contents of a database, use @option{--list} option.
When used without any arguments it will list the @samp{cache}
database:

@example
@group
$ @kbd{mfdbtool --list}
abrakat@@mail.com           success Thu Aug 24 15:28:58 2006
baccl@@EDnet.NS.CA          not_found Fri Aug 25 10:04:18 2006
bhzxhnyl@@chello.pl       not_found Fri Aug 25 10:11:57 2006
brqp@@aaanet.ru:24.1.173.165  not_found Fri Aug 25 14:16:06 2006
@end group
@end example

  You can also list data for any particular key or keys.  To do so,
give the keys as arguments to @command{mfdbtool}:

@example
@group
$ @kbd{mfdbtool --list abrakat@@mail.com brqp@@aaanet.ru:24.1.173.165}
abrakat@@mail.com           success Thu Aug 24 15:28:58 2006
brqp@@aaanet.ru:24.1.173.165  not_found Fri Aug 25 14:16:06 2006
@end group
@end example

@xopindex{format, mfdbtool, introduced}
@xopindex{format, mfdbtool, using with @option{--list}}
  To list another database, give its format identifier with the
@option{--format} (@option{-H}) option.  For example, to list the
@samp{rate} database:

@example
@group
$ @kbd{mfdbtool --list --format=rate}
sam@@mail.net-62.12.4.3 Wed Sep  6 19:41:42 2006  139   3 0.0216 6.82e-06
axw@@rame.com-59.39.165.172 Wed Sep  6 20:26:24 2006  0  1  N/A  N/A
@end group
@end example

  The @option{--format} option can be used with any database
management option, described below.

@anchor{estimated time of sending}
@cindex estimated time of sending, prediction of
@xopindex{predict, mfdbtool, introduced}
  Another useful operation you can do while listing @samp{rate}
database is the prediction of @dfn{estimated time of sending},
i.e. the time when the user will be able to send mail if currently his
mail sending rate has exceeded the limit.  This is done using
@option{--predict} option.  The option takes an argument, specifying
the mail sending rate limit, e.g. (the second line is split for readability):

@example
@group
$ @kbd{mfdbtool --predict="180 per 1 minute"}
ed@@fae.net-21.10.1.2 Wed Sep 13 03:53:40 2006  0 1 N/A N/A; free to send
service@@19.netlay.com-69.44.129.19 Wed Sep 13 15:46:07 2006 7 2
   0.286   0.0224; in 46 sec. on Wed Sep 13 15:49:00 2006
@end group
@end example

@noindent
Notice, that there is no need to use @option{--list --format=rate}
along with this option, although doing so is not an error.

@anchor{deleting from databases}
@xopindex{delete, mfdbtool, introduced}
  To delete an entry from the database, use @option{--delete} option,
for example: @kbd{mfdbtool --delete abrakat@@mail.com}.  You can give
any number of keys to delete in the command line.

@node Database Maintenance
@subsection Database Maintenance
@cindex database maintenance
@cindex maintenance, database
  There are two principal operations of database management:
expiration and compaction.  @dfn{Expiration} consists in removing
expired entries from the database.  In fact, it is rarely needed,
since the expired entries are removed in the process of normal
@command{mailfromd} work.  Nevertheless, a special option is provided
in case an explicit expiration is needed (for example, before dumping
the database to another format, to avoid transferring useless
information).

@xopindex{expire, mfdbtool, introduced}
  The command line option @option{--expire} instructs
@command{mfdbtool} to delete expired entries from the specified database.  As
usual, the database is specified using @option{--format} option.  If
it is not given explicitly, @samp{cache} is assumed.

@cindex database compaction
@cindex compaction, database
  While removing expired entries the space they occupied is marked as
free, so it can be used by subsequent inserts.  The database does
not shrink after expiration is finished.  To actually return the
unused space to the file system you should @dfn{compact} your
database.

@anchor{compaction}
@xopindex{compact, mfdbtool, introduced}
  This is done by running @kbd{mfdbtool --compact} (and, optionally,
specifying the database to operate upon with @option{--format}
option).  Notice, that compacting a database needs roughly as
much disk space on the partition where the database resides as is
currently used by the database.  Database compaction runs in three phases.
First, the database is scanned and all non-expired records are stored
in the memory.  Secondly, a temporary database is created in the state
directory and all the cached entries are flushed into it.  This
database is named after the @acronym{PID} of the running
@command{mfdbtool} process.  Finally, the temporary database is
renamed to the source database.

@anchor{compact cronjob}
@xopindex{all, mfdbtool, introduced}
  Both @option{--compact} and @option{--expire} can be applied to all
databases by combining them with @option{--all}.  It is useful, for
example, in @file{crontab} files.  For example, I have the following
monthly job in my @file{crontab}:

@example
0 1 1 * * /usr/bin/mfdbtool --compact --all
@end example

@node Testing Filter Scripts
@section Testing Filter Scripts

  It is important to check your filter script before actually starting
to use it.  There are several ways to do so.

@cindex lint mode
@cindex syntax check
@cindex script file checking
@xopindex{lint, mailfromd, introduced}
@xopindex{syntax-check, mailfromd, introduced}
  To test the syntax of your filter script, use the @option{--lint}
option.  It will cause @command{mailfromd} to exit
immediately after attempting to compile the script file.  If the
compilation succeeds, the program will exit with code 0.  Otherwise,
it will exit with error code 78 (@samp{configuration error}).  In the
latter case, @command{mailfromd} will also print a diagnostic message,
describing the error along with the exact location where the error was
diagnosed, for example:

@example
@group
mailfromd: /etc/mailfromd.mfl:39: syntax error, unexpected reject
@end group
@end example

@xopindex{location-column, mailfromd, described}
  The error location is indicated by the name of the file and the
number of the line when the error occurred.  By using the
@option{--location-column} option you instruct @command{mailfromd} to
also print the @dfn{column number}.  E.g. with this option the above
error message may look like:

@example
@group
mailfromd: /etc/mailfromd.mfl:39.12 syntax error, unexpected reject
@end group
@end example

  Here, @samp{39} is the line and @samp{12} is the column number.

@cindex cross-reference
  For complex scripts you may wish to obtain a listing of variables
used in the script.  This can be achieved using @option{--xref}
command line option:

The output it produces consists of four columns:

@table @asis
@item Variable name
@item Data type
Either @code{number} or @code{string}.
@item Offset in data segment
Measured in words.
@item References
A comma-separated list of locations where the variable was
referenced.  Each location is represented as @var{file}:@var{line}.
If several locations pertain to the same @var{file}, the file name is
listed only once.
@end table

@noindent
Here is an example of the cross-reference output:

@example
@group
$ @kbd{mailfromd --xref}
Cross-references:
-----------------
cache_used               number 5   /etc/mailfromd.mfl:48
clamav_virus_name        string 9   /etc/mailfromd.mfl:240,240
db                       string 15  /etc/mailfromd.mfl:135,194,215
dns_record_ttl           number 16  /etc/mailfromd.mfl:136,172,173
ehlo_domain              string 11
gltime                   number 13  /etc/mailfromd.mfl:37,219,220,222,223
greylist_seconds_left    number 1   /etc/mailfromd.mfl:220,226,227
last_poll_host           string 2
@end group
@end example

@anchor{test mode}
@cindex filter script, debugging
@cindex debugging the filter script
@cindex filter script, running in test mode
@xopindex{test, mailfromd, introduced}
  If the script passes syntax check, the next step is often to test if
it works as you expect it to.  This is done with @option{--test}
(@option{-t}) command line option.  This option runs the
@code{envfrom} handler (or another one, see below) and prints the
result of its execution.

@cindex sendmail macros, setting from the command line
  When running your script in test mode, you will need to supply the
values of @command{Sendmail} macros it needs.  You do this by placing
the necessary assignments in the command line.  For example, this is
how to supply initial values for @code{f} and @code{client_addr}
macros:

@example
$ @kbd{mailfromd --test f=gray@@gnu.org client_addr=127.0.0.1}
@end example

@anchor{overriding initial values}
@cindex overriding initial variable values
@cindex variable values, setting from the command line
@xopindex{variable, mailfromd, introduced}
  You may also need to alter initial values of some global variables
your script uses.  To do so, use @option{-v} (@option{--variable})
command line option.  This option takes a single argument consisting
of the variable name and its initial value, separated by an equals
sign.  For example, here is how to change the value of
@code{ehlo_domain} global variable:

@example
$ @kbd{mailfromd -v ehlo_domain=mydomain.org}
@end example

  The @option{--test} option is often useful in conjunction with options
@option{--debug}, @option{--trace} and @option{--transcript}
(@pxref{Logging and Debugging}.  The following example shows what the
author got while debugging the filter script described in
@ref{Filter Script Example}:

@example
@group
$ @kbd{mailfromd --test --debug=50 f=gray@@gnu.org client_addr=127.0.0.1}
MX 20 mx20.gnu.org
MX 10 mx10.gnu.org
MX 10 mx10.gnu.org
MX 20 mx20.gnu.org
getting cache info for gray@@gnu.org
found status: success (0), time: Thu Sep 14 14:54:41 2006
getting rate info for gray@@gnu.org-127.0.0.1
found time: 1158245710, interval: 29, count: 5, rate: 0.172414
rate for gray@@gnu.org-127.0.0.1 is 0.162162
updating gray@@gnu.org-127.0.0.1 rates
SET REPLY 450 4.7.0 Mail sending rate exceeded.  Try again later
State envfrom: tempfail
@end group
@end example

@xopindex{echo, mailfromd, echo to stdout or file}.
  If your script uses @code{echo} statements (@pxref{Echo}), they will
print their output on standard error.  To direct them to the standard
output, use the @option{--echo} option.  You can also redirect the
@code{echo} output to arbitrary file, by supplying its name as
argument, as in: @option{--echo=@var{file}}.  @pxref{echo option}.

@xopindex{test, mailfromd, specifying handler name}
  To test any handler, other than @samp{envfrom}, give its name as the
argument to @option{--test} option.  Since this argument is optional,
it is important that it be given immediately after the option, without
any intervening white space, for example @kbd{mailfromd --test=helo},
or @kbd{mailfromd -thelo}.

@cindex @command{mtasim}, introduced
  This method allows to test one handler at a time.  To test the
script as a whole, use @command{mtasim} utility.  When
started it enters interactive mode, similar to that of
@command{sendmail -bs}, where it expects @acronym{SMTP} commands on
its standard input and sends answers to the standard output.  The
@option{--port=auto} command line option instructs it to start
@command{mailfromd} and to create a unique socket for communication
with it.  For the detailed description of the program and the ways to
use it, @xref{mtasim}.

@node Run Mode
@section Run Mode
@cindex run mode
  Mailfromd provides a special option that allows to run arbitrary
@acronym{MFL} scripts.

@xopindex{run, mailfromd, described}
@cindex main, MFL function
  When given the @option{--run} command line option,
@command{mailfromd} loads the script given in its command line, looks for
the function called @samp{main}, and runs it.

This function must be declared as:

@example
func main(...) returns number
@end example

Mailfromd passes all command line arguments that follow the script
name as arguments to that function.  When the function returns, its
return value is used by @command{mailfromd} as exit code.

As an example, suppose the file @file{script.mfl} contains the
following:

@example
func main (...)
  returns number
do
  loop for number i 1,
       while i <= $#,
       set i i + 1
  do
    echo "arg %i=" . $(i)
  done
done
@end example

This function prints all its arguments (@xref{variadic functions}, for
a detailed description of functions with variable
number of arguments).  Now running:

@example
$ @kbd{mailfromd --run script.mfl 1 file dest}
@end example

@noindent
displays the following:

@example
@cartouche
arg 1=1
arg 2=file
arg 3=dest
@end cartouche
@end example

You can direct the script output to the standard output by using the
@option{--echo}, as described above, e.g.:

@example
$ @kbd{mailfromd --echo --run script.mfl 1 file dest}
@end example

Note, that @acronym{MFL} does not have a direct equivalent of
shell's @code{$0} argument.  If your function needs to know the name
of the script that is being executed, use @code{__file__} built-in
constant instead (@pxref{Built-in constants, __file__}).

The name @code{main} is not hard-coded.  You can use the
@option{--run} option to run any function, provided that its
definition is as discussed above.  Just give the name of this
function as the argument to the option.  This argument is optional,
therefore it must be separated from the option by an equals sign (with
no whitespace from either side).  For example, given the command line below,
@command{mailfromd} will load the file @file{script.mfl} and execute the
function @samp{start}:

@example
$ @kbd{mailfromd --run=start script.mfl}
@end example

  If you need to define sendmail macros (@pxref{Sendmail Macros}) for
use in the run mode, place the @var{macro}=@var{value} assignments
@emph{before} the script name, e.g.:

@example
$ @kbd{mailfromd --run=start i=feedbeef client_addr=::1 script.mfl}
@end example

 To summarize, the command line when using the run mode is:

@example
mailfromd [@var{options}] --run [@var{macro}=@var{value}] @var{file} @var{args...}
@end example

Finally, notice that @var{file} together with @var{args...} can be
omitted.  In this case the default script file will be used
(@pxref{default script file}).

@noindent
The @samp{@var{macro}=@var{value}} assignments define Sendmail macros,
@var{args...} are passed as arguments to the @code{main} function
defined in @var{file}, and @var{option} stands for any other
@command{mailfromd} options that might be needed.

@menu
* top-block::   The Top of a Script File.
* getopt::      Parsing Command Line Arguments.
@end menu

@node top-block
@subsection The Top of a Script File

@cindex @samp{#!} shell magic sequence
The @option{--run} option makes it possible to use @command{mailfromd}
scripts as standalone programs.  The traditional way to do so was to
set the executable bit on the script file and to begin the script with
the @dfn{interpreter selector}, i.e. the characters @samp{#!} followed by
the name of the @command{mailfromd} executable, e.g.:

@example
#! /usr/sbin/mailfromd --run
@end example

This would cause the shell to invoke @command{mailfromd} with the
command line constructed from the @option{--run} option, the name
of the invoked script file itself, and any actual arguments from
the invocation.  Once invoked, @command{mailfromd} would treat the
initial @samp{#!} line as a usual single-line comment
(@pxref{Comments}).

However, the interpretation of the @samp{#!} by shells has various
deficiencies, which depend on the actual shell being used.  For
example, some shells pass any characters following the whitespace
after the interpreter name as a single argument, some others silently
truncate the command line after some number of characters, etc.  This
often make it impossible to pass additional arguments to
@command{mailfromd}.  For example, a script which begins with the
following line would most probably fail to be executed properly:

@example
#! /usr/sbin/mailfromd --echo --run
@end example

@cindex @samp{#! ... !#} initial comment
To compensate for these deficiencies and to allow for more complex
invocation sequences, @command{mailfromd} handles initial @samp{#}
in a special way.  If the first line of a source file begins with
@samp{#!/} or @samp{#! /} (with a single space between @samp{!} and
@samp{/}), it is treated as a start of a multi-line comment, which is
closed by the two characters @samp{!#} on a line by themselves.

Thus, the correct way to begin a @command{mailfromd} script is:

@example
#! /usr/sbin/mailfromd --run
!#
@end example

Using this feature, you can start the @command{mailfromd} with
arbitrary shell code, provided it ends with an @code{exec} statement
invoking the interpreter itself.  For example:

@example
#!/bin/sh
exec /usr/sbin/mailfromd --echo --run $0 $@@
!#

func main(...)
  returns number
do
  /* actual mfl code goes here */
done
@end example

Note the use of @samp{$0} and @samp{$@@} to pass the actual script file
name and command line arguments to @command{mailfromd}.

@node getopt
@subsection Parsing Command Line Arguments
@cindex command line arguments, parsing in MFL
@cindex scripting, parsing command line arguments
@cindex parsing command line arguments

A special function is provided to break (parse) the command
line into options, and to check them for validity.  It uses the GNU
getopt routines (@pxref{Getopt, getopt, , libc, The GNU C Library
Reference Manual}).

@deftypefn {Built-in Function} string getopt (number @var{argc}, @
  pointer @var{argv}, ...)
The @command{getopt} function parses the command line arguments, as
supplied by @var{argc} and @var{argv}.  The @var{argc} argument is the
argument count, and @var{argv} is an opaque data structure,
representing the array of arguments@footnote{When @acronym{MFL} has
array data type, the second argument will change to array of
strings.}.  The operator @code{vaptr} (@pxref{vaptr}) is
provided to initialize this argument.

An argument that starts with @samp{-} (and is not exactly @samp{-} or
@samp{--}), is an option element.  An argument that starts with a
@samp{-} is called @dfn{short} or @dfn{traditional} option.  The
characters of this element, except for the initial @samp{-} are option
characters.  Each option character represents a separate option.  An
argument that starts with @samp{--} is called @dfn{long} or @dfn{GNU}
option.  The characters of this element, except for the initial
@samp{--} form the @dfn{option name}.

Options may have arguments.  The argument to a short option is
supplied immediately after the option character, or as the next word
in command line.  E.g., if option @option{-f} takes a mandatory
argument, then it may be given either as @option{-farg} or as
@option{-f arg}.  The argument to a long option is either given
immediately after it and separated from the option name by an equals
sign (as @option{--file=arg}), or is given as the next word in the
command line (e.g.@: @option{--file arg}).

If the option argument is optional, i.e. it may not necessarily be
given, then only the first form is allowed (i.e. either @option{-farg} or
@option{--file=arg}.

The @samp{--} command line argument ends the option list.  Any
arguments following it are not considered options, even if they begin
with a dash.

If @code{getopt} is called repeatedly, it returns successively each of
the option characters from each of the option elements (for short
options) and each option name (for long options).  In this case, the
actual arguments are supplied only to the first invocation.
Subsequent calls must be given two nulls as arguments.  Such
invocation instructs @code{getopt} to use the values saved on the
previous invocation.

When the function finds another option, it returns its character or name
updating the external variable @code{optind} (see below) so that the
next call to @code{getopt} can resume the scan with the following
option.

When there are no more options left, or a @samp{--} argument is
encountered, @code{getopt} returns an empty string.  Then
@code{optind} gives the index in @var{argv} of the first element that
is not an option.

The legitimate options and their characteristics are supplied in
additional arguments to @code{getopt}.  Each such argument is a string
consisting of two parts, separated by a vertical bar (@samp{|}).  Any
one of these parts is optional, but at least one of them must be
present.  The first part specifies short option character.  If it is
followed by a colon, this character takes mandatory argument.  If it
is followed by two colons, this character takes an optional argument.
If only the first part is present, the @samp{|} separator may be
omitted.  Examples:

@table @asis
@item "c"
@itemx "c|"
Short option @option{-c}.

@item "f:"
@itemx "f:|"
Short option @option{-f}, taking a mandatory argument.

@item "f::"
@itemx "f::|"
Short option @option{-f}, taking an optional argument.
@end table

If the vertical bar is present and is followed by any characters, these
characters specify the name of a long option, synonymous to the short
one, specified by the first part.  Any mandatory or optional arguments
to the short option remain mandatory or optional for the corresponding
long option.  Examples:

@table @asis
@item "f:|file"
Short option @option{-f}, or long option @option{--file}, requiring an
argument.

@item "f::|file"
Short option @option{-f}, or long option @option{--file}, taking an
optional argument.
@end table

In any of the above cases, if this option appears in the command line,
@command{getopt} returns its short option character.

To define a long option without a short equivalent, begin it with a
bar, e.g.:

@table @asis
@item "|help"
@end table

If this option is to take an argument, this is specified using the
mechanism described above, except that the short option character
is replaced with a minus sign.  For example:

@table @asis
@item "-:|output"
Long option @option{--output}, which takes a mandatory argument.

@item "-::|output"
Long option @option{--output}, which takes an optional argument.
@end table

@vrindex optarg
If an option is returned that has an argument in the command line,
@code{getopt} stores this argument in the variable @code{optarg}.

@vrindex optind
After each invocation, @code{getopt} sets the variable @code{optind}
to the index of the next @var{argv} element to be parsed.  Thus,
when the list of options is exhausted and the function returned an
empty string, @code{optind} contains the index of the the first
element that is not an option.

@vrindex optopt
When @code{getopt} encounters an option that is not described in its
arguments or if it detects a missing option argument it prints an
error message using @command{mailfromd} logging facilities, stores
the offending option in the variable @code{optopt}, and returns @samp{?}.

@vrindex opterr
If printing error message is not desired (e.g.@: the application is going
to take care of error messaging), it can be disabled by setting the
variable @code{opterr} to @samp{0}.

@cindex controlling argument, getopt
The third argument to @code{getopt}, called @dfn{controlling argument},
may be used to control the behavior of the function.  If it is a
colon, it disables printing the error message for unrecognized options
and missing option arguments (as setting @code{opterr} to @samp{0}
does).  In this case @code{getopt} returns @samp{:}, instead of
@samp{?} to indicate missing option argument.

If the controlling argument is a plus sign, or the environment
variable @env{POSIXLY_CORRECT} is set, then option processing stops as
soon as a non-option argument is encountered.  By default, if options
and non optional arguments are intermixed in @var{argv}, @code{getopt}
permutes them so that the options go first, followed by non-optional
arguments.

If the controlling argument is @samp{-}, then each non-option
element in @var{argv} is handled as if it were the argument of
an option with character code 1 (@samp{"\001"}, in @acronym{MFL}
notation.  This can used by programs that are written to expect options and
other @var{argv}-elements in any order and that care about the
ordering of the two.

Any other value of the controlling argument is handled as an option
definition.
@end deftypefn

@anchor{vaptr}
@cindex vaptr
A special language construct is provided to supply the second
argument (@var{argv}) to @code{getopt} and similar functions:

@example
vaptr(@var{param})
@end example

@noindent
where @var{param} is a positional parameter, from which to start the
array of @var{argv}.  For example:

@example
func main(...)
  returns number
do
  set rc getopt($#, vaptr($1), "|help")
  ...
@end example

Here, @code{vaptr($1)} constructs the @var{argv} array from all the
arguments, supplied to the function @code{main}.

To illustrate the use of @code{getopt} function, let's suppose you
write a script that takes the following options:

@table @option
@item -f @var{file}
@itemx --file=@var{file}

@item --output[=@var{dir}]

@item --help
@end table

Then, the corresponding @code{getopt} invocation will be:

@example
func main(...)
  returns number
do
  loop for string rc getopt($#, vaptr($1),
                            "f:|file", "-::|output", "h|help"),
       while rc != "",
       set rc getopt(0, 0)
  do
    switch rc
    do
      case "f":
        set file optarg
      case "output"
        set output 1
        set output_dir optarg
      case "h"
        help()
      default:
        return 1
    done
    ...
@end example

@node Examining Defaults
@section Examining Default Values
  Sometimes you may need to check what are the default settings of the
@command{mailfromd} binary and what values it uses actually.  Both
tasks are accomplished using the @option{--show-defaults} option.
When used alone, it shows the settings actually in use (default
values, eventually modified by your configuration settings).  When
used together with @command{--no-config}, it displays the compiled
defaults.

The output of @command{mailfromd --show-defaults} looks like this:

@example
@group
version:                   @value{VERSION}
script file:               /etc/mailfromd.mfl
preprocessor:              /usr/bin/m4 -s -DWITH_DKIM -DWITH_MFMOD
                           /var/mailfromd/@value{VERSION}/include/pp-setup
user:                      mail
statedir:                  /var/lib/mailfromd
socket:                    mailfrom
pidfile:                   mailfromd.pid
default syslog:            blocking
include path:              /etc/mailfromd:/usr/share/mailfromd/include:
                           /usr/share/mailfromd/8.14.94/include
module path:               /usr/share/mailfromd:
                           /usr/share/mailfromd/@value{VERSION}
mfmod path:                /usr/lib/mailfromd
optional features:         DKIM, mfmod, STARTTLS
supported database types:  gdbm, bdb
default database type:     bdb
greylist database:         /var/lib/mailfromd/greylist.db
greylist expiration:       86400
tbf database:              /var/lib/mailfromd/tbf.db
tbf expiration:            86400
rate database:             /var/lib/mailfromd/rates.db
rate expiration:           86400
cache database:            /var/lib/mailfromd/mailfromd.db
cache positive expiration: 604800
cache negative expiration: 86400
@end group
@end example

The above format, called @dfn{human-readable}, with two-column output
and long lines split across several physical lines, is used if
@command{mailfromd} is linked with GNU @command{libmailutils}
library version 3.16 or later and its standard output is connected to
a terminal.  Otherwise, @dfn{machine-readable} output format is used,
in which additional whitespace is elided, and long lines are retained
verbatim.  This makes it possible to easily extract default values
using familiar text processing tools, e.g.:

@example
@group
$ @kbd{mailfromd --show-defaults --no-config | grep '^script file:'}
script file:/etc/mailfromd.mfl
$ @kbd{mailfromd --show-defaults --no-config | sed -ne '/^script file:/s///p'}
/etc/mailfromd.mfl
@end group
@end example

The following table describes each line of the output in detail:

@table @asis
@item version
Program version.

@item script file
The script file used by the program.  It is empty if the script file
is not found.

@item preprocessor
Preprocessor command line.  @xref{Preprocessor}.  This value can be
changed in configuration: @xref{conf-preprocessor}.

@item user
System user @command{mailfromd} runs as.  @xref{conf-priv}.

@item statedir
@command{mailfromd} local state directory.  @xref{statedir}.

@item socket
The socket @command{mailfromd} listens on.  If UNIX socket, the
filename is shown.  Unless it begins with @samp{/}, it is relative to
the local state directory. TCP sockets are shown in @ref{milter port
specification}.

@xref{conf-server, listen}.

@item pidfile
PID file name (relative to local state directory, unless absolute).

@xref{conf-base, pidfile}.

@item default syslog
Syslog implementation used: either @samp{blocking}, or
@samp{non-blocking}.

@xref{syslog-async, Using non-blocking syslog}.  See also @ref{Logging and Debugging}.

@item include path
Include search path.  @xref{include search path}.

It can be changed from the command line, using the @option{-I} option
(@pxref{General Settings}), and in configuration file, using the
@code{include-path} statement (@pxref{conf-base, include-path}).

@item module path
Search path for MFL modules.  @pxref{module search path}.

It can be changed from the command line, using the @option{-P}
(@option{--module-path}) option (@pxref{General Settings}), and in
configuration file, using the @code{module-path} statement
(@pxref{conf-base, module-path}).

@item mfmod path
Search path for dynamically loaded modules.  @pxref{mfmod-path}.

@item optional features
Comma-delimited list of optional features, included to
@command{mailfromd} at compile time.  It can contain the following
feature names:

@multitable @columnfractions 0.3 0.6
@headitem Feature @tab Reference
@item DKIM @tab @xref{DKIM}.
@item GeoIP2 @tab @xref{Geolocation functions}.
@item mfmod @tab @xref{mfmod, Dynamically Loaded Modules}.
@item STARTTLS @tab @xref{conf-callout, STARTTLS in call-out}.
@end multitable

@item supported database types
Comma-delimited list of supported database types.  @xref{Databases}.
These types can be used as scheme prefixes in database names
(@pxref{DBM scheme}).

@item default database type
Type of the DBM used by default.  @xref{Databases}.

@item greylist database
@itemx greylist expiration
File name and record expiration time of the greylisting database.
@xref{greylist database}.

@item tbf database
@itemx tbf expiration
File name and record expiration time of the token-bucket filter
rate-limiting database.  @xref{tbf database}.

@item rate database
@itemx rate expiration
@xref{rate database}
File name and record expiration time of the legacy rate-limiting
database.  @xref{Rate limiting functions}.

@item cache database
@itemx cache positive expiration
@itemx cache negative expiration
File name and record expiration times of the call-out cache database.
@xref{cache database}.
@end table

The database settings can be changed using @ref{conf-database}.

@node Logging and Debugging
@section Logging and Debugging

@cindex diagnostics channel
@cindex standard error, using for diagnostics output
@cindex syslog, using for diagnostics output
  Depending on its operation mode, @command{mailfromd} tries to guess
whether it is appropriate to print its diagnostics and informational
messages on standard error or to send them to syslog.  Standard error
is assumed if the program is run with one of the following command
line options:

@itemize @bullet
@item @option{--test} (@pxref{Testing Filter Scripts})
@item @option{--run} (@pxref{Run Mode})
@item @option{--lint} (@pxref{Testing Filter Scripts})
@item @option{--dump-code} (@pxref{Logging and Debugging Options})
@item @option{--dump-grammar-trace} (@pxref{Logging and Debugging Options})
@item @option{--dump-lex-trace} (@pxref{Logging and Debugging Options})
@item @option{--dump-macros} (@pxref{Logging and Debugging Options})
@item @option{--dump-tree} (@pxref{Logging and Debugging Options})
@item @option{--xref} (or @option{--dump-xref}) (@pxref{Testing Filter Scripts})
@end itemize

@cindex syslog, default implementation
@cindex syslog, non-blocking
@cindex syslog, asynchronous
@cindex asynchronous syslog
@cindex non-blocking syslog
@xopindex{logger, mailfromd, introduced}
If none of these are used, @command{mailfromd} switches to syslog as
soon as it finishes its startup.  There are two ways to communicate
with the @command{syslogd} daemon: using the @code{syslog}
function from the system @file{libc} library, which is a @dfn{blocking}
implementation in most cases, or via internal, @dfn{asynchronous},
syslog implementation.  Whether the latter is compiled in and which
implementation is used by default is determined when compiling
the package, as described in @ref{syslog-async, Using non-blocking syslog}.

The @option{--logger} command line option allows you to manually
select the diagnostic channel:

@table @option
@item --logger=stderr
Log everything to the standard error.

@item --logger=syslog
Log to syslog.

@item --logger=syslog:async
Log to syslog using the asynchronous syslog implementation.
@end table

Another way to select the diagnostic channel is by using the
@code{logger} statement in the configuration file.  The statement
takes the same argument as its command line counterpart.

The rest of details regarding diagnostic output are controlled by
the @code{logging} configuration statement.

@cindex syslog facility, selecting
@cindex selecting syslog facility
@cindex syslog facility, default
@cindex default syslog facility
@xopindex{log-facility, mailfromd, introduced}
  The default syslog facility is @samp{mail}; it can be changed
using the @option{--log-facility} command line option or
@code{facility} statement.  Argument in both cases is a valid
facility name, i.e. one of: @samp{user}, @samp{daemon},
@samp{auth}, @samp{authpriv}, @samp{mail}, and @samp{local0}
through @samp{local7}.  The argument can be given in upper, lower or
mixed cases, and it can be prefixed with @samp{log_}:

@anchor{syslog tag}
@cindex syslog tag
@xopindex{log-tag, mailfromd, introduced}
  Another syslog-related parameter that can be configured is the
@dfn{tag}, which identifies @command{mailfromd} messages.  The default tag
is the program name.  It is changed by the @option{--log-tag}
(@option{-L} command line option and the @code{tag} logging statement.

  The following example configures both the syslog facility and tag:

@example
logging @{
  facility local7;
  tag "mfd";
@}
@end example

  As any other @acronym{UNIX} utility, @command{mailfromd} is very quiet unless it
has something important to communicate, such as, e.g.@: an error condition.
A set of command line options is provided for
controlling the verbosity of its output.

@xopindex{trace, mailfromd, introduced}
  The @option{--trace} option enables tracing Sendmail actions
executed during message verifications.  When this option is given,
any @code{accept}, @code{discard}, @code{continue}, etc. triggered
during execution of your filter program will leave their traces in
the log file.  Here is an example of how it looks like (syslog time
stamp, tag and @acronym{PID} removed for readability):

@example
@group
k8DHxvO9030656: /etc/mailfromd.mfl:45: reject 550 5.1.1 Sender validity
not confirmed
@end group
@end example

@noindent
This shows that while verifying the message with @acronym{ID}
@samp{k8DHxvO9030656} the @code{reject} action was executed by filter
script @file{/etc/mailfromd.mfl} at line 45.

@anchor{Message-ID}
@cindex Message-ID, using in @command{mailfromd} logs
@cindex Message-ID, exporting
  The use of message @acronym{ID} in the log deserves a special
notice.  The program will always identify its log messages with
the @samp{Message-Id}, when it is available.  Your responsibility as an
administrator is to make sure it is available by configuring
your @acronym{MTA} to export the macro @samp{i} to @command{mailfromd}.
The rule of thumb is: make @samp{i} available to the very first
handler @command{mailfromd} executes.  It is not necessary to export
it to the rest of the handlers, since @command{mailfromd} will cache
it.  For example, if your filter script contains @samp{envfrom} and
@samp{envrcpt} handlers, export @samp{i} for @samp{envfrom}.
The exact instructions on how to ensure it depend on the
@acronym{MTA} you use.  For @samp{Sendmail}, refer to @ref{Sendmail}.
For MeTA1, see @ref{MeTA1}, and @ref{pmult-macros}.  For
@samp{Postfix}, see @ref{Postfix}.

@xopindex{debug, mailfromd, introduced}
@cindex debugging level
@cindex verbosity level
  To push log verbosity further, use the @code{debug}
configuration statement (@pxref{conf-debug}) or its command line
equivalent, @option{--debug} (@option{-d}, @pxref{--debug}).  Its
argument is a @dfn{debugging level}, whose syntax is described
in @uref{http://mailutils.org/wiki/Debug_level}.

@anchor{debugging level specification}
The debugging output is controlled by a set of levels, each of which
can be set independently of others.  Each debug level consists of a
category name, which identifies the part of package for which
additional debugging is desired, and a level number, which indicates
how verbose should its output be.

Valid debug levels are:

@table @asis
@item error
Displays error conditions which are normally not reported, but passed
to the caller layers for handling.

@item trace0 through trace9
Ten levels of verbosity, @code{trace0} producing less output,
@code{trace9} producing the maximum amount of output.

@item prot
Displays network protocol interaction, where applicable.
@end table

The overall debugging level is specified as a list of individual levels,
delimited with semicolons.  Each individual level can be specified as
one of:

@table @asis
@item !@var{category}
Disables all levels for the specified category.

@item @var{category}
Enables all levels for the specified category.

@item @var{category}.@var{level}
For this category, enables all levels from @samp{error} to @var{level}, inclusive.

@item @var{category}.=@var{level}
Enables only the given @var{level} in this @var{category}.

@item @var{category}.!@var{level}
Disables all levels from @samp{error} to @var{level}, inclusive, in
this @var{category}.

@item @var{category}.!=@var{level}
Disables only the given @var{level} in this @var{category}.

@item @var{category}.@var{levelA}-@var{levelB}
Enables all levels in the range from @var{levelA} to @var{levelB}, inclusive.

@item @var{category}.!@var{levelA}-@var{levelB}
Disables all levels in the range from @var{levelA} to @var{levelB}, inclusive.
@end table

Additionally, a comma-separated list of level specifications is
allowed after the dot.  For example, the following specification:

@example
acl.prot,!=trace9,!trace2
@end example

@noindent
enables in category acl all levels, except trace9, trace0, trace1, and trace2.

Implementation and applicability of each level of debugging differs
between various categories.  Categories built-in to mailutils are
described in @uref{http://mailutils.org/wiki/Debug_level}.  Mailfromd
introduces the following additional categories:

@table @asis
@item db
@table @asis
@item trace0
Detailed debugging info about expiration and compaction.
@item trace5
List records being removed.
@end table

@item dns
@table @asis
@item trace8
Verbose information about attempted DNS queries and their results.
@item trace9
Enables @samp{libadns} internal debugging.
@end table

@item srvman
@table @asis
@item trace0
Additional information about normal conditions, such as subprocess
exiting successfully or a remote party being allowed access by ACL.
@item trace1
Detailed transcript of server manager actions: startup, shutdown,
subprocess cleanups, etc.
@item trace3
Additional info about fd sets.
@item trace4
Individual subserver status information.
@item trace5
Subprocess registration.
@end table

@item pmult
@table @asis
@item trace1
Verbosely list incoming connections, functions being executed and
erroneous conditions: missing headers in SMFIR_CHGHEADER, undefined
macros, etc.
@item trace2
List milter requests being processed.
@item trace7
List SMTP body content in SMFIR_REPLBODY requests.
@item error
Verbosely list mild errors encountered: bad recipient addresses, etc.
@end table

@item callout
@table @asis
@item trace0
Verification session transcript.
@item trace1
MX servers checks.
@item trace5
List emails being checked.
@item trace9
Additional info.
@end table

@item main
@table @asis
@item trace5
Info about hostnames in relayed domain list
@end table

@item engine
Debugging of the virtual engine.
@table @asis
@item trace5
Message modification lists.
@item trace6
Debug message modification operations and Sendmail macros registered.
@item trace7
List SMTP stages (@samp{xxfi_*} calls).
@item trace9
Cleanup calls.
@end table

@item pp
Preprocessor.

@table @asis
@item trace1
Show command line of the preprocessor being run.
@end table

@item prog
@table @asis
@item trace8
Stack operations
@item trace9
Debug exception state save/restore operations.
@end table

@item spf
@table @asis
@item error
Mild errors.
@item trace0
List calls to @samp{spf_eval_record}, @samp{spf_test_record},
@samp{spf_check_host_internal}, etc.
@item trace1
General debug info.
@item trace6
Explicitly list A records obtained when processing the @samp{a} SPF mechanism.
@end table
@end table

Categories starting with @samp{bi_} debug built-in modules:

@table @asis
@item bi_db
Database functions.
@table @asis
@item trace5
List database look-ups.
@item trace6
Trace operations on the greylisting database.
@end table

@item bi_sa
SpamAssassin and ClamAV API.
@table @asis
@item trace1
Report the findings of the @samp{clamav} function.
@item trace9
Trace payload in interactions with @samp{spamd}.
@end table

@item bi_io
I/O functions.
@table @asis
@item trace1
Debug the following functions: @code{open}, @code{spawn}, @code{write}.
@item trace2
Report stderr redirection.
@item trace3
Report external commands being run.
@end table

@item bi_mbox
Mailbox functions.
@table @asis
@item trace1
Report opened mailboxes.
@end table

@item bi_other
Other built-ins.
@table @asis
@item trace1
Report results of checks for existence of usernames.
@end table
@end table

  For example, the following invocation enables levels up to
@samp{trace2} in category @samp{engine}, all levels in category
@samp{savsrv} and levels up to @samp{trace0} in category
@samp{srvman}:

@example
$ @kbd{mailfromd --debug='engine.trace2;savsrv;srvman.trace0'}
@end example

  You need to have sufficient knowledge about @command{mailfromd}
internal structure to use this form of the @option{--debug} option.

@anchor{SMTP transcript}
@xopindex{transcript, mailfromd, introduced}
  To control the execution of the sender verification functions
(@pxref{SMTP Callout functions}), you may use
@option{--transcript} (@option{-X}) command line option which enables
transcripts of @acronym{SMTP} sessions in the logs.  Here is an example
of the output produced running @kbd{mailfromd --transcript}:

@xopindex{transcript, mailfromd, output example}
@example
@group
k8DHxlCa001774: RECV: 220 spf-jail1.us4.outblaze.com ESMTP Postfix
k8DHxlCa001774: SEND: HELO mail.gnu.org.ua
k8DHxlCa001774: RECV: 250 spf-jail1.us4.outblaze.com
k8DHxlCa001774: SEND: MAIL FROM: <>
k8DHxlCa001774: RECV: 250 Ok
k8DHxlCa001774: SEND: RCPT TO: <t1Kmx17Q@@malaysia.net>
k8DHxlCa001774: RECV: 550 <>: No thank you rejected: Account
 Unavailable: Possible Forgery
k8DHxlCa001774: poll exited with status: not_found; sent
 "RCPT TO: <t1Kmx17Q@@malaysia.net>", got "550 <>: No thank you
 rejected: Account Unavailable: Possible Forgery"
k8DHxlCa001774: SEND: QUIT
@end group
@end example

@node Runtime errors
@section Runtime Errors
@cindex runtime error
  A @dfn{runtime error} is a special condition encountered during
execution of the filter program, that makes further execution of
the program impossible.  There are two kinds of runtime errors: fatal
errors, and uncaught exceptions.  Whenever a runtime error occurs,
@command{mailfromd} writes into the log file the following message:

@example
RUNTIME ERROR near @var{file}:@var{line}: @var{text}
@end example

@noindent
where @var{file}:@var{line} indicates approximate source file location
where the error occurred and @var{text} gives the textual description
of the error.

@subheading Fatal runtime errors
@cindex runtime errors, fatal
@cindex fatal runtime errors
  Fatal runtime errors are caused by a condition that is impossible to
fix at run time.  For version @value{VERSION} these are:

@table @asis
@cindex @samp{Not enough memory}, runtime error
@item Not enough memory
  There is not enough memory for the execution of the program.  Try to
make more memory available for @command{mailfromd} or to reduce
its memory requirements by rewriting your filter script.

@cindex @samp{Out of stack space; increase #pragma stacksize}, runtime error
@item Out of stack space; increase #pragma stacksize
@cindex @samp{Heap overrun; increase #pragma stacksize}, runtime error
@itemx Heap overrun; increase #pragma stacksize
@cindex @samp{memory chunk too big to fit into heap}, runtime error
@itemx memory chunk too big to fit into heap
  These errors are reported when there is not enough space left on
stack to perform the requested operation, and the attempt to resize the stack
has failed.  Usually @command{mailfromd} expands the stack when the need
arises (@pxref{automatic stack resizing}).  This runtime error
indicates that there were no more memory available for stack
expansion.  Try to make more memory available for @command{mailfromd}
or to reduce its memory requirements by rewriting your filter script.

@cindex @samp{Stack underflow}, runtime error
@item Stack underflow
  Program attempted to pop a value off the stack but the stack was
already empty.  This indicates an internal error in the
@acronym{MFL} compiler or @command{mailfromd} runtime engine.  If you
ever encounter this error,
please report it to @email{bug-mailfromd@@gnu.org.ua}.  Include
the log fragment (about 10-15 lines before and after this log message)
and your filter script.  @xref{Reporting Bugs}, for more
information about bug reporting.

@cindex @samp{pc out of range}, runtime error
@item pc out of range
  The @dfn{program counter} is out of allowed range.  This is a severe
error, indicating an internal inconsistency in @command{mailfromd}
runtime engine.  If you encounter it, please report it to
@email{bug-mailfromd@@gnu.org.ua}.  Include the log fragment (about
10-15 lines before and after this log message) and your filter script.
@xref{Reporting Bugs}, for more information about how to report a
bug.
@end table

@subheading Programmatic runtime errors
  These indicate a programmatic error in your filter script, which the
@acronym{MFL} compiler was unable to discover at compilation stage:

@table @asis
@cindex @samp{Invalid exception number}, runtime error
@item Invalid exception number: @var{n}
  The @code{throw} statement used a not existent exception number @var{n}.
Fix the statement and restart @command{mailfromd}.  @xref{throw}, for
the information about @code{throw} statement and see @ref{Exceptions},
for the list of available exception codes.

@cindex @samp{No previous regular expression}, runtime error
@item No previous regular expression
  You have used a back-reference (@pxref{Back references}), where there
is no previous regular expression to refer to.  Fix this line in
your code and restart the program.

@cindex @samp{Invalid back-reference number}, runtime error
@item Invalid back-reference number
  You have used a back-reference (@pxref{Back references}), with a
number greater than the number of available groups in the previous
regular expression.  For example:

@example
@group
  if $f matches "(.*)@@gnu.org"
    # @r{Wrong: there is only one group in the regexp above!}
    set x \2
  @dots{}
@end group
@end example

  Fix your code and restart the daemon.
@end table

@anchor{uncaught exceptions}
@subheading Uncaught exceptions
  Another kind of runtime errors are @dfn{uncaught exceptions},
i.e. exceptional conditions for which no handler was installed
(@xref{Exceptions}, for information on exceptions and on how to
handle them).  These errors mean that the programmer (i.e. you), made
no provision for some specific condition.  For example, consider the
following code:

@example
@group
prog envfrom
do
  if $f mx matches "yahoo.com"
    foo()
  fi
done
@end group
@end example

@noindent
It is syntactically correct, but it overlooks the fact that @code{mx
matches} may generate @code{e_temp_failure} exception, if the underlying
@acronym{DNS} query has timed out (@pxref{Special comparisons}).  If
this happens, @command{mailfromd} has no instructions on what to do
next and reports an error.  This can easily be fixed using a
@code{try}/@code{catch} (@pxref{Catch and Throw}) statement, e.g.:

@example
prog envfrom
do
  try
  do
    if $f mx matches "yahoo.com"
      foo()
    fi
  done
  # Catch @acronym{DNS} errors
  catch e_temp_failure or e_failure
  do
    tempfail 451 4.1.1 "MX verification failed"
  done
done
@end example

  Another common case are undefined Sendmail macros.  In this case the
@code{e_macroundef} exception is generated:

@example
RUNTIME ERROR near foo.c:34: Macro not defined: @{client_adr@}
@end example

@noindent
These can be caused either by misspelling the macro name (as in the
example message above) or by failing to export the required name in
Sendmail milter configuration (@pxref{exporting macros}).  This error
should be fixed either in your source code or in @file{sendmail.cf}
file, but if you wish to provide a special handling for it, you can
use the following catch statement:

@example
@group
catch e_macroundef
do
  @dots{}
done
@end group
@end example

@anchor{tracing runtime errors}
@cindex runtime errors, tracing
  Sometimes the location indicated with the runtime error message is
not enough to trace the origin of the error.  For example, an error
can be generated explicitly with @code{throw} statement
(@pxref{throw}):

@example
RUNTIME ERROR near match_cidr.mfl:30: invalid CIDR (text)
@end example

  If you look in module @file{match_cidr.mfl}, you will see
the following code (line numbers added for reference):

@example
23 func match_cidr(string ipstr, string cidr)
24   returns number
25 do
26   number netmask
27
28   if cidr matches '^(([0-9]@{1,3@}\.)@{3@}[0-9]@{1,3@})/([0-9][0-9]?)'
29     return inet_aton(ipstr) & len_to_netmask(\3) = inet_aton(\1)
30   else
31     throw invcidr "invalid CIDR (%cidr)"
32   fi
33   return 0
34 done
@end example

@xopindex{stack-trace, mailfromd, explained}
  Now, it is obvious that the value of @code{cidr} argument to
@code{match_cidr} was wrong, but how to find the caller that passed
the wrong value to it?  The special command line option
@option{--stack-trace} is provided for this.  This option enables
dumping @dfn{stack traces} when a fatal error occurs.  Traces
contain information about function calls.  Continuing our example,
using the @option{--stack-trace} option you will see the following diagnostics:

@example
RUNTIME ERROR near match_cidr.mfl:30: invalid CIDR (127%)
mailfromd: Stack trace:
mailfromd: 0077: match_cidr.mfl:31: match_cidr
mailfromd: 0096: test.mfl:13: bar
mailfromd: 0110: mailfromd.mfl:18: foo
mailfromd: Stack trace finishes
mailfromd: Execution of the configuration program was not finished
@end example

@cindex stack traces, reading
  Each trace line describes one stack frame.  The lines appear in the
order of most recently called to least recently called.  Each frame
consists of:

@enumerate 1
@item Value of the program counter at the time of its execution;
@item Source code location, if available;
@item Name of the function called.
@end enumerate

  Thus, the example above can be read as: ``the function
@code{match_cidr} was called by the function @code{bar} in file
@file{test.mfl} at line 13.  This function was called from
the function @code{bar}, in file @file{test.mfl} at line 13.  In its turn,
@code{bar} was called by the function @code{foo}, in file
@file{mailfromd.mfl} at line 18''.

  Examining caller functions will help you localize the source of
the error and fix it.

@cindex @code{stack_trace} function, introduced
  You can also request a stack trace any place in your code, by
calling the @code{stack_trace} function.  This can be useful for
debugging.

@node Notes
@section Notes and Cautions

  This section discusses some potential culprits in the
@acronym{MFL}.

  It is important to execute special caution when writing format
strings for @code{sprintf} (@pxref{String formatting}) and @code{strftime}
(@pxref{strftime}) functions.  They use @samp{%} as a character
introducing conversion specifiers, while the same character is used to
expand a @acronym{MFL} variable within a string.  To prevent this
misinterpretation, always enclose format specification in @emph{single
quotes} (@pxref{singe-vs-double}).  To illustrate this, let's consider
the following example:

@example
echo sprintf ("Mail from %s", $f)
@end example

  If a variable @code{s} is not declared, this line will produce the
@samp{Variable s is not defined} error message, which will allow you
to identify and fix the bug.  The situation is considerably worse if
@code{s} is declared.  In that case you will see no warning message,
as the statement is perfectly valid, but at the run-time the variable
@code{s} will be interpreted within the format string, and its value
will replace @code{%s}.  To prevent this from happening, single quotes
must be used:

@example
echo sprintf ('Mail from %s', $f)
@end example

This does not limit the functionality, since there is no need to fall
back to variable interpretation in format strings.

@anchor{variable--constant clashes}
Yet another dangerous feature of the language is the way to refer to
variable and constant names within literal strings.  To expand a
variable or a constant the same notation is used
(@xref{Variables}, and @pxref{Constants}).  Now, lets consider the
following code:

@example
const x 2
string x "X"

prog envfrom
do
  echo "X is %x"
done
@end example

Does @code{%x} in @code{echo} refers to the variable or to the
constant?  The correct answer is @samp{to the variable}.  When executed, this
code will print @samp{X is X}.

As of version @value{VERSION}, @command{mailfromd} will always print a
diagnostic message whenever it stumbles upon a variable having the
same name as a previously defined constant or vice versa.  The
resolution of such name clashes is described in detail in
@xref{variable--constant shadowing}.

Future versions of the program may provide a non-ambiguous way of
referring to variables and constants from literal strings.

@node MFL
@chapter Mail Filtering Language
@cindex MFL
@cindex mail filtering language

  The @dfn{mail filtering language}, or @acronym{MFL}, is a special
language designed for writing filter scripts.  It has a simple syntax,
similar to that of Bourne shell.  In contrast to the most existing
programming languages, @acronym{MFL} does not have any special
terminating or separating characters (like, e.g. newlines and
semicolons in shell)@footnote{There are two noteworthy exceptions:
@code{module} and @code{from ... import} statements, which must be
terminated with a period.  For details, refer to @ref{module structure}, and @ref{import}.}.  All syntactical entities
are separated by any amount of white-space characters (i.e. spaces,
tabulations or newlines).

  The following sections describe @acronym{MFL} syntax in detail.

@menu
* Comments::                    Comments.
* include::
* line::
* Generated warnings and errors::
* Pragmas::                     Pragmatic comments.
* Data Types::
* Numbers::
* Literals::
* Here Documents::
* Sendmail Macros::
* Constants::
* Variables::
* Back references::
* Handlers::
* Special handlers::            Initialization and cleanup handlers.
* Functions::                   Functions.
* Expressions::                 Expressions.
* Shadowing::                   Variable and Constant Shadowing.
* Statements::
* Conditionals::                Conditional Statements.
* Loops::                       Loop Statements.
* Exceptions::                  Exceptional Conditions and their Handling.
* Polling::                     Sender Verification Tests.
* Modules::                     Modules are Collections of Useful Functions.
* mfmod::                       Dynamically Loaded Modules.
* Preprocessor::                Input Text Is Preprocessed.
* Filter Script Example::       A Working Filter Script Explained.
* Reserved Words::              A Reference List of Reserved Words.
@end menu

@node Comments
@section Comments

@cindex comments
  Two types of comments are allowed: @sc{c}-style, enclosed between
@samp{/*} and @samp{*/}, and shell-style, starting with @samp{#}
character and extending up to the end of line:

@example
@group
/* This is
   a comment. */
# And this too.
@end group
@end example

  There are, however, several special cases, where the characters
following @samp{#} are not ignored:

  If the first line begins with @samp{#!/} or @samp{#! /}, this is
treated as a start of a multi-line comment, which is closed by the
characters @samp{!#} on a line by themselves.  This feature allows for
writing sophisticated scripts.  @xref{top-block}, for a detailed
description.

  A @samp{#} is followed by @samp{include}, @samp{include_once},
@samp{line}, @samp{error}, or @samp{warning} is treated specially.
These cases are covered by the subsequent sections.

@node include
@section #include and #include_once
@cindex #include statement
@cindex including files
@kwindex #include
  If @samp{#} is followed by word @samp{include} (with
optional whitespace between them), this statement requires inclusion
of the specified file, as in @sc{c}.  There are two forms of the
@samp{#include} statement:

@enumerate 1
@item @code{#include <@var{file}>}
@item @code{#include "@var{file}"}
@end enumerate

  The quotes around @var{file} in the second form quotes are optional.

@anchor{include search path}
@cindex include search path, introduced
  Both forms are equivalent if @var{file} is an absolute file name.
Otherwise, the first form will look for @var{file} in the @dfn{include
search path}.  The second one will look for it in the current working
directory first, and, if not found there, in the include search
path.

The default include search path is:

@enumerate 1
@item @file{@var{prefix}/share/mailfromd/include}
@item @file{@var{prefix}/share/mailfromd/@value{VERSION}/include}
@end enumerate

@noindent
where @var{prefix} is the installation prefix.

  New directories can be appended in front of it using @option{-I}
(@option{--include-path}) command line option, or @code{include-path}
configuration statement (@pxref{conf-base, include-path}).

For example, invoking

@example
$ @kbd{mailfromd -I/var/mailfromd -I/com/mailfromd}
@end example

@noindent
creates the following include search path

@enumerate 1
@item @file{/var/mailfromd}
@item @file{/com/mailfromd}
@item @file{@var{prefix}/share/mailfromd/include}
@item @file{@var{prefix}/share/mailfromd/@value{VERSION}/include}
@end enumerate

@anchor{include_once}
@cindex include_once
  Along with @code{#include}, there is also a special form
@code{#include_once}, that has the same syntax:

@example
#include_once <@var{file}>
#include_once "@var{file}"
@end example

  This form works exactly as @code{#include}, except that, if the
@var{file} has already been included, it will not be included
again.  As the name suggests, it will be included only once.

  This form should be used to prevent re-inclusions of a code, which
can cause problems due to function redefinitions, variable
reassignments etc.

@node line
@section #line
@cindex line, @code{#line} statement
@kwindex #line

  A line in the form

@example
#line @var{number} "@var{identifier}"
@end example

@noindent
causes the @acronym{MFL} compiler to believe, for purposes of error
diagnostics, that the line number of the next source line is given by
@var{number} and the current input file is named by @var{identifier}.
If the identifier is absent, the remembered file name does not change.

  Line directives in @command{cpp} style are also understood:

@example
# @var{number} "@var{identifier}"
@end example

@node Generated warnings and errors
@section #warning and #error
@kwindex #warning
@cindex #warning
If @samp{#} is followed by the word @samp{warning}, any amount of
whitespace and a string in double quotes, the compiler will use that
string to generate a warning message at that point.  As usual, 
whitespace characters are allowed between @samp{#} and @samp{warning}:

@example
# warning "The code below is suspicious"
@end example

@kwindex #error
@cindex #error
Similarly, @samp{#} followed by the word @samp{error}, whitespace and
a doubly-quoted string causes the compiler to generate a compilation
error at that point.

To use backslash or double quote in the message text, precede them
with a single slash, e.g.:

@example
#error "the \"quoted\" text"
@end example

A backslash in front of any other character is retained.

@node Pragmas
@section Pragmatic comments
@cindex pragmatic comments
@cindex #pragma statement
@kwindex #pragma
  If @samp{#} is immediately followed by word @samp{pragma} (with
optional whitespace between them), such a construct introduces a
@dfn{pragmatic comment}, i.e. an instruction that controls some
configuration setting.

  The available pragma types are described in the following subsections.

@menu
* prereq::          Pragma prereq.
* stacksize::       Pragma stacksize.
* regex::           Pragma regex.
* dbprop::          Pragma dbprop.
* greylist::        Pragma greylist.
* miltermacros::    Pragma miltermacros.
* provide-callout:: Pragma provide-callout.
@end menu

@node prereq
@subsection Pragma prereq

  The @code{#pragma prereq} statement ensures that the correct
@command{mailfromd} version is used to compile the source file it
appears in.  It takes version number as its arguments and produces
a compilation error if the actual @command{mailfromd} version number
is earlier than that.  For example, the following statement:

@example
#pragma prereq 7.0.94
@end example

@noindent
results in error if compiled with @command{mailfromd} version 7.0.93
or prior.

@node stacksize
@subsection Pragma stacksize

  The @code{stacksize} pragma sets the initial size of the run-time
stack and may also define the policy of its growing, in case it
becomes full.  The default stack size is @value{STACK_SIZE} words.  You may
need to increase this number if your configuration program uses
recursive functions or does an excessive amount of string manipulations.

@deffn {pragma} stacksize size [incr [max]]
  Sets stack size to @var{size} units.  Optional @var{incr} and
@var{max} define stack growth policy (see below).  The default
@dfn{units} are words.  The following example sets the stack
size to 7168 words:

@example
#pragma stacksize 7168
@end example

The @var{size} may end with a @dfn{unit size} suffix:

@float Table, unit-size
@caption{Unit Size Suffix}
@multitable @columnfractions 0.3 0.6
@headitem Suffix @tab Meaning
@item k @tab Kiloword, i.e. 1024 words
@item m @tab Megawords, i.e. 1048576 words
@item g @tab Gigawords,
@item t @tab Terawords (ouch!)
@end multitable
@end float

File suffixes are case-insensitive, so the following two pragmas are
equivalent and set the stack size to @code{7*1048576 = 7340032} words:

@example
@group
#pragma stacksize 7m
#pragma stacksize 7M
@end group
@end example

@anchor{automatic stack resizing}
When the @acronym{MFL} engine notices that there is no more stack space
available, it attempts to expand the stack.  If this attempt succeeds, the
operation continues.  Otherwise, a runtime error is reported and the
execution of the filter stops.

@cindex stack growth policy
@cindex growth policy, stack
The optional @var{incr} argument to @code{#pragma stacksize} defines growth
policy for the stack.  Two growth policies are implemented:
@dfn{fixed increment policy}, which expands stack in a fixed
number of @dfn{expansion chunks}, and @dfn{exponential growth policy}, which
duplicates the stack size until it is able to accommodate the needed
number of words.  The fixed increment policy is the default.  The default
chunk size is @value{STACK_INCR} words.

  If @var{incr} is the word @samp{twice}, the duplicate policy is
selected.  Otherwise @var{incr} must be a positive number optionally
suffixed with a size suffix (see above).  This indicates the expansion
chunk size for the fixed increment policy.

  The following example sets initial stack size to 10240, and
expansion chunk size to 2048 words:

@example
#pragma stacksize 10M 2K
@end example

  The pragma below enables exponential stack growth policy:

@example
#pragma stacksize 10240 twice
@end example

In this case, when the run-time evaluator hits the stack size limit,
it expands the stack to twice the size it had before.  So, in the
example above, the stack will be sequentially expanded to the
following sizes: 20480, 40960, 81920, 163840, etc.

  The optional @var{max} argument defines the maximum size of
the stack.  If stack grows beyond this limit, the execution of the
script will be aborted.
@end deffn

  If you are concerned about the execution time of your script, you
may wish to avoid stack reallocations.  To help you find out the
optimal stack size, each time the stack is expanded,
@command{mailfromd} issues a warning in its log file, which looks like
this:

@example
warning: stack segment expanded, new size=8192
@end example

You can use these messages to adjust your stack size configuration
settings.

@node regex
@subsection Pragma regex
@anchor{pragma regex}
  The @samp{#pragma regex}, controls
compilation of regular expressions.  You can use any number
of such pragma directives in your @file{mailfromd.mfl}.  The scope of
@samp{#pragma regex} extends to the next occurrence of this directive
or to the end of the script file, whichever occurs first.

@deffn {pragma} regex [push|pop] flags
  The optional @var{push|pop} parameter is one of the words @samp{push} or
@samp{pop} and is discussed in detail below.  The @var{flags}
parameter is a whitespace-separated list of @dfn{regex flags}.  Each
regex-flag is a word specifying some regex feature.  It can be
preceded by @samp{+} to enable this feature (this is the default), by
@samp{-} to disable it or by @samp{=} to reset regex flags to its
value.  Valid regex-flags are:

@table @samp
@item extended
Use @acronym{POSIX} Extended Regular Expression syntax when
interpreting regex.  If not set, @acronym{POSIX} Basic Regular
Expression syntax is used.

@item icase
Do not differentiate case.  Subsequent regex searches will be case
insensitive.

@item newline
@dfn{Match-any-character} operators don't match a newline.

A non-matching list (@samp{[^...]}) not containing a newline does not
match a newline.

@dfn{Match-beginning-of-line} operator (@samp{^}) matches the empty
string immediately after a newline.

@dfn{Match-end-of-line} operator (@samp{$}) matches the empty string
immediately before a newline.
@end table

For example, the following pragma enables @acronym{POSIX} extended,
case insensitive matching (a good thing to start your
@file{mailfromd.mfl} with):

@example
#pragma regex +extended +icase
@end example
@end deffn

Optional modifiers @samp{push} and @samp{pop} can be used to maintain
a stack of regex flags.  The statement

@example
#pragma regex push [@var{flags}]
@end example

@noindent
saves current regex flags on stack and then optionally modifies them
as requested by @var{flags}.

The statement

@example
#pragma regex pop [@var{flags}]
@end example

@noindent
does the opposite: restores the current regex flags from the top of
stack and applies @var{flags} to it.

This statement is useful in module and include files to avoid disturbing user
regex settings.  E.g.:

@example
@group
#pragma regex push +extended +icase
 .
 .
 .
#pragma regex pop
@end group
@end example

@node dbprop
@subsection Pragma dbprop

@deffn {pragma} dbprop pattern prop @dots{}
  This pragma configures properties for a @acronym{DBM} database.
@xref{Database functions}, for its detailed description.
@end deffn

@node greylist
@subsection Pragma greylist

@deffn {pragma} greylist type
  Selects the greylisting implementation to use.  Allowed
values for @var{type} are:

@table @asis
@item traditional
@itemx gray
Use the traditional greylisting implementation.  This is the default.

@item con-tassios
@itemx ct
Use Con Tassios greylisting implementation.
@end table

@xref{greylisting types}, for a detailed description of these
greylisting implementations.
@end deffn

Notice, that this pragma can be used only once.  A second use of this
pragma would constitute an error, because you cannot use both greylisting
implementations in the same program.

@node miltermacros
@subsection Pragma miltermacros
@deffn {pragma} miltermacros handler macro @dots{}
Declare that the Milter stage @var{handler} uses @acronym{MTA} macro
listed as the rest of arguments.  The @var{handler} must be a valid
handler name (@pxref{Handlers}).
@end deffn

  The @command{mailfromd} parser collects the names of the macros
referred to by a @samp{$@var{name}} construct within a handler
(@pxref{Sendmail Macros}) and declares them automatically for
corresponding handlers.  It is, however, unable to track macros
used in functions called from handler as well as those referred to
via @code{getmacro} and @code{macro_defined} functions.  Such
macros should be declared using @samp{#pragma miltermacros}.

  During initial negotiation with the @acronym{MTA},
@command{mailfromd} will ask it to export the macro names declared
automatically or by using the @samp{#pragma miltermacros}.  The
@acronym{MTA} is free to honor or to ignore this request.  In
particular, Sendmail versions prior to 8.14.0 and Postfix versions
prior to 2.5 do not support this feature.  If you use one of these,
you will need to export the needed macros explicitly in the
@acronym{MTA} configuration.  For more details, refer to the section
in @ref{MTA Configuration} corresponding to your @acronym{MTA} type.

@node provide-callout
@subsection Pragma provide-callout

  The @code{#pragma provide-callout} statement is used in the
@file{callout} module to inform @command{mailfromd} that the
module has been loaded.

  Do not use this pragma.

@node Data Types
@section Data Types

  The @command{mailfromd} filter script language operates on entities
of two types: numeric and string.

  The @dfn{numeric} type is represented internally as a signed long
integer.  Depending on the machine architecture, its size can vary.
For example, on machines with Intel-based @acronym{CPU}s it is 32 bits long.

  A @dfn{string} is a string of characters of arbitrary length.
Strings can contain any characters except @acronym{ASCII} @sc{nul}.

  There is also a @dfn{generic pointer}, which is designed to
facilitate certain operations.  It appears only in the @code{body}
handler.  @xref{body handler}, for more information about it.

@node Numbers
@section Numbers

  A @dfn{decimal number} is any sequence of decimal digits, not
beginning with @samp{0}.

  An @dfn{octal number} is @samp{0} followed by any number of octal
digits (@samp{0} through @samp{7}), for example: @code{0340}.

  A @dfn{hex number} is @samp{0x} or @samp{0X} followed by any number
of hex digits (@samp{0} through @samp{9} and @samp{a} through @samp{f}
or @samp{A} through @samp{F}), for example: @code{0x3ef1}.

@node Literals
@section Literals

@cindex literals
  A literal is any sequence of characters enclosed in single or
double quotes.

  After @code{tempfail} and @code{reject} actions two special kinds of
literals are recognized: three-digit numeric values represent
@acronym{RFC} 2821 reply codes, and literals consisting of tree digit
groups separated by dots represent an extended reply code as per
@acronym{RFC} 1893/2034.  For example:

@example
@group
510   # @r{A reply code}
5.7.1 # @r{An extended reply code}
@end group
@end example

@anchor{Double-quoted strings}
@subheading Double-quoted strings

  String literals enclosed in double quotation marks
(@dfn{double-quoted strings}) are subject to @dfn{backslash interpretation},
@dfn{macro expansion}, @dfn{variable interpretation} and @dfn{back reference
interpretation}.

@cindex backslash interpretation
  @dfn{Backslash interpretation} is performed at compilation time.  It
consists in replacing the following @dfn{escape sequences} with the
corresponding single characters:

@float Table, backslash-interpretation
@caption{Backslash escapes}
@multitable @columnfractions 0.30 .5
@item Sequence @tab Replaced with
@item \a @tab Audible bell character (@acronym{ASCII} 7)
@item \b @tab Backspace character (@acronym{ASCII} 8)
@item \f @tab Form-feed character (@acronym{ASCII} 12)
@item \n @tab Newline character (@acronym{ASCII} 10)
@item \r @tab Carriage return character (@acronym{ASCII} 13)
@item \t @tab Horizontal tabulation character (@acronym{ASCII} 9)
@item \v @tab Vertical tabulation character (@acronym{ASCII} 11)
@end multitable
@end float

  In addition, the sequence @samp{\@var{newline}} has the same
effect as @samp{\n}, for example:

@example
@group
"a string with\
 embedded newline"
"a string with\n embedded newline"
@end group
@end example

Any escape sequence of the form @samp{\x@var{hh}}, where @var{h}
denotes any hex digit is replaced with the character whose @acronym{ASCII} value
is @var{hh}.  For example:

@example
"\x61nother" @result{} "another"
@end example

  Similarly, an escape sequence of the form @samp{\0@var{ooo}}, where
@var{o} is an octal digit, is replaced with the character whose @acronym{ASCII}
value is @var{ooo}.

@cindex variable interpretation
@cindex macro expansion
  Macro expansion and variable interpretation occur at run-time.  During
these phases all Sendmail macros (@pxref{Sendmail Macros}),
@command{mailfromd} variables (@pxref{Variables}), and constants
(@pxref{Constants}) referenced in the string are replaced by their
actual values.  For example, if the Sendmail macro @code{f} has the
value @samp{postmaster@@gnu.org.ua} and the variable @code{last_ip}
has the value @samp{127.0.0.1}, then the
string@footnote{Implementation note: actually, the references
are not interpreted within the string, instead, each such string is
split at compilation time into a series of concatenated atoms.  Thus,
our sample string will actually be compiled as:

@example
$f . " last connected from " . last_ip . ";"
@end example

@xref{Concatenation}, for a description of this construct.  You can
easily see how various strings are interpreted by using
@option{--dump-tree} option (@pxref{--dump-tree}).  In this case,
it will produce:

@example
  CONCAT:
    CONCAT:
      CONCAT:
        SYMBOL: f
        CONSTANT: " last connected from "
      VARIABLE last_ip (13)
    CONSTANT: ";"
@end example
}

@example
"$f last connected from %last_ip;"
@end example

@noindent
will be expanded to

@example
"postmaster@@gnu.org.ua last connected from 127.0.0.1;"
@end example

@cindex back reference interpretation
  A @dfn{back reference} is a sequence @samp{\@var{d}}, where @var{d}
is a decimal number.  It refers to the @var{d}th parenthesized
subexpression in the last @command{matches} statement@footnote{The
subexpressions are numbered by the positions of their opening
parentheses, left to right.}.  Any back reference occurring within a
double-quoted string is replaced by the value of the corresponding
subexpression.  @xref{Special comparisons}, for a detailed
description of this process.  Back reference interpretation is
performed at run time.

@anchor{singe-vs-double}
@subheading Single-quoted strings

  Any characters enclosed in single quotation marks are read unmodified.

  The following examples contain pairs of equivalent strings:

@example
@group
"a string"
'a string'

"\\(.*\\):"
'\(.*\):'
@end group
@end example

  Notice the last example.  Single quotes are particularly useful in writing
regular expressions (@pxref{Special comparisons}).

@node Here Documents
@section Here Documents

@cindex here document
@cindex multiline strings
  @dfn{Here-document} is a special form of a string literal is, allowing to
specify multiline strings without having to use backslash
escapes.  The format of here-documents is:

@example
@group
<<[@var{flags}]@var{word}
@dots{}
@var{word}
@end group
@end example

  The @code{<<@var{word}} construct instructs the parser to read all
the following lines up to the line containing only @var{word}, with
possible trailing blanks.  The lines thus read are concatenated
together into a single string.  For example:

@example
@group
set str <<EOT
A multiline
string
EOT
@end group
@end example

  The body of a here-document is interpreted the same way as
double-quoted strings (@pxref{Double-quoted strings}).  For example,
if Sendmail macro @code{f} has the value @code{jsmith@@some.com} and
the variable @code{count} is set to @code{10}, then the following string:

@example
@group
set s <<EOT
<$f> has tried to send %count mails.
Please see docs for more info.
EOT
@end group
@end example

@noindent
will be expanded to:

@example
@group
<jsmith@@some.com> has tried to send 10 mails.
Please see docs for more info.
@end group
@end example

  If the @var{word} is quoted, either by enclosing it in single quote
characters or by prepending it with a backslash, all interpretations
and expansions within the document body are suppressed.  For
example:

@example
@group
set s <<'EOT'
The following line is read verbatim:
<$f> has tried to send %count mails.
Please see docs for more info.
EOT
@end group
@end example

  Optional @var{flags} in the here-document construct control the way
leading white space is handled.  If @var{flags} is @code{-} (a dash),
then all leading tab characters are stripped from input lines and the
line containing @var{word}.  Furthermore, if @code{-} is followed by a
single space, all leading whitespace is stripped from them.  This
allows here-documents within configuration scripts to be indented in a
natural fashion.  Examples:

@example
@group
<<- TEXT
    <$f> has tried to send %count mails.
    Please see docs for more info.
TEXT
@end group
@end example

  Here-documents are particularly useful with @code{reject} actions
(@pxref{reject and tempfail syntax}).

@node Sendmail Macros
@section Sendmail Macros
@cindex macros, referencing
@cindex Sendmail macros, referencing
  Sendmail macros are referenced exactly the same way they are in
@file{sendmail.cf} configuration file, i.e.@: @samp{$@var{name}},
where @var{name} represents the macro name.  Notice, that the notation
is the same for both single-character and multi-character macro names.
For consistency with the @command{Sendmail} configuration the
@samp{$@{@var{name}@}} notation is also accepted.

  Another way to reference Sendmail macros is by using function
@code{getmacro} (@pxref{Macro access}).

  Sendmail macros evaluate to string values.

  Notice, that to reference a macro, you must properly export it in
your @acronym{MTA} configuration.  Attempt to reference a not exported
macro will result in raising a @code{e_macroundef} exception at the run time
(@pxref{uncaught exceptions}).

@node Constants
@section Constants

@cindex constants, defining
@cindex const
  A @dfn{constant} is a symbolic name for an @acronym{MFL} value.
Constants are defined using @code{const} statement:

@example
[@var{qualifier}] const @var{name} @var{expr}
@end example

@noindent
where @var{name} is an identifier, and @var{expr} is any valid
@acronym{MFL} expression evaluating immediately to a constant literal
or numeric value.  Optional @var{qualifier} defines the scope of
visibility for that constant (@pxref{scope of visibility}): either
@code{public} or @code{static}.

@cindex constants, using in program text
  Once defined, any appearance of @var{name} in the program text is
replaced by its value.  For example:

@example
const x 10/5
const text "X is "
@end example

@noindent
defines the numeric constant @samp{x} with the value @samp{5}, and the
literal constant @samp{text} with the value @samp{X is }.

@cindex enumeration
  A special construct is provided to define a series of numeric
constants (an @dfn{enumeration}):

@example
@group
[@var{qualifier}] const
do
  @var{name0} [@var{expr0}]
  @var{name1} [@var{expr1}]
  ...
  @var{nameN} [@var{exprN}]
done
@end group
@end example

@noindent
Each @var{exprN}, if present, must evaluate to a constant numeric
expression.  The resulting value will be assigned to constant
@var{nameN}.  If @var{exprN} is not supplied, the constant will be
defined to the value of the previous constant plus one.  If
@var{expr0} is not supplied, 0 is assumed.

For example, consider the following statement

@example
@group
const
do
  A
  B
  C 10
  D
done
@end group
@end example

@noindent
This defines @samp{A} to 0, @samp{B} to 1, @samp{C} to 10 and @samp{D}
to 11.

As a matter of fact, @var{exprN} may also evaluate to a constant
string expression, provided that all expressions in the enumeration
@samp{const} statement are provided.  That is, the following is
correct:

@example
@group
const
do
  A "one"
  B "two"
  C "three"
  D "four"
done
@end group
@end example

@noindent
whereas the following is not:

@example
@group
const
do
  A "one"
  B
  C "three"
  D "four"
done
@end group
@end example

Trying to compile the latter example will produce:

@example
mailfromd: @var{filename}:5.3: initializer element is not numeric
@end example

@noindent
which means that @command{mailfromd} was trying to create constant
@samp{B} with the value of @samp{A} incremented by one, but was unable
to do so, because the value in question was not numeric.

@cindex constants, using in literals
  Constants can be used in normal MFL expressions as well as in
literals.  To expand a constant within a literal string, prepend a
percent sign to its name, e.g.:

@example
echo "New %text %x" @result{} "New X is 2"
@end example

  This way of expanding constants creates an ambiguity if there happen
to be a variable of the same name as the constant.
@xref{variable--constant clashes}, for more information of this case
and ways to handle it.

@menu
* Built-in constants::
@end menu

@node Built-in constants
@subsection Built-in constants
@cindex built-in constants
@cindex constants, built-in
  Several constants are built into the @acronym{MFL}
compiler.  To discern them from user-defined ones,
their names start and end with two underscores (@samp{__}).

  The following constants are defined in @command{mailfromd} version
@value{VERSION}:

@deftypevr {Built-in constant} string __file__
Expands to the name of the current source file.
@end deftypevr

@deftypevr {Built-in constant} string __function__
Expands to the name of the current lexical context,
i.e. the function or handler name.
@end deftypevr

@deftypevr {Built-in constant} string __git__
This built-in constant is defined for alpha versions only.
Its value is the Git tag of the recent commit corresponding to that
version of the package.  If the release contains some uncommitted
changes, the value of the @samp{__git__} constant ends with
the suffix @samp{-dirty}.
@end deftypevr

@deftypevr {Built-in constant} number __line__
Expands to the current line number in the input source file.
@end deftypevr

@deftypevr {Built-in constant} number __major__
Expands to the major version number.

  The following example uses @code{__major__} constant to determine
if some version-dependent feature can be used:

@example
@group
if __major__ > 2
  # @r{Use some version-specific feature}
fi
@end group
@end example

@end deftypevr

@deftypevr {Built-in constant} number __minor__
Expands to the minor version number.
@end deftypevr

@deftypevr {Built-in constant} string __module__
Expands to the name of the current module (@pxref{Modules}).
@end deftypevr

@deftypevr {Built-in constant} string __package__
Expands to the package name (@samp{mailfromd})
@end deftypevr

@deftypevr {Built-in constant} number __patch__
For alpha versions and maintenance releases expands to the version
patch level.  For stable versions, expands to @samp{0}.
@end deftypevr

@deftypevr {Built-in constant} string __defpreproc__
  Expands to the default external preprocessor command line, if the
preprocessor is used, or to an empty string if it is not, e.g.:

@example
__defpreproc__ @result{} "/usr/bin/m4 -s"
@end example

  @xref{Preprocessor}, for information on preprocessor and its
features.
@end deftypevr

@deftypevr {Built-in constant} string __preproc__
  Expands to the current external preprocessor command line, if the
preprocessor is used, or to an empty string if it is not.  Notice,
that it equals @code{__defpreproc__}, unless the preprocessor was
redefined using @option{--preprocessor} command line option
(@pxref{Preprocessor, --preprocessor}).
@end deftypevr

@deftypevr {Built-in constant} string __version__
Expands to the textual representation of the program version
(e.g.@: @samp{3.0.90})
@end deftypevr

@deftypevr {Built-in constant} string __defstatedir__
Expands to the default state directory (@pxref{statedir}).
@end deftypevr

@deftypevr {Built-in constant} string __statedir__
Expands to the current value of the program state directory
(@pxref{statedir}).  Notice, that it is the same as
@code{__defstatedir__} unless the state directory was redefined at run
time.
@end deftypevr

  Built-in constants can be used as variables, this allows to expand them
within strings or here-documents.  The following example illustrates
the common practice used for debugging configuration scripts:

@example
@group
func foo(number x)
do
  echo "%__file__:%__line__: foo called with arg %x"
  @dots{}
done
@end group
@end example

  If the function @code{foo} were called in line 28 of the
script file @code{/etc/mailfromd.mfl}, like this:
@code{foo(10)}, you will see the following string in your logs:

@example
/etc/mailfromd.mfl:28: foo called with arg 10
@end example

@node Variables
@section Variables
@cindex variables, defined
  Variables represent regions of memory used to hold variable data.
These memory regions are identified by @dfn{variable names}.  A
variable name must begin with a letter or underscore and must consist
of letters, digits and underscores.

@cindex scope of visibility, variables
@cindex variable lexical scope
@anchor{variable scope}
  Each variable is associated with its @dfn{scope of visibility},
which defines the part of source code where it can be used
(@pxref{scope of visibility}).  Depending on the scope, we discern
three main classes of variables: public, static and automatic (or local).

  @dfn{Public variables} have indefinite lexical scope, so they may
be referred to anywhere in the program.  @dfn{Static} are variables
visible only within their module (@pxref{Modules}).  @dfn{Automatic}
or @dfn{local variables} are visible only within the given function or
handler.

  Public and static variables are sometimes collectively called
@dfn{global}.

  These variable classes occupy separate @dfn{namespaces}, so that an
automatic variable can have the same name as an existing public or
static one.  In this case this variable is said to @dfn{shadow} its global
counterpart.  All references to such a name will refer to the automatic
variable until the end of its scope is reached, where the global one
becomes visible again.

  Likewise, a static variable may have the same name as a static
variable defined in another module.  However, it may not have the
same name as a public variable.

@cindex variables, declaring
@cindex variable declarations
  A variable is @dfn{declared} using the following syntax:

@example
[@var{qualifiers}] @var{type} @var{name}
@end example

@noindent
where @var{name} is the variable name, @var{type} is the type of
the data it is supposed to hold.  It is @samp{string} for string
variables and @samp{number} for numeric ones.

  For example, this is a declaration of a string variable @samp{var}:

@example
string var
@end example

  If a variable declaration occurs within a function
(@pxref{Functions,User-defined}) or handler (@pxref{Handlers}), it
declares an automatic variable, local to this function or handler.
Otherwise, it declares a global variable.

@cindex qualifiers, variable declaration
@kwindex public
@kwindex static
Optional @var{qualifiers} are allowed only in global declarations, i.e.
in the variable declarations that appear outside of functions.  They
specify the scope of the variable.  The @code{public} qualifier
declares the variable as public and the @code{static} qualifier
declares it as static.  The default scope is @samp{public},
unless specified otherwise in the module declaration (@pxref{module
structure}).

@kwindex precious
@cindex variables, precious
Additionally, @var{qualifiers} may contain the word @code{precious},
which instructs the compiler to mark this variable as @dfn{precious}.
(@pxref{rset,, precious variables}).  The value of the precious variable
is not affected by the @acronym{SMTP} @samp{RSET} command.  If both
scope qualifier and @code{precious} are used, they may appear in any
order, e.g.:

@example
static precious string rcpt_list
@end example

@noindent
or

@example
precious static string rcpt_list
@end example

  Declaration can be followed by any valid @acronym{MFL}
expression, which supplies the initial value or @dfn{initializer} for the
variable, for example:

@example
string var "test"
@end example

  A variable declared without initializer is implicitly
initialized to a null value, no matter what its scope: a numeric
variable assumes initial value 0, a string variables is initialized to
an empty string.

@cindex variable, assigning a value
@cindex variable assignment
@kwindex set
  A variable is assigned a value using the @code{set} statement:

@example
set @var{name} @var{expr}
@end example

@noindent
where @var{name} is the variable name and @var{expr} is a
@command{mailfromd} expression (@pxref{Expressions}).  The effect of
this statement is that the @var{expr} is evaluated and the value it
yields is assigned to the variable @var{name}.

  If the @code{set} statement is located outside a function or handler
definition, the @var{expr} must be a constant expression, i.e. the
compiler should be able to evaluate it immediately.  @FIXME-xref{optimizer}.

  It is not an error to assign a value to a variable that is not
declared.  In this case the assignment first declares a global or automatic
variable having the type of @var{expr} and then assigns a value to it.
Automatic variable is created if the assignment occurs within a
function or handler, global variable is declared if it occurs at
topmost lexical level.  This is called @dfn{implicit variable
declaration}.

@cindex variables, referencing
  In the @acronym{MFL} program, variables are referenced by their
name.  When appearing inside a double-quoted string, variables are
referenced using the notation @samp{%@var{name}}.  Any variable being
referenced must have been declared earlier (either explicitly or
implicitly).

@menu
* Predefined variables::
@end menu

@node Predefined variables
@subsection Predefined Variables

@cindex predefined variables
@cindex variables, predefined
  Several variables are predefined.  In @command{mailfromd} version
@value{VERSION} these are:

@deftypevr {Predefined Variable} number milter_state
Identifies the current @dfn{milter state} (@pxref{milter state}).
@flindex milter.mfl
The module @file{milter.mfl} defines the following symbolic names:

@table @code
@kwindex milter_state_none
@item milter_state_none

@kwindex milter_state_startup
@item milter_state_startup

@kwindex milter_state_shutdown
@item milter_state_shutdown

@kwindex milter_state_begin
@item milter_state_begin

@kwindex milter_state_end
@item milter_state_end

@kwindex milter_state_connect
@item milter_state_connect

@kwindex milter_state_helo
@item milter_state_helo

@kwindex milter_state_envfrom
@item milter_state_envfrom

@kwindex milter_state_envrcpt
@item milter_state_envrcpt

@kwindex milter_state_data
@item milter_state_data

@kwindex milter_state_header
@item milter_state_header

@kwindex milter_state_eoh
@item milter_state_eoh

@kwindex milter_state_body
@item milter_state_body

@kwindex milter_state_eom
@item milter_state_eom

@kwindex milter_action
@item milter_action
@end table

Use the @code{milter_state_name} function to obtain the corresponding
textual string (@pxref{Informative Functions, milter_state_name}).
@end deftypevr

@anchor{milter_server_id}
@deftypevr {Predefined Variable} string milter_server_id
Identifier of the milter server which executes the code.  This is the
string passed to the @code{id} statement in the @code{server} section
of the configuration file (@pxref{conf-server}),
@end deftypevr

@deftypevr {Predefined Variable} string milter_server_address
Address of the socket the milter server is listening to.  This is
defined by the @code{listen} statement in the @code{server} section
of the configuration file (@pxref{conf-server}),
@end deftypevr

@deftypevr {Predefined Variable} number milter_server_family
Address family of the milter server address, as defined by the
@code{listen} statement in the @code{server} section of the
configuration file (@pxref{conf-server}).
See the @code{FAMILY_} constants in @ref{socket-families}.
@end deftypevr

@deftypevr {Predefined Variable} string milter_client_address
Address of the milter client which initiated the connection.
@end deftypevr

@deftypevr {Predefined Variable} number milter_client_family
Address family of @code{milter_client_address}.
See the @code{FAMILY_} constants in @ref{socket-families}.
@end deftypevr

@deftypevr {Predefined Variable} number cache_used
This variable is set by @code{stdpoll} and @code{strictpoll} built-ins
(and, consequently, by the @code{on poll} statement).  Its value is
@samp{1} if the function used the cached data instead of directly
polling the host, and @samp{0} if the polling took place.
@xref{SMTP Callout functions}.

@anchor{cache_used example}
@cindex @code{cache_used} variable, usage example
@cindex reject messages, marking cached rejects
You can use this variable to make your reject message more informative
for the remote party.  The common paradigm is to define a function,
returning empty string if the result was obtained from polling, or
some notice if cached data were used, and to use the function in the
@code{reject} text, for example:

@example
@group
func cachestr() returns string
do
  if cache_used
    return "[CACHED] "
  else
    return ""
  fi
done
@end group
@end example

@noindent
Then, in @code{prog envfrom} one can use:

@example
@group
on poll $f
do
when not_found or failure:
  reject 550 5.1.0 cachestr() . "Sender validity not confirmed"
done
@end group
@end example

@end deftypevr

@deftypevr {Predefined Variable} number clamav_stream_max_length
@cindex @code{StreamMaxLength}, @file{clamd.conf} parameter.
@cindex clamd.conf
  Maximum size of a message chunk, used when sending message to
@command{ClamAV} daemon.  Must be less than or equal to the value of
@code{StreamMaxLength} parameter in @file{clamd.conf}.

  Default value is 4096.

  @xref{ClamAV}.
@end deftypevr
  
@deftypevr {Predefined Variable} string clamav_virus_name
  Name of virus identified by @command{ClamAV}.  Set by @code{clamav}
function (@pxref{ClamAV}).
@end deftypevr

@deftypevr {Predefined Variable} number greylist_seconds_left
  Number of seconds left to the end of greylisting period.  Set by
@code{greylist} and @code{is_greylisted} functions (@pxref{Special test
functions}).
@end deftypevr

@anchor{ehlo_domain}
@deftypevr {Predefined Variable} string ehlo_domain
  Name of the domain used by polling functions in @acronym{SMTP}
@code{EHLO} or @code{HELO} command.  Default value is the fully
qualified domain name of the host where @command{mailfromd} is run.
@xref{Polling}.
@end deftypevr

@deftypevr {Predefined Variable} string last_poll_greeting
Callout functions (@pxref{SMTP Callout functions}) set this variable before
returning.  It contains the initial SMTP reply from the last polled
host.
@end deftypevr

@deftypevr {Predefined Variable} string last_poll_helo
Callout functions (@pxref{SMTP Callout functions}) set this variable before
returning.  It contains the reply to the @code{HELO} (@code{EHLO})
command, received from the last polled host.
@end deftypevr

@deftypevr {Predefined Variable} string last_poll_host
Callout functions (@pxref{SMTP Callout functions}) set this variable before
returning.  It contains the host name or @acronym{IP} address of the
last polled host.
@end deftypevr

@deftypevr {Predefined Variable} string last_poll_recv
Callout functions (@pxref{SMTP Callout functions}) set this variable before
returning.  It contains the last @acronym{SMTP} reply received from
the remote host.  In case of multi-line replies, only the first line is
stored.  If nothing was received the variable contains the string
@samp{nothing}.
@end deftypevr

@deftypevr {Predefined Variable} string last_poll_sent
Callout functions (@pxref{SMTP Callout functions}) set this variable before
returning.  It contains the last @acronym{SMTP} command sent to the
polled host.  If nothing was sent, @code{last_poll_sent} contains the string
@samp{nothing}.
@end deftypevr

@anchor{mailfrom_address}
@deftypevr {Predefined Variable} string mailfrom_address
  Email address used by polling functions in @acronym{SMTP} @code{MAIL
FROM} command (@pxref{Polling}.).  Default is @samp{<>}.  Here is an
example of how to change it:

@example
set mailfrom_address "postmaster@@my.domain.com"
@end example

@cindex multiple sender addresses
  You can set this value to a comma-separated list of email addresses,
in which case the probing will try each address until either the
remote party accepts it or the list of addresses is exhausted,
whichever happens first.

  It is not necessary to enclose emails in angle brackets, as they
will be added automatically where appropriate.  The only exception is
null return address, when used in a list of addresses.  In this case,
it should always be written as @samp{<>}.  For example:

@example
set mailfrom_address "postmaster@@my.domain.com, <>"
@end example

@end deftypevr

@deftypevr {Predefined Variable} number sa_code
Spam score for the message, set by @code{sa} function (@pxref{sa}).
@end deftypevr

@deftypevr {Predefined Variable} number rcpt_count
  The variable @code{rcpt_count} keeps the number of recipients given so
far by @code{RCPT TO} commands.  It is defined only in @samp{envrcpt}
handlers.
@end deftypevr

@deftypevr {Predefined Variable} number sa_threshold
Spam threshold, set by @code{sa} function (@pxref{sa}).
@end deftypevr

@deftypevr {Predefined Variable} string sa_keywords
Spam keywords for the message, set by @code{sa} function (@pxref{sa}).
@end deftypevr

@deftypevr {Predefined Variable} number safedb_verbose
This variable controls the verbosity of the exception-safe database
functions.  @xref{safedb_verbose}.
@end deftypevr

@node Back references
@section Back references

@cindex back references, in program text
  A @dfn{back reference} is a sequence @samp{\@var{d}}, where @var{d}
is a decimal number.  It refers to the @var{d}th parenthesized
subexpression in the last @command{matches} statement@footnote{The
subexpressions are numbered by the positions of their opening
parentheses, left to right.}.  Any back reference occurring within a
double-quoted string is replaced with the value of the corresponding
subexpression.  For example:

@example
@group
if $f matches '.*@@\(.*\)\.gnu\.org\.ua'
  set host \1
fi
@end group
@end example

  If the value of @code{f} macro is @samp{smith@@unza.gnu.org.ua}, the
above code will assign the string @samp{unza} to the variable
@code{host}.

  Notice, that each occurrence of @code{matches} will reset the table
of back references, so try to use them as early as possible.  The
following example illustrates a common error, when the back
reference is used after the reference table has been reused by another
matching:

@example
@group
# @r{Wrong!}
if $f matches '.*@@\(.*\)\.gnu\.org\.ua'
  if $f matches 'some.*'
    set host \1
  fi
fi
@end group
@end example

  This will produce the following run time error:

@example
mailfromd: RUNTIME ERROR near file.mfl:3: Invalid back-reference number
@end example

@noindent
because the inner match (@samp{some.*}) does not have any parenthesized
subexpressions.

@xref{Special comparisons}, for more information about @code{matches}
operator.

@node Handlers
@section Handlers

@cindex milter stage handler, defined
@cindex stage handler, defined
@cindex handler, defined
@kwindex prog
  @dfn{Milter stage handler} (or @dfn{handler}, for short) is a
subroutine responsible for processing a particular milter state.
There are eight handlers available.  Their order of invocation and
arguments are described in @ref{milter-control-flow}.

  A handler is defined using the following construct:

@example
@group
prog @var{handler-name}
do
  @var{handler-body}
done
@end group
@end example

@noindent
where @var{handler-name} is the name of the handler (@pxref{handler
names}), @var{handler-body} is the list of filter statements composing
the handler body.  Some handlers take arguments, which can be accessed
within the @var{handler-body} using the notation @var{$@var{n}},
where @var{n} is the ordinal number of the argument.  Here we describe
the available handlers and their arguments:

@anchor{connect handler}
@deffn {Handler} connect (string $1, number $2, number $3, string $4)
@table @b
@item Invocation:
This handler is called once at the beginning of each @acronym{SMTP} connection.

@item Arguments:
@enumerate 1
@item @code{string};
@anchor{hostname in connect handler}
The host name of the message sender, as reported by @acronym{MTA}.  Usually it
is determined by a reverse lookup on the host address.  If the reverse
lookup fails, @samp{$1} will contain the message sender's @acronym{IP} address
enclosed in square brackets (e.g.@: @samp{[127.0.0.1]}).

@item @code{number};
Socket address family.  You need to require the @samp{status} module
to get symbolic definitions for the address families.  Supported
families are:

@cindex FAMILY_STDIO
@cindex FAMILY_UNIX
@cindex FAMILY_INET
@cindex FAMILY_INET6
@float Table, socket-families
@caption{Supported socket families}
@multitable @columnfractions 0.25 .10 0.45
@headitem Constant @tab Value @tab Meaning
@item FAMILY_STDIO @tab  0    @tab Standard input/output (the @acronym{MTA} is
run with @option{-bs} option)
@item FAMILY_UNIX  @tab  1    @tab @acronym{UNIX} socket
@item FAMILY_INET  @tab  2    @tab @acronym{IP}v4 protocol
@item FAMILY_INET6 @tab  3    @tab @acronym{IP}v6 protocol
@end multitable
@end float

@item @code{number};
Port number if @samp{$2} is @samp{FAMILY_INET}.

@item @code{string};
Remote @acronym{IP} address if @samp{$2} is @samp{FAMILY_INET} or full file name
of the socket if @samp{$2} is @samp{FAMILY_UNIX}.  If @samp{$2} is
@samp{FAMILY_STDIO}, @samp{$4} is an empty string.
@end enumerate
@end table

@cindex actions, using in @code{connect} handler
  The actions (@pxref{Actions}) appearing in this handler
are handled by Sendmail in a special way.  First of all, any textual
message is ignored.  Secondly, the only action that immediately closes
the connection is @code{tempfail 421}.  Any other reply codes result in
Sendmail switching to @dfn{nullserver} mode, where it accepts any
commands, but answers with a failure to any of them, except for the
following: @code{QUIT}, @code{HELO}, @code{NOOP}, which are processed
as usual.

  The following table summarizes the Sendmail behavior depending on
the action used:

@table @code
@item tempfail 421 @var{excode} @var{message}
  The caller is returned the following error message:

@example
421 4.7.0 @var{hostname} closing connection
@end example

@noindent
Both @var{excode} and @var{message} are ignored.

@item tempfail 4@var{xx} @var{excode} @var{message}
  (where @var{xx} represents any digits, except @samp{21})
  Both @var{excode} and @var{message} are ignored.  Sendmail switches
to nullserver mode.  Any subsequent command, excepting the ones listed above,
is answered with

@example
454 4.3.0 Please try again later
@end example

@item reject 5@var{xx} @var{excode} @var{message}
 (where @var{xx} represents any digits).  All arguments are
ignored.  Sendmail switches to nullserver mode.  Any subsequent
command, excepting ones listed above, is answered with

@example
550 5.0.0 Command rejected
@end example
@end table

  Regarding reply codes, this behavior complies with @acronym{RFC}
2821 (section 3.9), which states:

@quotation
  An @acronym{SMTP} server @emph{must not} intentionally close the connection except:@*
  [@dots{}]@*
  -  After detecting the need to shut down the @acronym{SMTP} service and
     returning a 421 response code.  This response code can be issued
     after the server receives any command or, if necessary,
     asynchronously from command receipt (on the assumption that the
     client will receive it after the next command is issued).
@end quotation

  However, the @acronym{RFC} says nothing about textual messages and
extended error codes, therefore Sendmail's ignoring of these is,
in my opinion, absurd.  My practice shows that it is often reasonable,
and even necessary, to return a meaningful textual message if the
initial connection is declined.  The opinion of @command{mailfromd}
users seems to support this view.  Bearing this in mind,
@command{mailfromd} is shipped with a patch for Sendmail,
which makes it honor both extended return code and textual message given
with the action.  Two versions are provided:
@file{etc/@/sendmail-8.13.7.connect.diff}, for
Sendmail versions 8.13.x, and
@file{etc/@/sendmail-8.14.3.connect.diff}, for Sendmail versions 8.14.3.

@end deffn

@deffn {Handler} helo (string $1)
@table @b
@item Invocation:
This handler is called whenever the @acronym{SMTP} client sends @code{HELO} or
@code{EHLO} command.  Depending on the actual @acronym{MTA} configuration, it
can be called several times or even not at all.

@item Arguments:
@enumerate 1
@item @code{string}; Argument to @code{HELO} (@code{EHLO}) commands.
@end enumerate

@item Notes:
According to @acronym{RFC} 28221, @code{$1} must be domain name of the
sending host, or, in case this is not available, its @acronym{IP} address
enclosed in square brackets.  Be careful when taking decisions based
on this value, because in practice many hosts send arbitrary strings.
We recommend to use @code{heloarg_test} function
(@pxref{heloarg_test}) if you wish to analyze this value.
@end table
@end deffn

@deffn {Handler} envfrom (string $1, string $2)
@table @b
@item Invocation:
Called when the @acronym{SMTP} client sends @code{MAIL FROM} command, i.e. once
at the beginning of each message.

@item Arguments:
@enumerate 1
@item @code{string}; First argument to the @code{MAIL FROM} command,
i.e. the email address of the sender.
@item @code{string}; Rest of arguments to @code{MAIL FROM} separated
by space character.  This argument can be @samp{""}.
@end enumerate

@item Notes
@enumerate 1
@item @code{$1} is not the same as @code{$f} Sendmail variable, because
the latter contains the sender email after address rewriting and
normalization, while @code{$1} contains exactly the value given by
sending party.

@item When the array type is implemented, @code{$2} will contain
an array of arguments.
@end enumerate
@end table
@end deffn

@deffn {Handler} envrcpt (string $1, string $2)
@table @b
@item Invocation:
Called once for each @code{RCPT TO} command, i.e. once for each
recipient, immediately after @code{envfrom}.
@item Arguments:
@enumerate 1
@item @code{string}; First argument to the @code{RCPT TO} command,
i.e. the email address of the recipient.
@item @code{string}; Rest of arguments to @code{RCPT TO} separated
by space character.  This argument can be @samp{""}.
@end enumerate

@item Notes:
When the array type is implemented, @code{$2} will contain
an array of arguments.
@end table
@end deffn

@deffn {Handler} data ()
@table @b
@item Invocation:
Called after the @acronym{MTA} receives @acronym{SMTP} @samp{DATA}
command.  Notice that this handler is not supported by Sendmail
versions prior to 8.14.0 and Postfix versions prior to 2.5.
@item Arguments:
None
@end table
@end deffn

@deffn {Handler} header (string $1, string $2)
@table @b
@item Invocation:
Called once for each header line received after @acronym{SMTP} @code{DATA} command.
@item Arguments:
@enumerate 1
@item @code{string}; Header field name.
@item @code{string}; Header field value.  The content of the header may
include folded white space, i.e., multiple lines with following white
space where lines are separated by @sc{lf} (@acronym{ASCII} 10).  The
trailing line terminator (@sc{cr/lf}) is removed.
@end enumerate
@end table
@end deffn

@deffn {Handler} eoh
@table @b
@item Invocation:
This handler is called once per message, after all headers have been
sent and processed.
@item Arguments:
None.
@end table
@end deffn

@anchor{body handler}
@deffn {Handler} body (pointer $1, number $2)
@table @b
@item Invocation:
This header is called zero or more times, for each piece of the
message body obtained from the remote host.
@item Arguments:
@enumerate 1
@item @code{pointer}; Piece of body text.  See @samp{Notes} below.
@item @code{number}; Length of data pointed to by @code{$1}, in bytes.
@end enumerate
@item Notes:
The first argument points to the body chunk.  Its size may be quite
considerable and passing it as a string may be costly both in terms of
memory and execution time.  For this reason it is not passed as a
string, but rather as a @dfn{generic pointer}, i.e. an object having
the same size as @code{number}, which can be used to retrieve the
actual contents of the body chunk if the need arises.

A special function @code{body_string} is provided to convert this
object to a regular @acronym{MFL} string (@pxref{Mail body
functions}).  Using it you can collect the entire body text into a
single global variable, as illustrated by the following example:

@example
string text

prog body
do
  set text text . body_string($1,$2)
done
@end example
@end table
@end deffn

The text collected this way can then be used in the @code{eom} handler
(see below) to parse and analyze it.

If you wish to analyze both the headers and mail body, the following
code fragment will do that for you:

@example
string text

# @r{Collect all headers.}
prog header
do
  set text text . $1 . ": " . $2 . "\n"
done

# @r{Append terminating newline to the headers.}
prog eoh
do
  set text "%text\n"
done

# @r{Collect message body.}
prog body
do
  set text text . body_string($1, $2)
done
@end example

@anchor{eom handler}
@deffn {Handler} eom
@table @b
@item Invocation:
This handler is called once per message, when the terminating dot
after @code{DATA} command has been received.
@item Arguments:
None
@item Notes:
This handler is useful for calling @dfn{message capturing} functions,
such as @code{sa} or @code{clamav}.  For more information about these,
refer to @ref{Interfaces to Third-Party Programs}.
@end table
@end deffn

For your reference, the following table shows each handler with its arguments:

@cindex milter stage handler arguments
@cindex stage handler arguments
@cindex handler arguments
@float Table, handler-arguments
@caption{State Handler Arguments}
@multitable @columnfractions 0.20 0.20 0.20 0.20 0.20
@headitem Handler @tab $1 @tab $2 @tab $3 @tab $4
@item connect @tab Hostname @tab Socket Family @tab Port @tab Remote address
@item helo @tab @code{HELO} domain @tab @acronym{N/A} @tab @acronym{N/A} @tab @acronym{N/A}
@item envfrom @tab Sender email address @tab Rest of arguments @tab @acronym{N/A} @tab @acronym{N/A}
@item envrcpt @tab Recipient email address @tab Rest of arguments @tab @acronym{N/A} @tab @acronym{N/A}
@item header @tab Header name @tab Header value @tab @acronym{N/A} @tab @acronym{N/A}
@item eoh @tab @acronym{N/A} @tab @acronym{N/A} @tab @acronym{N/A} @tab @acronym{N/A}
@item body @tab Body segment (pointer) @tab Length of the segment
(numeric) @tab @acronym{N/A} @tab @acronym{N/A}
@item eom @tab @acronym{N/A} @tab @acronym{N/A} @tab @acronym{N/A} @tab @acronym{N/A}
@end multitable
@end float

@node Multiple handler definitions
@subsection Multiple Handler Definitions

Any handler may be declared multiple times.  When compiling the filter
program, @command{mailfromd} combines the code from all @code{prog}
declarations having the same handler name into one code block and
compiles it.  The resulting code is guaranteed to be executed in the
order in which it appears in the source files.

@node Special handlers
@section Initialization and Cleanup Handlers

  Apart from the milter handlers described in the previous section,
@acronym{MFL} provides several @dfn{special handlers}, that serve
as @dfn{hooks}, allowing the programmer to insert code in certain
important points of the control flow.

  Syntactically, special handlers are similar to milter state handlers,
i.e. they are defined as:

@example
@group
prog @var{handler}
do
  ...
done
@end group
@end example

@noindent
(@var{handler} being the handler name).

  Special handlers can be subdivided into three groups.
  
  The first group are @code{begin} and @code{end} handlers.  These
are run at the beginning and before the end of each SMTP session and
are used to provide a session-specific initialization and cleanup
routines.

  The second group are @code{startup} and @code{shutdown} handlers,
which provide global initialization and cleanup routines.  These
handlers are invoked exactly once: @code{startup} when
@command{mailfromd} has started up, but hasn't yet begun to serve
milter requests, and @code{shutdown} when @command{mailfromd} is about
to terminate.

  Finally, the @code{action} handler is run before executing each
reply action (@pxref{reply actions}).

@menu
* begin/end::        Session @samp{begin} and @samp{end} special handlers.
* startup/shutdown:: Global startup and shutdown handlers.
* action hook::      Action hook handler.
@end menu

@node begin/end
@subsection The @samp{begin} and @samp{end} special handlers
@cindex begin, special handler
@cindex end, special handler
@cindex session startup handler
@cindex handler, session startup
@cindex handler, session initialization
@cindex session cleanup handler
@cindex handler, session cleanup
  These two special handlers are executed once for each session,
marking its beginning and end.  Neither of them takes any arguments:

@deffn {Handler} begin
@example
# @r{Begin handler}
prog begin
do
  @dots{}
done
@end example

The @code{begin} handler is run once for each @acronym{SMTP} session,
after the connection has been established but before the first milter
handler has been called.
@end deffn

@deffn {Handler} end
@example
# @r{End handler}
prog end
do
  @dots{}
done
@end example

The @code{end} handler is run once for each @acronym{SMTP} session,
after all other handlers have finished their work and
@command{mailfromd} has already returned the resulting status to the
MTA and closed connection.
@end deffn

@anchor{multiple special handlers}
@FIXME{}
  Multiple @samp{begin} and @samp{end} handlers are a useful feature
for writing modules (@pxref{Modules}), because each module can thus
have its own initialization and cleanup blocks.  Notice, however, that
in this case the order in which subsequent @samp{begin} and @samp{end}
blocks are executed is not defined.  It is only warranted that all
@samp{begin} blocks are executed at startup and all @samp{end} blocks
are executed at shutdown.  It is also warranted that all @samp{begin}
and @samp{end} blocks defined within a compilation unit (i.e. a single
abstract source file, with all @code{#include} and
@code{#include_once} statements expanded in place) are executed in
order of their appearance in the unit.

@anchor{begin/end restrictions}
@cindex @samp{begin}, handler restrictions
@cindex @samp{end}, handler restrictions
  Due to their special nature, the startup and cleanup blocks impose
certain restrictions on the statements that can be used within them:

@enumerate 1
@cindex @code{return} in @samp{begin}
@cindex @samp{begin} and @code{return}
@cindex @code{return} in @samp{end}
@cindex @samp{end} and @code{return}
@item @code{return} cannot be used in @samp{begin} and @samp{end}
handlers.

@cindex @code{accept} in @samp{begin}
@cindex @samp{begin} and @code{accept}
@cindex @code{accept} in @samp{end}
@cindex @samp{end} and @code{accept}
@cindex @code{continue} in @samp{begin}
@cindex @samp{begin} and @code{continue}
@cindex @code{continue} in @samp{end}
@cindex @samp{end} and @code{continue}
@cindex @code{discard} in @samp{begin}
@cindex @samp{begin} and @code{discard}
@cindex @code{discard} in @samp{end}
@cindex @samp{end} and @code{discard}
@cindex @code{reject} in @samp{begin}
@cindex @samp{begin} and @code{reject}
@cindex @code{reject} in @samp{end}
@cindex @samp{end} and @code{reject}
@cindex @code{tempfail} in @samp{begin}
@cindex @samp{begin} and @code{tempfail}
@cindex @code{tempfail} in @samp{end}
@cindex @samp{end} and @code{tempfail}
@item The following Sendmail actions cannot be used in them:
@code{accept}, @code{continue}, @code{discard}, @code{reject},
@code{tempfail}.  They can, however, be used in @code{catch}
statements, declared in @samp{begin} blocks (see example below).

@cindex @code{add} in @samp{begin}
@cindex @samp{begin} and @code{add}
@cindex @code{add} in @samp{end}
@cindex @samp{end} and @code{add}
@cindex @code{replace} in @samp{begin}
@cindex @samp{begin} and @code{replace}
@cindex @code{replace} in @samp{end}
@cindex @samp{end} and @code{replace}
@cindex @code{delete} in @samp{begin}
@cindex @samp{begin} and @code{delete}
@cindex @code{delete} in @samp{end}
@cindex @samp{end} and @code{delete}

@item Header manipulation actions (@pxref{header manipulation}) cannot be
used in @samp{end} handler.

@end enumerate

  The @samp{begin} handlers are the usual place to put global
initialization code to.  For example, if you do not want to use
@acronym{DNS} caching, you can do it this way:

@example
@group
prog begin
do
  db_set_active("dns", 0)
done
@end group
@end example

  Additionally, you can set up global exception handling routines
there.  For example, the following @samp{begin} statement installs a
handler for all exceptions not handled otherwise that logs the
exception along with the stack trace and continues processing the
message:

@example
@group
prog begin
do
  catch *
  do
    echo "Caught exception $1: $2"
    stack_trace()
    continue
  done
done
@end group
@end example

@node startup/shutdown
@subsection Global startup and shutdown handlers.
@cindex startup, special handler
@cindex shutdown, special handler
@cindex global startup handler
@cindex handler, global startup
@cindex global cleanup handler
@cindex handler, global cleanup

  Yet another pair of special handlers, @code{startup} and
@code{shutdown}, can be used for global initialization and cleanup.

@deffn {Handler} startup
The @code{startup} handler is called exactly once, as a part of
@code{mailfromd} startup session.

This handler is normally used in @dfn{mfmod} interface modules to
load the shared library part (@pxref{mfmod}).
@end deffn

@deffn {Handler} shutdown
This handler is called during the normal program shutdown sequence,
before exiting.
@end deffn

Both handlers bear certain similarity to @code{begin} and @code{end}:
they take no arguments, and their use is subject to the same
restrictions (@pxref{begin/end restrictions}).  Additionally,
the following features cannot be used in global handlers:

@enumerate 1
@item Sendmail macros.
@item Message modification functions.
@end enumerate

@node action hook
@subsection Action Hook
@cindex action hook
  Action hook handler is run implicitly before executing each reply
action, such as @code{accept}, @code{reject}, etc.  @xref{reply
actions}, for a discussion of reply action statements.

  Upon invocation, the handler is passed six arguments:

@enumerate 1
@item Identifier of the action which is about to be performed.
Each action is assigned a numeric identifier.  The @code{status}
module defines the following symbolic names for action identifiers:

@anchor{action identifiers}
@table @code
@kwindex ACCEPT_ACTION 
@item ACCEPT_ACTION
@kwindex CONTINUE_ACTION
@item CONTINUE_ACTION
@kwindex DISCARD_ACTION
@item DISCARD_ACTION
@kwindex REJECT_ACTION
@item REJECT_ACTION
@kwindex TEMPFAIL_ACTION
@item TEMPFAIL_ACTION
@end table

To convert these to textual action names, use the
@code{milter_action_name} function (@pxref{Informative Functions,
milter_action_name}).

@item SMTP code of the reply

@item Extended reply code

@item Textual message passed to the action.

@item File name of the source location where the action was called.

@item Line number of source location where the action was called.
@end enumerate

  Last three arguments are meaningful only for @code{reject} and
@code{tempfail} actions.  For the remaining three actions
(@code{accept}, @code{discard}, and @code{continue}), empty strings
are passed.

  Action hook handlers are useful mainly for logging and accounting
purposes.  For example, the code fragment below assumes that the
@command{openmetrics} module is used (@pxref{Top,
mfmod_openmetrics,, mfmod_openmetrics, mfmod_openmetrics reference}).
It increases the corresponding metrics before taking the action.
Additionally, for @code{reject} and @code{tempfail} actions, the
metrics @samp{reject_@var{code}} and @samp{tempfail_@var{code}} are
increased, where @var{code} is the three-digit SMTP status code being
sent to the server.

@example
@group
prog action
do
  openmetrics_incr(milter_action_name($1))
  switch $1
  do
  case REJECT_ACTION:
     openmetrics_incr("reject_" . $2)
  case TEMPFAIL_ACTION:
     openmetrics_incr("tempfail_" . $2)
  done
done  
@end group
@end example

@node Functions
@section Functions

  A @dfn{function} is a named @command{mailfromd} subroutine, which
takes zero or more @dfn{parameters} and optionally returns a certain
value.  Depending on the return value, functions can be
subdivided into @dfn{string functions} and @dfn{number functions}.
A function may have @dfn{mandatory} and @dfn{optional parameters}.
When invoked, the function must be supplied exactly as many
@dfn{actual arguments} as the number of its mandatory parameters.

  Functions are invoked using the following syntax:

@example
  @var{name} (@var{args})
@end example

@noindent
where @var{name} is the function name and @var{args} is a
comma-separated list of expressions.  For example, the following are valid
function calls:

@example
@group
  foo(10)
  interval("1 hour")
  greylist("/var/my.db", 180)
@end group
@end example

  The number of parameters a function takes and their data types
compose the @dfn{function signature}.  When actual arguments are
passed to the function, they are converted to types of the
corresponding formal parameters.

  There are two major groups of functions: @dfn{built-in} functions,
that are implemented in the @command{mailfromd} binary, and
@dfn{user-defined} functions, that are written in @acronym{MFL}.  The
invocation syntax is the same for both groups.

  @command{Mailfromd} is shipped with a rich set of @dfn{library
functions}.  These are described in @ref{Library}.  In addition to
these you can define your own functions.

@anchor{User-defined}
@cindex function definition, syntax of
  Function definitions can appear anywhere between the handler
declarations in a filter program, the only requirement being that the
function definition occur before the place where the function is
invoked.

  The syntax of a function definition is:

@cindex @code{returns} statement, function definition
@cindex @code{func} statement, function definition
@example
@group
[@var{qualifier}] func @var{name} (@var{param-decl})
  [returns @var{data-type}]
do
  @var{function-body}
done
@end group
@end example

@noindent
where @var{name} is the name of the function to define, @var{param-decl} is
a comma-separated list of parameter declarations.  The syntax of the
latter is the same as that of variable declarations (@pxref{Variables,
Variable declarations}), i.e.:

@example
@var{type} @var{name}
@end example

@noindent
declares the parameter @var{name} having the type @var{type}.  The
@var{type} is @code{string} or @code{number}.

@cindex qualifier, function declaration
@kwindex public
@kwindex static
@cindex scope of visibility, functions
@anchor{function scope of visibility}
  Optional @var{qualifier} declares the scope of visibility for that
function (@pxref{scope of visibility}).  It is similar to that of
variables, except that functions cannot be local (i.e. you cannot
declare function within another function).

  The @code{public} qualifier declares a function that may be referred
to from any module, whereas the @code{static} qualifier declares a
function that may be called only from the current module
(@pxref{Modules}).  The default scope is @samp{public},
unless specified otherwise in the module declaration (@pxref{module
structure}).

  For example, the following declares a function @samp{sum}, that takes
two numeric arguments and returns a numeric value:

@example
func sum(number x, number y) returns number
@end example

  Similarly, the following is a declaration of a static function:

@example
static func sum(number x, number y) returns number
@end example

  Parameters are referenced in the @var{function-body} by their name,
the same way as other variables.  Similarly, the value of a parameter can be
altered using @code{set} statement.

@cindex optional arguments to a function
@cindex arguments, optional
  A function can be declared to take a certain number of @dfn{optional
arguments}.  In a function declaration, optional abstract arguments
must be placed after the mandatory ones, and must be separated from
them with a semicolon.  The following example is a definition of
function @code{foo}, which takes two mandatory and two optional
arguments:

@example
func foo(string msg, string email; number x, string pfx)
@end example

@noindent
@kwindex $#
@cindex function arguments, getting the number of
@cindex function arguments, counting
@cindex number of actual arguments
@cindex @@@var{var}, special construct
@cindex argument number in the list of arguments
Mandatory parameters are: @code{msg} and @code{email}.  Optional
parameters are: @code{x} and @code{pfx}.  The actual number of
arguments supplied to the function is returned by a special construct
@code{$#}.  In addition, the special construct @code{@@@var{arg}}
evaluates to the ordinal number of variable @var{arg} in the list of
formal parameters (the first argument has number @samp{0}).  These two
constructs can be used to verify whether an argument is supplied to
the function.

@cindex optional arguments, checking if supplied
  When an actual argument for parameter @code{n} is supplied, the number
of actual arguments (@code{$#}) is greater than the ordinal number
of that parameter in the declaration list (@code{@@@var{n}}).  Thus,
the following construct can be used to check if an optional argument
@var{arg} is actually supplied:

@example
func foo(string msg, string email; number x, string arg)
do
  if $# > @@arg
    @dots{}
  fi
@end example

  The default @command{mailfromd} installation provides a special
macro for this purpose: @pxref{defined}.  Using it, the example above
could be rewritten as:

@example
func foo(string msg, string email; number x, string arg)
do
  if defined(arg)
    @dots{}
  fi
@end example

  Within a function body, optional arguments are referenced
exactly the same way as the mandatory ones.  Attempt to dereference an
optional argument for which no actual parameter was supplied, results
in an undefined value, so be sure to check whether a parameter is
passed before dereferencing it.

@anchor{variadic functions}
@cindex variadic function
@cindex variable number of arguments
  A function can also take variable number of arguments (such
functions are called @dfn{variadic}).  This is
indicated by ellipsis in place of the last abstract parameter name.  The
statement below defines a function @code{foo} taking one mandatory, one
optional and any number of additional arguments:

@example
func foo (string a ; string b, string ...)
@end example

The data type before the ellipsis indicates the type to promote all
actual arguments to.  If it is omitted, @code{string} is assumed, so
the above declaration can also be written as:

@example
func foo (string a ; string b, ...)
@end example

@cindex $(@var{n})
To refer to the actual arguments in the function body, the following
construct is used:

@example
$(@var{expr})
@end example

@noindent
where @var{expr} is any valid @acronym{MFL} expression, evaluating to
a number @var{n}.  This construct refers to the value of @var{n}th
actual parameter from the variable argument list.  Parameters are
numbered from @samp{1}, so the first variable parameter is @code{$(1)},
and the last one is @code{$($# - @var{Nm} - @var{No})}, where @var{Nm}
and @var{No} are numbers of mandatory and optional parameters to the
function.

@cindex $@var{n}
The construct @samp{$(@var{n})} where 1 <= @var{n} <= 9 can also be
written as @samp{$@var{n}}.

For example, the function below prints all its arguments:

@example
func pargs (string text, ...)
do
  echo "text=%text"
  loop for number i 1,
       while i < $# - @@text,
       set i i + 1
  do
    echo "arg %i=" . $(i)
  done
done
@end example

@noindent
Note how the ordinal number operator is used to compute the upper
limit.

  As another example, the function below computes the sum of its
arguments.

@example
@group
func sum(number ...)
do
  number s 0
  loop for number i 1,
       while i <= $#,
       set i i + 1
  do
    set s s + $(i)
  done
  return s
done
@end group
@end example

@cindex passing variable arguments on to another function
@kwindex $@@
  Sometimes it is necessary to pass all variable arguments passed to a
variadic function on to another variadic function.  To do so, use the
@code{$@@} operator.  For example:

@example
@group
func y(string x, number ...)
do
  echo "x is " . sum($@@)
done
@end group
@end example

Suppose @code{y} is called as @code{y("test", 1, 3, 5)}.  Then, it
will call @code{sum} as: @code{sum(1, 3, 5)}.

The @code{$@@} can be used with a numeric argument, which indicates
number of arguments to remove from the resulted argument list.  This
is similar to @code{shift} statement in other languages.  Thus, if
@code{y} in the above example were written as:

@example
@group
func y(string x, ...)
do
  x($@@(2))
done
@end group
@end example

then @code{y("test", "a", "b", "c")}, it will call @code{x} as
follows: @code{x("c")}.

  Notice the following important points.  First, @code{$@@} can be
used only as the last argument in the argument list.  Secondly, it
cannot be used to pass mandatory and optional arguments to a function.
In other words, arguments passed via @code{$@@} must correspond to
ellipsis in the function declaration.  Finally, passing shift count
greater than the actual number of variable arguments results in a
runtime error.

@cindex return statement, defined
  The @var{function-body} is any list of valid @command{mailfromd}
statements.  In addition to the statements discussed below
(@pxref{Statements}) it can also contain the @code{return} statement,
which is used to return a value from the function.  The syntax of the
return statement is

@example
  return @var{value}
@end example

@anchor{sum--example}
  As an example of this, consider the following code snippet that
defines the function @samp{sum} to return a sum of its two arguments:

@example
@group
func sum(number x, number y) returns number
do
        return x + y
done
@end group
@end example

@cindex procedures
@cindex function returning void
@cindex void functions
  The @code{returns} part in the function declaration is optional.  A
declaration lacking it defines a @dfn{procedure}, or @dfn{void
function}, i.e. a function that is not supposed to return any value.
Such functions cannot be used in expressions, instead they are
used as statements (@pxref{Statements}).  The following example
shows a function that emits a customized temporary failure notice:

@example
@group
func stdtf()
do
  tempfail 451 4.3.5 "Try again later"
done
@end group
@end example

@anchor{function alias}
@cindex aliases
@kwindex alias
  A function may have several names.  An alternative name (or
@dfn{alias}) can be assigned to a function by using @code{alias}
keyword, placed after @var{param-decl} part, for example:

@example
func foo()
alias bar
returns string
do
  @dots{}
done
@end example

After this declaration, both @code{foo()} and @code{bar()} will refer
to the same function.

The number of function aliases is unlimited.  The following fragment
declares a function having three names:

@example
func foo()
alias bar
alias baz
returns string
do
  @dots{}
done
@end example

Although this feature is rarely needed, there are sometimes cases when
it may be necessary.

@cindex automatic variables
@cindex variables, local
@cindex local variables
@cindex variables, automatic
  A variable declared within a function becomes a local variable to
this function.  Its lexical scope ends with the terminating
@code{done} statement.

  Parameters, local variables and global variables are using
separate namespaces, so a parameter name can coincide with the name of
a global, in which case a parameter is said to @dfn{shadow} the
global.  All references to its name will refer to the parameter,
until the end of its scope is reached, where the global one
becomes visible again.  Consider the following example:

@example
@group
number x

func foo(string x)
do
  echo "foo: %x"
done

prog envfrom
do
  set x "Global"
  foo("Local")
  echo x
done
@end group
@end example

@noindent
  Running @command{mailfromd --test} with this configuration will
display:

@example
@cartouche
foo: Local
Global
@end cartouche
@end example

@menu
* Some Useful Functions::
@end menu

@node Some Useful Functions
@subsection Some Useful Functions

  To illustrate the concept of user-defined functions, this subsection
shows the definitions of some of the library functions shipped with
@command{mailfromd}@footnote{Notice that these are intended for
educational purposes and do not necessarily coincide with the actual
definitions of these functions in Mailfromd version @value{VERSION}.}.
These functions are contained in modules installed along with the
@command{mailfromd} binary.  To use any of them in your code, require
the appropriate module as described in @ref{import}, e.g. to use the
@code{revip} function, do @code{require 'revip'}.

  Functions and their definitions:

@enumerate 1
@cindex revip, definition of
@item @code{revip}@*

The function @code{revip}, that was used in releases of
@command{mailfromd} up to 9.0 (@pxref{revip}) was implemented as follows:

@example
func revip(string ip) returns string
do
  return inet_ntoa(ntohl(inet_aton(ip)))
done
@end example

  Previously it was implemented using regular expressions.  Below we include
this variant as well, as an illustration for the use of regular expressions:

@example
@group
#pragma regex push +extended
func revip(string ip) returns string
do
  if ip matches '([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)'
    return "\4.\3.\2.\1"
  fi
  return ip
done
#pragma regex pop
@end group
@end example

@cindex strip_domain_part, definition of
@item @code{strip_domain_part}@*

This function returns at most @var{n} last components of the domain
name @var{domain} (@pxref{strip_domain_part}).

@example
@group
#pragma regex push +extended

func strip_domain_part(string domain, number n) returns string
do
  if n > 0 and
    domain matches '.*((\.[^.]+)@{' . $2 . '@})'
    return substring(\1, 1, -1)
  else
    return domain
  fi
done
#pragma regex pop
@end group
@end example

@cindex valid_domain, definition
@item @code{valid_domain}@*

@xref{valid_domain}, for a description of this function.  Its
definition follows:

@example
@group
require dns

func valid_domain(string domain) returns number
do
  return not (resolve(domain) = "0" and not hasmx(domain))
done
@end group
@end example

@cindex match_dnsbl, definition
@item @code{match_dnsbl}@*

The function @code{match_dnsbl} (@pxref{match_dnsbl}) is defined as
follows:

@example
require dns
require match_cidr
#pragma regex push +extended

func match_dnsbl(string address, string zone, string range)
    returns number
do
  string rbl_ip
  if range = 'ANY'
    set rbl_ip '127.0.0.0/8'
  else
    set rbl_ip range
    if not range matches '^([0-9]@{1,3@}\.)@{3@}[0-9]@{1,3@}$'
      return 0
    fi
  fi

  if not (address matches '^([0-9]@{1,3@}\.)@{3@}[0-9]@{1,3@}$'
          and address != range)
    return 0
  fi

  if address matches
        '^([0-9]@{1,3@})\.([0-9]@{1,3@})\.([0-9]@{1,3@})\.([0-9]@{1,3@})$'
    if match_cidr (resolve ("\4.\3.\2.\1", zone), rbl_ip)
      return 1
    else
      return 0
    fi
  fi
  # never reached
done
@end example

@end enumerate

@node Expressions
@section Expressions

@cindex expressions
  Expressions are language constructs, that evaluate to a value, that
can subsequently be echoed, tested in a conditional statement,
assigned to a variable or passed to a function.

@menu
* Constant expressions::      String and Numeric Constants.
* Function calls::            A Function Call is an Expression.
* Concatenation::             String Concatenation.
* Arithmetic operations::     @samp{+}, @samp{-}, etc.
* Bitwise shifts::            @samp{<<} and @samp{>>}.
* Relational expressions::    @samp{=}, @samp{<}, etc.
* Special comparisons::       @code{matches}, @code{mx matches}, etc.
* Boolean expressions::       @code{and}, @code{or}, @code{not}.
* Precedence::                How various operators nest.
* Type casting::
@end menu

@node Constant expressions
@subsection Constant Expressions

  Literals and numbers are @dfn{constant expressions}.  They evaluate
to string and numeric types.

@node Function calls
@subsection Function Calls

  A function call is an expression.  Its type is the return type of
the function.

@node Concatenation
@subsection Concatenation

@cindex concatenation
  Concatenation operator is @samp{.} (a dot).  For example, if
@code{$f} is @samp{smith}, and @code{$client_addr} is
@samp{10.10.1.1}, then:

@example
$f . "-" . $client_addr @result{} "smith-10.10.1.1"
@end example

@cindex literal concatenation
  Any two adjacent literal strings are concatenated, producing a new
string, e.g.

@example
"GNU's" " not " "UNIX" @result{} "GNU's not UNIX"
@end example

@node Arithmetic operations
@subsection Arithmetic Operations

  The filter script language offers the common arithmetic operators:
@samp{+}, @samp{-}, @samp{*} and @samp{/}.  In addition, the @samp{%}
is a @dfn{modulo} operator, i.e. it computes the remainder of division
of its operands.

All of them follow usual precedence rules and work as you would expect
them to.

@node Bitwise shifts
@subsection Bitwise shifts

  The @samp{<<} represents a @dfn{bitwise shift left} operation, which
shifts the binary representation of the operand on its left by the
number of bits given by the operand on its right.

  Similarly, the @samp{>>} represents a @dfn{bitwise shift right}.

@node Relational expressions
@subsection Relational Expressions

  Relational expressions are:
@cindex @code{<} (left angle bracket), @code{<} operator
@cindex left angle bracket (@code{<}), @code{<} operator
@cindex @code{<} (left angle bracket), @code{<=} operator
@cindex left angle bracket (@code{<}), @code{<=} operator
@cindex @code{>} (right angle bracket), @code{>=} operator
@cindex right angle bracket (@code{>}), @code{>=} operator
@cindex @code{>} (right angle bracket), @code{>} operator
@cindex right angle bracket (@code{>}), @code{>} operator
@cindex @code{=} (equals sign), @code{=} operator
@cindex equals sign (@code{=}), @code{=} operator
@cindex @code{!} (exclamation point), @code{!=} operator
@cindex exclamation point (@code{!}), @code{!=} operator
@float Table,table-relational-expr
@caption{Relational Expressions}
@multitable @columnfractions .25 .75
@headitem Expression @tab Result
@item @var{x} @code{<} @var{y} @tab True if @var{x} is less than @var{y}.
@item @var{x} @code{<=} @var{y} @tab True if @var{x} is less than or equal to @var{y}.
@item @var{x} @code{>} @var{y} @tab True if @var{x} is greater than @var{y}.
@item @var{x} @code{>=} @var{y} @tab True if @var{x} is greater than or equal to @var{y}.
@item @var{x} @code{=} @var{y} @tab True if @var{x} is equal to @var{y}.
@item @var{x} @code{!=} @var{y} @tab True if @var{x} is not equal to @var{y}.
@end multitable
@end float

  The relational expressions apply to string as well as to numbers.
When a relational operation applies to strings, case-sensitive
comparison is used, e.g.:

@example
@group
"String" = "string" @result{} False
"String" < "string" @result{} True
@end group
@end example

@node Special comparisons
@subsection Special Comparisons

  In addition to the traditional relational operators, described
above, @command{mailfromd} provides two operators for regular
expression matching:

@kwindex matches
@kwindex fnmatches
@cindex regular expression matching
@cindex globbing patterns
@float Table,table-special-comp-expr
@caption{Regular Expression Matching}
@multitable @columnfractions .25 .75
@headitem Expression @tab Result
@item @var{x} @code{matches} @var{y} @tab True if the string @var{x} matches the
regexp denoted by @var{y}.
@item @var{x} @code{fnmatches} @var{y} @tab True if the string @var{x} matches the
globbing pattern denoted by @var{y}.
@end multitable
@end float

  The type of the regular expression used by @code{matches} operator
is controlled by @code{#pragma regex} (@pxref{pragma regex}).  For example:

@example
@group
$f @result{} "gray@@gnu.org.ua"
$f matches '.*@@gnu\.org\.ua' @result{} @code{true}
$f matches '.*@@GNU\.ORG\.UA' @result{} @code{false}
#pragma regex +icase
$f matches '.*@@GNU\.ORG\.UA' @result{} @code{true}
@end group
@end example

  The @code{fnmatches} operator compares its left-hand operand with a
globbing pattern (see @cite{glob(7)}) given as its right-hand side
operand.  For example:

@example
@group
$f @result{} "gray@@gnu.org.ua"
$f fnmatches "*ua" @result{} @code{true}
$f fnmatches "*org" @result{} @code{false}
$f fnmatches "*org*" @result{} @code{true}
@end group
@end example

@anchor{mx matches}
@cindex mx matches
  Both operators have a special form, for @dfn{@samp{MX} pattern matching}.
The expression:

@example
  @var{x} mx matches @var{y}
@end example

@noindent
is evaluated as follows: first, the expression @var{x} is analyzed and, if
it is an email address, its domain part is selected.  If it is not,
its value is used verbatim.  Then the list of @samp{MX}s for this domain is
looked up.  Each of @samp{MX} names is then compared with the regular
expression @var{y}.  If any of the names matches, the expression
returns true.  Otherwise, its result is false.

@cindex mx fnmatches
  Similarly, the expression:

@example
  @var{x} mx fnmatches @var{y}
@end example

@noindent
returns true only if any of the @samp{MX}s for (domain or email) @var{x}
match the globbing pattern @var{y}.

  Both @code{mx matches} and @code{mx fnmatches} can signal the
following exceptions: @code{e_temp_failure}, @code{e_failure}.

  The value of any parenthesized subexpression occurring within the
right-hand side argument to @code{matches} or @code{mx matches} can be
referenced using the notation @samp{\@var{d}}, where @var{d} is the
ordinal number of the subexpression (subexpressions are numbered from
left to right, starting at 1).  This notation is allowed in the
program text as well as within double-quoted strings and
here-documents, for example:

@example
@group
if $f matches '.*@@\(.*\)\.gnu\.org\.ua'
  set message "Your host name is \1;"
fi
@end group
@end example

  Remember that the grouping symbols are @samp{\(} and @samp{\)} for
basic regular expressions, and @samp{(} and @samp{)} for extended
regular expressions.  Also make sure you properly escape all special
characters (backslashes in particular) in double-quoted strings, or
use single-quoted strings to avoid having to do so
(@pxref{singe-vs-double}, for a comparison of the two forms).

@node Boolean expressions
@subsection Boolean Expressions

@cindex and
@cindex or
@cindex not
  A @dfn{boolean expression} is a combination of relational or
matching expressions using the boolean operators @code{and}, @code{or}
and @code{not}, and, eventually, parentheses to control nesting:

@float table, table boolean-expr
@caption{Boolean Operators}
@multitable @columnfractions .25 .75
@headitem Expression @tab Result
@item @var{x} @code{and} @var{y} @tab True only if both @var{x} and
@var{y} are true.
@item @var{x} @code{or} @var{y} @tab True if any of @var{x} or @var{y}
is true.
@item @code{not} @var{x} @tab True if @var{x} is false.
@end multitable
@end float

  Binary boolean expressions are computed using @dfn{shortcut evaluation}:

@table @code
@item @var{x} and @var{y}
If @code{@var{x} @result{} @code{false}}, the result is @code{false}
and @var{y} is not evaluated.

@item @var{x} or @var{y}
If @code{@var{x} @result{} @code{true}}, the result is @code{true} and
@var{y} is not evaluated.
@end table

@node Precedence
@subsection Operator Precedence

@cindex operator precedence, defined
@cindex precedence, operators
  Operator @dfn{precedence} is an abstract value associated with each
language operator, that determines the order in which operators are
executed when they appear together within a single expression.
Operators with higher precedence are executed first.  For example,
@samp{*} has a higher precedence than @samp{+}, therefore the
expression @code{a + b * c} is evaluated in the following order: first
@code{b} is multiplied by @code{c}, then @code{a} is added to the
product.

@cindex operator associativity
@cindex associativity, operators
  When operators of equal precedence are used together they are
evaluated from left to right (i.e., they are @dfn{left-associative}),
except for comparison operators, which are non-associative (these are
explicitly marked as such in the table below).  This means that you
cannot write:

@example
if 5 <= x <= 10
@end example

@noindent
Instead, you should write:

@example
if 5 <= x and x <= 10
@end example

  The precedence of the @command{mailfromd} operators where selected
so as to match that used in most programming languages.@footnote{The
only exception is @samp{not}, whose precedence in @acronym{MFL} is
much lower than usual (in most programming languages it has the same
precedence as unary @samp{-}).  This allows to write conditional
expressions in more understandable manner.  Consider the following
condition:

@example
if not x < 2 and y = 3
@end example

It is understood as ``if @code{x} is not less than 2 and @code{y} equals 3'',
whereas with the usual precedence for @samp{not} it would have meant
``if negated @code{x} is less than 2 and @code{y} equals 3''.}

The following table lists all operators in order of decreasing precedence:

@table @code
@item (...)
Grouping

@item $ %
@command{Sendmail} macros and @command{mailfromd} variables

@item * /
Multiplication, division

@item + -
Addition, subtraction

@item << >>
Bitwise shift left and right

@item < <= >= >
Relational operators (non-associative)

@item = != matches fnmatches
Equality and special comparison (non-associative)

@item &
Logical (bitwise) @sc{and}

@item ^
Logical (bitwise) @sc{xor}

@item |
Logical (bitwise) @sc{or}

@item not
Boolean negation

@item and
Logical @samp{and}.

@item or
Logical @samp{or}

@item .
String concatenation

@end table

@node Type casting
@subsection Type Casting

@cindex type casts, implicit
@cindex implicit type casts
  When two operands on each side of a binary expression have
different type, @command{mailfromd} evaluator coerces them to a
common type.  This is known as @dfn{implicit type casting}.  The rules
for implicit type casting are:

@enumerate 1
@item
Both arguments to an arithmetical operation are cast to numeric
type.

@item
Both arguments to the concatenation operation are cast to string.

@item
Both arguments to `match' or `fnmatch' function are cast to string.

@item
The argument of the unary negation (arithmetical or boolean) is
cast to numeric.

@item
Otherwise the right-hand side argument is cast to the type of the
left-hand side argument.
@end enumerate

@cindex type casts, explicit
@cindex explicit type casts
@anchor{explicit type casts}
  The construct for explicit type cast is:

@example
@var{type}(@var{expr})
@end example

@noindent
where @var{type} is the name of the type to coerce @var{expr} to.  For
example:

@example
string(2 + 4*8) @result{} "34"
@end example

@anchor{void type cast}
@kwindex void
A special case of type casting is cast to @code{void}.  It is used to
ignore return value of a function call between the braces, e.g.:

@example
void(dlcall(libh, "extlog", "s", text))
@end example

@node Shadowing
@section Variable and Constant Shadowing
@cindex shadowing, defined
@cindex name clashes
  When any two named entities happen to have the same name we say that a
@dfn{name clash} occurs.  The handling of name clashes depends on
types of the entities involved in it.

@subheading function -- any
A name of a constant or variable can coincide with that of a function,
it does not produce any warnings or errors because functions,
variables and constants use different namespaces.  For example, the
following code is correct:

@example
const a 4

func a()
do
  echo a
done
@end example

When executed, it prints @samp{4}.

@subheading function -- function, handler -- function, and function -- handler
Redefinition of a function or using a predefined handler name
(@pxref{Handlers}) as a function name results in a fatal error.  For
example, compiling this code:

@example
func a()
do
  echo "1"
done

func a()
do
  echo "2"
done
@end example

@noindent
causes the following error message:

@example
mailfromd: sample.mfl:9: syntax error, unexpected
FUNCTION_PROC, expecting IDENTIFIER
@end example

@subheading handler -- variable
A variable name can coincide with a handler name.  For example, the
following code is perfectly OK:

@example
string envfrom "M"
prog envfrom
do
        echo envfrom
done
@end example

@subheading handler -- handler
If two handlers with the same name are defined, the definition that
appears further in the source text replaces the previous one.  A
warning message is issued, indicating locations of both definitions,
e.g.:

@example
mailfromd: sample.mfl:116: Warning: Redefinition of handler
`envfrom'
mailfromd: sample.mfl:34: Warning: This is the location of the
previous definition
@end example

@subheading variable -- variable
@cindex variable shadowing
@cindex shadowing, variable
Defining a variable having the same name as an already defined one results
in a warning message being displayed.  The compilation succeeds.  The
second variable @dfn{shadows} the first, that is any subsequent
references to the variable name will refer to the second variable.
For example:

@example
string x "Text"
number x 1

prog envfrom
do
  echo x
done
@end example

Compiling this code results in the following diagnostics:

@example
mailfromd: sample.mfl:4: Redeclaring `x' as different data type
mailfromd: sample.mfl:2: This is the location of the previous
definition
@end example

Executing it prints @samp{1}, i.e. the value of the last definition of
@code{x}.

The scope of the shadowing depends on storage classes of the two
variables.  If both of them have external storage class (i.e. are
global ones), the shadowing remains in effect until the end of input.
In other words, the previous definition of the variable is effectively
forgotten.

If the previous definition is a global, and the shadowing definition
is an automatic variable or a function parameter, the scope of this
shadowing ends with the scope of the second variable, after which the
previous definition (global) becomes visible again.  Consider the
following code:

@example
set x "initial"

func foo(string x) returns string
do
  return x
done

prog envfrom
do
  echo foo("param")
  echo x
done
@end example

Its compilation produces the following warning:

@example
mailfromd: sample.mfl:3: Warning: Parameter `x' is shadowing a global
@end example

When executed, it produces the following output:

@example
param
initial
State envfrom: continue
@end example

@anchor{variable--constant shadowing}
@subheading variable -- constant
@cindex shadowing, variable--constant
If a constant is defined which has the same name as a previously
defined variable (the constant @dfn{shadows} the variable), the
compiler prints the following diagnostic message:

@example
@var{file}:@var{line}: Warning: Constant name `@var{name}' clashes with a variable name
@var{file}:@var{line}: Warning: This is the location of the previous definition
@end example

A similar diagnostics is issued if a variable is defined whose name
coincides with a previously defined constant (the variable shadows
the constant).

In any case, any subsequent notation %@var{name} refers to the last
defined symbol, be it variable or constant.

Notice, that shadowing occurs only when using %@var{name} notation.
Referring to the constant using its name without @samp{%} allows to
avoid shadowing effects.

If a variable shadows a constant, the scope of the shadowing depends
on the storage class of the variable.  For automatic variables and
function parameters, it ends with the final @code{done} closing the
function.  For global variables, it lasts up to the end of input.

For example, consider the following code:

@example
const a 4

func foo(string a)
do
  echo a
done

prog envfrom
do
  foo(10)
  echo a
done
@end example

When run, it produces the following output:

@example
$ @kbd{mailfromd --test sample.mfl}
mailfromd: sample.mfl:3: Warning: Variable name `a' clashes with a
constant name
mailfromd: sample.mfl:1: Warning: This is the location of the previous
definition
10
4
State envfrom: continue
@end example

@subheading constant -- constant
@cindex shadowing, constant--constant
Redefining a constant produces a warning message.  The latter
definition shadows the former.  Shadowing remains in effect until
the end of input.

@node Statements
@section Statements

@cindex statements
  Statements are language constructs, that, unlike expressions, do not
return any value.  Statements execute some actions, such as assigning
a value to a variable, or serve to control the execution flow in the
program.

@menu
* Actions::                     Actions control the handling of the mail.
* Assignments::
* Pass::
* Echo::
@c Return statement::
@c Conditionals
@c Exception handlers
@end menu

@node Actions
@subsection Action Statements

@cindex actions
  An @dfn{action} statement instructs @command{mailfromd} to
perform a certain action over the message being processed.  There are
two kinds of actions: return actions and header manipulation actions.

@subsubheading Reply Actions
@anchor{reply actions}
Reply actions tell @command{Sendmail} to return given response code
to the remote party.  There are five such actions:

@table @code
@item accept
@cindex accept action, defined
@kwindex accept
  Return an @code{accept} reply.  The remote party will continue
transmitting its message.

@item reject @var{code} @var{excode} @var{message-expr}
@itemx reject (@var{code-expr}, @var{excode-expr}, @var{message-expr})
@cindex reject action, defined
@kwindex reject
  Return a @code{reject} reply.  The remote party will have to
cancel transmitting its message.  The three arguments are optional,
their usage is described below.

@item tempfail @var{code} @var{excode} @var{message}
@itemx tempfail (@var{code-expr}, @var{excode-expr}, @var{message-expr})
@cindex tempfail action, defined
@kwindex tempfail
  Return a @samp{temporary failure} reply.  The remote party can retry
to send its message later.  The three arguments are optional,
their usage is described below.

@item discard
@cindex discard action, defined
@kwindex discard
  Instructs @command{Sendmail} to accept the message and silently discard
it without delivering it to any recipient.

@item continue
@cindex continue action, defined
@kwindex continue
  Stops the current handler and instructs @command{Sendmail} to
continue processing of the message.
@end table

@anchor{reject and tempfail syntax}
  Two actions, @code{reject} and @code{tempfail} can take up to three
optional parameters.  There are two forms of supplying these
parameters.

In the first form, called @dfn{literal} or @dfn{traditional} notation,
the arguments are supplied as additional words after the action name,
and are separated by whitespace.   The first argument is a three-digit
@acronym{RFC} 2821 reply code.  It must begin with @samp{5} for
@code{reject} and with @samp{4} for @code{tempfail}.  If two arguments
are supplied, the second argument must be either an @dfn{extended
reply code} (@acronym{RFC} 1893/2034) or a textual string to be
returned along with the @acronym{SMTP} reply.  Finally, if all three
arguments are supplied, then the second one must be an extended reply
code and the third one must give the textual string.  The following
examples illustrate the possible ways of using the @code{reject}
statement:

@example
@group
reject
reject 503
reject 503 5.0.0
reject 503 "Need HELO command"
reject 503 5.0.0 "Need HELO command"
@end group
@end example

Used without arguments, @code{reject} is equivalent to

@example
reject @value{DEFAULT_REJECT_CODE}
@end example

@noindent
and @code{tempfail} to

@example
tempfail @value{DEFAULT_TEMPFAIL_CODE}
@end example

In literal notation, the values of code and extendended code (if
supplied) must be literal strings.  The third argument (textual
message) can be either a literal string or @acronym{MFL} expression
that evaluates to string.

The second form of supplying arguments is called @dfn{functional}
notation, because it resembles the function syntax.  When used in this
form, the action word is followed by a parenthesized group of exactly
three arguments, separated by commas.  Each argument is a
@acronym{MFL} expression.  The meaning and ordering of the arguments is
the same as in literal form.  Any or all of these three arguments may
be absent, in which case the corresponding default value will be
used@footnote{The default value for code is
@value{DEFAULT_REJECT_CODE} for @code{reject} and
@value{DEFAULT_TEMPFAIL_CODE} for @code{tempfail}.  The remaining two
arguments default to empty strings.}.  To
illustrate this, here are the statements from the previous example,
written in functional notation:

@example
@group
reject(,,)
reject(503,,)
reject(503, 5.0.0)
reject(503, , "Need HELO command")
reject(503, 5.0.0, "Need HELO command")
@end group
@end example

  Notice that there is an important difference between the two
notations.  The functional notation allows to compute both reply codes
at run time, e.g.:

@example
  reject(500 + dig2*10 + dig3, "5.%edig2.%edig2")
@end example

@subsubheading Header Actions

@anchor{header manipulation}
@cindex header manipulation actions
@cindex actions, header manipulation
  Header manipulation actions provide basic means to add, delete or modify
the message @acronym{RFC} 2822 headers.

@table @command
@item add @var{name} @var{string}
@cindex add action, defined
@kwindex add
  Add the header @var{name} with the value @var{string}.  E.g.:

@example
add "X-Seen-By" "Mailfromd @value{VERSION}"
@end example

@noindent
(notice argument quoting)

@item replace @var{name} @var{string}
@cindex replace action, defined
@kwindex replace
  The same as @code{add}, but if the header @var{name} already
exists, it will be removed first, for example:

@example
replace "X-Last-Processor" "Mailfromd @value{VERSION}"
@end example

@item delete @var{name}
@cindex delete action, defined
@kwindex delete
  Delete the header named @var{name}:

@example
delete "X-Envelope-Date"
@end example
@end table

  These actions impose some restrictions.  First of all, their first
argument must be a literal string (not a variable or expression).
Secondly, there is no way to select a particular header instance
to delete or replace, which may be necessary to properly handle
multiple headers (e.g.@: @samp{Received}).  For more elaborate ways of
header modifications, see @ref{Header modification functions}.

@node Assignments
@subsection Variable Assignments

@cindex assignment, defined
@cindex variable assignment
@kwindex set
  An @dfn{assignment} is a special statement that assigns a value to
the variable.  It has the following syntax:

@example
set @var{name} @var{value}
@end example

@noindent
where @var{name} is the variable name and @var{value} is the value to
be assigned to it.

  Assignment statements can appear in any part of a filter program.
If an assignment occurs outside of function or handler definition,
the @var{value} must be a literal value (@pxref{Literals}).  If it
occurs within a function or handler definition, @var{value} can be any
valid @command{mailfromd} expression (@pxref{Expressions}).  In this
case, the expression will be evaluated and its value will be assigned
to the variable.  For example:

@example
@group
set delay 150

prog envfrom
do
  set delay delay * 2
  @dots{}
done
@end group
@end example

@node Pass
@subsection The @code{pass} statement
@kwindex pass
  The @code{pass} statement has no effect.  It is used in places
where no statement is needed, but the language syntax requires one:

@example
@group
on poll $f do
when success:
  pass
when not_found or failure:
  reject 550
done
@end group
@end example

@node Echo
@subsection The @code{echo} statement
@kwindex echo
@cindex debugging
  The @command{echo} statement concatenates all its arguments into a single
string and sends it to the @command{syslog} using the priority
@samp{info}.  It is useful for debugging your script, in
conjunction with built-in constants (@pxref{Built-in constants}), for
example:

@example
@group
func foo(number x)
do
  echo "%__file__:%__line__: foo called with arg %x"
  @dots{}
done
@end group
@end example

@node Conditionals
@section Conditional Statements

@cindex conditional statements
@cindex statements, conditional
  @dfn{Conditional expressions}, or conditionals for short, test
some conditions and alter the control flow depending on the
result.  There are two kinds of conditional statements: @dfn{if-else}
branches and @dfn{switch} statements.

@kwindex if
@kwindex elif
@kwindex else
@kwindex fi
  The syntax of an @dfn{if-else} branching construct is:

@example
  if @var{condition} @var{then-body} [else @var{else-body}] fi
@end example

@noindent
Here, @var{condition} is an expression that governs control flow
within the statement.  Both @var{then-body} and @var{else-body} are
lists of @command{mailfromd} statements.  If @var{condition} is
true, @var{then-body} is executed, if it is false, @var{else-body} is
executed.  The @samp{else} part of the statement is optional.  The
condition is considered false if it evaluates to zero, otherwise it is
considered true.  For example:

@example
@group
if $f = ""
  accept
else
  reject
fi
@end group
@end example

@noindent
This will accept the message if the value of the @command{Sendmail}
macro @code{$f} is an empty string, and reject it otherwise.  Both
@var{then-body} and @var{else-body} can be compound statements
including other @code{if} statements.  Nesting level of
conditional statements is not limited.

  To facilitate writing complex conditional statements, the @code{elif}
keyword can be used to introduce alternative conditions, for example:

@example
@group
if $f = ""
  accept
elif $f = "root"
  echo "Mail from root!"
else
  reject
fi
@end group
@end example

@anchor{switch}
@kwindex switch
@kwindex case
@cindex @code{switch} statement
@cindex @code{case}, @code{switch} statement
  Another type of branching instruction is @code{switch} statement:

@example
@group
switch @var{condition}
do
case @var{x1} [or @var{x2} @dots{}]:
  @var{stmt1}
case @var{y1} [or @var{y2} @dots{}]:
  @var{stmt2}
  .
  .
  .
[default:
  @var{stmt}]
done
@end group
@end example

@noindent
Here, @var{x1}, @var{x2}, @var{y1}, @var{y2} are literal expressions;
@var{stmt1}, @var{stmt2} and @var{stmt} are arbitrary
@command{mailfromd} statements (possibly compound); @var{condition} is
the controlling expression.  The vertical dotted row represent another
eventual @samp{case} branches.

  This statement is executed as follows: the @var{condition}
expression is evaluated and if its value equals @var{x1} or @var{x2}
(or any other @var{x} from the first @code{case}), then
@var{stmt1} is executed.  Otherwise, if  @var{condition} evaluates
to @var{y1} or @var{y2} (or any other @var{y} from the second
@code{case}), then @var{stmt2} is executed.  Other @code{case}
branches are tried in turn.  If none of them matches, @var{stmt}
(called the @dfn{default branch}) is executed.

  There can be as many @code{case} branches as you wish.  The
@code{default} branch is optional.  There can be at most one
@code{default} branch.

  An example of @code{switch} statement follows:

@example
@group
switch x
do
case 1 or 3:
  add "X-Branch" "1"
  accept
case 2 or 4 or 6:
  add "X-Branch" "2"
default:
  reject
done
@end group
@end example

  If the value of @command{mailfromd} variable @code{x} is 2 or 3,
it will accept the message immediately, and add a @samp{X-Branch: 1}
header to it.  If @code{x} equals 2 or 4 or 6, this code will add
@samp{X-Branch: 2} header to the message and will continue processing
it.  Otherwise, it will reject the message.

  The controlling condition of a @code{switch} statement may evaluate
to numeric or string type.  The type of the condition governs the
type of comparisons used in @code{case} branches: for numeric types,
numeric equality will be used, whereas for string types, string
equality is used.

@node Loops
@section Loop Statements

@kwindex loop
@kwindex while
@cindex loop statement
  The loop statement allows for repeated execution of a block of code,
controlled by some conditional expression.  It has the following form:

@example
@group
loop [@var{label}]
     [for @var{stmt1}] [,while @var{expr1}] [,@var{stmt2}]
do
  @var{stmt3}
done [while @var{expr2}]
@end group
@end example

@noindent
where @var{stmt1}, @var{stmt2}, and @var{stmt3} are statement lists,
@var{expr1} and @var{expr2} are expressions.

  The control flow is as follows:

@enumerate 1
@item
   If @var{stmt1} is specified, execute it.

@item
   Evaluate @var{expr1}.  If it is zero, go to 6.  Otherwise, continue.

@item
   Execute @var{stmt3}.

@item
   If @var{stmt2} is supplied, execute it.

@item
   If @var{expr2} is given, evaluate it.  If it is zero, go to 6.
Otherwise, go to 2.

@item
   End.
@end enumerate

  Thus, @var{stmt3} is executed until either @var{expr1} or
@var{expr2} yield a zero value.

@cindex loop body
  The @dfn{loop body} -- @var{stmt3} -- can contain special
statements:

@table @code
@kwindex break
@cindex break statement
@item break [@var{label}]
  Terminates the loop immediately.  Control passes to @samp{6} (End)
in the formal definition above.  If @var{label} is supplied, the
statement terminates the loop statement marked with that label.  This
allows to break from nested loops.

  It is similar to @code{break} statement in @sc{c} or shell.

@kwindex next
@cindex next statement
@item next [@var{label}]
  Initiates next iteration of the loop.  Control passes to @samp{4} in
the formal definition above.  If @var{label} is supplied, the
statement starts next iteration of the loop statement marked with that
label.  This allows to request next iteration of an upper-level
loop from a nested loop statement.
@end table

  The @code{loop} statement can be used to create iterative statements
of arbitrary complexity.  Let's illustrate it in comparison with @sc{c}.

@cindex infinite loop
@cindex loop, infinite
The statement:

@example
@group
loop
do
  @var{stmt-list}
done
@end group
@end example

@noindent
creates an infinite loop.  The only way to exit from such a loop is to
call @code{break} (or @code{return}, if used within a function),
somewhere in @var{stmt-list}.

@cindex loop, while-style
@cindex while loop
  The following statement is equivalent to @code{while (@var{expr1})
@var{stmt-list}} in @sc{c}:

@example
@group
loop while @var{expr}
do
  @var{stmt-list}
done
@end group
@end example

@cindex loop, for-style
@cindex for loop
  The @sc{c} construct @code{for (@var{expr1}; @var{expr2}; @var{expr3})}
is written in @acronym{MFL} as follows:

@example
@group
loop for @var{stmt1}, while @var{expr2}, @var{stmt2}
do
  @var{stmt3}
done
@end group
@end example

  For example, to repeat @var{stmt3} 10 times:

@example
@group
loop for set i 0, while i < 10, set i i + 1
do
  @var{stmt3}
done
@end group
@end example

@cindex loop, do-style
@cindex do loop
  Finally, the @sc{c} @samp{do} loop is implemented as follows:

@example
@group
loop
do
  @var{stmt-list}
done while @var{expr}
@end group
@end example

  As a real-life example of a loop statement, let's consider the
implementation of function @code{ptr_validate}, which takes a single
argument @var{ipstr}, and checks its validity using the following algorithm:

Perform a @acronym{DNS} reverse-mapping for @var{ipstr}, looking up the
corresponding @code{PTR} record in @samp{in-addr.arpa}.  For each record
returned, look up its @acronym{IP} addresses (A records).  If @var{ipstr} is
among the returned @acronym{IP} addresses, return 1 (@code{true}), otherwise
return 0 (@code{false}).

  The implementation of this function in @acronym{MFL} is:

@example
#pragma regex push +extended

func ptr_validate(string ipstr) returns number
do
  loop for string names dns_getname(ipstr) . " "
           number i index(names, " "),
       while i != -1,
       set names substr(names, i + 1)
       set i index(names, " ")
  do
    loop for string addrs dns_getaddr(substr(names, 0, i)) . " "
             number j index(addrs, " "),
         while j != -1,
         set addrs substr(addrs, j + 1)
         set j index(addrs, " ")
    do
      if ipstr == substr(addrs, 0, j)
        return 1
      fi
    done
  done
  return 0
done
@end example

@node Exceptions
@section Exceptional Conditions
@cindex exceptions, defined
  When the running program encounters a condition it is not able
to handle, it signals an @dfn{exception}.  To illustrate the concept,
let's consider the execution of the following code fragment:

@example
@group
  if primitive_hasmx(domainpart($f))
    accept
  fi
@end group
@end example

@noindent
The function @code{primitive_hasmx} (@pxref{primitive_hasmx}) tests whether the
domain name given as its argument has any @samp{MX} records.  It should
return a boolean value.  However, when querying the Domain Name
System, it may fail to get a definite result.  For example, the @acronym{DNS}
server can be down or temporary unavailable.  In other words,
@code{primitive_hasmx} can be in a situation when, instead of returning
@samp{yes} or @samp{no}, it has to return @samp{don't know}.  It has
no way of doing so, therefore it signals an @dfn{exception}.

@anchor{status.mf}
@anchor{status.mfl}
@cindex exception types
@cindex @file{status.mfl}, module
@cindex exceptions, symbolic names
@flindex status.mfl
  Each exception is identified by @dfn{exception type}, an integer
number associated with it.

@menu
* Built-in Exceptions::
* User-defined Exceptions::
* Catch and Throw::
@end menu

@node Built-in Exceptions
@subsection Built-in Exceptions
@flindex status.mfl
The first @value{MAX_E_EXCEPTIONS} exception numbers are reserved for
@dfn{built-in exceptions}.  These are declared in module @file{status.mfl}.
The following table summarizes all built-in exception types implemented by
@command{mailfromd} version @value{VERSION}.  Exceptions are listed in
lexicographic order.

@anchor{exception names}
@defvr {Exception} e_badmmq
  The called function cannot finish its task because an incompatible
message modification function was called at some point before it.
For details, @ref{MMQ and dkim_sign}.
@end defvr

@defvr {Exception} e_dbfailure
  General database failure.  For example, the database cannot be
opened.  This exception can be signaled by any function that queries
any @acronym{DBM} database.
@end defvr

@defvr {Exception} e_divzero
Division by zero.
@end defvr

@defvr {Exception} e_exists
  This exception is emitted by @code{dbinsert} built-in if the
requested key is already present in the database (@pxref{Database
functions,dbinsert}).
@end defvr

@defvr {Exception} e_eof
  Function reached end of file while reading.  @xref{I/O functions},
for a description of functions that can signal this exception.
@end defvr

@defvr {Exception} e_failure
  A general failure has occurred.  In particular, this exception is
signaled by @acronym{DNS} lookup functions when any permanent failure occurs.
This exception can be signaled by any @acronym{DNS}-related function
(@code{hasmx}, @code{poll}, etc.) or operation (@code{mx matches}).
@end defvr

@defvr {Exception} e_format
  Invalid input format.  This exception is signaled if input data to a
function are improperly formatted.  In version @value{VERSION} it is
signaled by @code{message_burst} function if its input message is not
formatted according to RFC 934.  @xref{Message digest functions}.
@end defvr

@defvr {Exception} e_ilseq
  Illegal byte sequence.  Signaled when a string cannot be converted
between character sets because a sequence of bytes was encountered
that is not defined for the source character set or cannot be
represented in the destination character set.

@xref{MIME decoding}, for details.
@end defvr

@defvr {Exception} e_inval
  Arguments supplied to a function are invalid.
@end defvr

@defvr {Exception} e_invcidr
  Invalid @acronym{CIDR} notation.  This is signaled by
@code{match_cidr} function when its second argument is not a valid
@acronym{CIDR}.
@end defvr

@defvr {Exception} e_invip
  Invalid @acronym{IP} address.  This is signaled by @code{match_cidr} function
when its first argument is not a valid @acronym{IP} address.
@end defvr

@defvr {Exception} e_invtime
  Invalid time interval specification.  It is signaled by
@code{interval} function if its argument is not a valid time interval
(@pxref{time interval specification}).
@end defvr

@defvr {Exception} e_io
  An error occurred during the input-output operation.  @xref{I/O
functions}, for a description of functions that can signal this
exception.
@end defvr

@defvr {Exception} e_macroundef
  A Sendmail macro is undefined.
@end defvr

@defvr {Exception} e_not_found
  Required entity is not found.  It is raised, for example, by
@code{message_find_header}, when the requested header is not present
in the message and by @acronym{DNS} resolver functions when unable to
resolve host name or @acronym{IP} address.
@end defvr

@defvr {Exception} e_range
  The supplied argument is outside the allowed range.  This is
signalled, for example, by @code{substring} function (@pxref{substring}).
@end defvr

@defvr {Exception} e_regcomp
  Regular expression cannot be compiled.  This can happen when a
regular expression (a right-hand argument of a @code{matches}
operator) is built at the runtime and the produced string is an
invalid regex.
@end defvr

@defvr {Exception} e_ston_conv
  String-to-number conversion failed.  This can be signaled when a
string is used in numeric context which cannot be converted to the numeric
data type.  For example:

@example
@group
 set x "10a"
 set y x / 2
@end group
@end example

@noindent
  In this code fragment, line 2 will raise the @code{e_ston_conv}
exception, since @samp{10a} cannot be converted to a number.
@end defvr

@defvr {Exception} e_success
  This is not an exception in the strict sense of the word, but a
constant indicating success.
@end defvr

@defvr {Exception} e_temp_failure
  A temporary failure has occurred.  This can be signaled by
@acronym{DNS}-related functions or operations.
@end defvr

@defvr {Exception} e_too_many
  Raised by various @acronym{DNS} functions when they encounter a long chain
of CNAME records when trying to resolve a hostname.  @xref{CNAME chains}.
@end defvr

@defvr {Exception} e_url
  The supplied @acronym{URL} is invalid.  @xref{Interfaces to
Third-Party Programs}.
@end defvr

@node User-defined Exceptions
@subsection User-defined Exceptions
@kwindex dclex
  You can define your own exception types using the @code{dclex}
statement:

@example
dclex @var{type}
@end example

In this statement, @var{type} must be a valid @acronym{MFL}
identifier, not used for another constant (@pxref{Constants}).
The @code{dclex} statement defines a new exception identified by
the constant @var{type} and allocates a new exception number for it.

The @var{type} can subsequently be used in @code{throw} and
@code{catch} statements, for example:

@example
dclex myrange

number fact(number val)
  returns number
do
  if val < 0
    throw myrange "fact argument is out of range"
  fi
  @dots{}
done
@end example

@node Catch and Throw
@subsection Exception Handling
@cindex exceptions, default handling
@cindex default exception handling
  Normally when an exception is signalled, the program execution is
terminated and the @acronym{MTA} is returned a @code{tempfail}
status.  Additional information regarding the exception is then output
to the logging channel (@pxref{Logging and Debugging}).  However, the
user can intercept any exception by installing his own
exception-handling routines.

@cindex @code{try} statement
@cindex @code{catch} statement
@cindex try--catch construct
@cindex exception-handling routines
@cindex exception handlers
@kwindex catch
  An exception-handling routine is introduced by a @dfn{try--catch}
statement, which has the following syntax:

@example
@group
try
do
  @var{stmtlist}
done
catch @var{exception-list}
do
  @var{handler-body}
done
@end group
@end example

@noindent
where @var{stmtlist} and @var{handler-body} are sequences of
@acronym{MFL} statements and @var{exception-list} is the list of
exception types, separated by the word @code{or}.  A special
@var{exception-list} @samp{*} is allowed and means all exceptions.

This construct works as follows.  First, the statements from
@var{stmtlist} are executed.  If the execution finishes
successfully, control is passed to the first statement after the
@samp{catch} block.  Otherwise, if an exception is signalled and this
exception is listed in @var{exception-list}, the execution is passed to the
@var{handler-body}.  If the exception is not listed in
@var{exception-list}, it is handled as usual.

  The following example shows a @samp{try--catch} construct used for
handling eventual exceptions, signalled by @code{primitive_hasmx}.

@example
@group
try
do
  if primitive_hasmx(domainpart($f))
    accept
  else
    reject
  fi
done
catch e_failure or e_temp_failure
do
  echo "primitive_hasmx failed"
  continue
done
@end group
@end example

@cindex scope of exception handlers
@cindex scope of a @code{catch}
@cindex @code{catch} scope
@cindex exception handler scope
  The @samp{try--catch} statement can appear anywhere inside a function or
a handler, but it cannot appear outside of them.  It can also be nested
within another @samp{try--catch}, in either of its parts.  Upon exit from a
function or milter handler, all exceptions are restored to the state
they had when it has been entered.

@cindex catch, standalone
@cindex standalone catch
  A @code{catch} block can also be used alone, without preceding @code{try}
part.  Such a construct is called a @dfn{standalone catch}.  It is
mostly useful for setting global exception handlers in a @code{begin}
statement (@pxref{begin/end}).  When used within a usual function or
handler, the exception handlers set by a standalone catch
remain in force until either another standalone catch appears further
in the same function or handler, or an end of the function is
encountered, whichever occurs first.

  A standalone catch defined within a function must return from
it by executing @code{return} statement.  If it does not do that
explicitly, the default value of 1 is returned.  A standalone catch
defined within a milter handler must end execution with any of the
following actions: @code{accept}, @code{continue}, @code{discard},
@code{reject}, @code{tempfail}.  By default, @code{continue} is
used.

  It is not recommended to mix @samp{try--catch} constructs and
standalone catches.  If a standalone catch appears within a
@samp{try--catch} statement, its scope of visibility is undefined.

@cindex returning from a catch
@cindex returning from an exception handler
@cindex @code{catch}, returning from
@cindex exception handler, returning from
@cindex @code{catch} arguments
@cindex arguments, @code{catch}
  Upon entry to a @var{handler-body}, two implicit positional arguments
are defined, which can be referenced in @var{handler-body} as @code{$1}
and @code{$2}@footnote{As of @command{mailfromd} version
@value{VERSION}, there is also a third implicit argument, which holds
the value of program counter where the exception occurred.  Currently
it is considered to be an implementation artifact.  Filter writers are
discouraged from relying on it.}.  The first argument gives the
numeric code of the exception that has occurred.  The second argument
is a textual string containing a human-readable description of the exception.

  The following is an improved version of the previous example, which
uses these parameters to supply more information about the failure:

@example
try
do
  if primitive_hasmx(domainpart($f))
    accept
  else
    reject
  fi
done
catch e_failure or e_temp_failure
do
  echo "Caught exception $1: $2"
  continue
done
@end example

@cindex @code{hasmx}, definition of the function
  The following example defines the function @code{hasmx} that
returns true if the domain part of its argument has any @samp{MX} records, and
false if it does not or if an exception occurs @footnote{This function is
part of the @code{mailfromd} library, @xref{hasmx}.}.

@example
func hasmx (string s)
  returns number
do
  try
  do
    return primitive_hasmx(domainpart(s))
  done
  catch *
  do
    return 0
  done
done
@end example

  The same function can written using standalone @code{catch}:

@example
func hasmx (string s)
  returns number
do
  catch *
  do
    return 0
  done
  return primitive_hasmx(domainpart(s))
done
@end example


@cindex @code{catch}, accessing variables from
@cindex variables, accessing from @code{catch}
@cindex accessing variables from @code{catch}
  All variables remain visible within @code{catch} body, with the
exception of positional arguments of the enclosing handler.  To access
positional arguments of a handler from the @code{catch} body, assign
them to local variables prior to the @samp{try--catch} construct, e.g.:

@example
@group
prog header
do
  string hname $1
  string hvalue $2
  try
  do
    @dots{}
  done
  catch *
  do
    echo "Exception $1 while processing header %hname: %hvalue"
    echo $2
    tempfail
  done
@end group
@end example

@anchor{throw}
@cindex exceptions, raising from code
@cindex raising exceptions
@kwindex throw
  You can also generate (or @dfn{raise}) exceptions explicitly in the
code, using @code{throw} statement:

@example
throw @var{excode} @var{descr}
@end example

  The arguments correspond exactly to the positional parameters of the
@code{catch} statement: @var{excode} gives the numeric code of the
exception, @var{descr} gives its textual description.  This statement
can be used in complex scripts to create non-local exits from deeply
nested statements.  @FIXME{Elaborate on that.}

  Notice, that the the @var{excode} argument must be an immediate
value: an exception identifier (either a built-in one or one declared
previously using a @code{dclex} statement).

@node Polling
@section Sender Verification Tests

@cindex sender verification, writing tests
  The filter script language provides a wide variety of functions for
sender address verification or @dfn{polling}, for short.  These
functions, which were described in @ref{SMTP Callout functions}, can be
used to implement any sender verification method.  The additional data
that can be needed is normally supplied by two global variables:
@code{ehlo_domain}, keeping the default domain for the @code{EHLO}
command, and @code{mailfrom_address}, which stores the sender address
for probe messages (@pxref{Predefined variables}).

  For example, a simplest way to implement standard polling would be:

@example
prog envfrom
do
  if stdpoll($1, ehlo_domain, mailfrom_address) == 0
    accept
  else
    reject 550 5.1.0 "Sender validity not confirmed"
  fi
done
@end example

  However, this does not take into account exceptions that
@code{stdpoll} can signal.  To handle them, one will have to use
@code{catch}, for example thus:

@example
require status

prog envfrom
do
  try
  do
    if stdpoll($1, ehlo_domain, mailfrom_address) == 0
      accept
    else
      reject 550 5.1.0 "Sender validity not confirmed"
    fi
  done
  catch e_failure or e_temp_failure
  do
    switch $1
    do
    case failure:
      reject 550 5.1.0 "Sender validity not confirmed"
    case temp_failure:
      tempfail 450 4.1.0 "Try again later"
    done
  done
done
@end example

  If polls are used often, one can define a wrapper function, and use
it instead.  The following example illustrates this approach:

@float Figure, figure-poll-wrapper
@caption{Building Poll Wrappers}
@example
func poll_wrapper(string email) returns number
do
  catch e_failure or e_temp_failure
  do
    return email
  done
  return stdpoll(email, ehlo_domain, mailfrom_address)
done

prog envfrom
do
  switch poll_wrapper($f)
  do
  case success:
    accept
  case not_found or failure:
    reject 550 5.1.0 "Sender validity not confirmed"
  case temp_failure:
    tempfail 450 4.1.0 "Try again later"
  done
done
@end example
@end float

  Notice the way @code{envfrom} handles @code{success} and
@code{not_found}, which are not exceptions in the strict sense of the
word.

@cindex @code{on} statement
@cindex @code{when} keyword
  The above paradigm is so common that @command{mailfromd} provides a
special language construct to simplify it: the @code{on} statement.
Instead of manually writing the wrapper function and using it as a
@code{switch} condition, you can rewrite the above example as:

@float Figure, figure-stdpoll
@caption{Standard poll example}
@example
@group
prog envfrom
do
  on stdpoll($1, ehlo_domain, mailfrom_address)
  do
  when success:
    accept
  when not_found or failure:
    reject 550 5.1.0 "Sender validity not confirmed"
  when temp_failure:
    tempfail 450 4.1.0 "Try again later"
  done
done
@end group
@end example
@end float

@noindent
As you see the statement is pretty similar to @code{switch}.  The
major syntactic difference is the use of the keyword @code{when} to
introduce conditional branches.

  General syntax of the @code{on} statement is:

@example
@group
on @var{condition}
do
  when @var{x1} [or @var{x2} @dots{}]:
    @var{stmt1}
  when @var{y1} [or @var{y2} @dots{}]:
    @var{stmt2}
    .
    .
    .
done
@end group
@end example

@noindent
The @var{condition} is either a function call or a special @code{poll}
statement (see below).  The values used in @code{when} branches are
normally symbolic exception names (@pxref{exception names}).

  When the compiler processes the @code{on} statement it does the
following:

@enumerate 1
@item
Builds a unique wrapper function, similar to that described in
@ref{figure-poll-wrapper};  The name of the function is constructed
from the @var{condition} function name and an unsigned number,
called @dfn{exception mask}, that is unique for each combination of
exceptions used in @code{when} branches;  To avoid name clashes with
the user-defined functions, the wrapper name begins and ends with
@samp{$} which normally is not allowed in the identifiers;

@item
Translates the @code{on} body to the corresponding @code{switch}
statement;
@end enumerate

@anchor{poll}
@cindex @code{poll} keyword
@cindex @code{poll} statement, defined
  A special form of the @var{condition} is @code{poll} keyword,
whose syntax is:

@example
@group
poll [for] @var{email}
     [host @var{host}]
     [from @var{domain}]
     [as @var{email}]
@end group
@end example

  The order of particular keywords in the @code{poll} statement is
arbitrary, for example @code{as @var{email}} can appear before
@var{email} as well as after it.

@cindex @code{poll} command, standard verification
@cindex standard verification with @code{poll}
  The simplest form, @code{poll @var{email}}, performs the standard
sender verification of email address @var{email}.  It is translated
to the following function call:

@example
  stdpoll(@var{email}, ehlo_domain, mailfrom_address)
@end example

@cindex @code{poll} command, strict verification
@cindex strict verification with @code{poll}
  The construct @code{poll @var{email} host @var{host}}, runs the
strict sender verification of address @var{email} on the given host.
It is translated to the following call:

@example
  strictpoll(@var{host}, @var{email}, ehlo_domain, mailfrom_address)
@end example

@cindex as
@cindex from
  Other keywords of the @code{poll} statement modify these two basic
forms.  The @code{as} keyword introduces the email address to be used
in the @acronym{SMTP} @code{MAIL FROM} command, instead of
@code{mailfrom_address}.  The @code{from} keyword sets the domain
name to be used in @code{EHLO} command.  So, for example the following
construct:

@example
  poll @var{email} host @var{host} from @var{domain} as @var{addr}
@end example

@noindent
is translated to

@example
  strictpoll(@var{host}, @var{email}, @var{domain}, @var{addr})
@end example

  To summarize the above, the code described in @ref{figure-stdpoll}
can be written as:

@example
@group
prog envfrom
do
  on poll $f do
  when success:
    accept
  when not_found or failure:
    reject 550 5.1.0 "Sender validity not confirmed"
  when temp_failure:
    tempfail 450 4.1.0 "Try again later"
  done
done
@end group
@end example

@node Modules
@section Modules

@cindex module, defined
  A @dfn{module} is a logically isolated part of code that implements a
separate concern or feature and contains a collection of conceptually
united functions and/or data.  Each module occupies a separate
compilation unit (i.e. file).  The functionality provided by
a module is incorporated into another module or the main program by
@dfn{requiring} this module or by @dfn{importing} the desired
components from it.

@menu
* module structure::    Declaring Modules
* scope of visibility::
* import::              Require and Import
@end menu

@node module structure
@subsection Declaring Modules
@cindex module declaration
@kwindex module
  A module file must begin with a @dfn{module declaration}:

@example
module @var{modname} [@var{interface-type}].
@end example

  Note the final dot.

The @var{modname} parameter declares the name of the module.  It is
recommended that it be the same as the file name without the
@samp{.mfl} extension.  The module name must be a valid @acronym{MFL}
literal.  It also must not coincide with any defined @acronym{MFL}
symbol, therefore we recommend to always quote it (see example below).

The optional parameter @var{interface-type} defines the @dfn{default
scope of visibility} for the symbols declared in this module.  If it is
@samp{public}, then all symbols declared in this module are made
public (importable) by default, unless explicitly declared otherwise
(@pxref{scope of visibility}).  If it is @samp{static}, then all
symbols, not explicitly marked as public, become static.  If the
@var{interface-type} is not given, @samp{public} is assumed.

The actual @acronym{MFL} code follows the @samp{module} line.

The module definition is terminated by the @dfn{logical end} of its
compilation unit, i.e. either by the end of file, or by the
keyword @code{bye}, whichever occurs first.

@kwindex bye
Special keyword @code{bye} may be used to prematurely end the current
compilation unit before the physical end of the containing file.
Any material between @code{bye} and the end of file is ignored by the
compiler.

Let's illustrate these concepts by writing a module @samp{revip}:

@example
@group
module 'revip' public.

func revip(string ip)
  returns string
do
  return inet_ntoa(ntohl(inet_aton(ip)))
done

bye

This text is ignored.  You may put any additional
documentation here.
@end group
@end example

@node scope of visibility
@subsection Scope of Visibility
@cindex scope of visibility
  @dfn{Scope of Visibility} of a symbol defines from where this symbol
may be referred to.  Symbols in @acronym{MFL} may have either of the
following two scopes:

@table @dfn
@item Public
  Public symbols are visible from the current module, as well as from
any external modules, including the main script file, provided that
they are properly imported (@pxref{import}).

@item Static
  Static symbols are visible only from the current module.  There is
no way to refer to them from outside.
@end table

  The default scope of visibility for all symbols declared within
a module is defined in the module declaration (@pxref{module
structure}).  It may be overridden for any individual symbol by
prefixing its declaration with an appropriate @dfn{qualifier}: either
@code{public} or @code{static}.

@node import
@subsection Require and Import
@cindex importing from modules
@cindex requiring modules
  Functions or variables declared in another module must be @dfn{imported}
prior to their actual use.  @acronym{MFL} provides two ways of doing
so: by @dfn{requiring} the entire module or by importing selected
symbols from it.

@anchor{module search path}
@cindex module search path
  Modules are looked up in the @dfn{module search path}.  The default
module search path consists of two directories:

@enumerate 1
@item @file{@var{prefix}/share/mailfromd}
@item @file{@var{prefix}/share/mailfromd/@value{VERSION}}
@end enumerate

@noindent
where @var{prefix} stands for the installation prefix (normally
@file{/usr} or @file{/usr/local}).

  Module search path can be changed in the configuration file,
using the @code{module-path} statement (@pxref{conf-base,
module-path}), or from the command line, using the @option{-P}
(@option{--module-path}) option (@pxref{--module-path}).

@deffn {Module Import} require modname
  The @code{require} statement instructs the compiler to locate the
module @var{modname} and to load all public interfaces from it.
@end deffn

  The compiler looks for the file @file{@var{modname}.mfl} in the
module search path.  If no such file is found, a compilation error is
reported.

  For example, the following statement:

@example
require revip
@end example

@noindent
imports all interfaces from the module @file{revip.mfl}.

@kwindex from
@kwindex import
@cindex @code{from @dots{} import}
  Another, more sophisticated way to import from a module is to use
the @samp{from ... import} construct:

@example
from @var{module} import @var{symbols}.
@end example

Note the final dot.  The @samp{from} and @samp{module} statements are
the only two constructs in @acronym{MFL} that require the delimiter.

The @var{module} has the same semantics as in the @code{require}
construct.  The @var{symbols} is a comma-separated list of symbol
names to import from @var{module}.  A symbol name may be given in
several forms:

@enumerate 1
@item Literal

Literals specify exact symbol names to import.  For example,
the following statement imports from module @file{A.mfl} symbols
@samp{foo} and @samp{bar}:

@example
from A import foo,bar.
@end example

@item Regular expression

Regular expressions must be surrounded by slashes.  A regular
expression instructs the compiler to import all symbols whose
names match that expression.  For example, the following statement
imports from @file{A.mfl} all symbols whose names begin with @samp{foo}
and contain at least one digit after it:

@example
from A import '/^foo.*[0-9]/'.
@end example

The type of regular expressions used in the @samp{from} statement is
controlled by @code{#pragma regex} (@pxref{regex}).

@item Regular expression with transformation

Regular expression may be followed by a @dfn{s-expression}, i.e. a
@command{sed}-like expression of the form:

@example
s/@var{regexp}/@var{replace}/[@var{flags}]
@end example

@noindent
where @var{regexp} is a @dfn{regular expression}, @var{replace} is a
replacement for each part of the input that matches @var{regexp}.
S-expressions and their parts are discussed in detail in
@ref{s-expression}.

The effect of such construct is to import all symbols that match the
regular expression and apply the s-expression to their names.

For example:

@example
from A import '/^foo.*[0-9]/s/.*/my_&/'.
@end example

This statement imports all symbols whose names begin with @samp{foo}
and contain at least one digit after it, and renames them, by prefixing
their names with the string @samp{my_}.  Thus, if @file{A.mfl} declared a
function @samp{foo_1}, it becomes visible under the name of @samp{my_foo_1}.

@end enumerate

@node mfmod
@section Dynamically Loaded Modules
@cindex mfmod
  Native mailfromd modules described above rely on the functions
provided by the @command{mailfromd} binary.  For more sophisticated
tasks you might need to use C functions, either for efficiency reasons
or to make use of some third-party library.  This is possible using
special kind of modules called @dfn{mfmod}.

An mfmod consists of two major parts: a dynamically loaded library
that provides its main functionality and a small @dfn{interface}
mailfromd module.  The convention is that for the module @var{x}
the library is named @file{mfmod_@var{x}.so}@footnote{The actual
suffix depends on operating system.  It is @samp{.so} on all POSIX
systems.}, and the interface module file is @file{@var{x}.mfl}.

At the time of this writing, three mfmods exist:

@table @asis
@item mfmod_pcre
Provides support for Perl-compatible regular expressions.  It also
contains a special function for scanning an email message for a match
to a regular expression.  @xref{Top, Mfmod_pcre,, mfmod_pcre,
Mfmod_pcre}.

@item mfmod_ldap
Functions for searching in LDAP directory.  @xref{Top, Mfmod_ldap,,
mfmod_ldap, Mfmod_ldap}.

@item mfmod_openmetrics
Openmetrics support for @command{mailfromd}.  @xref{Top,
mfmod_openmetrics,, mfmod_openmetrics, mfmod_openmetrics reference}.
@end table

The subsections below describe the internal structure of an mfmod in
detail.

@menu
* Loadable Library::
* Interface Module::
* mfmodnew::  Creating a Mfmod Structure
@end menu

@node Loadable Library
@subsection Loadable Library
@cindex mfmod, loadable library
External functions in the loadable library must be declared as

@example
int funcname(long count, MFMOD_PARAM *param, MFMOD_PARAM *retval);
@end example

@findex mfmod.h
The @code{MFMOD_PARAM} type is declared in the header file
@file{mailfromd/mfmod.h}, which must be included at the start of the
source code.

@deftp {mfmod type} MFMOD_PARAM type string number
This type is defined as follows:

@example
typedef struct mfmod_param @{
        mfmod_data_type type;
        union @{
                char *string;
                long number;
                mu_message_t message;
        @};
@} MFMOD_PARAM;
@end example

The @code{type} fields defines the type of the data represented by the
object.  Its possible values are:

@defvr {mfmod constant} mfmod_string
String data.
@end defvr

@defvr {mfmod constant} mfmod_number
Numeric data.
@end defvr

@defvr {mfmod constant} mfmod_message
A @code{mailutils} message object (@code{mu_message_t}).
@end defvr

The actual data are accessed as @code{string}, @code{number}, or
@code{message}, depending on the value of @code{type}.
@end deftp

The first parameter in the external function declaration, @var{count},
is the number of arguments passed to that function.  Actual arguments are
passed in the @code{MFMOD_PARAM} array @var{param}.  The function should
never modify its elements.  If the function returns a value to MFL, it
must pass it in the @var{retval} parameter.  For example, the
following code returns the numeric value @samp{1}:

@example
retval->type = mfmod_number;
retval->number = 1;
@end example

To return a string value, allocate it using @code{malloc},
@code{calloc} or a similar function, like this:

@example
retval->type = mfmod_string;
retval->string = strdup("text");
@end example

If a message is returned, it should be created using mailutils message
creation primitives.  @command{Mailutils} will call
@code{mu_message_destroy} on it, when it is no longer used.

The return value (in the C sense) of the function is used to determine
whether it succeeded or not.  Zero means success.  Returning -1 causes
a runtime exception @code{e_failure} with a generic error text
indicating the names of the module and function that caused the
exception.  Any other non-zero value is treated as a
@command{mailfromd} exception code (@pxref{Exceptions}).  In this case
an additional textual explanation of the error can be supplied in the
@code{retval} variable, whose type must then be set to @code{mfmod_string}.
This explanation string must be allocated using @code{malloc}.

To facilitate error handling, the following functions are provided
(declared in the @file{mailfromd/mfmod.h} header file):

@deftypefn {mfmod} int mfmod_error (MFMOD_PARAM *@var{retval}, int @var{ecode}, @
  char const *@var{fmt}, ...)
Raises exception @var{ecode} with the error message formatted from the
variadic arguments using @code{printf}-style format string @var{fmt}.

Example use:

@example
@group
if (error_condition)
   return mfmod_error(retval, "error %s occurred", error_text);
@end group
@end example
@end deftypefn
  
@deftypefn {mfmod} int mfmod_error_argtype (MFMOD_PARAM *@var{param}, MFMOD_PARAM *@var{retval}, @
  int @var{n}, int @var{exptype})
Reports argument type mismatch error (@code{e_inval} with
appropriately formatted error text).  Arguments are:

@table @var
@item param
@itemx retval
The two arguments passed to the interface function.

@item n
0-based index of the erroneous argument in @var{param}.

@item exptype
Expected data type of @code{@var{param}[@var{n}]}.
@end table

You will seldom need to use this function directly.  Instead, use the
@code{ASSERT_ARGTYPE} macro described below.
@end deftypefn

@deftypefn {mfmod} {char const *} mfmod_data_type_str(mfmod_data_type @var{type})
Returns the MFL name of the mfmod data type @var{type}.
@end deftypefn

The following convenience macros are provided for checking the number
of argument and their types and returning error if necessary:

@deffn {mfmod macro} ASSERT_ARGCOUNT (MFMOD_PARAM *@var{retval}, long @var{count}, long @var{expcount})
Assert that the number of arguments (@var{count}) equals the expected
number (@var{expcount}).  If it does not, return the @code{e_inval}
exception with a descriptive error text.

@var{retval} and @var{count} are corresponding arguments from the
calling function.
@end deffn

@deffn {mfmod macro} ASSERT_ARGTYPE (MFMOD_PARAM *@var{param}, MFMOD_PARAM *@var{retval}, @
  int @var{n}, int @var{exptype})
Check if the data type of the @var{n}th parameter
(i.e. @code{@var{param}[@var{n}]}) is @var{exptype} and return the
@code{e_inval} exception if it does not.
@end deffn

As an example, suppose you want to write an interface to the system
@code{crypt} function.  The loadable library source,
@file{mfmod_crypt.c}, will look as follows:

@example
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <mailfromd/mfmod.h>
#include <mailfromd/exceptions.h>

/*
 * Arguments:
 *   param[0] - key string to hash.
 *   param[1] - salt value.
 */
int
cryptval(long count, MFMOD_PARAM *param, MFMOD_PARAM *retval)
@{
     char *hash;

     /* Check if input arguments are correct: */
     ASSERT_ARGCOUNT(retval, count, 2);
     ASSERT_ARGTYPE(param, retval, 0, mfmod_string);
     ASSERT_ARGTYPE(param, retval, 1, mfmod_string);

     /* Hash the key string. */
     hash = crypt(param[0].string, param[1].string);

     /* Return string to MFL */
     retval->type = mfmod_string;
     retval->string = strdup(hash);

     /* Throw exception if out of memory */
     if (retval->string == NULL)
          return -1;

     return 0;
@}
@end example

The exact way of building a loadable library from this source file
depends on the operating system.  For example, on GNU/Linux you
would do:

@example
cc -shared -fPIC -DPIC -omfmod_crypt.so -lcrypt mfmod_crypt.c
@end example

The preferred and portable way of doing so is via @command{libtool}
(@pxref{Top, Shared library support for GNU,,libtool,Libtool}).
@command{Mailfromd} provides a special command @command{mfmodnew} that
creates infrastructure necessary for building loadable modules.
@xref{mfmodnew}.

@node Interface Module
@subsection Interface Module
@cindex mfmod, interface module
The interface module is responsible for loading the library, and
providing MFL wrappers over external functions defined in it.

For the first task, the @code{dlopen} function is provided.  It takes
a single argument, the file name of the library to load.  This can
be an absolute pathname, in which case it is used as is, or a relative
file name, which will be searched in the @dfn{library search path}
(@pxref{mfmod-path}).  On success, the function returns the
@dfn{library handle}, which will be used in subsequent calls to
identify that library.  On error, a runtime exception is signalled.

It is common to call the @code{dlopen} function in the @code{startup}
section of the interface module (@pxref{startup/shutdown}), so that
the library gets loaded at the program startup.  For example:

@example
static number libh

prog startup
do
  set libh dlopen("mfmod_crypt.so")
done
@end example

The function @code{dlcall} is provided to call a function from the
already loaded library.  It is a variadic function with three
mandatory parameters:

@enumerate 1
@item The handle of the loadable library as returned by @code{dlopen}.
@item Name of the external function to call.
@item Type string
@end enumerate

The @dfn{type string} argument declares data types of the variable
arguments.  It contains a single letter for each additional argument
passed to @code{dlcall}.  The valid letters are:

@table @asis
@item s
The argument is of string type.

@item n
@itemx d
The argument is of numeric type.

@item m
The argument is of message type.
@end table

For example, the following will call the @code{cryptval} function
defined in the previous section (supposing @code{key} and @code{salt}
are two string MFL variables):

@example
set x dlcall(libh, "cryptval", "ss", key, salt)
@end example

The last letter in type string can be @samp{+} or @samp{*}.  Both mean
that any number of arguments are allowed (all of the type given by
the penultimate type letter).  The difference between the two is that
@samp{+} allows for one or more arguments, while @samp{*} allows for
zero or more arguments.  For example, @samp{n+} means one or more
numeric arguments, and @samp{n*} means zero or more such arguments.
Both are intended to be used in variadic functions, e.g.:

@example
@group
func pringstddev(number ...)
  returns number
do
  return dlcall(libh, "stddev", "n*", $@@)
done
@end group
@end example

@kwindex void
The @code{dlcall} function returns the value returned by the
library function it invoked.  If the library function returns
no meaningful value, it is recommended to use the @code{void}
type cast around the @code{dlcall} invocation (@pxref{void type
cast}).  E.g.:

@example
@group
func out(string text)
do
  void(dlcall(libh, "output", "s", text))
done
@end group
@end example

Without @code{void} type cast, the definition above will produce the
following warning when compiled:

@example
return from dlcall is ignored
@end example

@node mfmodnew
@subsection Creating a Mfmod Structure
@cindex mfmodnew
  The @command{mfmodnew} provides a convenient start for writing a new
@i{mfmod}.  Given a name of the planned module, this command creates
directory @file{mfmod_@var{name}} and populates it with the files
necessary for building the new module using GNU autotools, as well as
boilerplate files for the loadable library and interface module.

  Let's see how to use it to create the @code{crypt} module outlined in
previous subsections.

  First, invoke the command:

@example
$ @kbd{mfmodnew crypt}
mfmodnew: setting up new module in mfmod_crypt
@end example

  Let's change to the new directory and see the files in it:

@example
$ @kbd{cd mfmod_crypt}
$ @kbd{ls}
Makefile.am  configure.ac  crypt.mfl  mfmod_crypt.c
@end example

  Now, open the @file{mfmod_crypt.c} file and add to it the definition
of the @code{cryptval} function (@pxref{Loadable Library}).  Then, add
the interface function definition from @ref{Interface Module} to file
@code{crypt.mfl}.

  The last thing to do is to edit @file{configure.ac}.  The
@code{crypt} function requires the @file{libcrypt} library, so the
following line should be added to the @samp{Checks for libraries.}
section.

@example
AC_CHECK_LIB([crypt], [crypt])
@end example

  Now, run @command{autoreconf}, as follows:

@example
$ @kbd{autoreconf -f -i -s}
@end example

  It will bootstrap the autotools infrastructure, importing additional
files as necessary.  Once done, you can build the project:

@example
$ @kbd{./configure}
$ @kbd{make}
@end example

Notice, that if the @code{autoreconf} stage ends abnormally with a
diagnostics like:

@example
configure.ac:21: error: possibly undefined macro: AC_MFMOD
@end example

@noindent
@findex mfmod.m4
that means that @command{autoconf} was unable to find the file
@file{mfmod.m4}, which provides that macro.  That's because the
directory where this file is installed is not searched by
@command{autoreconf}.  To fix this, supply the name of that directory
using the @option{-I} option.  E.g. assuming @file{mfmod.m4} is
installed in @file{/usr/local/share}:

@example
$ @kbd{autoreconf -fis -I /usr/local/share/aclocal}
@end example

@menu
* mfmodnew invocation::
@end menu

@node mfmodnew invocation
@subsubsection mfmodnew invocation

The @command{mfmodnew} is invoked as:

@example
mfmodnew [@var{options}] @var{modname} [@var{dir}]
@end example

@noindent
where @var{modname} is the name of the new module and @var{dir} is the
directory where to store the module infrastructure files.  Normally
you would omit @var{dir} altogether: in this case the utility will use
@file{mfmod_@var{modname}} as the directory name.

Options are:

@table @option
@item -C @var{dir}
Search for template files in @var{dir}, instead of the default
location.

@item -e @var{email}
Supply the author's email.  The email is passed as argument to
the @code{AC_INIT} macro in @file{configure.ac}.  By default it
is constructed as @samp{@var{username}@@@var{hostname}}.  If it is
incorrect, you can either edit @file{configure.ac} afterwards, or
just supply the correct one using this option.

@item -q
Suppress informative messages.

@item -h
Display a short command line usage help.
@end table

@node Preprocessor
@section @acronym{MFL} Preprocessor
@cindex preprocessor
@cindex m4
  Before compiling the script file, @command{mailfromd} preprocesses
it.  The built-in preprocessor handles only file inclusion
(@pxref{include}), while the rest of traditional facilities, such as
macro expansion, are supported via @command{m4}, which is used as an
external preprocessor.

  The detailed description of @command{m4} facilities lies far beyond
the scope of this document.  You will find a complete user manual in
@ref{Top, GNU M4 manual, GNU M4, m4, GNU M4 macro processor}.  For the
rest of this section we assume the reader is sufficiently
acquainted with @command{m4} macro processor.

@flindex pp-setup
@cindex preprocessor setup file
  The external preprocessor is invoked with @option{-s} flag, instructing
it to include line synchronization information in its output, which
is subsequently used by @acronym{MFL} compiler for purposes of error
reporting.  The initial set of macro definitions is supplied in
@dfn{preprocessor setup} file @file{pp-setup}, located in the library
search path@footnote{It is usually located in
@file{/usr/local/share/mailfromd/@value{VERSION}/include/pp-setup}.},
which is fed to the preprocessor input before the script file itself.
The default @file{pp-setup} file renames all @command{m4} built-in
macro names so they all start with the prefix @samp{m4_}@footnote{This
is similar to GNU m4 @option{--prefix-builtin} options.  This approach
was chosen to allow for using non-GNU @command{m4} implementations as
well.}.  It changes comment characters to @samp{/*}, @samp{*/} pair,
and leaves the default quoting characters, grave (@samp{`}) and acute
(@samp{'}) accents without change.  Finally, @file{pp-setup} defines
several useful macros (@pxref{m4 macros}).

@node Configuring Preprocessor
@subsection Preprocessor Configuration
@cindex preprocessor configuration
@cindex configuring the preprocessor
  The preprocessor is configured in the mailfromd configuration file,
using the preprocessor statement (@pxref{conf-preprocessor}).  The
default settings correspond to the following configuration:

@example
@group
preprocessor @{
  # Enable preprocessor
  enable yes;
  # Preprocessor command line stub.
  command "m4 -s";
  # Pass current include path to the preprocessor via -I options.
  pass-includes false;
  # Pass to the preprocessor the feature definitions via -D options
  # as well as any -D/-U options from the command line
  pass-defines true;
  # Name of the preprocessor setup file.  Unless absolute, it is
  # looked up in the include path.
  setup-file "pp-setup";
@}
@end group
@end example

@kwindex command
@kwindex preprocessor.command
@kwindex pass-includes
@kwindex preprocessor.pass-includes
If @code{pass-includes} is true, the @code{command} value
is augmented by zero or more @option{-I} options supplying it the
mailfromd include search path (@pxref{include search path}).

@kwindex pass-defines
@kwindex preprocessor.pass-defines
Furthermore, if @code{pass-defines} is set, zero or more
@option{-D} options defining optional features are passed to it (e.g.
@option{-DWITH_DKIM}) as well as any @option{-D} and @option{-U}
options from the mailfromd command line.

@kwindex setup-file
@kwindex preprocessor.setup-file
Unless the value of @code{setup-file} begins with a slash,
the file with this name is looked up in the current include search
path.  If found, its absolute name is passed to the preprocessor as
first argument.

If it begins with a slash, it is passed to the preprocessor as is.

@node Preprocessor Usage
@subsection Preprocessor Usage
@cindex E, -E @r{option, described}
  You can obtain the preprocessed output, without starting actual
compilation, using @option{-E} command line option:

@example
$ @kbd{mailfromd -E file.mfl}
@end example

The output is in the form of preprocessed source code, which is sent
to the standard output.  This can be useful, among others, to debug
your own macro definitions.

Macro definitions and deletions can be made on the command line, by
using the @option{-D} and @option{-U} options, provided that their use
is allowed by the @code{pass-defines} preprocessor configuration
setting (@pxref{Configuring Preprocessor}.  They have the following format:

@table @option
@xopindex{define, mailfromd, described}
@cindex D, -D @r{option, described}
@item -D @var{name}[=@var{value}]
@itemx --define=@var{name}[=@var{value}]
Define a symbol @var{name} to have a value @var{value}.  If
@var{value} is not supplied, the value is taken to be the empty
string.  The @var{value} can be any string, and the macro can be
defined to take arguments, just as if it was defined from within the
input using the @code{m4_define} statement.

For example, the following invocation defines symbol @code{COMPAT} to
have a value @code{43}:

@example
$ @kbd{mailfromd -DCOMPAT=43}
@end example

@xopindex{undefine, mailfromd, described}
@cindex U, -U @r{option, described}
@item -U @var{name}
@item --undefine=@var{name}
A counterpart of the @option{-D} option is the option @option{-U}
(@option{--undefine}).  It undefines a preprocessor symbol whose name
is given as its argument.  The following example undefines the symbol
@code{COMPAT}:

@example
$ @kbd{mailfromd -UCOMPAT}
@end example
@end table

The following two options are supplied mainly for debugging purposes:

@table @option
@xopindex{no-preprocessor, mailfromd, usage}
@item --no-preprocessor
Disables the external preprocessor.

@xopindex{preprocessor, mailfromd, usage}
@item --preprocessor[=@var{command}]
Use @var{command} as external preprocessor.  If @var{command} is not
supplied, use the default preprocessor, overriding the @code{enable}
preprocessor configuration setting.

Be especially careful with this option, because @command{mailfromd}
cannot verify whether @var{command} is actually some kind of a
preprocessor or not.
@end table

@node m4 macros
@subsection Preprocessor Macros
@anchor{defined}
@deftypefn {M4 Macro} boolean defined (@var{identifier})
The @var{identifier} must be the name of an optional abstract
argument to the function.  This macro must be used only within a function
definition.  It expands to the @acronym{MFL} expression that yields
@code{true} if the actual parameter is supplied for @var{identifier}.
For example:

@example
func rcut(string text; number num)
  returns string
do
  if (defined(num))
    return substr(text, length(text) - num)
  else
    return text
  fi
done
@end example

  This function will return last @var{num} characters of @var{text} if
@var{num} is supplied, and entire @var{text} otherwise, e.g.:

@example
@group
rcut("text string") @result{} "text string"
rcut("text string", 3) @result{} "ing"
@end group
@end example

  Invoking the @code{defined} macro with the name of a mandatory argument
yields @code{true}
@end deftypefn

@deffn {M4 Macro} printf (@var{format}, @dots{})
  Provides a @code{printf} statement, that formats its optional
parameters in accordance with @var{format} and sends the resulting
string to the current log output (@pxref{Logging and Debugging}).
@xref{String formatting}, for a description of @var{format}.

  Example usage:

@example
printf('Function %s returned %d', funcname, retcode)
@end example
@end deffn

@deftypefn {M4 Macro} string _ (@var{msgid})
A convenience macro.  Expands to a call to @code{gettext} (@pxref{NLS
Functions}).
@end deftypefn

@deffn {M4 Macro} string_list_iterate (@var{list}, @var{delim}, @var{var}, @var{code})
This macro intends to compensate for the lack of array data type
in @acronym{MFL}.  It splits the string @var{list} into segments
delimited by string @var{delim}.  For each segment, the @acronym{MFL}
code @var{code} is executed.  The code can use the variable @var{var}
to refer to the segment string.

For example, the following fragment prints names of all existing
directories listed in the @env{PATH} environment variable:

@example
string path getenv("PATH")
string seg

string_list_iterate(path, ":", seg, `
     if access(seg, F_OK)
       echo "%seg exists"
     fi')
@end example

Care should be taken to properly quote its arguments.  In the code
below the string @code{str} is treated as a comma-separated list of
values.  To avoid interpreting the comma as argument delimiter the
second argument must be quoted:

@example
string_list_iterate(str, `","', seg, `
     echo "next segment: " . seg')
@end example
@end deffn

@deffn {M4 Macro} N_ (@var{msgid})
A convenience macro, that expands to @var{msgid} verbatim.  It is
intended to mark the literal strings that should appear in the
@file{.po} file, where actual call to @code{gettext} (@pxref{NLS
Functions}) cannot be used.  For example:

@example
/* Mark the variable for translation: cannot use gettext here */
string message N_("Mail accepted")

prog envfrom
do
  @dots{}
  /* Translate and log the message */
  echo gettext(message)
@end example
@end deffn

@node Filter Script Example
@section Example of a Filter Script File

  In this section we will discuss a working example of the filter
script file.  For the ease of illustration, it is divided in several
sections.  Each section is prefaced with a comment explaining its
function.

  This filter assumes that the @file{mailfromd.conf} file contains the
following:

@example
relayed-domain-file (/etc/mail/sendmail.cw,
                     /etc/mail/relay-domains);
io-timeout 33;
database cache @{
  negative-expire-interval 1 day;
  positive-expire-interval 2 weeks;
@};
@end example

  Of course, the exact parameter settings may vary, what is important
is that they be declared.  @xref{Mailfromd Configuration}, for a
description of @command{mailfromd} configuration file syntax.

  Now, let's return to the script.  Its first part defines the
configuration settings for this host:

@example
#pragma regex +extended +icase

set mailfrom_address "<>"
set ehlo_domain "gnu.org.ua"
@end example

  The second part loads the necessary source modules:

@example
@group
require 'status'
require 'dns'
require 'rateok'
@end group
@end example

  Next we define @code{envfrom} handler.  In the first two rules, it
accepts all mails coming from the null address and from the machines
which we relay:

@example
prog envfrom
do
  if $f = ""
    accept
  elif relayed hostname($client_addr)
    accept
  elif hostname($client_addr) = $client_addr
    reject 550 5.7.7 "IP address does not resolve"
@end example

  Next rule rejects all messages coming from hosts with dynamic @acronym{IP}
addresses.  A regular expression used to catch such hosts is not 100%
fail-proof, but it tries to cover most existing host naming patterns:

@example
@group
   elif hostname($client_addr) matches
         ".*(adsl|sdsl|hdsl|ldsl|xdsl|dialin|dialup|\
ppp|dhcp|dynamic|[-.]cpe[-.]).*"
     reject 550 5.7.1 "Use your SMTP relay"
@end group
@end example

  Messages coming from the machines whose host names contain
something similar to an @acronym{IP} are subject to strict checking:

@example
   elif hostname($client_addr) matches
   ".*[0-9]@{1,3@}[-.][0-9]@{1,3@}[-.][0-9]@{1,3@}[-.][0-9]@{1,3@}.*"
     on poll host $client_addr for $f do
     when success:
       pass
     when not_found or failure:
       reject 550 5.1.0 "Sender validity not confirmed"
     when temp_failure:
       tempfail
     done
@end example

  If the sender domain is relayed by any of the @indicateurl{yahoo.com}
or @indicateurl{nameserver.com} @samp{MX}s, no checks are performed.  We
will greylist this message in @code{envrcpt} handler:

@example
@group
   elif $f mx fnmatches "*.yahoo.com"
        or $f mx fnmatches "*.namaeserver.com"
     pass
@end group
@end example

  Finally, if the message does not meet any of the above conditions,
it is verified by the standard procedure:

@example
   else
     on poll $f do
     when success:
       pass
     when not_found or failure:
       reject 550 5.1.0 "Sender validity not confirmed"
     when temp_failure:
       tempfail
     done
   fi
@end example

  At the end of the handler we check if the sender-client pair does
not exceed allowed mail sending rate:

@example
@group
   if not rateok("$f-$client_addr", interval("1 hour 30 minutes"), 100)
     tempfail 450 4.7.0 "Mail sending rate exceeded.  Try again later"
   fi
done
@end group
@end example

  Next part defines the @code{envrcpt} handler.  Its primary purpose
is to greylist messages from some domains that could not be checked
otherwise:

@example
prog envrcpt
do
  set gltime 300
  if $f mx fnmatches "*.yahoo.com"
     or $f mx fnmatches "*.namaeserver.com"
     and not dbmap("/var/run/whitelist.db", $client_addr)
    if greylist("$client_addr-$f-$rcpt_addr", gltime)
      if greylist_seconds_left = gltime
        tempfail 450 4.7.0
               "You are greylisted for %gltime seconds"
      else
        tempfail 450 4.7.0
               "Still greylisted for " .
               %greylist_seconds_left . " seconds"
      fi
    fi
  fi
done
@end example

@node Reserved Words
@section Reserved Words
@cindex reserved words
@cindex keywords

     For your reference, here is an alphabetical list of all reserved
words:

@itemize @bullet
@item __defpreproc__
@item __defstatedir__
@item __file__
@item __function__
@item __line__
@item __major__
@item __minor__
@item __module__
@item __package__
@item __patch__
@item __preproc__
@item __statedir__
@item __version__
@item accept
@item add
@item and
@item alias
@item begin
@item break
@item bye
@item case
@item catch
@item const
@item continue
@item default
@item delete
@item discard
@item do
@item done
@item echo
@item end
@item elif
@item else
@item fi
@item fnmatches
@item for
@item from
@item func
@item if
@item import
@item loop
@item matches
@item module
@item next
@item not
@item number
@item on
@item or
@item pass
@item precious
@item prog
@item public
@item reject
@item replace
@item return
@item returns
@item require
@item set
@item static
@item string
@item switch
@item tempfail
@item throw
@item try
@item vaptr
@item when
@item while
@end itemize

Several keywords are context-dependent: @code{mx} is a keyword if it
appears before @code{matches} or @code{fnmatches}.  Following strings
are keywords in @code{on} context:

@itemize
@item as
@item host
@item poll
@end itemize

The following keywords are preprocessor macros:

@itemize
@item defined
@item _ (an underscore)
@item N_
@end itemize

Any keyword beginning with a @samp{m4_} prefix is a reserved
preprocessor symbol.

@node Library
@chapter The MFL Library Functions
@include functions.texi

@node Using MFL Mode
@chapter Using the GNU Emacs MFL Mode
@cindex Emacs, @acronym{MFL} mode
@cindex GNU Emacs, @acronym{MFL} mode
@cindex @acronym{MFL} mode, GNU Emacs
  @acronym{MFL} sources are usual @acronym{ASCII} files and you may
edit them with any editor you like.  However, the best choice for this
job (as well as for many others) is, without doubt, GNU Emacs.  To ease
the work of editing script files, the @command{mailfromd} package
provides a special Emacs mode, called @dfn{@acronym{MFL} mode}.

@flindex mfl-mode.el
@flindex site-start.el
@flindex ~/.emacs
@cindex Enabling @acronym{MFL} mode
@cindex @acronym{MFL} mode, enabling
@cindex MFL mode,
  The elisp source file providing this mode, @file{mfl-mode.el}, is
installed automatically, provided that GNU Emacs is present on your
machine.  To enable the mode, add the following lines to your Emacs setup
file (either system-wide @file{site-start.el}, or your personal one,
@file{~/.emacs}):

@smalllisp
@group
(autoload 'mfl-mode "mfl-mode")
(setq auto-mode-alist (append auto-mode-alist
                       '(("\\.mfl$" . mfl-mode))))
@end group
@end smalllisp

  The first directive loads the @acronym{MFL} mode, and the second one
tells Emacs to apply it to any file whose name ends in
@samp{.mfl} suffix.

@cindex indentation, @acronym{MFL}, default
  @acronym{MFL} mode provides automatic indentation and syntax
highlighting for @acronym{MFL} sources.  The default indentation setup
is the same as the one used throughout this book:

@itemize @bullet
@item Handler and function definitions start at column 1;
@item A block statement, i.e.@: @samp{do}, @samp{done}, @samp{if},
@samp{else}, @samp{elif} and @samp{fi}, occupies a line by itself,
with the only exception that @samp{do} after an @samp{on} statement is
located on the same line with it;
@item A @samp{do} statement that follows function or handler
definition is placed in column 1.
@item Each subsequent level of nesting is indented two columns to the
right (@pxref{mfl-basic-offset}).
@item A closing statement (@samp{done}, @samp{else}, @samp{elif},
@samp{fi}) is placed at the same column as the corresponding opening
statement;
@item Branch statements (@samp{case} and @samp{when}) are placed in
the same column as their controlling keyword (@samp{switch} and
@samp{on}, correspondingly (@pxref{mfl-case-line-offset}).
@item Loop substatements (@pxref{Loops}) are offset 5 columns to the right
from the controlling @code{loop} keyword.
(@pxref{mfl-loop-statement-offset}).  Continuation statements within
loop header are offset 5 columns from the indentation of their
controlling keyword, either @code{for} or @code{while}
(@pxref{mfl-loop-continuation-offset}).
@end itemize

@cindex Finding function definition
@cindex Navigating through function definitions
  The mode provides two special commands that help navigate through
the complex filter scripts:

@table @kbd
@item C-M-a
Move to the beginning of current function or handler definition.

@item C-M-e
Move to the end of current function or handler definition.
@end table

  Here, @dfn{current function or handler} means the one within which
your cursor currently stays.

  You can use @kbd{C-M-e} repeatedly to walk through all function
and handler definitions in your script files.  Similarly, repeatedly
pressing @kbd{C-M-a} will visit all the definitions in the opposite
direction (from the last up to the very first one).

@cindex Verifying script syntax
  Another special command, @kbd{C-c C-c}, allows to verify the
syntax of your script file.  This command runs @command{mailfromd} in
syntax check mode (@pxref{Testing Filter Scripts}) and displays its
output in a secondary window, which allows to navigate through
eventual diagnostic messages and to jump to source locations
described by them.

@cindex customization, Emacs
@cindex customization, @acronym{MFL} mode
  All @acronym{MFL} mode settings are customizable.  To change any of
them, press @kbd{M-x customize} and visit @samp{Environment/Unix/Mfl}
customization group.  This group offers two subgroups: @samp{Mfl Lint
group} and @samp{Mfl Indentation group}.

@samp{Mfl Lint group} controls invocation of mailfromd by @kbd{C-c
C-c}.  This group contains two variables:

@defvr {MFL-mode setting} mfl-mailfromd-command
The @command{mailfromd} to be invoked.  By default, it is
@samp{mailfromd}.  You will have to change it, if @command{mailfromd}
cannot be found using @env{PATH} environment variable, or if you wish
to pass it some special options.  However, do not include
@option{--lint} or @option{-I} options in this variable.  The
@option{--lint} option is given automatically, and include paths are
controlled by @code{mfl-include-path} variable (see below).
@end defvr

@defvr {MFL-mode setting} mfl-include-path
A list of directories to be appended to @command{mailfromd} include
search path (@pxref{include search path}).  By default it is empty.
@end defvr

@samp{Mfl Indentation group} controls automatic indentation of
@acronym{MFL} scripts.  This group contains the following settings:

@anchor{mfl-basic-offset}
@defvr {MFL-mode setting} mfl-basic-offset
This variable sets the basic indentation increment.  It is set to 2,
by default, which corresponds to the following indentation style:

@example
prog envfrom
do
  if $f = ""
    accept
  else
    @dots{}
  fi
done
@end example
@end defvr

@anchor{mfl-case-line-offset}
@defvr {MFL-mode setting} mfl-case-line-offset
Indentation offset for @code{case} and @code{when} lines, relative to
the column of their controlling keyword.  The default is 0, i.e.:

@example
switch x
do
case 0:
  @dots{}
default:
  @dots{}
done
@end example
@end defvr

@defvr {MFL-mode setting} mfl-returns-offset
Indentation offset of @code{returns} and @code{alias} statements,
relative to the controlling @code{func} keyword.  The default value is
2, which corresponds to:

@example
@group
func foo()
  alias bar
  returns string
@end group
@end example
@end defvr

@defvr {MFL-mode setting} mfl-comment-offset
Indentation increment for multi-line comments.  The default value is
1, which makes:

@example
@group
   /* first comment line
      second comment line */
@end group
@end example
@end defvr

@anchor{mfl-loop-statement-offset}
@defvr {MFL-mode setting} mfl-loop-statement-offset
Indentation increment for parts of a @code{loop} statement.  The
default value is 5, which corresponds to the following style:

@example
@group
loop for @var{stmt},
     while @var{cond},
     @var{incr}
do
@end group
@end example
@end defvr

@anchor{mfl-loop-continuation-offset}
@defvr {MFL-mode setting} mfl-loop-continuation-offset
If any of the @code{loop} parts occupies several lines, the
indentation of continuation lines relative to the first line is
controlled by @code{mfl-loop-continuation-offset}, which defaults
to 5:

@example
@group
loop for set n 0
          set z 1,
     while n != 10
          or z != 2,
     set n n + 1
@end group
@end example
@end defvr

@node Mailfromd Configuration
@chapter Configuring @command{mailfromd}

  Upon startup, @command{mailfromd} checks if the file
@file{/etc/mailfromd.conf} exists.@footnote{The exact location is
determined at compile time: the @file{/etc} directory is the system
configuration directory set when building @command{mailfromd}
(@pxref{Building}).}  If it does, the program attempts to retrieve
its configuration settings from that file.

  The @file{mailfromd.conf} file must be written in the @dfn{GNU mailutils
configuration format}, as described in @ref{conf-syntax, Configuration
File Syntax,, mailutils, GNU Mailutils Manual}.  This format can be
summarized as follows:

@table @asis
@item Comments
Inline comments begin with @samp{//} or @samp{#} and end at the end of
the line.  Multiline comments are delimited by @samp{/*} and
@samp{*/}.  Multiline comments cannot be nested, but can contain
inline comment markers.
@item Empty lines and whitespace
Empty lines are ignored.  Whitespace characters (i.e. horizontal, vertical
space, and newline) are ignored, except as they serve to separate tokens.
@item Statements
A statement consists of a keyword and a value, separated by
whitespace.  Statements terminate with a semicolon.  E.g.

@example
pidfile /var/run/mailfromd.pid;
@end example
@item Block statements
A block statement consists of a keyword and a list of statements
enclosed in @samp{@{} and @samp{@}} characters.  Optional @dfn{label}
can appear between the keyword and opening curly brace.  E.g.:

@example
logging @{
  syslog on;
  facility mail;
@}
@end example

Block statement is not required to terminate with a semicolon,
although it is allowed to.
@item File Inclusion
The @code{include} statement causes
inclusion of the file listed as its value:

@example
include /usr/share/mailfromd/config.inc;
@end example
@end table

The @file{mailfromd.conf} file is used by all programs that form the
@samp{mailfromd} package, i.e.  @command{mailfromd},
@command{calloutd}, @command{mfdbtool}, and @command{pmult}.  Since the sets
of statements understood by each of them differ, special syntactic means are
provided to separate program-specific configurations from each other.

First of all, if the argument to @code{include} is a directory, then
the program will search that directory for a file with the same name
as the base name of the program itself.  If found, this file will be
loaded after finishing parsing the @file{mailfromd.conf} file.
Otherwise, this statement is ignored.

Secondly, the special block statement @code{program @var{tag}} is
processed only if @var{tag} matches the base name of the program being
run.  Again, it is processed after the main @file{mailfromd.conf}
file.

Thus, if you need to provide configuration for the @command{calloutd}
component, there are two ways of doing so.  First, you can place it to
a file named @file{calloutd} placed in a separate directory (say,
@file{/etc/mailfromd.d}), and use the name of that directory in a
@code{include} statement in the main configuration file:

@example
include /etc/mailfromd.d;
@end example

Secondly, you can use the @code{program} statement as follows:

@example
program calloutd @{
  ...
@}
@end example

@menu
* conf-types::        Special Configuration Data Types
* conf-base::         Base Mailfromd Configuration
* conf-preprocessor:: Preprocessor Configuration
* conf-resolver::     DNS Resolver Configuration
* conf-server::       Server Configuration
* conf-milter::       Milter Connection Configuration
* conf-debug::        Logging and Debugging configuration
* conf-timeout::      Timeout Configuration
* conf-callout::      Call-out Configuration
* conf-priv::         Privilege Configuration
* conf-database::     Database Configuration
* conf-runtime::      Runtime Constants
* conf-mailutils::    Standard Mailutils Statements
@end menu

@node conf-types
@section Special Configuration Data Types

  In addition to the usual data types (@pxref{Statements,,,
mailutils, GNU Mailutils Manual}), @command{mailfromd} configuration
introduces the following two special ones:

@table @asis
@item time interval specification
@cindex Time Interval Specification
@anchor{time interval specification}
  The @dfn{time interval specification} is a string that defines an
interval, much the same way we do this in English: it consists of one
or more pairs @samp{number}-@samp{time unit}.  For example, the
following are valid interval specifications:

@example
@group
1 hour
2 hours 35 seconds
1 year 7 months 2 weeks 2 days 11 hours 12 seconds
@end group
@end example

@noindent
The pairs can occur in any order, however unusual it may sound to a
human ear, e.g.@: @samp{2 days 1 year}.  If the @samp{time unit} is
omitted, seconds are supposed.

@item Connection URL
@anchor{milter port specification}
@table @asis
@item unix://@var{file}
@itemx unix:@var{file}
@itemx local://@var{file}
@itemx local:@var{file}
  A named pipe (socket).

@item inet://@var{address}:@var{port}
@itemx inet:@var{port}@@@var{address}
  An @acronym{IP}v4 connection to host @var{address} at port @var{port}.  Port
must be specified either as a decimal number or as a string representing
the port name in @file{/etc/services}.

@item inet6:@var{port}@@@var{address}
An @acronym{IP}v6 connection to host @var{address} at port @var{port}.  This
port type is not yet supported.
@end table
@end table

@node conf-base
@section Base Mailfromd Configuration

@deffn {Mailfromd Conf} script-file file
Read filter script from @var{file}.  By default it is
@file{@var{sysconfdir}/mailfromd.mfl}.
@end deffn

@deffn {Mailfromd Conf} setvar name value
Initialize @acronym{MFL} variable @var{name} to @var{value}.
@end deffn

@deffn {Mailfromd Conf} include-path path
Add directories to the list of directories to be searched for include
files.  @xref{include search path}.

Argument is a list of directory names separated by colons.  These
names are added to the search path before the default search path
directories, but after any directories added by the previous
@code{include-path} statements, if any.

@xref{--include-path}
@end deffn

@deffn {Mailfromd Conf} module-path path
Add directories to the module search path  @xref{module search path}.

Argument is a list of directory names separated by colons.  These
names are added to the search path before the default search path
directories, but after any directories added by the previous
@code{module-path} statements, if any.

@xref{--module-path}
@end deffn

@deffn {Mailfromd Conf} state-directory dir
Set program state directory.  @xref{statedir}.
@end deffn

@deffn {Mailfromd Conf} relayed-domain-file file
Append domain names from the named @var{file} to the list of relayed
domains.  This list can be inspected from @acronym{MFL} script using
the @samp{relayed} function (@pxref{relayed}).

The @var{file} argument is either a single file name or a
list of file names, e.g.:

@example
relayed-domain-file /etc/mail/sendmail.cw;
relayed-domain-file (/etc/mail/sendmail.cw, /etc/mail/relay-domains);
@end example
@end deffn

@deffn {Mailfromd Conf} source-ip ipaddr
Set source IP address for outgoing TCP connections.
@end deffn

@deffn {Mailfromd Conf} pidfile file
Set the name of the file to store PID value in.  The file must be
writable for the user or group @command{mailfromd} runs as
(@pxref{conf-priv}).
@end deffn

@node conf-preprocessor
@section Preprocessor Configuration
@kwindex preprocessor

MFL preprocessor is configured using the @code{preprocessor}
statement (default values shown):

@example
@group
preprocessor @{
  # Enable preprocessor
  enable yes;
  # Preprocessor command line stub.
  command "m4 -s";
  # Pass current include path to the preprocessor via -I options.
  pass-includes false;
  # Pass to the preprocessor the feature definitions via -D options
  # as well as any -D/-U options from the command line
  pass-defines true;
  # Name of the preprocessor setup file.  Unless absolute, it is
  # looked up in the include path.
  setup-file "pp-setup";
@}
@end group
@end example

Its substatements are:

@deffn {preprocessor} enable @var{bool}
Enable or disable preprocessor.  If disabled in configuration file,
preprocessor can be forcefully enabled in the command line by using
the @option{--preprocessor} option.
@end deffn

@deffn {preprocessor} command @var{string}
The stub for the preprocessor command line.  Before use, it will be
eventually augmented by @option{-I} and @option{-D} options, depending
on the @code{pass-includes} and @code{pass-defines} settings in this
section.  This is described in detail in @ref{Configuring Preprocessor}.
@end deffn

@deffn {preprocessor} pass-includes @var{bool}
If @code{true} the directories from include search path
(@pxref{include search path} will be passed to the preprocessor via
the @option{-I} options.
@end deffn

@deffn {preprocessor} pass-defines @var{bool}
If @code{true}, the -D options defining optional features will be
passed to the preprocessor (e.g. @option{-DWITH_DKIM}), as well as any
@option{-D} and @option{-U} options from the @command{mailfromd} command line.
@end deffn

@deffn {preprocessor} setup-file @var{string}
Name of the preprocessor setup file (@pxref{Preprocessor}). Unless
@var{string} begins with a slash, the file with this name is looked up
in the current include search path (@pxref{include search path}).  If
found, its absolute name is passed to the preprocessor as its first
argument.

If @var{string} begins with a slash, it is passed to the preprocessor
as is.
@end deffn

@node conf-resolver
@section DNS Resolver Configuration
@kwindex resolver
  DNS resolver settings are configured using the @code{resolver}
compound statement:

@example
@group
resolver @{
  config @var{filename};
  max-cname-chain @var{num};
@}
@end group
@end example

@deffn {resolver} config @var{filename}
@findex /etc/resolv.conf
Name of the resolver configuration file to use, instead of the default
@file{/etc/resolv.conf}.
@end deffn

@anchor{CNAME chains}
@deffn {resolver} max-cname-chain @var{num}
@cindex CNAME chains
Maximum allowed length of a DNS @dfn{CNAME chain} that will be
followed.  A @dfn{CNAME chain} is a sequence of @code{CNAME} records
pointing to another @code{CNAME}s.  Although CNAME chains are not
considered a good practice, many sites still use them.  By default the
@command{mailfromd} resolver allows at most one CNAME record pointing
to a CNAME (this corresponds to @code{max-cname-chain 2}).  If you
need to follow longer chains, raise this value.  Note however, that
using values greater than 5 is not a good idea, anyway.
@end deffn

@node conf-server
@section Server Configuration
  A single @command{mailfromd} daemon can run several @dfn{servers}.
These are configured in the following statement:

@example
@group
server @var{type} @{
  id @var{name};
  listen @var{url};
  backlog @var{num};
  max-instances @var{num};
  single-process @var{bool};
  reuseaddr @var{bool};
  option @var{list};
  default @var{bool};
  callout @var{url};
  acl @{ @dots{} @}
@}
@end group
@end example

@deffn {Mailfromd Conf} server @var{type}
  Define a server.  The @var{type} is either @samp{milter} or
@samp{callout}.  @xref{SMTP Timeouts}, for a description of various
types of servers.

  The substatements in the @code{server} block provide parameters for
configuring this server.
@end deffn

@deffn {server} id name
  Assign an identifier to this server.  This identifier is used as
a suffix to syslog tag (@pxref{syslog tag}) in messages related to
this server.  For example, if a server block had the following
statement in it:

@example
id main;
@end example

@noindent
then all messages related to this server will be marked with tag
@samp{mailfromd#main}.

  The part before the @samp{#} is set using the @code{tag} statement
in @code{logging} block (@pxref{Logging Statement, Mailutils
Configuration File,, mailutils, GNU Mailutils Manual}).

  This identifier can be inspected from the MFL code using the
@code{milter_server_id} variable (@pxref{milter_server_id}).
@end deffn

@deffn {server} listen url
Listen for connections on the given @acronym{URL}.
@xref{milter port specification}, for a description of allowed
@var{url} formats.

Example:

@example
listen inet://10.10.10.1:3331;
@end example

The MFL code can access its address using the
@code{milter_server_address} and @code{milter_server_family} variables.
@end deffn

@deffn {server} backlog @var{num}
Configures the size of the queue of pending connections.  Default
value is 8.
@end deffn

@deffn {server} max-instances number
Sets the maximum number of instances allowed for this server.
@end deffn

@deffn {server} single-process bool
When set to @samp{yes}, this server will run in @dfn{single-process}
mode, i.e. it will not fork sub-processes to serve requests.  This
option is meant exclusively to assist in debugging
@command{mailfromd}.  Don't use it for anything else but for
debugging!
@end deffn

@deffn {server} reuseaddr bool
When set to @samp{yes}, @command{mailfromd} will attempt to reuse
existing socket addresses.  This is the default behavior.
@end deffn

  If the server @var{type} is @samp{callout}, the following statement
is also allowed:

@deffn {server} option @var{list}
Configures server options.  As of version @value{VERSION} only one
option is defined:

@table @asis
@item default
Mark this server as the default one.  This means it will be used by
every milter server that doesn't define the @code{callout-url}
statement.
@end table
@end deffn

@deffn {server} default bool
When set to @samp{yes}, this server is marked as a default callout server
for all milter servers declared in the configuration.  This is
equivalent to @code{option default}.
@end deffn

  if the server @var{type} is @samp{milter}, you can use the following
statement to query a remote callout server:

@anchor{configuring default callout server}
@deffn {server} callout url
  Use a callout server at @var{url} (@pxref{milter port specification}).
@end deffn

  You can also set a @dfn{global callout server}, which will be used
by all milter servers that do not set the @code{callout} statement:

@deffn {Mailfromd Conf} callout-url url
  Set global callout server.  @xref{milter port specification}, for
allowed @var{url} formats.
@end deffn

@node conf-milter
@section Milter Connection Configuration

@deffn {Mailfromd Conf} milter-timeout time
  Sets the timeout value for connection between the filter
and the @acronym{MTA}.  Default value is @value{GACOPYZ_TIMEOUT} seconds.  You
normally do not need to change this value.
@end deffn

@deffn {Mailfromd Conf} acl
  This block statement configures @dfn{access control list} for
incoming Milter connections.  @xref{ACL Statement, Mailutils
Configuration File,, mailutils, GNU Mailutils Manual}, for a
description of its syntax.  E.g.:

@example
acl @{
  allow from 10.10.10.0/24;
  deny from any;
@}
@end example
@end deffn

@node conf-debug
@section Logging and Debugging configuration

@deffn {Mailfromd Conf} logger mech
  Set default logger mechanism.  Allowed values for
@var{mech} are:

@table @asis
@item stderr
Log everything to the standard error.

@item syslog
Log to syslog.

@item syslog:async
Log to syslog using the asynchronous syslog implementation.
@end table

@xref{Logging and Debugging}, for a detailed discussion.
See also @ref{syslog-async, Using non-blocking syslog}, for
information on how to set default syslog implementation at compile time.
@end deffn

@deffn {Mailfromd Conf} debug spec
  Set @command{mailfromd} debug verbosity level.  The @var{spec}
must be a valid debugging level specification (@pxref{debugging level specification}).
@end deffn

@deffn {Mailfromd Conf} stack-trace bool
  Enables stack trace dumps on runtime errors.  This feature is
useful for locating the source of an error, especially in complex
scripts.  @xref{tracing runtime errors}, for a detailed description.
@end deffn

@deffn {Mailfromd Conf} trace-actions bool
  Enable action tracing.  If @var{bool} is @samp{true},
@command{mailfromd} will log all executed actions.  @xref{Logging and
Debugging}, for a detailed description of action tracing.
@end deffn

@deffn {Mailfromd Conf} trace-program modlist
  Enable program instruction tracing for modules in @var{modlist},
a comma-separated list of source code modules, e.g.:

@example
trace-program (bi_io,bi_db);
@end example

This statement enables tracing for functions from modules
@file{bi_io.c} and @file{bi_db.c} (notice, that you need not give file
suffixes).

  This tracing is useful for debugging @command{mailfromd}, but is not
advised to use otherwise, since it is very time-costly.
@end deffn

@deffn {Mailfromd Conf} transcript bool
Enable transcripts of call-out @acronym{SMTP} sessions.
@xref{SMTP transcript}, for a detailed description of
@acronym{SMTP} transcripts.
@end deffn

@node conf-timeout
@section Timeout Configuration
  The @acronym{SMTP} timeouts used in callout sessions are configured via
@code{smtp-timeout} statement:

@subheading Syntax
@kwindex smtp-timeout
@example
@group
smtp-timeout @var{type} @{
  connection @var{interval};
  initial-response @var{interval};
  helo @var{interval};
  mail @var{interval};
  rcpt @var{interval};
  rset @var{interval};
  quit @var{interval};
@}
@end group
@end example

@deffn {Mailfromd Conf} smtp-timeout @var{type}
Declare @acronym{SMTP} timeouts of the given @var{type}, which may be
@samp{soft} or @samp{hard}.

Callout @acronym{SMTP} sessions initiated by polling functions, are
controlled by two sets of timeouts: @samp{soft} and @samp{hard}.
@dfn{Soft timeouts} are used by the mailfromd milter servers.  @dfn{Hard
timeouts} are used by callout servers (@pxref{callout server}).
When a soft timeout is exceeded, the calling procedure is
delivered an @samp{e_temp_failure} exception and the session is
scheduled for processing by a callout server.  The latter re-runs the
session using hard timeouts.  If a hard timeout is exceeded, the
address is marked as @samp{not_found} and is stored in the cache
database with that status.

Normally, soft timeouts are set to shorter values, suitable for use
in @acronym{MFL} scripts without causing excessive delays.  Hard
timeouts are set to large values, as requested by RFC 2822 and
guarantee obtaining a definite answer (see below for the default
values).
@end deffn

@subheading Statements
  The @var{time} argument for all @code{smtp-timeout} sub-statements
is expressed in time interval units, as described in @ref{time
interval specification}.

@deffn {smtp-timeout} connection time
  Sets initial connection timeout for callout tests.  If the
connection is not established within this time, the corresponding
callout function returns temporary failure.
@end deffn

@deffn {smtp-timeout} initial-response time
  Sets the time to wait for the initial @acronym{SMTP} response.
@end deffn

@deffn {smtp-timeout} helo time
  Timeout for a response to @samp{HELO} (or @samp{EHLO}) command.
@end deffn

@deffn {smtp-timeout} mail time
  Timeout for a response to @samp{MAIL} command.
@end deffn

@deffn {smtp-timeout} rcpt time
  Timeout for a response to @samp{RCPT} command.
@end deffn

@deffn {smtp-timeout} rset time
  Timeout for a response to @samp{RSET} command.
@end deffn

@deffn {smtp-timeout} quit time
   Timeout for a response to @samp{QUIT} command.
@end deffn

@subheading Default Values
The default timeout settings are:

@float Table, smtp-timeouts
@caption{Default SMTP timeouts}
@multitable @columnfractions 0.5 0.25 0.25
@headitem Timeout      @tab Soft @tab Hard
@item connection       @tab  10s @tab 5m
@item initial-response @tab  30s @tab 5m
@item helo             @tab  I/O @tab 5m
@item mail             @tab  I/O @tab 10m
@item rcpt             @tab  I/O @tab 5m
@item rset             @tab  I/O @tab 5m
@item quit             @tab  I/O @tab 2m
@end multitable
@end float

@deffn {Mailfromd Conf} io-timeout time
  Sets a general @acronym{SMTP I/O} operation timeout.  This timeout
is used as the default for entries marked with @samp{I/O} in the above
table.  The default is 3 seconds.
@end deffn

@node conf-callout
@section Call-out Configuration

@deffn {Mailfromd Conf} ehlo-domain string
  Sets default domain used in @samp{EHLO} (or @samp{HELO})
@acronym{SMTP} command when probing the remote host.  This value can be
overridden by @samp{from} parameter to @command{poll} command
(@pxref{poll}).

  This statement assigns the value @var{string} to the
@samp{ehlo_domain} variable (@pxref{ehlo_domain}), and is therefore
equivalent to

@example
setvar ehlo_domain @var{string};
@end example
@end deffn

@deffn {Mailfromd Conf} mail-from-address string
  Sets default email addresses used in @samp{MAIL FROM:}
@acronym{SMTP} command when probing the remote host.  This value can be
overridden by @samp{as} parameter to @command{poll} command
(@pxref{poll}).

  This statement assigns the value @var{string} to the
@samp{mailfrom_address} variable (@pxref{mailfrom_address}), and is therefore
equivalent to

@example
setvar mailfrom_address @var{string};
@end example
@end deffn

@deffn {Mailfromd Conf} enable-vrfy bool
@cindex VRFY, SMTP statement
  Enables the use of SMTP VRFY statement prior to normal callout
sequence.  If VRFY is supported by the remote server,
@command{mailfromd} relies on its reply and does not perform normal
callout.

  The use of this statement is not recommended, because many existing
VRFY implementations always return affirmative result, no matter is
the requested email handled by the server or not.

  The default is @code{enable-vrfy no}, i.e.@: VRFY is disabled.
@end deffn

@deffn {Mailfromd Conf} smtp-starttls string
@cindex STARTTLS in SMTP callout sessions
@cindex TLS support
Configures whether to issue the @code{STARTTLS} command if the mail
server offers such capability.  Allowed values are:

@defvr {smtp-starttls value} never
Never use @code{STARTTLS}.
@end defvr

@defvr {smtp-starttls value} always
Always use @code{STARTTLS} if supported by the server.
@end defvr

@defvr {smtp-starttls value} ondemand
Use @code{STARTTLS} only if @code{MAIL FROM:} command failed with the code
530 (Authorization required).
@end defvr

Default is @samp{ondemand}.

Notice, that the @code{smtp-starttls} feature depends on whether
GnuTLS support is available in @code{libmailutils}.  You can check
whether it is available by inspecting the output of @code{mailfromd
--show-defaults} (@pxref{Examining Defaults}): if so, the
@samp{optional features} line will contain the word @samp{STARTTLS}.

@end deffn

@deffn {Mailfromd Conf} tls @{ ... @}
Configures TLS settings for the callout.  This is a compound
statement.  The two most important statements in this compound are:

@deffn {Mailfromd TLS} ssl-priorities @var{string}
Configures the TLS session's handshake algorithms and options in a
compact, easy-to-use format.  @xref{Priority strings,,,GnuTLS,GnuTLS},
for a detailed description of the priority string format.

Default value is @samp{NORMAL:%COMPAT}.  You may need to adjust it in
order to work with older or misconfigured servers, e.g.:

@example
@group
tls @{
  ssl-priorities "LEGACY:%COMPAT";
@}
@end group
@end example
@end deffn

@deffn {Mailfromd TLS} handshake-timeout @var{n}
Sets the timeout for TLS handshake to @var{n} seconds.
@end deffn

The remaining three statements are not of much importance for
callout.  They are described here for completeness sake:

@deffn {Mailfromd TLS} ssl-ca-file file
Specifies the pathname of the certificate authority file.
@end deffn

@deffn {Mailfromd TLS} ssl-certificate-file file
Specifies the pathname of the certificate file.
@end deffn

@deffn {Mailfromd TLS} ssl-key-file file
Specifies the pathname of the certificate key file.
@end deffn
@end deffn

@node conf-priv
@section Privilege Configuration

@deffn {Mailfromd Conf} user name
  Switch to this user's privileges after startup.
@xref{Starting and Stopping}, for a discussion of
the privileges @command{mailfromd} runs under and the options that
affect them.  See also @code{group} below.
@end deffn

@deffn {Mailfromd Conf} group name
  Retain the supplementary group @var{name} when switching to user
privileges.  By default @command{mailfromd} clears the list of
supplementary groups when switching to user privileges, but this statement
allows to retain the given group.  It can be specified multiple times
to retain several groups.  This option may be necessary to
maintain proper access rights for various files.  @xref{Starting and
Stopping}.
@end deffn

@node conf-database
@section Database Configuration

@subheading Syntax
@kwindex database
@example
database @var{dbname} @{
  file @var{name};
  enable @var{bool};
  expire-interval @var{interval};
  positive-expire-interval @var{interval};
  negative-expire-interval @var{interval};
@}
@end example

@deffn {Mailfromd Conf} database dbname
  The @code{database} statement controls run-time parameters of
the @acronym{DBM} database identified by @var{dbname}.  Allowed
values for the latter are:

@table @code
@item cache
Callout cache database.

@item rate
Sending rate database.  @xref{Sending Rate}.

@item tbf
Token-bucket filter database.  @xref{TBF}.

@item greylist
Greylisting database.  @xref{Greylisting}.
@end table
@end deffn

@subheading Statements

@deffn {database} file name
@cindex DBM scheme
@anchor{DBM scheme}
Set the database file name.  The file name can be prefixed with the
@dfn{DBM scheme}, which indicates the desired DBM implementation to
use.  The scheme consists of DBM type name followed by a colon and
two slashes.  To obtain the list of available DBM types,
run @command{mailfromd --show-defaults} (@pxref{Examining Defaults})
and examine the @samp{supported database types:} line, e.g.:

@example
@group
$ @kbd{mailfromd --show-defaults | grep '^supported database types:'}
supported database types:gdbm, bdb
@end group
@end example

@cindex default database type
@cindex DBM implementation, default
If no DBM scheme is supplied, the default DBM implementation will be
used.  The default implementation is set up on compile time and
can be inspected using the following command:

@example
mailfromd --show-defaults | grep '^default database type:'
@end example

The part that follows DBM scheme can be either absolute or relative
file name.  Absolute file name (begining with @samp{/}) is used
verbatim.  Relative file name (i.e. the name that begins with any
character except @samp{/}) is modified by prepending it with the
mailfromd local state directory (@pxref{statedir}).

To illustrate the above, suppose that the local state directory is
@file{/var/mailfromd} and default DBM implementation is @code{gdbm}.  Then:

@multitable @columnfractions 0.3 0.3 0.2
@headitem Value of @code{file} @tab Resulting filename @tab DBM type
@item @code{tbf.db} @tab /var/mailfromd/tbf.db @tab GDBM
@item @code{bdb://tbf.db} @tab /var/mailfromd/tbf.db @tab Berkeley DB
@item @code{/run/tbf.db} @tab /run/tbf.db @tab GDBM
@item @code{bdb:///run/tbf.db} @tab /run/tbf.db @tab Berkeley DB
@end multitable
@end deffn

@deffn {database} enable bool
Enable or disable this database.
@end deffn

@deffn {database} expire-interval time
@anchor{expire-interval-conf}
  Set the expiration interval for this database @var{dbname}.
@xref{time interval specification}, for a description of @var{time}
format.

@end deffn

@deffn {database} positive-expire-interval time
  This statement is valid only for @samp{cache} database.  It sets
the expiration interval for positive (@samp{success}) cache entries.
@end deffn

@deffn {database} negative-expire-interval time
  This statement is valid only for @samp{cache} database, where
  it sets expiration interval for negative (@samp{not_found}) cache entries.
@end deffn

@subheading Additional Statements

@deffn {Mailfromd Conf} database-type @var{type}
Set default database type.  @var{type} is one of the database types
supported by mailutils (i.e., for Mailutils 3.0: @samp{gdbm},
@samp{ndbm}, @samp{bdb}, @samp{kc}, and @samp{tc}).  Run

@example
mailfromd --show-defaults | grep 'supported databases:'
@end example
@noindent

to get a list of type names supported by your build of
@command{mailfromd} (@pxref{Examining Defaults}).
@end deffn

@deffn {Mailfromd Conf} database-mode @var{mode}
Defines file mode for newly created database files.  @var{mode} must
be a valid file mode in octal.
@end deffn

@node conf-runtime
@section Runtime Constants Configuration

@deffn {Mailfromd Conf} runtime @{ statements @}
The statements in the @code{runtime} section configure various values
used by @acronym{MFL} builtin functions.
@end deffn

@deffn {runtime} max-streams number
Sets the maximum number of stream descriptors that can be opened
simultaneously.  Default is @value{MAX_IOSTREAMS}.  @xref{I/O functions}.
@end deffn

@deffn {runtime} max-open-mailboxes number
Sets the maximum number of available mailbox descriptors.  This value
is used by @acronym{MFL} mailbox functions (@pxref{Mailbox functions}).
@end deffn

@deffn {runtime} max-open-messages number
Sets the maximum number of messages that can be opened simultaneously
using the @code{mailbox_get_message} function.  @xref{Message
functions}, for details.
@end deffn

@deffn {runtime} max-mfmods number
Maximum number of dynamically loaded modules (@dfn{mfmod}s) that can
be used simultaneously.  @xref{mfmod}.
@end deffn

@deffn {runtime} mfmod-path string
@anchor{mfmod-path}
Search path for dynamically loaded modules -- a colon-delimited list
of directories where to look for libraries loaded using the
@code{dlopen} function (@pxref{dlopen}).  Default @code{mfmod-path}
value is @code{@var{prefix}/mailfromd/lib}, where @var{prefix} is the
installation prefix directory.  @xref{mfmod}, for details.
@end deffn

@node conf-mailutils
@section Standard Mailutils Statements

The following standard Mailutils statements are understood:

@multitable @columnfractions 0.3 0.6
@headitem Statement @tab Reference
@item auth @tab @xref{auth statement, Mailutils Configuration File,,
mailutils, GNU Mailutils Manual}.
@item debug @tab @xref{debug statement, Mailutils Configuration File,,
mailutils, GNU Mailutils Manual}.
@item include @tab @xref{include, Mailutils Configuration File,,
mailutils, GNU Mailutils Manual}.
@item logging @tab @xref{logging statement, Mailutils Configuration File,,
mailutils, GNU Mailutils Manual}.
@item mailer @tab @xref{mailer statement, Mailutils Configuration File,,
mailutils, GNU Mailutils Manual}.
@item locking @tab @xref{locking statement, Mailutils Configuration File,,
mailutils, GNU Mailutils Manual}.
@end multitable

@node Invocation
@chapter @command{Mailfromd} Command Line Syntax
@cindex invocation
@cindex command line, @command{mailfromd} invocation syntax
  The @command{mailfromd} binary is started from the command line
using the following syntax (brackets indicate optional parts):

@example
$ @kbd{mailfromd [@var{options}] [@var{asgn}] [@file{@var{script}}]}
@end example

@noindent
The meaning of each invocation part is described in the table below:

@table @var
@item options
  The command line options (@pxref{options}).

@item asgn
  Sendmail macro assignments.  These are currently meaningful only with
the @option{--test} option (@pxref{test mode}), but this
may change in the future.  Each assignment has the form:

@example
@var{var}=@var{value}
@end example

@noindent
where @var{var} is the name of a Sendmail macro and @var{value} is the
value to be assigned to it.

@item script
  The file name of the filter script, if other than the default one.
@end table

@menu
* options::                     Command Line Options.
* Starting and Stopping::       How to Start and Shut Down the Daemon.
@end menu

@node options
@section Command Line Options.

@menu
* Operation Modifiers::
* General Settings::
* Preprocessor Options::
* Timeout Control::
* Logging and Debugging Options::
* Informational Options::
@end menu

@node Operation Modifiers
@subsection Operation Modifiers

@table @option
@opsummary{daemon, mailfromd}
@item --daemon
Run in daemon mode (default).

@opsummary{run, mailfromd}
@item --run[=@var{start}]
Load the script named in the command line and execute the function
named @var{start}, or @samp{main}, if @var{start} is not given.

The full invocation syntax for this mode is:

@example
mailfromd [@var{options}] --run[=@var{start} \
          [@var{macro}=@var{value}] [@var{file} @var{args...}]
@end example

Its parts are:

@table @var
@item options
Any mailfromd options needed to tune its behavior.

@item start
Start symbol.  Defaults to @code{main}.

@item @var{macro}=@var{value}
Any number of Sendmail macro definitions.  @var{macro} is the macro
name and @var{value} is the value to initialize it to.

@item file
Script file name.  If omitted, default script file will be used
(@pxref{default script file}).

@item args...
Command line arguments that will be pased to the @var{start} MFL
function as parameters.
@end table

The @var{start} function must be defined as:

@example
func @var{start} (...) returns number
@end example

@xref{Run Mode}, for a detailed discussion of this feature.

@opsummary{test, mailfromd}
@item -t[@var{state}]
@itemx --test[=@var{state}]
Run in test mode.  @xref{Testing Filter Scripts}.  Default @var{state} is
@samp{envfrom}.  This option implies @option{--stderr} (@pxref{--stderr}).

@end table

@ignore
------------------------------------------------------------------------
 -- This is to pacify make check-docs.
 -- These options are reserved for future use

@opindex load, --load, mailfromd option
@opindex load-dir, --load-dir, mailfromd option
@opindex compile, --compile, mailfromd option
------------------------------------------------------------------------
@end ignore

@node General Settings
@subsection General Settings

@table @option
@opsummary{callout-socket, mailfromd}
@item --callout-socket=@var{string}
Set socket for the default callout server.
This is mainly useful together with the @option{--mtasim} option.

@opsummary{echo, mailfromd}
@anchor{echo option}
@item --echo
@itemx --echo=@var{file}
This option controls where the output of the @code{echo} statement
(@pxref{Echo}) goes when @command{mailfromd} is run in test mode,
i.e. using the @option{--test} or @option{--run} options
(@pxref{Testing Filter Scripts}).

When used without argument it directs the @code{echo} output to the
standard output stream.  If an argument is supplied (second form
above), the output goes to the named file.  The file will be created
if it doesn't exist.  Notice, that the use of @samp{=} is compulsory
when specifying an argument.  A single dash as a filename means
standard output, so that @option{--echo=-} and @option{--echo} are
equivalent. 

@opsummary{foreground, mailfromd}
@item --foreground
Stay in foreground.  When given this option, @command{mailfromd} will
not disconnect itself from the controlling terminal and will run in
the foreground.

@opsummary{group, mailfromd}
@item -g @var{name}
@itemx --group=@var{name}

Retain the group @var{name} when switching to user
privileges.  @xref{Starting and Stopping}.

This option complements the @code{group} configuration statement
(@pxref{conf-priv, group}).

@opsummary{include-path, mailfromd}
@item --include-path=@var{dir}
@itemx -I @var{dir}
Add the directory @var{dir} to the list of directories to be searched
for include files (@pxref{include search path}).  This setting affects
the @code{#include} statement (@pxref{include}) and is used to locate
preprocessor setup file (@pxref{Preprocessor}).  If enabled in the
configuration (@pxref{conf-preprocessor, pass-includes}), directories
from the include search path will also be added to the preprocessor
command line as additional @option{-I} options.

@opsummary{module-path, mailfromd}
@item --module-path=@var{dir}
@itemx -P @var{dir}
Add the directory @var{dir} to the list of directories searched for
MFL module files (@pxref{module search path}).  This setting affects
the @code{require} and @code{import} MFL statements.  @xref{Modules}.

@opsummary{mailer, mailfromd}
@item --mailer=@var{url}
@itemx -M @var{url}
Set the @acronym{URL} of the mailer to use.  @xref{Mail Sending
Functions}.

@opsummary{mtasim, mailfromd}
@item --mtasim
This option is reserved for use by @command{mtasim} (@pxref{mtasim}).

@opsummary{optimize, mailfromd}
@item -O[@var{level}]
@itemx --optimize[=@var{level}]
Set optimization level for code generator.  Two levels are
implemented: @samp{0}, meaning no optimization, and @samp{1}, meaning
full optimization.

@opsummary{port, mailfromd}
@opsummary{milter-socket, mailfromd}
@item -p @var{string}
@itemx --port=@var{string}
@itemx --milter-socket=@var{string}
Set communication socket.  Overrides the @code{listen} configuration
statement, which you are advised to use instead (@pxref{conf-server, listen}).

@opsummary{pidfile, mailfromd}
@item --pidfile=@var{file}
Set pidfile name.  Overrides the @code{pidfile} configuration
statement, which you are advised to use instead (@pxref{conf-base, pidfile}).

@opsummary{relayed-domain-file, mailfromd}
@item --relayed-domain-file=@var{file}
@anchor{option relay}
Read relayed domains from @var{file}.  Overrides the
@code{relayed-domain-file} configuration statement (@pxref{conf-base,
relayed-domain-file}), which you are advised to use instead.
@xref{Avoiding Verification Loops}, and the description of
@code{relayed} function (@pxref{relayed}) for more information.

@opsummary{resolv-conf-file, mailfromd}
@item --resolv-conf-file=@var{file}
Read resolver settings from @var{file}, instead of the default
@file{/etc/resolv.conf}.

@opsummary{state-directory, mailfromd}
@item --state-directory=@var{dir}
Set new program state directory.  @xref{statedir, Local state directory}, for
the description of this directory and its purposes.  This option
overrides the settings of @code{state-directory} configuration statement,
described in @ref{conf-base, state-directory}.

@opsummary{source-ip, mailfromd}
@item -S @var{ip}
@itemx --source-ip=@var{ip}
Set source address for @acronym{TCP} connections.  Overrides the
@samp{source-ip} configuration statement, which you are advised to use
instead (@pxref{conf-base, source-ip}).

@opsummary{user, mailfromd}
@item -u @var{name}
@itemx  --user @var{name}
Switch to this user privileges after startup.  Overrides the @code{user}
configuration file statement, which you are advised to use instead
(@pxref{conf-priv, user}).  Default user is @samp{@value{DEFAULT_USER}}.

@opsummary{variable, mailfromd}
@item -v @var{var}=@var{value}
@itemx --variable @var{var}=@var{value}
Assign @var{value} to the global variable @var{var}.  The variable
must be declared in your startup script.  @xref{overriding initial
values}, for a detailed discussion of this option.

@ignore
-- Reserved for future use
@opindex compile
@end ignore
@end table

@node Preprocessor Options
@subsection Preprocessor Options

Following command line options control the preprocessor
feature.  @xref{Preprocessor}, for a detailed discussion of these.

@table @option
@opsummary{no-preprocessor, mailfromd}
@item --no-preprocessor
Do not run the preprocessor.

@opsummary{preprocessor, mailfromd}
@item --preprocessor[=@var{command}]
If @var{command} is supplied, use it as the external preprocessor
instead of the default.  This overrides the
@code{preprocessor.command} setting (@pxref{conf-preprocessor}).

If used without arguments, forces the use of the configured
preprocessor.  This form is intended as a way to manually enable the
use of the preprocessor if disabled in the configuration.

@opsummary{define, mailfromd}
@cindex D, -D @r{option, summary}
@item -D @var{name}[=@var{value}]
@itemx --define=@var{name}[=@var{value}]
Define a preprocessor symbol @var{name} to have a value @var{value}.
This option is ignored if the @code{preprocessor.pass-defines}
configuration setting is @code{false}.

@opsummary{undefine, mailfromd}
@cindex U, -U @r{option, summary}
@item -U @var{name}
@itemx --undefine=@var{name}
Undefine the preprocessor symbol @var{name}.
This option is ignored if the @code{preprocessor.pass-defines}
configuration setting is @code{false}.

@cindex E, -E @r{option, summary}
@item -E
Stop after the preprocessing stage; do not run the compiler proper.
The output is in the form of preprocessed source code, which is sent
to the standard output.
@end table

@node Timeout Control
@subsection Timeout Control

@xref{time interval specification}, for information on @var{interval} format.

@table @option
@opsummary{milter-timeout, mailfromd}
@item --milter-timeout=@var{interval}
Set @acronym{MTA} connection timeout.  Overrides @code{milter-timeout}
statement in the @command{mailfromd} configuration file, which you are
advised to use instead (@pxref{conf-milter, milter-timeout}).
@end table

@node Logging and Debugging Options
@subsection Logging and Debugging Options

@table @option
@opsummary{location-column, mailfromd}
@item --location-column
@itemx --no-location-column
Mention column number in error messages.  @xref{Testing Filter
Scripts,, location-column}.  Use @option{--no-location-column} to
disable

@opsummary{debug, mailfromd}
@item -d @var{string}
@itemx --debug=@var{string}
Set debugging level.  @xref{Logging and Debugging}.

@opsummary{dump-code, mailfromd}
@item --dump-code
Parse and compile the script file and dump the disassembled
listing of the produced code to the terminal.  @xref{Logging and Debugging}.

@opsummary{dump-grammar-trace, mailfromd}
@item --dump-grammar-trace
Enable debugging the script file parser.  While parsing the
file, the detailed dump of the parser states and tokens seen will be
output.

@opsummary{dump-lex-trace, mailfromd}
@item --dump-lex-trace
Enable debugging the lexical analyzer.  While parsing the
script file, the detailed dump of the lexer states and matched
rules will be output.

@opsummary{dump-macros, mailfromd}
@item --dump-macros
Show Sendmail macros used in the script file.  The macro names
are displayed as comma-separated lists, grouped by handler names.
@xref{Sendmail}, for a detailed description of this
option and its usage.

@opsummary{dump-tree, mailfromd}
@item --dump-tree
Parse and compile the script file and dump the parse tree in a
printable form to the terminal.

@opsummary{dump-xref, mailfromd}
@item --dump-xref
Print a cross-reference of variables used in the filter script.
@xref{Testing Filter Scripts}.

@cindex E, -E @r{option, summary}
@item -E
Stop after the preprocessing stage; do not run the compiler proper.
The output is in the form of preprocessed source code, which is sent
to the standard output.  @xref{Preprocessor}.

@opsummary{lint, mailfromd}
@item --lint
Check script file syntax and exit.  If the file is @sc{ok}, return 0
to the shell, otherwise print appropriate messages to stderr and exit
with code 78 (@samp{configuration error}).

@opsummary{single-process, mailfromd}
@item --single-process
Do not fork sub-processes to serve requests.  This option is meant to
assist in debugging @command{mailfromd}.  Don't use it for anything
else but for debugging, as it terribly degrades performance!

@opsummary{stack-trace, mailfromd}
@item --stack-trace
@itemx --no-stack-trace
Add @acronym{MFL} stack trace information to runtime error output.
Overrides @code{stack-trace} configuration statement.  Use the
@option{--no-stack-trace} to disable trace information.

@xref{tracing runtime errors}, for more information on this feature.

@anchor{gacopyz-log option}
@opsummary{gacopyz-log, mailfromd}
@item --gacopyz-log=@var{level}
Set desired logging level for @command{gacopyz} library
(@pxref{Gacopyz}).  There are five logging levels.  The following
table lists them in order of decreasing priority:

@table @asis
@item fatal
Log fatal errors.

@item err
Log error messages.

@item warn
Log warning messages.

@item info
Log informational messages.  In particular, this enables printing
messages on each subprocess startup and termination, which look like
that:

@example
Apr 28 09:00:11 host mailfromd[9411]: connect
    from 192.168.10.1:50398
Apr 28 09:00:11 host mailfromd[9411]: finishing
    connection
@end example

This level can be useful for debugging your scripts.

@item debug
Log debugging information.

@item proto
Log Milter protocol interactions.  This level prints huge amounts of
information, in particular it displays dumps of each Milter packet
sent and received.

@end table

Although it is possible to set these levels independently of each
other, it is seldom practical.  Therefore, the option
@option{--gacopyz-log=@var{level}} enables all logging levels from
@var{level} up.  For example, @option{--gacopyz-log=warn} enables
log levels @samp{warn}, @samp{err} and @samp{fatal}.  It is the
default.  If you need to trace each subprocess startup and shutdown,
set @option{--gacopyz-log=info}.  Setting the logging level to
@samp{proto} can be needed only for @command{Gacopyz} developers, to
debug the protocol.

@xref{Testing Filter Scripts}.

@opsummary{logger, mailfromd}
@item --logger=@var{mech}
Set logger mechanism (@var{mech} is one of @samp{stderr},
@samp{syslog}, @samp{syslog:async}).  @xref{Logging and Debugging}.

@opsummary{log-facility, mailfromd}
@item --log-facility=@var{facility}
Output logs to syslog @var{facility}.

@opsummary{log-tag, mailfromd}
@item --log-tag=@var{string}
Tag syslog entries with the given @var{string}, instead of the program name.

@opsummary{source-info, mailfromd}
@item --source-info
@itemx --no-source-info
Include @sc{c} source information in debugging messages.  This is
similar to setting @code{line-info yes} in the @code{debug}
configuration block (@pxref{debug statement, line-info,,
mailutils, GNU Mailutils Manual}).

The @option{--no-source-info} can be used to cancel the effect of the
@code{line-info yes} configuration statement.

You do not need this option, unless you are developing or debugging
@command{mailfromd}.

@opsummary{syntax-check, mailfromd}
@item --syntax-check
Synonym for @option{--lint}.

@opsummary{trace, mailfromd}
@item --trace
@item --no-trace
Enable or disable action tracing.  If @option{--trace} is given,
@command{mailfromd} will log all executed actions.  @xref{Logging and
Debugging}.

@opsummary{trace-program, mailfromd}
@item --trace-program[=@var{string}]
Enable program instruction tracing.  With this option
@command{mailfromd} will log execution of every instruction in the
compiled filter program.  The optional arguments allows to specify a
comma-separated list of source code modules for which the tracing is
to be enabled, for example @code{--trace-program=bi_io,bi_db} enables
tracing for functions from modules @file{bi_io.c} and @file{bi_db.c}
(notice, that you need not give file suffixes in @var{string}).

  This option is useful for debugging @command{mailfromd}, but is not
advised to use otherwise, since it is very time-costly.

@opsummary{transcript, mailfromd}
@item -X
@itemx --transcript
@itemx --no-transcript
Enable or disable transcript of the @acronym{SMTP} sessions to the log
channel.  @xref{Logging and Debugging}.

@opsummary{syslog, mailfromd}
@item --syslog
Selects default syslog mechanism for diagnostic output.

@opsummary{stderr, mailfromd}
@item --stderr
Directs all logging to standard output.  Similar to @option{--logger=stderr}.

@opsummary{xref, mailfromd}
@item --xref
Same as @option{--dump-xref}.  @xref{Logging and Debugging}.

@end table

@node Informational Options
@subsection Informational Options

@table @option
@item -?
@itemx --help
Give a short help summary.

@item --usage
Give a short usage message.

@item -V
@itemx --version
Print program version.

@opsummary{show-defaults, mailfromd}
@item --show-defaults
Show compilation defaults.  @xref{Examining Defaults}.
@end table

@node Starting and Stopping
@section Starting and Stopping

@cindex startup
@cindex user privileges
@cindex groups
@cindex supplementary groups
  Right after startup, when @command{mailfromd} has done the
operations that require root privileges, it switches to the privileges
of the user it is configured to run as (@pxref{default user privileges}) or
the one given in its configuration file (@pxref{conf-priv, user}).  During
this process it will drop all supplementary groups and switch to the
principal group of that user.

  Such limited privileges of the daemon can cause difficulties if your
filter script needs to access some files (e.g.@: @command{Sendmail}
databases) that are not accessible to that user and group.  For
example, the following fragment using @code{dbmap} function:

@example
@group
if dbmap("/etc/mail/aliases.db", $f, 1)
  @dots{}
fi
@end group
@end example

@noindent
will normally fail, because @file{/etc/mail/aliases.db} is readable only
to the root and members of the group @samp{smmsp}.

  In such situations you need to instruct @command{mailfromd} to
retain the privileges of one or several supplementary groups when
switching to the user privileges.  This is done using @code{group}
statement in the @command{mailfromd} configuration file
(@pxref{conf-priv, group}).  In example above, you need to use the following
settings:

@example
group smmsp;
@end example

@noindent
(The same effect can be achieved with @option{--group} command line
option: @kbd{mailfromd --group=smmsp}).

@cindex signals
@cindex SIGQUIT
@cindex SIGTERM
@cindex SIGINT
  To stop a running instance of @command{mailfromd} use one of the
following signals: @code{SIGQUIT}, @code{SIGTERM}, @code{SIGINT}.
All three signals have the same effect: the program cancels handling any
pending requests, deinitializes the communication socket (if it is a
@acronym{UNIX} socket, the program unlinks it) and exits.

@cindex SIGHUP
  To restart the running @command{mailfromd} instance, send it
@code{SIGHUP}.  For restart to be possible, two conditions must be
met: @command{mailfromd} must be invoked with the full file name, and the
configuration file name must be full as well.  If either of them is not
met, @command{mailfromd} displays a similar warning message:

@example
@group
warning: script file is given without full file name
warning: restart (SIGHUP) will not work
@end group
@end example

@noindent
or:

@example
@group
warning: mailfromd started without full file name
warning: restart (SIGHUP) will not work
@end group
@end example

The reaction of @command{mailfromd} on @code{SIGHUP} in this case is
the same as on the three signals described previously, i.e. cleanup
and exit immediately.

@cindex pidfile
  The @acronym{PID} of the master instance of @command{mailfromd} is
kept on the @dfn{pidfile}, which is named @file{mailfromd.pid} and is
located in the program @dfn{state directory}.  Assuming the default
location of the latter, the following command will stop the running
instance of the daemon:

@example
kill -TERM `head -n1 /usr/local/var/mailfromd/mailfromd.pid`
@end example

The default pidfile location is shown in the output of @command{mailfromd
--show-defaults} (@pxref{Examining Defaults}), and can be changed at
run time using @code{pidfile} statement (@pxref{conf-base, pidfile}).

@cindex @file{rc.mailfromd}
@cindex system-wide startup script
  To facilitate the use of @command{mailfromd}, it is shipped with a shell
script that can be used to launch it on system startup and shut it
down when the system goes down.  The script, called
@file{rc.mailfromd}, is located in the directory @file{/etc} of the
distribution.  It takes a single argument, specifying the action that
should be taken:

@table @asis
@item start
Start the program.

@item stop
Shut down the program

@item reload
Reload the program, by sending it @code{SIGHUP} signal.

@item restart
Shut down the program and start it again.

@item status
Display program status.  It displays the @acronym{PID} of the master
process and its command line, for example:

@example
@group
$ @kbd{/etc/rc.d/rc.mailfromd status}
mailformd appears to be running at 26030
26030 /usr/local/sbin/mailfromd @/--group smmsp
@end group
@end example

@noindent
If the second line is not displayed, this most probably mean that
there is a @samp{stale} pidfile, i.e. the one left though the program
is not running.

An empty @kbd{rc.mailfromd status} output means that
@command{mailfromd} is not running.

@item configtest [@var{file}]
Check the script file syntax, report any errors found and exit.  If
@var{file} is given it is checked instead of the default one.

@item macros [-c] [@var{file}]
Parse the script file (or @var{file}, if it is given, extract
the names of Sendmail macros it uses and generate corresponding export
statements usable in the Sendmail configuration file.  By default,
@file{mc} statements are generated.  If @option{-c} (@option{--cf}) is
given, the statements for @file{sendmail.cf} are output.  See the next
chapter for the detailed description of this mode.
@end table

  You can pass any additional arguments to @command{mailfromd} by
editing @code{ARGS} variable near line 22.

  The script is not installed by default.  You will have to copy it to
the directory where your system start-up scripts reside and ensure it
is called during the system startup and shut down.  The exact
instructions on how to do so depend on the operating system you use
and are beyond the scope of this manual.

@node MTA Configuration
@chapter Using @command{mailfromd} with Various @acronym{MTA}s
  The following sections describe how to configure various Milter-capable
@acronym{MTA}s to work with @command{mailfromd}.

@menu
* Sendmail::
* MeTA1::
* Postfix::
@end menu

@node Sendmail
@section Using @command{mailfromd} with Sendmail.

  This chapter assumes you are familiar with Sendmail configuration in
general and with Milter configuration directives in particular.  It
concentrates only on issues, specific for @command{mailfromd}.

@cindex INPUT_MAIL_FILTER, mc file directive
  To prepare @command{Sendmail} to communicate with
@command{mailfromd} you need first to set up the @dfn{milter port}.
This is done with @code{INPUT_MAIL_FILTER} statement in your
@command{Sendmail} file:

@example
INPUT_MAIL_FILTER(`mailfrom', `S=unix:/usr/local/var/mailfromd/mailfrom')
@end example

  Make sure that the value of @samp{S} matches the value of
@code{listen} statement in your @file{mailfromd.conf} file
(@pxref{conf-milter, listen}).  Notice, however, that they
may not be literally the same, because @code{listen} allows to
specify socket address in various formats, whereas Sendmail's @samp{S}
accepts only milter format.

  If you prefer to fiddle directly with @file{sendmail.cf} file, use
this statement instead:

@example
Xmailfrom, S=unix:/usr/local/var/mailfromd/mailfrom
@end example

@anchor{exporting macros}
@cindex f, @command{Sendmail} macro
@cindex i, @command{Sendmail} macro
@cindex client_addr, @command{Sendmail} macro
@cindex confMILTER_MACROS_ENVFROM, mc file directive
@cindex @command{Sendmail} macros, exporting
@cindex Message-ID, exporting in @file{mc} file
  If you are using Sendmail version 8.14.0 or newer, you may skip to
the end of this section.  These versions implement newer Milter
protocol that enables @command{mailfromd} to negotiate with the
@acronym{MTA} the macros it needs for each state.

  Older versions of Sendmail do not offer this feature.  For Sendmail
versions prior to 8.14.0, you need to manually configure
@command{Sendmail} to export macros you need in your
@command{mailfromd.mfl} file.  The simplest way to do so is using
@command{rc.mailfromd} script, introduced in the previous chapter.
Run it with @code{macros} command line argument and copy its output to
your @file{sendmail.mc} configuration file:

@example
$ @kbd{rc.mailfromd macros}
@end example

If you prefer to work with @file{sendmail.cf} directly, use
@option{-c} (@option{--cf}) command line option:

@example
$ @kbd{rc.mailfromd macros -c}
@end example

Finally, if you use other mailfromd script file than that
already installed (for example, you are preparing a new configuration
while the old one is still being used in production environment), give
its name in the command line:

@example
$ @kbd{rc.mailfromd macros newscript.mfl}
# @r{or:}
$ @kbd{rc.mailfromd macros -c newscript.mfl}
@end example

  If you use this method, you can skip the rest of this chapter.
However, if you are a daring sort of person and prefer to do everything
manually, follow the instructions below.

@xopindex{dump-macros, mailfromd, described}
  First of all you need to build a list of macros used by handlers in
your @file{mailfromd.mfl} file.  You can obtain it running
@command{mailfromd --dump-macros}.  This will display all macros used
in your handlers, grouped by handler name, for example:

@example
@group
envfrom i, f, @{client_addr@}
envrcpt f, @{client_addr@}, @{rcpt_addr@}
@end group
@end example

  Now, modify @code{confMILTER_MACROS_@var{handler}} macros in your
@file{mc} file.  Here, @var{handler} means the uppercase name of the
@command{mailfromd} handler you want to export macros to, i.e. the
first word on each line of the above @command{mailfromd --dump-macros}
output.  @emph{Notice,} that in addition to these macros, you should
also export the macro @code{i} for the very first handler
(@command{rc.mailfromd macros} takes care of it automatically, but you
preferred to do everything yourself...)  It is necessary in
order for @command{mailfromd} to include @samp{Message-ID} in its log
messages (@pxref{Message-ID}).

  For example, given the above macros listing, which corresponds to our
sample configuration (@pxref{Filter Script Example}), the
@file{sendmail.mc} snippet will contain:

@example
@group
define(`confMILTER_MACROS_ENVFROM',dnl
confMILTER_MACROS_ENVFROM `, i, f, @{client_addr@}')
define(`confMILTER_MACROS_ENVRCPT',dnl
confMILTER_MACROS_ENVRCPT `, f, @{client_addr@}, @{rcpt_addr@}')
@end group
@end example

@cindex s, @command{Sendmail} macro
  Special attention should be paid to @code{s} macro
(@samp{HELO} domain name).  In @command{Sendmail} versions up to
8.13.7 (at least) it is available only to @code{helo} handler.  If you
wish to make it available elsewhere you will need to use the method
described in @ref{HELO Domain}

  Now, if you are a @emph{really} daring person and prefer to do everything
manually @emph{and} to hack your @file{sendmail.cf} file directly, you
certainly don't need any advices.  Nonetheless, here's how the two
statements above @emph{could} look in this case:

@example
@group
O Milter.macros.envfrom=i, @{auth_type@}, @{auth_authen@}, \
   @{auth_ssf@}, @{auth_author@}, @{mail_mailer@}, @{mail_host@}, \
   @{mail_addr@} ,@{mail_addr@}, @{client_addr@}, f
O Milter.macros.envrcpt=@{rcpt_mailer@}, @{rcpt_host@}, \
   @{rcpt_addr@} ,i, f, @{client_addr@}
@end group
@end example

@node MeTA1
@section Using @command{mailfromd} with MeTA1.
@cindex meta1
  MeTA1 (@uref{http://www.meta1.org}) is an @acronym{MTA} of next
generation which is designed to provide the following main features:

@itemize
@item Security
@item Reliability
@item Efficiency
@item Configurability
@item Extendibility
@end itemize

Instead of using Sendmail-compatible Milter protocol,
it implements a new protocol, called @dfn{policy milter}, therefore an
additional program is required to communicate with
@command{mailfromd}.  This program is a @dfn{Pmilter--Milter
multiplexer} @command{pmult}, which is part of the @samp{Mailfromd}
distribution.  @xref{pmult}, for a detailed description of its configuration.

The configuration of @samp{Meta1--Mailfromf} interaction can be
subdivided into three tasks.

@enumerate 1
@item Configure @command{mailfromd}

This was already covered in previous chapters.  No special
@samp{MeTA1}-dependent configuration is needed.

@item Configure @command{pmult} to communicate with @command{mailfromd}

This is described in detail in @ref{pmult}.

@item Set up MeTA1 to communicate with @command{pmult}

The MeTA1 configuration file is located in @file{/etc/meta1/meta1.conf}.
Configure the @command{smtps} component, by adding the following
section:

@example
policy_milter @{
    socket @{
        type = @var{type};
        address = @var{addr};
        [path = @var{path};]
        [port = @var{port-no};]
    @};
    [timeout = @var{interval};]
    [flags = @{ @var{flag} @};]
@};
@end example

Statements in square brackets are optional.  The meaning of each
instruction is as follows:

@table @code
@item type = @var{type}
Set the type of the socket to communicate with @command{pmult}.
Allowed values for @var{type} are:

@table @asis
@item inet
Use @acronym{INET} socket.  The socket address and port number are
set using the @code{address} and @code{port} statements (see below).

@item unix
Use @acronym{UNIX} socket.  The socket path is given by the
@code{path} statement (see below).
@end table

Notice, that depending on the @code{type} setting you have to set up
either @code{address}/@code{port} or @code{path}, but not both.

@item address = @var{addr}
Configure the socket address for @code{type = inet}.  @var{Addr} is
the IP address on which @command{pmult} is listening
(@pxref{pmult-conf, listen statement}).

@item port = @var{port-no}
Port number @command{pmult} is listening on
(@pxref{pmult-conf, listen statement}).

@item path = @var{socket-file}
Full pathname of the socket file, if @code{type = unix}.

@item timeout = @var{interval}
Sets the maximum amount of time to wait for a reply from
@command{pmult}.

The behavior of @command{smtps} in case of time out depends
on the @code{flags} settings:

@item flags = @{ @var{flag} @}

@var{Flag} is one of the following:

@table @asis
@item abort
If @command{pmult} does not respond, abort the current @acronym{SMTP}
session with a @samp{421} error.

@item accept_but_reconnect
If @command{pmult} does not respond, continue the current session
but try to reconnect for the next session.
@end table
@end table
@end enumerate

For example, if the @command{pmult} configuration has:

@example
listen inet://127.0.0.1:3333;
@end example

@noindent
then the corresponding part in @file{/etc/meta1/meta1.conf} will be

@example
smtps @{
    policy_milter @{
        socket @{
            type = inet;
            address = 127.0.0.1;
            port = 3333;
        @};
        @dots{}
    @};
    @dots{}
@};
@end example

Similarly, if the @command{pmult} configuration has:

@example
listen unix:///var/spool/meta1/pmult/socket;
@end example

@noindent
then the @file{/etc/meta1/meta1.conf} should have:

@example
smtps @{
    policy_milter @{
        socket @{
            type = unix;
            path = /var/spool/meta1/pmult/socket;
        @};
        @dots{}
    @};
    @dots{}
@};
@end example

@node Postfix
@section Using @command{mailfromd} with Postfix
@cindex Postfix
@cindex smtpd_milters, postfix configuration
@cindex non_smtpd_milters, postfix configuration
@findex /etc/postfix/main.cf
  To configure @command{postfix} to work with your filter, you
need to inform it about the socket your filter is listening on.
The @code{smtpd_milters} (or @code{non_smtpd_milters}) statement in
@file{/etc/postfix/main.cf} serves this purpose.  If the filter is
to handle mail that arrives via @acronym{SMTP}, use
@code{smtpd_milters}.  If it is to handle mail submitted locally to the
queue, use @code{non_smtpd_milters}.  In both cases, the value
is a whitespace-separated list of socket addresses.  Note, that Postfix
syntax for socket addresses differs from that used by Sendmail and
mailfromd.  The differences are summarized in the following table:

@float Table, postfix-socket
@caption{Socket addresses in various formats}
@multitable @columnfractions .33 .33 .33
@headitem Sendmail @tab Mailfromd @tab Postfix
@item inet:@var{port}@@@var{host}
@tab inet://@var{host}:@var{port}
@tab inet:@var{host}:@var{port}
@item unix:@var{file}
@tab unix://@var{file}
@tab unix:@var{file}
@end multitable
@end float

  For example, if your @command{mailfromd} listens on
@samp{inet://127.0.0.1:4111}, add the following to
@file{/etc/postfix/main.cf}:

@example
smtpd_milters = inet:127.0.0.1:4111
@end example

  @command{Mailfromd} uses Milter protocol version 6.  Postfix, starting from
version 2.6 uses the same version.  Older versions of Postfix use
Milter protocol 2 by default.  Normally, it should not be a problem,
as @command{mailfromd} tries to detect what version the server is
speaking.  If, however, it fails to select the proper version, you
will have to instruct Postfix what version to use.  To do so, add the
following statement to @file{/etc/postfix/main.cf}:

@example
milter_protocol = 6
@end example

@cindex i, @command{Sendmail} macro in @command{Postfix}
The way Postfix handles macros differs from that of Sendmail.
Postfix emulates a limited subset of Sendmail macros, and not
all of them are are available when you would expect them to.  In
particular, the @samp{i} macro is not available before the @samp{DATA}
stage, which brings two consequences.  First, @command{mailfromd} log
messages will not include message ID until the @samp{DATA} stage is
reached.  Secondly, you cannot use @samp{i} in handlers @samp{connect},
@samp{helo}, @samp{envfrom} and @samp{envrcpt},

@findex postfix-macros.sed
  If you wish to tailor Postfix defaults to export the actual macros
used by your filter, run @command{mailfromd --dump-macros} and filter
its output through the @file{postfix-macros.sed} filter, which is
installed to the @file{@var{prefix}/share/mailfromd} directory,
e.g.:

@example
@group
$ @kbd{mailfromd --dump-macros | \
  sed -f /usr/share/mailfromd/postfix-macros.sed}
milter_helo_macros = @{s@}
milter_mail_macros = @{client_addr@} @{s@} @{f@}
milter_rcpt_macros = @{rcpt_addr@} @{f@} @{client_addr@}
milter_end_of_data_macros = @{i@}
@end group
@end example

  Cut and paste its output to your @file{/etc/postfix/main.cf}.

For more details regarding Postfix interaction with Milter and
available Postfix configuration options, see
@uref{http://www.postfix.org/MILTER_README.html, Postfix before-queue
Milter support}.

@node calloutd
@chapter @command{calloutd}
@include calloutd.texi

@node mfdbtool
@chapter @command{mfdbtool}
@include mfdbtool.texi

@node mtasim
@chapter @command{mtasim} --- a testing tool
@include mtasim.texi

@node pmult
@chapter Pmilter multiplexer program.
@include pmult.texi

@node Reporting Bugs
@chapter How to Report a Bug

@quotation
@i{Documentation is like sex: when it is good, it is very, very
good; and when it is bad, it is better than nothing.}@*
Dick Brandon
@end quotation

  Although the author has tried to make this documentation as
detailed as is possible and practical, he is well aware that the
result is rather ``better than nothing'', than ``very good''.  So, if
you find that some piece of explanation is lousy or if you find
anything that should have been mentioned here, but is not, please
report it to @email{bug-mailfromd@@gnu.org.ua}.

  Similarly, if the program itself fails to meet your expectations, or
does not do what is described in this document; if you have found a
bug or happen to have any suggestion... or have written a useful
function you wish to share with the rest of @command{mailfromd} users,
or wish to express your thanks, email it to the same address,
@email{bug-mailfromd@@gnu.org.ua}.

  If you think you've found a bug, please be sure to include maximum
information needed to reliably reproduce it, or at least to analyze
it.  The information needed is:

@itemize
@item Version of the package you are using.
@item Compilation options used when configuring the package.
@item Run-time configuration (@file{mailfromd.mfl} file and the command
line options used).
@item Conditions under which the bug appears.
@end itemize

@node Gacopyz
@appendix Gacopyz
@include gacopyz.texi

@node Time and Date Formats
@appendix Time and Date Formats
@include strftime.texi

@node Upgrading
@appendix Upgrading
@include upgrade.texi

@node Copying This Manual
@appendix GNU Free Documentation License
@include fdl.texi

@node Concept Index
@unnumbered Concept Index

This is a general index of all issues discussed in this manual

@printindex cp

@ifset WEBDOC
@ifhtml
@node This Manual in Other Formats
@unnumbered This Manual in Other Formats
@include otherdoc.texi
@end ifhtml
@end ifset

@bye