.TH leafnode 8 "@VERSION@" leafnode \" -*- nroff -*-
.\" DO NOT EDIT: @configure_input@
.\"
.\" Written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995 Troll
.\" Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47 22646949.
.\"
.\" Use, modification and distribution is allowed without limitation,
.\" warranty, or liability of any kind.
.\"
.\" (C) of modifications by Matthias Andree, 2001 - 2021
.\" see COPYING for license.
.\"
.\" $Id: leafnode.8.in,v 1.45 2006/01/15 12:35:43 emma Exp $
.\"
.\" hilit19 is stupid: "
.SH NAME
leafnode \- NNTP server for small (dialup) sites

.SH SYNOPSIS
.B leafnode

.SH DESCRIPTION
.B Leafnode
is a USENET package intended for small sites, where there are few
users and little disk space, but where a large number of groups is
desired.
.PP
The design of
.B leafnode
is intended to self-repair after problems, and to require no
manual maintenance.
.PP
The
.B leafnode
program itself is the NNTP server.  It is run from
.BR inetd (8)
or
.BR xinetd (8)
when someone wants to read news.  The other parts of the package,
.B fetchnews
and
.B texpire,
are responsible for fetching new news from another server, and for
deleting old news.

.SH "ACCESS CONTROL"
No authentication or access control is supported.  This is a
deliberate omission: Implementing this is a job which should not be
redone for each and every service.
.PP
.B It is mandatory that you use external access control mechanisms like
.B tcpd, inetd/xinetd compiled with libwrap support option and the like
.B and that these are in effect.
tcpd and libwrap are components of Wietse Venema's fine
.B tcp_wrappers
package.
.PP
As a very rough last line of defense against abuse, leafnode will drop
connections from outside your LANs by default. You can configure
leafnode to let go of this restriction (look for the allowstrangers
option), but do not do that unless tight access control is in place.
Someone will abuse your computer sooner or later. Promised.
.PP
I recommend that either firewalling or tcpd be used for access control.

.SH FILES
All these files and directories
.I must
be readable by the user "news". It is recommended that, unless otherwise
stated, that the user "news" be the only user in the group "news" and
these files belong to "root:news" (user:group) so leafnode cannot modify
your configuration or filter files.
.PP
.I @msysconfdir@
should not be writable by the user "news", but it must be executable for
at least any of the group that the user "news" is in.
.I @msysconfdir@/config
contains the configuration parameters for
.B leafnode.
It \fBmust not\fR be writable by the user "news". Set this to owner
root:news and mode 640. For details, see CONFIGURATION below.
.PP
.I @SPOOLDIR@
must also be readable and writable by the user "news".
It contains the news articles; e.g.,
.I @SPOOLDIR@/alt/fan/agulbra
contains the articles in the
.I alt.fan.agulbra
group.  Each directory contains articles in numbered files (decimal
numbers, monotonically increasing), and a special file called
.I .overview
which contains the "Subject", "From", "Date", "Message\-ID",
"References", "Bytes" and "Lines" headers for each article in the
group.
.PP
Several subdirectories are special:
.PP
.I @SPOOLDIR@/leaf.node
contains the files that leafnode creates during operation, for example the
.BR groupinfo
file which contains information about each USENET newsgroup. This file
is built by
.BR fetchnews
(8). You can force a complete rebuild of the groupinfo file by calling
.BR fetchnews
with the parameter \-f (see
.BR fetchnews
(8)).
.PP
.I @SPOOLDIR@/out.going
contains local postings that
.BR fetchnews (8)
is to pass to the upstream NNTP server. After a posting has been successfully
written to disk, its u+r permission flag is set. This flag is
interpreted by
.BR fetchnews(8)
as "you may post this article". This prevents fetchnews from posting
articles that are still being received from newsreaders. (Note: versions 1.9.23 to 1.9.32
inclusively used u+x instead, which caused some "stuck post" problems
with articles in the spool when a prior leafnode version was updated to
one of these 10 versions. Updating to leafnode 1.9.33 or later fixes the
problem.)
.PP
.I @SPOOLDIR@/failed.postings
contains local postings that the upstream server rejected.
.BR fetchnews (8)
will create files in this directory, but none of the
.B leafnode
programs will delete anything in it.
.PP
.I @SPOOLDIR@/message.id
contains hard links to each message; this is used in place of the
.B dbz
database typically used by bigger servers.  (A directory such as this
is probably more efficient for the small servers
.B leafnode
is designed for but scales very badly.)
.PP
.I @SPOOLDIR@/interesting.groups
contains one file for each group an NNTP client has asked to read.
.B leafnode
will update the \fBctime\fR (ls\ \-l usually shows the \fBmtime\fR, try
ls\ \-lc) of the relevant file when a LISTGROUP, XOVER, XHDR, STAT, HEAD,
BODY or ARTICLE command is issued, when a GROUP or LIST ACTIVE command
(the latter only with a single group, not with patterns)
is issued for an interesting group (to avoid unsubscribing low-traffic groups that are
still read) and
.BR fetchnews (8)
will retrieve all new articles in all groups whose files have been
either
.IP
- touched during the past two days, or
.IP
- touched more than once, and at least once within the past week.
.PP
The timeout is configurable through the \fIconfig\fR file variables
\fBtimeout_short\fR and \fBtimeout_long\fR. See also \fBfetchnews\fR(8) for
the \fB\-n\fR option.
.PP
.IR /etc/inetd.conf
or
.IR /etc/xinetd.conf
contains the configuration which starts
.B leafnode.
It is strongly recommended to start
.B leafnode
as user news.

.SH ENVIRONMENT
.TP
LN_REJECT_POST_PRE
If this variable exists, all POST commands are rejected with a 400 code.
Use only for debugging clients.
.TP
LN_REJECT_POST_POST
If this variable exists, the POST command is rejected with a 400 code
after the article and CRLF.CRLF has been received. Use only for
debugging clients.

.SH CONFIGURATION
All configuration is done using the file
.IR @msysconfdir@/config ,
which may include a filter description file, filterfile for short, as
described below.
.PP
For the purposes of this section, whitespace shall be defined as an
arbitrary sequence consisting of one or more SPACE or HTAB characters,
ASCII positions 32 and 9, respectively.
.PP
The configuration file is strictly line-oriented with LF or CRLF as line
terminator.
.PP
Empty lines and lines consisting of only whitespace, possibly followed
by a comment (introduced by a hash mark (#) and extending through the
end of the line), are skipped.
.PP
All other lines have exactly three mandatory fields, a plain text
parameter, an assignment character (=) optionally surrounded by
whitespace and a value.  The value is either plain text or \[en] new since
leafnode v1.11 \[en] a string in double quotes with trivial backslash escape
(see below).
.PP
Plain text starts at the first non-whitespace character and extends
through the last non-whitespace character on the line that is not a
comment. A trailing comment on a line is skipped.
.PP
Quoted strings are enclosed in double quote characters (").
The backslash character (\\) is skipped, but it copies the immediately
following character verbatim, so that you can specify the backslash
itself by doubling it (\\\\) or a double quote character as part of the
string by preceding it with a backslash (\\"); the hash mark has no
special meaning as command introducer inside quoted strings. Text after
the end of the string is silently ignored (this may change in future
versions).  Comments after quoted strings are ignored.
.PP
.B MANDATORY PARAMETERS
.PP
These parameters must be specified for leafnode to work.
.TP
server = news02.example.com
"server" is used by
.B fetchnews
(8) to select what NNTP server(s) to retrieve news from and to post your
articles to. You can specify more than one news server; in that case,
the servers will be queried from the top down. If you want to post
articles, at least one of your servers should allow you to post.
In the example above, news02.example.com is the news server.

This parameter can be given more than once. Each \fBserver\fR starts
with a fresh set of default configuration options, no inheritance takes
place from the previous server definition. Only options explicitly
marked "server-specific" can be set on a per server basis, "general" options
are set for all servers at the same time.
.TP
expire = 5
"expire" is the number of days an article should be kept around. In the
example, five days after the article has last been read, it is deleted by
.B texpire
(8). This value MUST be at least 1. This parameter is global, see the
introductory paragraph of the following
.B GENERAL OPTION PARAMETERS
section to find out what this means.
.PP
.B GENERAL OPTIONAL PARAMETERS
.PP
These options can only be configured once in the configuration file, and
take effect for leafnode as a whole. It does not matter where these are
specified relative to server= options, but for clarity, you are encouraged
to place these before the first server= line. Specifying each of the
global options more than once lets the last copy take effect, but may
cause errors in the future.
.TP
hostname = host.domain.country
By default, leafnode tries hard to figure the host name of your computer,
skipping inadequate (non-unique) names if possible. It will look up your
computer's host name with gethostname(3) and then try to qualify the
name with gethostbyname(3) if necessary. Common sources for the full
name therefore are /etc/hosts, NIS and DNS, but consult your system
documentation for details.

If leafnode fails to determine the host name,
this is usually a hint that your system is not configured properly, or
it has a hostname that is unsuitable for the domain part of a
Message\-ID, for example, "localhost.localdomain", and you should fix the
name service configuration. Adding a unique fully-qualified host name to
/etc/hosts is usually sufficient. Please see \fIREADME-FQDN\fR for more
details.

You can configure the unique fully-qualified host name here as well, but
this is not recommended and discouraged.
.TP
create_all_links = 1
Normally, fetchnews will store articles only in the newsgroups which it
considers interesting. With this option set, fetchnews will create
hardlinks for all newsgroups in the Newsgroups: header that it knows
about. This may be of interest if you want to apply a score- or killfile
to the local Xref: line.
.TP
maxfetch = 1000
"maxfetch" specifies the maximum number of articles
.B fetchnews
(8) should fetch from the upstream server in each group. Its use is not
advised, because if you use it you will not see all the traffic in a
group. By default there is no limit.
.TP
initialfetch = 1
"initialfetch" defines how many articles from a newly subscribed group
should be fetched. The default is to fetch all old articles, which can
get quite time-consuming when subscribing to a very busy group. This
is equivalent to setting initialfetch to zero. If you want to get no
old articles when subscribing to a new group, you should set initialfetch
to one, as in the example above.
.PP
groupexpire very.crowded.group = 1
.TP
groupexpire very.crowded.hierarchy.* = 1
"groupexpire" makes it possible to adjust expiry times for individual
groups. Expiry times are given in days. 0 means "use the default",
negative values prevent the expire process for this group altogether (you can
consider this an archive mode). This value is used by
.B texpire
(8). You can specify as many groupexpire lines as you like. It is
possible to specify
.B glob
(7)-like wildcard expressions.
.TP
maxage = 10
If an article turns up on your upstream news server which is older than
"maxage" days it will not been fetched even if you do not have it yet.
This is useful if your upstream server gets occasional "hiccups". The
default is set to 10. If you want to switch this feature off, set
maxage to some very large value, such as 20000 (this is equivalent
to roughly 54 years).
.TP
maxold = 10
Is synonymous to maxage, see above.
.TP
maxlines = 2000
If you want to avoid receiving very large articles, you may set the
"maxlines" parameter to the maximal number of lines an article should
have. By default, this feature is switched off.
.TP
minlines = 2
Sometimes newsgroups are spammed with empty postings. To reject these
postings, you can set the "minlines" parameter. Setting minlines to
a value larger 4 is probably not a good idea since you will also start
to kill "real" postings then. By default, this feature is switched off.
.TP
maxbytes = 100000
If you want to avoid receiving very large articles, instead of using the
"maxlines" parameter you can also use the "maxbytes" parameter. By
default, this feature is switched off.
.TP
maxcrosspost = 5
If you want to combat spam, you can filter out all postings that are
posted to more than a certain number of newsgroups. The number is
defined by setting "maxcrosspost". Setting this parameter to very low
values is probably a bad idea. This feature is switched off by default.
.TP
maxgroups = 5
Synonymous for maxcrosspost. See above.
.TP
filterfile = @msysconfdir@/filters
Leafnode can filter the input headers for arbitrary regular expressions.
These are stored in a file designated "filterfile". The format of
"filterfile" is very simple: one perl-compatible regular expression per
line. If one of the regular expressions fits to a header to be
downloaded, the body of that article will be rejected. This feature is
switched off by default. The format of the regular expressions is
described in
.BR pcre (3).
.TP
timeout_short = 2
By default, a group that has been accidentally touched is being fetched
for two days. You can change this time by changing timeout_short.
.TP
timeout_long = 7
By default, a group that has not been read at all is being fetched for
seven days before being unsubscribed. This interval can be changed by
setting timeout_long to a different value.
.TP
timeout_active = 90
By default, active files from the upstream servers are re-read every
90 days. This interval can be changed by setting timeout_active to
a different value. Be aware that reading an active file transfers
about one MB of information if the server that you are using carries
a reasonable number of groups (i. e. around 20,000).
.TP
timeout_client = 900 (since v1.9.23)
By default, leafnode will drop the connection 900 seconds (15 minutes)
after seeing the last command from the client. You can change the
timeout here. Setting it too low (like below 5 minutes) will annoy your
users and consume more system resources for re-reading all the files.
.TP
timeout_fetchnews = 300 (since v1.9.52)
Fetchnews will, since v1.9.52, assume the upstream server has become
wedged after waiting for a reply for 300 seconds. You can change the
timeout here.
.TP
timeout_lock = 5 (since v1.9.54)
Configure how many seconds the leafnode programs (applyfilter,
checkgroups, fetchnews, texpire) will wait for the lock
file before aborting. Setting this to 0 means to wait indefinitely.
.B NOTE:
you can override this by setting the environment variable
LN_LOCK_TIMEOUT (note it is not LN_TIMEOUT_LOCK).
The default is 5 seconds.
.TP
delaybody = 1
With this option set,
.B fetchnews
(8) fetches only the headers of an article for visual inspection. Only
when the headers have been read, the bodies of the articles will
be retrieved the next time
.B fetchnews
(8) is called. This can save a huge amount of download time and disk space.
.TP
delaybody_in_situ = 1 (since v1.9.41)
This is only applicable with delaybody=1.

By default, leafnode will give the full downloaded article a new article
number so they appear as new in your newsreader. This does not work for
all newsreaders. With this option set, leafnode will retain the original
article number. You'll have to figure out how to tell your newsreader to
show old articles. This option defaults to 0. It is
.B highly
recommended to leave it unset.

.TP
debugmode = 1
With this option set,
.B fetchnews
(8),
.B texpire
(8)
and
.B leafnode
(8) will start to log lots of debugging output via
.B syslog
(8) at facility news and priority debug. Use it for tracking down
problems with your feed. debugmode should be left at 0 for regular use
because it can log enormous amounts of data. The higher the number, the
more will be logged. Choosing a figure greater than 3 will not make a
difference at the moment.
.TP
allow_8bit_headers = 1 (since v1.9.25)
By default, leafnode rejects local posts that have 8-bit characters in
their headers, because they violate relevant standards, particularly RFC-2822 (which RFC-1036 is based
on) that demands that Usenet news headers (as mail headers) must be pure
7-bit US-ASCII, with only whitespace allowed from the control
characters.

However, as UTF-8 is to come, and some national hierarchies,
particularly the Norwegian and Danish (no.*, dk.*) seem to have agreed
on preferring just-send-eight over RFC-2047, you can set this option to
allow 8-bit data in headers. Leafnode will however add a warning header
if 8-bit data is present, stating that the site administrator allowed
this.

There is no way to make leafnode accept non-whitespace control
characters in headers.
.TP
allowSTRANGERS = MAGIC (since v1.9.23)
By default, leafnode refuses connections from outside your LANs. Check
config.example for how to use this parameter to let strangers connect to
your leafnode. Instead of MAGIC, you have to write a number as mentioned
in \fIconfig.example\fR. Note that capitalization matters.
.TP
linebuffer = 1
By default, stdout and sometimes stderr of applications are set to "fully
buffered" unless connected to terminals. Use this option to explicitly
request line buffered mode for stdout and stderr.
.TP
clamp_maxage = 0
By default, leafnode will derive a "maxage" argument from the expire
time that is applicable to the group (groupexpire if set, expire
otherwise), to prevent fetching articles again that were once there and
then cleared by \fBtexpire\fR(8). Set clamp_maxage=0 to get rid of this
behaviour.
.TP
article_despite_filter = 1 (since v1.9.33)
By default, fetchnews will request HEAD and BODY separately if a filter
file is defined and delaybody is off. For high latency, high throughput
links (such as interleaved DSL or satellite links), it may be faster to
request head and body together with an ARTICLE command and ignore the
body if the filters apply (though it may not be cheaper if you pay per
MByte), enabling this option will force leafnode to use the ARTICLE
command in spite of filters being defined. (Note that in delaybody mode,
HEAD and BODY will ALWAYS be requested separately.)
.TP
newsadmin = news@leafnode.example.org (since v1.9.47)
This option sets the From: address for the placeholder article, it
should be the news administrator's mail address.
It defaults to news@HOSTNAME, where HOSTNAME is leafnode's hostname.
.PP
.B SERVER-SPECIFIC OPTIONAL PARAMETERS
.PP
These options can only be placed after the server= line of the server to
which you would like these to apply, and they always pertain to the
preceding server= line. Specifying them before the first server= line is
an error.
.TP
username = myname
If any of your news servers requires authentication, you can enter
your username on that server here. This field may occur multiple times,
once after each server definition. See the introduction of this
CONFIGURATION section for information on how to quote \fImyname\fR.
.TP
password = mypassword
If any of your news servers requires authentication, you can enter
your password on that server here. This field may occur multiple times,
once after each server definition.  Since the password is available in clear
text, it is recommended that you set the rights on the config file as
restrictive as possible, otherwise other users of your computer will be able
to get your password(s) from that file. See the introduction of this
CONFIGURATION section for information on how to quote \fImypassword\fR.
.TP
port = 8000
By default, fetchnews tries to connect to its upstream news servers on the
NNTP port (119). If your servers run on a different port, you can specify
those here. This field may occur multiple times, once after each server
definition.
.IP ""
.B Note:
to modify the port your own leafnode servers listens on, change the
inetd.conf or xinetd.conf configuration file.
leafnode does not set up its listen port itself.
.TP
timeout = 30
By default, leafnode tries to connect for 10 seconds to a server and then
gives up. If you have a slow server, you can try for a longer time by
setting the timeout higher (in this example, 30 seconds). The timeout
can be tuned individually for each server.
.TP
noactive = ANYTHING (v1.9.25 ... v1.11.4)
.TP
noactive = 1 (since v1.11.5)
If this parameter is set, the active file is never downloaded from this
server. Use this for very slow servers unless they carry groups that
other servers do not offer. Leafnode versions 1.9.25 to 1.11.4 would
always assume that "ANYTHING" had been 1. "noactive = 0" is supported
since v1.11.5.
.TP
nodesc = ANYTHING (until v1.11.4)
.TP
nodesc = 1 (since v1.11.5)
Some servers do not deliver news groups descriptions correctly because
they cannot parse the XGTITLE and LIST NEWSGROUPS commands. In that
case, put this line after the "server" line. Leafnode versions up to
v1.11.4 would always assume that "ANYTHING" had been 1. "nodesc = 0" is
supported since v1.11.5.
.TP
nopost = 1 (since v1.9.23)
Prevent posting to this server. You can use this if the upstream will not let you
post but still greet leafnode with 200 or if the upstream does not
forward your postings reliably.
.TP
noread = 1 (since v1.9.33)
Prevent fetching news articles or active files from this server. You can
use this if the upstream is good to post, but too slow to fetch news
from.
.TP
noxover = 1 (since v1.9.47)
Prevent the use of XOVER on the current server. Fetchnews will use XHDR
instead.
.TP
only_groups_match_all = 1 (since v1.9.52)
Usually, when cross-posting an article, fetchnews will post the article
if ANY group listed in the Newsgroups: header is matched by the PCRE.
With this option on, ALL groups listed in the Newsgroups: header must
match. This can be used to avoid "poison" groups when you have multiple
upstream servers.
.TP
only_groups_pcre = PCRE (since v1.9.28)
This parameter lists the Perl-compatible regular expression of groups
that are fetched or posted to this server. The PCRE is automatically
anchored at the left hand side, so you can omit the leading ^. Remember
to escape dots, as in:
.br
de\\.comp\\.|de\\.comm\\.
.IP
If this parameter is omitted, all groups are fetched from and posted to
this server.
.IP
Note: you must run fetchnews with the \fB\-f\fP option after changing, adding
or removing any only_groups_pcre option.
.IP
Hint: you can use something like this to check your only_groups_pcre
settings:
.br
cut \-f1 \-d" " @spooldir@/leaf.node/groupinfo \\
.br
| pcregrep 'PATTERN'
.TP
post_anygroup = 1 (since v1.9.37)
This parameter makes leafnode post on this server without checking if it
carries the group an article is posted to. The default is post_anygroup =
0, which means that leafnode will check with a "GROUP" command if the
server carries the group the articles is posted into. Use this on
post-only servers that do not allow the "GROUP" command. Note:
inconsiderate use of this parameter may cause articles to end up in
the failed.postings directory.
.PP
.B OBSOLETE PARAMETERS
.TP
supplement
is synonymous to server. Do not use it on new installations.
.TP
fqdn
is synonymous to hostname. Do not use it on new installations.

.SH PROTOCOL
Here are the NNTP commands supported by this server:
.PP
ARTICLE, BODY, DATE, GROUP, HDR, HEAD, HELP, LAST, LIST, LIST ACTIVE, LIST
ACTIVE.TIMES, LIST EXTENSIONS, LIST NEWSGROUPS, LIST OVERVIEW.FMT,
LISTGROUP, MODE, NEWGROUPS, NEXT, POST, OVER, SLAVE, STAT, XHDR, XOVER.
These commands follow RFC-977 and RFC-2980, except HDR and OVER which
are recognized in anticipation of current NNTP drafts.
.PP
Note that the syntax of HDR and OVER may change.
.SH BUGS
Leafnode is totally unaware of UTF-8 and will reject a client that posts
UTF-8 characters in the header. Current Usefor drafts claim all article
headers UTF-8 encoded Unicode. Leafnode still expects RFC-2047
instead which may eventually be phased out in favour of UTF-8.

Leafnode stops reading a line at the first NUL character.

Leafnode may not cope well with crosspostings that cross hierarchies if
you have multiple upstream feeds and use the only_groups_pcre
configuration option.

Leafnode will only bother to determine the time zone offset for
generated Date: headers for posts that lack them on systems that offer
the tm_gmtoff member in struct tm (commonly BSD and GNU systems).

.SH AUTHOR
Written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995
Troll Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47
22646949.
.PP
Modified by Cornelius Krasel <krasel@wpxx02.toxi.uni\-wuerzburg.de>,
Randolf Skerka <Randolf.Skerka@gmx.de> and
Markus Enzenberger <enz@cip.physik.uni\-muenchen.de>.
Copyright of the modifications 1997 \[en] 1999.
.PP
Modified by Matthias Andree <matthias.andree@gmx.de>, Copyright 1999 \[en] 2002.
Modified by Ralf Wildenhues <ralf.wildenhues@gmx.de>, Copyright 2002.
.PP
Jonathan Larmour <jifl@jifvik.org> contributed the timeout_client
feature.

Andreas Meininger <a.meininger@gmx.net> contributed the code to
let texpire ignore groupexpire = \-1 groups.

Mark Brown
<broonie@debian.org> added the noactive option.
.PP
Numerous contributions by other people.
.PP
The initial development of
.B leafnode
has been paid for by Uninett AS (http://www.uninett.no/).

.SH SEE ALSO
.BR applyfilter (8),
.BR checkgroups (8),
.BR fetchnews (8),
.BR newsq (1),
.BR texpire (8).
.PP
.BR tcpd (8),
.BR hosts_access (5),
.BR glob (7),
.BR pcre2 (3),
.BR "RFC 977,"
.BR "RFC 2980."