File: fetchnews.8.in

package info (click to toggle)
leafnode 1.11.11-1
  • links: PTS
  • area: main
  • in suites: bullseye, buster, sid
  • size: 2,468 kB
  • sloc: ansic: 10,914; sh: 1,748; xml: 628; makefile: 291; perl: 84; sed: 4
file content (238 lines) | stat: -rw-r--r-- 8,077 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
.TH fetchnews 8 "@VERSION@" leafnode \" -*- nroff -*-
.\" DO NOT EDIT: @configure_input@
.\"
.\" Written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995-96 Troll
.\" Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47 22646949.
.\"
.\" Use, modification and distribution is allowed without limitation,
.\" warranty, or liability of any kind.
.\"
.\" $Id: fetchnews.8.in,v 1.22 2006/03/18 12:30:11 emma Exp $
.\"
.\" hilit19 is stupid: "
.SH NAME
fetchnews \- fetch news via NNTP

.SH SYNOPSIS
.B fetchnews
[\fB\-q\fR] [\fB\-v\fR] ... [\fB\-x\fR \fInumber\fR] [\fB\-l\fR] [\fB\-n\fR]
[\fB\-f\fR] [\fB\-P\fR] [\fB\-w\fR]

.SH DESCRIPTION
.B Leafnode
is a USENET package intended for small sites, where there are few
users and little disk space, but where a large number of groups is
desired.
.PP
The design of
.B leafnode
is intended to self-repair after problems, and to require no
manual maintenance.
.PP
.B Fetchnews
(formerly called "fetch") is the program which submits and retrieves new
articles to or from the upstream NNTP server.

.SH OPTIONS
.TP
.B \-q
Be quiet. Suppresses some warnings. This flag cancels any previously
encountered \fB\-v\fR flags.
.TP
.B \-v
Be verbose.  This flag cancels any previously encountered \fB\-q\fR
flags.  A maximum of four
.B \-v
can be used (the more v, the more verbose):
.TP
.B \-v
Warn about non-critical errors.
.TP
.B \-vv
Be verbose whenever a group command occurs.
.TP
.B \-vvv
Be verbose whenever an article command occurs.
.TP
.B \-vvvv
For debugging purposes. Don't expect the additional output to make sense.
.TP
.B \-w
Wait mode. This causes the XOVER update (which runs after the article
fetch phase) to run in the foreground. By
default, the XOVER update runs in the background, detached from the
shell, on systems that offer a working fork(2) call. Helpful for
debugging and to avoid terminal clobbering when fetchnews is run
manually.
.TP
\fB\-x\fR \fInumber\fR
Go
.I number
articles back and try to re-fetch what's missing. This can be useful to
pull data from broken servers that have either overview and group data
out of synch or create articles out of article number order.
.TP
.B \-l
Don't use supplementary servers (\fI\-l\fR is supposed to mean "local").
This does speed up the operation quite a bit.
.TP
.B \-n
Do not expire newsgroup subscriptions. Technically: do not unlink files in the
.I interesting.groups
directory.
.TP
.B \-f
Erase the active.read file, thus force redownloading the whole active file from all upstream servers. This is
usually occurring only very infrequently because it takes a long time.
See \fItimeout_active\fR in \fBleafnode\fR(8).
.B DO NOT
use this option in cron jobs or otherwise regularly.
.TP
.B \-P
Only post articles that have accumulated in the
.I out.going
directory but do not fetch articles from upstream servers.

.SH "EXIT STATUS"
.BR fetchnews
will exit with code 0 if it could connect to all servers and encountered
no other errors. It exits with code 2 when it was unable to connect to
at least one server, and it exits with code 1 on other errors.

.SH "FILES AND CONFIGURATION"
.TP
.B @SPOOLDIR@/leaf.node/active.read
is a file to track when the active files have been successfully
downloaded from the upstream servers the last time. If it is missing,
this has the same effect as giving the \fB\-f\fR option. It is only
created after the active file of each upstream servers has been
downloaded successfully.
.PP
See
.BR leafnode (8)
for the list of other files affected, and for information on configuring
.B fetchnews.

.SH ENVIRONMENT
.TP
\fBLN_SKIP_GROUPS=\fR\fIpattern[,pattern[...]]\fR (since v1.9.53)
If set, all groups that match one of the
.BR wildmat (3)
.IR pattern s
will be skipped during the fetch. This can be used as a quick
way of skipping binary groups during daytime, for instance, example:
.IP
env 'LN_SKIP_GROUPS=*binary*,alt.*' @sbindir@/fetchnews \-nq

.TP
.B LN_LOCK_TIMEOUT
This variable is parsed as an unsigned integer value and determines how
many seconds fetchnews will wait when trying to obtain the lock file
from another leafnode program. 0 means to wait indefinitely. This
variable takes precedence over the configuration file.

.TP
.B LN_SUPPRESS_DATE
(since v1.11.7)
If this variable is defined (regardless of its value, including empty),
leafnode will skip checking the upstream server's time by means of the
NNTP DATE command.

.SH ALGORITHM
Here is a brief description of what
.B fetchnews
does.
.PP
First, after connecting to the upstream NNTP server,
all articles that have the u+r permission (0400) flag set in
.B @SPOOLDIR@/out.going
are posted using POST.  If a posting fails, that article is moved to
.B @SPOOLDIR@/failed.postings
which an administrator has to clean up once in a while.  Moving a file
back to
.B @SPOOLDIR@/out.going
is sufficient to retry the post it. The u+r flag is checked to prevent posting
an (incomplete) article that \fBleafnode\fR(8) is still receiving at the
same time.  If a posting succeeds, the article is removed from out.going
in order to not post it twice into a moderated newsgroup on a different
server (which would send multiple copies of the article to the
moderator).
.B Note:
older fetchnews versions tried to post on all servers first, but failed
to do so when one of the servers could not be reached. If you need to
work around a dodgy upstream server that is fast but doesn't forward
your posts reliably, prohibit posting to this server using the nopost
server option in the configuration file, see
.BR leafnode (8).
.PP
Second, the upstream server's list of newsgroups (obtained using LIST
and LIST NEWSGROUPS) is merged into
.BR leafnode "'s"
(if timeout_active has expired, the active.read file is not present or
the active file is empty) or updated (with NEWGROUPS).
.PP
Third,
.B fetchnews
iterates over the list of newsgroups, performing a GROUP, an XOVER (if
supported) or XHDR Message\-ID and a number of HEAD and BODY or ARTICLE
commands for each group which has been read recently. It assumes that
for the particular group it is looking at, "maxage" was set to the
groupexpire or global expire. As a consequence, it makes only sense to
set maxage lower than or equal to expire.

.SH NOTES
.B Fetchnews
does not use NEWNEWS.  NEWNEWS is not used because it is often very
expensive and therefore also disabled at many sites.
.PP
After some experiments, I decided that it was better to fetch all the
articles in a group than to optimize away some; users would complain
too loudly when the optmizer guessed wrongly.
.PP
The only way to control what newsgroups are accessible to local users
is to control what the upstream server lets
.B fetchnews
read, and that will not keep out all crossposts (endemic in e.g. the
alt.sex groups).  If your site is big enough that you need to control
newsgroup access, consider using a news server which is designed for a
big site.
.PP
.B Fetchnews
sets its real and effective uid to "news" when started as root and
refuses to run if it cannot get this user ID.

.SH BUGS
.B Fetchnews
ignores the "Supersedes:" and "Control:" headers.

.SH HISTORY
Older versions of
.B fetchnews
posted any articles found in out.going. Since v1.9.23,
.B fetchnews
will only post article files that have their user executable bit set.
.B Leafnode
sets this bit after it has successfully received a post. This behaviour
avoids posting incomplete articles when
.B leafnode
receives a post at the same time
.B fetchnews
is posting.

.SH AUTHOR
Written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995\\-\[en]\\-96
Troll Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47
22646949.
.PP
Modified by Cornelius Krasel <krasel@wpxx02.toxi.uni\-wuerzburg.de>,
Markus Enzenberger <enz@cip.physik.uni\-muenchen.de>
and Randolf Skerka <Randolf.Skerka@gmx.de>.
Copyright of the modifications 1997\\-\[en]\\-1999.
Modified by Matthias Andree <matthias.andree@gmx.de>, copyright of the
modifications 2001\\-\[en]\\-2004.

.SH SEE ALSO
.BR leafnode (8),
.BR texpire (8),
.BR newsq (1),
.BR @sysconfdir@/leafnode/filters.example .