Newsx - Frequently Asked Questions
		 ==================================


Fetching new groups seem to take forever.

	If you are connected to a server with a big news spool, many
	newsgroups may contain thousands of news articles. If you start
	fetching a new newsgroup, newsx will try to fetch all of these.

	It is often convenient to limit the number of articles fetched
	in previously unseen groups, for instance by giving the option:

		--maxnew 100


How do I find out about future newsx versions?

	Send an email message to majordomo@kvaleberg.no with the 
	following message body:

		subscribe newsx-announce

	This should add your name to the newsx announcement list, where
	future versions and possible bug alerts will be announced.


I fetch most of my news from one provider, but in addition I want to pick
up `linux.*' from another source. Is it possible? I can't find any newsx
configuration file where I can specify this.

	That is because newsx relies on the configuration file already
	present in your news spool. This conveniently coordinates the 
	actions of the outbound stream prepared by the news server itself
	with the inbound stream handled by newsx.

	Assuming you already has your main newsserver set up (see README),
	you have to define the second news source. For C News, you would
	then add to the `sys' file:

		foo/foo.bar:linux.all/all:FL:

	Similarily for the INN `newsfeeds' file:

		foo/foo.bar:linux.*:Tf,Wf:

	This will tell newsx to only pick up the linux hierarchy to the
	foo spool, and tell the local news server to place local postings
	for the `linux' hierarchy in the foo spool.

	Assuming news.acme.net don't carry these groups, you will never get
	any linux news from news.acme.net. But to prevent postings to be
	sent there in vain, and to avoid attempts at fetching these
	groups, the acme entry needs to be modified:

		acme/newsx:all,!linux.all,!junk,!control/all:FL:

	And for INN:

		acme/newsx:*,!linux.*,!junk,!control:Tf,Wf:

	To fetch from both sources, you would need to invoke newsx twice:

		newsx acme news.acme.net &
		newsx foo  news.foo.bar  &


How do I configure a local newsgroups?

	If you define newsgroups that should be local to your own machine,
	you need to exclude these from the external feeds. Assuming these
	groups are called `local.something', the entry in the C News sys
	file should be:

		acme/newsx:all,!local.all,!junk,!control/all:FL:

	Similarily for the INN newsfeeds file:

		acme/newsx:*,!local.*,!junk,!control:Tf,Wf:

	With regards to INN, notice that the trailing `.*' is required.
	The `local.*' could be written as `local*' but this would then
	also include any `localities' hierarchy, for instance.


I have tried to set up a local newsgroup, but newsx insists on fetching
the local group from other servers.

	This surely means that there is something wrong with the 
	configuration in your sys or newsfeeds file. Ensure that the
	entry for every external news server excludes the local groups.

	To find out what actually happens, run newsx with a sufficient
	debug level (e.g. -ddddd). You should see something like:

		group local.something disallowed by newsfeeds


Should I use the distribution field for limiting the distribution of
local newsgroups?

	In general, the answer is a definitive no. The distribution 
	field is a mechanism that is really best suited for limiting
	distribution of specific news articles.


How can I automatically ensure that my local spool carries all available
groups, subject to the limitations of my sys/newsfeeds file?

	You can make newsx produce a list of newsgroups that the
	remote newsserver carries, and feed that list to a small
	script that handles the actual task of local subscriptions.

	This can be combined with normal news fetching by adding the
	option: 

		--newlist "|addgroup.all" 

	This will pipe the list of new groups to the script specified, 
	which incidently is available in the lib subdirectory in the
	newsx distribution.

	On servers with many newsgroups this does take some time, so it 
	will be useful not to do it too often. A standalone effort may
	thus be convenient, e.g.:
		
		newsx acme news.acme.net --newlist "|addgroup.all" \
				 --no-fetch --no-post --locks none


How can newsx be set up to handle the increasing problem of spamming?

	Any of the following methods are suggested:

	   A. Use a news source that does spam filtering.
	   B. Do spam filtering by way of external filters like cleanfeed.
	   C. Use NoCeM to stop spam.
	   D. Do spam filtering in the newsreader by use of kill-files and
	      article scoring mechanisms.

	newsx can be set up to interface with a filter like
	cleanfeed by use of the --filter option.


I've set up filtering, but I've noticed that the articles being
filtered out still are transferred. It surely makes sense not to pick 
up the spam article in the first place?

	The answer to that is not as obvious as it might seem, and
	all depends on how large percentage of the message stream
	that would be stopped by an automated spam filter.

	The streaming mode of operation used by newsx gives a very
	significant transfer speed increase over other methods. It does,
	however, require that articles are transferred in their
	entirety.

	Many of the more useful filtering techniques relies on
	access to the article body anyway.

	Filtering by article header only means that article transfers
	must be handled in a less efficient manner, either by using the 
	overview mechanism (which is not entirely standardized and
	universally available anyway), or by reading the article header 
	and body separately.

	This extra overhead means that the ratio of spam content must be 
	very high to warrant such a scheme in newsx. 

	There is one exception to this, and that is methods involving
	spam killing based on the message ID alone. The --filterm
	option is provided for this purpose.


How can newsx be set up to take advantage of the NoCeM effort?

	The c-nocem package from Olaf Titz can be used with both INN
	and C News. It is available via:

		http://www.inka.de/~bigred/sw/c-nocem.txt

	c-nocem will delete articles that already has reached the
	spool. Just as importantly, it will mark articles not arrived
	yet in the history database.

	Since newsx refers to the history database for every article,
	this does in fact mean that newsx will NOT fetch any articles
	that has already been flagged by NoCeM.

	This method will work better the fresher NoCeM info you can get.
	It will be sensible to fetch and handle the NoCeM newsgroups
	before starting a new ordinary transfer.

	Note that the volume of the NoCeM newsgroups is pretty huge.
	For a small spool, this will mean that the total news
	transfer volume will most probably INCREASE when using NoCeM.
	NoCeM may still serve a useful purpose in reducing the number
	of undesired articles popping up in the newsreader.


How can I make newsx run quicker?

	You can try to increase the size of the window. The default
	value is 10. In some circumstances, it may help to increase
	this, e.g:

		--window 25


I want to limit the number of groups that I actually fetch from a 
particular server. Should I use the -e mechanism for that?

	It depends. It will usually be better to set up your requirements
	in the sys/newsfeeds file. For instance by splitting into several
	incoming spools for a single news source.

	But by all means, the -e approach does work. You must manually
	edit and add a tag line in the server active file found in 
	`/var/spool/news/in.hosts/spool', where `spool' is the name of
	the spool that appears in the sys/newsfeeds file. Rearrange it
	so that all groups you want appears before the tag, and all those
	that you don't want appears after it. The tag line could look like
	this:

		:end

	Remember to use the corresponding command line option:

		-e end

	The thing with the -e option is that since you have limited the 
	number of groups, newsx won't be allowed to start fetching new
	groups that appear in the active file. So you can't simply add 
	groups with addgroup, you will also have to manually create an 
	entry for them in the in.hosts file.


Why did the configure script fail to find the news home directory?

	The configure script goes through a list of probable directory 
	names in some order of priority to find the one you are using, 
	looking for the either an `bin/config' (C News) or
	`innshellvars' (INN) file.

	One reason can be that the directory name you are using is not
	on that list. See the variable VISIT in configure.in

	If your system is using another location, run configure with
	the option --with-newshome="/my/news/home". Or set the environment
	variable NEWSHOME before you invoke configure. Please alse send 
	the relevant information to egil@kvaleberg.no, so that future 
	versions of the newsx configure can be updated accordingly.

	Another reason can be that the configuration file is missing
	altogether, which must be corrected because it will cause trouble
	with other news software.

	It could also be that access to the news home directory has been
	prohibited.


Why did the configure script come up with another news home directory than
the one I am using?

	Presumably, the script have found some traces of a previous news
	installation, or perhaps some symbolic link to the actual news
	directory. Try specifying the news home explicitly with the
	--with-newshome configure option. In the latter case, you may get
	away with keeping the configuration as it is.


Why did the configure say that the configure file is `inaccessible'?

	Usually because it *is* inaccessible. It is commonplace to protect
	access to this configure file from other users than he actual news
	user. The solution here is to install the software as user news, 
	or extend the read permission of the configuration file.


Why did the configure script say I am using INN when I am actually using
C News, or vice versa?

	Presumably, the script have found some traces of a previous news 
	installation, and has become confused.


Why did the configure script come up with another news spool directory 
than the one my system is in fact using?

	The news spool name is read from the news config file. In C News,
	the file is `bin/config' and the variable is NEWSARTS, In INN, the
	file is `innshellvars' and the variable is SPOOL. Either way, if
	this doesn't match the actual directory you are using, there is
	something wrong with the setup of your news server software. 


Why did the configure script come up with another history file than the
one my system is in fact using?

	The reason is most probably that the history database is in a non-
	standard location. 
	
	If you are running INN, this should have been defined properly by
	the news configuration file, so there must be something wrong 
	with news server setup.
	
	If you are using C News, you can either supply the configure
	script with the --with-history=FILE option, append a (non-standard)
	definition of HISTORY to `bin/config', or solve this by
	installing appropriate symbolic links for the files `history',
	`history.dir' and `history.pag' from the news home directory.


Newsx compiled fine, but doesn't work?

	Run newsx from the command line, adding the option `-dddddd'.
	You will then be told what newsx does step by step, and
	finding out what happens should be self-explanatory ;-).


How do I install the binary and manual together with the news binaries?

	By default, newsx is installed in /usr/local/bin and /usr/local/man,
	respectively. To install it in NEWS/bin and NEWS/man, try these
	configure options:

		--exec-prefix=NEWS --prefix=NEWS


How do I make the newsx binary as small as possible?

	Instead of `make install', try:

		make install-strip


Newsx seems to fetch news without complaint, but when I start up my
newsreader I don't see anything?

	The articles are probably still residing in the 
	`/var/spool/news/in.coming'. To move these files to the news
	spool proper, you must do in C News:

		newsrun

	And in INN:

		rnews -U


But the incoming spool is also empty?

	The local server has probably rejected the articles, in which 
	case you will find them in `/var/spool/news/in.coming/bad'.
	Have a look in the logs to find out why they were rejected.
	For C News, look at `/usr/lib/news/errlog'.

	When you have corrected the configuration, do:

		cd /var/spool/news/in.coming
		mv bad/* .

	Finally rerun newsrun/rnews as shown in the previous question
	to try again.


My outgoing posts are not going out. Why?

	Outgoing posts are selected by the newsserver itself, placing a 
	referrence to outgoing articles in the out.going spool for the
	external newsserver.

	It will appear as one text line per outgoing article. Newsx excepts
	the first item on any line to be the file name of the article.

	If a locally posted article does not appear in out.going, this is
	either because the newsfeeds (C news: sys) file is configured wrong, 
	or because rnews -U (C news: newsrun) has not been run yet.


How can I verify that history database lookups are indeed working?

	Find the message ID of any article in the newsspool, and feed it
	to newsx via the -q option. The response should be `yes'. Likewise,
	any non-existing message ID should respond with `no'.


Why doesn't the history lookup function work?  

	The reason is probably that the history database is in a non-
	standard location, see the question about history configuration.
	
	It could also be that the database format is wrong. The standard
	configuration assumes that the news server uses the `dbz V3.4'
	database manager, or the `dbz V6.1.1'. If your system has been
	configured otherwise, you have to manipulate the configuration 
	variables DBZ_VERSION et al.


What shall I do? The `ps' process status doesn't work, and/or crashes
the system.

	Give the --without-ps-status option when running configure, and
	re-make. Also, please run config.guess and report what it says to
	egil@kvaleberg.no so that the configure script can be made to
	disable this feature by default for future releases for the system
	you are using.

	The dynamic process status probably has some portability   
	problems some systems, but is simply too useful to take away.


What on earth is this tiny.server this in included in the distribution?

	One of the special features of newsx is that it can fetch
	news using NNTP relayed over Telnet.

	The point of the tiny.server is to offer people who can log
	onto machines that does not offer any NNTP connections a way 
	of fetching news. Since tiny.server is a Unix shell script, you 
	don't even need access to a C compiler. 

	Note that the shell script idea makes tiny.server quite         
	inefficient. So tiny.server mostly exists for its hack.value.
	Also, only a subset of NNTP is offered.


Does newsx figure out who to send outgoing news to in the same way that
it downloaded news?

	By design of the news system, it is in fact INN (or C News) that
	decides on the outgoing news. newsx simply sends on whatever appears
	in the outgoing queue for the external news server. 

	On the other hand, it is newsx that decides what to fetch. The reason
	why this appears to be a symmetric process is that newsx interprets
	the newsfeeds (or sys) file in the same manner as the rest of the
	news system.


I got a `ctlinnd flush failed' message. What does that mean?

	When running with INN, newsx performs an INN `ctlinnd flush'
	operation. If this operation fails, the above message will appear.
	Any error message from ctlinnd should appear just before this
	message, e.g. `No such site'. Consult the INN documentation for
	more information: `man ctlinnd' might be a good start.


When I fetch articles, I get error messages that says
"newsx: ARTICLE 13225 in misc.test did not contain 'acme' in 'Path:'"

	Take a look at the next line, it should say something in the
	order of:

		newsx: Path line was 'news.acme.net!peer.zoomnews.net!
		       newsfeed.somewhere.edu!news.force.no!not-for-mail'

	The significant part here is the first (or sometimes second) item.
	For INN to properly know where news should be forwarded, all
	remote server entries in your 'newsfeeds' file must be set up
	with the corresponding path 'exclude' name. In this case, probably
	something like:

		acme/news.acme.net:*,!junk*,!control*:Tf,Wf:

	For more information, try:

		man newsfeeds

	For C news, the same applies, except the setup file is called 'sys'.

	By setting the exclude string to `newsx', newsx will add the
	string `newsx' to the path of incoming news. For uncomplicated news
	sites, this is a simple manner of handling this problem.


 In the INN outgoing spool directory, I have found a file with a `.retry'
 suffix. What is that for?

	The file with the `.retry' suffix is used by newsx to handle retries
	of article posting. The exact procedure is:

	 1. Apply lock for other newsxes
	 2. Open outgoing spool
	 3. If a spool.togo exists: rename/append it to spool.retry
	 4. Rename the spool file to spool.togo
	 5. Do a ctlinnd flush spoolname
	 6. If a spool.retry exists: Append it to spool.togo and erase
	 7. When posting: Any articles with problems go to spool.retry
	 8. When finished: Get rid of spool.togo
	 9. Rename spool.retry to spool.togo if it exists


I am trying to understand how newsx is compiled, but I find the Makefile and
configure script very hard to understand. Why?

	That is because these files were never meant for for human 
	consumption or maintenance. They were produced by the GNU tools
	automake and autoconf, respectively. The real source files are
	Makefile.am and configure.in, which should be a bit more readable.
	Especially if referring to the automake and autoconf documentation.


I find the newsx trace log confusing.

	It is especially so since newsx by default runs with windowing
	(streaming) enabled. To get a more logical flow of NNTP requests,
	try running with windowing disabled:

		--window 0


What is the significance of the ME field?

	It depends on which local newsserver you are using.

	In Cnews, the ME field is the local site's general subscription
	list for incoming news.

	In INN, the content of ME is less intuitive, but perhaps more
	useful: Its content is simply prepended to all other entries,
	and bears no direct influence on incoming news that INN will accept.


Does newsx open up several connections in parallell?

	No, it doesn't. The --window option controls the number of
	outstanding requests that may be sent to the other party.


$Id: FAQ,v 1.22 1999/04/08 08:12:12 src Exp $