A Tutorial on Gatewaying Mail and News with Sendmail, C-News and newsgate by George M. Jones george@cis.ohio-state.edu August 7, 1991 This document describes how to set up gateways between mailing lists and newsgroups. Some gateways are different. Your mileage may vary. Introduction Mailing lists and newsgroups are similar. Both provide one-to-many communication on selected topics. There are differences in the delivery, storage and display mechanisms. Some people prefer to receive information via news reading mechanisms some people prefer to receive information via mail. News readers tend to have the advantage of being able to categorize information better (by delivering it to different newsgroups) and thus to enabling the user to deal with a higher volume of information. News also has the advantage of reducing storage requirements by allowing all users to read the same copy of a stored message. This document describes how to set up mail/news gateways on UNIX systems running Sendmail 5.61 or later, Cnews and Rich Salz's mail2news and news2mail programs. It is assumed that the local sendmail.cf configuration supports mailing to newsgroups (e.g. mail to "comp.org.usenix" would land in the newsgroup). Theory of operation The basic job that has to be accomplished in mailing lists and newsgroups is to assure that everything that appears in the newsgroup is sent to the mailing list and that everything that is sent to the mailing list appears in the newsgroup, and duplication must be avoided (all messages should appear once, and only once in both the newsgroup and on the mailing list). Files, Programs, and Flags The following files and programs are important in the process of gatewaying mailing lists and newsgroups The sys file Usually /var/lib/news/sys. This file determines what news groups are sent to your "neighbor" sites and what program is called to deliver it to them. Newsgroups can be gatewayed to mailing lists by putting an entry in the sys file specifying which groups are to be sent to the list and specifying the news2mail program (with proper arguments) as the means of getting it there. A typical line sys file entry is internet:gnu.emacs.gnus,!gnu.emacs.gnus.all/all::\ /usr/bin/news2mail info-gnus-english info-gnus-english-mail \ info-gnus-english-request cis.ohio-state.edu See news(5) for more information on the sys file. news2mail News2mail is a program that is responsible for taking a news message, hacking the headers to make it compliant with RFC 822, and sending it out to the list. See news2mail(1) for more information. The aliases file Usually /etc/aliases. This contains various aliases pertaining to the mailing list. Typically, there are aliases of the form list-name: post-list-name if the list is locally maintained, or list-name-local: post-list-name,some,user,names if the list is a local exploder of a list maintained elsewhere. The post-* alias is Typically something like post-list-name: "|/usr/bin/mail2news -n some.group \ -d dist -o '.Gateway .Org .Name' -x internet" With the result that mail sent to "list-name" is posted to the list, or mail sent to "list-name-local" is both posted to the list and sent to several local users. If the mailing list is maintained locally, there will be an alias of the form list-name-request: address-of-list-maintainer to which requests about the list are to be sent. If there are problems in delivering mail to "list-name", and there is an alias of the form owner-list-name: who-gets-the-bad-news then bounces and errors will be sent to the owner of the list, not the sender of the message. Often the addresses of locally maintained mailing lists are maintained in a file, rather than individually in the aliases file. This allows maintenance of mailing list membership by people who do not have the necessary permissions to update the aliases file. These sorts of aliases take the form list-name-mail: :include:/news/mailing-lists/list-name.list Sometimes, the an entry in the included name-list sends messages back to the post-list-name address. So, putting it all together, a real life example is info-gnus-english: post-info-gnus-english@news post-info-gnus-english: "|/usr/bin/mail2news -n gnu.emacs.gnus \ -d gnu -o '.GNUs .Not .Usenet' -x internet" info-gnus-english-mail: :include:/usr/lib/news/info-gnus-english/Aliases info-gnus-english-request: romig owner-info-gnus-english: info-gnus-english-request mail2news mail2news is responsible for taking an RFC822 mail message, hacking on the headers sufficiently to make it acceptable as a news article and handing it off to inews(1). See mail2news(1) for a complete description of the options and header hacking. Problems with NNTP inews Note that mail2news calls inews directly and passes certain flags to it. In the examples above, the -d and -x flags are passed to inews. The use of the -x flag is crucial to avoiding duplication (see below). Also note that NNTP version of inews ("mini-inews") accepts only the -h flag and ignores all others. These two points become extremely relevant if you have news configured in an environment with one machine acting as the central news machine (say, with everyone else NFS mounting the same spool directory) and all other machines normally using NNTP to post. In order for things to work, mail2news has to be able to execute the "real" inews. Solutions There are two solutions. One, hack your sendmail.cf and aliases file such that all mail to mailing lists is delivered to the machine acting as the central news repository and having a copy of the "real" inews, not the NNTP version of inews. This is the solution used in the example above. Sendmail.cf hacking is left as an exercise for the reader. Two, put a shell wrapper around mail2news (or inews) and use rsh(1) to force mail2news (or inews) to execute on the central news machine. This second option can be problematic if you don't have rsh, if not all users are allowed to log in to the central news machine, or if you are not well versed in shell quoting and rsh argument passing mechanisms. Avoiding duplication One of the major issues in mail/news gatwaying is how to avoid duplication. The news software solves the problem of avoiding duplicates in the newsgroups by maintaining a list of the message-ids (the history file) that have been seen and rejecting messages with the same id. Avoiding duplication on mailing lists is somewhat more difficult. You want mail sent to the list to be posted. You want everything posted to be sent back to the list, but you don't want mail that goes through the gateway and is posted to be sent back to the list. The solution as implemented in the examples above is as follows: mail that comes in to the mailing list address is sent through mail2news with the inews -x flag to exclude distribution of the article to a pseudo news neighbor called "internet". The sys file has an entry that sends all articles posted to the group to the pseudo news neighbor "internet" via the news2mail program...soooo...things coming from the list will be posted, but not sent back to the list because of the "-x internet" flag and things posted locally to the group (or transferred to us as news by other neighbors) will be sent to the list. Digital watches are pretty neat ! Another possible solution is to use the distribution header to avoid duplication. The basic idea is to force messages coming in from the mailing list to be posted with a specific distribution (say, by using the "-d fromlist" flag to mail2news in the aliases file entry for the list), and then having the sys file send everything to the list EXCEPT messages with a distribution of "fromlist" (see news(5) for details). The biggest problem with this is that you would then have to pass the "fromlist" distribution to neighbors interested in receiving the gatewayed mailing list as news. Examples The following two examples illustrate how to set up two of the most common types of gateways, a bi-directional gateway for a locally maintained mailing list, and a bi-directional gateway for a mailing list maintained elsewhere. Issues not addressed are one-way gateways, since they are simplifications of the examples below, and propagation of newsgroups to neighbors (you may be gatewaying a popular mailing list for the convenience of local users who prefer it in news, or you may be acting as "the" gateway for a newsgroup that is propagated world wide). These examples can be used as models for setting up new gateways Locally maintained mailing list Aliases file entries info-gnus-english: post-info-gnus-english@news post-info-gnus-english: "|/usr/bin/mail2news -n gnu.emacs.gnus -d gnu -o '.GNUs .Not .Usenet' -x internet" info-gnus-english-mail: :include:/news/mailing-lists/info-gnus-english.list info-gnus-english-request: william owner-info-gnus-english: info-gnus-english-request where the .list file contains a list of email addresses, one per line. sys file entries internet:gnu.emacs.gnus,!gnu.emacs.gnus.all/all::\ /usr/bin/news2mail info-gnus-english info-gnus-english-mail info-gnus-english-request cis.ohio-state.edu Externally maintained mailing list Aliases file entries sun-nets: sun-nets@umiacs.umd.edu sun-nets-local: post-sun-nets@news, tom, richard, harry post-sun-nets: "|/usr/bin/mail2news -n osu.sys.sun.nets -d osu -o '.Sun .Nets .Mailing .List' -x internet" owner-sun-nets-local: sally owner-sun-nets: usenet owner-post-sun-nets: usenet sys file entries internet:osu.sys.sun.nets,!osu.sys.sun.nets.all/all::\ /usr/bin/news2mail sun-nets sun-nets sun-nets-request umiacs.umd.edu It is highly recommended that you check with the maintainer of a mailing list before gatewaying the mailing list into a newsgroup and propagating the newsgroup beyond the local organization. Acknowledgments The setup described here is largely a description of how mail/news gatewaying is done at The Ohio State University Department of Computer and Information Science. Many of the ideas were originaly implenmented by Karl Kleinpaste and Bob Sutterfield using B-News and Eric Fair's nrecnews and gateway scripts.