File: INSTALL

package info (click to toggle)
diablo 1.13-1
  • links: PTS
  • area: non-free
  • in suites: hamm
  • size: 804 kB
  • ctags: 875
  • sloc: ansic: 8,308; perl: 1,908; sh: 186; csh: 81; makefile: 67
file content (301 lines) | stat: -rw-r--r-- 11,829 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301

				INSTALLATION

    NOTICE!  UPGRADING TO *** 1.xx *** FROM VERSIONS <= 1.07 REQUIRES A 
    DHISTORY FILE RELOAD AND COMPLETE BINARY REPLACEMENT.

    READ the section at the end entitled 'UPGRADING TO 1.xx' for instructions,
    at the end of this file.  You do not have to reload the dhistory file if
    you are upgrading from 1.08

    NOTICE!  WHEN UPGRADING TO *** 1.xx *** FROM 1.10 or earlier, you need to
    mess with the addgroup/delgroup stuff in your dnewsfeeds file a little.
    Please see the sample/dnewsfeeds file and the RELEASE_NOTES for more
    information.

    Get and install XMake if you don't have it.  You can get XMake
    from http://www.backplane.com/xmake/

    Edit lib/vendor.h

	To use default hash table size or specifically request a small
	or large hash table.  To maintain dhistory compatibility with
	releases < V1.11, set USE_SMALL_HASH.  To maintain dhistory
	compatibility with releaes > V1.11 from releases < V1.11,
	use USE_LARGE_HASH.  The default is to use a small hash table
	in releases < V1.09, and a large one in releases >= 1.09.  

	>> We suggest that you follow the instructions at the end of this file
	if you wish to upgrade to a larger hash table size. <<

    Edit lib/config.h

	To port to a new architecture or to refine an existing port

    Edit XMakefile.inc

	To set compiler flags

    xmake clean
    xmake

    Create a news home called /news (in recognition that usenet news is
    a major subsystem, we have none of this /usr/local/news crap).  /news
    can be a softlink to wherever the real news home is.

    Create a news userid and a news groupid.

    mkdir /news/dbin
    mkdir /news/spool
    mkdir /news/spool/news
    chown -R news.news /news

    xmake install

    Your news spool partition should be mounted as /news/spool/news.  Be
    sure that both the underlying mount point and the partition is
    properly chown'd and chmod'd.  Do the same with /news.

    cd /news

    copy the contents of the samples directory to /news and edit as
    appropriate, then run rc.news as root.

    Don't forget the adm directory with the sample crontab.  You will have
    to edit the sample crontab as appropriate for your system, and then
    install it for the 'news' user.

    And you are off.

    WHEN UPGRADING:  when upgrading from an earlier release of diablo, always
    diff the source tree adm directory against your installed adm directory
    and update the appropriate files.  Note that for any given release of the
    code, corruption may result if you do not properly update your 
    administrative scripts.


		    /NEWS and /NEWS/SPOOL/NEWS HINTS

		    ** FOR A DEDICATED NEWSFEEDS MACHINE **

    /news should generally own its own partition, as should /news/spool/news.

    /news can usually share the root disk

    /news/spool/news should be given its own private disk(s).  The MINIMUM 
    recommended spool size is 8 GB.

    Under no circumstances should you use physical disks with storage
    capacities greater then 4G.

    The minimum recommended configuration when taking a full feed is:

	128 MB ram, parity SIMM.

	Three 4G seagate barracudas (or better)  Use one for the root 
	drive and /news, and stripe the other two (e.g. with ccd) to make
	your spool.  The suggested stripe size is 2 cylinders (usually 
	around 1024 blocks).

	pentium 90 or better motherboard (I personally suggest a pentium
	pro motherboard, which gives better cpu performance and hardware
	ECC when using parity-simm memory).

	run FreeBSD, BSDI, or Linux.  My personal favorite is FreeBSD.



		    ** FOR A DEDICATED NEWSREADER MACHINE **

    /news should generally own its own partition, as should /news/spool/news.

    /news can reside on the root disk *IF* you have enough memory not to
    swap, otherwise it should own its own disk.  

    /news/spool/news should be given its own private disk(s).  The MINIMUM
    recommended spool size is 16 GB if you are taking a full feed, 8 or 12
    if you are not.

    Under no circumstances should you use physical disks with storage
    capacities greater then 4G.

    The minimum recommended configuration for a newsreader machine taking
    a full feed is:

	128 MB ram, parity SIMMs, 256 MB ram if over 100 readers may become
	active at once.

	Five 4G seagate barracudas (or better).  Use one for the root drive 
	and /news, and stripe the other four (e.g. with ccd) to make your 
	spool.  The suggested stripe size is 2 cylinders (usually around 
	1024 blocks).

	Pentium-pro 200, Adaptec PCI SCSI controller

	running FreeBSD.


		    FREEBSD / PENTIUM BOX / ADAPTEC CONTROLLER

    We use pentium boxes running FreeBSD with seagate drives and an 
    adaptec controller.  We have had serious firmware problems with other 
    drives, especially quantums (even the new ones).  If you are going 
    with a pentium/FreeBSD solution for news, I suggest sticking with
    the newer seagate drives.

    For any news system, you should not underestimate the memory 
    requirements.  Most modern UNIX operating systems are able to use any
    free memory as a disk buffer cache, and that can be very important
    when it comes to news.  News machines cannot generally afford to 
    swap, but it depends heavily on the software you are using.  Diablo,
    for example, can handle swapping/paging much better then something
    like INN.

			CHECKLIST ONCE IT IS RUNNING

    Here's a checklist for once you have Diablo running:

    * Make sure syslog is logging news

    * fgrep DIABLO /var/log/news (or wherever your syslog puts the log) and
      use the uptime and 'arts' numbers to calculate the articles per minute.
      The uptime is in HH:MM, so calculate:  arts / (HH * 60 + MM) .  For
      a full feed as of March 1997, you should wind up with between 
      190 and 250 articles/min if you properly primed the history file.
      If you have not primed the history file and are taking multiple feeds,
      you may see a higher article rate for several days until Diablo's
      history file stabilizes.

    * After DIABLO has been running for a day, check the logs generated by
      the cron scripts.  Make sure that the expire is being run every four
      hours.

    * After DIABLO has been running for a few days, check the logs again.
      Make sure that expire is still running properly, and that the 
      history rewrite (biweekly.rehist) is running properly.

    * After four weeks of operation, double check that biweekly.rehist has
      been running properly and that the dhistory file has stabilized to
      a reasonable size (as of July 1997, between 200 and 300 MBytes).

      If you have a lot of feeds and a lot of memory, it may be worthwhile 
      increasing the dhistory table hash size from 4 million entries to 
      8 million, see adm/biweekly.atrim and the manual page to diload for
      more information.


		UPGRADING TO 1.xx FROM 1.07 OR EARLIER VERSIONS

    Release 1.08 and beyond uses a different dhistory file format... the 
    new format adds a header to the dhistory database file and adds new 
    fields to the History entries to support the new 
    multiple-articles-in-one-file storage scheme (see the release notes for 
    a list of 1.08 features).

    To upgrade to 1.xx from 1.07 or lower, the following steps must be
    taken.  If upgrading to 1.xx from 1.08, you have already done all of
    this and do not need to do anything more.

    (1) disable all cron jobs for news
    (2)	shutdown diablo
    (3) kill all currently running dnewslinks
    (4) ensure that no other diablo-specific processes are running
    (5) remove all binaries in ~news/dbin/

    (6) recompile a whole new set of binaries and install in ~news/dbin/

    (7) make a backup of your current dhistory file and regenerate it, using
	the following sequence:

	mv dhistory dhistory.SAVE
	mv dhistory.param dhistory.SAVE.param

	didump -o dhistory.SAVE | diload -f dhistory

	If you kill the dump/reload before it finishes, you must remove
	dhistory before rerunning the dump/reload.  Do not remove the
	old-format backup for at least a day, just in case.

	WARNING!  Earlier diablo releases could easily corrupt the dhistory
	file.  It is possible that didump -o will not cleanly dump your
	old history file.  If this is the case, see note (11) below.  1.08
	has a built-in check/repair algorithm to handle any corruption
	that might creep in.

    (8a) Compare your cron jobs against the new distribution samples and
	 make appropriate adjustments.

    (8b) Restart diablo

    (9) reenable your cron jobs for the 'news' user

    (10) NOTE regarding dexpire -- the new dexpire runs strictly FIFO and is
	 also single-threaded with the expectation that the article files
	 are now very large.  However, you will still have hundreds of 
	 thousands of smaller article files in your spool from the previous
	 version of diablo.

	 dexpire will expire these files, but it may not be able to remove
	 them quickly enough to prevent your spool from filling up.  We 
	 suggest that you manually rm -rf most of your old spool directories, 
	 running several rm -rf's in parallel on different directories, and 
	 that you watch your spool space carefully until the old spool files
	 have had a chance to cycle out.  If you have some outgoing feeds
	 that are backed up, you may wish to delay removal of some of
	 these directories since dnewslink will still be able to access
	 the old spool files for outgoing feeds.

	 Once the old spool files have cycled out, dexpire should have no
	 trouble keeping up... in fact, it will tend to run 10 times
	 faster then before.

    (11) NOTE regarding irrevocably corrupted dhistory files.. if you wound
	 up destroying your dhistory file trying to upgrade, the best thing
	 to do is to simply remove it and allow diablo to build a new one
	 from scratch.  You will then have the option of allowing it to
	 settle-out on its own (takes around 3 days), or you can reload
	 it from one of your newsreader machine's innd history files using
	 diconvhist.  If you choose the latter, you have the choice of
	 reloading it while diablo is active (you can't use the -f option
	 in this case), or reloading it before turning on diablo (you CAN
	 use the -f option in this case).

	 If you start a new dhistory file from scratch, be aware that your
	 spool will fill around 4 times faster then normal since diablo will
	 basically start accepting everything.  You will have to watch over
	 your spool space carefully for at least 3 days.

    Additional notes:

	* dclean no longer exists, and it will tell you so if you try to run
	  it.

	* for this release, dexpire.ctl has no effect except for entries with
	  expirations of 0 (which rejects an article).

	* dexpire now operates in a strictly FIFO fashion, removing the oldest
	  article files first.  dexpire is now much faster.

	* article files now contain MULTIPLE articles.  In fact, only one file
	  is generated per session.  Diablo generates feed files which include
	  the file offset and number of bytes which dnewslink properly 
	  interprets.

	* innxmit is no longer supported by dspoolout (because it can't handle 
	  file offset / size info)

	Also note that dnewslink will still be able to transfer the older
	spool files.. it detects the difference because the queue file format
	for the feed is different (has no file offset/length info).

    Files to scrap:

	* dexpire.factor is no longer used

	* dhistory.param is no longer used (but the file is still applicable to
	  the backup of the original dhistory file you had before upgrading,
	  so rename it along with the original dhistory file).

	* dclean is now an empty shell, but we suggest leaving it in your
	  crontab in case a use is found for it in some future release.