File: gindexd.doc

package info (click to toggle)
gopher 2.3-2
  • links: PTS
  • area: non-free
  • in suites: hamm
  • size: 2,364 kB
  • ctags: 2,030
  • sloc: ansic: 22,451; perl: 1,950; sh: 1,510; makefile: 397; asm: 1
file content (207 lines) | stat: -rw-r--r-- 6,666 bytes parent folder | download | duplicates (3)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
What is gindexd?
----------------

Gindexd allows you to serve indexes of data residing on a gopher
server.  Gindexd will allow you to search through a voluminous amount
of data quickly.  It is preferred that you use the built in indexing
in the gopher server though, see the gopherd man page for more
information.

As of release 1.0, gindexd is now a symbolic link to gopherd.


How To Set Up An Index Server
-----------------------------

An full-text index is used to rapidly find data in a set of files.
The first step in making a gopher index server is to build the index
on your data files.

For the NeXT this command is called ixBuild.
For the WAIS server this command is called waisindex.

Okay, how about a real world example?  Suppose that you have a
directory structure that contains your grandmother's favorite recipes
located in /home/mudhoney/recipes/.  Also assume that there are
subdirectories for cakes, pies, and stews i.e. :

-------
% pwd
/home/mudhoney/recipes

% ls 
cakes/ pies/  stews/ 

-------

Assume also that you will be running a gopher data server on the
directory /home/mudhoney/recipes.


Building the index with the NeXT and ixBuild.
---------------------------------------------

Go into the directory you want as the root level of your index.  If
you want to index all of the recipes you'd type the following:

  cd /home/mudhoney/recipes
  mkdir .index
  ixBuild -V

This will make an index in the file:

  /home/mudhoney/recipes/.index/index.ixif 

If you wanted to just index the pies subdirectory you would do the
following:

  cd /home/mudhoney/recipes/pies
  mkdir ../.index
  ixBuild -V -i../.index/index.ixif

...or

  cd /home/mudhoney/recipes/pies
  mkdir .index
  ixBuild -V -i.index/index.ixif pies

It's important that the filenames that are generated by ixBuild have
the same format that is given by the gopher server!  You can easily
test this by using ixFind.  Make sure it doesn't contain the entire
path.  Also make sure that when you're indexing a subdirectory that the
previous directories up to the root directory get added to the path by
ixBuild.


Building the Index with waisindex
---------------------------------

With waisindex it isn't necessary to be in the root directory of the
gopher server.  The WAIS indexer stores filenames with their absolute
paths, this causes problems later.

If you want to index the whole recipe collection in
/home/mudhoney/recipes you'd do the following:

  waisindex -r /home/mudhoney/recipes

The waisindex program will create a bunch of files starting with
index.  These files together comprise an index to your data.


If you wanted to just index the pies subdirectory you would do the
following:

  waisindex -r /home/mudhoney/recipes/pies


Starting Up the Index Server
----------------------------

Now that you have indexes you can actually run the gopher index server
on them.  The first step you should do is to move the index files into
a directory all their own.  Let's say that you chose
/home/mudhoney/indexes/recipes.

So, for the NeXT you would:

  mkdir /home/mudhoney/indexes/recipes
  mkdir /home/mudhoney/indexes/recipes/.index
  cp {originaldir}/.index/index.ixif /home/mudhoney/indexes/recipes/.index

For the WAIS version you would:

  mkdir /home/mudhoney/indexes/recipes
  cp index.* /home/mudhoney/indexes/recipes

The next step is very important.  You must create a "hostdata" file in
the same directory as the indexes.  In this case you'd make a file
called "/home/mudhoney/indexes/recipes/hostdata".

In this file you need 2 to 3 pieces of information.  The first line of
the hostdata file should contain the hostname of the machine that is
running a gopher server for the indexed data.  The second line
contains the port that the data server is running at.

If you're running the WAIS version of gindexd then you'll have to add
a third line.  This line should contain the directory of the gopher
data server.  Gindexd will strip this off of all responses it sends
back.  (This is why the WAIS indexing engine causes problems, it puts
in the whole path, not the relative path)

For example, our recipes index "hostdata" file would look like the
following:
 
  mudhoney.micro.umn.edu
  150
  /home/mudhoney/recipes

This assumes that there is a gopher data server running on
mudhoney.micro.umn.edu at port 150.  The nice part about using the
hostdata file is that you can move indexes off to other machines.  Put
the indexes on the big sexy computers.  And put the actual data off on
some sluggish machine.

{As of version 0.7 you can override the default title name returned by 
 gindexd.  Add the keyword "ShowHeadline" at the end of the hostdata
 file.

 Also you can add the keyword "ShowDate" in the hostdata file.  This will
 put the date of the document in the Title.
}

After this step is completed you can actually run the gindexd daemon!
There two required options.  The usage is:

  gindexd [-DI] <indexdirectory>/[databasename]  <portnumber> 

The -p sets the port to run at.  You can use any unused port.  The -i
sets the index directory to look for indexes.  (For the NeXT version
do *not* specify the subdirectory ".index".  Give the directory above
it instead).  The -D specifies that we should run in debug mode.  The
-d parameter is only useful for the Wais indexer, it allows you to
specify a database name other than "index".

The -I parameter should be used when you are starting the indexer from
inetd.  It's also useful when testing whether or not your indexes have
been built successfully.

The -H parameter can be used to specify an alternative hostdata file.

Here is the command we would use to start our recipe index daemon:

  gindexd /home/mudhoney/indexes/recipes 151



Setting Up Links To Index Servers
---------------------------------

This part is pretty simple.  Just create a .link file on your gopher
data server that contains the pertinant information.  Just make sure
that you get the "Type=" line correct.  An index server is itemtype 7.

Here's what we would use for our recipe example:

  Type=7
  Host=mudhoney.micro.umn.edu
  Port=151
  Path=
  Name=Grandma's Recipe Book


By fiddling with the Path= parameter and changing the type to "Type=1"
you can do some interesting things.  When the user accesses the
following list they will get a directory of all the stuff that has
rhubarb pies in it.

  Type=1
  Host=mudhoney.micro.umn.edu
  Port=151
  Path=rhubarb pie
  Name=Grandma's Rhubarb pie recipes.
  
This type of thing is especially usefull for data that changes
rapidly, like news.  We've set up one such link that searches our
campus newspaper for the "What's Doing" schedule of events.