.Id $Id: gopherd.8,v 3.9 1995/11/03 21:20:06 lindner Exp $
.TH GOPHERD 8 Local
.SH NAME
gopherd - serve data using the gopher protocol.
.SH SYNOPSIS
.B gopherd
.RB [ -DCIc ] [-o 
.IR optionfile ]
.RB [ -L
.IR loadavg ]
.RB [ -l
.IR logfile ]
.RB [ -u
.IR userid ]
.RB [ -U
.IR uid ]
.B
.I Datadir
.I Port
.SH OPTIONS
.I Datadir
is a directory where the information for the gopher server resides

.I Port
is the socket port to use (or if running under inetd, the port that
the server is running under.)

.I 
Changing the name of the gopherd executable will change how gopherd
will behave.  If invoked as 
.B gopherls
, the program will list out the Datadir to the screen, processing
all the links and .cap things.  If invoked as gindexd, the program can
serve up the single index Datadir.  We discourage the use of gindexd, 
since it adds an extra process that must be run.  It is included for
backwards compatibility.

.B -D
Enable copious debugging output.
.LP
.B -L
.RS 3
If the server is compiled with the LOADRESTRICT option then this option
sets the load average at which connections are turned away.
.RE
.LP
.B -C
Disables caching of directory requests.
.LP
.B -l
Log connections to 
.I logfile
.LP
.RS 3
If a logfile is specified, the server will log the time, host and
transaction of each connection.  If there isn't a -l option passed to
gopherd then no logging is done.
.RE
.LP
.B -I
Specify when running under inetd
.RS 3

This is also useful for testing server code changes.
.RE
.LP
.B -o
Specifies an alternative/optional "gopherd.conf" file.  See the
gopherd.conf(5) man page for more information about the format of this
file.
.LP
.B -C
Turns server side caching 
.B off
.RS 3

The caching mechanism works as follows:  The server checks for a file called
.B .cache
when doing a directory lookup.  If it finds a relatively new one,
it blasts it across the
connection, no sorting required.  If it doesn't find one, or finds an old one
it re-sorts through the directory and makes a new .cache file.

.RE
.LP
.B -c
Do not
.IR chroot(2)
before processing connections. (Waterlooism)

.RS 3
This relies on code that (tries to) ensure(s)
that files outside the data directory cannot be retrieved.  This
has the potential to be a security hole.  Use with care, and with the
.B -u
or
.B -U
options.  Also, if you are using it with inetd, be sure to specify the
-I flag *first*.

This is very useful if you wish to have symbolic links to files outside
the data directory available through the gopher server.
For example, you might make a symlink to some local man pages, to
avoid duplicating the information.

If you are running
.B without
the -c flag you'll find that certain things might not work right as
you expect.  These are situations where you need to have some binary
executed under the
.IR gopherd ,
or need to have links to files which in reality are outside the gopher
data root.
Such cases include for example local man-pages etc.  Symlinking them is
no answer, either a physical link into another directory on same disk,
or using a loop-back filesystem (where available) are the ways to do.

.RE

.LP
.B -u
Run as user
.IR userid .
(Waterlooism)

.RS 3
The server is run with reduced permissions (i.e. other than root) which
can ensure that only publically readable files are available from the server.

This option can be used with or without the -c flag.  If you're
running a gopher server on top of an anonymous ftp directory you'll
want to run with the option "-u ftp" to preserve the same level of
security as anonymous ftp.

.RE

.LP
.B -U
Like -u, but takes a numeric uid instead. (Waterlooism)


.SH DESCRIPTION
.IX  "gopherd command"  ""  "\fLgopherd\fP \(em Serve data using gopher protocol"
.B Gopherd
serves data using the Gopher protocol.  The 
.I gopherd 
.B 
server understands how to serve text and directory objects.
(For indexes see gindexd(8))  The server also supports the creation of 
"Links".  Thus it is possible to represent Telnet sessions, CSO servers, etc.

The gopher server also contains enough intelligence to represent mail spool
files as directories.

.SH ADDING INFORMATION TO THE SERVER

.LP
The gopher server gets its information from the files and directories
in 
.I Datadir.
By making changes in this directory tree, you will change what the
gopher clients see.

.SS DEFAULT TRANSALTIONS

NOTE: The defaults mentioned in this section apply to the gopherd.conf
file that is distributed.  If you change this file, then this default
behavior will be different.

All files/directories that start with a dot, or are named
.I etc, usr, bin, dev, core 
are ignored or processed specially.

The title of each object presented to the client is the filename of
the file/directory.

Directories in the tree are served as Gopher directory objects.  

Text files are represented as Gopher text objects.

Ulaw sound files are are represented as sound objects.

Compressed files are assumed to be text, and have their titles
changed to remove the".Z" extension.

Mail spool files are represented as directories.  When the client
attempts to view the mail-spool file, the Subject lines are presented
as the title of each mail message within the file.

Executable shell scripts are assumed to be of text object type.  When
the client requests to view the shell-script, the script is run, and
the results sent to the client.

Files that end with .html are assumed to be WWW hypertext files.  They are
served up with type 'h'.

Files that end with .src are assumed to be WAIS searches.  If the server has
WAIS indexing compiled in, then these will show up as a search engine/

The server sorts directories it sends out alphabetically.  The server
does distinguish between upper and lower case when sorting.

Consult the gopherd.conf file and the gophed.conf(5) manual page for more
information about translations of filename extensions.

.SS LINKS

You can override the defaults by creating what are known as 
.I Links
in the Gopher data directory tree.


The ability to make links to
other hosts is how gopher distributes itself among multiple hosts.
There are two different ways to make a link.  The first and simplest
is to create a link file that contains the data needed by the server.
By default all files in the gopher data directory starting with a
period are taken to be link files.  A link file can contain multiple
links.  To define a link you need to put five lines in a link file that
define the needed characteristics for the document.  Here is an
example of a link.

.RS 3
 Name=Cheese Ball Recipes
 Numb=1
 Type=1
 Port=150
 Path=1/Moo/Cheesy
 Host=zippy.micro.umn.edu
.RE

The Name= line is what the user will see when cruising through the
database.  In this case the name is "Cheese Ball Recipes".  The
"Type=" defines what kind of document this object is.  The following
is a list of all the defined types:

.RS 3
 -  -- Ignore this entry (for .cap/XX -files)
 0  -- Text File
 1  -- Directory
 2  -- CSO name server
 4  -- Mac HQX file.
 5  -- PC binary
 7  -- Full Text Index (Gopher menu)
 8  -- Telnet Session
 9  -- Binary File
 s  -- Sound
 e  -- Event    (not in 2.06)
 I  -- Image (other than GIF)
 M  -- MIME multipart/mixed message
 T  -- TN3270 Session
 c  -- Calendar (not in 2.06)
 g  -- GIF image
 h  -- HTML, HyperText Markup Language
.RE

The "Path=" line contains the selector string that the client will use
to retrieve the actual document.  The Numb= specifies that this entry
should be presented first in the directory list (instead of being
alphabetized).  The "Numb=" line is optional.  If it is present it
cannot be the last line of the link.  The "Port=" and "Host=" lines
specify a fully qualified domain name (FQDN) and a port respectively.
You may substitute a plus '+' for these two parameters if you wish.
The server will insert the current hostname and the current port when
it sees a plus in either of these two fields.

An easy way to retrieve links is to use the Curses Gopher Client.  By
pressing '=' You can get information suitable for inclusion in a link
file. 

.SS OVERRIDING DEFAULTS

If you define CAPFILES for SERVEROPTS in Makefile.config then the
server looks for a directory called
.I .cap
when parsing a directory.  The server then checks to see if the .cap
directory contains a file with the same name as the file it's
parsing.  If this file exists then the server will open it for
reading.  The server parses this file just like a link file.  However
instead of making a new object, the parameters inside the .cap/ file
are used to override any of the server supplied default values. 

For instance say you wanted to change the Title of a text file for
gopher, but don't want to change the filename.  You also don't want it
alphabetized, instead you want it second in the directory listing.
You could make a set-aside file in the .cap directory with the same
filename that contained the following lines:

.RS 3
 Name=New Long Cool Name
 Numb=2
.RE

The replacement (and default) for .cap files are extended link files.
The equivilant is to create a file that begins with a dot (.) in the
.I same
directory as the file you wish to override.  If the name of the file
was 
.I "file-to-change"
then you could create a file called
.I .names
with the following contents

.RS 3
 Path=./file-to-change
 Name=New Long Cool Name
 Numb=2
.RE


.SS ADDING COOL LINKS

One cool thing you can do with .Links is to add neato services to your
gopher server.  Adding a link like this:

.RS 3
 Name=Cool ftp directory
 Type=1
 Path=ftp:hostname@/path/
 Host=+
 Port=+
 
 Name=Cool ftp file
 Type=0
 Path=ftp:hostname@/path/file
 Host=+
 Port=+
.RE

Will allow you to link in any ftp site into your gopher.  Make sure
that there is a /tmp directory to store the files for the gateway.
Note that if you're running without the -c option, you must create a
"tmp" directory at the top level of the gopher-data directory.

You can easily add a finger site to your gopher server thusly:

.RS 3
 Name=Finger information
 Type=0
 Path=lindner
 Host=mudhoney.micro.umn.edu
 Port=79
.RE

Another neat thing you can do is to execute shell scripts:

.RS 3
 Name=Execed command name
 Type={a type}
 Path=exec:"args":/scriptname
 Host=+
 Port=+
.RE

This is usually used by other types of gateway scripts.  For instance,
The first script might take a search and get a few hits.  It could then
generate "exec" scripts that would retrieve the actual document the hit
referred to.

Note that the scriptname *must* begin with the magic character "#!/".
It also must be executable.

.SS HIDING AN ENTRY

This kind of trick may be necessary in some cases, when gopherd.conf's
"ignore:" tags don't, or can't cover something specific, and thus for
object "fred", the overriding .names file entry would be:

.RS 3
 Type=X
 Path=./fred
.RE

by overriding default type to be "X".
This kind of hideouts may be usefull, when for some reason there are
symlinks (or whatever) in the directory at which the Gopherd looks at,
and those entries are not desired to be shown at all.

.SH FULL TEXT INDEXES

As of the 0.8 release of gopher, full text indices can be accessed
directly using gopherd.  Just put the indexes someplace in the
gopher-data directory and then create a link.  For example, assume you
have a directory that contains some indexes called "fred" in the top
level of your Gopher data directory.

.SS Making a WAIS link

To make the "fred" directory into a WAIS full text index searcher, add
the following lines to .cap/fred

.RS 3
 Type=7
 Path=7/fred/index
.RE 

This will change the type into a full text index (7).  The Path=
format for a WAIS index is "7{path}/{dbname}".

.SS Making a NeXT Digital Librarian link

To make the "fred" directory into a NeXT full text index searcher,
build an index in the index directory and add the following lines to
.cap/fred

.RS 
 Type=7
 Path=7/fred
.RE

This will change the type into a full text index (7).  The Path=
format for a NeXT index is "7{path}".

.SH Merging multiple indexes (formerly mindexd)

.PP
The gopher server can spread an index query to multiple search items,
even items on other hosts.  This allows one to split indexes into
smaller, more managable pieces, and take advantage of a sort of
fine-grain parallelism.

A multiple search is specified as follows.  A file ending with .mindex
is placed on the server, inside this file are lines like the
following: 

.RS 3
 <host.domain|localhost> <port> <pathname>
.RE

Here's an example:

.RS 3
 #
 #Recipes
 #
 localhost 70 7/indexes/otherrecipes
 ashpool.micro.umn.edu 70 7/indexes/recipes
 #
 # computer info
 joeboy.micro.umn.edu 70 7/indexes/computer
.RE

The server displays this to the user as a single searchable item.  A
search to this item will contain the results of searches on the two
databases.

If you specify "localhost" as the hostname then the server will do the
search directly on the local machine in series.  Otherwise a copy of
the gopher daemon will be forked for each index.  Using localhost is
very efficient, and should be used when the indexes to be searched are
on the local hard disk.

.SH GOPHER+ FORMS (a.k.a. ASK BLOCKS)

Gopher+ forms allow you to request a variety of data from a person
using a Gopher+ client.  When the user completes the form, the
responses are fed back to the gopher server for processing.

Ask blocks are implemented in two parts.
.RS 3
The first is a file that defines the form layout.
.RE
.RS 3
The second is a command script that acts on the responses 
returned to the server.
.RE
.RS 3

If you want an ask block named "framis"
.RS 3

 framis.ask    holds the form definition
 framis        holds the script
.RE
.RE

If either is missing, the item is ignored.

.B FORM ITEMS

.TP
.I   Ask: prompt [<TAB> defaultvalue]
displays all text in the prompt up to about 40 chars.
and solicits a one-line of input.

.TP
.I AskP: prompt [<TAB> defaultvalue]
does same but does not echo what you type.

.TP
.I AskL: prompt
prompts for a multi-lined response.

.TP
.I Note: text
displays "text" on the screen.

.TP
.I Select: prompt
displays prompt followed by a "check box".

.TP
.I Choose: prompt <TAB> choice <TAB> choice ...
displays prompt and list of choices in radio-button format.  The first
item is the default item for the list.
.LP
.LP
.B SERVER SCRIPT.
.LP
First line of script must contain the launching command (e.g.    #!/bin/sh  ) 
Otherwise, the server will not act on the script.  The script must
also have the execute bit turned on.
.LP
Each response from the form is fed to the script one line at a time.
This allows you to simulate user input by just running your script and
typing the following items for each line of the FORM code.
.TP
.TP
.I Ask and AskP
returns the user typed text as one line
.TP
.INote
items are ignored.
.TP
.I Select
returns
 1 if selected,
 0 otherwise
.TP
.I Choose
 returns the name of the item chosen from the list.
.TP
.I AskL
 sends the number of lines that were typed first, followed by
the actual typed multi-line text.  The script should read the number
and then read the next n lines to get the whole response.
.P
If your script contains any "echo" commands or other screen output,
that output is returned to the client as a text document.  Use this to
return feedback to the user, or as a thank-you note.

.SH WWW and HTML support.

Gopherd can be queried by WWW clients in the hypertext mode.  The server
does not currently cache the HTML pages it makes.  You can include HTML stuff
that will appear above the directory by making a file 
called \*(lq.about.html\*(rq The gopher server will send this before the menu.

References to Gopherd-WWW pages are as follows:

.RI "gopher://" "hostname.domainname" ":" "port" "/hh/" "gopherpath"


.SH EXAMPLES

/usr/local/etc/gopherd /home/gopher-data 70

This command will serve data from the directory 
.I /home/gopher-data
.B on port
.I 70
.B

The gopher daemon forks off and will then start accepting connections.
You can easily test the server by using telnet to connect to the port
specified on the command line.  Once connected, type return.  A list
of things in the gopher-data directory should be returned.  For
instance let's say that the server was started on the machine
mudhoney.  To test it you'd do the following: 

.RS 3
telnet mudhoney 70
.RE

Here is an example of running the gopherd server from inetd.  This
example assumes that you have the files /etc/inetd.conf and
/etc/services.  Other machines and architectures may vary.  Assume
that we have a directory /gopher-data and we want to run at the
standard port for gopher (70).

 In /etc/services add the following line:

gopher 70/tcp

In /etc/inetd.conf add the following line:

 gopher stream tcp nowait root /etc/gopherd gopherd -I /datadir 70

AIX 3.2 users take note; when adding gopher to /etc/inetd.conf, the
command string must be 50 characters of less.  This is because under
AIX the inetd information is stored in the ODM object InetServ in
/etc/objrepos.  If you use more than 50 characters, the command will
just be truncated when /etc/services and /etc/inetd.conf get imported
into /etc/objrepos/InetServ by the inetimp command.

For reference you can get the description of the InetServ object
class by going 'cd /etc/objrepos' and 'odmshow InetServ'.

To see the contents of the InetServ object itself, go into
/etc/objrepos and do 'odmget InetServ'.



.SH FULL-TEXT INDEX EXAMPLES

An full-text index is used to rapidly find data in a set of files.
The first step in making a gopher server with indexes is to build the
index on your data files.

For some canned indexing scripts look in the misc/indexing directory.
They might be more intelligible than these instructions.

.RS 3
When using the NeXT digital librarian this command is 
.B ixBuild
, when using the WAIS software this command is
.B waisindex
.RE

Okay, how about a real world example?  Suppose that you have a
directory structure that contains your grandmother's favorite recipes
located in /home/mudhoney/recipes/.  Also assume that there are
subdirectories for cakes, pies, and stews i.e. :

.RS 3
 % pwd
 /home/mudhoney/recipes
 
 % ls 
 cakes/ pies/  stews/ 
.RE	

In this example we will be running a gopher server on the directory
/home/mudhoney/recipes.


.SS Building the index with the NeXT and ixBuild.

Go into the directory you want as the root level of your index.  If
you want to index all of the recipes you'd type the following:

.RS 3
 cd /home/mudhoney/recipes
 mkdir .index
 ixBuild -Vv
.RE

This will make an index in the file:

.RS 3
 /home/mudhoney/recipes/.index/index.ixif 
.RE

If you wanted to just index the pies subdirectory you would do the
following:

.RS 3
 cd /home/mudhoney/recipes/pies
 mkdir ../.index
 ixBuild -V -i../.index/index.ixif
.RE

or...

.RS 3
 cd /home/mudhoney/recipes/pies
 mkdir .index
 ixBuild -V -i.index/index.ixif pies
.RE

It's important that the filenames that are generated by ixBuild have
the same format that is given by the gopher server!  You can easily
test this by using 
.B ixFind
.  Make sure the filename returned doesn't contain the entire
path.  Also make sure that when you're indexing a subdirectory that the
previous directories up to the gopher data directory get added to the path by
ixBuild.


.SS Building the Index with waisindex

With waisindex it isn't necessary to be in the root directory of the
gopher server.  The WAIS indexer stores filenames with their absolute
paths, this causes problems later.

If you want to index the whole recipe collection in
/home/mudhoney/recipes you'd do the following:

.RS 3
 mkdir /home/mudhoney/recipes/.index
 cd /home/mudhoney/recipes/.index
 waisindex -r /home/mudhoney/recipes
.RE 

The waisindex program will create a bunch of files starting with
index.  These files together comprise an index to your data.

If you wanted to just index the pies subdirectory you would do the
following:

.RS 3
 mkdir /home/mudhoney/recipes/.index
 cd /home/mudhoney/recipes/.index
 waisindex -r /home/mudhoney/recipes/pies
.RE

.SS Linking in the Sample indexes

Both of these indexes are in a directory that starts with a period.
Thus the server will ignore these directories when sending out
information.  So now you need to make a link to these indexes so
people can start searching them.  For the NeXT you would put this in a
file
.B /home/mudhoney/recipes/.Link

.RS 3
 Type=7
 Name=Recipe Index
 Port=+
 Host=+
 Path=7/
.RE

For a WAIS based gopher server you would put the following into the
file 
.B /home/mudhoney/recipes/.Link

.RS 3
 Type=7
 Name=Recipe Index
 Host=+
 Port=+
 Path=7/.index/index
.RE

.SS Changing the defaults for full text indexes...

You may want to change what gets returned for a full text index.
Especially if you are storing indexes on a machine that doesn't have
the actual data.  In this case you can use what is known as a "hostdata"
file to change what the server returns given a full-text search.

The server will look for a file called "<indexname>.hostdata" to get
information from.  If it doesn't find this file it will try just
"hostdata".

A hostdata file consists of a text file with three lines in the
following order:

.RS 3
 <hostname>
 <portnum>
 <path to strip>
.RE

These fields are used to override the defaults.  For instance, the
following hostdata file would substitute the hostname
"goober.micro.umn.edu" for the current server host, the port 8000 for
the current server port, and strip off the path /home/goober from the
pathname entries in the index:

.RS 3
 goober.micro.umn.edu
 8000
 /home/goober
.RE


.LP
.SH "SEE ALSO"
.BR gopher (1);
.BR gopherd.conf (5);
.IR "The Internet Gopher Protocol" ", March 1993, RFC 1436"