.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.27.
.TH ARROW "1" "November 2002" "arrow 0.2" "User Commands"
.SH NAME
arrow \- manual page for arrow 
.SH SYNOPSIS
.B arrow
[\fIOPTION\fR...] [\fIARG\fR...]
.SH DESCRIPTION
Arrow is a document retrieval front-end to libbow, it uses TFIDF 
to retrieve relevant documents. 
.SH EXAMPLES
If you have a database of documents in 
.B foo
you would just need to type
.B  arrow \-\-index foo
to create the database. You could then make queries  by typing
.B arrow \-\-query
then typing your query, and pressing Control-D.
.PP
If you want to make many queries, it will be more efficient to run
.B arrow 
as a server, and query it multiple times without restarts by
communicating through a socket.  Type, for example,
.B arrow \-\-query-server=9876
and access it through port number 9876.  For example:
.B telnet localhost 9876
In this mode there is no need to press Control-D to end a query.
Simply type your query on one line, and press return.
.SH OPTIONS
General options
.IP
For building data structures from text files:
.TP
\fB\-i\fR, \fB\-\-index\fR
tokenize training documents found under ARG...,
build weight vectors, and save them to disk
.IP
For doing document retrieval using the data structures built with \fB\-i\fR:
.TP
\fB\-c\fR, \fB\-\-compare\fR=\fIFILE\fR
Print the TFIDF cosine similarity metric of the
query with this FILE.
.TP
\fB\-n\fR, \fB\-\-num\-hits\-to\-show\fR=\fIN\fR
Show the N documents that are most similar to the
query text (default N=1)
.TP
\fB\-q\fR, \fB\-\-query\fR[=\fIFILE\fR]
tokenize input from stdin [or FILE], then print
document most like it
.TP
\fB\-\-query\-forking\-server\fR=\fIPORTNUM\fR
Run arrow in socket server mode, forking a new
process with every connection.  Allows multiple
simultaneous connections.
.HP
\fB\-\-query\-server\fR=\fIPORTNUM\fR Run arrow in socket server mode.
.IP
Diagnostics
.TP
\fB\-\-print\-coo\fR
Print word co-occurrence statistics.
.TP
\fB\-\-print\-idf\fR
Print, in unsorted order the IDF of all words in
the model's vocabulary
.TP
\fB\-\-annotations\fR=\fIFILE\fR
The sarray file containing annotations for the
files in the index
.TP
\fB\-b\fR, \fB\-\-no\-backspaces\fR
Don't use backspace when verbosifying progress
(good for use in emacs)
.TP
\fB\-d\fR, \fB\-\-data\-dir\fR=\fIDIR\fR
Set the directory in which to read/write
word-vector data (default=~/.<program_name>).
.TP
\fB\-\-random\-seed\fR=\fINUM\fR
The non-negative integer to use for seeding the
random number generator
.TP
\fB\-\-score\-precision\fR=\fINUM\fR
The number of decimal digits to print when
displaying document scores
.TP
\fB\-v\fR, \fB\-\-verbosity\fR=\fILEVEL\fR
Set amount of info printed while running;
(0=silent, 1=quiet, 2=show-progess,...5=max)
.IP
Lexing options
.TP
\fB\-\-append\-stoplist\-file\fR=\fIFILE\fR
Add words in FILE to the stoplist.
.TP
\fB\-\-exclude\-filename\fR=\fIFILENAME\fR
When scanning directories for text files, skip
files with name matching FILENAME.
.TP
\fB\-g\fR, \fB\-\-gram\-size\fR=\fIN\fR
Create tokens for all 1-grams,... N-grams.
.TP
\fB\-h\fR, \fB\-\-skip\-header\fR
Avoid lexing news/mail headers by scanning forward
until two newlines.
.TP
\fB\-\-istext\-avoid\-uuencode\fR
Check for uuencoded blocks before saying that
the file is text, and say no if there are many
lines of the same length.
.TP
\fB\-\-lex\-pipe\-command\fR=\fISHELLCMD\fR
Pipe files through this shell command before
lexing them.
.TP
\fB\-\-max\-num\-words\-per\-document\fR=\fIN\fR
Only tokenize the first N words in each document.
.TP
\fB\-\-no\-stemming\fR
Do not modify lexed words with a stemming
function. (usually the default, depending on
lexer)
.TP
\fB\-\-replace\-stoplist\-file\fR=\fIFILE\fR
Empty the default stoplist, and add
space-delimited words from FILE.
.TP
\fB\-s\fR, \fB\-\-no\-stoplist\fR
Do not toss lexed words that appear in the
stoplist.
.TP
\fB\-\-shortest\-word\fR=\fILENGTH\fR Toss lexed words that are shorter than LENGTH.
Default is usually 2.
.TP
\fB\-S\fR, \fB\-\-use\-stemming\fR
Modify lexed words with the `Porter' stemming
function.
.TP
\fB\-\-use\-stoplist\fR
Toss lexed words that appear in the stoplist.
(usually the default SMART stoplist, depending on
lexer)
.TP
\fB\-\-use\-unknown\-word\fR
When used in conjunction with \fB\-O\fR or \fB\-D\fR, captures
all words with occurrence counts below threshold
as the `<unknown>' token
.TP
\fB\-\-xxx\-words\-only\fR
Only tokenize words with `xxx' in them
.IP
Mutually exclusive choice of lexers
.TP
\fB\-\-flex\-mail\fR
Use a mail-specific flex lexer
.TP
\fB\-\-flex\-tagged\fR
Use a tagged flex lexer
.TP
\fB\-H\fR, \fB\-\-skip\-html\fR
Skip HTML tokens when lexing.
.TP
\fB\-\-lex\-alphanum\fR
Use a special lexer that includes digits in
tokens, delimiting tokens only by non-alphanumeric
characters.
.TP
\fB\-\-lex\-infix\-string\fR=\fIARG\fR Use only the characters after ARG in each word for
stoplisting and stemming.  If a word does not
contain ARG, the entire word is used.
.TP
\fB\-\-lex\-suffixing\fR
Use a special lexer that adds suffixes depending
on Email-style headers.
.TP
\fB\-\-lex\-white\fR
Use a special lexer that delimits tokens by
whitespace only, and does not change the contents
of the token at all---no downcasing, no stemming,
no stoplist, nothing.  Ideal for use with an
externally-written lexer interfaced to rainbow
with \fB\-\-lex\-pipe\-cmd\fR.
.IP
Feature-selection options
.TP
\fB\-D\fR, \fB\-\-prune\-vocab\-by\-doc\-count\fR=\fIN\fR
Remove words that occur in N or fewer documents.
.TP
\fB\-O\fR, \fB\-\-prune\-vocab\-by\-occur\-count\fR=\fIN\fR
Remove words that occur less than N times.
.TP
\fB\-T\fR, \fB\-\-prune\-vocab\-by\-infogain\fR=\fIN\fR
Remove all but the top N words by selecting words
with highest information gain.
.IP
Weight-vector setting/scoring method options
.TP
\fB\-\-binary\-word\-counts\fR
Instead of using integer occurrence counts of
words to set weights, use binary absence/presence.
.TP
\fB\-\-event\-document\-then\-word\-document\-length\fR=\fINUM\fR
Set the normalized length of documents when
\fB\-\-event\-model\fR=\fIdocument\-then\-word\fR
.TP
\fB\-\-event\-model\fR=\fIEVENTNAME\fR
Set what objects will be considered the
`events' of the probabilistic model.  EVENTNAME
can be one of: word, document, document-then-word.
.IP
Default is `word'.
.TP
\fB\-\-infogain\-event\-model\fR=\fIEVENTNAME\fR
Set what objects will be considered the `events'
when information gain is calculated.  EVENTNAME
can be one of: word, document, document-then-word.
.IP
Default is `document'.
.TP
\fB\-m\fR, \fB\-\-method\fR=\fIMETHOD\fR
Set the word weight-setting method; METHOD may be
one of: tfidf_words, tfidf_log_words,
tfidf_log_occur, tfidf, default=naivebayes.
.TP
\fB\-\-print\-word\-scores\fR
During scoring, print the contribution of each
word to each class.
.TP
\fB\-\-smoothing\-dirichlet\-filename\fR=\fIFILE\fR
The file containing the alphas for the dirichlet
smoothing.
.TP
\fB\-\-smoothing\-dirichlet\-weight\fR=\fINUM\fR
The weighting factor by which to muliply the
alphas for dirichlet smoothing.
.TP
\fB\-\-smoothing\-goodturing\-k\fR=\fINUM\fR
Smooth word probabilities for words that occur NUM
or less times. The default is 7.
.TP
\fB\-\-smoothing\-method\fR=\fIMETHOD\fR
Set the method for smoothing word
probabilities to avoid zeros; METHOD may be one
of: goodturing, laplace, mestimate, wittenbell
.TP
\fB\-\-uniform\-class\-priors\fR When setting weights, calculating infogain and
scoring, use equal prior probabilities on classes.
.TP
-?, \fB\-\-help\fR
Give this help list
.TP
\fB\-\-usage\fR
Give a short usage message
.TP
\fB\-V\fR, \fB\-\-version\fR
Print program version
.PP
Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.
.SH "REPORTING BUGS"
Please report bugs related to this program to Andrew McCallum
<mccallum@cs.cmu.edu>. If the bugs are related to the Debian package 
send bugs to submit@bugs.debian.org
.SH "SEE ALSO"
.BR archer (1),
.BR crossbow (1),
.BR rainbow (1).
.PP
The full documentation for
.B arrow
will be provided as a Texinfo manual.  If the
.B info
and
.B arrow
programs are properly installed at your site, the command
.IP
.B info arrow
.PP
should give you access to the complete manual.
.PP 
You can also find documentation and updates for 
.B libbow 
at http://www.cs.cmu.edu/~mccallum/bow