.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.27.
.TH CROSSBOW "1" "November 2002" "crossbow" "User Commands"
.SH NAME
crossbow \-  a front-end with hierarchical clustering and deterministic annealing
.SH SYNOPSIS
.B crossbow
[\fIOPTION\fR...] [\fIARG\fR...]
.SH DESCRIPTION
Crossbow is document clustering front-end to libbow. This brief manpage
was written for the Debian GNU/Linux distribution since there is none
available in the main package.
.PP
Note that 
.B crossbow
is not a supported program.
.SH OPTIONS
.IP
For building data structures from text files:
.TP
\fB\-\-build\-hier\-from\-dir\fR
When indexing a single directory, use the
directory structure to build a class hierarchy
.TP
\fB\-c\fR, \fB\-\-cluster\fR
cluster the documents, and write the results to
disk
.TP
\fB\-\-classify\fR
Split the data into train/test, and classify the
test data, outputing results in rainbow format
.TP
\fB\-\-classify\-files\fR=\fIDIRNAME\fR
Classify documents in DIRNAME, outputing
`filename classname' pairs on each line.
.TP
\fB\-\-cluster\-output\-dir\fR=\fIDIR\fR
After clustering is finished, write the
cluster to directory DIR
.TP
\fB\-i\fR, \fB\-\-index\fR
tokenize training documents found under ARG...,
build weight vectors, and save them to disk
.TP
\fB\-\-index\-multiclass\-list\fR=\fIFILE\fR
Index the files listed in FILE.  Each line of FILE
should contain a filenames followed by a list of
classnames to which that file belongs.
.TP
\fB\-\-print\-doc\-names\fR[=\fITAG\fR]
Print the filenames of documents contained in
the model.  If the optional TAG argument is given,
print only the documents that have the specified
tag.
.TP
\fB\-\-print\-matrix\fR
Print the word/document count matrix in an awk- or
perl-accessible format.  Format is sparse and
includes the words and the counts.
.TP
\fB\-\-print\-word\-probabilities\fR=\fIFILEPREFIX\fR
Print the word probability distribution in each
leaf to files named FILEPREFIX-classname
.TP
\fB\-\-query\-server\fR=\fIPORTNUM\fR Run crossbow in server mode, listening on socket
number PORTNUM.  You can try it by executing this
command, then in a different shell window on the
same machine typing `telnet localhost PORTNUM'.
.TP
\fB\-\-use\-vocab\-in\-file\fR=\fIFILENAME\fR
Limit vocabulary to just those words read as
space-separated strings from FILE.
.IP
Splitting options:
.TP
\fB\-\-ignore\-set\fR=\fISOURCE\fR
How to select the ignored documents.  Same format
as \fB\-\-test\-set\fR.  Default is `0'.
.TP
\fB\-\-set\-files\-use\-basename\fR[=\fIN\fR]
When using files to specify doc types, compare
only the last N components the doc's pathname.
That is use the filename and the last N-1
directory names.  If N is not specified, it
defaults to 1.
.TP
\fB\-\-test\-set\fR=\fISOURCE\fR
How to select the testing documents.  A number
between 0 and 1 inclusive with a decimal point
indicates a random fraction of all documents.  The
number of documents selected from each class is
determined by attempting to match the proportions
of the non-ignore documents.  A number with no
decimal point indicates the number of documents to
select randomly.  Alternatively, a suffix of `pc'
indicates the number of documents per-class to
tag.  The suffix 't' for a number or proportion
indicates to tag documents from the pool of
training documents, not the untagged documents.
`remaining' selects all documents that remain
untagged at the end.  Anything else is interpreted
as a filename listing documents to select.
Default is `0.0'.
.TP
\fB\-\-train\-set\fR=\fISOURCE\fR
How to select the training documents.  Same format
as \fB\-\-test\-set\fR.  Default is `remaining'.
.TP
\fB\-\-unlabeled\-set\fR=\fISOURCE\fR How to select the unlabeled documents.
Same
format as \fB\-\-test\-set\fR.  Default is `0'.
.TP
\fB\-\-validation\-set\fR=\fISOURCE\fR
How to select the validation documents.  Same
format as \fB\-\-test\-set\fR.  Default is `0'.
.IP
Hierarchical EM Clustering options:
.TP
\fB\-\-hem\-branching\-factor\fR=\fINUM\fR
Number of clusters to create.  Default is 2.
.TP
\fB\-\-hem\-deterministic\-horizontal\fR
In the horizontal E-step for a document, set to
zero the membership probabilities of all leaves,
except the one matching the document's filename
.TP
\fB\-\-hem\-garbage\-collection\fR
Add extra /Misc/ children to every internal
node of the hierarchy, and keep their local word
distributions flat
.TP
\fB\-\-hem\-incremental\-labeling\fR
Instead of using all unlabeled documents in
the M-step, use only the labeled documents, and
incrementally label those unlabeled documents that
are most confidently classified in the E-step
.TP
\fB\-\-hem\-lambdas\-from\-validation\fR=\fINUM\fR
Instead of setting the lambdas from the
labeled/unlabeled data (possibly with LOO),
instead set the lambdas using held-out validation
data.  0<NUM<1 is the fraction of unlabeled
documents just before EM training of the
classifier begins.  Default is 0, which leaves
this option off.
.TP
\fB\-\-hem\-max\-num\-iterations\fR=\fINUM\fR
Do no more iterations of EM than this.
.TP
\fB\-\-hem\-maximum\-depth\fR=\fINUM\fR
The hierarchy depth beyond which it will not
split.  Default is 6.
.TP
\fB\-\-hem\-no\-loo\fR
Do not use leave-one-out evaluation during the
E-step.
.TP
\fB\-\-hem\-no\-shrinkage\fR
Use only the clusters at the leaves; do not do
anything with the hierarchy.
.TP
\fB\-\-hem\-no\-vertical\-word\-movement\fR
Use EM just to set the vertical priors, not to set
the vertical word distribution; i.e. do not to
`full-EM'.
.TP
\fB\-\-hem\-pseudo\-labeled\fR
After using the labels to set the starting point
for EM, change all training documents to
unlabeled, so that they can have their class
labels re-assigned by EM.  Useful for imperfectly
labeled training data.
.TP
\fB\-\-hem\-restricted\-horizontal\fR
In the horizontal E-step for a document, set to
zero the membership probabilities of all leaves
whose names are not found in the document's
filename
.TP
\fB\-\-hem\-split\-kl\-threshold\fR=\fINUM\fR
KL divergence value at which tree leaves will be
split. Default is 0.2
.TP
\fB\-\-hem\-temperature\-decay\fR=\fINUM\fR
Temperature decay factor.  Default is 0.9.
.TP
\fB\-\-hem\-temperature\-end\fR=\fINUM\fR
The final value of T.  Default is 1.
.TP
\fB\-\-hem\-temperature\-start\fR=\fINUM\fR
The initial value of T.
.IP
General options
.TP
\fB\-\-annotations\fR=\fIFILE\fR
The sarray file containing annotations for the
files in the index
.TP
\fB\-b\fR, \fB\-\-no\-backspaces\fR
Don't use backspace when verbosifying progress
(good for use in emacs)
.TP
\fB\-d\fR, \fB\-\-data\-dir\fR=\fIDIR\fR
Set the directory in which to read/write
word-vector data (default=~/.<program_name>).
.TP
\fB\-\-random\-seed\fR=\fINUM\fR
The non-negative integer to use for seeding the
random number generator
.TP
\fB\-\-score\-precision\fR=\fINUM\fR
The number of decimal digits to print when
displaying document scores
.TP
\fB\-v\fR, \fB\-\-verbosity\fR=\fILEVEL\fR
Set amount of info printed while running;
(0=silent, 1=quiet, 2=show-progess,...5=max)
.IP
Lexing options
.TP
\fB\-\-append\-stoplist\-file\fR=\fIFILE\fR
Add words in FILE to the stoplist.
.TP
\fB\-\-exclude\-filename\fR=\fIFILENAME\fR
When scanning directories for text files, skip
files with name matching FILENAME.
.TP
\fB\-g\fR, \fB\-\-gram\-size\fR=\fIN\fR
Create tokens for all 1-grams,... N-grams.
.TP
\fB\-h\fR, \fB\-\-skip\-header\fR
Avoid lexing news/mail headers by scanning forward
until two newlines.
.TP
\fB\-\-istext\-avoid\-uuencode\fR
Check for uuencoded blocks before saying that
the file is text, and say no if there are many
lines of the same length.
.TP
\fB\-\-lex\-pipe\-command\fR=\fISHELLCMD\fR
Pipe files through this shell command before
lexing them.
.TP
\fB\-\-max\-num\-words\-per\-document\fR=\fIN\fR
Only tokenize the first N words in each document.
.TP
\fB\-\-no\-stemming\fR
Do not modify lexed words with a stemming
function. (usually the default, depending on
lexer)
.TP
\fB\-\-replace\-stoplist\-file\fR=\fIFILE\fR
Empty the default stoplist, and add
space-delimited words from FILE.
.TP
\fB\-s\fR, \fB\-\-no\-stoplist\fR
Do not toss lexed words that appear in the
stoplist.
.TP
\fB\-\-shortest\-word\fR=\fILENGTH\fR Toss lexed words that are shorter than LENGTH.
Default is usually 2.
.TP
\fB\-S\fR, \fB\-\-use\-stemming\fR
Modify lexed words with the `Porter' stemming
function.
.TP
\fB\-\-use\-stoplist\fR
Toss lexed words that appear in the stoplist.
(usually the default SMART stoplist, depending on
lexer)
.TP
\fB\-\-use\-unknown\-word\fR
When used in conjunction with \fB\-O\fR or \fB\-D\fR, captures
all words with occurrence counts below threshold
as the `<unknown>' token
.TP
\fB\-\-xxx\-words\-only\fR
Only tokenize words with `xxx' in them
.IP
Mutually exclusive choice of lexers
.TP
\fB\-\-flex\-mail\fR
Use a mail-specific flex lexer
.TP
\fB\-\-flex\-tagged\fR
Use a tagged flex lexer
.TP
\fB\-H\fR, \fB\-\-skip\-html\fR
Skip HTML tokens when lexing.
.TP
\fB\-\-lex\-alphanum\fR
Use a special lexer that includes digits in
tokens, delimiting tokens only by non-alphanumeric
characters.
.TP
\fB\-\-lex\-infix\-string\fR=\fIARG\fR Use only the characters after ARG in each word for
stoplisting and stemming.  If a word does not
contain ARG, the entire word is used.
.TP
\fB\-\-lex\-suffixing\fR
Use a special lexer that adds suffixes depending
on Email-style headers.
.TP
\fB\-\-lex\-white\fR
Use a special lexer that delimits tokens by
whitespace only, and does not change the contents
of the token at all---no downcasing, no stemming,
no stoplist, nothing.  Ideal for use with an
externally-written lexer interfaced to rainbow
with \fB\-\-lex\-pipe\-cmd\fR.
.IP
Feature-selection options
.TP
\fB\-D\fR, \fB\-\-prune\-vocab\-by\-doc\-count\fR=\fIN\fR
Remove words that occur in N or fewer documents.
.TP
\fB\-O\fR, \fB\-\-prune\-vocab\-by\-occur\-count\fR=\fIN\fR
Remove words that occur less than N times.
.TP
\fB\-T\fR, \fB\-\-prune\-vocab\-by\-infogain\fR=\fIN\fR
Remove all but the top N words by selecting words
with highest information gain.
.IP
Weight-vector setting/scoring method options
.TP
\fB\-\-binary\-word\-counts\fR
Instead of using integer occurrence counts of
words to set weights, use binary absence/presence.
.TP
\fB\-\-event\-document\-then\-word\-document\-length\fR=\fINUM\fR
Set the normalized length of documents when
\fB\-\-event\-model\fR=\fIdocument\-then\-word\fR
.TP
\fB\-\-event\-model\fR=\fIEVENTNAME\fR
Set what objects will be considered the
`events' of the probabilistic model.  EVENTNAME
can be one of: word, document, document-then-word.
.IP
Default is `word'.
.TP
\fB\-\-infogain\-event\-model\fR=\fIEVENTNAME\fR
Set what objects will be considered the `events'
when information gain is calculated.  EVENTNAME
can be one of: word, document, document-then-word.
.IP
Default is `document'.
.TP
\fB\-m\fR, \fB\-\-method\fR=\fIMETHOD\fR
Set the word weight-setting method; METHOD may be
one of: fienberg-classify, hem-classify,
hem-cluster, multiclass, default=naivebayes.
.TP
\fB\-\-print\-word\-scores\fR
During scoring, print the contribution of each
word to each class.
.TP
\fB\-\-smoothing\-dirichlet\-filename\fR=\fIFILE\fR
The file containing the alphas for the dirichlet
smoothing.
.TP
\fB\-\-smoothing\-dirichlet\-weight\fR=\fINUM\fR
The weighting factor by which to muliply the
alphas for dirichlet smoothing.
.TP
\fB\-\-smoothing\-goodturing\-k\fR=\fINUM\fR
Smooth word probabilities for words that occur NUM
or less times. The default is 7.
.TP
\fB\-\-smoothing\-method\fR=\fIMETHOD\fR
Set the method for smoothing word
probabilities to avoid zeros; METHOD may be one
of: goodturing, laplace, mestimate, wittenbell
.TP
\fB\-\-uniform\-class\-priors\fR When setting weights, calculating infogain and
scoring, use equal prior probabilities on classes.
.TP
-?, \fB\-\-help\fR
Give this help list
.TP
\fB\-\-usage\fR
Give a short usage message
.TP
\fB\-V\fR, \fB\-\-version\fR
Print program version
.PP
Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.
.SH "REPORTING BUGS"
Please report bugs related to this program to Andrew McCallum
<mccallum@cs.cmu.edu>. If the bugs are related to the Debian package 
send bugs to submit@bugs.debian.org
.SH "SEE ALSO"
.BR arrow (1),
.BR archer (1),
.BR rainbow (1).
The full documentation for
.B crossbow
will be provided as a Texinfo manual.  If the
.B info
and
.B crossbow
programs are properly installed at your site, the command
.IP
.B info crossbow
.PP
should give you access to the complete manual.
.PP 
You can also find documentation and updates for 
.B libbow 
at http://www.cs.cmu.edu/~mccallum/bow