File: NoSQL-3.html

package info (click to toggle)
nosql 0.9-0
links: PTS
area: main
in suites: hamm
size: 1,364 kB
ctags: 225
sloc: perl: 3,766; sh: 476; makefile: 41
file content (1856 lines) | stat: -rw-r--r-- 66,624 bytes
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Draft//EN">
<HTML>
<HEAD>
<TITLE>NoSQL: NoSQL Operators</TITLE>
</HEAD>
<BODY>
<A HREF="NoSQL-2.html">Previous</A>
<A HREF="NoSQL-4.html">Next</A>
<A HREF="NoSQL.html#toc3">Contents</A>
<HR>
<H2><A NAME="s3">3. NoSQL Operators</A>  </H2>

<P>The NoSQL system comprises a set of programs called Operators.
<P>Each operator is a separate program module that performs
a unique function on the data.  Operators can be grouped into data
movers, report generators, and utilities.
<P>The data movers are operators that extract or rearrange
the data in some way.  They each read an rdbtable via STDIN
and write a rdbtable via STDOUT, and so are frequently
connected using the UNIX pipe function to form a larger task.
Each operator in such a "pipeline" style of operation gets
its input from the output of the previous operator in the
"pipeline".  The data movers include:
<P>
<DL>
<DT><B><B>nsq-col</B></B><DD><P>Picks columns by name, outputs columns in listed
order.
<P>
<DT><B><B>nsq-compute</B></B><DD><P>Computes an arbitrary expression using column
names.
<P>
<DT><B><B>nsq-headchg</B></B><DD><P>Generates and replaces (or removes) the header
of an rdbtable.
<P>
<DT><B><B>nsq-header</B></B><DD><P>Extracts the header from an rdbtable.
<P>
<DT><B><B>nsq-join</B></B><DD><P>Natural or "Master/Detail" join of two
rdbtables.
<P>
<DT><B><B>nsq-listtotable</B></B><DD><P>Converts files from /rdb <EM>list</EM>
to /rdb <EM>table</EM> format.
<P>
<DT><B><B>nsq-merge</B></B><DD><P>Merges two like rdbtables.
<P>
<DT><B><B>nsq-n2r</B></B><DD><P>Converts tables from NoSQL to /rdb format.
<P>
<DT><B><B>nsq-r2n</B></B><DD><P>Converts tables from /rdb to NoSQL format.
<P>
<DT><B><B>nsq-row</B></B><DD><P>Selects rows based on arbitrary expressions.
<P>
<DT><B><B>nsq-search</B></B><DD><P>Selects rows based on a multi-column key of
a sorted or indexed rdbtable.
<P>
<DT><B><B>nsq-sort</B></B><DD><P>Sorts a datafile by one or more columns.
<P>
<DT><B><B>nsq-subtot</B></B><DD><P>Lists subtotals of specified columns.
<P>
<DT><B><B>nsq-tabletolist</B></B><DD><P>Converts files from /rdb <EM>table</EM>
to /rdb <EM>list</EM> format.
<P>
<DT><B><B>nsq-tee</B></B><DD><P>Safely [over]writes an rdbtable.
<P>
<DT><B><B>nsq-uniq</B></B><DD><P>Makes an rdbtable unique on specified columns.
</DL>
<P>The report generators each read an rdbtable via STDIN
and produce a report on STDOUT, so when they are in a
"pipeline" of operators they will be the operator at the end.
The report generators are:
<DL>
<DT><B>nsq-pr</B><DD><P>Quick and easy printing of output formatted from
information in the header.
<P>
<DT><B>nsq-report</B><DD><P>Best form of output, with definable format.
<P>
<DT><B>nsq-summ</B><DD><P>Summary/Statistical information about data
values in an rdbtable.
<P>
<DT><B>nsq-valid</B><DD><P>Verifies the structure of an rdbtable.
</DL>
 
<P>The utilities are used for manipulating the structure and
content of rdbtables and are generally used as separate
tasks,i.e. they do not read STDIN. The utilities are:
<P>
<DL>
<DT><B>nsq</B><DD><P>NoSQL interactive terminal interface.
<P>
<DT><B>nsq-ed</B><DD><P>Uses an editor to allow modifications to an rdbtable.
<P>
<DT><B>nsq-index</B><DD><P>Generate rdbtable index files to be used by
'nsq-search'.
<P>
<DT><B><B>nsq-lock</B></B><DD><P>Trivial table locking program, for
serializing access to rdbtables.
<P>
<DT><B>nsq-repair</B><DD><P>Attempts to repair candidate NoSQL datafiles.
<P>
<DT><B><B>nsq-updseq</B></B><DD><P>Updates/Creates a unique record ID on each row
of an rdbtable. The field must be the first column in the table.
</DL>
<P>All operators take a '-h[elp]' option to show details of
operation online.  Following is a section for each
operator, in alphabetic order.
<P>
<H2><A NAME="ss3.1">3.1 nsq</A>
        </H2>

<P>Usage:  <B>nsq  [options]  [rdbtable]</B>
<P>Interactive analysis tool for NoSQL data tables (rdbtables).
This is especially useful for rdbtables that were made from
spreadsheet data.
<P>Uses nine NoSQL modules: nsq-col, nsq-ed, nsq-pr, nsq-report, nsq-row,
nsq-sort, nsq-t2l, nsq-summ, nsq-valid.
<P>Specific module options are available using the '-help' option of
individual modules. This utility uses the PAGER environment
variable.
<P>
<H2><A NAME="ss3.2">3.2 nsq-col</A>
        </H2>

<P>Usage:  <B>nsq-col  [options]  list</B>
<P>Selects ("<EM>projects</EM>") columns by name (and order) and
outputs an rdbtable
with these columns.  Can effectively select, order, add,
delete, or duplicate columns.
<P>The value 'list' is normally a list of column names.
If 'list' contains a triplicate of the form '-c NAME NEW'
then column name 'NAME' will be changed to 'NEW'.  If 'list'
contains a triplicate of the form '-a  NAME  DEFN' then a
new (null) column is added, at that point in the list of
column names, with name 'NAME' and definition 'DEFN'.
<P>This NoSQL operator reads an rdbtable from STDIN and writes
an rdbtable to STDOUT.  Options may be abbreviated.
<P>As an example using the sample rdbtable from the DATA section
(named sample), to select columns named 'NAME' and 'COUNT'
the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-col  NAME  COUNT  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>To select all columns except column 'NAME' the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-col  -v  NAME  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>To add a new column named 'LENGTH' with a size of 10 the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-col  -v  -a  LENGTH  10  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>Note that to include documentation with the new column
definition the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-col  -v  -a  LENGTH  '10 length in meters'  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>The '10 length in meters' must be quoted so that it will
be treated  as a single token.
<P>
<H2><A NAME="ss3.3">3.3 nsq-compute</A>
  </H2>

<P>Usage:  <B>nsq-compute  [options]  [statements]</B>
<P>Computes values for data fields based on arbitrary statements
using column names. Any characters that are special to the
UNIX shell must be quoted.
<P>Comparison operators may be of the form: gt, ge, lt, le,
eq, ne. For example 'NAME  eq  Hobbs'.  Logical constructors
'or' and 'and' may be used; as well as 'null' to indicate an
empty data value.  The supplied statements may be essentially
any valid PERL statements.
<P>All of the Comparison operators and Logical constructors
are reserved and should not be used as column names (they
are all lower case and four characters or less).
<P>This operator reads a rdbtable via STDIN and writes a rdbtable via
STDOUT. Options may be abbreviated.
<P>If a file is used to contain the statements any line in
the file that starts with a sharp sign (#) is treated
as a comment and ignored. Also if there is a sharp sign
preceded by a space character anywhere on the line the
rest of the line is also treated as a comment.
<P>Since column names and reserved words are parsed by the
program, do not put the entire expression in a single pair
of quotes as that will prevent the parsing.  Also note
that column names and reserved words need to be surrounded
by blank spaces if they are not individually quoted.
For example either form below is fine:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row   NAME    eq   &quot;L Brown&quot;  &lt;  sample

      nsq-row  &quot;NAME&quot;  &quot;eq&quot;  &quot;L Brown&quot;  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>but do not use this form:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row  &quot;NAME  eq  L Brown&quot;  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>Example rdbtable (named cfile):
<P>
<BLOCKQUOTE><CODE>
<PRE>
      name    count   type    amt
      6       5N      4       5N
      Bush    3       A       133
      Hansen  39      A       23
      Newton  8       E       8
      Hobbs   42      B       144
      Hart    2       C       55
      Jones   4       B       244
      Smith   5       D       1111
    
</PRE>
</CODE></BLOCKQUOTE>
<P>The command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-compute  count  +=  100  if  type  lt  D  &lt;  cfile  |  nsq-pr
    
</PRE>
</CODE></BLOCKQUOTE>
<P>gives the output:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      name    count  type    amt
      ------  -----  ----  -----
      Bush      103  A       133
      Hansen    139  A        23
      Newton      8  E         8
      Hobbs     142  B       144
      Hart      102  C        55
      Jones     104  B       244
      Smith       5  D      1111
    
</PRE>
</CODE></BLOCKQUOTE>
<P>Example file of commands named 'XXX':
<P>
<BLOCKQUOTE><CODE>
<PRE>
      if( type eq A ){
          name = NEW ;
          amt = count * 2 ;
          type = 'AAA' ;
      }
      else{
          name = OLD ;
          amt = count + 1000 ;
          type = 'ZZZ' ;
      }
    
</PRE>
</CODE></BLOCKQUOTE>
<P>Output from command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-compute  -fXXX  &lt;  cfile  |  nsq-pr
    
</PRE>
</CODE></BLOCKQUOTE>
<P>would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      name    count  type    amt
      ------  -----  ----  -----
      NEW         3  AAA       6
      NEW        39  AAA      78
      OLD         8  ZZZ    1008
      OLD        42  ZZZ    1042
      OLD         2  ZZZ    1002
      OLD         4  ZZZ    1004
      OLD         5  ZZZ    1005
    
</PRE>
</CODE></BLOCKQUOTE>
<P>
<H2><A NAME="ss3.4">3.4 nsq-ed</A>
    </H2>

<P>Usage:  <B>nsq-ed  [options]  rdbtable [col_spec]
[line_spec]  [pat_spec]</B>
<P>This utility calls an editor to allow the editing of
selected lines and/or columns of (or the entire) rdbtable.
Options may be abbreviated.
<P>A "col_spec" is a list of column names.
<P>A "line_spec" is a list of line numbers, of increasing
value, optionally separated by a dash to specify a
range, e.g. "10-20". The form "N-" means from line
N to end of file. The header is always included,
so do not specify lines 1 or 2 (except as the first
part of a larger group, e.g. "1-10").
<P>A "pat_spec" is a single pattern (of the form: /pat/ )
optionally followed by one or more column names, and may
be preceded with the reserved word 'ne' to negate the
meaning (e.g. the pattern should NOT match).
<P>The order of "col_spec", "line_spec", and "pat_spec"
is significant only to the extent that "col_spec" must
precede "pat_spec" in the command line if both are given.
<P>If none of "col_spec", "line_spec", or "pat_spec" are
given then the entire rdbtable will be edited. If one
or more of the three above options are given then the
selected subset of the rdbtable will be edited. The option
"col_spec" identifies which columns of the rdbtable are
to be edited, and options "line_spec" and "pat_spec"
determine which lines will be selected for editing,
either by direct reference ("line_spec" given "col_spec"
not given) or by pattern matching ("col_spec" given
"line_spec" not given).  If both "line_spec" and "pat_spec"
are given then only lines within the bounds of "line_spec"
will be considered for selection by pattern matching.
<P>If "pat_spec" does not include column names then the
pattern (any PERL regular expression) is matched against
each entire row; a row is selected if there is a match
anywhere in the row.  If column names are included the
pattern is matched against only the specified columns.
In this case a row is selected if a match is found in
any specified column. If the "ne" option precedes the
"pat_spec" without column names then an entire row is
selected if the pattern does not match anywhere in the row,
and if column names are given then the row is selected
if the pattern does not match in any specified column.
<P>The form of the file to be edited is either "column"
with visible column delimiters (the default) or "list"
format where the column names are on the left and the data
is on the right.  The default editor is specified by the
environment variable EDITOR if set, otherwise the editor
'vi' is used.
<P>In either form of editing the delimiter is a "pipe" symbol
(|). Care should be taken when editing not to use any
"pipe" symbols in the data, or to delete any existing
pipe symbols in the file. Also, in the case of "list"
form, one or more blank lines must separate each record.
<P>The rbdtable may be an existing file, or it may be
automatically checked out from RCS.  In the latter case
it will be checked back into RCS after the editing is
complete.  The default action is that if the rdbtable does
not exist an attempt will be made to find the rdbtable
under RCS (the '-RCS' option may be used to force the
use of an RCS file).
<P>Afterward, except in the RCS case, the original contents
of the rdbtable will be left in a file of the same name
preceded with a comma, e.g.  "sample" will be ",sample".
<P>Uses NoSQL operators: nsq-col, nsq-pr, nsq-mktbl, nsq-t2l, nsq-l2t.
<P><EM>WARNING</EM>: If line_spec is given the number of
columns must not be changed by editing, or if col_spec
and/or "pat_spec" is given the number of lines must
not be changed by editing, otherwise the results may
be unpredictable.
<P>An example command to edit the rdbtable (named sample)
from the DATA section would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
          nsq-ed  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>which would edit the entire rdbtable. The file as it is
ready to edit is shown in Table 3.  The pipe character '|'
must not be removed during the editing process, although
it may be moved left or right if necessary and the spaces
around the pipe character may be deleted if desired.
<P>This form of editing is fine if the rdbtable is not large.
If it is large then editing only those parts that need
changes is faster and less error prone. To edit only
columns 'NAME', 'COUNT', and 'AMT', the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-ed  sample  NAME  COUNT  AMT
    
</PRE>
</CODE></BLOCKQUOTE>
<P>To edit only lines five thru seven the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-ed  sample  5-7
    
                            Table 3

           RDBTABLE (SAMPLE) READY TO EDIT, COLUMN FORM

      NAME   | COUNT | TYP  |   AMT | OTHER    |    RIGHT
      6      |    5N | 4    |    5N | 8        |       8&gt;
      Bush   |    44 | A    |   133 | Another  |     This
      Hansen |    44 | A    |    23 | One      |       Is
      Jones  |    77 | X    |    77 | Here     |       On
      Perry  |    77 | B    |   244 | And      |      The
      Hart   |    77 | D    |  1111 | So       |    Right
      Holmes |    65 | D    |  1111 | On       |     Edge
    
</PRE>
</CODE></BLOCKQUOTE>
<P>To edit only lines five thru seven of only columns 'NAME',
'COUNT', and 'AMT' the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-ed  sample  NAME  COUNT  AMT  5-7
    
</PRE>
</CODE></BLOCKQUOTE>
<P>and the file to edit would look like:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      ..&gt;&gt;&gt; 1 2 CONTROL LINE, DO NOT TOUCH &lt;&lt;&lt;
      NAME   | COUNT |   AMT
      6      |    5N |    5N
      ..&gt;&gt;&gt; 5 3 CONTROL LINE, DO NOT TOUCH &lt;&lt;&lt;
      Jones  |    77 |    77
      Perry  |    77 |   244
      Hart   |    77 |  1111
    
</PRE>
</CODE></BLOCKQUOTE>
   
<P>Note that whenever a line_spec is given, control lines
(starting with '..&gt;&gt;&gt;' are inserted into the file to
edit. They must not be modified during the editing process.
They are used to reconstruct the rdbtable after editing.
<P>If the rdbtable has data fields that are long, i.e. longer
than convenient to edit in the column form shown above,
the 'list' form is the preferred method. The usage of
line_spec and col_spec are unchanged but the form of the
file to edit is different. For example consider an rdbtable
(named sample3) which is shown in Table 4, where the
TAB characters are represented by '&lt;T&gt;and the newline
characters are represented by &lt;N&gt;This small rdbtable
looks very incoherent in raw form, and a file of any real
size with long data fields is even  more so.  The command
to edit the file sample3 in 'list' form would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
                           Table 4
  
               RDBTABLE (SAMPLE3) ACTUAL CONTENT
  
name&lt;T&gt;datatype&lt;T&gt;agencysrc&lt;T&gt;dbms&lt;T&gt;contact&lt;T&gt;contents&lt;T&gt;notes&lt;N&gt;
46&lt;T&gt;15&lt;T&gt;60&lt;T&gt;15&lt;T&gt;21&lt;T&gt;530&lt;T&gt;600&lt;N&gt;
ACAS (Air Combat Assessment)&lt;T&gt;BDA&lt;T&gt;Bigplace AFB&lt;T&gt;File&lt;T&gt;Starr&lt;T&gt;Air
Combat Assessment BDA data.  Duplicates data under ACAS (Air Combat
Assessment) BDA Sorties, diskettes nr 1,2,3.&lt;T&gt;On two 3.5 inch
diskettes.&lt;N&gt;
ACAS (Air Combat Assessment) BDA Sorties&lt;T&gt;BDA&lt;T&gt;Sawyer AFB&lt;T&gt;File&lt;T&gt;
Hobbs/Emerson&lt;T&gt;85 files, 2 per day containing 12 and 24 hour reports.
This data is different from that under ACAS (Air Combat Assessment) BDA
Data, from diskettes 1,2,3.&lt;T&gt;Received 5/6/91.&lt;N&gt;
ATO (Air Tasking Orders) Original&lt;T&gt;ATO&lt;T&gt;HQ USAF, Universal AFB&lt;T&gt;
File&lt;T&gt;Marshall&lt;T&gt;Original ATO messages.  Both sets are incomplete.&lt;T&gt;
To be joined into single file and edited.  Missing sections not yet
ordered.  May be parsed completely, or only for key comments. Much data
to be processed.&lt;N&gt;
ABC Original&lt;T&gt;ABC&lt;T&gt;HQ USAF, Universal AFB&lt;T&gt;File&lt;T&gt;Marshall&lt;T&gt;Original
ATO messages.  Both sets are incomplete.&lt;T&gt;To be joined into single file
and edited.  Missing sections not yet ordered. May be parsed completely,
or only for key comments. Much data to be processed.&lt;N&gt;
    
</PRE>
</CODE></BLOCKQUOTE>
<P>The command to edit the file sample3 in 'list' form would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-ed  -list  sample3
    
</PRE>
</CODE></BLOCKQUOTE>
<P>which would produce a file to edit as shown in Table 5.
Note that each section holds information relating to one row
in the rdbtable and that the first section holds information
relating to the header of the rdbtable.  Also note that
each section is separated by a blank line (it could be any
number of blank lines).
<P>  
Each row in a section relates to a single data value.
The pipe character '|' must not be removed during the
editing process, although it may be moved left or right
if necessary. Only one pipe character is to be in the
information relating to one data value, although that
information may be physically on more than one line in the
section if the data value is long. The spaces on both
sides of the pipe character as well as the spaces around
the column names are only for readability; they may be
moved or even deleted if desired.
<P>
<BLOCKQUOTE><CODE>
<PRE>
                           Table 5

           RDBTABLE (SAMPLE3) READY TO EDIT, LIST FORM

      name | 46
  datatype | 15
 agencysrc | 60
      dbms | 15
   contact | 21
  contents | 530
     notes | 600

      name | ACAS (Air Combat Assessment)
  datatype | BDA
 agencysrc | Bigplace AFB
      dbms | File
   contact | Starr
  contents | Air Combat Assessment BDA data.  Duplicates data under ACAS
             (Air Combat Assessment) BDA Sorties, diskettes nr 1,2,3.
     notes | On two 3.5 inch diskettes.

      name | ACAS (Air Combat Assessment) BDA Sorties
  datatype | BDA
 agencysrc | Sawyer AFB
      dbms | File
   contact | Hobbs/Emerson
  contents | 85 files, 2 per day containing 12 and 24 hour reports. This
             data is different from that under ACAS (Air Combat
             Assessment) BDA Data, from diskettes 1,2,3.
     notes | Received 5/6/91.
    
      name | ATO (Air Tasking Orders) Original
  datatype | ATO
 agencysrc | HQ USAF, Universal AFB
      dbms | File
   contact | Marshall
  contents | Original ATO messages.  Both sets are incomplete.
     notes | To be joined into single file and edited.  Missing sections
             not yet ordered.  May be parsed completely, or only for key
             comments. Much data to be processed.
    
      name | ABC Original
  datatype | ABC
 agencysrc | HQ USAF, Universal AFB
      dbms | File
   contact | Marshall
  contents | Original ATO messages.  Both sets are incomplete.
     notes | To be joined into single file and edited.  Missing sections
             not yet ordered.  May be parsed completely, or only for key
             comments.  Much data to be processed.
    
</PRE>
</CODE></BLOCKQUOTE>
<P>The advantage of this form of edit file is that even with
very large data values most, if not all, of the information
from each row of an rdbtable will be visible on the screen
at once.
<P>
<H2><A NAME="ss3.5">3.5 nsq-headchg</A>
    </H2>

<P>Usage:  <B>nsq-headchg  [options]  file.tpl</B>
<P> 
Replaces the header (first two rows) of an rdbtable with a
header generated from information in the template file
'file.tpl'.  Options are available to add, copy, or
delete the header, or to generate a template file from
an existing rdbtable.
<P>Each line of the Template file contains info about
a column, in order.  The lines contain: (optional)
index number (starting at 0 or 1), column name,
definition, and (optional) comments or documentation,
white space separated.  If column name contains spaces
it must be enclosed in double quotes.  Names containing
space characters are not recommended, however, as it is
generally troublesome and error prone. A good substitute
is the underscore character (_).
<P>Lines that start with a sharp character '#' are skipped,
as are blank lines.  To start a column name with a sharp
character '#'  the name must be enclosed in double quotes.
(but this is not recommended).
<P>The number of columns in the header is normally reported
on STDERR. This operator reads an rdbtable via STDIN
and writes an rdbtable via STDOUT.  Options may be
abbreviated. This operator uses the NoSQL operator: nsq-valid.
<P>As an example, to generate a template file named 'new.tpl'
from the rdbtable (named sample) from the DATA section, the command
would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-headchg  -templ  &lt;  sample  &gt;  new.tpl
    
</PRE>
</CODE></BLOCKQUOTE>
<P>The contents of file 'new.tpl' would then be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
     0                 NAME  6
     1                COUNT  5N
     2                  TYP  4
     3                  AMT  5N
     4                OTHER  8
     5                RIGHT  8&gt;
    
</PRE>
</CODE></BLOCKQUOTE>
<P>To change the header of rdbtable 'sample', the procedure
is to edit the file 'new.tpl', and then run 'nsq-headchg'
using the modified file.  For example, to change the names
so that only the first letters are upper case and to make
column 'OTHER' numeric, edit file 'new.tpl' so it looks
like the following:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      0                 Name  6    All names are first letter upper case.
      1                Count  5N
      2                  Typ  4
      3                  Amt  5N
      4                Other  8N   Now numeric.
      5                Right  8&gt;
    
</PRE>
</CODE></BLOCKQUOTE>
<P>Note the index in the zeroth column and the documentation
in the fourth column, both of which are optional, but
recommended.  The command to change the header of rdbtable
'sample' and make a new rdbtable called 'new.sample'
would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-headchg  new.tpl  &lt;  sample  &gt;  new.sample
    
</PRE>
</CODE></BLOCKQUOTE>
   
<P>
<H2><A NAME="ss3.6">3.6 nsq-header     </A>
</H2>

<P>Usage:  <B>nsq-header  [options]</B>
<P>Section yet to be written. See 'nsq-header -help' in the
meantime.
<P>
<H2><A NAME="ss3.7">3.7 nsq-index</A>
    </H2>

<P>Usage:
<P><B>nsq-index  [options]  rdbtbl  column  ...</B>
<P>or
<P><B>nsq-index  [options]  -update  [ index_file  ... ]</B>
<P> 
The first form of usage of this utility generates an
index file for the column(s) given, that refers to the
specified rdbtbl.  An index file is actually another
(smaller) rdbtable containing only the column(s)
given plus a column for index information. An index
file can be used by the operator 'nsq-search' to quickly
locate rows of data in the referenced rdbtable.
<P>Index files are named by appending an 'x' and the
column name(s) (seperated by a dots) to the base name
of the rdbtable it refers to. For example an index
file for the rdbtable "area.rdb" on column "strip"
would be "area.x.strip".  An index file that was also
on column "depth" would be "area.x.strip.depth".
<P>The second form of usage of this utility updates
the index file(s) given. If no files are given all
index files in the current directory are updated.
An update of an index file is necessary when the
rdbtable it refers to has been modified.
<P>This utility writes or rewrites rdbtables with
defined names in the current directory.  Options may
be abbreviated.
<P>
<H2><A NAME="ss3.8">3.8 nsq-join</A>
        </H2>

<P>Usage:  <B>nsq-join  [options]  col.name[=col.name_2]
rdbtable_2  &lt; rdbtable_1</B>
<P>where rdbtable_2 is the "secondary" rdbtable, while rdbtable_1 is
the the "primary" one.
<P>Does a join of two rdbtables on the column(s) specified.
The default is a "natural" join, with optional
"Master/Detail" or cartesian (cross-product) type joins.
Options may be abbreviated.
<P>The Table from STDIN is the master. A natural join
produces a new rdbtable that contains only rows from the
input rdbtables that match on the specified columns (key
columns). A master-detail join produces a new rdbtable
that contains all rows from the master rdbtable and those
rows from the secondary rdbtable that match.  A cartesian
join produces an rdbtable that contains all rows of both
input rdbtables.
<P>Each item in the list of column(s) may specify column names
that are different in the two rdbtables, i.e. '=column_2',
if given, refers to a name in rdbtable_2 that corresponds
to 'column' in rdbtable_1.  If '=column_2' is not given it
means that the corresponding column name in both rdbtables
is the same.
<P>If different column names are specified, the name of
the join columns in the output rdbtable will be from
rdbtable_1.
<P>Note that the two rdbtables must be sorted on the columns
specified in order for a join operation to function
correctly.
<P>The order of columns in the output rdbtable will be:
first the join columns, then the other columns from
rdbtable_1, then the other columns from rdbtable_2.
<P>This operator reads an rdbtable via STDIN and writes an
rdbtable via STDOUT.
<P>If we have the rdbtable (named samplej) here:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      name     nr     typ     amt
      6        2      4       4
      Bush     1      A       133
      Bush     2      A       134
      Hansen   3      A       143
      Hobbs    4      B       144
      Hobbs    5      B       144
      Jones    6      C       155
      Perry    7      D       244
      Perry    8      D       311
    
</PRE>
</CODE></BLOCKQUOTE>
   
<P>and the rdbtable (named samplej2) here:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      name     cnt    typ     amt
      6        5N     4       5N
      Hobbs    41     A       141
      Hobbs    42     BB      142
      Hobbs    51     BB      144
      Hobbs    43     CC      143
        
</PRE>
</CODE></BLOCKQUOTE>
<P>then the command to do a natural join of samplej and
samplej2 on  column name is:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-join  name  samplej2  &lt;  samplej
        
</PRE>
</CODE></BLOCKQUOTE>
<P>and the result is shown in Table 6.
<P>
<BLOCKQUOTE><CODE>
<PRE>
                          Table 6

         NATURAL JOIN OF RDBTABLES SAMPLEJ AND SAMPLEJ2

      name    nr      typ     amt     cnt     typ     amt
      6       2       4       4       5N      4       5N
      Hobbs   4       B       144     41      A       141
      Hobbs   4       B       144     42      BB      142
      Hobbs   4       B       144     51      BB      144
      Hobbs   4       B       144     43      CC      143
      Hobbs   5       B       144     41      A       141
      Hobbs   5       B       144     42      BB      142
      Hobbs   5       B       144     51      BB      144
      Hobbs   5       B       144     43      CC      143
        
</PRE>
</CODE></BLOCKQUOTE>
<P>The command to do a "masterdetail" join of the same
two rdbtables on column name is:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-join  -md  name  samplej2  &lt;  samplej
        
</PRE>
</CODE></BLOCKQUOTE>
<P>and the result is shown in Table 7. 
<P>
<BLOCKQUOTE><CODE>
<PRE>
                           Table 7

       MASTER-DETAIL JOIN OF RDBTABLES SAMPLEJ AND SAMPLEJ2

      name    nr      typ     amt     cnt     typ     amt
      6       2       4       4       5N      4       5N
      Bush    1       A       133
      Bush    2       A       134
      Hansen  3       A       143
      Hobbs   4       B       144     41      A       141
      Hobbs   4       B       144     42      BB      142
      Hobbs   4       B       144     51      BB      144
      Hobbs   4       B       144     43      CC      143
      Hobbs   5       B       144     41      A       141
      Hobbs   5       B       144     42      BB      142
      Hobbs   5       B       144     51      BB      144
      Hobbs   5       B       144     43      CC      143
      Jones   6       C       155
      Perry   7       D       244
      Perry   8       D       311
        
</PRE>
</CODE></BLOCKQUOTE>
<P>
<H2><A NAME="ss3.9">3.9 nsq-l2t     </A>
</H2>

<P>Usage:  <B>nsq-l2t  [options]</B>
<P>Converts a file in "list" format to an rdbtable.  Long data
fields may be folded.  This operator is mainly used by
other operators. Options may be abbreviated.
<P>
<H2><A NAME="ss3.10">3.10 nsq-listtotable     </A>
</H2>

<P>Usage:  <B>nsq-listtotable  [options]</B>
<P>Section yet to be written. See 'nsq-listtotable -help' in the
meantime.
<P>
<H2><A NAME="ss3.11">3.11 nsq-lock     </A>
</H2>

<P>Usage:  <B>nsq-lock  [options]</B>
<P>Section yet to be written. See 'nsq-lock -help' in the
meantime.
<P>
<H2><A NAME="ss3.12">3.12 nsq-merge</A>
        </H2>

<P>Usage:  <B>nsq-merge  [options]  &lt; old_table  column  ...
merge_table</B>
<P>This operator merges and/or deletes rows of 'old_table'
based on data values in 'merge_table' in the specified
column(s).  Both tables should be sorted on the specified
column(s).
<P>In the normal case, one or more rows in 'merge_table'
either replace one or more existing rows in 'old_table'
if the key column(s) match, or are inserted in order if
the key column(s) do NOT match.
<P>If the delete option is specified on the command line,
one or more existing rows in 'old_table' will be deleted
if there is a key column(s) match and the data in the
delete column is equal to the delete string, "&gt;&gt;DEL&lt;&lt;"
(without the quotes) by default.  The delete column is
the first non-key column in 'merge_table'.
<P>Both tables should have similar data structures. The
header for the new rdbtable is taken from 'merge_table',
thus allowing a change of header  information to be made.
<P>This operator writes an rdbtable via STDOUT.  Options may be
abbreviated.
<P>
<H2><A NAME="ss3.13">3.13 nsq-mktbl</A>
        </H2>

<P>Usage: <B>nsq-mktbl [options]</B>
<P>Makes a file of data in columns (with visible column
delimiters) into an rdbtable.  The column delimiter is
the pipe symbol (|).  This operator is mainly used by
other operators.
<P>This operator reads a file via STDIN and writes an rdbtable
via STDOUT.  Options may be abbreviated.
<P>
<H2><A NAME="ss3.14">3.14 nsq-n2r     </A>
</H2>

<P>Usage:  <B>nsq-n2r  [options]</B>
<P>Section yet to be written. See 'nsq-n2r -help' in the
meantime.
<P>
<P>
<H2><A NAME="ss3.15">3.15 nsq-pr</A>
        </H2>

<P>Usage:  <B>nsq-pr  [options]</B>
<P>This operator used for quick and easy printing of an
rdbtable, in a simple but useful form.  It prints an
rdbtable using formatting information from the header.
<P>The printing of each row of data will be on one line if
possible, but when multiple lines are necessary the second
and later lines are indented for readability. Also when
multiple lines are necessary a simple space availability
algorithm is used to minimize the number of lines printed
for each row of data. This may result in the order of
some data values being rearranged from their order in the
rdbtable. The '-b0' option can override this algorithm
and force the same printing order as in the rdbtable.
<P>This NoSQL operator reads an rdbtable from STDIN and writes
a formatted report on STDOUT. Options may be abbreviated.
<P>As an example using the sample rdbtable from the DATA
section (named sample), the command to view this rdbtable
would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-pr &lt; sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>which would produce the output shown in Table 8.  The same command
with a page heading for printing:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-pr  -PP  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>produces the output as shown in Table 9.  Using an
rdbtable (named sample4) that has long data values,
shown  in Table 10, the command to print the rdbtable
using the truncate option is:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-pr  -t  &lt;  sample4
  
                         Table 8

          PRINTING RDBTABLE (SAMPLE) USING NSQ-PR

      NAME    COUNT  TYP     AMT  OTHER        RIGHT
      ------  -----  ----  -----  --------  --------
      Bush       44  A       133  Another       This
      Hansen     44  A        23  One             Is
      Jones      77  X        77  Here            On
      Perry      77  B       244  And            The
      Hart       77  D      1111  So           Right
      Holmes     65  D      1111  On            Edge
  

                         Table 9

  PRINTING RDBTABLE (SAMPLE) WITH PAGE HEADING USING NSQ-PR

      Page   1   Mon Dec  2 16:56:43 PST 1991

      NAME    COUNT  TYP     AMT  OTHER        RIGHT
      ------  -----  ----  -----  --------  --------
      Bush       44  A       133  Another       This
      Hansen     44  A        23  One             Is
      Jones      77  X        77  Here            On
      Perry      77  B       244  And            The
      Hart       77  D      1111  So           Right
      Holmes     65  D      1111  On            Edge

                          Table 10

    RDBTABLE WITH LONG DATA VALUES (SAMPLE4) ACTUAL CONTENT

  name&lt;T&gt;type&lt;T&gt;contact&lt;T&gt;contents&lt;N&gt;
  10&lt;T&gt;4&lt;T&gt;21&lt;T&gt;20&lt;N&gt;
  Hansen&lt;T&gt;AAA&lt;T&gt;R. Starr at the UCLA & USC&lt;T&gt;Duplicate data under
  processing order number 55-7.&lt;N&gt;
  Hart&lt;T&gt;CCC&lt;T&gt;Hobbs/Emerson at RAND Corporation&lt;T&gt;85 files, 2 per
  day containing 12 and 24 hour reports.&lt;N&gt;
  Hobbs&lt;T&gt;EEE&lt;T&gt;Marshall at Universal AFB&lt;T&gt;Original PAF messages.
  Both sets are incomplete.&lt;N&gt;
  Bush&lt;T&gt;KKK&lt;T&gt;General USAF personnel&lt;T&gt;Duplicate ATO messages,
  incomplete.&lt;N&gt;
  Lender&lt;T&gt;RRR&lt;T&gt;Army base in Nevada&lt;T&gt;Nothing.&lt;N&gt;
  Emerson&lt;T&gt;UUU&lt;T&gt;Navy at Washington DC&lt;T&gt;More than we thought at
  first.&lt;N&gt;
      
</PRE>
</CODE></BLOCKQUOTE>
<P>which will produce output with the data values truncated
to the defined column width as in Table 11.  Using the
same rdbtable with the fold option:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-pr  -fold  &lt;  sample4
    
</PRE>
</CODE></BLOCKQUOTE>
<P>produces output with the long data values 'folded'
within their defined column widths as shown in Table 12.
Note that each line is repeated until the entire data
value for each column is completely shown. This makes
this type of output variable length.
<P>If you need a quick and easy way to look at the data in an
rdbtable use the -win option.  This option will cause nsq-pr
to list as many columns as possible in single line records
that will fit in the current window or terminal width.
Note that you do not have to type the column names (or
even know them) to use this option.
<P>
<BLOCKQUOTE><CODE>
<PRE>
                           Table 11

        PRINTING RDBTABLE (SAMPLE4) WITH NSQ-PR -TRUNC OPTION

   name        type  contact                contents
   ----------  ----  ---------------------  --------------------
   Hansen      AAA   R. Starr at the UCLA   Duplicate data under
   Hart        CCC   Hobbs/Emerson at RAND  85 files, 2 per day
   Hobbs       EEE   Marshall at Universal  Original PAF message
   Bush        KKK   General USAF personne  Duplicate ATO messag
   Lender      RRR   Army base in Nevada    Nothing.
   Emerson     UUU   Navy at Washington DC  More than we thought
    


                           Table 12

         PRINTING RDBTABLE (SAMPLE4) WITH NSQ-PR -FOLD OPTION

   name        type  contact                contents
   ----------  ----  ---------------------  --------------------
   Hansen      AAA   R. Starr at the UCLA   Duplicate data under
                     & USC                  processing order 
                                            number 55-7.
   Hart        CCC   Hobbs/Emerson at RAND  85 files, 2 per day
                     Corporation            containing 12 and 24
                                            hour reports.
   Hobbs       EEE   Marshall at Universal  Original PAF
                     AFB                    messages.  Both sets
                                            are incomplete.
   Bush        KKK   General USAF           Duplicate ATO
                     personnel              messages,
                                            incomplete.
   Lender      RRR   Army base in Nevada    Nothing.
   Emerson     UUU   Navy at Washington DC  More than we thought
                                            at first.
        
</PRE>
</CODE></BLOCKQUOTE>
<P>It may be combined with the -t option to increase the
number of columns of data shown on each line at the
expense of some column width.
<P>For example the command 'nsq-pr &lt; d11c' on an 80 character
wide window or terminal produces the following:
<P>
<BLOCKQUOTE><CODE>
<PRE>
    name    count            type  amt    n1                        n3
    ------  ---------------  ----  -----  ------------------------  ------------
        n2                        n4                        n5
        ------------------------  ------------------------  ------------
        n6            n7
        ------------  ------------
    Bush    3                A     133    alpha22.307               117722
     baker                     DDBBx17                   other
        124567        8GGXXH17
    Hansen  39               A     23     beta222.307               117723
    charlie                   DDBBx18                   data
        1239870       GGXXH17
    Newton  8                E     8      gama22.333                117724
    dog                       DDBBx19                   exists
        1239870       GGXXH17
    Hobbs   42               B     144    delta3.3.118              117725
    echo                      DDBBx20                   here
        1239870       GGXXH17
    Hart    2                C     55     epslion33.118             117726
    foxtrot                   DDBBx21                   also
        1239870       GGXXH17
    
</PRE>
</CODE></BLOCKQUOTE>
<P>This is readable, but not very nice to look at, and even
worse if there are more columns.  The command
<P>
<P>
<BLOCKQUOTE><CODE>
<PRE>
          nsq-pr -win &lt; d11c
        
</PRE>
</CODE></BLOCKQUOTE>
<P>produces:
<P>
<BLOCKQUOTE><CODE>
<PRE>
 name   count           type amt   n1                       n3
 ------ --------------- ---- ----- ------------------------ ------------
 Bush   3               A    133   alpha22.307              117722
 Hansen 39              A    23    beta222.307              117723
 Newton 8               E    8     gama22.333               117724
 Hobbs  42              B    144   delta3.3.118             117725
 Hart   2               C    55    epslion33.118            117726
        
</PRE>
</CODE></BLOCKQUOTE>
<P>Not all the data is listed, but the first few columns (sometimes
the most important) are easier to view. The command
<P>
<P>
<BLOCKQUOTE><CODE>
<PRE>
          nsq-pr -win -t6 &lt; d11c
        
</PRE>
</CODE></BLOCKQUOTE>
<P>shows even more of the data, at the expense of some data width:
<P>
<BLOCKQUOTE><CODE>
<PRE>
 name   count  type amt   n1     n2     n3     n4     n5     n6     n7
 ------ ------ ---- ----- ------ ------ ------ ------ ------ ------ ------
 Bush   3      A    133   alpha2 baker  117722 DDBBx1 other  124567 8GGXXH
 Hansen 39     A    23    beta22 charli 117723 DDBBx1 data   123987 GGXXH1
 Newton 8      E    8     gama22 dog    117724 DDBBx1 exists 123987 GGXXH1
 Hobbs  42     B    144   delta3 echo   117725 DDBBx2 here   123987 GGXXH1
 Hart   2      C    55    epslio foxtro 117726 DDBBx2 also   123987 GGXXH1
        
</PRE>
</CODE></BLOCKQUOTE>
<P>
<H2><A NAME="ss3.16">3.16 nsq-r2n     </A>
</H2>

<P>Usage:  <B>nsq-r2n  [options]</B>
<P>Section yet to be written. See 'nsq-r2n -help' in the
meantime.
<P>
<H2><A NAME="ss3.17">3.17 nsq-repair</A>
        </H2>

<P>Usage:  <B>nsq-repair  [options]  file  ...</B>
<P>Attempts to repair candidate NoSQL datafiles, e.g. files
that have been ported from a MacIntosh or PC (MSDOS
computer) in spreadsheet form but that do not yet have
valid rdbtable structure.  Generates definition lines
(second line of header). The width of all data values
is checked and the maximum width for a column is used as
the column width in the definition line for that Table.
<P>It also works with existing rdbtables ('-exist' option)
and is convenient for removing leading and trailing space
characters from data values (-blank option).
<P>Adds fields as necessary to rows (null), or to header
(DUM1, DUM2, ...) to make the Table structure valid.
<P>The new rdbtables will be in the current directory (even if
the input files are not) and will have the suffix changed
(or added) to '.rdb' by default.
<P>Options may be abbreviated.
<P>
<H2><A NAME="ss3.18">3.18 nsq-report</A>
        </H2>

<P>Usage:  <B>nsq-report  [options]  file.frm</B>
<P>Formats and prints an arbitrary style report, with the
format specified in the file "file.frm". A page header
may be specified.
<P>This NoSQL operator reads an rdbtable from STDIN and writes
a formatted report on STDOUT. Options may be abbreviated.
<P>The "file.frm" file (or form file) shows pictorially
one 'record' of output, which will contain data from
one row of an rdbtable.  An optional page header may
be defined as well.
<P>The form file contains regular text, picture fields,
and associated column names.  Regular text prints just as
given. Picture fields define the  width and justification
for printing a data value from a column. The names of
the associated columns are listed on the line following
the picture fields and in the same order.  Note that
this file should not contain any TAB characters; space
characters should be used instead.
<P>Picture fields start with either '@' or '^' and are
followed by one of three primary characters to define the
width of the field.  The three characters are '&lt;', '&gt;',
or '|' to specify left, right, or  center justification
respectively.  There is also an alternate right
justification character for printing numeric data, with
optional decimal point.  The character is the sharp sign
'#', and a period specifies the decimal point placement,
as in '@########.##'.
<P>A numeric picture field has the following features:
<UL>
<LI>Data is lined up on the decimal point (if any).</LI>
<LI>Automatic rounding of data.</LI>
<LI>Automatic conversion of data in scientific notation.</LI>
</UL>
<P>Numeric Data may be in the form of integers, fixed point,
or scientific notation' e.g. 12345, 4567.345, or  1.678E17.
<P>Normally picture fields start with the '@' character. That
means to put the referenced data value into the defined
picture field, or as much of the data as will fit into
the field, if the data is larger than the field.  If the
field starts with the '^' character it means to repeat the
field on as many lines as necessary in order to print the
entire data value.  This is useful for large data fields,
such as comments or free text.
<P>Instead of a column name there are some special names that
can be used to have other information inserted. This are
especially useful if there is a page header.  The special
names and what they mean are:
<P>
<UL>
<LI>_pgnr_  -  current page number</LI>
<LI>_date_  -  current date</LI>
<LI>_rcnr_  -  current record number (row number) </LI>
<LI>_`cmd arg1 ... argN`_  -  the UNIX command is executed once,
and its output is put into the associated picture field.  Note that
they are BACKTICKS (grave accents) not single quotes.</LI>
<LI>_COLNAME_cd_ - the column documentation for column name 'COLNAME'.</LI>
<LI>_tbld_ - the table documentation, all lines.</LI>
<LI>_tbld_3.7_ - the table documentation, lines 3 thru 7. If either
first or second number is missing it means line 1  or the last line
of the header, respectively.</LI>
</UL>
<P>An example of a form file for use with rdbtable 'sample'
is shown in Table 13.
<P>The first and last lines (that start with 'format' or
a single period) define the pictorial records and must
be as shown. The first record defines the header and is
optional. If this form file (named sample.frm) were used
in the command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-report  sample.frm  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>it would produce the one page report as in Table 14.
<P>
<BLOCKQUOTE><CODE>
<PRE>
                            Table 13

                 FORM FILE FOR RDBTABLE (SAMPLE)

format top =
Page @&gt;,            The Page Header     @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
   _pgnr_                                       _date_

.
format =
  Name:   @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;          Type:  @&gt;&gt;&gt;&gt;
               NAME                          TYP
                  Total: @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;     Other:  @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
                         AMT                        OTHER
.




                           Table 14

            PRINTING RDBTABLE (SAMPLE) WITH NSQ-REPORT

Page  1,            The Page Header     Mon Dec  2 16:21:18 PST 1991

Name:   Bush                                        Type:      A
                  Total: 133           Other:  Another
Name:   Hansen                                      Type:      A
                  Total: 23            Other:  One
Name:   Jones                                       Type:      X
                  Total: 77            Other:  Here
Name:   Perry                                       Type:      B
                  Total: 244           Other:  And
Name:   Hart                                        Type:      D
                  Total: 1111          Other:  So
Name:   Holmes                                      Type:      D
                  Total: 1111          Other:  On
        
</PRE>
</CODE></BLOCKQUOTE>
<P>For another example, one might want to have a date on a
report in other than the standard date output format, and
an idea of who executed the program, and have a reference
count of the records being produced. The form file might
be as in Table 15.  which could produce:
<P>
<BLOCKQUOTE><CODE>
<PRE>
                            Table 15

                        ANOTHER FORM FILE

format top =
Run By: @&lt;&lt;&lt;&lt;&lt;&lt;&lt;        The Date/Time is    @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
        _`whoami`_                           _`date &quot;+%m/%d/%y %H:%M&quot;`_
.
format =
    RecordNr:  @&gt;&gt;      @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
              _rcnr_    OTHER
.



Run By: hobbs           The Date/Time is    10/15/91 09:43

RecordNr:        1      Other data here
RecordNr:        2      and here,
RecordNr:        3      and so on.
...             ...     ...
        
</PRE>
</CODE></BLOCKQUOTE>
<P>Another example shows how longer data values can be
handled. If the form file (named sample2.frm) looks like
Table 16, and if the following command is used:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-report  sample2.frm  &lt;  sample2
    
</PRE>
</CODE></BLOCKQUOTE>
<P>then the output would be as shown in Table 17.
<P>
<BLOCKQUOTE><CODE>
<PRE>
                            Table 16

                 FORM FILE FOR RDBTABLE (SAMPLE2)

format top =
Page @&gt;,                Page Header here                 @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
_pgnr_                                      _`date &quot;+%m/%d/%y %H:%M&quot;`_
                Executed by @&lt;&lt;&lt;&lt;&lt;&lt;&lt; on: @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
                            `whoami`     _`hostname`_

format =
    Name:   @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;  Other: @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;        Type: @&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;
            NAME              OTHER                            TYP
 Comment:   ^&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;    Long: ^&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;    Right: @&gt;&gt;&gt;&gt;&gt;&gt;&gt;
            COMMENT                   LONG                     RIGHT
=======================================================================



                              Table 17

              PRINTING RDBTABLE (SAMPLE2) WITH NSQ-REPORT

Page  1,                Page Header here                 12/02/91 16:32
                Executed by hobbs    on: id

    Name:   Bush              Other: Another            Type: A
 Comment:   A comment          Long: This a long        Right:     This
            here.                    message for
                                     test.
=======================================================================
    Name:   Hansen            Other: One                Type: A
 Comment:   A longer           Long: This a long        Right:       Is
            comment here.            message for
                                     test.
=======================================================================
    Name:   Jones             Other: Here               Type: X
 Comment:   A longer,          Long: Short test.        Right:       On
            longer comment
            here.
=======================================================================
    Name:   Perry             Other: And                Type: B
 Comment:   A short comment    Long: This a long        Right:      The
            here.                    message for
                                     test.
=======================================================================
    Name:   Hart              Other: So                 Type: D
 Comment:   Little here.       Long: Here too.          Right:    Right
=======================================================================
    Name:   Holmes            Other: On                 Type: D
 Comment:   A comment here     Long: A short            Right:     Edge
            that is a                message.
            little ongoing,
            so to speak.
=======================================================================
        
</PRE>
</CODE></BLOCKQUOTE>
<P>Note that since there were two picture fields that
started with the '^' character on one line the length of
output records varies according to the length of the two
associated data values.
<P>
<H2><A NAME="ss3.19">3.19 nsq-row</A>
        </H2>

<P>Usage:  <B>nsq-row  [options]  expression</B>
<P>Selects rows from the input rdbtable based on an arbitrary expression
using column names.  Characters that are special to the UNIX shell
must be quoted.
<P>Logical constructors 'or' and 'and' may be used; as well as
'null' to indicate empty data values.  Comparison operators
may be of the form: gt, ge, lt, le, eq, ne, mat, nmat.
The first six are the usual operators, E.g 'name eq Hobbs'
or 'COUNT gt 100'.  The last two stand for 'match' and
'non-match' and are used for pattern matching. They are
exactally the same as using the PERL operators '=&nbsp' or
'!&nbsp' respectively, except that pattern matching can be
specified easier in expressions, as in:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      NAME  mat   /[Hh]obbs/  &lt;&lt;&lt; First letter either case
      NAME  mat   /hobbs/i    &lt;&lt;&lt; any combination of case
      NAME  nmat  /[aeiou]/i  &lt;&lt;&lt; names without vowels
        
</PRE>
</CODE></BLOCKQUOTE>
<P>where 'NAME' and 'COUNT' are column names, of course.
A warning message is produced on STDERR if either of
'mat' or 'nmat' is used with a numeric type column, but
the execution continues.  It does not check the '=&nbsp' or
'!&nbsp' forms.
<P>All of the Comparison operators and Logical constructors
are reserved and should not be used as column names
(they are all lower case and four characters or less).
<P>Since column names and reserved words are parsed by the
program, do not put the entire expression in a single pair
of quotes as that will prevent the parsing.  Also note
that column names and reserved words need to be surrounded
by blank spaces if they are not individually quoted.
For example either form below is fine:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row   NAME    eq   &quot;L Brown&quot;  &lt;  sample
      nsq-row  &quot;NAME&quot;  &quot;eq&quot;  &quot;L Brown&quot;  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>but do not use this form:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row  &quot;NAME  eq  L Brown&quot;  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>This operator reads an rdbtable via STDIN and writes an
rdbtable via STDOUT.  Options may be abbreviated.
<P>   
As an example using the sample rdbtable from the DATA
section (named sample), to select rows that have the NAME
column equal to 'Hansen'  the command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row  NAME  eq  Hansen  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>which would produce:
<P>
<BLOCKQUOTE><CODE>
<PRE>
          NAME    COUNT   TYP     AMT     OTHER   RIGHT
          6       5N      4       5N      8       8&gt;
          Hansen  44      A       23      One     Is
        
</PRE>
</CODE></BLOCKQUOTE>
<P>to select rows that have the TYP column equal to 'A' or
that have the AMT column greater than 1000 the command
would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row  TYP  eq  A  or  AMT  gt  1000  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>producing:
<P>
<BLOCKQUOTE><CODE>
<PRE>
          NAME    COUNT   TYP     AMT     OTHER   RIGHT
          6       5N      4       5N      8       8&gt;
          Bush    44      A       133     Another This
          Hansen  44      A       23      One     Is
          Hart    77      D       1111    So      Right
          Holmes  65      D       1111    On      Edge
        
</PRE>
</CODE></BLOCKQUOTE>
<P>Note that in some rare cases there could be a column
name that is identical to a data value specified in an
expression using another column name that might cause
a problem (this actually happened).  For example if
two column names are 'N' and 'T', and column 'N' has a
data value of 'T', to select all rows where column 'N'
is equal to 'T' the normal command would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row  &lt;  table  N  eq  T
        
</PRE>
</CODE></BLOCKQUOTE>
<P>Unfortunately the 'T' in the expression gets translated
to 'column name T', not used as 'data value T'. That
is, the expression askes for all rows where the data
value of column N equals the data value of column T,
legal, but not what was wanted.  There is a simple
workaround however. The 'T' in the expression can be
escaped with a backslash to prevent the translation to
a column name, as in the revised command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-row  &lt;  table  N  eq  '\T'
        
</PRE>
</CODE></BLOCKQUOTE>
<P>Thus either meaning can be specified, as desired.
<P>
<H2><A NAME="ss3.20">3.20 nsq-search</A>
        </H2>

<P>Usage:
<P><B>nsq-search  [options]  rdbtbl  &lt;  keytbl</B>
<P>or:
<P>nsq-search  [options]  -ind  index_file 
[rdbtbl]  &lt; keytbl
<P>This operator does a fast search of 'rdbtbl' (or
index_file) using a binary search on a key of of
one or more columns. The 'rdbtbl' (or index_file)
must be sorted on the key columns.  Each column in
the key may be of type string or type numeric (but be
carefull with numeric data and exact matches). In the
second form of usage for this operator if 'rdbtbl' is
not given its name will be inferred from the name of
index_file. For example if index_file is 'skb.x.typ'
then the rdbtbl name inferred will be 'skb.rdb'.
<P>The column(s) in the file 'keytbl' specify both the
key column name(s) and the argument values to search
for. File 'Keytbl' is in rdbtable format.
<P>Normally an argument value and a data field must
compare exactally for a match to occur (exact
match). If the paritial match otpion (-part) is
selected, and if the argument value compares with
the initial part of the data field it is considered
a match. This applies to string type data only. Note
that for numeric type data an exact match is always
necessary.
<P>Normally all rows that match will be written to
the new rdbtable, in the same order as in the old
rdbtable. If only a single row key match is appropriate
some execution time can be saved by specifing the
'-sgl' option.
<P>This operator writes an rdbtable via STDOUT.
Options may be abbreviated.  Returns the number of
non-finds at exit.
<P>
<H2><A NAME="ss3.21">3.21 nsq-sort</A>
        </H2>

<P>Usage:  <B>nsq-sort  [options]  [-r]  column
[[-r]  column]  ...</B>
<P>Sorts an rdbtable on one or more columns. Each column may
be sorted in normal (ascending) or reverse (descending)
order. Also a column of monthnames (Jan, Apr, ...) in
any case letters, may be sorted.
<P>This operator reads an rdbtable via STDIN and writes an
rdbtable via STDOUT.  Options may be abbreviated. Uses
the UNIX 'sort' routine.
<P>For example, using the sample data file from the DATA
section (named sample) in the following command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-sort  COUNT  TYP  &lt;  sample
        
</PRE>
</CODE></BLOCKQUOTE>
<P>would produce:
<P>
<BLOCKQUOTE><CODE>
<PRE>
    
      NAME    COUNT   TYP     AMT     OTHER   RIGHT
      6       5N      4       5N      8       8&gt;
      Bush    44      A       133     Another This
      Hansen  44      A       23      One     Is
      Holmes  65      D       1111    On      Edge
      Perry   77      B       244     And     The
      Hart    77      D       1111    So      Right
      Jones   77      X       77      Here    On

    
</PRE>
</CODE></BLOCKQUOTE>
<P>Of course it would look better if it was piped through 'nsq-pr'.
The command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-sort  COUNT  -r  AMT  &lt;  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>would produce:
<P>
<BLOCKQUOTE><CODE>
<PRE>
    
      NAME    COUNT   TYP     AMT     OTHER   RIGHT
      6       5N      4       5N      8       8&gt;
      Bush    44      A       133     Another This
      Hansen  44      A       23      One     Is
      Holmes  65      D       1111    On      Edge
      Hart    77      D       1111    So      Right
      Perry   77      B       244     And     The
      Jones   77      X       77      Here    On

        
</PRE>
</CODE></BLOCKQUOTE>
<P>
<H2><A NAME="ss3.22">3.22 nsq-subtot</A>
        </H2>

<P>Usage:  <B>nsq-subtot  [options]  B_column  ... 
-s  column  ...</B>
<P>This operator lists subtotals of specified column(s)
whenever the value of specified break columns(s)
(B_column(s)) changes.  A single break column will
produce a sub-total of all specified columns on each
line. If there is more than one break column given then
in addition whenever the value of the first break column
changes an additional line will be printed showing the
sub-total for that group.
<P>If no break column is given the first column is used;
if no subtotal column is given then all columns of type
numeric are sub-totaled.
<P>This operator reads an rdbtable via STDIN and writes an
rdbtable via STDOUT.  Options may be abbreviated.
<P>Example rdbtable (named small):
<P>
<BLOCKQUOTE><CODE>
<PRE>

      name      amt    typ   count  n
      6         5N     4     5N     2
      Hansen    39     A     23     3
      Hansen    9      A     3      3
      Hansen    9      B     3      4
      Jones     42     B     144    5
      Jones     4      B     14     5
      Hart      9      C     3      5
      Hart      2      C     55     6
      Hart      2      D     55     6
      Hobbs     57     X     7      4
      Hobbs     5      X     57     4
    
        
</PRE>
</CODE></BLOCKQUOTE>
<P>The output from the command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-subtot  name  -s  amt  &lt;  small  |  nsq-pr
    
</PRE>
</CODE></BLOCKQUOTE>
<P>would be:
<P>
<BLOCKQUOTE><CODE>
<PRE>

      name      amt
      ------  -----
      Hansen     57
      Jones      46
      Hart       13
      Hobbs      62

        
</PRE>
</CODE></BLOCKQUOTE>
<P>The output from the command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-subtot  name  typ  -s  amt  count  &lt;  small  |  nsq-pr
        
</PRE>
</CODE></BLOCKQUOTE>
<P>is shown in Table 18.
<P>
<BLOCKQUOTE><CODE>
<PRE>

                             Table 18
  
                  OUTPUT FROM THE NSQ-SUBTOT OPERATOR

      name    typ     amt  count
      ------  ----  -----  -----
      Hansen  A        48     26
      Hansen  B         9      3
                       57     29
      
      Jones   B        46    158
                       46    158
      
      Hart    C        11     58
      Hart    D         2     55
                       13    113
      
      Hobbs   X        62     64
                       62     64
      
        
</PRE>
</CODE></BLOCKQUOTE>
<P>
<H2><A NAME="ss3.23">3.23 nsq-summ</A>
        </H2>

<P>Usage:  <B>nsq-summ  [options]  [column ...]</B>
<P>Produces "summary" information about the rdbtable. If
no columns are given then information about all columns
is produced.  A Count of the data rows is always shown.
<P>This operator reads an rdbtable via STDIN and writes a
summary report via STDOUT.  Options may be abbreviated.
<P>
<H2><A NAME="ss3.24">3.24 nsq-t2l</A>
    </H2>

<P>Usage:  <B>nsq-t2l  [options]</B>
<P>Converts an rdbtable to "list" format. Long data fields are
folded. This operator is mainly used by other operators.
<P>This NoSQL operator reads an rdbtable from STDIN and writes
an rdbtable to STDOUT.  Options may be abbreviated.
<P>
<H2><A NAME="ss3.25">3.25 nsq-tabletolist     </A>
</H2>

<P>Usage:  <B>nsq-tabletolist  [options]</B>
<P>Section yet to be written. See 'nsq-tabletolist -help' in the
meantime.
<P>
<H2><A NAME="ss3.26">3.26 nsq-tee</A>
        </H2>

<P>Usage:  <B>nsq-tee  [options]  rdbtable</B>
<P>Reads an rdbtable via STDIN and [over]writes the
rdbtable given as a command line argument,
abiding by the NoSQL table locking scheme. This operator
is named after yhe UNIX utility <EM>tee</EM>.
<P>This operator can be safely used in constructs like :
<P>
<BLOCKQUOTE><CODE>
<PRE>
          nsq-compute ... &lt; table.rdb | nsq-tee table.rdb | ...
    
</PRE>
</CODE></BLOCKQUOTE>
<P>without worring that the original input file table.rdb
be destroyed before the first pipeline has completed.
<P>If no output stream is desired on STDOUT then the latter
can be directed to /dev/null, as usual:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-compute ... &lt; table.rdb | nsq-tee table.rdb &gt; /dev/null
    
</PRE>
</CODE></BLOCKQUOTE>
<P>This operator is especially meant to be used in programs
that need to modify rdbtables 'in place'.  It has been
designed for the GNU Bourne Again Shell (Bash), but it
should work just as well with other Bourne compatible shells.
<P>Note: the output table name given on the command line 
may not start with the carachter "<B>,</B>" , to prevent
possible clobbering of the backup table.
<P>
<H2><A NAME="ss3.27">3.27 nsq-uniq</A>
        </H2>

<P>Usage:  <B>nsq-uniq  [options]  column  ...</B>
<P>Reads the input rdbtable and compares adjacent rows. The
second and succeeding copies of repeated rows, considering
only the selected columns, are removed.  That is, adjacent
rows are considered equal if the data values in all of
the selected columns are equal.  The remaining rows are
written to the output rdbtable.
<P>Note that repeated rows must be adjacent in order to be
found. Normally this means that the input rdbtable should
be sorted on the selected columns for this capability to
work properly.
<P>This NoSQL operator reads an rdbtable from STDIN and writes
an rdbtable to STDOUT.  Options may be abbreviated.
<P>
<H2><A NAME="ss3.28">3.28 nsq-updseq     </A>
</H2>

<P>Usage:  <B>nsq-updseq  [options]</B>
<P>Section yet to be written. See 'nsq-updseq -help' in the
meantime.
<P>
<H2><A NAME="ss3.29">3.29 nsq-valid</A>
        </H2>

<P>Usage:  <B>nsq-valid  [options]  [file ...]</B>
<P>Validates the structure of one or more rdbtables.  Checks
number of data fields per line, max width of column names
and data values, and checks numeric data type values.
Reports errors by line number and column name.
<P>Reads from STDIN if filenames are not given.  Writes
diagnostic information on STDOUT.  Options may be
abbreviated.
<P>If there is more than one file given each file will be
identified on the output.
<P>The '-size' option has proven very useful as it shows the
actual size of the largest data value for each column,
in addition to the template information.  The command:
<P>
<BLOCKQUOTE><CODE>
<PRE>
      nsq-valid  -size  sample
    
</PRE>
</CODE></BLOCKQUOTE>
<P>shows the following output:
<P>
<BLOCKQUOTE><CODE>
<PRE>
    
    
          0                 NAME  6     6
          1                COUNT  5N    2
          2                  TYP  4     1
          3                  AMT  5N    4
          4                OTHER  8     7
          5                RIGHT  8&gt;    5
          Columns: 6, Rows: 6, File format valid  sample

        
</PRE>
</CODE></BLOCKQUOTE>
<P>The last two columns above show the defined size of each
column in the rdbtable, and the actual maximum size of
the data values for each column in the rdbtable.
<P>
<HR>
<A HREF="NoSQL-2.html">Previous</A>
<A HREF="NoSQL-4.html">Next</A>
<A HREF="NoSQL.html#toc3">Contents</A>
</BODY>
</HTML>