File: ffe.html

package info (click to toggle)
ffe 0.1.6-1
links: PTS
area: main
in suites: etch, etch-m68k
size: 1,020 kB
ctags: 273
sloc: sh: 3,529; ansic: 2,619; makefile: 67
file content (998 lines) | stat: -rw-r--r-- 39,225 bytes
<html lang="en">
<head>
<title>ffe - flat file extractor</title>
<meta http-equiv="Content-Type" content="text/html">
<meta name="description" content="ffe - flat file extractor">
<meta name="generator" content="makeinfo 4.8">
<link title="Top" rel="top" href="#Top">
<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
<!--
This file documents version 0.1.6 of `ffe', a flat file extractor.

Copyright (C) 2006 Timo Savinen

     Permission is granted to make and distribute verbatim copies of
     this manual provided the copyright notice and this permission
     notice are preserved on all copies.

     Permission is granted to copy and distribute modified versions of
     this manual under the conditions for verbatim copying, provided
     that the entire resulting derived work is distributed under the
     terms of a permission notice identical to this one.

     Permission is granted to copy and distribute translations of this
     manual into another language, under the above conditions for
     modified versions.
   -->
<meta http-equiv="Content-Style-Type" content="text/css">
<style type="text/css"><!--
  pre.display { font-family:inherit }
  pre.format  { font-family:inherit }
  pre.smalldisplay { font-family:inherit; font-size:smaller }
  pre.smallformat  { font-family:inherit; font-size:smaller }
  pre.smallexample { font-size:smaller }
  pre.smalllisp    { font-size:smaller }
  span.sc    { font-variant:small-caps }
  span.roman { font-family:serif; font-weight:normal; } 
  span.sansserif { font-family:sans-serif; font-weight:normal; } 
body {
	margin: 1%;
	padding: 0 5%;
	background: white;
	font-family: serif;
	text-align: justify;
}

h1,h2,h3,h4,h5 {
    padding: 0.5em 0 0 0;
    font-weight: bold;
    font-family: sans-serif;
}

h1 {
    padding: 0.5em 0 0.5em 1em;
    color: white;
    background: #575;
}

pre {
  margin: 0;
  padding: 0.5em 0.5em 0.5em 0;
}

pre.example {
  background: #eee;
}

pre.verbatim, .menu {
  border: solid 1px gray;
  background: white;
  padding-bottom: 1em;
}

div.node {
  background: #ccc;
  margin: 0;
  padding: 0 1.5em;
  font-weight: lighter;
  color: #000;
  text-align: right;
}

.node a {
  color: #770000;
}

.node a:visited {
  color: #550000;
}

dd, li {
  padding-top: 0.1em;
  padding-bottom: 0.1em;
}

samp {
	font: inherit;
}

code {
	font-size: inherit;
    font-weight: bold;
}

pre, code { 
	font-family: monospace;
}

.command, .file {
   font-family: monospace;
} 

div.node hr {
    display:none;
}
--></style>
</head>
<body>
<h1 class="settitle">ffe - flat file extractor</h1>
<div class="node">
<p><hr>
<a name="Top"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Overview">Overview</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#dir">(dir)</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#dir">(dir)</a>

</div>

<h2 class="unnumbered">ffe</h2>

<p>This file documents version 0.1.6 of <samp><span class="command">ffe</span></samp>, a flat file extractor.

   <p>Copyright &copy; 2006 Timo Savinen

   <blockquote>
Permission is granted to make and distribute verbatim copies of
this manual provided the copyright notice and this permission notice
are preserved on all copies.

        <p>Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the entire
resulting derived work is distributed under the terms of a permission
notice identical to this one.

        <p>Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions. 
</blockquote>

<!-- All the menus can be updated with the EMACS command -->
<!-- texinfo-all-menus-update, which is normally bound to C-c C-u C-a. -->
<ul class="menu">
<li><a accesskey="1" href="#Overview">Overview</a>:                     Preliminary information. 
<li><a accesskey="2" href="#Samples">Samples</a>:                      Samples using <samp><span class="command">ffe</span></samp>. 
<li><a accesskey="3" href="#Invoking-ffe">Invoking ffe</a>:                 How to run <samp><span class="command">ffe</span></samp>. 
<li><a accesskey="4" href="#ffe-configuration">ffe configuration</a>:            How <samp><span class="command">ffe</span></samp> works. 
<li><a accesskey="5" href="#Problems">Problems</a>:                     Reporting bugs. 
</ul>

<div class="node">
<p><hr>
<a name="Overview"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Samples">Samples</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Top">Top</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Top">Top</a>

</div>

<h2 class="chapter">1 Preliminary information</h2>

<p><a name="index-greetings-1"></a><a name="index-overview-2"></a>
The <samp><span class="command">ffe</span></samp> program is a used to extract fields from flat files and to print them in different
formats. The input file structure and printing definitions are defined in a configuration file, which
is always required. Default configuration file is <samp><span class="file">~/.fferc</span></samp>.

   <p><samp><span class="command">ffe</span></samp> is a command-line tool developed for GNU/linux and unix systems. <samp><span class="command">ffe</span></samp> can read from
standard input and write to standard output, so it can be used as a part of a pipeline.

<div class="node">
<p><hr>
<a name="Samples"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Invoking-ffe">Invoking ffe</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Overview">Overview</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Top">Top</a>

</div>

<h2 class="chapter">2 Samples using <samp><span class="command">ffe</span></samp></h2>

<p><a name="index-sample-3"></a>
One example of using <samp><span class="command">ffe</span></samp> for printing personnel information in xml format from fixed length flat file:

<pre class="example">     $ cat personnel
     john     Ripper       23
     Scott    Tiger        45
     Mary     Moore        41
     $
</pre>
   <p class="noindent">A file <samp><span class="file">personnel</span></samp> contains three fixed length fields: `<samp><span class="samp">FirstName</span></samp>', `<samp><span class="samp">LastName</span></samp>' and `<samp><span class="samp">Age</span></samp>',
their respective lengths are 9,13 and 2.

<p class="noindent">In order to print data above in xml, following configuration file must be available:

<pre class="example">     $cat personnel.fferc
     structure personel {
         type fixed
         output xml
         record person {
             field FirstName 9
             field LastName  13
             field Age 2
         }
     }
     
     output xml {
         file_header "&lt;?xml version=\"1.0\" encoding=\"ISO-8859-1\"?&gt;\n"
         data "&lt;%n&gt;%t&lt;/%n&gt;\n"
         record_header "&lt;%r&gt;\n"
         record_trailer "&lt;/%r&gt;\n"
         indent " "
     }
     $
</pre>
   <p class="noindent">Using ffe:

<pre class="example">     $ffe -c personnel.fferc personnel
     &lt;?xml version="1.0" encoding="ISO-8859-1"?&gt;
      &lt;person&gt;
       &lt;FirstName&gt;john&lt;/FirstName&gt;
       &lt;LastName&gt;Ripper&lt;/LastName&gt;
       &lt;Age&gt;23&lt;/Age&gt;
      &lt;/person&gt;
      &lt;person&gt;
       &lt;FirstName&gt;Scott&lt;/FirstName&gt;
       &lt;LastName&gt;Tiger&lt;/LastName&gt;
       &lt;Age&gt;45&lt;/Age&gt;
      &lt;/person&gt;
      &lt;person&gt;
       &lt;FirstName&gt;Mary&lt;/FirstName&gt;
       &lt;LastName&gt;Moore&lt;/LastName&gt;
       &lt;Age&gt;41&lt;/Age&gt;
      &lt;/person&gt;
     $
</pre>
   <div class="node">
<p><hr>
<a name="Invoking-ffe"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#ffe-configuration">ffe configuration</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Samples">Samples</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Top">Top</a>

</div>

<h2 class="chapter">3 How to run <samp><span class="command">ffe</span></samp></h2>

<p><a name="index-running-ffe-4"></a><a name="index-using-5"></a>
<samp><span class="command">ffe</span></samp> is a command-line tool. Normally <samp><span class="command">ffe</span></samp> can be invoked as:

   <p><code>ffe -o OUTPUTFILE INPUTFILE</code>

<p class="noindent"><samp><span class="command">ffe</span></samp> uses the definitions from <samp><span class="file">~/.fferc</span></samp> and tries to guess the inputfile
structure using the first 10 000 lines or 1 MB of input data.

<p class="noindent">If the structure cannot be guessed the option <samp><span class="option">-s</span></samp> must be used.

<ul class="menu">
<li><a accesskey="1" href="#Invocation">Invocation</a>:       Program invocation
<li><a accesskey="2" href="#Configuration">Configuration</a>:    Input and printing definitions
<li><a accesskey="3" href="#Guessing">Guessing</a>:         How ffe identifies input structure
<li><a accesskey="4" href="#Limits">Limits</a>:           Limitations
</ul>

<div class="node">
<p><hr>
<a name="Invocation"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Configuration">Configuration</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Invoking-ffe">Invoking ffe</a>

</div>

<h3 class="section">3.1 Program invocation</h3>

<p><a name="index-options-6"></a>
The format for running the <samp><span class="command">ffe</span></samp> program is:

<pre class="example">     ffe <var>option</var> ...
</pre>
   <p class="noindent"><samp><span class="command">ffe</span></samp> supports the following options:

<!-- Formatting copied from the Texinfo 4.0 manual. -->
<dl>
<dt><code>-c </code><var>file</var><dt><code>--configuration=</code><var>file</var><dd>Configuration is read from <var>file</var>, instead of <samp><span class="file">~/.fferc</span></samp>.

     <br><dt><code>-s </code><var>structure</var><dt><code>--structure=</code><var>structure</var><dd>Use structure <var>structure</var> for input file, suppresses guessing.

     <br><dt><code>-p </code><var>output</var><dt><code>--print=</code><var>output</var><dd>Use output format <var>output</var> for printing. If not defined, then the record or structure related
output format is used. Printing can be suppressed using format <var>no</var>.

     <br><dt><code>-o </code><var>file</var><dt><code>--output=</code><var>file</var><dd>Write output to <var>file</var> instead of standard output.

     <br><dt><code>-f </code><var>list</var><dt><code>--field-list=</code><var>list</var><dd>Print only fields expresses in comma separated list <var>list</var>. Order of fields in
<var>list</var> defines also the printing order.

     <br><dt><code>-e </code><var>expression</var><dt><code>--expression=</code><var>expression</var><dd>Print only those records for which the <var>expression</var> evaluates to true.

     <br><dt><code>-a</code><dt><code>--and</code><dd>Expressions are combined with logical and, default is logical or.

     <br><dt><code>-v</code><dt><code>--invert-match</code><dd>Print only those records which don't match the expression.

     <br><dt><code>-l</code><dt><code>--loose</code><dd>Normally <samp><span class="command">ffe</span></samp> stops when it encounteres an input line which doesn't match any of
the records in selected structure. Defining this option causes <samp><span class="command">ffe</span></samp> continue despite the error.

     <br><dt><code>-?</code><dt><code>--help</code><dd>Print an informative help message describing the options and then exit
successfully.

     <br><dt><code>-V</code><dt><code>--version</code><dd>Print the version number of <samp><span class="command">ffe</span></samp> and then exit successfully. 
</dl>

   <p>All remaining options are names of input files, if no input files are specified or <code>-</code> is given, then the standard input is read.

<h4 class="subheading">Expressions (option <samp><span class="option">-e</span></samp>, <samp><span class="option">--expression</span></samp>)</h4>

<p>Expression can be used to select specific records comparing field values. 
Expression has syntax <var>field</var><strong>x</strong><var>value</var>, where <strong>x</strong> is the comparison operator. 
Expression is used to compare field's contents to <var>value</var> and if comparison is successfull
the record is printed. Several expressions can be defined and at least one must evaluate to true in
order to print a record. If option <samp><span class="option">-a</span></samp> is defined all expressions must evaluate to true.

<p class="noindent">Expressions can be defined as:

     <dl>
<dt><var>field</var><strong>=</strong><var>value</var><dd>Field <var>field</var> is equal to <var>value</var>.

     <br><dt><var>field</var><strong>^</strong><var>value</var><dd>Field <var>field</var> starts with <var>value</var>.

     <br><dt><var>field</var><strong>~</strong><var>value</var><dd>Field <var>field</var> contains <var>value</var>.

     <br><dt><var>field</var><strong>!</strong><var>value</var><dd>Field <var>field</var> is not equal to <var>value</var>. 
</dl>

<div class="node">
<p><hr>
<a name="Configuration"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Guessing">Guessing</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Invocation">Invocation</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Invoking-ffe">Invoking ffe</a>

</div>

<h3 class="section">3.2 Configuration</h3>

<p><a name="index-configuration-7"></a>
<samp><span class="command">ffe</span></samp> uses a configuration file in order to read the input file and print the output. 
Default configuration file is <samp><span class="file">~/.fferc</span></samp>.

   <p>Configuration file for <samp><span class="command">ffe</span></samp> is a text file. The file may contain empty lines. 
Commands are case-sensitive. Comments  begin with the <code>#</code>-character and end at the end of the line. 
The <code>string</code> definitions can be enclosed in double quotation <code>"</code> characters. 
<code>char</code> is a single character. <code>string</code> and <code>char</code> can contain following escape codes:
<code>\a</code>, <code>\b</code>, <code>\t</code>, <code>\n</code>, <code>\v</code>, <code>\f</code>, <code>\r</code>, <code>\"</code> and <code>\#</code>. 
A backslash can be escaped as <code>\\</code>.

   <p>Configuration has two main parts: the structure, which defines the input file structure and
the output, which defines how the input data is formated for output.

<h4 class="subheading">Common syntax</h4>

<p>Common syntax for configuration file is:

<pre class="example">     #comment
     structure <var>name</var> {
         <i>option value</i> ...
         ...
         record <var>name</var> {
             <i>option value</i> ...
             ...
         }
         record <var>name</var> {
             <i>option value</i> ...
             ...
         }
         ...
     }
     structure <var>name</var> {
         ...
     }
     ...
     output <var>name</var> {
         <i>option value</i> ...
         ...
     }
     output <var>name</var> {
         ...
     }
     ...
</pre>
   <h4 class="subheading">Structure</h4>

<p>Keyword <code>structure</code> is used to define an input file content. An input file can contain several
types of records (or lines). E.g. file can have a header, data and trailer record types. Records
must be distinguishable from each other, this can be achieved defining different 'keys'
(<code>id</code> in record definition) or having different line lengths (for fixed length structure) or different count
of fields (for separated structure) for different records.

<p class="noindent">Typically a <code>structure</code> maps to a file and a <code>records</code> maps to a line in the file.

<p class="noindent">A structure is defined as:
<br>

<pre class="example">     structure <var>name</var> {
         <i>option value</i> ...
         ...
     }
</pre>
   <p class="noindent">A structure can contain following options:

     <dl>
<dt><code>type fixed|separated [</code><var>char</var><code>] [*]</code><dd>The fields in the input are fixed length fields or separated by <var>char</var>. If * is  defined,
multiple sequential separators are considered as one. Default separator is comma.

     <br><dt><code>quoted [</code><var>char</var><code>]</code><dd>Fields may be quoted with char, default quotation mark is double quotation mark '"'. 
A quotation mark is assumed be escaped as \<var>char</var> or doubling the mark as <var>charchar</var> in input. 
Non escaped quotation marks are not preserved in output.

     <br><dt><code>header first|all|no</code><dd>Controls the occurrence of the header line. Default is no. If set as <em>first</em> or <em>all</em>, the first line
of the first input file is considered as header line containing the names of  the  fields. <em>first</em>
means  that  only  the  first  file  has  a header, <em>all</em> means means that all files have a header,
allthough the names are still taken from the the header of the first file. Header line is handled
according the record definition, meaning that the name positions, separators etc. are the same as
for fields.

     <br><dt><code>output </code><var>name</var><dd>All records belonging this structure are printed according output format name. 
Default is to use output named as `<samp><span class="samp">default</span></samp>'.

     <br><dt><code>record </code><var>name</var><code> {</code><i>options</i><code> ...}</code><dd>Defines one record for a structure. A structure can contain several record types. 
</dl>

<h4 class="subheading">Record</h4>

<p>A record defines one type of input line in a file. Different records can be distinguished using
the <code>id</code> option or different line lengths or field counts.

<p class="noindent">A record can contain following options:

     <dl>
<dt><code>id </code><var>position</var> <var>string</var><dd>Identifies a record in the input file. Records are identified by the string in input record position
<var>position</var>. For fixed length input the position is the byte position of input record and for
separated input the position is the <var>position</var>'th field of the input record. Positions start from one.

     <p>A record definition can contain several id's, then all id'd must match the input line
(<code>id</code>'s are <em>anded</em>).

     <br><dt><code>field </code><var>name</var><code>|FILLER|* [</code><var>length</var><code>]</code><dd>Defines  one  field  in  input  structure.  length is mandatory for fixed length input structure. 
Length is also used for printing the fields in fixed length format (directive <code>%D</code> in output definitions). 
The order of fields in configuration file is essential, it defines the field order in a record.

     <p>If <em>*</em> is defined instead of the name, then the <var>name</var> will be the ordinal number of the field,
or if the <code>header</code> option has value <em>first</em> or <em>all</em>, then the name of the field will taken from
the header line (first line of the input).

     <p>If field is named as <em>FILLER</em>, the field will not appear in output. 
<br><dt><code>fields-from </code><var>record</var><dd>Fields for this record are the same as for record <var>record</var>. <code>field</code> and <code>fields-from</code> are mutually
exclusive. 
</dl>

<h4 class="subheading">Output</h4>

<p>Keyword <code>output</code> defines one output format for formatting the flat file data. Formatting
is controlled using options and printf-style directives. An output defition is allways independent
from structure, so one output format can be used with different input file formats.

   <p>Actual formatting and printing is controlled using <em>pictures</em> in output options. Pictures can contain
following printf-style directives:

     <dl>
<dt><code>%f</code><dd>Name of the input file. 
<br><dt><code>%s</code><dd>Name of the current structure. 
<br><dt><code>%r</code><dd>Name of the current record. 
<br><dt><code>%o</code><dd>Input record number in current file. 
<br><dt><code>%O</code><dd>Input record number starting from the first file. 
<br><dt><code>%n</code><dd>Field name. 
<br><dt><code>%t</code><dd>Field contents, without leading and trailing whitespaces. 
<br><dt><code>%d</code><dd>Field contents. 
<br><dt><code>%D</code><dd>Field contents, right padded to the field length (requires length definition for the field). 
<br><dt><code>%e</code><dd>Does not print anything, causes still the "field empty" check to be performed. 
Can be  used  when only the names of non-empty fields should be printed. 
<br><dt><code>%%</code><dd>Percent sign. 
</dl>

<p class="noindent">Output options:
     <dl>
<dt><code>file_header </code><var>picture</var><dd><var>picture</var> is printed once before file contents. 
<br><dt><code>file_trailer </code><var>picture</var><dd><var>picture</var> is printed once after file contents. 
<br><dt><code>header </code><var>picture</var><dd>If defined, then the header line describing the field names is printed before records. 
Every field name is printed according the <var>picture</var> using the same separator and fields length as
defined for the fields. Picture can contain only <code>%n</code> directive. 
<br><dt><code>data </code><var>picture</var><dd>Field contents is printed according <var>picture</var>. 
<br><dt><code>separator </code><var>string</var><dd>All fields are terminated by <var>string</var>, except the last field of the record. 
Default is not to print separator. 
<br><dt><code>record_header </code><var>picture</var><dd>All records are started by <var>picture</var>. Default is not to print the record header. 
<br><dt><code>record_trailer </code><var>picture</var><dd>All records are ended with <var>picture</var>. Default is newline. 
<br><dt><code>justify left|right|</code><var>char</var><dd>The output from the <code>data</code> option is left or right justified. 
<var>char</var> justifies output according the first occurrence of <var>char</var>
in the data picture. Default is left. 
<br><dt><code>indent </code><var>string</var><dd>Record contents is intended by <var>string</var>. 
Field contents is intended by two times the string. Default is not to indent. 
<br><dt><code>field-list </code><var>field1</var><code>,</code><var>field2</var><code>,...</code><dd>Only fields named as <var>field1</var>,<var>field2</var>,<small class="dots">...</small> are printed, same effect as has option <samp><span class="option">-f</span></samp>. 
Default is print all fields. Fields are also printed in the same order as they are listed. 
<br><dt><code>no-data-print yes|no</code><dd>If <code>field-list</code> is defined and and this is set as no and none of the fields in <code>field-list</code>
does not belong to the current record, then the <code>record_header</code> and <code>record_trailer</code> are not printed. 
Default is yes. 
<br><dt><code>field-empty-print yes|no</code><dd>When set as no, nothing is printed for fields which consist entirely of characters from <code>empty-chars</code>. 
If none of the fields of a record are printed, then the printing of <code>record_trailer</code> is also suppressed. 
Default is yes. 
<br><dt><code>empty-chars </code><var>string</var><dd><var>string</var> defines a set of characters which define an "empty" field. Default is
"&nbsp;\f\n\r\t\v"<!-- /@w --> (space, form-feed, newline, carriage return, horizontal tab and vertical tab). 
</dl>

<div class="node">
<p><hr>
<a name="Guessing"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Limits">Limits</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Configuration">Configuration</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Invoking-ffe">Invoking ffe</a>

</div>

<h3 class="section">3.3 Guessing</h3>

<p><a name="index-guess-8"></a>If <samp><span class="option">-s</span></samp> is not given, <samp><span class="command">ffe</span></samp> tries to guess the input structure. 
<samp><span class="command">ffe</span></samp> reads first 10 000 lines or 1 MB of input data and tries to match the structure definitions
from configuration file to input stream. If all lines match one and only one structure, the structure is used
for reading the input file.

<p class="noindent">Guessing uses following execution cycle:

     <ol type=1 start=1>
<li>Input line is read
<li>All record <code>id</code>'s are compared to the input line, if all <code>id</code>'s of a record match
the input line and the
records line length matches the total length (or total count for separated structure) of the fields,
the record is considered to match the input line. If there are no <code>id</code>'s,
only the line length or field count is checked. 
<li>If all lines match at least one of the records in a particular structure, the structure is considered as selected. 
There must be only one structure matching all lines used for guessing.
        </ol>

<div class="node">
<p><hr>
<a name="Limits"></a>
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Guessing">Guessing</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Invoking-ffe">Invoking ffe</a>

</div>

<h3 class="section">3.4 Limitations</h3>

<p><a name="index-big-files-9"></a><a name="index-limits-10"></a>
At least in GNU/Linux <samp><span class="command">ffe</span></samp> should be able to handle big files (&gt; 4 GB), other
systems are  not tested.

<div class="node">
<p><hr>
<a name="ffe-configuration"></a>
Next:&nbsp;<a rel="next" accesskey="n" href="#Problems">Problems</a>,
Previous:&nbsp;<a rel="previous" accesskey="p" href="#Invoking-ffe">Invoking ffe</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Top">Top</a>

</div>

<h2 class="chapter">4 How <samp><span class="command">ffe</span></samp> works</h2>

<p>Following examples use two different input files:

<h4 class="subheading">Fixed length example</h4>

<p>Fixed length personnel file with header and trailer, line (record) is identified by the
first byte (H = Header, E = Employee, B = Boss, T = trailer).
<pre class="example">     $cat personnel.fix
     H2006-02-25
     EJohn     Ripper       23
     BScott    Tiger        45
     EMary     Moore        41
     ERidge    Forrester    31
     T0004
     $
</pre>
   <p class="noindent">Structure for reading file above. Note that record `<samp><span class="samp">boss</span></samp>' reuses fields from `<samp><span class="samp">employee</span></samp>'.

<pre class="example">     structure personel_fix {
         type fixed
         record header {
             id 1 H
             field type 1
             field date 10
         }
         record employee {
             id 1 E
             field type 1
             field FirstName 9
             field LastName  13
             field Age 2
         }
         record boss {
             id 1 B
             fields-from employee
         }
         record trailer {
             id 1 T
             field type 1
             field count 4
         }
     }
</pre>
   <h4 class="subheading">Separated example</h4>

<p>Sama file as above, but now separated by colon.

<pre class="example">     $cat personnel.sep
     H,2006-02-25
     E,john,Ripper,23
     B,Scott,Tiger,45
     E,Mary,Moore,41
     E,Ridge,Forrester,31
     T,0004
     $
</pre>
   <p class="noindent">Structure for reading file above. Note that the field lengths are not needed in separated format.

<pre class="example">     structure personel_sep {
         type separated ,
         record header {
             id 1 H
             field type
             field date
         }
         record employee {
             id 1 E
             field type
             field FirstName
             field LastName
             field Age
         }
         record boss {
             id 1 B
             fields-from employee
         }
         record trailer {
             id 1 T
             field type
             field count
         }
     }
</pre>
   <h4 class="subheading">Printing in XML-format</h4>

<p>Data in examples above can be printed in xml using output definition like:

<pre class="example">     output xml {
         file_header "&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;\n"
         data "&lt;%n&gt;%t&lt;/%n&gt;\n"
         record_header "&lt;%r&gt;\n"
         record_trailer "&lt;/%r&gt;\n"
         indent " "
     }
</pre>
   <p class="noindent">Example output using command (assuming definitions above are saved in ~/.fferc)

   <p><code>ffe -p xml personnel.sep</code>

<pre class="example">     &lt;?xml version="1.0" encoding="UTF-8"?&gt;
      &lt;header&gt;
       &lt;type&gt;H&lt;/type&gt;
       &lt;date&gt;2006-02-25&lt;/date&gt;
      &lt;/header&gt;
      &lt;employee&gt;
       &lt;type&gt;E&lt;/type&gt;
       &lt;FirstName&gt;john&lt;/FirstName&gt;
       &lt;LastName&gt;Ripper&lt;/LastName&gt;
       &lt;Age&gt;23&lt;/Age&gt;
      &lt;/employee&gt;
      &lt;boss&gt;
       &lt;type&gt;B&lt;/type&gt;
       &lt;FirstName&gt;Scott&lt;/FirstName&gt;
       &lt;LastName&gt;Tiger&lt;/LastName&gt;
       &lt;Age&gt;45&lt;/Age&gt;
      &lt;/boss&gt;
      &lt;employee&gt;
       &lt;type&gt;E&lt;/type&gt;
       &lt;FirstName&gt;Mary&lt;/FirstName&gt;
       &lt;LastName&gt;Moore&lt;/LastName&gt;
       &lt;Age&gt;41&lt;/Age&gt;
      &lt;/employee&gt;
      &lt;employee&gt;
       &lt;type&gt;E&lt;/type&gt;
       &lt;FirstName&gt;Ridge&lt;/FirstName&gt;
       &lt;LastName&gt;Forrester&lt;/LastName&gt;
       &lt;Age&gt;31&lt;/Age&gt;
      &lt;/employee&gt;
      &lt;trailer&gt;
       &lt;type&gt;T&lt;/type&gt;
       &lt;count&gt;0004&lt;/count&gt;
      &lt;/trailer&gt;
</pre>
   <h4 class="subheading">Printing sql-commands</h4>

<p>Data in examples above can be loaded to database by generated sql-commands. Note that the header and trailer
are not loaded, because only fields `<samp><span class="samp">FirstName</span></samp>',`<samp><span class="samp">LastName</span></samp>' and `<samp><span class="samp">Age</span></samp>' are printed and `<samp><span class="samp">no-data-print</span></samp>'
is set as no. This prevents the `<samp><span class="samp">record_header</span></samp>' and `<samp><span class="samp">record_trailer</span></samp>' to be printed for file header and trailer.

<pre class="example">     output sql {
         file_header "delete table boss;\ndelete table employee;\n"
         record_header "insert into %r values("
         data "'%t'"
         separator ","
         record_trailer ");\n"
         file_trailer "commit\nquit\n"
         no-data-print no
         field-list FirstName,LastName,Age
     }
</pre>
   <p class="noindent">Output from command

   <p><code>ffe -p sql personnel.sep</code>

<pre class="example">     delete table boss;
     delete table employee;
     insert into employee values('john','Ripper','23');
     insert into boss values('Scott','Tiger','45');
     insert into employee values('Mary','Moore','41');
     insert into employee values('Ridge','Forrester','31');
     commit
     quit
</pre>
   <h4 class="subheading">Human readable output</h4>

<p>This output format shows the fields suitable for displaying in screen or printing.

<pre class="example">     output nice {
         record_header "%s - %r - %f - %o\n"
         data "%n=%t\n"
         justify =
         indent " "
     }
</pre>
   <p class="noindent">Output from command

   <p><code>ffe -p nice personnel.fix</code>
<pre class="example">      personel - header - personnel.fix - 1
       type=H
       date=2006-02-25
     
      personel - employee - personnel.fix - 2
            type=E
       FirstName=John
        LastName=Ripper
             Age=23
     
      personel - boss - personnel.fix - 3
            type=B
       FirstName=Scott
        LastName=Tiger
             Age=45
     
      personel - employee - personnel.fix - 4
            type=E
       FirstName=Mary
        LastName=Moore
             Age=41
     
      personel - employee - personnel.fix - 5
            type=E
       FirstName=Ridge
        LastName=Forrester
             Age=31
     
      personel - trailer - personnel.fix - 6
        type=T
       count=0004
</pre>
   <h4 class="subheading">HTML table</h4>

<p>Personnel data can be displayed as HTML table using output like:

<pre class="example">     output html {
         file_header "&lt;html&gt;\n&lt;head&gt;\n&lt;/head&gt;\n&lt;body&gt;\n&lt;table border=\"1\"&gt;\n&lt;tr&gt;\n"
         header "&lt;th&gt;%n&lt;/th&gt;\n"
         record_header "&lt;tr&gt;\n"
         data "&lt;td&gt;%t&lt;/td&gt;\n"
         file_trailer "&lt;/table&gt;\n&lt;/body&gt;\n&lt;/html&gt;\n"
         no-data-print no
     }
</pre>
   <p class="noindent">Output from command

   <p><code>ffe -p html -f FirstName,LastName,Age personnel.fix</code>
<pre class="example">     &lt;html&gt;
     &lt;head&gt;
     &lt;/head&gt;
     &lt;body&gt;
     &lt;table border="1"&gt;
     &lt;tr&gt;
     &lt;th&gt;FirstName&lt;/th&gt;
     &lt;th&gt;LastName&lt;/th&gt;
     &lt;th&gt;Age&lt;/th&gt;
     
     &lt;tr&gt;
     &lt;td&gt;John&lt;/td&gt;
     &lt;td&gt;Ripper&lt;/td&gt;
     &lt;td&gt;23&lt;/td&gt;
     
     &lt;tr&gt;
     &lt;td&gt;Scott&lt;/td&gt;
     &lt;td&gt;Tiger&lt;/td&gt;
     &lt;td&gt;45&lt;/td&gt;
     
     &lt;tr&gt;
     &lt;td&gt;Mary&lt;/td&gt;
     &lt;td&gt;Moore&lt;/td&gt;
     &lt;td&gt;41&lt;/td&gt;
     
     &lt;tr&gt;
     &lt;td&gt;Ridge&lt;/td&gt;
     &lt;td&gt;Forrester&lt;/td&gt;
     &lt;td&gt;31&lt;/td&gt;
     
     &lt;/table&gt;
     &lt;/body&gt;
     &lt;/html&gt;
</pre>
   <h4 class="subheading">Using expression</h4>

<p>Printing only Scott's record using expression with previous example:

   <p><code>ffe -p html -f FirstName,LastName,Age -e FirstName^Scott personnel.fix</code>
<pre class="example">     &lt;html&gt;
     &lt;head&gt;
     &lt;/head&gt;
     &lt;body&gt;
     &lt;table border="1"&gt;
     &lt;tr&gt;
     &lt;th&gt;FirstName&lt;/th&gt;
     &lt;th&gt;LastName&lt;/th&gt;
     &lt;th&gt;Age&lt;/th&gt;
     
     &lt;tr&gt;
     &lt;td&gt;Scott&lt;/td&gt;
     &lt;td&gt;Tiger&lt;/td&gt;
     &lt;td&gt;45&lt;/td&gt;
     
     &lt;/table&gt;
     &lt;/body&gt;
     &lt;/html&gt;
</pre>
   <h4 class="subheading">The whole configuration file used in examples</h4>

<pre class="example">     structure personel_fix {
         type fixed
         record header {
             id 1 H
             field type 1
             field date 10
         }
         record employee {
             id 1 E
             field type 1
             field FirstName 9
             field LastName  13
             field Age 2
         }
         record boss {
             id 1 B
             fields-from employee
         }
         record trailer {
             id 1 T
             field type 1
             field count 4
         }
     }
     
     structure personel_sep {
         type separated ,
         record header {
             id 1 H
             field type
             field date
         }
         record employee {
             id 1 E
             field type
             field FirstName
             field LastName
             field Age
         }
         record boss {
             id 1 B
             fields-from employee
         }
             record trailer {
             id 1 T
             field type
             field count
         }
     }
     
     output xml {
         file_header "&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;\n"
         data "&lt;%n&gt;%t&lt;/%n&gt;\n"
         record_header "&lt;%r&gt;\n"
         record_trailer "&lt;/%r&gt;\n"
         indent " "
     }
     
     output sql {
         file_header "delete table boss;\ndelete table employee;\n"
         record_header "insert into %r values("
         data "'%t'"
         separator ","
         record_trailer ");\n"
         file_trailer "commit\nquit\n"
         no-data-print no
         field-list FirstName,LastName,Age
     }
     
     output nice {
         record_header "%s - %r - %f - %o\n"
         data "%n=%t\n"
         justify =
         indent " "
     }
     
     output html {
         file_header "&lt;html&gt;\n&lt;head&gt;\n&lt;/head&gt;\n&lt;body&gt;\n&lt;table border=\"1\"&gt;\n&lt;tr&gt;\n"
         header "&lt;th&gt;%n&lt;/th&gt;\n"
         record_header "&lt;tr&gt;\n"
         data "&lt;td&gt;%t&lt;/td&gt;\n"
         file_trailer "&lt;/table&gt;\n&lt;/body&gt;\n&lt;/html&gt;\n"
         no-data-print no
     }
</pre>
<h4 class="subheading">Using <samp><span class="command">ffe</span></samp> to test file integrity</h4>

<p><samp><span class="command">ffe</span></samp> can be used to check flat file integrity, because <samp><span class="command">ffe</span></samp>
checks for all lines the line length and id's for fixed length structure
and field count and id's for separated structure.

<p class="noindent">Integrity can be checked using command

   <p><code>ffe -p no -l inputfiles...</code>

<p class="noindent">Because option <samp><span class="option">-p</span></samp> has value <code>no</code> nothing is printed to output except the error messages. 
Option <samp><span class="option">-l</span></samp> causes all errorneous lines to be reported, not just the first one.

<p class="noindent">Example output:

<pre class="example">     ffe: Invalid input line in file 'inputfileB', line 14550
     ffe: Invalid input line in file 'inputfileD', line 12
</pre>
   <div class="node">
<p><hr>
<a name="Problems"></a>
Previous:&nbsp;<a rel="previous" accesskey="p" href="#ffe-configuration">ffe configuration</a>,
Up:&nbsp;<a rel="up" accesskey="u" href="#Top">Top</a>

</div>

<h2 class="chapter">5 Reporting Bugs</h2>

<p><a name="index-bugs-11"></a><a name="index-problems-12"></a>
If you find a bug in <samp><span class="command">ffe</span></samp>, please send electronic mail to
<a href="mailto:tjsa@iki.fi">tjsa@iki.fi</a>.  Include the version number, which you can find by
running `<samp><span class="samp">ffe&nbsp;--version</span></samp>'<!-- /@w -->.  Also include in your message the
output that the program produced and the output you expected.

   <p>If you have other questions, comments or suggestions about
<samp><span class="command">ffe</span></samp>, contact the author via electronic mail to
<a href="mailto:tjsa@iki.fi">tjsa@iki.fi</a>.  The author will try to help you out, although he
may not have time to fix your problems.

   <div class="contents">
<h2>Table of Contents</h2>
<ul>
<li><a name="toc_Top" href="#Top">ffe</a>
<li><a name="toc_Overview" href="#Overview">1 Preliminary information</a>
<li><a name="toc_Samples" href="#Samples">2 Samples using <samp><span class="command">ffe</span></samp></a>
<li><a name="toc_Invoking-ffe" href="#Invoking-ffe">3 How to run <samp><span class="command">ffe</span></samp></a>
<ul>
<li><a href="#Invocation">3.1 Program invocation</a>
<li><a href="#Configuration">3.2 Configuration</a>
<li><a href="#Guessing">3.3 Guessing</a>
<li><a href="#Limits">3.4 Limitations</a>
</li></ul>
<li><a name="toc_ffe-configuration" href="#ffe-configuration">4 How <samp><span class="command">ffe</span></samp> works</a>
<li><a name="toc_Problems" href="#Problems">5 Reporting Bugs</a>
</li></ul>
</div>

</body></html>