=begin sdf
H1: Copyright

Copyright 2000 by Robb Canfield. Released under the same terms as Perl

Robb Canfield, Canfield Research Group, {{EMAIL robbc@canfield.com}}

=cut


package Exten::Table;

#use Data::Dumper;           # DEBUG

use Carp;

use HTML::Filter;
@ISA = qw/HTML::Filter/;

use strict;


=begin sdf

H1: parse_table

Parses the table data embedded in the HTML string passed. The string passed
should include all the HTML data B<between> the table tags but B<not> including
the table tags!

Uses C<TH> to extract headers. A row that starts with C<TD> ends the
header extraction. Optionally pass a header count
and that many data rows are converted to headers without regard to the column
type.

If no headers are found or the header count is 0 then no labels header
is defined and all rows are considered data.

Returns an array of each row of data. Each element contains the
data for that column. Each possible column is always set. If a cell spanned
one or more cells then spanned cells are set to undef(). This is different
from empty cells which are assigned a blank string C<"">. The B<first> element
is special and contains the deduced type for the row of data. If any C<TH> elements
were found then the row is labeled C<HEADER> otherwise it is labled C<DATA>. Note
that multiple header rows can appear anywhere in the table.

Example\:
>   [
>       [HEADER, undef, Tempurature, undef, undef, Precipitation, undef, undef],
>       [HEADER, Place, Today, Tonight, Saturday, Today, Tonight, Saturday],
>       [DATA, Seattle, 51, 42, 49, 60, 100, 80],
>       [DATA, Everett, 50, 41, 48, 60, 100, 80],
>   ]

Note that the first header row used colspan 3 which are represented by the two
undefs following C<Tempurature> and C<Precipitation>.

H2: Extensions

The following data elements are added to the standard Filter object. These are
all internal and should B<not> be used by outside methods. All are placed under
the key C<TABLE>:

!block table
Element         Description
table_active    True if currently processing a table
data            Array for parsed table, retrieve via L<"table_data">
row_id          Current row within the C<data> element
col_id          Current col within the C<data> element
type            Type (HEADER, DATA) of row
skip_eof        Skips all further parsing, used to prevent going beyond end of table
column_count    Set to the number of columns wide the table is, not counting internal columns
jagged          Set to TRUE if the table has uneven rows, normally automatically fixed by L<"_clean_up_table"> and changed to FALSE
force_blank     Another intanal variable, used to handle setting cells to undefine during C<COLSPAN> processing
no_out          Prevent output if TRUE, used to avoid output while inside a table but outside a cell
pre_on          True if the C<PRE> HTML tage is active, used to know if spaces/newlines should be trimmed
!end_block

The following are global for the process
!block table
Element         Description
NEW_TEXT        New processed text, see L<"get_output">
TABLE_STACK     Contains pending table data (all data in above table) when processing nested tables
POST_PROCESS    Name/reference to method for table post processing - applies to all tables found during parsing
!end_block


H2: How to use

The simplist way to use this module is:
!block example
    use Exten::Table;

    my $filter = Exten::Table->new();
    $filter->parse_file($ARGV[0]);          # or $filter->parse(some-string)

    # Retrieve the formatted text
    $print $filter->get_result();
!end block

Since this module acts as a filter it has to have an input and an output. The input comes via
L<"parse"> or L<"parse_file"> and the output is placed within the object itself and retrieved
via L<"get_result">. If you want to control the table filtering more carefuly then overload
the L<"table_post_processing"> method with your own. Typically this is done to gain fine control
over the HTML page being scanned. Combined with the L<"parse"> method it becomes trivial to scan
the HTML page looking for a particular table, extract it, process it with the filter, post-process
it as desired, and replace it if needed.

Note: The L<"set_table_post_process"> can also be used to change the post-processing method, this
is often easier for simple code such as in the example below.


!block example
    # An example of a custom table post processing. This example scans the input
    # file for a table containing 'Precipitation' (as an example) and then
    # custom formats that table.

    use Exten::Table;

    # Define a simple post-process method
    sub my_post_process {
        my $self =  shift;

        # Do a header join and then dump the table using table_out. This
        # has a very different look than the hierarchial output.
        $self->join_headers();
        $self->drop_blank_columns;      # usually a good idea

        # User method returns a STRING that is then forwarded to
        # the L<"output"> method *internal processing). The L<"output"> can
        # be overloaded to accept and process the result of this method
        # as desired. See also L<"end"> the 'table' section.
        "<HR width='5'>" . $self->table_out() . "<HR width='5'>";
    }

    # Read a file into a string for ease of debugging.
    open(T, $ARGV[0])
        or die "Unable to open $ARGV[0], $!";


    my $tmp = join("\n", <T>);

    # Massage string to make it easier to look for table. This is a trick I often
    # use to make the REGX faster, easier and more reliable.
    $tmp =~ s/(<table)/\xFF$1/gi;
    $tmp =~ s/(<\/table\s*>)/$1\xFE/gi;

    # Scan each table looking for Precip table
    $tmp =~ s/\xFF([^\xFF\xFE]+)\xFE/
        $data = $1;                         # Remember it in case I need to use another REGX

        # If proper table is found, process it and use the output as the
        # replacement text.
        if($data =~ m'Precipitation') {     # There, I needed to use a REGX
            # this is the table I want, create a new object for it
            # and use my own post processing, easier to use this way than create another module
            # for overloading the default post processor.
            $parser = Exten::Table->new();
            $parser->set_table_post_process(\&my_post_process);
            $parser->parse($data);
            $parser->get_result();
        } else {
            $data;
        }
    /ge;

    $tmp =~ s/\xFF|\xFE//g;

    print $tmp;

!end_block


If you want to process the HTML at the tag level instead of filter then:
    * Process the HTML per above then re-process using your own module that
    uses the HTML::Parser or HTML::Filter module directory (Recomened)
    * OR overload the this module to add the functionaility you need. This is not
    normally the best way since the code can quickly become convoluted, but it
    is handy for quick and dirty fixes. Your choice.

H2: History
    Date: 2000-04-11
        -- Fixed bugs
            --- A tempory debug file C<t.txt> was still being written, it is now gone. Please
            delete any copy on your drive.
            --- A bug in L<"force_headers"> caused all rows after the header count reached
            -1 to become HEADER rows

        -- Added a new method L<"set_table_conversion_process">. This allows the user
        to intercept the table data immediatly after the end C<TABLE> tag is found. It
        is easier to use than overloading <"set_table_post_process">, which is a bit more
        complex than it should be. By default the table conversion does nothing making it
        very easy to overload. I use this to extend table processing for specific applications,
        such as the documention for the Palm game F<Dragon Bane> which does NOT use C<TH>
        in the Spells document. I wrote a conversion that checks for tables wider than
        5 columns and if found forces the first row to a header.

!block example
        sub my_conversion_process {
            my $self =  shift;

            # Table's first ROW is a header if it has 5 or more columns
            # usually only for SPELLS html page
            if($self->column_count >= 5) {
                $self->force_headers(1) unless $self->row_type() eq 'HEADER';
            }

            $self;
        }
!end_block
}

=cut


=begin sdf

H2: Globals

Variables defined globably. These should probably be defined as part of the object
but they are here for either testing, experimenatation of laziness.

=cut

=begin sdf

H3: $Column_Seperator

HTML text that is used to seperate columns in a table. This is only used when there
are no headings for the table and the columns are two wide to place side by side.

=cut
#my $Column_Seperator = '<HR WIDTH="25%">';
my $Column_Seperator = '<BR>';

=begin sdf

H3: $Row_Seperator

HTML text that is used to seperate columns in a table. This is only used when there
are no headings for the table and the columns are two wide to place side by side.

=cut
#my $Row_Seperator = '<HR WIDTH="75%">';
my $Row_Seperator = '<BR>';

=begin sdf

H3: $Table_Start

HTML text that is used to mark the start of a table. If the output format can
handle tables then use C<<TABLE>> otherwise set as desired.

=cut
#my $Table_Start = '<HR SIZE="2">';
my $Table_Start = "<TABLE>";

=begin sdf

H3: $Table_End

HTML text that is used to mark the end of a table. If the output format can
handle tables then use C<</TABLE>> otherwise set as desired.

=cut
#my $Table_End = '<HR SIZE="5">';      # A little larger than Start, allows nested tables to be noticed (mostly for debugging)
my $Table_End = '</table>';



=begin sdf

H2: Methods

=cut


=begin sdf

H3: new

Stub for creating a new object

=cut

sub new {
    my $class = shift;

    my $p = HTML::Filter->new(@_);


    $p = bless $p, $class;

    $p->{TABLE}{no_out} = 0;           # output can occur by default
    $p->{TABLE}{data} = [];            # Initialize to empty array
    $p->{NEW_TEXT} = '';                # Default output text to blank
    $p->{TABLE_STACK} = [];             # Default is no tables on stack
    $p->set_table_post_process('_default_post_process');    # Allows for OO style calling
    $p->set_table_conversion_process('_default_conversion_process');    # Allows for OO style calling

    $p;
}


=begin sdf

H3: parse

Overloads the parent's C<parse> method for future expansion.

=cut

sub parse {
    my $self = shift;

    $self->SUPER::parse(@_);
    $self;
}


=begin sdf

H3: parse_file

Overloads the parent's C<parse_file> method for future expansion.

=cut

sub parse_file {
    my $self = shift;

    $self->SUPER::parse_file(@_);
    $self;
}


=begin sdf

H3: set_table_post_process

Sets the table's post processing to the method name or reference passed. If a string is passed
it is used as the method name for a standard object oriented call. The post porcess method
method is called with the filter's object. Note that each new table encountered purges the previous
so archive the data if you need to merge or work with more than one table at a time.

Returns the previoust post process method. If undef is passed NO changes are done to the
current post process (used for reading current).

See also L<"_default_post_process">

=cut

sub set_table_post_process {
    my $self = shift;
    my(
        $method,
    ) = @_;

    my $old_method = $self->{POST_PROCESS};

    if(defined $method) {
        $self->{POST_PROCESS} = $method;
    }

    $old_method;
}


=begin sdf

H3: set_table_conversion_process

Sets the table's pre processing to the method name or reference passed. If a string is passed
it is used as the method name for a standard object oriented call. The post porcess method
method is called with the filter's object. Note that each new table encountered purges the previous
so archive the data if you need to merge or work with more than one table at a time.

Returns the previoust pre process method. If undef is passed NO changes are done to the
current post process (used for reading current).

See also L<"_default_conversion_process">
=cut

sub set_table_conversion_process {
    my $self = shift;
    my(
        $method,
    ) = @_;

    my $old_method = $self->{CONVERSION_PROCESS};

    if(defined $method) {
        $self->{CONVERSION_PROCESS} = $method;
    }

    $old_method;
}


=begin sdf

H3: table_out

Formats the table using by guesing its structure:
    * If there are no columns in the table then a blank string is returned

    * If there are no rows in the table then a blank string is returned

    * If the first line does not contain any headers then
        ** If one column, display as <BR>
        ** If only 2 columns, combine columns using C<' - '> and display each row on its own line using <BR>
        ** If more than 3 columns then display first column then indent using <UL>, <LI> for each subsequent column

    * If first line is a header:
        ** Combine all headers, see L<"join_headers">.
        ** Display first header name followed by a C<':<BR>'> then loop through all rows and
        display first column of data with each subsequent column is indented via C<UL> and C<LI> with
        column name and data using C<': '> to concatenate the strings.


The defaults can be altered via paramaters to this method. For more complex operations process the
data structure directly.


Returns the string with the new HTML code. undef() indicates an error occured. A blank string
indicates an empty table.

=cut

sub table_out {
    my $self = shift;

    my $result = '';            # Result string
    my $text;                   # Return text from user call
    my $column_count = $self->column_count();

    # No table, no output
    return '' unless $column_count;
    return '' unless $self->row_count();

    # Check for any headers
    if($self->row_type() ne 'HEADER') {
        if($column_count == 1) {
            $result = $self->process_table(
                sub {
                    my ($self, $type, $row) = @_;
                    return '' if $type ne 'DATA';
                    $row->[0] = '' if ! defined $row->[0];
                    "$row->[0]<BR>\n";
                }, 
            );
        } elsif($column_count == 2) {
            $result = $self->process_table(
                sub {
                    my ($self, $type, $row) = @_;
                    return '' if $type ne 'DATA';
                    foreach(@$row) {
                        $_ ||= '';
                    }
                    # Guess a nice length where the '-' doesn't make sense.
                    # Most often triggered on wide tables, especially tables containing
                    # other tables.
                    if(length($self->text_only($row->[0] . $row->[1])) > 40) {
                        "$row->[0]<BR>$row->[1]$Row_Seperator\n";
                    } else {
                        "$row->[0] - $row->[1]<BR>\n";
                    }
                }, 
            );
        } else {
            $result = $self->process_table(
                sub {
                    my ($self, $type, $row) = @_;
                    return '' if $type ne 'DATA';
                    foreach(@$row) {
                        $_ = '' if ! defined $_;
                    }

                    # Handle properly if nested within another table
                    if($self->{TABLE}{table_active}) {
                        join($Column_Seperator, @$row) . $Row_Seperator;
                    } else {
                        "$row->[0]<BR><UL><LI>" . join("<LI>", @{$row}[1 .. ($column_count-1)]) . "</UL>\n";
                    }
                }, 
            );
        }

    } else {
        if($column_count == 1) {
            $result = $self->process_table(
                sub {
                    my ($self, $type, $row) = @_;
                    return '' if $type ne 'DATA';
                    $row->[0] = '' if ! defined $row->[0];
                    "$row->[0]<BR>\n";
                }, 
                1,          # Skip header column
            );

            # Add column header
            $result = $self->get_row(0, 1)->[0] . ":<BR>\n$result";

        } elsif($column_count == 2) {
            $result = $self->process_table(
                sub {
                    my ($self, $type, $row) = @_;
                    return '' if $type ne 'DATA';
                    foreach(@$row) {
                        $_ = '' if ! defined $_;
                    }
                    # Guess a nice length where the '-' doesn't make sense.
                    if(length($self->text_only($row->[0] . $row->[1])) > 40) {
                        "$row->[0]<hr>$row->[1]<br>";
                    } else {
                        "$row->[0] - $row->[1]<BR>\n";
                    }
                }, 
                1,          # Skip header column
            );

            # prefix column headers
            $result = join(" - ", $self->get_row(0, 1)) . "<BR>\n$result";

        } else {
            # Complex multi-column table

            # Build hierarchial column headers - takes less width than joining headers into a single string
            my $user_data = $self->headers_to_tree();

            # now process remaining rows
            $result .= $self->process_table(
                sub {
                    my ($self, $type, $row, $err, $format) = @_;
                    return '' if $type ne 'DATA';

                    # Format the row using the user-data header, convert undefined into blanks
                    # to avoid Perl's warnings.
                    foreach(@$row) {
                        $_ = '' if ! defined $_;
                    }
                    sprintf($format, @$row);
                },
                1,          # Skip header row
                undef,      # go to end of table
                $user_data,   # This will be passed to the user method defined above
            );
        }
    }


    $result;
}


=begin sdf

H3: headers_to_tree

Designed to convert a table into a n HTML list (C<<UL>>, <<LI>>). In essence this trades the table width
for length making it fit better on shallow screesn such as the Palm Pilot. Originally designed for
use in SiteScooper files for output to iSilo.

Builds an array of headers for each column of the table. The actual table data is NOT modified. This
method is handy when a table is using column groups:

>        |       Tempurature          |      Precipitation
>  Place | Today | Tonight | Saturday |  Today | Tonight | Saturday

Note that if the grouping must be done using C<COLSPAN> attribute otherwise there is no
reliable way to know which group applies to which column. If C<COLSPAN> was not used then
the grouping will appear incorrect.

The following rules were determined to provide the most flexible and accurate output. Sample
output contains only the HTML tags (header text is appended). The first rule that matches
is used and all others are skipped.
    * Undefined columns are skipped
    * Blank columns are skipped
    * If there is text in the same column of the previous row then it is the first column in the conitionation of a multi-column heading: C<<UL><LI>>
    * If there is no previous column then treat as a blank (see next rule)
    * If the previous column of the current row is undefined then it is another multi-column header: C<</ul><UL><LI>>
    * If the previous column of the current row is blank ("") then it is a new level: C<<UL><LI>>
    * If the previous column of the current row contains text (only option left) then it is a normal column: C<<LI>>

Note that these rules were determined trhough experimentation and appear to work on most tables. Tables
that do not use header tags C<<TH>> properly or strangly formatted tables will probably cause weird
header output. But in general even that is better than simply scrapping all table formatting! The
first column is indentented like all others for consitancy and to avoid strange gaps in spacing that
iSilo uses between before the first C<<UL>> tag.

Returns a header format string designed for sprintf style formatting. Typically each
row can be outputed by: C<sprintf($header->[$column_count], @row_data)>.

Note: Alignment is ignored since all data is displayed in a veritcal list. All other format tags
are utilized, such as C<<FONT>>, C<<STRONG>>, C<<EM>>, etc...

Note: It is recomended that L<"drop_blank_columns"> be called before this method. It will allow
for more compressed table displays especially in cases where a column of images was nulled out. It
is not auticamatically called since this method is not allowed to modify the table data.

Note: Normally headers should be in bold, but that makes the output too hard to read
since headers are repeated over and over. Instead use a bold ']' after the header text and
before the column text. This seems to work well on the desktop and the Palm screen.

=cut

sub headers_to_tree {
    my $self = shift;
    my(
        $offset,
    ) = @_;

    $offset ||= 0;

    # Handle from end-of-table offsets
    if($offset < 0) {
        $offset = $self->row_count() - $offset;
        $offset = 0 if $offset < 0;
    }

    # Used to seperate the column headings from the cell contents
    my $seperator = "<STRONG>]</strong> ";

    # Starting at the row specified scan for headers building tree.
    my(
        @header,                    # Header format
        $column_text,               # Contents of current column
        $column_loop,               # Current column being processed
        $max_depth,                 # Maximum header depth (offset)
        $depth,                     # Current depth in header
        $open_tags,                 # How many open tags exist for the current column
        $tmp,                       # whatever...
        $column_is_group,           # False until any row in the column is determined to be part of a group. Used to help adjust the table for better readability
    );

    my($type, $data, $max);
    $max = $self->row_count();
    $max_depth = 0;
    $open_tags = 0;
    for($column_loop = 0; $column_loop < $self->column_count(); $column_loop++) {

        $column_is_group = 0;

        # Loops until a non-header row is found
        for($depth=0;;$depth++) {
            ($type, $data) = $self->get_row($offset + $depth);
            if($type ne 'HEADER') {
                last;
            }

            $max_depth ||= $depth;      # Remember maximum depth of header, only bother to set once

            $column_text = $data->[$column_loop];       # Easier to work with

            PROCESS: {
                # If current column is undefined then skip it
                (! defined $column_text) && do {
                    $column_is_group = 1;               # Column is part of a group
                    last;
                };

                # If current column is blank then append to header (used for side effect of defining header if it is currently undefined)
                ($column_text eq '') && do {
                    # If on first row of header close out all open tags
                    # This normally happens between two group columns or on the first column (where $open_tags is 0 anyway)
                    if($depth == 0) {
                        $header[$column_loop] = "</ul>" x $open_tags;
                        $open_tags = 0;
                    } else {
                        $header[$column_loop] .= $column_text;
                    }

                    last;
                };

                # *** At this point it is known that $column_text contains some kind of data

                # If there is data in the column header then a there must be data in one of the previous rows
                # for this column and therfor this header is probably another level of previous rows header
                if(defined $header[$column_loop]) {
                    $header[$column_loop] .= "<UL><LI>";
                    $open_tags++;
                    $header[$column_loop] .= $column_text;
                } else {
                    # No data in previous row for this header, so see how far to back out of the
                    # list tags.

                    # See how many closeing tags must be generated, one for each level deep into the
                    # header block. Note that 0 results in no text being appending.
                    $tmp = $max_depth - $depth;
                    if($tmp > $open_tags) {
                        $tmp = $open_tags;
                    }
                    $header[$column_loop] .= ("</ul>" x $tmp);
                    $open_tags -= $tmp;

                    # If there are no open tags then add one, usually only occurs when at top of
                    # first column
                    if(! $open_tags) {
                        $header[$column_loop] .= "<UL>";
                        $open_tags++;
                    }

                    # Now add another list element for this level of header
                    $header[$column_loop] .= "<LI>$column_text";
                }

            }   # End of PROCESS


            # See if ANY rows in this column belong to a group that was started in this column.
            if($column_loop != $self->column_count()-1 && ! defined $data->[$column_loop+1]) {
                $column_is_group = 1;
            }

        }   # Loop for rows


        # If on the first column and it is NOT part of a group then adjust paramaters tp
        # bring this column seperate from the others. Often the first column is a key or
        # label column.
        if($column_loop == 0 && ! $column_is_group) {
            # This causes the next column to NOT close this open tag
            $open_tags = 0;
        }


    }   # Loop for columns

    # Append the '%s' to each header to display the actual data column text
    # Normally headers should be in bold, but that makes the output too hard to read
    # since headers are repeated over and over. Instead use a ']' and make it bold.
    # This seems to work well.
    foreach(@header) {
        $_ .= "$seperator%s\n";
    }

    # Combine header array into a single string
    my $header = join('', @header);


    # Append appropiate number of </ul> tags to each header by counting the number of open tags versus closing
    # tags.
    # This longer algorthm was chosen over simply looking at the open tag count because the open tag
    # count is NOT reliable for knowing the total number of open tags (funny that). The reason is that
    # the open tag count can be adjusted to highlight column 1 by placing it at its own list depth.
    # This algorithm also recovers from any errors above (there shouldn't be any, but...)
    my($open, $close) = (0, 0);
    $header =~ s/(<UL>)/$open++; $1/gie;
    $header =~ s/(<\/UL>)/$close++; $1/gie;

    if($close >= $open) {
        die "Internal error: The number of close ULs ($close) created exceeds or equals the number of open ULs ($open)";
    }

    $header . ("</ul>" x ($open - $close));
}



=begin sdf

H3: row_type

Returns type of row it is. A row offset (first row is 0) may be provided,
defaults to the first row (0). Rows may be specified from end of table by
using a negative offset, -1 = last row in table.

=cut

sub row_type {
    my $self = shift;
    my(
        $offset,
    ) = @_;
    
    $offset ||= 0;

    # Handle from end-of-table offsets
    if($offset < 0) {
        $offset = $self->row_count() - $offset;
        $offset = 0 if $offset < 0;
    }

    $self->{TABLE}{data}[$offset][0];
}


=begin sdf

H3: row_count

Returns the number of rows in the table.

=cut

sub row_count {
    my $self = shift;

    scalar(@{$self->{TABLE}{data}});
}


=begin sdf

H3: column_count

Returns the number of user columns in the table.

=cut

sub column_count {
    my $self = shift;

    scalar($self->{TABLE}{column_count});
}


=begin sdf

H3: get_row

Returns the row type and a reference to the data. The data is adjusted to remove all internal
data elements. If a scalar was requested then only the data is returned. Returns undef for
data if the table is empty or a request is made beyond the table range.

If $normalize is TRUE then all undefined cells become blanks. Useful for quick
processing and Perl's warnings are on.

Note: The L<"process_table"> is a much more effecient way to process multiple rows. This method
was created to allow easy retrieval of one or two rows such as headers.

Note: Use negative values to retrive data from the end of the table, where -1 = last row in table.

=cut

sub get_row {
    my $self = shift;
    my(
        $offset,
        $normalize,         # If true causes all undefined cells to become blanks
    ) = @_;

    $offset ||= 0;

    # Handle from end-of-table offsets
    if($offset < 0) {
        $offset = $self->row_count() - $offset;
    }

    if($offset < 0 || $offset >= $self->row_count()) {
        if(wantarray) {
            (undef, undef);
        } else {
            undef;
        }
    } else {
        my @data = @{$self->{TABLE}{data}[$offset]};
        my $type = shift @data;
        if($normalize) {
            foreach(@data) {
                $_ = '' if ! defined $_;
            }
        }

        if(wantarray) {
            ($type, \@data);
        } else {
            \@data;
        }
    }
}


=begin sdf

H3: process_table

Allows all the table data to be processed. Loops through each row of the table calling
a user defined method with the following:
!block table
Paramater       Description
$self           Reference to underlying table object
$row_type       Type of row, C<HEADER> or C<DATA>
\$data          Reference to data elements, the first data element is in [0]. Note that this data
                is isolated from the rest of the table and can be directly modified without effecting
                the underlying table. Also note that internal elements (such as data type) are removed.
\$err           Optional error code. The user method can set this to 0 for ALL ok (default) or to a negative
                value to inidcate an error. Negative error codes abort table processing. The user
                error is returned by this method depending on how it was called, see return.
$user_data      Any type of data the caller wishes to be passed to the user method. Typically this is
                a refernce to an array or hash that allows variables that are needed from row to row.
                Undefined is passed if no $user_data was received.

!end_block

The user method can be defined as follows, no matter how passed the paramaters it receives
are identical.
    * Recomended: As a string, in this case the call is done via the object allowing full object oriented
    interface.
    * As a standard refernece to a method.

The user method should return:
    * Text to use for this row. All text is combined into a single string on return by this
    method without any addition characters added.
    * Optionally adjust the \$err passed. If set to a negative value that value is returned
    to the original caller of this method. Postive return codes are reserved for future use.

Returns\:
    * If an array was requested then returns: ($output_text, $rows_processed, $err) where $err is
    < 0 as returned by user defined method. Note that on error $output_text is probably invalid or
    at least incomplete.
    * If a scalar was requested then returns: $output_text, or undef on error.

=cut

sub process_table {
    my $self = shift;
    my(
        $user_method,               # Reference to user's method or a string for use with OO style
        $start_row,                 # Row to start on, default is 0 (first row)
        $row_count,                 # Number of rows to process, default is entire table
        $user_data,                 # Special user data
    ) = @_;


    my $out;            # Output o return
    my @data;           # Used to isolate data from underlying table and mask internal elements
    my $type;           # Holds row type
    my $err;            # Error from user method
    my $user_rtn;       # Data returned by user method
    my $offset;         # Current location into table array
    $start_row ||= 0;
    $start_row = 0 if $start_row < 0;
    $row_count = undef if(defined $row_count && $row_count < 0);
    $row_count = $self->row_count() if ! defined $row_count;

    for($offset = $start_row; $offset < $row_count; $offset++) {
        @data = @{$self->{TABLE}{data}[$offset]};
        $type = shift @data;
        $err = 0;
        if(! ref $user_method) {
            $user_rtn = $self->$user_method($type, \@data, \$err, $user_data);
        } else {
            $user_rtn = &$user_method($self, $type, \@data, \$err, $user_data);
        }

        last if $err;

        $out .= $user_rtn;
    }

    if($err > 0) {
        croak("Unexpected return code of '$err' from user method '$user_method'. Postive return codes are reserved for use by this module ");
    }


    # Determin what to return
    if(wantarray) {
        ($offset, $out, $err);
    } else {
        if($err) {
            undef;
        } else {
            $out;
        }
    }
}


=begin sdf

H3: force_headers

Forces the first x rows to be header rows, all other rows are forced to data. Use
this then a table fails to utilize C<TH> to indicate a header or uses C<TH> when C<TD>
should have been used.

Returns the parser object.

=cut

sub force_headers {
    my $self = shift;
    my(
        $header_count,
    ) = @_;

    my $data;

    foreach $data (@{$self->{TABLE}{data}}) {
        if($header_count > 0) {
            $data->[0] = 'HEADER';
            $header_count--;
        } else {
            $data->[0] = 'DATA';
        }
    }


    $self;
}



=begin sdf

H3: drop_column

Causes the column number specified to be dropped (default 0, first column). The optional
$count (default 1) can be provided to drop more than one column. Pass a count of -1 to
drop columns to end of table.

Returns the parser object passed.

=cut

sub drop_column {
    my $self = shift;
    my(
        $start,
        $count,
    ) = @_;

    $count = 1 if !defined $count;


    if($start < 0) {
        croak "Starting paramater ($start) cannot be less than 0";
    }

    if($count < -1) {
        croak "Count paramater ($count) cannot be less than -1";
    }

    # handle end of table paramater
    if($count == -1) {
        $count = $self->column_count() - $start;
    }

    $start++;           # Skip past the internal data column

    if($count > 0) {
        if($count + $start > $self->column_count()+1 ) {
            croak "Start paramater ($start) plus count paramater ($count) cannot exceed table width (" . $self->{TABLE}{column_count} . " columns)";
        }
    
        foreach(@{$self->{TABLE}{data}}) {
            splice(@$_, $start, $count);
        }

        # Adjust table width tracker
        $self->{TABLE}{column_count} -= $count;
    }

    $self;
}


=begin sdf

H3: drop_blank_columns

Drops all columns that have every row blank (either undefined or an empty string or all white space). Normally
done to clean-up a table where columns were post processed, such as removing images.

=cut

sub drop_blank_columns {
    my $self = shift;

    my $offset = 0;             # current row offset
    my @data_found = ();        # Tracks if data was found for each column
    my $data;                   # Contains referecne to table data, becomes undefined at end of table
    my $column;                 # current column offset

    # Loop through all rows looking for blank columns
    # Stop scan if all columns are determined to be non-blank
    while($data = $self->get_row($offset++)) {
        $column = 0;
        foreach(@$data) {
            if(defined $_ && length($_)) {
                $data_found[$column] = 1;
            }

            $column++;
        }

        # Exit unless at least one column is still blank
        if(@data_found) {
            last unless(grep $_, @data_found);
        }
    }

    # Work backwords to avoid having to recalculate offsets - skip column 0 (internal column type indicator)
    for($offset = scalar(@data_found)-1; $offset >= 0; $offset--) {
        next if $data_found[$offset];

        $self->drop_column($offset);
    }

    $self;
}

        



=begin sdf

H3: drop_row

Drops the row number specified from the data. Normally this is used to remove
a header that is not needed or wanted. If no value is passed a value of zero
is assumed and the first row is dropped (the most common application). If $count
is specified then that number of rows is droped, default is one. If 0 is passed
no action will occur.

Returns the parser object.

Note: Currently values less than 0 for $count are treated as 0, but the use of a negative
count is reserved for future expansion.

Note: Remember that rows start at 0 B<not> 1!

=cut

sub drop_row {
    my $self = shift;
    my(
        $start,
        $count,
    ) = @_;

    $count = 1 if !defined $count;

    if($start < 0) {
        croak "Starting paramater ($start) cannot be less than 0";
    }

    if($count < -1) {
        croak "Count paramater ($count) cannot be less than -1";
    }

    # handle end of table paramater
    if($count == -1) {
        $count = $self->row_count() - $start;
    }

    if($count > 0) {
        $start ||= 0;

        splice(@{$self->{TABLE}{data}}, $start, $count);
    }

    $self;
}




=begin sdf

H3: join_headers

Merges multiple consecitive header rows into a single row using the connector
string passed. Normally used when a table ues group headers:
>        |       Tempurature          |      Precipitation
>  Place | Today | Tonight | Saturday |  Today | Tonight | Saturday

becomes (if C<: > is used as the connector string)
> Place | Tempurature: Today | Tempurature: Tonight | etc...


Note that the C<Place> column from the second header line was used as is. This
allows for cases where a table begins grouping columns after column 1, a rather
common occurance.

This also effects single header rows that used the C<COLSPAN> attribute. The
extra undefined cells created during parsing are filled with the primary
cell name. Headings defined 

Returns the parser object passed.

Note: Scans all the rows and merges ANY multiple header rows found.

Note: Use this method with caution, the standard min_width_text() method does
a pretty good job of organizing multiple heading outputs.

Note: The connector is not encoded into HTML allow it to contan HTML tags (such as <B>, <EM>, etc)
if desired.


=cut


sub join_headers {
    my $self = shift;
    my(
        $connector,
    ) = @_;

    my @new_data = ();      # new table data is placed here, still related to $self for all data rows. Actual memory requirements for this variable are minimal.
    my $last;               # Reference to last row addedif it was a header row
    my $row;                # Current row being processed
    my $tmp;                # whatever... 

    $connector = ':' if ! defined $connector;

    foreach $row (@{$self->{TABLE}{data}}) {

        # fill any undef elements if this is a header row
        if($row->[0] eq 'HEADER') {

            # No need to skip first element since it is never undef and
            # second element can never be undef, although it could be blank.
            # See end() TD/TH processing if unsure why...
            foreach(@$row) {
                if(defined($_)) {
                    $tmp = $_;
                } else {
                    $_ = $tmp;
                }
            }
        }


        if(! @new_data) {
        # If no data then seed the new data
            push(@new_data, $row);
        } else {
        # See if this row is a header, if so merge with previous
        # if it also was also header
            if($row->[0] eq 'HEADER' && $last) {
            # Merge the rows

                # Increase size of $last if necessary
                while(scalar(@$last) < scalar(@$row)) {
                    push(@$last, $last->[-1]);
                }

                # Increase size of $row if necessary
                while(scalar(@$last) > scalar(@$row)) {
                    push(@$row, $row->[-1]);
                }


                # Now merge current row with last row, using connector, skip row type [0] element
                # If no text is in last row then do NOT perform conactenation, handles
                # cases were grouped columns start after column 1 (rather common).
                for($tmp = 1; $tmp < scalar(@$last); $tmp++) {
                    if($last->[$tmp]) {
                        $last->[$tmp] .= $connector . $row->[$tmp];
                    } else {
                        $last->[$tmp] = $row->[$tmp];
                    }
                }
                    
            } else {
            # Not a multiple header sequence so add row normally
                push @new_data, $row;
            }
        }


        # Remember if current row is a header
        if($row->[0] eq 'HEADER') {
            $last = $new_data[$#new_data];
        } else {
            $last = undef;
        }

    }

    # Transfer modified array back to object
    $self->{TABLE}{data} = \@new_data;

    $self;
}



=begin sdf

H3: join_columns

Synopsis\: join_columns($connector, \@join_list1 {,\@join_list2, ...}, $smart_join, \@forced_keep)

Joins two or more columns into a single column using the connector string
supplied. Note that this joins ALL columns in the table including headers. Multiple column joins
are allowed during the same call. For multiple joins each works from the original rows offsets.

If $smart_join is turnd on then when two heading columns are joined a check is done to
see if the column names are the same, if they are the text is left unchanged. This
only applies to heading rows, data rows are always joined.

For example the following:
>   Today | Tonight | Saturday | Today | Tonight | Saturday

when merged on 0,3 then 1,3 then 2,3 using connector ':'
> $parser->join_columns(':', [0, 3]);
> $parser->join_columns(':', [1, 3]);
> $parser->join_columns(':', [2, 3]);


without $smart_join
>   Today:Today | Tonight:Tonight | Saturday:Saturday

with $smart_join (for example: C<$parser->join_columns(':', [0, 3], 1)> )
>   Today | Tonight | Saturday

Or use the single pass option to do all with one call. Note that the offsets
are based on the original row. This format is also a bit faster.
>   $parser->join_columns(':', [1, 3], [2, 4], [3, 5], 1);


See also L<"join_same_columns">

This is normally used to compress a table that used group columns into an easier to read
format.

Returns the parser object passed.

Note: Columns must be passed by reference!

Note: Columns are removed as they are joined so that when multiple joins take place, as in the
example above where 3 seperate column pairs were joined, you must adjust the offsets
properly. You are guarenteed that the offsets do not change during a single call, so
a list of 2,3,4 will join columns 2,3,4 into a single column 2. On the next call remember
that columns 3,4 are now something different.

Note: Column 0 is the first column of the row (even through internally [0] is used for type,
this translation is handled transparently to the caller).

Note: Column numbers may be in any order and will be merged in the order provided with all
text appearing in the B<first> column in the list. The remaining columns will be removed
when the join is done.

Note: Column values less than 0 cause a fatel exception, column numbers beyond the length
of the table also cause a fatal exception.

Note: Columns must be unique within the list.

Note: The connector is not encoded into HTML allow it to contan HTML tags (such as <B>, <EM>, etc)
if desired.

Note: Errors are indicated by column number and group number, where the first group
in the list is 1.

Note: On occasion it is necessary to keep a column that would otherwise be deleted. This
can be done via the optional \@forced_keep paramater. For example if 2,3,4 are merged
(into 2) normally 3 and 4 are deleted. But if \@forced_keep is [3,4] then 3 and 4 will remain.
Forced keep only applies to this call and a column need only be mentioned once even it was
merged several times.

H4: To do
This method got way out of hand and does far more than it was originally designed to.
I have tested it as well as I can given time.  A re-thining of the code may be in order and
some test code.

=cut

sub join_columns {
    #
    # Extract paramaters, a bit weird but handy
    #
    my $self = shift;
    my $connector = shift;
    # Scan remaining paramaters until a non-array reference is found
    my @column_list;
    while(ref($_[0]) =~ /ARRAY/) {
        # Create a unique array to avoid mucking with passed data - never muck with user data without good reason
        push @column_list, [@{$_[0]}];
        shift;
    }
    # Get remaining paramaters
    my(
        $smart_join,
        $forced_keep,   # refernce to a list of columns that should never be deleted
    ) = @_;


    # now define some variables
    my $column;         # Refernce to current column being processed
    my $row;            # current row being processed
    my $target;         # target column
    my $group;          # tracks which column group is being worked on
    my %delete_list;    # Contains columns that will be deleted (1) or forced saved (-1)
    my $tmp;            # whatever
    my %tmp;            # whatever
    my @tmp;            # whatever
    my $offset;
        

    #
    # Make sure column counts are valid and adjust to skip past internal column (type)
    #
    $tmp = '';
    %tmp = ();
    $group = 1;
    foreach $column (@column_list) {
        $offset = 0;
        %tmp = ();
        foreach(@$column) {
            $_ += 0;            # Force to be a number
            if($_ < 0) {
                $tmp .= "Negative column #$_  in group $group is not allowed. ";
            } elsif($_ > $self->{TABLE}{column_count}) {
                $tmp .= "Column #$_ in group $group exceeds maximum table width of $self->{TABLE}{column_count}. ";
            }


            # Adjust to skip past internal type column
            $_++;

            # Do NOT allow duplicates
            $tmp{$_}++;

            # Add to delete list all target columns (2nd+ in a group)
            if($offset++) {
                $delete_list{$_} = 1;
            }

        }

        if(@$column < 2) {
            $tmp .= "At least two columns must be provided (group $group). ";
        }
        
        foreach(sort {$a <=> $b} %tmp) {
            if($tmp{$_} > 1) {
                $tmp .= sprintf("Column #%d is repeated %d times in group $group, must be unqiue within the group. ", $_-1, $group);

            }
        }


        $group++;
    }
    if($tmp) {
        croak("Errors with column IDs passed: $tmp");
    }


    #
    # Clean up the delete list, remove forced keep columns
    #
    foreach(@$forced_keep) {
        delete $delete_list{$_};
    }

    #
    # Clean up removed groups (forced keep)
    #
    @column_list = grep(defined $_, @column_list);


    #
    # Now join the columns
    #
    foreach $row (@{$self->{TABLE}{data}}) {
        foreach $column (@column_list) {
            @tmp = @$column;            # Isolate so it doesn't get changed
            $target = shift @tmp;        # Get starting column

            foreach(@tmp) {
                if(! $smart_join || $row->[0] ne 'HEADER' || $row->[$target] ne $row->[$_]) {
                    $row->[$target] .= $connector . $row->[$_];
                }
            }
        }

        # Now delete the columns that were joined, delete from end to start
        # to avoid having to recalculate offsets
        foreach(sort {$b <=> $a} keys %delete_list) {
            splice(@$row, $_, 1);
        }
    }


    $self;

}


=begin sdf

H3: join_same_columns

Uses the column names from the header row specified (defaults to 0) and checks to see
if any columns have the same names. If so the columns are marked for joining. All columns
in the table are joined using this this list. This was originall written to handle common
cases where a table is grouped into multiple sections such as:
!block example
0 :         |         Tempurature        |     Precipiation
1 :  Place  | Today | Tonight | Saturday | Today | Tonight | Saturday
2 :  Seattle| 51    | 42      | 49       | 60    | 100     | 80
!end_block

In this example the first row (0) would probably be deleted, see L<"delete_row">,
and then columns merged using this method. If the connector string was C<"/"> then
the result would be:

!block example
0 :  Place  | Today | Tonight | Saturday
1 :  Seattle| 51/60 | 42/100  | 49/80
!end_block

This can compress the table data considerable.

See also L<"join_columns">

Note: This method uses L<"join_columns"> with the $samrt_join enabled.

Note: The column names must match exactly including whitespace and HTML tags
    
=cut

sub join_same_columns {
    my $self = shift;
    my(
        $connector,
        $header_row,
    ) = @_;


    $header_row ||= 0;

    if($header_row < 0) {
        croak "Header row paramater ($header_row) cannot be less than 0";
    }

    if($self->{TABLE}{data}{$header_row}[0] ne 'HEADER') {
        croak "Header row paramater ($header_row) references a row that is not a header.";
    }

    my %lookup;     # Used to track same name columns
    my $offset;
    for($offset = 1; $offset <= $self->{TABLE}{column_count}; $offset++) {
        # Use column name to track groups - adjust offset to be from user's
        # perspective (not counting internal type column)
        push @{$lookup{$self->{TABLE}{data}[$header_row][$offset]}}, $offset-1;
    }
    
    # Build group list for all joins where more than one column is present
    my @join_list = ();
    foreach(values %lookup) {
        next if @$_ < 2;
        push @join_list, $_;
    }

    # Join the columns
    if(@join_list) {
        $self->join_columns($connector, @join_list, 1);
    }

    $self;
}



=begin sdf

H3: start

Intercepts the following table tags:
    * table
    * tr
    * th
    * td
    * /pre - B<not> skiped, but prevents newline/space conversion, see ouput()

These tags are removed and output offsets are initialized (on C<TR>). See L<end> for
incrementing offsets and setting of row data type.

Output is turned on when the start of a CELL is found.

=cut

sub start {
    my $self = shift;       # Parser object
    my(
        $tag,               # tag name in lower case
        $attr,              # hash of all attributes, attribute names in lower case
        $attrseq,           # array containing attributes in order found
        $origtext,          # original full HTML text
    ) = @_;

    return if $self->{TABLE}{skip_eof};

    PROCESS:{
        # Strip the table tags from the output
        $tag eq 'table' && do {
            # Save any pending table definition on stack if nesting
            if($self->{TABLE}{table_active}) {
                push (@{$self->{TABLE_STACK}}, $self->{TABLE});
            }

            # Reset Table stuff
            delete $self->{TABLE};              # Remove all current data
            $self->{TABLE}{row_id} = 0;
            $self->{TABLE}{jagged} = -1;        # Tracks if the table was determined to be jagged, an increment of 1 is unavoidable if speed is to be maintained (see end()) so set to -1 to compensate.
            $self->{TABLE}{column_count} = 0;
            $self->{TABLE}{row_id} = 0;
            $self->{TABLE}{col_id} = 0;
            $self->{TABLE}{type} = undef;
            $self->{TABLE}{data} = [];
            $self->{TABLE}{table_active} = 1;
            $self->{TABLE}{no_out} = 1;        # No more ouput until a cell is encountered
            $self->{TABLE}{force_blank} = 0;

            last;
        };

        $tag eq 'tr'    && do {
            $self->{TABLE}{col_id} = 1;        # [0] is reserved for row data type
            $self->{TABLE}{type} = 'DATA';     # Assume data until otherwise determined

            # Remeber callspan attribute for processing by end()
            if(exists $attr->{colspan} && $attr->{colspan} > 1) {
                $self->{TABLE}{force_blank} = $attr->{colspan} - 1;
            }

            last;
        };


        $tag eq 'th'    && do {
            $self->{TABLE}{type} = 'HEADER';
            $self->{TABLE}{no_out} = 0;        # Output is now ok, inside a cell

            # Remeber callspan attribute for processing by end()
            if(exists $attr->{colspan} && $attr->{colspan} > 1) {
                $self->{TABLE}{force_blank} = $attr->{colspan} - 1;
            }

            last;
        };

        $tag eq 'td'    && do {
            $self->{TABLE}{no_out} = 0;        # Output is now ok, inside a cell
            last;
        };

        $tag eq 'pre'   && do {
            # Turn on <PRE> processing, see output() for details
            $self->{TABLE}{pre_on}++;
            last;
        };

        # all other tags are processed normally
        $self->SUPER::start(@_);
    };
}


=begin sdf

H3: end

Intercepts the following table tags:
    * /table
    * /tr
    * /th
    * /td
    * /pre - B<not> skiped, but prevents newline/space conversion, see ouput()

These tags are removed and output offsets incremented. The C</TR> tag also triggers
the setting of the row type for the current row and checking for maximum column
count and jaggedness.

Output is turned off when the end of a CELL is found.

Note: Table data has all leading and trailing spaces removed!

=cut

sub end {
    my $self = shift;       # Parser object
    my(
        $tag,               # tag name in lower case
        $origtext,          # original full HTML text
    ) = @_;

    return if $self->{TABLE}{skip_eof};

    PROCESS:{
        # Strip the table tags from the output
        # As long as the parent start() is not called the tags will vanish from
        # the output.
        $tag eq 'table' && do {
            # Make sure table is cleaned up, otherwise weird and wild things may occur
            $self->_clean_up_table();

            # Handle nested tables properly, this way process can adjust its output
            # if nested tables are being processed.
            if(scalar(@{$self->{TABLE_STACK}}) > 0) {
                $self->{TABLE}{table_active} = 1;
            } else {
                $self->{TABLE}{table_active} = 0;
            }

            $self->{TABLE}{no_out} = 0;        # Restore normal output
            my $tmp = $self->set_table_conversion_process();        # Do any table conversion, should be done in place
            $self = $self->$tmp();                                  # Result is object to use going forward
            $tmp = $self->set_table_post_process();
            $tmp = $self->$tmp();               # Remember result

            # If there is a table stack pending then grab result of last table
            # and add it to the appropiate cell for table on stack restoring
            # table on stack as current
            if(scalar(@{$self->{TABLE_STACK}}) > 0) {
                $self->{TABLE} = pop @{$self->{TABLE_STACK}};   # Restore parent table
            }

            $self->output($tmp);                            # save result into appropiate area
            last;
        };

        $tag eq 'tr'    && do {
            # Set type based on encountering any header like columns
            $self->{TABLE}{data}[$self->{TABLE}{row_id}][0] = $self->{TABLE}{type};

            # Check to see if the table is jagged and adjust width count
            # Anytime the number of columns in a row do NOT equal the previous count
            # indicates a jagged table.
            my $tmp;
            $tmp = scalar(@{$self->{TABLE}{data}[$self->{TABLE}{row_id}]}) - 1;       # Ignore the type column
            $self->{TABLE}{jagged}++ if $tmp != $self->{TABLE}{column_count};
            if($tmp > $self->{TABLE}{column_count}) {
                # Only remember row
                $self->{TABLE}{column_count} = $tmp;
            }

            $self->{TABLE}{row_id}++;
            last;
        };


        # TH and TD are the same for end tag processing
        ($tag eq 'th' || 
        $tag eq 'td')    && do {

            # Force a blank string to ouput just in case the CELL has no data defined
            # in it. Differenates from a spanned cell which is undefined.
            # Note that I don't bother to add a check statement first since
            # that just complicates maintenance and has minimal speed impact for
            # this application.
            my $tmp = \$self->{TABLE}{data}[$self->{TABLE}{row_id}][$self->{TABLE}{col_id}];
            $$tmp .= "";
            $$tmp =~ s/^\s+//g;
            $$tmp =~ s/\s+$//g;

# Didn't end up being a good idea, too many bolds mixed up display
# instead try bolding the seperator between the header name and data, see table_out
# keeping code just in case it is later needed and to prevent re-inventing the wheel
#            # TH by default places text in bold, so follow suit, unless cell is blank
#            if($tag eq 'th') {
#                if($$tmp) {
#                    $$tmp = "<STRONG>$$tmp</strong>";
#                }
#            }

            # Update offsets to prepare for next cell
            $self->{TABLE}{col_id}++;

            # See if colspan is in effect, if so then create false entries
            if($self->{TABLE}{force_blank} > 0) {
                while($self->{TABLE}{force_blank}--) {
                    $self->output();
                    $self->{TABLE}{col_id}++;
                }
            }

            $self->{TABLE}{no_out} = 1;        # all output outside a cell is ignored

            last;
        };


        $tag eq 'pre'   && do {
            # Turn off <PRE> processing, see output() for details
            $self->{TABLE}{pre_on}--;
            last;
        };

        # all other tags are processed normally
        $self->SUPER::end(@_);
    };
}




=begin sdf

H3: output

Saves data in the appropiate row/column of the table data. Does not adjust
output offsets. If there is no table currently being processed the data is
appended to a holding buffer. All data can be retrieved via L<"get_result">.

If the C<PRE> tag is not active then convert newlines to spaces and collapse multiple
spaces into one. Leading and trailing spaces remain, these are handled in the L<"end">
method, see the TR/TH processing section.

Note: Pass an empty string to force the cell to become defined but empty. Pass undef() to
create a new cell that is not defined.

=cut


sub output {
    my $self = shift;
    my(
        $out
    ) = @_;

    # See if NO output should be done
    if(! $self->{TABLE}{no_out}) {
        if(! $self->{TABLE}{table_active}) {
            $self->{NEW_TEXT} .= $out;
            return;
        }

        # Handle when undef() is in a cell of a table. It must remain undefined only
        # if cell is already undefined.
        if(! defined $out) {
            if(! defined $self->{TABLE}{data}[$self->{TABLE}{row_id}][$self->{TABLE}{col_id}]) {
                $self->{TABLE}{data}[$self->{TABLE}{row_id}][$self->{TABLE}{col_id}] = undef;
            }

            # otherwise don't bother doing anything.
        } else {
            if(! $self->{TABLE}{pre_on}) {
                # If PRE is NOT on (allows nested) then convert all newlines
                # to spaces and collapse multiple spaces into one.
                $out =~ s/[\n\s]+/ /g;
            }

            $self->{TABLE}{data}[$self->{TABLE}{row_id}][$self->{TABLE}{col_id}] .= $out;
        }
    }
}


=begin sdf

H3: get_result

Returns the text string resulting from parsing.

=cut

sub get_result {
    my $self = shift;

    $self->{NEW_TEXT};
}


=begin sdf

H3: clear_output

Clears all pending output for the object. Does B<not> clear the table data.

Returns the object passed.

=cut

sub clear_output {
    my $self = shift;

    $self->{NEW_TEXT} = '';
}


=begin sdf

H3: text_only

Returns the text with all HTML tags removed. Designed to be very quick and
therfor uses a brute force method of removing tags:
    * Locate and remove all <..> patterns!

Originally written to help calculate the amount of text characters a cell would
require to help better convert table data to non-table data.

=cut

sub text_only {
    my $self = shift;
    my(
        $in,
    ) = shift;

    $in =~ s/<[^>]+>//gs;            # treat input string as a single line of text
    $in;
}



=begin sdf

H2: Private Methods

These methods should not be called by methods outside this module.

=cut


=begin sdf

H3: _clean_up_table

Does the following clean up for the parsed table data:
    * Foces all rows to have the same number of columns. All added columns
    contain a blank string.

    * Sets 'column_count' element to column width of table

Normally called automatically by L<"parse"> and L<"parse_file">, this method is
critical to prevent weird errors during column/row manipulation and display.

Returns parser object passed.

See also L<"parse">, L<"parse_file">

=cut

sub _clean_up_table {
    my $self = shift;

    # Scan adjusting all columns to be maximum width if table was found to be jagged
    if($self->{TABLE}{jagged}) {
        my $max = $self->column_count() + 1;

        foreach(@{$self->{TABLE}{data}}) {
            while(scalar(@$_) < $max) {
                push @$_, '';
            }
        }

        # Table is no longer jagged
        $self->{TABLE}{jagged} = 0;
    }

    $self;
}


=begin sdf

H3: _default_post_process

Default handler for processing tables for this module. The default is to run the table
through the L<"table_out"> method and take the resulting string and add it to the table
output string. All blank columns (see L<"drop_blank_columns">) are first droped to
help simplify the table.

Also place text in a TABLE block. This causes iSilo (and others) to properly handle preceeding
<CENTER> tags and other things as well as highlighting the table in some manner.

Users can override this default via L<"set_table_post_process"> or by overloading
this method.

The results are returned as a string which are passed to the L<"output"> method,
which the user can overload if desired to accept/process whatever data format is
desired.

Note: The $self->{TABLE}{table_active} variable is guarenteed to be:
    * 0 - Text is not nested within another table
    * 1 - Text is nested within another table

Note: If there is no non-whitespace data for the table then the entire table
is ignored, a blank string is returned.

=cut

sub _default_post_process {
    my $self = shift;

    $self->drop_blank_columns();

    my $tmp = $self->table_out();
    if($tmp =~ /^\s*$/) {
        # If no data then empty table (data was probably stripped by another process)
        # so return nothing.
        '';
    } else {
# DEBUG
#open(T, ">>t") && do {
#    print T $tmp, "\n---------------\n\n";

        $Table_Start . $tmp . $Table_End;
    }
}


=begin sdf

H3: _default_conversion_process

Default handler for conversion processing the converted table date (BEFORE L<"_default_post_process">).

This method does nothing and is only a place holder for user functionality. Typically
the user method would perform row type conversion, such as forcing a row to be
a header, removing data, changine data, etc. See L<"default_post_process"> for output
processing. All conversions should be done to the current table object.

Originallly defined so that the more comples L<"_default_post_process"> could be left alone.

See also L<"set_table_conversion_process">


Note: The $self->{TABLE}{table_active} variable is guarenteed to be:
    * 0 - Text is not nested within another table
    * 1 - Text is nested within another table

Returns table object that the rest of the processing will use.

=cut

sub _default_conversion_process {
    my $self = shift;

    $self;
}

1;