File: okopnik.html

package info (click to toggle)
lg-issue97 2-1
links: PTS
area: main
in suites: sarge
size: 1,280 kB
ctags: 65
sloc: perl: 68; makefile: 34; sh: 34
file content (495 lines) | stat: -rw-r--r-- 17,170 bytes

<html>
<head>
<link href="../lg.css" rel="stylesheet" type="text/css" media="screen, projection"  />
<title>Using the HTML::Template module LG #97</title>

<style type="text/css" media="screen, projection">
<!--


.articlecontent {
	position:absolute;
	top:143px;
}


-->
</style>


</head>

<body>


<img src="../gx/2003/newlogo-blank-200-gold2.jpg" id="logo" alt="Linux Gazette"/>
<p id="fun">...making Linux just a little more fun!</p>


<div class="content articlecontent">

<div id="previousnexttop">
<A HREF="moen.html" >&lt;-- prev</A> | <A HREF="oregan.html" >next --&gt;</A>
</div>



<h1>Using the HTML::Template module</h1>
<p id="by"><b>By <A HREF="../authors/okopnik.html">Ben Okopnik</A></b></p>

<p>
<p>
Recently, I needed to generate a Web page - the Linux Gazette's <a
href="http://linuxgazette.net/mirrors.html">"Mirrors and Translations"</a>
page, actually - based on the contents of a database. Perl is famous for
its ability to connect to almost any database via a common interface, given
its <code>DBD::DBI</code> module kit; however, the challenge in this case
came from the front end, the HTML generation. Sure, I could use the CGI
module to output whatever I needed - but in this case, I already had the
static page that I wanted to create, and saw no reason to rewrite all the
static content in CGI. Also, the final product was not to be a CGI file but
a generated HTML page. In fact, everything in this case hinted at
<b>templating</b>, a process in which I would use the static HTML with a
few special tags and a script which would then apply processing based on
those tags. This made especially good sense since it drew a clean
separating line between writing HTML and creating code, very different
tasks and ones for which I have different mental states (layout designer
vs. programmer.)

<p>
As with anything in Perl, <a
href="http://okopnik.freeshell.org/acronyms.cgi?string=TMTOWTDI&type=acronym">TMTOWTDI</a>
- there was a number of modules available on <a href="http://cpan.org">CPAN</a>
(the Comprehensive Perl Archive Network) that could do the job. However, I
had used the <code>HTML::Template</code> module in the past, and the job
wasn't particularly complicated (although <code>HTML::Template</code> can
handle some very complex jobs indeed), so that's what I settled on. My
first task was to hunt through the HTML, removing the dozens of repetitive
stanzas and replacing them with the appropriate tag framework that the
module would utilize later. We had also made the decision not to display
the maintainers email addresses, even in the <a
href="../issue86/okopnik.html">munged form</a> that I use to deter
spammers; those of you who use our mirrors and want to thank these fine
folks for making LG available should be able to find an address link on the
mirror site without much trouble.
<p>

<b>Fragment of the old page (there were several dozen entries like this):</b>
<pre><hr>
...
&lt;A name="AU"&gt;&lt;/A&gt;
&lt;DT&gt;&lt;B&gt;&lt;font color="maroon"&gt;AUSTRALIA (AU)&lt;/font&gt;&lt;/B&gt;&lt;/DT&gt;
&lt;DD&gt;
&lt;STRONG&gt;&lt;FONT COLOR="green"&gt;&lt;TT&gt;[WWW]&lt;/TT&gt;&lt;/FONT&gt;&lt;/STRONG&gt;
&lt;A HREF="http://www.localnet.com.au/lg/index.html"&gt;http://www.localnet.com.au/lg/index.html&lt;/A&gt;
&lt;BR&gt;
&lt;SMALL&gt;
Maintainer: Jim McGregor &amp;lt;&lt;A HREF="mailto:nospam@here.please"&gt;nospam@here.please&lt;/A&gt;&amp;gt; &amp;nbsp;
&lt;/SMALL&gt;
&lt;P&gt;
&lt;/DD&gt;

&lt;DD&gt;
&lt;STRONG&gt;&lt;FONT COLOR="green"&gt;&lt;TT&gt;[WWW]&lt;/TT&gt;&lt;/FONT&gt;&lt;/STRONG&gt;
&lt;A HREF="http://www.eastwood.apana.org.au/Linux/LinuxGazette/"&gt;http://www.eastwood.apana.org.au/Linux/LinuxGazette/&lt;/A&gt;
&lt;BR&gt;
&lt;SMALL&gt;
Maintainer: Mick Stock &amp;lt;&lt;A HREF="mailto:nospam@here.please"&gt;nospam@here.please&lt;/A&gt;&amp;gt; &amp;nbsp;
&lt;/SMALL&gt;
&lt;P&gt;
&lt;/DD&gt;
</BODY></HTML>
...
<hr></pre>

<b>Single-stanza replacement for all the entries (new template):</b><br>
<pre><hr>
...
&lt;a name="&lt;TMPL_VAR NAME=FQDN&gt;"&gt;&lt;/a&gt;
&lt;dt&gt;&lt;b&gt;&lt;font color="maroon"&gt;&lt;TMPL_VAR NAME=FQDN&gt; (&lt;TMPL_VAR NAME=TLD&gt;)&lt;/font&gt;&lt;/b&gt;&lt;/dt&gt;
&lt;dd&gt;&lt;strong&gt;&lt;font color="green"&gt;&lt;tt&gt;[WWW]&lt;/tt&gt;&lt;/font&gt;&lt;/strong&gt;
&lt;a href="&lt;TMPL_VAR NAME=HTTP&gt;"&gt;&lt;TMPL_VAR NAME=HTTP&gt;&lt;/a&gt;
&lt;br&gt;
&lt;strong&gt;&lt;font color="red"&gt;&lt;tt&gt;[FTP]&lt;/tt&gt;&lt;/font&gt;&lt;/strong&gt;
&lt;a href="&lt;TMPL_VAR NAME=FTP&gt;"&gt;&lt;TMPL_VAR NAME=FTP&gt;&lt;/a&gt;
&lt;br&gt;
&lt;small&gt;
Maintainer: &lt;TMPL_VAR NAME=MAINT&gt;
&lt;/small&gt;
&lt;p&gt;
&lt;/dd&gt;
</BODY></HTML>
...
<hr></pre>

Now, the challenge had shifted away from generating the HTML to just
dealing with code. What I needed to do was sort the data into groups and
subgroups - that is, there would some number of "country" headings, some
number of "mirror" headings under each of those, and either one or two
(WWW, FTP, or both) hosts plus a maintainer under each "mirror" heading. In
programmatic terms, these are known as "nested loops", and are not that
difficult to code. However, translating that into HTML terms could be an
exercise in kind of language abilities of which your mother would not
approve... if it wasn't for <code>HTML::Template</code>.

<p> 
<b>Note:</b> Using <code>HTML::Template</code> is normally very simple; in
fact, learning the basics of using it usually takes only a minute or two
(see the example at the top of "perldoc HTML::Template".) However, in this
instance, we're creating <i>nested lists</i> - a rather more complex issue
than simple variable/tag replacement - and thus, the coding issues get a bit
deeper. However, this isn't due to <code>HTML::Template</code>; if you
think about the issues inherent in modeling what is already a complex data
structure and then transferring that structure into a "passive" layout
language... truth to tell, I'm somewhat surprised that it can be done at
all. Kudos and my hat's off to Sam Tregar (author of the module) and Jesse
Erlbaum (the man responsible for TMPL_LOOP.)

<p>
<h3>References</h3>

The area that seems to strike fear into the heart of neophyte programmers,
more so than anything else, is the topic of <i>references</i>. Particularly
in Perl, where everything is supposed to be warm, fuzzy, and easy to
understand. However, understanding references and objects - in my opinion -
are the very things that take one from being a Perl user to a Perl
programmer. I'm going to simply show how references work with a bit of an
explanation, but the real comprehensive reference for references :) is
included with the standard Perl documentation. Simply type "perldoc
perlreftut" at your command line for a good introduction, and be sure to
take a look at "perldoc perlref" for the complete documentation.

<p>
First, in order to understand how the data structure must be laid out to
create the pattern that we need, let's take a look at that pattern.
Fortunately, in Perl it's easy to lay out the data structures to match what
they represent (whitespace is arbitrary, so you can follow your preferences
- but see "perldoc perlstyle".) What we'll want to do here is build the
structure that contains all the values we want to assign within the loop as
well as the names which are associated with those values. Those of you with
a little Perl experience are nodding and smiling already: the word
"associated" points very clearly to the type of variable we need - a hash!
Taking a single "row" (per-country entry) - Austria, as a random example -
here is how it looks:

<pre><hr>
%row = (
	tld	=&gt; AT,
	fqdn	=&gt; Austria,
	sites	=&gt; [
			{ 
			  http	=&gt;  "http://www.luchs.at/linuxgazette/",
			  maint =&gt;  "Rene Pfeiffer"
			},

			{ 
			  http  =&gt;  "http://info.ccone.at/lg/",
			  maint =&gt;  "Gerhard Beck"
			},

			{
			  http	=&gt;  "http://linuxgazette.tuwien.ac.at/",
			  ftp	=&gt;  "ftp://linuxgazette.tuwien.ac.at/pub/linuxgazette/",
			  maint	=&gt;  "Tony Sprinzl"
			}
		   ]
);
<hr></pre>

The above hash, <code>%row</code>, matches our requirements exactly: its
keys will be used to match (case-insensitively) the tag names in the HTML,
and the values associated with those keys will be used to replace those
tags. That is, every instance of <code>&lt;TMPL_VAR NAME=FQDN&gt;</code>
in the template will be replaced by "Austria" while this entry is being
processed.  Here are some of the less-obvious points of the above
structure:

<p>
<ul>
<li>The <i>anonymous hash constructor</i>, defined by the curly braces
surrounding each group, stores all the data in an <font
color="red"><b>anonymous hash</b></font> and returns a reference to it.
<li>In turn, the <i>anonymous array constructor</i>, defined by the square
braces surrounding the list of groups, stores all of the above references
in an <font color="green"><b>anonymous array</b></font> and returns a reference to it.
<li>The <code><b>sites</b></code> key points to (is associated with) the
reference to the above <font color="green"><b>anonymous array</b></font>,
and is the name of the loop that we'll use within the HTML to iterate
through all of the above data.
</ul>

As we create a "row" for each country, we will need to store all of them in
a list. Each entry in this list must, of course, contain a reference to
each row that we have built:

<pre><hr>
# Add the hashref to the end of the array
push @mirrors, \%row;
<hr></pre>

Note the '\' preceding the <code>%row</code>; this stores a reference to
<code>%row</code> rather than the hash itself (stuffing a hash into an
array would result in a generally unusable mess - key/value pairs in
effectively random order as array elements.) This is a standard mechanism
for creating multidimensional arrays, lists-of-hashes, etc. in Perl.

<p>
And - one more time, <b><i>with gusto</i></b> - HTML::Template's
<code>param()</code> subroutine, as most other subroutines in Perl and many
other languages, expects a reference to the array rather than the array
itself:

<pre><hr>
# Create a new HTML::Template object
my $t = HTML::Template -&gt; new( filename =&gt; "mirrors.tmpl" );

# Pass the listref to param()
$t -&gt; param( MIRR =&gt; \@mirrors );
<hr></pre>

"<b>And</b>", as Austin Powers would say, "<b>Oi'm <i>spent.</i></b>" Those of you scared
of the Big Bad References may come out from under the bed now. :)

<p>
Looking at it from the other end, the matching part of the template for
this loop looks like this:

<pre><hr>
&lt;dl&gt;
&lt;TMPL_LOOP NAME=MIRR&gt;

&lt;dt&gt;&lt;b&gt;&lt;font color="maroon"&gt;
&lt;a name="&lt;TMPL_VAR NAME=TLD&gt;"&gt;
&lt;img src="gx/flags/&lt;TMPL_VAR NAME=TLD&gt;.jpg" border="1"&gt;
&lt;/a&gt;
&lt;TMPL_VAR FQDN&gt; [&lt;TMPL_VAR NAME=TLD&gt;]
&lt;/font&gt;&lt;/b&gt;&lt;/dt&gt;

&lt;TMPL_LOOP NAME=SITES&gt;

&lt;dd&gt;
&lt;TMPL_IF NAME="HTTP"&gt;
&lt;strong&gt;&lt;font color="green"&gt;&lt;tt&gt;[WWW]&lt;/tt&gt;&lt;/font&gt;&lt;/strong&gt;
&lt;a href=&lt;TMPL_VAR HTTP&gt;&gt;
&lt;TMPL_VAR HTTP&gt;
&lt;/a&gt;&lt;br&gt;
&lt;/TMPL_IF&gt;

&lt;TMPL_IF NAME="FTP"&gt;
&lt;strong&gt;&lt;font color="green"&gt;&lt;tt&gt;[FTP]&lt;/tt&gt;&lt;/font&gt;&lt;/strong&gt;
&lt;a href=&lt;TMPL_VAR NAME=FTP&gt;&gt;
&lt;TMPL_VAR NAME=FTP&gt;
&lt;/a&gt;&lt;br&gt;
&lt;/TMPL_IF&gt;
&lt;small&gt;
Maintainer:
&lt;TMPL_VAR NAME=MAINT&gt;
&lt;/small&gt;
&lt;p&gt;
&lt;/dd&gt;

&lt;/TMPL_LOOP&gt;
&lt;/TMPL_LOOP&gt;

&lt;/dl&gt;
</BODY></HTML>
<hr></pre>

To recap what we're looking at, there are two loops defined in the above
template, one inside the other: <code>&lt;TMPL_LOOP NAME=MIRR&gt;</code> and
<code>&lt;TMPL_LOOP NAME=SITES&gt;</code>. Note that the outside loop corresponds
to the name of the parameter key that we assigned when passing the data
construct to <code>param()</code>, and the name of the inside loop is the
same as the key associated with the groups inside the hash we created.

<p><br></p>

However, fine as the above may be for static data that we can simply type
into those anonymous hashes in the 'groups' listref, static data isn't
often what we get in the real world. Databases are updated, file contents
change - and we obviously need to reflect this in our HTML. So, let's take
a look at a code fragment that does this:

<pre><hr>
for $tld ( @tlds ){
	# Set some temporary (per-loop) variables
	my @sites;
	my %row;
	my %line;

	# Here's the inner loop!
	for ( grep /^$tld/, @mirr ){
		# Parse the CSV into fields
		my @rec;
		my %site;
		s/\\,/&amp;comma;/g;
		@rec = split /,/;
		s/&amp;comma;/,/g for @rec;

		# Mirror listings don't require much data
		$site{ http  } = $rec[2];
		$site{ ftp   } = $rec[3];
		$site{ maint } = $rec[4];
		# Load it up!
		push @sites, \%site;
	}
	
	# Outer loop vars
	$row{ tld } = $tld;
	$row{ country } = $country{ $tld };
	# Ref to the inner loop, attached
	$row{ sites } = \@sites;
	
	# ...and load up the total into the array to be passed to param()
	push @mirrs, \%row;
}

# Feed the data to the hungry HTML::Template object
$t -&gt; param( MIRR =&gt; \@mirrs );
<hr></pre>

By the way, the data we're reading in looks like this:

<pre><hr>
AT,,http://www.luchs.at/linuxgazette/,,Rene Pfeiffer,nospam@here.please,
AT,,http://info.ccone.at/lg/,,Gerhard Beck,nospam@here.please,
BE,,http://linuxgazette.linuxbe.org/,,Cedric Gavage,nospam@here.please,
CA,,http://blue7green.crosswinds.net/hobbies/lg/,,Jim Pierce,nospam@here.please,
<hr></pre>

Now we have a highly dynamic chunk of code that will process the data that
we give it, generate the necessary data structure on the fly, and feed it
out to the template. <i>Voila!</i>

<p>
If you want to see the complete script that I wrote for this project, go <a
href="../issue97/misc/okopnik/mirrorgen.txt">here</a>; the template can be
found <a href="../issue97/misc/okopnik/mirrors.tmpl">here</a>. If you would
like to see the latest generated page, go <a
href="http://linuxgazette.net/mirrors.html">here</a>. If you would like to
change the way the page looks and do something great for the Linux
community, join the folks on the list and become a mirror maintainer:
commit some of your disk space and bandwidth and let the Linux Gazette
"mirrors and translations" person - that's me! - know about it <a
href="mailto:&#109;&#105;&#114;&#114;&#111;&#114;&#115;&#64;&#108;&#105;&#110;&#117;&#120;&#103;&#97;&#122;&#101;&#116;&#116;&#101;&#46;&#110;&#101;&#116;">here</a>.

<p>
Happy Linuxing to all!

<hr>
<b>Source material: (I <i>was</i> going to write "References"... :)</b>
<p>
<pre>
perldoc perlreftut
perldoc perlref
perldoc HTML::Template
</pre>
<b>Motivation:</b>
<p>
My annoyance at the lack of good documentation for nested loops under
HTML::Template. :)

</p>


<!-- *** BEGIN author bio *** -->
<P>&nbsp;
<P>
<P> Ben is a Contributing Editor for Linux Gazette and a member of
The Answer Gang.

<!-- *** BEGIN bio *** -->
<P>
<IMG ALT="picture" SRC="../gx/2002/tagbio/ben-okopnik.jpg" WIDTH="199"
   HEIGHT="200" ALIGN="left"  HSPACE="10" VSPACE="10">
<em>
Ben was born in Moscow, Russia in 1962. He became interested in
electricity at age six--promptly demonstrating it by sticking a fork into
a socket and starting a fire--and has been falling down technological mineshafts
ever since. He has been working with computers since the Elder Days, when
they had to be built by soldering parts onto printed circuit boards and
programs had to fit into 4k of memory.  He would gladly pay good money to any
psychologist who can cure him of the resulting nightmares.

<p>Ben's subsequent experiences include creating software in nearly a dozen
languages, network and database maintenance during the approach of a hurricane,
and writing articles for publications ranging from sailing magazines to
technological journals. Having recently completed a seven-year
Atlantic/Caribbean cruise under sail, he is currently docked in Baltimore, MD,
where he works as a technical instructor for Sun Microsystems.

<p>Ben has been working with Linux since 1997, and credits it with his complete
loss of interest in waging nuclear warfare on parts of the Pacific Northwest.
</em>
<br CLEAR="all">
<!-- *** END bio *** -->

<!-- *** END author bio *** -->

<div id="articlefooter">

<p>
Copyright &copy; 2003, Ben Okopnik. Copying license 
<a href="http://linuxgazette.net/copying.html">http://linuxgazette.net/copying.html</a>
</p>

<p>
Published in Issue 97 of Linux Gazette, December 2003
</p>

</div>


<div id="previousnextbottom">
<A HREF="moen.html" >&lt;-- prev</A> | <A HREF="oregan.html" >next --&gt;</A>
</div>


</div>






<div id="navigation">

<a href="../index.html">Home</a>
<a href="../faq/index.html">FAQ</a>
<a href="../lg_index.html">Site Map</a>
<a href="../mirrors.html">Mirrors</a>
<a href="../mirrors.html">Translations</a>
<a href="../search.html">Search</a>
<a href="../archives.html">Archives</a>
<a href="../authors/index.html">Authors</a>
<a href="../contact.html">Contact Us</a>

</div>



<div id="breadcrumbs">

<a href="../index.html">Home</a> &gt; 
<a href="index.html">December 2003 (#97)</a> &gt; 
Article

</div>





<img src="../gx/2003/sit3-shine.7-2.gif" id="tux" alt="Tux"/>




</body>
</html>