1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
|
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>3.DOM Record Model Configuration</title><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot"><link rel="home" href="index.html" title="Zebra - User's Guide and Reference"><link rel="up" href="record-model-domxml.html" title="Chapter7.DOM XML Record Model and Filter Module"><link rel="prev" href="record-model-domxml-pipeline.html" title="2.DOM XML filter pipeline configuration"><link rel="next" href="record-model-alvisxslt.html" title="Chapter8.ALVIS XML Record Model and Filter Module"></head><body><link rel="stylesheet" type="text/css" href="common/style1.css"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">3.<acronym class="acronym">DOM</acronym> Record Model Configuration</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="record-model-domxml-pipeline.html">Prev</a></td><th width="60%" align="center">Chapter7.<acronym class="acronym">DOM</acronym> <acronym class="acronym">XML</acronym> Record Model and Filter Module</th><td width="20%" align="right"><a accesskey="n" href="record-model-alvisxslt.html">Next</a></td></tr></table><hr></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="record-model-domxml-conf"></a>3.<acronym class="acronym">DOM</acronym> Record Model Configuration</h2></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="record-model-domxml-index"></a>3.1.<acronym class="acronym">DOM</acronym> Indexing Configuration</h3></div></div></div><p>
As mentioned above, there can be only one indexing pipeline,
and configuration of the indexing process is a synonym
of writing an <acronym class="acronym">XSLT</acronym> stylesheet which produces <acronym class="acronym">XML</acronym> output containing the
magic processing instructions or elements discussed in
<a class="xref" href="record-model-domxml-pipeline.html#record-model-domxml-canonical-index" title="2.5.Canonical Indexing Format">Section2.5, “Canonical Indexing Format”</a>.
Obviously, there are million of different ways to accomplish this
task, and some comments and code snippets are in order to
enlighten the wary.
</p><p>
Stylesheets can be written in the <span class="emphasis"><em>pull</em></span> or
the <span class="emphasis"><em>push</em></span> style: <span class="emphasis"><em>pull</em></span>
means that the output <acronym class="acronym">XML</acronym> structure is taken as starting point of
the internal structure of the <acronym class="acronym">XSLT</acronym> stylesheet, and portions of
the input <acronym class="acronym">XML</acronym> are <span class="emphasis"><em>pulled</em></span> out and inserted
into the right spots of the output <acronym class="acronym">XML</acronym> structure.
On the other
side, <span class="emphasis"><em>push</em></span> <acronym class="acronym">XSLT</acronym> stylesheets are recursively
calling their template definitions, a process which is commanded
by the input <acronym class="acronym">XML</acronym> structure, and is triggered to produce
some output <acronym class="acronym">XML</acronym>
whenever some special conditions in the input stylesheets are
met. The <span class="emphasis"><em>pull</em></span> type is well-suited for input
<acronym class="acronym">XML</acronym> with strong and well-defined structure and semantics, like the
following <acronym class="acronym">OAI</acronym> indexing example, whereas the
<span class="emphasis"><em>push</em></span> type might be the only possible way to
sort out deeply recursive input <acronym class="acronym">XML</acronym> formats.
</p><p>
A <span class="emphasis"><em>pull</em></span> stylesheet example used to index
<acronym class="acronym">OAI</acronym> harvested records could use some of the following template
definitions:
</p><pre class="screen">
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:z="http://indexdata.com/zebra-2.0"
xmlns:oai="http://www.openarchives.org/&acro.oai;/2.0/"
xmlns:oai_dc="http://www.openarchives.org/&acro.oai;/2.0/oai_dc/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
version="1.0">
<!-- Example pull and magic element style Zebra indexing -->
<xsl:output indent="yes" method="xml" version="1.0" encoding="UTF-8"/>
<!-- disable all default text node output -->
<xsl:template match="text()"/>
<!-- disable all default recursive element node transversal -->
<xsl:template match="node()"/>
<!-- match only on oai xml record root -->
<xsl:template match="/">
<z:record z:id="{normalize-space(oai:record/oai:header/oai:identifier)}">
<!-- you may use z:rank="{some XSLT; function here}" -->
<!-- explicetly calling defined templates -->
<xsl:apply-templates/>
</z:record>
</xsl:template>
<!-- OAI indexing templates -->
<xsl:template match="oai:record/oai:header/oai:identifier">
<z:index name="oai_identifier:0">
<xsl:value-of select="."/>
</z:index>
</xsl:template>
<!-- etc, etc -->
<!-- DC specific indexing templates -->
<xsl:template match="oai:record/oai:metadata/oai_dc:dc/dc:title">
<z:index name="dc_any:w dc_title:w dc_title:p dc_title:s ">
<xsl:value-of select="."/>
</z:index>
</xsl:template>
<!-- etc, etc -->
</xsl:stylesheet>
</pre><p>
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="record-model-domxml-index-marc"></a>3.2.<acronym class="acronym">DOM</acronym> Indexing <acronym class="acronym">MARCXML</acronym></h3></div></div></div><p>
The <acronym class="acronym">DOM</acronym> filter allows indexing of both binary <acronym class="acronym">MARC</acronym> records
and <acronym class="acronym">MARCXML</acronym> records, depending on its configuration.
A typical <acronym class="acronym">MARCXML</acronym> record might look like this:
</p><pre class="screen">
<record xmlns="http://www.loc.gov/MARC21/slim">
<rank>42</rank>
<leader>00366nam 22001698a 4500</leader>
<controlfield tag="001"> 11224466 </controlfield>
<controlfield tag="003">DLC </controlfield>
<controlfield tag="005">00000000000000.0 </controlfield>
<controlfield tag="008">910710c19910701nju 00010 eng </controlfield>
<datafield tag="010" ind1=" " ind2=" ">
<subfield code="a"> 11224466 </subfield>
</datafield>
<datafield tag="040" ind1=" " ind2=" ">
<subfield code="a">DLC</subfield>
<subfield code="c">DLC</subfield>
</datafield>
<datafield tag="050" ind1="0" ind2="0">
<subfield code="a">123-xyz</subfield>
</datafield>
<datafield tag="100" ind1="1" ind2="0">
<subfield code="a">Jack Collins</subfield>
</datafield>
<datafield tag="245" ind1="1" ind2="0">
<subfield code="a">How to program a computer</subfield>
</datafield>
<datafield tag="260" ind1="1" ind2=" ">
<subfield code="a">Penguin</subfield>
</datafield>
<datafield tag="263" ind1=" " ind2=" ">
<subfield code="a">8710</subfield>
</datafield>
<datafield tag="300" ind1=" " ind2=" ">
<subfield code="a">p. cm.</subfield>
</datafield>
</record>
</pre><p>
</p><p>
It is easily possible to make string manipulation in the <acronym class="acronym">DOM</acronym>
filter. For example, if you want to drop some leading articles
in the indexing of sort fields, you might want to pick out the
<acronym class="acronym">MARCXML</acronym> indicator attributes to chop of leading substrings. If
the above <acronym class="acronym">XML</acronym> example would have an indicator
<code class="literal">ind2="8"</code> in the title field
<code class="literal">245</code>, i.e.
</p><pre class="screen">
<datafield tag="245" ind1="1" ind2="8">
<subfield code="a">How to program a computer</subfield>
</datafield>
</pre><p>
one could write a template taking into account this information
to chop the first <code class="literal">8</code> characters from the
sorting index <code class="literal">title:s</code> like this:
</p><pre class="screen">
<xsl:template match="m:datafield[@tag='245']">
<xsl:variable name="chop">
<xsl:choose>
<xsl:when test="not(number(@ind2))">0</xsl:when>
<xsl:otherwise><xsl:value-of select="number(@ind2)"/></xsl:otherwise>
</xsl:choose>
</xsl:variable>
<z:index name="title:w title:p any:w">
<xsl:value-of select="m:subfield[@code='a']"/>
</z:index>
<z:index name="title:s">
<xsl:value-of select="substring(m:subfield[@code='a'], $chop)"/>
</z:index>
</xsl:template>
</pre><p>
The output of the above <acronym class="acronym">MARCXML</acronym> and <acronym class="acronym">XSLT</acronym> excerpt would then be:
</p><pre class="screen">
<z:index name="title:w title:p any:w">How to program a computer</z:index>
<z:index name="title:s">program a computer</z:index>
</pre><p>
and the record would be sorted in the title index under 'P', not 'H'.
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="record-model-domxml-index-wizzard"></a>3.3.<acronym class="acronym">DOM</acronym> Indexing Wizardry</h3></div></div></div><p>
The names and types of the indexes can be defined in the
indexing <acronym class="acronym">XSLT</acronym> stylesheet <span class="emphasis"><em>dynamically according to
content in the original <acronym class="acronym">XML</acronym> records</em></span>, which has
opportunities for great power and wizardry as well as grande
disaster.
</p><p>
The following excerpt of a <span class="emphasis"><em>push</em></span> stylesheet
<span class="emphasis"><em>might</em></span>
be a good idea according to your strict control of the <acronym class="acronym">XML</acronym>
input format (due to rigorous checking against well-defined and
tight RelaxNG or <acronym class="acronym">XML</acronym> Schema's, for example):
</p><pre class="screen">
<xsl:template name="element-name-indexes">
<z:index name="{name()}:w">
<xsl:value-of select="'1'"/>
</z:index>
</xsl:template>
</pre><p>
This template creates indexes which have the name of the working
node of any input <acronym class="acronym">XML</acronym> file, and assigns a '1' to the index.
The example query
<code class="literal">find @attr 1=xyz 1</code>
finds all files which contain at least one
<code class="literal">xyz</code> <acronym class="acronym">XML</acronym> element. In case you can not control
which element names the input files contain, you might ask for
disaster and bad karma using this technique.
</p><p>
One variation over the theme <span class="emphasis"><em>dynamically created
indexes</em></span> will definitely be unwise:
</p><pre class="screen">
<!-- match on oai xml record root -->
<xsl:template match="/">
<z:record>
<!-- create dynamic index name from input content -->
<xsl:variable name="dynamic_content">
<xsl:value-of select="oai:record/oai:header/oai:identifier"/>
</xsl:variable>
<!-- create zillions of indexes with unknown names -->
<z:index name="{$dynamic_content}:w">
<xsl:value-of select="oai:record/oai:metadata/oai_dc:dc"/>
</z:index>
</z:record>
</xsl:template>
</pre><p>
Don't be tempted to play too smart tricks with the power of
<acronym class="acronym">XSLT</acronym>, the above example will create zillions of
indexes with unpredictable names, resulting in severe <span class="application">Zebra</span>
index pollution..
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="record-model-domxml-debug"></a>3.4.Debuggig <acronym class="acronym">DOM</acronym> Filter Configurations</h3></div></div></div><p>
It can be very hard to debug a <acronym class="acronym">DOM</acronym> filter setup due to the many
successive <acronym class="acronym">MARC</acronym> syntax translations, <acronym class="acronym">XML</acronym> stream splitting and
<acronym class="acronym">XSLT</acronym> transformations involved. As an aid, you have always the
power of the <code class="literal">-s</code> command line switch to the
<code class="literal">zebraidz</code> indexing command at your hand:
</p><pre class="screen">
zebraidx -s -c zebra.cfg update some_record_stream.xml
</pre><p>
This command line simulates indexing and dumps a lot of debug
information in the logs, telling exactly which transformations
have been applied, how the documents look like after each
transformation, and which record ids and terms are send to the indexer.
</p></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="record-model-domxml-pipeline.html">Prev</a></td><td width="20%" align="center"><a accesskey="u" href="record-model-domxml.html">Up</a></td><td width="40%" align="right"><a accesskey="n" href="record-model-alvisxslt.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">2.<acronym class="acronym">DOM</acronym> <acronym class="acronym">XML</acronym> filter pipeline configuration</td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top">Chapter8.ALVIS <acronym class="acronym">XML</acronym> Record Model and Filter Module</td></tr></table></div></body></html>
|
