File: categorisation.html

package info (click to toggle)
xapian-core 1.2.3-2
links: PTS, VCS
area: main
in suites: squeeze
size: 17,528 kB
ctags: 13,287
sloc: cpp: 99,474; sh: 10,626; ansic: 7,551; perl: 758; makefile: 611; python: 40
file content (375 lines) | stat: -rw-r--r-- 10,859 bytes
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
<title>Xapian Categorisation Support</title>
<style type="text/css">

/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 5196 2007-06-03 20:25:28Z wiemann $
:Copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the HTML output of Docutils.

See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/

/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
  border: 0 }

table.borderless td, table.borderless th {
  /* Override padding for "table.docutils td" with "! important".
     The right padding separates the table cells. */
  padding: 0 0.5em 0 0 ! important }

.first {
  /* Override more specific margin styles with "! important". */
  margin-top: 0 ! important }

.last, .with-subtitle {
  margin-bottom: 0 ! important }

.hidden {
  display: none }

a.toc-backref {
  text-decoration: none ;
  color: black }

blockquote.epigraph {
  margin: 2em 5em ; }

dl.docutils dd {
  margin-bottom: 0.5em }

/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
  font-weight: bold }
*/

div.abstract {
  margin: 2em 5em }

div.abstract p.topic-title {
  font-weight: bold ;
  text-align: center }

div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
  margin: 2em ;
  border: medium outset ;
  padding: 1em }

div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
  font-weight: bold ;
  font-family: sans-serif }

div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
  color: red ;
  font-weight: bold ;
  font-family: sans-serif }

/* Uncomment (and remove this text!) to get reduced vertical space in
   compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
  margin-bottom: 0.5em }

div.compound .compound-last, div.compound .compound-middle {
  margin-top: 0.5em }
*/

div.dedication {
  margin: 2em 5em ;
  text-align: center ;
  font-style: italic }

div.dedication p.topic-title {
  font-weight: bold ;
  font-style: normal }

div.figure {
  margin-left: 2em ;
  margin-right: 2em }

div.footer, div.header {
  clear: both;
  font-size: smaller }

div.line-block {
  display: block ;
  margin-top: 1em ;
  margin-bottom: 1em }

div.line-block div.line-block {
  margin-top: 0 ;
  margin-bottom: 0 ;
  margin-left: 1.5em }

div.sidebar {
  margin: 0 0 0.5em 1em ;
  border: medium outset ;
  padding: 1em ;
  background-color: #ffffee ;
  width: 40% ;
  float: right ;
  clear: right }

div.sidebar p.rubric {
  font-family: sans-serif ;
  font-size: medium }

div.system-messages {
  margin: 5em }

div.system-messages h1 {
  color: red }

div.system-message {
  border: medium outset ;
  padding: 1em }

div.system-message p.system-message-title {
  color: red ;
  font-weight: bold }

div.topic {
  margin: 2em }

h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
  margin-top: 0.4em }

h1.title {
  text-align: center }

h2.subtitle {
  text-align: center }

hr.docutils {
  width: 75% }

img.align-left {
  clear: left }

img.align-right {
  clear: right }

ol.simple, ul.simple {
  margin-bottom: 1em }

ol.arabic {
  list-style: decimal }

ol.loweralpha {
  list-style: lower-alpha }

ol.upperalpha {
  list-style: upper-alpha }

ol.lowerroman {
  list-style: lower-roman }

ol.upperroman {
  list-style: upper-roman }

p.attribution {
  text-align: right ;
  margin-left: 50% }

p.caption {
  font-style: italic }

p.credits {
  font-style: italic ;
  font-size: smaller }

p.label {
  white-space: nowrap }

p.rubric {
  font-weight: bold ;
  font-size: larger ;
  color: maroon ;
  text-align: center }

p.sidebar-title {
  font-family: sans-serif ;
  font-weight: bold ;
  font-size: larger }

p.sidebar-subtitle {
  font-family: sans-serif ;
  font-weight: bold }

p.topic-title {
  font-weight: bold }

pre.address {
  margin-bottom: 0 ;
  margin-top: 0 ;
  font-family: serif ;
  font-size: 100% }

pre.literal-block, pre.doctest-block {
  margin-left: 2em ;
  margin-right: 2em }

span.classifier {
  font-family: sans-serif ;
  font-style: oblique }

span.classifier-delimiter {
  font-family: sans-serif ;
  font-weight: bold }

span.interpreted {
  font-family: sans-serif }

span.option {
  white-space: nowrap }

span.pre {
  white-space: pre }

span.problematic {
  color: red }

span.section-subtitle {
  /* font-size relative to parent (h1..h6 element) */
  font-size: 80% }

table.citation {
  border-left: solid 1px gray;
  margin-left: 1px }

table.docinfo {
  margin: 2em 4em }

table.docutils {
  margin-top: 0.5em ;
  margin-bottom: 0.5em }

table.footnote {
  border-left: solid 1px black;
  margin-left: 1px }

table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
  padding-left: 0.5em ;
  padding-right: 0.5em ;
  vertical-align: top }

table.docutils th.field-name, table.docinfo th.docinfo-name {
  font-weight: bold ;
  text-align: left ;
  white-space: nowrap ;
  padding-left: 0 }

h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
  font-size: 100% }

ul.auto-toc {
  list-style-type: none }

</style>
</head>
<body>
<div class="document" id="xapian-categorisation-support">
<h1 class="title">Xapian Categorisation Support</h1>

<!-- Copyright (C) 2007 Olly Betts -->
<!-- Copyright (C) 2009 Lemur Consulting Ltd -->
<div class="contents topic" id="table-of-contents">
<p class="topic-title first">Table of contents</p>
<ul class="simple">
<li><a class="reference internal" href="#introduction" id="id1">Introduction</a></li>
<li><a class="reference internal" href="#how-to-make-use-of-the-categorisation-functionality" id="id2">How to make use of the categorisation functionality</a><ul>
<li><a class="reference internal" href="#indexing" id="id3">Indexing</a></li>
<li><a class="reference internal" href="#searching" id="id4">Searching</a></li>
<li><a class="reference internal" href="#restricting-by-category-values" id="id5">Restricting by category values</a></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="introduction">
<h1><a class="toc-backref" href="#id1">Introduction</a></h1>
<p>Xapian provides functionality which allows you to dynamically generate complete
lists of category values which feature in matching documents.  There are
numerous potential uses this can be put to, but a common one is to offer the
user the ability to narrow down their search by filtering it to only include
documents with a particular value of a particular category.  This is often
referred to as <tt class="docutils literal"><span class="pre">faceted</span> <span class="pre">search</span></tt>.</p>
<p>In some applications, you may have many different categories (for example
colour, price, width, height) but not always want to offer all of them
for every search.  If all the results are red, and none have width, it's
not useful to offer to narrow the search by colour or width.  Also, the
user interface may not have room to include every category, so you may
want to select the &quot;best&quot; few categories to show the user.</p>
</div>
<div class="section" id="how-to-make-use-of-the-categorisation-functionality">
<h1><a class="toc-backref" href="#id2">How to make use of the categorisation functionality</a></h1>
<div class="section" id="indexing">
<h2><a class="toc-backref" href="#id3">Indexing</a></h2>
<p>When indexing a document, you need to add each category in a different
number value slot.</p>
</div>
<div class="section" id="searching">
<h2><a class="toc-backref" href="#id4">Searching</a></h2>
<p>At search time, you need to pass a <tt class="docutils literal"><span class="pre">Xapian::ValueCountMatchSpy</span></tt> object for
each category you want to look at to <tt class="docutils literal"><span class="pre">Xapian::Enquire::add_matchspy()</span></tt>, like
so:</p>
<pre class="literal-block">
Xapian::ValueCountMatchSpy spy0(0);
Xapian::ValueCountMatchSpy spy1(1);
Xapian::ValueCountMatchSpy spy3(3);

Xapian::Enquire enq(db);
enq.add_matchspy(spy0);
enq.add_matchspy(spy1);
enq.add_matchspy(spy3);

enq.set_query(query);

Xapian::MSet mset = enq.get_mset(0, 10, 10000);
</pre>
<p>The <tt class="docutils literal"><span class="pre">10000</span></tt> in the call to <tt class="docutils literal"><span class="pre">get_mset()</span></tt> tells Xapian to check at least
10000 documents, so the MatchSpies will be passed at least 10000 documents
to tally category information from (unless fewer than 10000 documents match the
query, in which case it will see all of them).  Setting this higher will make
the counts exact, but Xapian will have to do more work for most queries so
searches will be slower.</p>
<p>The <tt class="docutils literal"><span class="pre">spy</span></tt> objects now contain the category information.  You can find out how
many documents they looked at by calling <tt class="docutils literal"><span class="pre">spy0.get_total()</span></tt>.  (All the spies
will have looked at the same number of documents.)  You can read the values
from, say, <tt class="docutils literal"><span class="pre">spy0</span></tt> like this:</p>
<pre class="literal-block">
const map&lt;string, size_t&gt; &amp; cat = spy0.get_values();
map&lt;string, size_t&gt;::const_iterator i;
for (i = cat.begin(); i != cat.end(); ++i) {
    cout &lt;&lt; i-&gt;first &lt;&lt; &quot;: &quot; &lt;&lt; i-&gt;second &lt;&lt; endl;
}
</pre>
</div>
<div class="section" id="restricting-by-category-values">
<h2><a class="toc-backref" href="#id5">Restricting by category values</a></h2>
<p>If you're using the categorisation to offer the user choices for narrowing down
their search results, you then need to be able to apply a suitable filter.</p>
<p>For a single value, you could use <tt class="docutils literal"><span class="pre">Xapian::Query::OP_VALUE_RANGE</span></tt> with the
same start and end, or <tt class="docutils literal"><span class="pre">Xapian::MatchDecider</span></tt>, but it's probably most
efficient to also index the categories as suitably prefixed boolean terms and
use those for filtering.</p>
</div>
</div>
</div>
</body>
</html>