File: queryparser.html

package info (click to toggle)
xapian-core 2.0.0-1
links: PTS, VCS
area: main
in suites: experimental
size: 25,008 kB
sloc: cpp: 136,717; ansic: 11,798; sh: 5,416; perl: 1,024; javascript: 551; makefile: 460; tcl: 299; python: 40
file content (552 lines) | stat: -rw-r--r-- 19,676 bytes
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.22.4: https://docutils.sourceforge.io/" />
<title>Xapian::QueryParser Syntax</title>
<style type="text/css">

/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 9511 2024-01-13 09:50:07Z milde $
:Copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the HTML output of Docutils.
Despite the name, some widely supported CSS2 features are used.

See https://docutils.sourceforge.io/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/

/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
  border: 0 }

table.borderless td, table.borderless th {
  /* Override padding for "table.docutils td" with "! important".
     The right padding separates the table cells. */
  padding: 0 0.5em 0 0 ! important }

.first {
  /* Override more specific margin styles with "! important". */
  margin-top: 0 ! important }

.last, .with-subtitle {
  margin-bottom: 0 ! important }

.hidden {
  display: none }

.subscript {
  vertical-align: sub;
  font-size: smaller }

.superscript {
  vertical-align: super;
  font-size: smaller }

a.toc-backref {
  text-decoration: none ;
  color: black }

blockquote.epigraph {
  margin: 2em 5em ; }

dl.docutils dd {
  margin-bottom: 0.5em }

object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
  overflow: hidden;
}

/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
  font-weight: bold }
*/

div.abstract {
  margin: 2em 5em }

div.abstract p.topic-title {
  font-weight: bold ;
  text-align: center }

div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
  margin: 2em ;
  border: medium outset ;
  padding: 1em }

div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
  font-weight: bold ;
  font-family: sans-serif }

div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title, .code .error {
  color: red ;
  font-weight: bold ;
  font-family: sans-serif }

/* Uncomment (and remove this text!) to get reduced vertical space in
   compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
  margin-bottom: 0.5em }

div.compound .compound-last, div.compound .compound-middle {
  margin-top: 0.5em }
*/

div.dedication {
  margin: 2em 5em ;
  text-align: center ;
  font-style: italic }

div.dedication p.topic-title {
  font-weight: bold ;
  font-style: normal }

div.figure {
  margin-left: 2em ;
  margin-right: 2em }

div.footer, div.header {
  clear: both;
  font-size: smaller }

div.line-block {
  display: block ;
  margin-top: 1em ;
  margin-bottom: 1em }

div.line-block div.line-block {
  margin-top: 0 ;
  margin-bottom: 0 ;
  margin-left: 1.5em }

div.sidebar {
  margin: 0 0 0.5em 1em ;
  border: medium outset ;
  padding: 1em ;
  background-color: #ffffee ;
  width: 40% ;
  float: right ;
  clear: right }

div.sidebar p.rubric {
  font-family: sans-serif ;
  font-size: medium }

div.system-messages {
  margin: 5em }

div.system-messages h1 {
  color: red }

div.system-message {
  border: medium outset ;
  padding: 1em }

div.system-message p.system-message-title {
  color: red ;
  font-weight: bold }

div.topic {
  margin: 2em }

h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
  margin-top: 0.4em }

h1.title {
  text-align: center }

h2.subtitle {
  text-align: center }

hr.docutils {
  width: 75% }

img.align-left, .figure.align-left, object.align-left, table.align-left {
  clear: left ;
  float: left ;
  margin-right: 1em }

img.align-right, .figure.align-right, object.align-right, table.align-right {
  clear: right ;
  float: right ;
  margin-left: 1em }

img.align-center, .figure.align-center, object.align-center {
  display: block;
  margin-left: auto;
  margin-right: auto;
}

table.align-center {
  margin-left: auto;
  margin-right: auto;
}

.align-left {
  text-align: left }

.align-center {
  clear: both ;
  text-align: center }

.align-right {
  text-align: right }

/* reset inner alignment in figures */
div.align-right {
  text-align: inherit }

/* div.align-center * { */
/*   text-align: left } */

.align-top    {
  vertical-align: top }

.align-middle {
  vertical-align: middle }

.align-bottom {
  vertical-align: bottom }

ol.simple, ul.simple {
  margin-bottom: 1em }

ol.arabic {
  list-style: decimal }

ol.loweralpha {
  list-style: lower-alpha }

ol.upperalpha {
  list-style: upper-alpha }

ol.lowerroman {
  list-style: lower-roman }

ol.upperroman {
  list-style: upper-roman }

p.attribution {
  text-align: right ;
  margin-left: 50% }

p.caption {
  font-style: italic }

p.credits {
  font-style: italic ;
  font-size: smaller }

p.label {
  white-space: nowrap }

p.rubric {
  font-weight: bold ;
  font-size: larger ;
  color: maroon ;
  text-align: center }

p.sidebar-title {
  font-family: sans-serif ;
  font-weight: bold ;
  font-size: larger }

p.sidebar-subtitle {
  font-family: sans-serif ;
  font-weight: bold }

p.topic-title {
  font-weight: bold }

pre.address {
  margin-bottom: 0 ;
  margin-top: 0 ;
  font: inherit }

pre.literal-block, pre.doctest-block, pre.math, pre.code {
  margin-left: 2em ;
  margin-right: 2em }

pre.code .ln { color: gray; } /* line numbers */
pre.code, code { background-color: #eeeeee }
pre.code .comment, code .comment { color: #5C6576 }
pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
pre.code .literal.string, code .literal.string { color: #0C5404 }
pre.code .name.builtin, code .name.builtin { color: #352B84 }
pre.code .deleted, code .deleted { background-color: #DEB0A1}
pre.code .inserted, code .inserted { background-color: #A3D289}

span.classifier {
  font-family: sans-serif ;
  font-style: oblique }

span.classifier-delimiter {
  font-family: sans-serif ;
  font-weight: bold }

span.interpreted {
  font-family: sans-serif }

span.option {
  white-space: nowrap }

span.pre {
  white-space: pre }

span.problematic, pre.problematic {
  color: red }

span.section-subtitle {
  /* font-size relative to parent (h1..h6 element) */
  font-size: 80% }

table.citation {
  border-left: solid 1px gray;
  margin-left: 1px }

table.docinfo {
  margin: 2em 4em }

table.docutils {
  margin-top: 0.5em ;
  margin-bottom: 0.5em }

table.footnote {
  border-left: solid 1px black;
  margin-left: 1px }

table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
  padding-left: 0.5em ;
  padding-right: 0.5em ;
  vertical-align: top }

table.docutils th.field-name, table.docinfo th.docinfo-name {
  font-weight: bold ;
  text-align: left ;
  white-space: nowrap ;
  padding-left: 0 }

/* "booktabs" style (no vertical lines) */
table.docutils.booktabs {
  border: 0px;
  border-top: 2px solid;
  border-bottom: 2px solid;
  border-collapse: collapse;
}
table.docutils.booktabs * {
  border: 0px;
}
table.docutils.booktabs th {
  border-bottom: thin solid;
  text-align: left;
}

h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
  font-size: 100% }

ul.auto-toc {
  list-style-type: none }

</style>
</head>
<body>
<div class="document" id="xapian-queryparser-syntax">
<h1 class="title">Xapian::QueryParser Syntax</h1>

<p>This document describes the query syntax supported by the
Xapian::QueryParser class. The syntax is designed to be similar to other
web based search engines, so that users familiar with them don't have to
learn a whole new syntax.</p>
<div class="section" id="operators">
<h1>Operators</h1>
<div class="section" id="and">
<h2>AND</h2>
<p><em>expression</em> AND <em>expression</em> matches documents which are matched by
both of the subexpressions.</p>
</div>
<div class="section" id="or">
<h2>OR</h2>
<p><em>expression</em> OR <em>expression</em> matches documents which are matched by
either of the subexpressions.</p>
</div>
<div class="section" id="not">
<h2>NOT</h2>
<p><em>expression</em> NOT <em>expression</em> matches documents which are matched by
only the first subexpression. This can also be written as <em>expression</em>
AND NOT <em>expression</em>. If <tt class="docutils literal">FLAG_PURE_NOT</tt> is enabled, then
NOT <em>expression</em> will match documents which don't match the
subexpression.</p>
</div>
<div class="section" id="xor">
<h2>XOR</h2>
<p><em>expression</em> XOR <em>expression</em> matches documents which are matched by one
or other of the subexpressions, but not both. XOR is probably a bit
esoteric.</p>
</div>
<div class="section" id="and-1">
<h2>'+' and '-'</h2>
<p>A group of terms with some marked with + and - will match documents
containing all of the + terms, but none of the - terms. Terms not marked
with + or - contribute towards the document rankings. You can also use +
and - on phrases and on bracketed expressions.</p>
</div>
<div class="section" id="near">
<h2>NEAR</h2>
<p><tt class="docutils literal">one NEAR two NEAR three</tt> matches documents containing those words
within 10 words of each other. You can set the threshold to <em>n</em> by using
<tt class="docutils literal">NEAR/n</tt> like so: <tt class="docutils literal">one NEAR/6 two</tt>.</p>
</div>
<div class="section" id="adj">
<h2>ADJ</h2>
<p><tt class="docutils literal">ADJ</tt> is like <tt class="docutils literal">NEAR</tt> but only matches if the words appear in the
same order as in the query. So <tt class="docutils literal">one ADJ two ADJ three</tt> matches
documents containing those three words in that order and within 10 words
of each other. You can set the threshold to <em>n</em> by using <tt class="docutils literal">ADJ/n</tt> like
so: <tt class="docutils literal">one ADJ/6 two</tt>.</p>
</div>
<div class="section" id="syn">
<h2>SYN</h2>
<p><tt class="docutils literal">SYN</tt> matches when any of its subqueries match (like <tt class="docutils literal">OR</tt> does)
but the ranking is done assuming the subqueries are synonyms and so treats the
entire <tt class="docutils literal">SYN</tt> subquery as a single term.</p>
</div>
<div class="section" id="bracketed-expressions">
<h2>Bracketed expressions</h2>
<p>You can control the precedence of operators using brackets.
In the query <tt class="docutils literal">one OR two AND three</tt> the AND takes precedence, so this
is the same as <tt class="docutils literal">one OR (two AND three)</tt>. You can override the
precedence using <tt class="docutils literal">(one OR two) AND three</tt>.</p>
<p>The default precedence from highest to lowest is:</p>
<ul class="simple">
<li>SYN</li>
<li>+, - (equal)</li>
<li>NEAR, ADJ (equal)</li>
<li>AND, NOT (equal)</li>
<li>XOR</li>
<li>OR</li>
</ul>
</div>
<div class="section" id="phrase-searches">
<h2>Phrase searches</h2>
<p>A phrase surrounded with double quotes (&quot;&quot;) matches documents containing
that exact phrase. Hyphenated words are also treated as phrases, as are
cases such as filenames and email addresses (e.g. <tt class="docutils literal">/etc/passwd</tt> or
<tt class="docutils literal">president&#64;whitehouse.gov</tt>).</p>
</div>
<div class="section" id="searching-within-a-free-text-field">
<h2>Searching within a free-text field</h2>
<p>If the database has been indexed with prefixes on terms generated from
certain free-text fields, you can set up a prefix map so that the user can
search within those fields. For example <tt class="docutils literal">author:dickens title:shop</tt>
might find documents by dickens with shop in the title. You can also
specify a prefix on a quoted phrase (e.g. <tt class="docutils literal"><span class="pre">author:&quot;charles</span> dickens&quot;</tt>)
or on a bracketed subexpression (e.g. <tt class="docutils literal"><span class="pre">title:(mice</span> men)</tt>).</p>
</div>
<div class="section" id="searching-for-proper-nouns">
<h2>Searching for proper nouns</h2>
<p>If stemming is enabled it can cause problems with some proper nouns in
some languages.  For example, the English stemmer conflates <tt class="docutils literal">Tony</tt> with
<tt class="docutils literal">Toni</tt> and <tt class="docutils literal">Keating</tt> with <tt class="docutils literal">Keats</tt>.  If you want a word to be
searched for unstemmed, you can quote it (like a phrase, but with just
one word).  This only works if unstemmed terms have been indexed (so
only for stem strategies <tt class="docutils literal">STEM_SOME</tt> and <tt class="docutils literal">STEM_SOME_FULL_POS</tt>).</p>
<p>The QueryParser also uses a heuristic which prevents stemming of words which
start with a capital letter.  Since Xapian 2.0.0, this heuristic is only
enabled for some languages as it is unhelpful for languages where proper
nouns are inflected, and also in German where all nouns are capitalised.</p>
</div>
<div class="section" id="range-searches">
<h2>Range searches</h2>
<p>The QueryParser <a class="reference external" href="valueranges.html">can be configured to support
range-searching</a> using document values.</p>
<p>The syntax for a range search is <tt class="docutils literal"><span class="pre">start..end</span></tt> - for example,
<tt class="docutils literal"><span class="pre">01/03/2007..04/04/2007</span></tt>, <tt class="docutils literal"><span class="pre">$10..100</span></tt>, <tt class="docutils literal"><span class="pre">5..10kg</span></tt>.</p>
<p>Open-ended ranges are also supported - an empty start or end is
interpreted as no limit, for example: <tt class="docutils literal"><span class="pre">..2010-06-17</span></tt>, <tt class="docutils literal">$10..</tt>,
<tt class="docutils literal"><span class="pre">$..100</span></tt>, <tt class="docutils literal">..5kg</tt>.</p>
</div>
<div class="section" id="synonyms">
<h2>Synonyms</h2>
<p>The QueryParser can be configured to support synonyms, which can either
be used when explicitly specified (using the syntax <tt class="docutils literal">~term</tt>) or
implicitly (synonyms will be used for all terms or groups of terms for
which they have been specified).</p>
</div>
<div class="section" id="wildcards">
<h2>Wildcards</h2>
<p>The QueryParser supports using wildcards, but this support is not
enabled by default.  Matching wildcard queries is inherently more
work which may be problematic for heavily used search systems.</p>
<p>Prior to Xapian 2.0.0, only a trailing <tt class="docutils literal">*</tt> wildcard was supported.
This matches any number of trailing characters, so <tt class="docutils literal">wildc*</tt> would match
wildcard, wildcarded, wildcards, wildcat, wildcats, etc.  This wildcard
mode is enabled by passing <tt class="docutils literal"><span class="pre">Xapian::QueryParser::FLAG_WILDCARD</span></tt> in the flags
argument of <tt class="docutils literal"><span class="pre">Xapian::QueryParser::parse_query(query_string,</span> flags)</tt>.</p>
<p>(In Xapian 1.2.x you also needed to tell the QueryParser which database to
expand wildcards from using the <tt class="docutils literal"><span class="pre">QueryParser::set_database(database)</span></tt> method.
Since Xapian 1.3.3 wildcards are only expanded when <tt class="docutils literal"><span class="pre">Enquire::get_mset()</span></tt>
is called, and expansion now uses the database being searched.)</p>
<p>Xapian 2.0.0 added an &quot;extended wildcard&quot; feature, which supports
both <tt class="docutils literal">*</tt> (matching zero or more characters) and <tt class="docutils literal">?</tt> (matching
exactly one character).  These can be used anywhere in the term,
and can appear multiple times in a term.  Extended wildcards are
enabled using flag <tt class="docutils literal"><span class="pre">Xapian::QueryParser::FLAG_WILDCARD_GLOB</span></tt>
(or <tt class="docutils literal"><span class="pre">Xapian::QueryParser::FLAG_WILDCARD_MULTI</span></tt> if you only
want to support <tt class="docutils literal">*</tt>, or <tt class="docutils literal"><span class="pre">Xapian::QueryParser::FLAG_WILDCARD_SINGLE</span></tt> if you
only want to support <tt class="docutils literal">?</tt>).  A term cannot consist entirely of wildcards.</p>
<p>You can specify a minimum length for the fixed initial portion in
wildcard pattern with <tt class="docutils literal"><span class="pre">QueryParser::set_min_wildcard_prefix()</span></tt>, for example
to prevent users searching for <tt class="docutils literal">e*</tt> which would expand to thousands of term
and be a fairly slow query.  By default there is no minimum length, so with
extended wildcards users can use wildcards at the start of a term.</p>
<p>You can limit the number of terms a wildcard will expand to by
calling <tt class="docutils literal"><span class="pre">Xapian::QueryParser::set_max_expansion()</span></tt>.  This supports
several different modes, and can also be used to limit expansion
performed via <tt class="docutils literal">FLAG_PARTIAL</tt> - see the API documentation for
details.  By default, there's no limit on wildcard expansion and
<tt class="docutils literal">FLAG_PARTIAL</tt> expands to the most frequent 100 terms.</p>
</div>
<div class="section" id="partially-entered-query-matching">
<h2>Partially entered query matching</h2>
<p>The QueryParser also supports performing a search with a query which has
only been partially entered. This is intended for use with &quot;incremental
search&quot; systems, which don't wait for the user to finish typing their
search before displaying an initial set of results. For example, in such
a system a user would enter a search, and the system would display a new
set of results after each letter, or whenever the user pauses for a
short period of time (or some other similar strategy).</p>
<p>The problem with this kind of search is that the last word in a
partially entered query often has no semantic relation to the completed
word. For example, a search for &quot;dynamic cat&quot; would return a quite
different set of results to a search for &quot;dynamic categorisation&quot;. This
results in the set of results displayed flicking rapidly as each new
character is entered. A much smoother result can be obtained if the
final word is treated as having an implicit terminating wildcard, so
that it matches all words starting with the entered characters - thus,
as each letter is entered, the set of results displayed narrows down to
the desired subject.</p>
<p>A similar effect could be obtained simply by enabling the wildcard
matching option, and appending a &quot;*&quot; character to each query string.
However, this would be confused by searches which ended with punctuation
or other characters.</p>
<p>This feature is disabled by default - pass
<tt class="docutils literal"><span class="pre">Xapian::QueryParser::FLAG_PARTIAL</span></tt> flag in the flags argument of
<tt class="docutils literal"><span class="pre">Xapian::QueryParser::parse_query(query_string,</span> flags)</tt> to enable it,
and tell the QueryParser which database to expand wildcards from using
the <tt class="docutils literal"><span class="pre">QueryParser::set_database(database)</span></tt> method.</p>
</div>
</div>
</div>
</body>
</html>