File: rdfandsparql.xml

package info (click to toggle)
virtuoso-opensource 7.2.5.1%2Bdfsg1-0.3
links: PTS, VCS
area: main
in suites: bookworm
size: 285,240 kB
sloc: ansic: 641,220; sql: 490,413; xml: 269,570; java: 83,893; javascript: 79,900; cpp: 36,927; sh: 31,653; cs: 25,702; php: 12,690; yacc: 10,227; lex: 7,601; makefile: 7,129; jsp: 4,523; awk: 1,697; perl: 1,013; ruby: 1,003; python: 326
file content (28755 lines) | stat: -rw-r--r-- 1,507,387 bytes
parent folder | download | duplicates (2)
<!--
 -
 -  This file is part of the OpenLink Software Virtuoso Open-Source (VOS)
 -  project.
 -
 -  Copyright (C) 1998-2018 OpenLink Software
 -
 -  This project is free software; you can redistribute it and/or modify it
 -  under the terms of the GNU General Public License as published by the
 -  Free Software Foundation; only version 2 of the License, dated June 1991.
 -
 -  This program is distributed in the hope that it will be useful, but
 -  WITHOUT ANY WARRANTY; without even the implied warranty of
 -  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
 -  General Public License for more details.
 -
 -  You should have received a copy of the GNU General Public License along
 -  with this program; if not, write to the Free Software Foundation, Inc.,
 -  51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
 -
-->
<chapter label="rdfandsparql.xml" id="rdfandsparql"><title>RDF Data Access and Data Management</title>
<abstract>
<para>
Starting with version 4.5, Virtuoso provides built-in support for SPARQL, the standard query language for RDF and the semantic web.

Adoption of SPARQL with Virtuoso is effortless, as any  existing SQL client applications and stored procedures can take advantage of SPARQL simply by using it in the place of or inside SQL queries.  Additionally, Virtuoso offers the standard SPARQL protocol to HTTP clients.
From version 5.0.7, Virtuoso can be used  as the RDF store/query processor of the Jena and Sesame RDF frameworks.
</para>

<para>
This chapter discusses Virtuoso's RDF triple storage and query capabilities. 
This discusses storing RDF data as well as mapping existing relational data into RDF for SPARQL access. Numerous  SPARQL language extensions and standard compliance are covered.
</para>

<para>In this chapter SPARQL and SPASQL are used as siblings.</para>

  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtodbcsparql">Virtuoso ODBC RDF extensions for SPASQL</link></listitem>
      <listitem><link linkend="sqlrefgeospatial">Geometry Data Types and Spatial Index Support</link></listitem>
    </itemizedlist>
  </tip>

</abstract>

<sect1 id="rdfdatarepresentation"><title>Data Representation</title>
<para>This section covers how Virtuoso stores RDF triples. The IRI_ID built-in data type is introduced, along with the default table structures used for triple persistency.
These details are mostly hidden from users of RDF, thus this section is not necessary reading for typical use of Virtuoso with RDF.
</para>
<sect2 id="rdfiriidtype"><title>IRI_ID Type</title>
<para>The central notion of RDF is the IRI, or URI, which serves as the globally unique label of named nodes. The subject and predicate of a triple are always IRI's and the object may be an IRI or any other XML Schema scalar data type. In any case, an IRI is always distinct from any instance of any other data type.</para>
<para>Virtuoso supports a native IRI_ID data type, internally an unsigned 32 bit or unsigned 64 bit integer value.
Small databases can use 32 bit values but if database becomes big then the administrator should execute
<function>DB.DBA.RDF_64BIT_UPGRADE</function>() procedure that will switch to 64-bit values. This procedure takes time so if it is known in advance that the database will grow to billions of nodes then it could be convenient to upgrade it while it is empty.
An IRI_ID is never equal to any instance of any other type.</para>
<para>Thus, the object column of a table storing triples can be declared as ANY and IRI values will be distinguishable without recourse to any extra flag and IRI's will naturally occupy their own contiguous segment in the ANY type collation sequence. Indices can be defined over such columns. An IRI_ID is never automatically cast into any other type nor any other type into IRI_ID.</para>
<para>The functions iri_id_num (in i IRI_ID) and iri_id_from_num (in n INT) convert between signed integers and IRI_ID's. The function isiri_id (in i any) returns nonzero if the argument is of type IRI_ID, zero otherwise.</para>
<para>The syntax for an IRI_ID literal is <emphasis>#i&lt;NNN&gt;</emphasis> or <emphasis>#ib&lt;NNN&gt;</emphasis>, where <emphasis>&lt;NNN&gt;</emphasis> is up to 20 decimal digits. <emphasis>#i12345</emphasis> is equal to <emphasis>iri_id_from_num (12345)</emphasis> and <emphasis>#ib12345</emphasis> is equal to <emphasis>iri_id_from_num (12345) + min_64bit_bnode_iri_id ()</emphasis>.</para>	
<para>When received by a SQL client application, the ODBC driver or
interactive SQL will bind an IRI_ID to a character buffer, producing
the <emphasis>#i&lt;NNN&gt;</emphasis> syntax. When passing IRI_ID's from a client, one can pass an
integer and use the iri_id_from_num () function in the statement to
convert server side. A SQL client will normally not be exposed to
IRI_ID's since the SPARQL implementation returns IRI's in their text
form, not as internal id's. These will however be seen if reading the
internal tables directly.</para>
<note><para>Nobody, even DBA, should write directly to internal RDF tables, because some data from that tables are cached in a special way and cache is not automatically updated when content of tables has changed.</para></note>
<para><emphasis>Example</emphasis></para>
<para>The following example demonstrates IRI type usage as Virtuoso PL function parameter:</para>
<programlisting><![CDATA[
SQL>create procedure vs_property_label (in _uri varchar)
  {
    declare res varchar;
    result_names (res);
    for (sparql define input:storage "" select distinct ?graph_rvr_fixed where { graph `iri(?:_uri)` { ?qmv virtrdf:qmGraphRange-rvrFixedValue ?graph_rvr_fixed}})
    do {
      result ("graph_rvr_fixed");
    }
  }
;
Done. -- 0 msec.

SQL>select vs_property_label('http://www.openlinksw.com/schemas/virtrdf#');
res
VARCHAR
_______________________________________________________________________________

http://demo.openlinksw.com/Northwind
http://demo.openlinksw.com/tpch
http://demo.openlinksw.com/tpcd
http://demo.openlinksw.com/bsbm
http://demo.openlinksw.com/tutorial/Northwind
http://demo.openlinksw.com/thalia
http://demo.openlinksw.com/tutorial_view
http://demo.openlinksw.com/ecrm
http://demo.openlinksw.com/sys
http://demo.openlinksw.com/Doc
http://demo.openlinksw.com/informix/stores_demo
http://demo.openlinksw.com/oraclehr
http://demo.openlinksw.com/db2sample
http://demo.openlinksw.com/ingrestut
http://demo.openlinksw.com/sybasepubs2
http://demo.openlinksw.com/MSPetShop#
http://demo.openlinksw.com/oracle#
http://demo.openlinksw.com/progress/isports
http://demo.openlinksw.com/wpl_v
http://demo.openlinksw.com/mw_v
http://demo.openlinksw.com/drupal_v
0

22 Rows. -- 241 msec.
]]></programlisting>
</sect2>
<sect2 id="rdfboxtype"><title>RDF_BOX Type</title>
<para>While strings, numbers, dates and XML entities are &quot;native&quot; SQL datatypes,
RDF literal with non-default type or language have no exact matches among standard SQL types.
Virtuoso introduces a special data type called "RDF_BOX" in order to handle that cases.
Instance of RDF_BOX consists of data type, language, the content (or beginning characters of a long content)
and a possible reference to DB.DBA.RDF_OBJ table if the object is too long to be held in-line in some table or should be outlined for free-text indexing.</para>
<para>Usually applications do not need to access internals of an RDF boxes. This datatype is used in system tables but almost all SPARQL and RDF operations use standard SQL datatypes for arguments and return values.</para>
</sect2>
<sect2 id="rdfquadtables"><title>RDF_QUAD and other tables</title>
<para>The main tables of the default RDF storage system are:</para>
<programlisting>
create table DB.DBA.RDF_QUAD (
  G IRI_ID,
  S IRI_ID,
  P IRI_ID,
  O any,
  primary key (G,S,P,O) );
create bitmap index RDF_QUAD_OGPS on DB.DBA.RDF_QUAD (O, G, P, S);
</programlisting>
<para>Each triple (more correctly, each quad) is represented by one row in RDF_QUAD.
The columns represent the graph, subject, predicate and object.
The IRI_ID type columns reference RDF_IRI, which translates the internal id to the external name of the IRI.
The O column is of type ANY.
If the O value is a non-string SQL scalar, such as a number or date or IRI, it is stored in its native binary representation.
If it is a "very short" string (20 characters or less), it is also stored "as is".
Long strings and RDF literal with non-default type or language are stored as RDF_BOX values.
Instance of rdf_box consists of data type, language, the content (or beginning characters of a long content)
and a possible reference to RDF_OBJ if the object is too long to be held in-line in this table or should be outlined for free-text indexing.
</para>
<programlisting>
create table DB.DBA.RDF_PREFIX (
  RP_NAME varchar primary key,
  RP_ID int not null unique );
create table DB.DBA.RDF_IRI (
  RI_NAME varchar primary key,
  RI_ID IRI_ID not null unique );
</programlisting>
<para>These two tables store a mapping between internal IRI id's and their external string form.
A memory-resident cache contains recently used IRIs to reduce access to this table.
Function id_to_iri (in id IRI_ID) returns the IRI by its ID.
Function iri_to_id (in iri varchar, in may_create_new_id) returns an IRI_ID for given string;
if the string is not used before as an IRI then either NULL is returned or a new ID is allocated, depending on the second argument.
</para>
<programlisting>
create table DB.DBA.RDF_OBJ (
  RO_ID integer primary key,
    RO_VAL varchar,
  RO_LONG long varchar,
  RO_DIGEST any
)
create index RO_VAL on DB.DBA.RDF_OBJ (RO_VAL)
create index RO_DIGEST on DB.DBA.RDF_OBJ (RO_DIGEST)
;
</programlisting>
<para>When an O value of RDF_QUAD is longer than a certain limit or should be free-text indexed, the value is stored in this table.
Depending on the length of the value, it goes into the varchar or the long varchar column.
The RO_ID is contained in rdf_box object that is stored in the O column.
Still, the truncated value of O can be used for determining equality and range matching,
even if &lt; and &gt; of closely matching values need to look at the real string in RDF_OBJ.
When RO_LONG is used to store very long value, RO_VAL contains a simple checksum of the value, to accelerate search for identical values when the table is populated by new values.
</para>
<programlisting>
create table DB.DBA.RDF_DATATYPE (
    RDT_IID IRI_ID not null primary key,
    RDT_TWOBYTE integer not null unique,
    RDT_QNAME varchar );
</programlisting>
<para>The XML Schema data type of a typed string O represented as 2 bytes in the O varchar value. This table maps this into the broader IRI space where the type URI is given an IRI number.</para>
<programlisting>
create table DB.DBA.RDF_LANGUAGE (
    RL_ID varchar not null primary key,
    RL_TWOBYTE integer not null unique );
</programlisting>
<para>The varchar representation of a O which is a string with language has a two byte field for language. This table maps the short integer language id to the real language name such as 'en', 'en-US' or 'x-any'.</para>
<para><emphasis>Note that unlike datatype names, language names are not URIs.</emphasis></para>
<para>A short integer value can be used in both RDF_DATATYPE and RDF_LANGUAGE tables for two different purposes. E.g. an integer 257 is for 'unspecified datatype' as well as for 'unspecified language'.</para>
</sect2>
<sect2 id="rdfsqlmodes"><title>Short, Long and SQL Values</title>
<para>When processing an O, the SPARQL implementation may have it in one of three internal formats, called "valmodes". The below cases apply for strings:</para>
<para>The short format is the format where an O is stored in RDF_QUAD.</para>
<para>The long value is similar to short one but an rdf_box object, that consists of six fields:</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>short integer id of type referencing RDT_TWOBYTE, 257   if the type is not specified,</listitem>
<listitem>the string as inlined in O or as stored in RO_VAL or   RO_LONG,</listitem>
<listitem>the RO_ID if the string is from RDF_OBJ (otherwise   zero),</listitem>
<listitem>the short integer id of language referencing RL_TWOBYTE, 257 if the language is not specified,</listitem>
<listitem>flag whether the stored string value is complete or it is only the beginning that is inlined in O.</listitem>
</itemizedlist>
<para>The SQL value is the string as a narrow string representing the UTF8 encoding of the value, stripped of data type and language tag.</para>
<para>The SQL form of an IRI is the string. The long and short forms are the IRI_ID referencing RU_IRI_ID of RDF_URL.</para>
<para>For all non-string, non-IRI types, the short, long and SQL values are the same SQL scalar of the appropriate native SQL type. A SQL host variable meant to receive an O should be of the ANY type.</para>
<para>The SPARQL implementation will usually translate results to the SQL format before returning them.
Internally, it uses the shortest possible form suited to the operation. For equalities and joining, the
short form is always good. For range comparisons, the long form is needed etc. For arithmetic,
all three forms will do since the arguments are expected to be numbers which are stored as their binary
selves in O, thus the O column unaltered and uncast will do as an argument of arithmetic or numeric
comparison with, say, SQL literal constants.</para>
</sect2>
<sect2 id="rdfsqlsparqlresolve"><title>Programatically resolving DB.DBA.RDF_QUAD.O to SQL</title>
<para>This section describes how to resolve programatically the internal representation of DB.DBA.RDF_QUAD.O to its SQL value.</para>
<para>When operating over RDF_QUAD table directly, in order to transform all values obtained from column
O to the explicit SQL type in a programmatic way, should be used the following hints depending on the case:
</para>
<itemizedlist mark="bullet">
  <listitem>The SQL value can be extracted as <emphasis>__ro2sq(O)</emphasis>.</listitem>
  <listitem>The datatype can be extracted as <emphasis>DB.DBA.RDF_DATATYPE_OF_OBJ (O)</emphasis> if IRI_ID of the type is enough or <emphasis>__ro2sq ( DB.DBA.RDF_DATATYPE_OF_OBJ(O))</emphasis></listitem>
  <listitem>The language can be extracted as <emphasis>DB.DBA.RDF_LANGUAGE_OF_OBJ (O)</emphasis>.</listitem>
</itemizedlist>

<para>It could be helpful to be created an Linked Data View for a custom table with formats rdfdf:default or rdfdf:default-nullable
for columns similar to O, and let SPARQL perform the rest.</para>

<para>To track SPARQL, use the following functions:</para>

<programlisting><![CDATA[
select sparql_to_sql_text ('query text here without a leading SPARQL keyword and trailing semicolon')
]]></programlisting>
<para>or</para>
<programlisting><![CDATA[
string_to_file ('filename.sql', sparql_to_sql_text ('query text'), -2);
]]></programlisting>
<para>So for example to track the following SPARQL query:</para>
<programlisting><![CDATA[
SPARQL define input:storage ""
select distinct ?graph_rvr_fixed
from <http://www.openlinksw.com/schemas/virtrdf#>
where { ?qmv virtrdf:qmGraphRange-rvrFixedValue ?graph_rvr_fixed }
]]></programlisting>
<para>execute</para>
<programlisting><![CDATA[
SQL>select sparql_to_sql_text('define input:storage "" select distinct ?graph_rvr_fixed from <http://www.openlinksw.com/schemas/virtrdf#> where { ?qmv virtrdf:qmGraphRange-rvrFixedValue ?graph_rvr_fixed }');
callret
VARCHAR
_______________________________________________________________________________

 SELECT  __ro2sq ("s-1-0_rbc"."graph_rvr_fixed") AS "graph_rvr_fixed" FROM (SELECT DISTINCT __rdf_sqlval_of_obj ( /*retval[*/ "s-1-1-t0"."O" /* graph_
rvr_fixed */ /*]retval*/ ) AS /*tmpl*/ "graph_rvr_fixed"
    FROM DB.DBA.RDF_QUAD AS "s-1-1-t0"
    WHERE /* field equal to URI ref */
        "s-1-1-t0"."G" = __i2idn ( /* UNAME as sqlval */ __box_flags_tweak ( 'http://www.openlinksw.com/schemas/virtrdf#' , 1))
        AND  /* field equal to URI ref */
        "s-1-1-t0"."P" = __i2idn ( /* UNAME as sqlval */ __box_flags_tweak ( 'http://www.openlinksw.com/schemas/virtrdf#qmGraphRange-rvrFixedValue' ,
1))
OPTION (QUIETCAST)) AS "s-1-0_rbc"

1 Rows. -- 321 msec.
]]></programlisting>
<para>or</para>
<programlisting><![CDATA[
SQL>string_to_file ('mytest.sql', sparql_to_sql_text ('define input:storage "" select distinct ?graph_rvr_fixed from <http://www.openlinksw.com/schemas/virtrdf#> where { ?qmv virtrdf:qmGraphRange-rvrFixedValue ?graph_rvr_fixed }'), -2);
]]></programlisting>
<para>As result will be created file with the given name, i.e. mytest.sql and its content should be:</para>
<programlisting><![CDATA[
 SELECT  __ro2sq ("s-1-0_rbc"."graph_rvr_fixed") AS "graph_rvr_fixed" FROM (SELECT DISTINCT __rdf_sqlval_of_obj ( /*retval[*/ "s-1-1-t0"."O" /* graph_rvr_fixed */ /*]retval*/ ) AS /*tmpl*/ "graph_rvr_fixed"
    FROM DB.DBA.RDF_QUAD AS "s-1-1-t0"
    WHERE /* field equal to URI ref */
        "s-1-1-t0"."G" = __i2idn ( /* UNAME as sqlval */ __box_flags_tweak ( 'http://www.openlinksw.com/schemas/virtrdf#' , 1))
        AND  /* field equal to URI ref */
        "s-1-1-t0"."P" = __i2idn ( /* UNAME as sqlval */ __box_flags_tweak ( 'http://www.openlinksw.com/schemas/virtrdf#qmGraphRange-rvrFixedValue' , 1))
OPTION (QUIETCAST)) AS "s-1-0_rbc"
]]></programlisting>
</sect2>
<sect2 id="rdfxmlschemacompat"><title>Special Cases and XML Schema Compatibility</title>
<para>We note that since we store numbers as the equivalent SQL binary type, we do not preserve the distinction of byte, boolean etc. These all become integer. If preserving such detail is for some reason important, then storage as a typed string is possible but is not done at present for reasons of compactness and performance.</para>
</sect2>
<sect2 id="rdfquietcast"><title>SQL Compiler Support - QUIETCAST option</title>
<para>The type cast behaviors of SQL and SPARQL are different. SQL will generally signal an error when an automatic cast fails. For example, a string can be compared to a date column if the string can be parsed as a date but otherwise the comparison should signal an error. In SPARQL, such situations are supposed to silently fail. Generally, SPARQL is much more relaxed with respect to data types.</para>
<para>These differences will be specially noticed if actual SQL data is processed with SPARQL via some sort of schema mapping translating references to triples into native tables and columns.</para>
<para>Also, even when dealing with the triple-oriented RDF_QUAD table, there are cases of joining between S and O such that the O can be a heterogeneous set of IRI's and other data whereas the S is always an IRI. The non-IRI to IRI comparison should not give cast errors but should silently fail. Also, in order to keep queries simple and easily optimizable, it should not be necessary to introduce extra predicates for testing if the O is n IRI before comparing with the S.</para>
<para>Due to these considerations, Virtuoso introduces a SQL statement option called QUIETCAST. When given in the OPTION clause of a SELECT, it switches to silent fail mode for automatic type casting.</para>
<para>The syntax is as follows:</para>
<programlisting>
SELECT ...
FROM .... OPTION (QUIETCAST)
</programlisting>
<para>This option is automatically added by the SPARQL to SQL translator. The scope is the enclosing procedure body.</para>
</sect2>
<sect2 id="rdfdynamiclocal"><title>Dynamic Renaming of Local IRI's</title>
<para>
    There are cases where it is desirable to have IRI's in RDF storage
    that will change to reflect a change of the host name of the containing
    store.  This is specifically true of DAV resource metadata for local
    DAV resources.
    Such IRI's must be stored prefixed with <computeroutput>local:</computeroutput>.  
</para>
<para>
    If a user application makes statements with such a URI, then these statements will be returned with local:
    substituted with a prefix taken from the context as described below.
</para>
<para>
    When returning IRI's from id's, this prefix is replaced by the Host header of the HTTP request 
    and if not running with HTTP, with the DefaultHost from URIQA.  This behavior is always in effect.
</para>

<para>
    When converting strings to IRI id's, the <computeroutput>local:</computeroutput> prefix may or may not be introduced depending on ini file and other context factors.
    If <link linkend="VIRTINI">DynamicLocal</link> defined in the [URIQA] section of the Virtuoso INI file is on and the host part of the IRI matches the Host header of the HTTP request in context or the DefaultHost if outside of HTTP context, then this is replaced with local: before looking up the IRI ID.  Even if DynamicLocal is not on and the <computeroutput>local:</computeroutput> prefix occurs in the IRI string being translated to id, the translating the IRI_ID back to the IRI name will depend on the context as described above.
</para>
<para>
    The effects of DynamicLocal = 1 can be very confusing since many names
    can refer to the exact same thing.  For example, if the DefaultHost is
    dbpedia.org,
    <computeroutput>iri_to_id ('http://dbpedia.org/resource/Paris') = iri_to_id ('local:///resource/Paris) </computeroutput>
    is true and so is  
    <computeroutput>'http://dbpedia.org/resource/Paris' = id_to_iri (iri_to_id ('local://resource/Paris'))</computeroutput>
    These hold in a SQL client context, i.e. also when connected through RDF frameworks like Jena or Sesame.
    When running a SPARQL protocol request, the Host: header influences the behavior, likewise when using web interactive SQL in Conductor.
    Also be careful when loading RDF files that may have URI's corresponding to the local host name.
</para>
</sect2>
</sect1>

<sect1 id="rdfsparql"><title>SPARQL</title>
<sect2 id="rdfsparqlimplementationextent"><title>SPARQL Implementation Details</title>
<para>Virtuoso's RDF support includes in-built support for the SPARQL query language. 
	It also includes a number of powerful extensions that cover path traversal and business 
intelligence features. In addition, there is in-built security based on Virtuoso's support 
for row level policy-based security, custom authentication, and named graphs.</para>
<para>The current implementation does not support some SPARQL features:</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>Unicode characters in names are not supported.</listitem>
<listitem>Comments inside SPARQL queries are not supported when the query is inlined in SQL code.</listitem>
</itemizedlist>
<para>On the other hand, Virtuoso implements some extensions to SPARQL:</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>SPARUL statements, such as <emphasis>insert</emphasis>, <emphasis>modify</emphasis>, <emphasis>load</emphasis> etc, are supported.</listitem>
<listitem>The SPARQL compiler can be configured using <emphasis>define ...</emphasis> clauses, e.g. <emphasis>define output:valmode "LONG"</emphasis>.</listitem>
<listitem>Expressions are allowed in triple patterns, both in a <emphasis>where</emphasis> clause and in constructor patterns. Such expressions are delimited by backquotes.</listitem>
<listitem>Expressions are allowed in select statement result lists.</listitem>
<listitem>Parameters can be passed to the query from outside, using <emphasis>?:variablename</emphasis> syntax.</listitem>
<listitem>Aggregate functions are supported.</listitem>
<listitem>Subqueries may appear where group patterns are allowed.</listitem>
<listitem>A set of operators has been added to configure the mapping of relational data to RDF (aka Linked Data Views).</listitem>
</itemizedlist>
<para>The following listing shows the SPARQL grammar expressed in BNF, including all Virtuoso extensions but excluding rules for the syntax of each lexical element. 
Rule numbers in square brackets are from W3C normative SPARQL grammar. An asterisk indicates that the rule differs from the W3C grammar due to Virtuoso extensions -
<emphasis>[Virt]</emphasis> means that the rule is Virtuoso-specific, 
<emphasis>[DML]</emphasis> indicates a data manipulation language extension from SPARUL.
</para>
<programlisting><![CDATA[
[1]*	Query		 ::=  Prolog ( QueryBody | SparulAction* | ( QmStmt ('.' QmStmt)* '.'? ) )
[1]	QueryBody	 ::=  SelectQuery | ConstructQuery | DescribeQuery | AskQuery
[2]*	Prolog		 ::=  Define* BaseDecl? PrefixDecl*
[Virt]	Define		 ::=  'DEFINE' QNAME (QNAME | Q_IRI_REF | String )
[3]	BaseDecl	 ::=  'BASE' Q_IRI_REF
[4]	PrefixDecl	 ::=  'PREFIX' QNAME_NS Q_IRI_REF
[5]*	SelectQuery	 ::=  'SELECT' 'DISTINCT'? ( ( Retcol ( ','? Retcol )* ) | '*' )
			DatasetClause* WhereClause SolutionModifier
[6]	ConstructQuery	 ::=  'CONSTRUCT' ConstructTemplate DatasetClause* WhereClause SolutionModifier
			DatasetClause* WhereClause? SolutionModifier
[8]	AskQuery	 ::=  'ASK' DatasetClause* WhereClause
[9]	DatasetClause	 ::=  'FROM' ( DefaultGraphClause | NamedGraphClause )
[10]*	DefaultGraphClause	 ::=  SourceSelector SpongeOptionList?
[11]*	NamedGraphClause	 ::=  'NAMED' SourceSelector SpongeOptionList?
[Virt]	SpongeOptionList	 ::=  'OPTION' '(' ( SpongeOption ( ',' SpongeOption )* )? ')'
[Virt]	SpongeOption	 ::=  QNAME PrecodeExpn
[Virt]	PrecodeExpn	 ::=  Expn	(* Only global variables can occur in Expn, local cannot *)
[13]	WhereClause	 ::=  'WHERE'? GroupGraphPattern
[14]	SolutionModifier	 ::=  OrderClause?
			((LimitClause OffsetClause?) | (OffsetClause LimitClause?))?
[15]	OrderClause	 ::=  'ORDER' 'BY' OrderCondition+
[16]*	OrderCondition	 ::=  ( 'ASC' | 'DESC' )?
			( FunctionCall | Var | ( '(' Expn ')' ) | ( '[' Expn ']' ) )
[17]	LimitClause	 ::=  'LIMIT' INTEGER
[17]	LimitClause	 ::=  'LIMIT' INTEGER
[18]	OffsetClause	 ::=  'OFFSET' INTEGER
[18]	OffsetClause	 ::=  'OFFSET' INTEGER
[19]*	GroupGraphPattern	 ::=  '{' ( GraphPattern | SelectQuery ) '}'
[20]	GraphPattern	 ::=  Triples? ( GraphPatternNotTriples '.'? GraphPattern )?
[21]*	GraphPatternNotTriples	 ::=
			QuadMapGraphPattern
			| OptionalGraphPattern
			| GroupOrUnionGraphPattern
			| GraphGraphPattern
			| Constraint
[22]	OptionalGraphPattern	 ::=  'OPTIONAL' GroupGraphPattern
[Virt]	QuadMapGraphPattern	 ::=  'QUAD' 'MAP' ( IRIref | '*' ) GroupGraphPattern
[23]	GraphGraphPattern	 ::=  'GRAPH' VarOrBlankNodeOrIRIref GroupGraphPattern
[24]	GroupOrUnionGraphPattern	 ::=  GroupGraphPattern ( 'UNION' GroupGraphPattern )*
[25]*	Constraint	 ::=  'FILTER' ( ( '(' Expn ')' ) | BuiltInCall | FunctionCall )
[26]*	ConstructTemplate	 ::=  '{' ConstructTriples '}'
[27]	ConstructTriples	 ::=  ( Triples1 ( '.' ConstructTriples )? )?
[28]	Triples		 ::=  Triples1 ( '.' Triples? )?
[29]	Triples1	 ::=  VarOrTerm PropertyListNotEmpty | TriplesNode PropertyList
[30]	PropertyList	 ::=  PropertyListNotEmpty?
[31]	PropertyListNotEmpty	 ::=  Verb ObjectList ( ';' PropertyList )?
[32]*	ObjectList	 ::=  ObjGraphNode ( ',' ObjectList )?
[Virt]	ObjGraphNode	 ::=  GraphNode TripleOptions?
[Virt]	TripleOptions	 ::=  'OPTION' '(' TripleOption ( ',' TripleOption )? ')'
[Virt]	TripleOption	 ::=  'INFERENCE' ( QNAME | Q_IRI_REF | SPARQL_STRING )
[33]	Verb		 ::=  VarOrBlankNodeOrIRIref | 'a'
[34]	TriplesNode	 ::=  Collection | BlankNodePropertyList
[35]	BlankNodePropertyList	 ::=  '[' PropertyListNotEmpty ']'
[36]	Collection	 ::=  '(' GraphNode* ')'
[37]	GraphNode	 ::=  VarOrTerm | TriplesNode
[38]	VarOrTerm	 ::=  Var | GraphTerm
[39]*	VarOrIRIrefOrBackquoted	 ::=  Var | IRIref | Backquoted
[40]*	VarOrBlankNodeOrIRIrefOrBackquoted	 ::=  Var | BlankNode | IRIref | Backquoted
[Virt]	Retcol	 ::=  ( Var | ( '(' Expn ')' ) | RetAggCall ) ( 'AS' ( VAR1 | VAR2 ) )?
[Virt]	RetAggCall	 ::=  AggName '(', ( '*' | ( 'DISTINCT'? Var ) ) ')'
[Virt]	AggName	 ::=  'COUNT' | 'AVG' | 'MIN' | 'MAX' | 'SUM'
[41]*	Var	 ::=  VAR1 | VAR2 | GlobalVar | ( Var ( '+>' | '*>' ) IRIref )
[Virt]	GlobalVar	 ::=  QUEST_COLON_PARAMNAME | DOLLAR_COLON_PARAMNAME
			| QUEST_COLON_PARAMNUM | DOLLAR_COLON_PARAMNUM
[42]*	GraphTerm	 ::=  IRIref | RDFLiteral | ( '-' | '+' )? NumericLiteral
			| BooleanLiteral | BlankNode | NIL | Backquoted
[Virt]	Backquoted	 ::=  '`' Expn '`'
[43]	Expn		 ::=  ConditionalOrExpn
[44]	ConditionalOrExpn	 ::=  ConditionalAndExpn ( '||' ConditionalAndExpn )*
[45]	ConditionalAndExpn	 ::=  ValueLogical ( '&&' ValueLogical )*
[46]	ValueLogical	 ::=  RelationalExpn
[47]*	RelationalExpn	 ::=  NumericExpn
			( ( ('='|'!='|'<'|'>'|'<='|'>='|'LIKE') NumericExpn )
			| ( 'IN' '(' Expns ')' ) )?
[49]	AdditiveExpn	 ::=  MultiplicativeExpn ( ('+'|'-') MultiplicativeExpn )*
[50]	MultiplicativeExpn	 ::=  UnaryExpn ( ('*'|'/') UnaryExpn )*
[51]	UnaryExpn	 ::=   ('!'|'+'|'-')? PrimaryExpn
[58]	PrimaryExpn	 ::=
			BracketedExpn | BuiltInCall | IRIrefOrFunction
			| RDFLiteral | NumericLiteral | BooleanLiteral | BlankNode | Var
[55]	IRIrefOrFunction	 ::=  IRIref ArgList?
[52]*	BuiltInCall	 ::=
			( 'STR' '(' Expn ')' )
			| ( 'IRI' '(' Expn ')' )
			| ( 'LANG' '(' Expn ')' )
			| ( 'LANGMATCHES' '(' Expn ',' Expn ')' )
			| ( 'DATATYPE' '(' Expn ')' )
			| ( 'BOUND' '(' Var ')' )
			| ( 'sameTERM' '(' Expn ',' Expn ')' )
			| ( 'isIRI' '(' Expn ')' )
			| ( 'isURI' '(' Expn ')' )
			| ( 'isBLANK' '(' Expn ')' )
			| ( 'isLITERAL' '(' Expn ')' )
			| RegexExpn
[53]	RegexExpn	 ::=  'REGEX' '(' Expn ',' Expn ( ',' Expn )? ')'
[54]	FunctionCall	 ::=  IRIref ArgList
[56]*	ArgList	 ::=  ( NIL | '(' Expns ')' )
[Virt]	Expns	 ::=  Expn ( ',' Expn )*
[59]	NumericLiteral	 ::=  INTEGER | DECIMAL | DOUBLE
[60]	RDFLiteral	 ::=  String ( LANGTAG | ( '^^' IRIref ) )?
[61]	BooleanLiteral	 ::=  'true' | 'false'
[63]	IRIref		 ::=  Q_IRI_REF | QName
[64]	QName		 ::=  QNAME | QNAME_NS
[65]*	BlankNode	 ::=  BLANK_NODE_LABEL | ( '[' ']' )
[DML]	SparulAction	 ::=
			CreateAction | DropAction | LoadAction
			| InsertAction | InsertDataAction | DeleteAction | DeleteDataAction
			| ModifyAction | ClearAction
[DML]*	InsertAction	 ::=
			'INSERT' ( ( 'IN' | 'INTO ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn
			ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )?
[DML]*	InsertDataAction	 ::=
			'INSERT' 'DATA' ( ( 'IN' | 'INTO ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? )?
			PrecodeExpn ConstructTemplate
[DML]*	DeleteAction	 ::=
			'DELETE' ( 'FROM' 'GRAPH' ( 'IDENTIFIED' 'BY' )? )? PrecodeExpn
			ConstructTemplate ( DatasetClause* WhereClause SolutionModifier )?
[DML]*	DeleteDataAction	 ::=
			'DELETE' 'DATA' ( 'FROM' 'GRAPH' ( 'IDENTIFIED' 'BY' )? )?
			PrecodeExpn ConstructTemplate
[DML]*	ModifyAction	 ::=
			'MODIFY' ( 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn?
			'DELETE' ConstructTemplate 'INSERT' ConstructTemplate
			( DatasetClause* WhereClause SolutionModifier )?
[DML]*	ClearAction	 ::=  'CLEAR' ( 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn )?
[DML]*	LoadAction	 ::=  'LOAD' PrecodeExpn
			( ( 'IN' | 'INTO' ) 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn )?
[DML]*	CreateAction	 ::=  'CREATE' 'SILENT'? 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn
[DML]*	DropAction	 ::=  'DROP' 'SILENT'? 'GRAPH' ( 'IDENTIFIED' 'BY' )? PrecodeExpn
[Virt]	QmStmt		 ::=  QmSimpleStmt | QmCreateStorage | QmAlterStorage
[Virt]	QmSimpleStmt	 ::=
			QmCreateIRIClass | QmCreateLiteralClass | QmDropIRIClass | QmDropLiteralClass
			| QmCreateIRISubclass | QmDropQuadStorage | QmDropQuadMap
[Virt]	QmCreateIRIClass	 ::=  'CREATE' 'IRI' 'CLASS' QmIRIrefConst
			( ( String QmSqlfuncArglist )
			| ( 'USING' QmSqlfuncHeader ',' QmSqlfuncHeader ) )
[Virt]	QmCreateLiteralClass	 ::=  'CREATE' 'LITERAL' 'CLASS' QmIRIrefConst
			'USING' QmSqlfuncHeader ',' QmSqlfuncHeader QmLiteralClassOptions?
[Virt]	QmDropIRIClass	 ::=  'DROP' 'IRI' 'CLASS' QmIRIrefConst
[Virt]	QmDropLiteralClass	 ::=  'DROP' 'LITERAL' 'CLASS' QmIRIrefConst
[Virt]	QmCreateIRISubclass	 ::=  'IRI' 'CLASS' QmIRIrefConst 'SUBCLASS' 'OF' QmIRIrefConst
[Virt]	QmIRIClassOptions	 ::=  'OPTION' '(' QmIRIClassOption (',' QmIRIClassOption)* ')'
[Virt]	QmIRIClassOption	 ::=
			'BIJECTION'
			| 'DEREF'
			| 'RETURNS' STRING ('UNION' STRING)*
[Virt]	QmLiteralClassOptions	 ::=  'OPTION' '(' QmLiteralClassOption (',' QmLiteralClassOption)* ')'
[Virt]	QmLiteralClassOption	 ::=
			( 'DATATYPE' QmIRIrefConst )
			| ( 'LANG' STRING )
			| ( 'LANG' STRING )
			| 'BIJECTION'
			| 'DEREF'
			| 'RETURNS' STRING ('UNION' STRING)*
[Virt]	QmCreateStorage	 ::=  'CREATE' 'QUAD' 'STORAGE' QmIRIrefConst QmSourceDecl* QmMapTopGroup
[Virt]	QmAlterStorage	 ::=  'ALTER' 'QUAD' 'STORAGE' QmIRIrefConst QmSourceDecl* QmMapTopGroup
[Virt]	QmDropStorage	 ::=  'DROP' 'QUAD' 'STORAGE' QmIRIrefConst
[Virt]	QmDropQuadMap	 ::=  'DROP' 'QUAD' 'MAP' 'GRAPH'? QmIRIrefConst
[Virt]	QmDrop	 ::=  'DROP' 'GRAPH'? QmIRIrefConst
[Virt]	QmSourceDecl	 ::=
			( 'FROM' QTABLE 'AS' PLAIN_ID QmTextLiteral* )
			| ( 'FROM' PLAIN_ID 'AS' PLAIN_ID QmTextLiteral* )
			| QmCondition
[Virt]	QmTextLiteral	 ::=  'TEXT' 'XML'? 'LITERAL' QmSqlCol ( 'OF' QmSqlCol )? QmTextLiteralOptions?
[Virt]	QmTextLiteralOptions	 ::=  'OPTION' '(' QmTextLiteralOption ( ',' QmTextLiteralOption )* ')'
[Virt]	QmMapTopGroup	 ::=  '{' QmMapTopOp ( '.' QmMapTopOp )* '.'? '}'
[Virt]	QmMapTopOp	 ::=  QmMapOp | QmDropQuadMap | QmDrop
[Virt]	QmMapGroup	 ::=  '{' QmMapOp ( '.' QmMapOp )* '.'? '}'
[Virt]	QmMapOp		 ::=
			( 'CREATE' QmIRIrefConst 'AS' QmMapIdDef )
			| ( 'CREATE' 'GRAPH'? QmIRIrefConst 'USING' 'STORAGE' QmIRIrefConst QmOptions? )
			| ( QmNamedField+ QmOptions? QmMapGroup )
			| QmTriples1
[Virt]	QmMapIdDef	 ::=  QmMapTriple | ( QmNamedField+ QmOptions? QmMapGroup )
[Virt]	QmMapTriple	 ::=  QmFieldOrBlank QmVerb QmObjField
[Virt]	QmTriples1	 ::=  QmFieldOrBlank QmProps
[Virt]	QmNamedField	 ::=  ('GRAPH'|'SUBJECT'|'PREDICATE'|'OBJECT') QmField
[Virt]	QmProps		 ::=  QmProp ( ';' QmProp )?
[Virt]	QmProp		 ::=  QmVerb QmObjField ( ',' QmObjField )*
[Virt]	QmObjField	 ::=  QmFieldOrBlank QmCondition* QmOptions?
[Virt]	QmIdSuffix	 ::=  'AS' QmIRIrefConst
[Virt]	QmVerb		 ::=  QmField | ( '[' ']' ) | 'a'
[Virt]	QmFieldOrBlank	 ::=  QmField | ( '[' ']' )
[Virt]	QmField		 ::=
			NumericLiteral
			| RdfLiteral
			| ( QmIRIrefConst ( '(' ( QmSqlCol ( ',' QmSqlCol )* )? ')' )? )
			| QmSqlCol
[Virt]	QmCondition	 ::=  'WHERE' ( ( '(' SQLTEXT ')' ) | String )
[Virt]	QmOptions	 ::=  'OPTION' '(' QmOption ( ',' QmOption )* ')'
[Virt]	QmOption	 ::=  ( 'SOFT'? 'EXCLUSIVE' ) | ( 'ORDER' INTEGER ) | ( 'USING' PLAIN_ID )
[Virt]	QmSqlfuncHeader	 ::=  'FUNCTION' SQL_QTABLECOLNAME QmSqlfuncArglist 'RETURNS' QmSqltype
[Virt]	QmSqlfuncArglist	 ::=  '(' ( QmSqlfuncArg ( ',' QmSqlfuncArg )* )? ')'
[Virt]	QmSqlfuncArg	 ::=  ('IN' | QmSqlId) QmSqlId QmSqltype
[Virt]	QmSqltype	 ::=  QmSqlId ( 'NOT' 'NULL' )?
[Virt]	QmSqlCol	 ::=  QmSqlId | spar_qm_sql_id
[Virt]	QmSqlId		 ::=  PLAIN_ID | 'TEXT' | 'XML'
[Virt]	QmIRIrefConst	 ::=  IRIref | ( 'IRI' '(' String ')' )
]]></programlisting>
<para><emphasis>Example: Using OFFSET and LIMIT</emphasis></para>
<para>
Virtuoso uses a zero-based index for OFFSET. Thus, in the example below, the query returns 
1000 rows starting from, and including, record 9001 of the result set. Note that the
default value of the MaxSortedTopRows parameter in the [Parameters] section of the virtuoso.ini configuration 
file defaults to 10000, so in this example its value will need to have been increased beforehand.
</para>
<programlisting><![CDATA[
SQL>SELECT ?name
ORDER BY ?name
OFFSET 9000
LIMIT 1000
]]></programlisting>
<para>LIMIT applies to the solution resulting from the graph patterns specified in the WHERE
CLAUSE. This implies that SELECT and CONSTRUCT/DESCRIBE queries will
behave a little differently. In the case of a SELECT, there is a straight
translation i.e. LIMIT 4 implies 4 records in the result set. In the
case of CONSTRUCTs where the solution is a graph (implying that the
existence of duplicates and/or unbound variables is common) LIMIT is
basically a maximum triples threshold of: [Solution Triples] x [LIMIT].
</para>
<para>
Example query:
</para>
<programlisting><![CDATA[
SQL>SPARQL
prefix dct:<http://purl.org/dc/terms/>
prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#>

CONSTRUCT { ?resource dct:title ?title ;
                      a ?type }

FROM <http://msone.computas.no/graphs/inferred/classification>
FROM <http://msone.computas.no/graphs>
FROM <http://msone.computas.no/graphs/instance/nfi>
FROM <http://msone.computas.no/graphs/instance/mo>
FROM <http://msone.computas.no/graphs/ontology/mediasone>
FROM <http://msone.computas.no/graphs/vocab/mediasone>
FROM <http://msone.computas.no/graphs/inferred/nfi/realisation1>
FROM <http://msone.computas.no/graphs/inferred/mo/realisation1>
FROM <http://msone.computas.no/graphs/inferred/nfi/realisation2>
FROM <http://msone.computas.no/graphs/inferred/mo/realisation2>
FROM <http://msone.computas.no/graphs/inferred/agent-classification>
FROM <http://msone.computas.no/graphs/ontology/mediasone/agent>

WHERE {
  {
?resource a ?type .
  FILTER (?type = <http://www.w3.org/2002/07/owl#Class> ) .
  ?resource rdfs:label ?title .
  } UNION {
?resource a ?type .
  FILTER (?type in (
          <http://musicbrainz.org/mm/mm-2.1#Track> ,
          <http://www.csd.abdn.ac.uk/~ggrimnes/dev/imdb/IMDB#Movie> ,
          <http://xmlns.com/foaf/0.1/Image> ,
          <http://www.computas.com/mediasone#Text> ) ) .
  ?resource dct:title ?title .
  }
  FILTER regex(?title, "turi", "i")
}
ORDER BY ?title LIMIT 4 OFFSET 0
]]></programlisting>
<para><emphasis>Example: Prevent Limits of Sorted LIMIT/OFFSET query</emphasis></para>
<para>The DBpedia SPARQL endpoint is configured with the following INI setting:</para>
<programlisting><![CDATA[
MaxSortedTopRows = 40000
]]></programlisting>
<para>The setting above sets a threshold for sorted rows. Thus, when using basic 
	SPARQL queries that include OFFSET and LIMIT the following query will still exist 
	the hard limit set in the INI:</para>
<programlisting><![CDATA[
DEFINE sql:big-data-const 0 
SELECT DISTINCT  ?p ?s
FROM <http://dbpedia.org>
WHERE
  { 
    ?s ?p <http://dbpedia.org/resource/Germany> 
  }
ORDER BY ASC(?p)
OFFSET  40000
LIMIT   1000
]]></programlisting>
<para>returns the following error on execution:</para>
<programlisting><![CDATA[
HttpException: 500 SPARQL Request Failed

Virtuoso 22023 Error SR353: Sorted TOP clause specifies more then 41000 rows to sort. 
Only 40000 are allowed. 
Either decrease the offset and/or row count or use a scrollable cursor	
]]></programlisting>
<para>To prevent the problem outlined above you can leverage the use of subqueries which make 
	better use of temporary storage associated with this kind of quest. An example would take the form:</para>
<programlisting><![CDATA[
SELECT ?p ?s 
WHERE 
  {
    {
      SELECT DISTINCT ?p ?s 
      FROM <http://dbpedia.org> 
      WHERE   
        { 
          ?s ?p <http://dbpedia.org/resource/Germany> 
        } ORDER BY ASC(?p) 
    }
  } 
OFFSET 50000 
LIMIT 1000	
]]></programlisting>
<sect3 id="rdfsparqlandxquery"><title>SPARQL and XQuery Core Function Library</title>
<para>In the current implementation, the XQuery Core Function Library is not available from SPARQL.</para>
<para>As a temporary workaround, string parsing functions are made available, because they are widely used in W3C DAWG examples and the like. They are:</para>
<programlisting>
xsd:boolean (in strg any) returns integer
xsd:dateTime (in strg any) returns datetime
xsd:double (in strg varchar) returns double precision
xsd:float (in strg varchar) returns float
xsd:integer (in strg varchar) returns integer
</programlisting>
<para>(assuming that the query contains the declaration: 'PREFIX xsd: &lt;http://www.w3.org/2001/XMLSchema#&gt;')</para>
</sect3>
</sect2>
<sect2 id="rdfpredicatessparql"><title>Query Constructs</title>
  <para>Starting from Version 5.0, Virtuoso supports filtering RDF objects triples by a given predicate.</para>
  <sect3 id="rdfpredicatessparqlexamples"><title>Examples</title>
    <para>The boolean functions bif:contains, bif:xcontains, bif:xpath_contains and bif:xquery_contains 
    	can be used for objects that come from Linked Data Views as well
as for regular "physical" triples. Each of these functions takes two arguments and returns a boolean value.
The first argument is a local variable which should also be used as an object field in
the group pattern where the filter condition is placed.
    </para>
    <para>In order to execute the examples below please run these commands:</para>
<programlisting><![CDATA[
SQL>SPARQL CLEAR GRAPH <http://MyTest.com>;
DB.DBA.RDF_QUAD_URI_L ('http://MyTest.com', 'sxml1', 'p_all1', xtree_doc ('<Hello>world</Hello>'));
DB.DBA.RDF_QUAD_URI_L ('http://MyTest.com', 'sxml2', 'p_all2', xtree_doc ('<Hello2>world</Hello2>'));
DB.DBA.RDF_QUAD_URI_L ('http://MyTest.com', 'nonxml1', 'p_all3', 'Hello world');
VT_INC_INDEX_DB_DBA_RDF_OBJ();
DB.DBA.RDF_OBJ_FT_RULE_ADD ('http://MyTest.com', null, 'My test RDF Data');
]]></programlisting>
    <para><emphasis>bif:contains</emphasis></para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://MyTest.com>
WHERE { ?s ?p ?o . ?o bif:contains "world" };

s             p         o
VARCHAR       VARCHAR   VARCHAR
_______________________________________________________________________________

sxml1         p_all1    <Hello>world</Hello>
nonxml1       p_all3    Hello world
sxml2         p_all2    <Hello2>world</Hello2>

3 Rows. -- 20 msec.
]]></programlisting>
<para><emphasis>bif:xcontains</emphasis></para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://MyTest.com>
WHERE { ?s ?p ?o . ?o bif:xcontains "//Hello[text-contains (., 'world')]" };
s                  p          o
VARCHAR            VARCHAR    VARCHAR
_______________________________________________________________________________

sxml1              p_all      <Hello>world</Hello>

1 Rows. -- 10 msec.
]]></programlisting>
<para><emphasis>bif:xpath_contains</emphasis></para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://MyTest.com>
WHERE { ?s ?p ?o . ?o bif:xpath_contains "//*" };

s             p         o
VARCHAR       VARCHAR   VARCHAR
_______________________________________________________________________________

sxml1         p_all1    <Hello>world</Hello>
sxml2         p_all2    <Hello2>world</Hello2>

2 Rows. -- 20 msec.
]]></programlisting>
<para><emphasis>bif:xquery_contains</emphasis></para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://MyTest.com>
WHERE { ?s ?p ?o . ?o bif:xquery_contains "//Hello2 , world" };

s             p         o
VARCHAR       VARCHAR   VARCHAR
_______________________________________________________________________________

sxml2         p_all2    <Hello2>world</Hello2>

1 Rows. -- 20 msec.
]]></programlisting>
  </sect3>
</sect2>
<sect2 id="rdfsparqlprotocolendpoint"><title>SPARQL Web Services &amp; APIs</title>
<sect3 id="rdfsparqlprotocolendpointintro"><title>Introduction</title>
<para>The Virtuoso SPARQL query service implements the <ulink url="http://www.w3.org/TR/rdf-sparql-protocol/">SPARQL Protocol for RDF</ulink>
(W3C Working Draft 25 January 2006) providing SPARQL query processing for RDF data available on the open internet.</para>
<para>The query processor extends the standard protocol to provide support for multiple output formats.
At present this uses additional query parameters.</para>
<para>Supported features include:</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>Support for GET and POST requests</listitem>
<listitem>Support for a variety of transfer MIME-types, including RDF /XML and TURTLE</listitem>
<listitem>Support for: <emphasis>default-graph-uri</emphasis>, <emphasis>named-graph-uri</emphasis>,
<emphasis>using-graph-uri</emphasis>, <emphasis>using-named-graph-uri</emphasis>, <emphasis>format</emphasis>
 parameters</listitem>
 <listitem>Support for: <emphasis>debug</emphasis>, <emphasis>timeout</emphasis>,
<emphasis>maxrows</emphasis>, <emphasis>qtxt</emphasis>, <emphasis>query</emphasis> parameters</listitem>
<listitem>Support for a variety of query types including CONSTRUCT, ASK, DESCRIBE</listitem>
</itemizedlist>
<para>Virtuoso also supports /sparql-graph-crud/ web service endpoint that
implements the current draft of <ulink url="http://www.w3.org/TR/sparql11-http-rdf-update/">W3C SPARQL Graph Update protocol</ulink>. 
Both /sparql /sparql-graph-crud/ endpoints use the same SPARQL user account, 
so this user should be member of SPARQL_UPDATE group in order to modify data via Graph Update protocol.
Note that /sparql/ endpoint has /sparql-auth/ variant that uses web authentication.
Similarly, /sparql-graph-crud/ has /sparql-graph-crud-auth/ variant. As
soon as user is member of SPARQL_UPDATE group, she/he can modify the stored
data via /sparql-graph-crud-auth/ as well as via /sparql-auth/ .

The /sparql-graph-crud/ endpoint is primarily for serving requests from applications, not for manual interactions via browser.

See more information in our <link linkend="sparqloauthendpointauth">SPARQL Authentication</link> section.

</para>
</sect3>

<sect3 id="rdfsupportedprotocolendpoint"><title>Service Endpoint</title>
<para>Virtuoso uses the pre-assigned endpoints "/sparql" and "/SPARQL" as the
defaults for exposing its REST based SPARQL Web Services.</para>
<para>The port number associated with the SPARQL services is determined by the 'ServerPort' key value
in the '[HTTPServer]' section of the virtuoso.ini file. Thus, if the Virtuoso instance is configured
to listen at a none default port e.g. 8890, the SPARQL endpoints would be accessible
at http://example.com:8890/sparql/.</para>
<para>The SPARQL endpoint supports both GET and POST requests. The client chooses between GET and POST
automatically, using the length of query text as the criterion. If the SPARQL endpoint is accessed without
any URL and requisite SPARQL protocol parameters, an interactive HTML page for capturing SPARQL input will
be presented.</para>
<sect4 id="rdfsupportedprotocolendpointuricustm"><title>Customizing SPARQL Endpoint Page</title>
<para>The SPARQL Endpoint Page can now be customized using a xsl stylesheet.</para>
<para>This works by adding the following line with isql:</para>
<programlisting><![CDATA[
SQL> registry_set ('sparql_endpoint_xsl', 'http://host:port/path/isparql.xsl');
]]></programlisting>
<para>where obviously host, port, path and the name isparql.xsl can be set to anything.</para>
</sect4>
</sect3>

<sect3 id="rdfrequestparamsextensions"><title>SPARQL Protocol Extensions</title>

<sect4 id="rdfrequestparamsofunctions"><title>Request Parameters</title>
<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Request Parameters List</title>
    <tgroup align="char" charoff="50" char="." cols="3">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>Parameter</entry>
        <entry>Notes</entry>
        <entry>Required?</entry>
      </row>
    </thead>
    <tbody>
      <row>
        <entry>service</entry>
        <entry>Service URI such as 'http://example.com/sparql/'</entry>
        <entry>Yes</entry>
      </row>
      <row>
        <entry>query</entry>
        <entry>Text of the query</entry>
        <entry>Yes</entry>
      </row>
      <row>
        <entry>dflt_graph</entry>
        <entry>Default graph URI (string or NULL)</entry>
        <entry>No</entry>
      </row>
      <row>
        <entry>named_graphs</entry>
        <entry>Vector of named graphs (or NULL to prevent overriding named graphs specified in the query)</entry>
        <entry>Yes</entry>
      </row>
      <row>
        <entry>req_hdr</entry>
        <entry>Additional HTTP headers that should be passed to the service, e.g. 'Host: ...'</entry>
        <entry>No</entry>
      </row>
      <row>
        <entry>maxrows</entry>
        <entry>Limit on the numbers of rows that should be returned (the actual size of the result set may differ)</entry>
        <entry>No</entry>
      </row>
      <row>
        <entry>xslt-uri</entry>
        <entry>Absolute URL of any XSLT stylesheet file to be applied to the SPARQL query results</entry>
        <entry>No</entry>
      </row>
      <row>
        <entry>timeout</entry>
        <entry>Timeout for &quot;anytime&quot; query execution, in milliseconds, values less than 1000 are ignored; see
<link linkend="anytimequeries">Anytime Queries</link> for more details</entry>
        <entry>No</entry>
      </row>
      <row>
        <entry>debug</entry>
        <entry>If set to on, SPARQL Compiler will check if all variables are declared, and if there
        	is variable that is not declared, an error will be raised</entry>
        <entry>No</entry>
      </row>
    </tbody>
  </tgroup>
</table>
</sect4>

<sect4 id="rdfresponsecodeofprotocol"><title>Response Codes</title>
<para>If the query is a CONSTRUCT or a DESCRIBE then the result set
consists of a single row and a single column. The value inside is a
dictionary of triples in 'long valmode'. Note that the dictionary
object cannot be sent to a SQL client, say, via ODBC. The client may 
lose the database connection trying to fetch a result set row that
contains a dictionary object. This disconnection does not disrupt the server,
so the client may readily reconnect to the server, but the disconnected
transaction will have been rolled back.</para>

  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtodbcsparql">Virtuoso ODBC RDF extensions for SPASQL</link></listitem>
    </itemizedlist>
  </tip>

</sect4>

<sect4 id="rdfsupportedmimesofprotocol"><title>Response Format</title>
<para>All the SPARQL protocol standard MIME types are supported by a SPARQL web service client.
Moreover, SPARQL web service endpont supports additional MIME types and in some cases additional
query types for standard MIME types.</para>
<sect5 id="rdfsupportedmimesofprotocolserver"><title>Server Response Formats</title>
<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Server Response Formats</title>
    <tgroup align="char" charoff="50" char="." cols="3">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>Content-Type</entry>
        <entry>SPARQL query type</entry>
        <entry>Description</entry>
      </row>
      <row>
        <entry>'application/sparql-results+xml'</entry>
        <entry>SELECT, ASK</entry>
        <entry>Canonical XML presentation of SPARQL result set</entry>
      </row>
      <row>
        <entry>'text/rdf+n3'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/rdf+ttl'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/rdf+turtle'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/turtle'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/n3'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'application/turtle'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'application/x-turtle'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'application/x-nice-turtle'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle, like above, but the server will try to use "list" and "[...]" notations to make the document easier to read. This formatting is a slow procedure so long results will be formatted as plain Turtle.</entry>
      </row>
      <row>
        <entry>'text/rdf+nt'</entry>
        <entry>SELECT</entry>
        <entry>Format for NT (each triple is printed separately without abbreviations)</entry>
      </row>
      <row>
        <entry>'text/plain'</entry>
        <entry>SELECT</entry>
        <entry>Format for NT (each triple is printed separately without abbreviations)</entry>
      </row>
      <row>
        <entry>'text/ntriples'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>Format for NT (each triple is printed separately without abbreviations)</entry>
      </row>
      <row>
        <entry>'application/x-trig'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>TriG syntax for result sets, triples and quads (or sets of graphs with triples). While it is not used for quads, the output is same as for Turtle.</entry>
      </row>
      <row>
        <entry>'application/rdf+xml'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>Canonical RDF/XML presentation</entry>
      </row>
      <row>
        <entry>'application/soap+xml'</entry>
        <entry>SELECT</entry>
        <entry>SOAP XML</entry>
      </row>
      <row>
        <entry>'application/soap+xml;11'</entry>
        <entry>SELECT</entry>
        <entry>SOAP XML</entry>
      </row>
      <row>
        <entry>'text/html'</entry>
        <entry>SELECT</entry>
        <entry>HTML document for plain browsing; it's a TABLE for result set and HTML with micro-data for triples</entry>
      </row>
      <row>
        <entry>'text/md+html'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>HTML with microdata; for triples only</entry>
      </row>
      <row>
        <entry>'text/microdata+html'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>HTML with microdata; for triples only</entry>
      </row>
      <row>
        <entry>'text/x-html+ul'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>HTML with triples grouped into hierarchical list</entry>
      </row>
      <row>
        <entry>'text/x-html+tr'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>HTML with triples in form of a table</entry>
      </row>
      <row>
        <entry>'application/vnd.ms-excel'</entry>
        <entry>SELECT</entry>
        <entry>HTML table for loading data into stylesheets</entry>
      </row>
      <row>
        <entry>'text/csv'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>Comma-separated values</entry>
      </row>
      <row>
        <entry>'text/tab-separated-values'</entry>
        <entry>SELECT</entry>
        <entry>Tab-separated values</entry>
      </row>
      <row>
        <entry>'application/javascript'</entry>
        <entry>SELECT</entry>
        <entry>JavaScript data fragment</entry>
      </row>
      <row>
        <entry>'application/json'</entry>
        <entry>SELECT</entry>
        <entry>JSON</entry>
      </row>
      <row>
        <entry>'application/sparql-results+json'</entry>
        <entry>SELECT, ASK</entry>
        <entry>JSON result set</entry>
      </row>
      <row>
        <entry>'application/odata+json'</entry>
        <entry>SELECT, ASK, CONSTRUCT, DESCRIBE</entry>
        <entry>JSON in ODATA style</entry>
      </row>
      <row>
        <entry>'application/microdata+json'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>Microdata as JSON; for triples only</entry>
      </row>
      <row>
        <entry>'application/rdf+json'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>JSON in TALIS style; for triples only</entry>
      </row>
      <row>
        <entry>'application/x-rdf+json'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>JSON in TALIS style; for triples only</entry>
      </row>
      <row>
        <entry>'application/x-json+ld'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>JSON in Linked Data style; for triples only</entry>
      </row>
      <row>
        <entry>'application/ld+json'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>JSON in Linked Data style; for triples only</entry>
      </row>
      <row>
        <entry>'text/cxml'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>CXML output for rendering in Pivot Viewer of MS SilverLight?. Result sets and triples are handled in different ways.</entry>
      </row>
      <row>
        <entry>'text/cxml+qrcode'</entry>
        <entry>SELECT,CONSTRUCT, DESCRIBE</entry>
        <entry>CXML output with QRcode imprinted into each picture; for result sets and triples</entry>
      </row>
      <row>
        <entry>'application/atom+xml'</entry>
        <entry>SELECT, CONSTRUCT, DESCRIBE</entry>
        <entry>Atom-style XML</entry>
      </row>
      <row>
        <entry>'application/xhtml+xml'</entry>
        <entry>SELECT</entry>
        <entry>RDFa placed into XHTML; for triples only</entry>
      </row>
    </thead>
    <tbody>
    </tbody>
  </tgroup>
</table>
</sect5>
<sect5 id="rdfsupportedmimesofprotocolclient"><title>Client Response Formats</title>
<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Client Response Formats</title>
    <tgroup align="char" charoff="50" char="." cols="3">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>Content-Type</entry>
        <entry>SPARQL query type</entry>
        <entry>Description</entry>
      </row>
      <row>
        <entry>'application/sparql-results+xml'</entry>
        <entry>SELECT, ASK</entry>
        <entry>Canonical XML presentation of SPARQL result set</entry>
      </row>
      <row>
        <entry>'text/rdf+n3'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/rdf+ttl'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/rdf+turtle'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/turtle'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'text/n3'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'application/turtle'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'application/x-turtle'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Turtle</entry>
      </row>
      <row>
        <entry>'application/rdf+xml'</entry>
        <entry>CONSTRUCT, DESCRIBE</entry>
        <entry>Canonical RDF/XML presentation</entry>
      </row>
    </thead>
    <tbody>
    </tbody>
  </tgroup>
</table>
</sect5>
<para>The current implementation does not support returning the results of SELECT as RDF/XML or 'sparql-results-2'.</para>
<para>If the HTTP header returned by the remote server does not contain a 'Content-Type' line, the client may guess MIME type from the text of the returned body.</para>
<para>Error messages returned from the service are returned as XML documents, using the MIME type application/xml. The documents consist of a single element containing an error message.</para>
</sect4>
<sect4 id="rdfsupportedmimesofprotocoladdselect"><title>Additional Response Formats -- SELECT</title>
<para>Use the format parameter to select one of the following alternate output formats:</para>

<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Additional Response formats list -- SELECT</title>
    <tgroup align="char" charoff="50" char="." cols="3">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>Format Value</entry>
        <entry>Description</entry>
        <entry>Mimetype</entry>
      </row>
    </thead>
    <tbody>
      <row>
        <entry>HTML</entry>
        <entry>The result is a HTML document containing query summary and tabular results. The format
is human-readable but not intended for using by applications because it makes strings undistinguishable
from IRIs and loses other details such as exact datatypes of returned values.</entry>
        <entry>text/html</entry>
      </row>
      <row>
        <entry>json</entry>
        <entry>Two separate MIME types exist for JSON: JSON serialization of results is
'application/sparql-results+json' and confirms to the draft specification "Serializing SPARQL
Query Results in JSON". JSON serialization of triples is 'application/rdf+json' and interoperable
with Talis. Sometimes a client needs a JSON but it does not know the type of query it sends to
Virtuoso web service endpoint. In this case the client can specify either one MIME-type
'application/json' or both 'application/sparql-results+json' and 'application/rdf+json' in the
"Accept" header line and Virtuoso will chose the appropriate one automatically. Similar trick
works for other sorts of result types: Virtuoso inspects the whole "Accept" header line to
find out the most appropriate return type for the given query.
        </entry>
        <entry>application/sparql-results+json</entry>
      </row>
      <row>
        <entry>json</entry>
        <entry></entry>
        <entry>application/rdf+json</entry>
      </row>
      <row>
        <entry>js</entry>
        <entry>Javascript serialization of results generates an HTML table with the CSS class sparql.
The table contains a column indicating row number and additional columns for each query variable.
Each query solution contributes one row of the table.
Unbound variables are indicated with a non-breaking space in the appropriate table cells.</entry>
        <entry>application/javascript</entry>
      </row>
      <row>
        <entry>table</entry>
        <entry></entry>
        <entry>text/html</entry>
      </row>
      <row>
        <entry>XML</entry>
        <entry></entry>
        <entry>text/html</entry>
      </row>
      <row>
        <entry>TURTLE</entry>
        <entry></entry>
        <entry>text/html</entry>
      </row>
    </tbody>
  </tgroup>
</table>
</sect4>

<sect4 id="rdfsupportedmimesofprotocoladdcons"><title>Additional Response Formats -- CONSTRUCT &amp; DESCRIBE</title>
<para><emphasis>Example output of DESCRIBE in rdf+json serialization format</emphasis></para>
<orderedlist>
  <listitem>Go to the sparql endpoint at http://host:port/sparql, for ex. at http://dbpedia.org/sparql</listitem>
  <listitem>Enter query in the "Query text" area, for ex.:
<programlisting><![CDATA[
DESCRIBE <http://dbpedia.org/resource/%22S%22_Bridge_II>
]]></programlisting>
</listitem>
 <listitem>Select for "Display Results As": JSON</listitem>
  <listitem>Click "Run Query" button.</listitem>
  <listitem>As result should be produced the following output:
<programlisting><![CDATA[
{
  { 'http://dbpedia.org/resource/%22S%22_Bridge_II' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/ontology/Place' } ,
      { 'type' : 'uri', 'value' : 'http://dbpedia.org/ontology/Resource' } ,
      { 'type' : 'uri', 'value' : 'http://dbpedia.org/ontology/HistoricPlace' } } ,
    { 'http://dbpedia.org/ontology/added' : { 'type' : 'literal', 'value' : '1973-04-23' , 'datatype' : 'http://www.w3.org/2001/XMLSchema#date' } } ,
    { 'http://www.w3.org/2003/01/geo/wgs84_pos#lat' : { 'type' : 'literal', 'value' : 39.99305725097656 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#float' } } ,
    { 'http://www.w3.org/2003/01/geo/wgs84_pos#long' : { 'type' : 'literal', 'value' : -81.74666595458984 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#float' } } ,
    { 'http://dbpedia.org/property/wikiPageUsesTemplate' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Template:infobox_nrhp' } } ,
    { 'http://dbpedia.org/property/name' : { 'type' : 'literal', 'value' : '"S" Bridge II' , 'lang' : 'en' } } ,
    { 'http://dbpedia.org/property/nearestCity' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/New_Concord%2C_Ohio' } ,
      { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Ohio' } } ,
    { 'http://dbpedia.org/property/latDirection' : { 'type' : 'literal', 'value' : 'N' , 'lang' : 'en' } } ,
    { 'http://dbpedia.org/property/governingBody' : { 'type' : 'literal', 'value' : 'State' , 'lang' : 'en' } } ,
    { 'http://www.georss.org/georss/point' : { 'type' : 'literal', 'value' : '39.99305556 -81.74666667' } ,
      { 'type' : 'literal', 'value' : '39.9930555556 -81.7466666667' } } ,
    { 'http://xmlns.com/foaf/0.1/name' : { 'type' : 'literal', 'value' : '"S" Bridge II' } } ,
    { 'http://dbpedia.org/property/latDegrees' : { 'type' : 'literal', 'value' : 39 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://dbpedia.org/property/latMinutes' : { 'type' : 'literal', 'value' : 59 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://dbpedia.org/property/latSeconds' : { 'type' : 'literal', 'value' : 35 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://dbpedia.org/property/longDirection' : { 'type' : 'literal', 'value' : 'W' , 'lang' : 'en' } } ,
    { 'http://dbpedia.org/property/architect' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Benjamin_Latrobe' } } ,
    { 'http://dbpedia.org/property/added' : { 'type' : 'literal', 'value' : '1973-04-23' , 'datatype' : 'http://www.w3.org/2001/XMLSchema#date' } } ,
    { 'http://www.w3.org/2000/01/rdf-schema#label' : { 'type' : 'literal', 'value' : '"S" Bridge II (Muskingum County, Ohio)' , 'lang' : 'nl' } ,
      { 'type' : 'literal', 'value' : '"S" Bridge II' , 'lang' : 'en' } } ,
    { 'http://dbpedia.org/ontology/architect' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Benjamin_Latrobe' } } ,
    { 'http://xmlns.com/foaf/0.1/img' : { 'type' : 'uri', 'value' : 'http://upload.wikimedia.org/wikipedia/commons/d/d4/FoxRunS-Bridge_NewConcordOH.jpg' } } ,
    { 'http://dbpedia.org/property/locmapin' : { 'type' : 'literal', 'value' : 'Ohio' , 'lang' : 'en' } } ,
    { 'http://dbpedia.org/property/refnum' : { 'type' : 'literal', 'value' : 73001513 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://dbpedia.org/property/abstract' : { 'type' : 'literal', 'value' : '"S" Bridge II is a historic S bridge near New Concord, Ohio, United States. A part of the National Road, the first federally-financed highway in the United States, it was built in 1828. Its peculiar shape, typical for an S bridge, is designed to minimize the span and allow easy access. In 1973, it was listed on the National Register of Historic Places.' , 'lang' : 'en' } ,
      { 'type' : 'literal', 'value' : '"S" Bridge II bij New Concord, Ohio, is een deel van de National Road, een van de eerste highways die door de federale overheid vanaf 1811 werden aangelegd. De vorm, die de brug als een S Brug kenmerkt, is bedoeld om de overspanning zo klein mogelijk te houden en toch gemakkelijk toegang tot de brug te verlenen. De brug staat sinds 1973 op de lijst van het National Register of Historic Places als monument vermeld.' , 'lang' : 'nl' } } ,
    { 'http://www.w3.org/2004/02/skos/core#subject' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Category:National_Register_of_Historic_Places_in_Ohio' } ,
      { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Category:Bridges_on_the_National_Register_of_Historic_Places' } } ,
    { 'http://dbpedia.org/ontology/nearestCity' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/Ohio' } ,
      { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/New_Concord%2C_Ohio' } } ,
    { 'http://xmlns.com/foaf/0.1/depiction' : { 'type' : 'uri', 'value' : 'http://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/FoxRunS-Bridge_NewConcordOH.jpg/200px-FoxRunS-Bridge_NewConcordOH.jpg' } } ,
    { 'http://dbpedia.org/property/caption' : { 'type' : 'literal', 'value' : 'The bridge in the fall' , 'lang' : 'en' } } ,
    { 'http://dbpedia.org/property/longDegrees' : { 'type' : 'literal', 'value' : 81 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://dbpedia.org/property/longMinutes' : { 'type' : 'literal', 'value' : 44 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://dbpedia.org/property/longSeconds' : { 'type' : 'literal', 'value' : 48 , 'datatype' : 'http://www.w3.org/2001/XMLSchema#integer' } } ,
    { 'http://www.w3.org/2000/01/rdf-schema#comment' : { 'type' : 'literal', 'value' : '"S" Bridge II is a historic S bridge near New Concord, Ohio, United States.' , 'lang' : 'en' } ,
      { 'type' : 'literal', 'value' : '"S" Bridge II bij New Concord, Ohio, is een deel van de National Road, een van de eerste highways die door de federale overheid vanaf 1811 werden aangelegd.' , 'lang' : 'nl' } } ,
    { 'http://xmlns.com/foaf/0.1/page' : { 'type' : 'uri', 'value' : 'http://en.wikipedia.org/wiki/%22S%22_Bridge_II' } } } ,
  { 'http://dbpedia.org/resource/%22S%22_Bridge_II_%28Muskingum_County%2C_Ohio%29' : { 'http://dbpedia.org/property/redirect' : { 'type' : 'uri', 'value' : 'http://dbpedia.org/resource/%22S%22_Bridge_II' } } }
}

]]></programlisting>
</listitem>
</orderedlist>
<para><emphasis>Example output of CONSTRUCT in rdf+json serialization format</emphasis></para>
<orderedlist>
  <listitem>Go to the sparql endpoint at http://host:port/sparql, for ex. at http://dbpedia.org/sparql</listitem>
  <listitem>Enter query in the "Query text" area, for ex.:
<programlisting><![CDATA[
CONSTRUCT
{
 ?s a ?Concept .
}
WHERE
{
 ?s a ?Concept .
}
LIMIT 10
]]></programlisting>
</listitem>
 <listitem>Select for "Display Results As": JSON</listitem>
  <listitem>Click "Run Query" button.</listitem>
  <listitem>As result should be produced the following output:
<programlisting><![CDATA[
{
  { 'http://dbpedia.org/ontology/Place' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/Area' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/City' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/River' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/Road' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/Lake' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/LunarCrater' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/ShoppingMall' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/Park' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } } ,
  { 'http://dbpedia.org/ontology/SiteOfSpecialScientificInterest' : { 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' : { 'type' : 'uri', 'value' : 'http://www.w3.org/2002/07/owl#Class' } } }
}
]]></programlisting>
</listitem>
</orderedlist>
<para>
For interoperability with clients that were developed before current versions of SPARQL protocol and
format specs are issued, Virtuoso supports some obsolete variants of standard MIME types.
'text/rdf+n3', 'text/rdf+ttl', 'application/turtle' and 'application/x-turtle' are understood for
TURTLE output, 'application/x-rdf+json' and 'application/rdf+json' are for "Serializing SPARQL Query
Results in JSON". When a client specifies obsolete MIME type but not its standard variant, an obsolete
variant is returned for interoperability.
</para>
</sect4>
<sect4 id="viewresultspagesparqldebug"><title>Strict checking of void variables</title>
  <para>To set SPARQL Endpoint to check if all variables in a given query are declared correctly,
  	one should hatch the the Options "Strict checking of void variables" check-box. In that case,
  	if a variable declaration is missing, i.e. not included in the SPARQL Query WHERE clause, then an error will be raised.
  	For example, on attempt to execute the following query:
  </para>
<programlisting><![CDATA[
-- 	Options "Strict checking of void variables" check-box is checked:
select ?y
where
 {
  ?s ?p ?o
 }
]]></programlisting>
   <para>since the variable <code>?y</code> is not declared, the following error will be raised:</para>
<programlisting><![CDATA[
Virtuoso 37000 Error SP031: SPARQL compiler:
Variable 'y' is used in the query result set but not assigned

SPARQL query:

#output-format:text/html
define sql:signal-void-variables 1 define sql:gs-app-callback "ODS" select ?y
where
{
?s ?p ?o
}
]]></programlisting>
</sect4>
<sect4 id="viewresultspagesparqlqex"><title>View Results Page of SPARQL Query Execution</title>
<para>To view SPARQL Endpoint Results page of SPARQL query execution should be used the parameter 
<emphasis>query</emphasis> i.e the SPARQL Protocol URL should look like:</para> 
<programlisting><![CDATA[
http://cname/sparql?default-graph-uri=&query=... 
]]></programlisting>
<para><emphasis>Example</emphasis></para>
<para>Suppose the following simple query:</para>
<programlisting><![CDATA[
SELECT * 
WHERE 
  {
    ?s ?p ?o
  }	
LIMIT 10  
]]></programlisting>
<para>See <ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=SELECT+*+%0D%0AWHERE+%0D%0A++{%0D%0A++++%3Fs+%3Fp+%3Fo%0D%0A++}%09%0D%0ALIMIT+10++&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">this example link</ulink> against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URL.</para>
</sect4>
<sect4 id="viewresultspagesparqlqed"><title>View Editor Page of SPARQL Query </title>
<para>To view the SPARQL Endpoint editor page of SPARQL query execution should be used the parameter 
<emphasis>qtxt</emphasis> i.e the SPARQL Protocol URL should look like:</para>
<programlisting><![CDATA[
http://cname/sparql?default-graph-uri=&qtxt=... 
]]></programlisting>
<para><emphasis>Example</emphasis></para>
<para>Suppose the following simple query:</para>
<programlisting><![CDATA[
SELECT * 
WHERE 
  {
    ?s ?p ?o
  }	
LIMIT 10  
]]></programlisting>
<para>Suppose also <ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=SELECT+*+%0D%0AWHERE+%0D%0A++{%0D%0A++++%3Fs+%3Fp+%3Fo%0D%0A++}%09%0D%0ALIMIT+10++&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">this results page link</ulink> against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URL. </para>
<para>Replace the parameter name <emphasis>query</emphasis> with <emphasis>qtxt</emphasis>.</para>
<para><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=SELECT+*+%0D%0AWHERE+%0D%0A++{%0D%0A++++%3Fs+%3Fp+%3Fo%0D%0A++}%09%0D%0ALIMIT+10++&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">Access the new link</ulink>, which should present the SPARQL Endpoint Editor page with "Query Text" area filled in with the SPARQL Query from above.</para>
</sect4>
<sect4 id="rdfsupportedmimesaddofprotocol"><title>Virtuoso/PL APIs</title>
<para>Virtuoso also provides SPARQL protocol client APIs in Virtuoso PL, so you can communicate with SPARQL
Query Services from Virtuoso stored procedures. The APIs are as follows:
</para>
<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Virtuoso/PL APIs</title>
    <tgroup align="char" charoff="50" char="." cols="2">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>API</entry>
        <entry>Notes</entry>
      </row>
    </thead>
    <tbody>
      <row>
        <entry>DB.DBA.SPARQL_REXEC</entry>
        <entry>Behaves like DBA.SPARQL_EVAL, but executes the query on the specified server. The procedure
does not return anything. Instead, it creates a result set.</entry>
      </row>
      <row>
        <entry>DB.DBA.SPARQL_REXEC_TO_ARRAY</entry>
        <entry>Behaves like DBA.SPARQL_EXEC_TO_ARRAY(), but executes the query on the specified server. The
function returns a vector of rows, where every row is represented by a vector of field values.</entry>
      </row>
      <row>
        <entry>DB.DBA.SPARQL_REXEC_WITH_META</entry>
        <entry>Has no local SPARQL_EVAL analog. It produces not only an array of result rows together with an array
of result set metadata in a format used by the exec() function.</entry>
      </row>
    </tbody>
  </tgroup>
</table>
</sect4>
   <sect4 id="anytimequeriessparql">
     <title>SPARQL Anytime Queries</title>
     <para>Starting with version 6, Virtuoso offers a partial query evaluation feature that guarantees
answers to arbitrary queries within a fixed time. This is intended for use in  publicly available SPARQL
or SQL end points on large databases. This enforces a finite duration to all queries and will strive to
return meaningful partial results. Thus this provides the same security as a transaction timeout but will
be more user friendly since results will generally be returned, also for aggregate queries. Outside of a
public query service, this may also be handy when exploring a large data set with unknown properties.
     </para>
     <para>The feature is activated with the statement</para>
<programlisting><![CDATA[
set result_timeout == <expression>;
]]></programlisting>
   <para>Find more detailed information in the <link linkend="anytimequeries">Anytime Queries</link> section.</para>
   <sect5 id="anytimequeriessparqlex"><title>Example Dump arbitrary query result as N-Triples</title>
   <para>Assume the following arbitrary query:</para>
<programlisting><![CDATA[
SPARQL define output:format "NT" 
CONSTRUCT { ?s a ?t } 
FROM virtrdf:
WHERE { ?s a ?t };	
]]></programlisting>   
      <para>For iteration over result-set of an arbitrary query, use 
      	<link linkend="fn_exec_next"><function>exec_next()</function></link> in a loop that begins with 
      	<link linkend="fn_exec"><function>exec()</function></link> with cursor output variable as 
      	an argument and ends with <link linkend="fn_exec_close"><function>exec_close()</function></link> 
      	after it is out of data.
      </para>
   </sect5>
   </sect4>
</sect3>

<sect3 id="rdfsupportedprotocolendpointuri"><title>Service Endpoint Security</title>
<para>Earlier releases of Virtuoso secured the SPARQL endpoint via privileges assigned to the service-
specific SQL user account "SPARQL". This account was optionally granted "SPARQL_SELECT" or
"SPARQL_UPDATE" roles. By default only the  "SPARQL_SELECT" role was assigned, enabling all users
to at least perform SELECT queries. The "SPARQL_UPDATE" role must be granted to allow updates
to the Quad Store - a pre-requisite for the Virtuoso Sponger services to be functional i.e. to
allow the Sponger to populate and update the Quad Store. In Virtuoso release 5.0.7, there is a new "SPARQL_SPONGE" role
which can be assigned specifically to allow Sponger services to update the Quad Store but not 
SPARQL users via the SPARQL endpoint.</para>
<para>Restricting a user's access to specific graphs can be done using Virtuoso Graph security
functionality, via one of the Virtuoso Data Access APIs: ODBC, JDBC, ADO.Net or PL code.</para>

  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtodbcsparql">Virtuoso ODBC RDF extensions for SPASQL</link></listitem>
    </itemizedlist>
  </tip>

<para>For example, users of OpenLink Data Space (ODS) applications are restricted in the RDF graphs accessible to them as follows:</para>
<programlisting><![CDATA[
DB.DBA.TABLE_DROP_POLICY ('DB.DBA.RDF_QUAD', 'S');

create procedure DB.DBA.RDF_POLICY (in tb varchar, in op varchar)
{
 declare chost, ret varchar;
 chost := DB.DBA.WA_CNAME ();  
 ret := sprintf ('(ID_TO_IRI (G) NOT LIKE \'http://%s/dataspace/%%/private#\' ' ||
 'OR G = IRI_TO_ID (sprintf (\'http://%s/dataspace/%%U/private#\', USER)))', chost, chost);
 return ret;
}
;

grant execute on DB.DBA.RDF_POLICY to public;

DB.DBA.TABLE_SET_POLICY ('DB.DBA.RDF_QUAD', 'DB.DBA.RDF_POLICY', 'S');
]]></programlisting>
<para>
where DB.DBA.WA_CNAME () is an ODS function returning the default host name.
</para>
<para>The effect of this policy is to restrict user 'user' to the graph http://cname/dataspace/user/private#</para>
<sect4 id="rdfsupportedprotocolendpointurisparqlauthex"><title>SPARQL Auth Endpoint Usage Example</title>
<para>Virtuoso reserves the path '/sparql-auth/' for a SPARQL service supporting authenticated SPARUL.
This endpoint allows specific SQL accounts to perform SPARUL over the SPARQL protocol.
To be allowed to login via SQL or ODBC and update physical triples, a user must be granted "SPARQL_UPDATE" privileges. To grant this role:
</para>
<orderedlist>
  <listitem>Go to the Virtuoso administration UI i.e. http://host:port/conductor</listitem>
  <listitem>Login as user dba</listitem>
  <listitem>Go to System Admin->User Accounts->Users
    <figure id="rdf1" float="1">
      <title>Conductor UI</title>
      <graphic fileref="ui/usr1.png"/>
    </figure>
  </listitem>
  <listitem>Click the link "Edit"</listitem>
  <listitem>Set "User type" to "SQL/ODBC Logins and WebDAV".</listitem>
  <listitem>Select from the list of available Account Roles "SPARQL_UPDATE" role and
click the ">>" button so to add it to the right-hand list.</listitem>
    <figure id="rdf2" float="1">
      <title>Conductor UI</title>
      <graphic fileref="ui/usr2.png"/>
    </figure>
  <listitem>Click the "Save" button.</listitem>
</orderedlist>
<para>Note that if a table is used in an Linked Data View, and this table is not granted to SPARQL_SELECT permission
(or SPARQL_UPDATE, which implicitly confers SPARQL_SELECT), then all SELECTs on a graph defined by an 
Linked Data View will return an access violation error as the user
account has no permissions to read the table. The user must have appropriate privileges on all tables included
in an Linked Data View in order to be able to select on <emphasis>all</emphasis> graphs.</para>
</sect4>
<sect4 id="sparqwebservicetbl"><title>Managing a SPARQL Web Service Endpoint</title>
<para>
Virtuoso web service endpoints may provide different default configurations for different host names mentioned in an HTTP request.
Host name configuration for SPARQL web service endpoints can be managed via the table <emphasis>DB.DBA.SYS_SPARQL_HOST</emphasis>.
</para>
<programlisting>
create table DB.DBA.SYS_SPARQL_HOST (
  SH_HOST	varchar not null primary key, -- host mask
  SH_GRAPH_URI varchar,                 -- default graph uri
  SH_USER_URI	varchar,      -- reserved for any use in applications
  SH_BASE_URI varchar,                  -- for future use (not used currently) to set BASE in sparql queries. Should be NULL for now.
  SH_DEFINES	long varchar,             -- additional defines for requests
  PRIMARY KEY (SH_HOST)
)
</programlisting>
<para>You can find detailed descriptions of the table columns <link linkend="rdfdefaultgraph">here</link>.
Also, please read <link linkend="rdfperfindexes">these notes</link> on managing public web service endpoints.</para>
</sect4>
<sect4 id="sparqloauthendpointauth"><title>Authentication</title>
<para>Virtuoso 5.0.7 introduced a new "SPARQL_SPONGE" role which can be assigned
specifically for controlling Sponger middleware services which perform writes and graph creation in
the RDF Quad Store. This role only allows updates through the Sponger. Quad Store updates via any other route 
require granting the SPARQL_UPDATE role.</para>
<para> 
Virtuoso 5.0.11 onwards added three new methods for securing SPARQL endpoints that include:
</para>
<itemizedlist mark="bullet">
  <listitem>SQL authentication</listitem>
  <listitem>OAuth</listitem>
  <listitem>WebID Protocol based authentication</listitem>
</itemizedlist>
<para> 
Each of these authentication methods is associated with a purpose specific default SPARQL endpoint along the following lines:
</para> 
<itemizedlist>
  <listitem>http://&lt;cname&gt;/sparql-auth (SQL authentication)</listitem>
  <listitem>http://&lt;cname&gt;/sparql-oauth (OAuth)</listitem>
  <listitem>http://&lt;cname&gt;/sparql-graph-crud-auth (OAuth CRUD)</listitem>
  <listitem>https://&lt;cname&gt;/sparql and https://&lt;cname&gt;/sparql-webid (WebID Protocol)</listitem>
</itemizedlist>
<para>Note: sparql-ssl is alias of sparql-webid.</para>
<para>The Virtuoso Authentication Server offers a UI with options for managing:</para>
<itemizedlist mark="bullet">
  <listitem>
Application keys and protected SPARQL endpoints: OAuth provides a secure data transmission
level mechanism for your SPARQL endpoint. It enables you to interact securely with your RDF database
from a variety of locations. It also allows you to provide controlled access to private data to selected
user profiles.
  </listitem>
  <listitem>WebID Protocol ACLs: WebID Protocol is an implementation of a conceptual authentication and authorization
protocol that links a Web ID to a public key to create a global, decentralized/distributed, and open
yet secure authentication system that functions with existing browsers.</listitem>
</itemizedlist>
<para>Virtuoso Authentication Server can be installed by downloading and installing the
conductor_dav.vad package.</para>
  <para>The Authentication UI is accessible from the Conductor UI -&gt; Linked Data -&gt; Access Control -&gt; SPARQL-WebID. Here is sample scenario:</para>
  <sect5 id="sparqloauthendpointauthexample"><title>SPARQL-WebID Authentication Example</title>
<orderedlist>
  <listitem>Download and install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> package.</listitem>
  <listitem><ulink url="http://ods.openlinksw.com/dataspace/dav/wiki/ODS/ODSGenerateX509Certificate">Generate an X.509 Certificate hosted WebID</ulink>.</listitem>
  <listitem>Go to http://&lt;cname&gt;:&lt;port&gt;/conductor, where &lt;cname&gt;:&lt;port&gt; are replaced by your 
  	local server values.</listitem> 
  <listitem>Log in as user "dba" or another user with DBA privileges.</listitem>
      <listitem>Go to Linked Data -&gt; Access Controls -&gt; SPARQL-WebID:
     <figure id="sparqlssl1" float="1">
           <title>SPARQL-WebID</title>
       <graphic fileref="ui/auth1.png"/>
     </figure>  	
</listitem>
  <listitem>Enter in the presented form Web ID for ex.:
<programlisting><![CDATA[
http://id.myopenlink.net/dataspace/person/demo#this
]]></programlisting>  	
  <para> and select "SPARQL Role" for ex. "Sponge".</para>
     <figure id="sparqlssl2" float="1">
          <title>SPARQL-WebID</title>
       <graphic fileref="ui/auth3.png"/>
     </figure>    
  </listitem>
  <listitem>Click the "Register" button.</listitem>
  <listitem>As result the WebID Protocol ACLs will be created:
     <figure id="sparqlssl3" float="1">
           <title>SPARQL-WebID</title>
       <graphic fileref="ui/auth4.png"/>
     </figure>  	
  </listitem>          
      <listitem>Go to the SPARQL-WebID endpoint https://&lt;cname&gt;:&lt;port&gt;/sparql-webid</listitem>          
  <listitem>Select the user's certificate from above:
     <figure id="sparqlssl4" float="1">
           <title>SPARQL-WebID</title>
       <graphic fileref="ui/auth5.png"/>
    </figure>
  </listitem>          
  <listitem>As result the SPARQL Query UI will be presented:
     <figure id="sparqlssl5" float="1">
           <title>SPARQL-WebID</title>
       <graphic fileref="ui/auth6.png"/>
     </figure>   	
  </listitem>          
  <listitem>Execute sample query and view the results:
  	 <figure id="sparqlssl7" float="1">
           <title>SPARQL-WebID</title>
           <graphic fileref="ui/auth6a.png"/>
     </figure>   	
  </listitem>                  
</orderedlist>
  </sect5>
</sect4>
<sect4 id="sparqloauthendpoint"><title>SPARQL OAuth Endpoint</title>
<para>OAuth provides a secure data transmission level mechanism for your SPARQL endpoint.
It enables you to interact securely with your RDF database from a variety of locations. It also
allows you to provide controlled access to private data to selected users.</para>
<para>Virtuoso OAuth Server can be installed by downloading and installing the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/ods_framework_dav.vad">ods_framework_dav.vad</ulink> package.
The OAuth UI is accessible from the URL http://cname:port/oauth</para>
<para>A user must have SQL privileges in order to run secured SPARQL statements.</para>
<para>Here is a sample scenario:</para>

<orderedlist>
  <listitem>Download and install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> 
and <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/ods_framework_dav.vad">ods_framework_dav.vad</ulink> packages.</listitem>

  <listitem><ulink url="http://ods.openlinksw.com/dataspace/dav/wiki/ODS/ODSGenerateX509Certificate">Generate an X.509 Certificate hosted WebID</ulink>.</listitem>
  
  <listitem>Go to http://&lt;cname&gt;:&lt;port&gt;/conductor, where &lt;cname&gt;:&lt;port&gt; are replaced by your 
  	local server values.</listitem> 
  	
  <listitem>Log in as user "dba" or another user with DBA privileges.</listitem>
  
  <listitem>Go to System Admin->User Accounts:
<figure id="sparqloauthendpoint1" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so1.png"/>
    </figure>
</listitem>

<listitem>Click "Create New Account":
<figure id="sparqloauthendpoint2" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so2.png"/>
    </figure>
</listitem>

<listitem>In the presented form enter respectively:
  <orderedlist>
    <listitem>Account name, for ex:demo1; a password and then confirm the password;</listitem>
    <listitem>User type: SQL/ODBC and WebDAV;</listitem>
    <listitem>Account role: SPARQL_UPDATE
<figure id="sparqloauthendpoint3" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so3.png"/>
    </figure>
</listitem>
</orderedlist>
</listitem>

<listitem>Click the "Save" button. </listitem>

<listitem>The created user should be shown in the list of registered users:
<figure id="sparqloauthendpoint4" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so4.png"/>
    </figure>
</listitem>


<listitem>Go to http://&lt;cname&gt;:&lt;port&gt;/oauth/, where &lt;cname&gt;:&lt;port&gt; are replaced by your local server values. 
<figure id="sparqloauthendpoint5" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so5.png"/>
    </figure>
</listitem>

<listitem>Click the "OAuth keys" link:
<figure id="sparqloauthendpoint6" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so6.png"/>
    </figure>	
	</listitem>
	
<listitem>Log in as user demo1: 
		<figure id="sparqloauthendpoint7" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so7.png"/>
    </figure>	
</listitem>

<listitem>The OAuth application registration form will be shown. 
		<figure id="sparqloauthendpoint8" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so8.png"/>
    </figure>
</listitem>

<listitem>Select SPARQL from the "Application name" list, and click the "Generate Keys" button. </listitem>

<listitem>A Consumer Key for SPARQL will be generated: 
<programlisting><![CDATA[
90baa79108b1d972525bacc76c0279c02d6421e8
]]></programlisting>
		<figure id="sparqloauthendpoint9" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so9.png"/>
    </figure>
</listitem>

<listitem>Click the "Back to main menu" link.</listitem>

<listitem>Click the "Protected SPARQL Endpoint" link.</listitem>

<listitem>The OpenLink Virtuoso SPARQL Query form will be displayed.
		<figure id="sparqloauthendpoint11" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so11.png"/>
    </figure>	
    		<figure id="sparqloauthendpoint12" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so12.png"/>
    </figure>
</listitem>

<listitem>Enter a simple query, for ex: 
<programlisting><![CDATA[
SELECT * 
WHERE 
  {
    ?s ?p ?o
  }
LIMIT 10
]]></programlisting>
</listitem>

<listitem>Enter the value from below for the "OAuth token":
<programlisting><![CDATA[
90baa79108b1d972525bacc76c0279c02d6421e8
]]></programlisting>
		<figure id="sparqloauthendpoint13" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so13.png"/>
    </figure>
</listitem>

<listitem>Click the "Run Query" button.</listitem>

<listitem>In the OAuth Authorization Service form enter the password for user demo1 and click the "Login" button.
		<figure id="sparqloauthendpoint14" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so16.png"/>
    </figure>
</listitem>

<listitem>Next you should authorize the request: 
		<figure id="sparqloauthendpoint15" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so15.png"/>
    </figure>
</listitem>

<listitem>On successful authentication and authorization, the query results should be shown: 
		<figure id="sparqloauthendpoint14" float="1">
      <title>SPARQL OAuth Endpoint</title>
      <graphic fileref="ui/so14.png"/>
    </figure>
</listitem>
</orderedlist>
</sect4>
<sect4 id="sparqloauthendpointfoafssl"><title>WebID Protocol ACLs</title>
<para>WebID Protocol is an implementation of a conceptual authentication and authorization protocol that
links a Web ID to a public key, to create a global decentralized/distributed, and open yet secure
authentication system that functions with existing browsers.</para>
<para>To use WebID Protocol, download and install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> 
VAD package. Once installed, to access the WebID Protocol ACLs UI, go to 
URL http://cname:port/conductor -&gt; Linked Data -&gt; Access Controls -&gt; SPARQL-WebID .</para>
  <figure id="sparqloauthendpoint1" float="1">
    <title>WebID</title>
    <graphic fileref="ui/auth4.png"/>
  </figure>
<para>
Configuring WebID Protocol ACLs is with a WebID Protocol certificate and a Web ID allows secure SPARQL queries to be performed against a Virtuoso SPARQL-WebID endpoint and viewing of the query results.
The SPARQL-WebID endpoint URL is of the form https://cname:port/sparql-webid</para>
<para>Note: SPARQL-SSL is alias of SPARQL-WebID.</para>
<para>See <link linkend="sparqloauthendpointauthexample">sample example </link> how to configure a sample  WebID Protocol ACL are outlined below:</para>
  <tip><title>See Also:</title>
    <para><link linkend="secureodbcx509foafsll">WebID Protocol ODBC Login</link></para>
  </tip>
</sect4>
<sect4 id="sparqloauthendpointfoafsslsparql"><title>Creating and Using a SPARQL-WebID based Endpoint</title>
<para>The following section describes the basic steps for setting up an SSL protected and WebID 
based SPARQL Endpoint (SPARQL-WebID). The guide also covers the use of Virtuoso PL functions and 
the Virtuoso Conductor for SPARQL endpoint creation and configuration. It also covers the use 
of cURL for exercising the newly generated SPARQL-SSL endpoint. Note: SPARQL-SSL is alias of SPARQL-WebID. 
</para>
<orderedlist>
  <listitem><link linkend="vfoafsslst509issuer">Setup the CA issuer and https listener</link></listitem>
  <listitem>To create the /sparql-webid endpoint, install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.3/virtuoso/policy_manager_dav.vad">policy_manager.vad</ulink> manage or manually 
  define the /sparql-webid endpoint on an HTTPS based listener (HTTPS service endpoint), for example using Virtuoso PL:
<programlisting><![CDATA[
DB.DBA.VHOST_DEFINE (
	 lhost=>'127.0.0.1:443',
	 vhost=>'localhost',
	 lpath=>'/sparql-webid',
	 ppath=>'/!sparql/',
	 is_dav=>1,
	 auth_fn=>'DB.DBA.FOAF_SSL_AUTH',
	 vsp_user=>'dba',
	 ses_vars=>0,
	 auth_opts=>vector ( 'https_cert', 
	                     'db:https_key_localhost', 
	                     'https_key', 
	                     'db:https_key_localhost', 
	                     'https_verify', 
	                     3, 
	                     'https_cv_depth', 
	                     10 ),
	 opts=>vector ('noinherit', 1),
	 is_default_host=>0
);
]]></programlisting>  	
</listitem>
  <listitem><link linkend="sparqloauthendpointfoafssl">Setup the SPARQL-WebID endpoint and define 
ACLs</link> using the Virtuoso Conductor</listitem>
  <listitem>Export your private key and its associated WebID based X.509 certificate from your 
Firefox browser or System's Key Manager into PEM (PKCS12) file
    <orderedlist>
      <listitem>If using Firefox use the menu path: Advanced -&gt; View Certificates, then click 
Backup for your certificate with name "mykey". </listitem>
      <listitem>The file "mykey.p12" will be created. To disable password protection so that you 
can use this file in non-interactive mode (e.g. with cURL and other HTTP clients) execute:
<programlisting><![CDATA[
openssl pkcs12 -in mykey.p12 -out mykey.pem -nodes 	
]]></programlisting>
</listitem>
    </orderedlist>
  </listitem>
  <listitem>Test the SPARQL-WebID endpoint with cURL: (listening on default HTTPS 443 port): 
    <itemizedlist mark="bullet">
      <listitem>Note: In this example we use the "-k / --insecure" option with cURL since we are going 
to be using self-signed X.509 certificates signed by self-signed root CA.</listitem>
    </itemizedlist>  	
<programlisting><![CDATA[
	curl -k -E mykey.pem "https://example.com/sparql-webid?query=select+*+where+\{+%3Fx+%3Fy+%3Fz+.+\}+limit+10&format=text%2Fn3"

@prefix res: <http://www.w3.org/2005/sparql-results#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
_:_ a res:ResultSet .
_:_ res:resultVariable "x" , "y" , "z" .
@prefix ns0:    <https://example.com/tutorial/> .
@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:hosting ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:xml ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:repl ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:rdfview ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:services ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:wap ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:bpeldemo ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:web ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:web2 ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:xmlxslt ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
]]></programlisting>    
</listitem>
  <listitem>Import your key it via Conductor UI:
  	<orderedlist>
     <listitem>Go to Conductor -> System Admin->User Accounts
     <figure id="foafsslsparql1" float="1">
       <title>Import key it via Conductor UI</title>
       <graphic fileref="ui/fsp1.png"/>
     </figure>
     </listitem>
     <listitem>Click "Edit" for your user
     <figure id="foafsslsparql2" float="1">
       <title>Import key it via Conductor UI</title>
       <graphic fileref="ui/fsp2.png"/>
     </figure>
     </listitem>
     <listitem>Change "User type" to: SQL/ODBC and WebDAV
     <figure id="foafsslsparql3" float="1">
       <title>Import key it via Conductor UI</title>
       <graphic fileref="ui/fsp3.png"/>
     </figure>
     </listitem>
     <listitem>Enter your ODS user WebID:
<programlisting><![CDATA[
http://cname:port/dataspace/person/username#this
]]></programlisting>    	
     <figure id="foafsslsparql4" float="1">
       <title>Import key it via Conductor UI</title>
       <graphic fileref="ui/fsp4.png"/>
     </figure>
     </listitem>
     <listitem>Click "Save"</listitem>
     <listitem>Click again "Edit" for your user</listitem>
     <listitem>In "PKCS12 file:" click the Browse" button and select your key.</listitem>
     <listitem>Enter a local Key Name, for e.g., "cli_key"</listitem>
     <listitem>Enter key password
     <figure id="foafsslsparql5" float="1">
       <title>Import key it via Conductor UI</title>
       <graphic fileref="ui/fsp5.png"/>
     </figure>
     </listitem>                
     <listitem>Click "Import Key"</listitem> 
     <listitem>As result the key will be stored with name for ex. cli_key 
     <figure id="foafsslsparql6" float="1">
       <title>Import key it via Conductor UI</title>
       <graphic fileref="ui/fsp6.png"/>
     </figure>
    </listitem> 
    <listitem>Click "Save"</listitem>    
  </orderedlist>
</listitem>
  <listitem>Test the SPARQL-WebID endpoint with http_client (listening on default HTTPS 443 port):
  	<orderedlist>
      <listitem>Log in at Virtuos ISQL with your user credentials: 
<programlisting><![CDATA[
C:\>isql localhost:1111 johndoe****
Connected to OpenLink Virtuoso
Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL>	
]]></programlisting>      	
</listitem>
      <listitem>Execute:
<programlisting><![CDATA[
SQL>select http_client ('https://example.com/sparql-webid?query=select+*+where+{+%3Fx+%3Fy+%3Fz+.+}+limit+10&format=text%2Fn3', cert_file=>'d
b:cli_key', insecure=>1);
callret
VARCHAR
_______________________________________________________________________________


@prefix res: <http://www.w3.org/2005/sparql-results#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
_:_ a res:ResultSet .
_:_ res:resultVariable "x" , "y" , "z" .
@prefix ns0:    <https://example.com/tutorial/> .
@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:hosting ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:xml ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:repl ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:rdfview ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:services ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:wap ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:bpeldemo ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [
      res:binding [ res:variable "x" ; res:value ns0:web ] ;
      res:binding [ res:variable "y" ; res:value rdf:type ] ;
      res:binding [ res:variable "z" ; res:value "Tutorial" ] ] .
_:_ res:solution [


1 Rows. -- 281 msec.
]]></programlisting>      	
</listitem>
    </orderedlist>
</listitem>
</orderedlist>

<tip><title>See Also:</title>
<para><ulink url="http://demo.openlinksw.com/tutorial/rdf/fs_s_1/fs_s_1.vsp">Demo Example</ulink> Using HTTP client to perform WebID Protocol connection.</para>
</tip>
</sect4>
<sect4 id="sparqloauthendpointfoafsslsparqldisable"><title>Disable Default SPARQL Endpoint</title>
  <sect5 id="sparqloauthendpointfoafsslsparqldisableisql"><title>Using iSQL:</title>
    <orderedlist>
      <listitem>To disable /sparql, execute:
<programlisting><![CDATA[
DB.DBA.VHOST_REMOVE (lpath=>'/sparql');	
]]></programlisting>      	
      </listitem>
      <listitem>To add the endpoint again via PL, execute:
<programlisting><![CDATA[
DB.DBA.VHOST_DEFINE (lpath=>'/sparql/', ppath => '/!sparql/', is_dav => 1, vsp_user => 'dba', opts => vector('noinherit', 1)); 	
]]></programlisting>      	
      </listitem>
    </orderedlist>    
  </sect5>
  <sect5 id="sparqloauthendpointfoafsslsparqldisablecond"><title>Using Conductor UI:</title>
    <orderedlist>
      <listitem>Go to http://cname:port/conductor .</listitem>
      <listitem>Enter user dba credentials.</listitem>
      <listitem>Go to "Web Application Server" -&gt; "Virtual Domains &amp; Directories".
        <figure id="s1" float="1">
          <title>Disable SPARQL Endpoint</title>
          <graphic fileref="ui/s1.png"/>
        </figure>      	
      </listitem>
      <listitem>Find the logical path "/sparql".
        <figure id="s2" float="1">
          <title>Disable SPARQL Endpoint</title>
          <graphic fileref="ui/s2.png"/>
        </figure>      	
      </listitem>
      <listitem>Click "Edit" from the "Action" column.
        <figure id="s3" float="1">
          <title>Disable SPARQL Endpoint</title>
          <graphic fileref="ui/s3.png"/>
        </figure>      	
      </listitem>
      <listitem>Change "VSP User" to "nobody".
        <figure id="s4" float="1">
          <title>Disable SPARQL Endpoint</title>
          <graphic fileref="ui/s4.png"/>
        </figure>      	
      </listitem>
      <listitem>Click "Save Changes".</listitem>
      <listitem>As result the SPARQL Endpoint should be shown as disabled:
        <figure id="s5" float="1">
          <title>Disable SPARQL Endpoint</title>
          <graphic fileref="ui/s5.png"/>
        </figure>      	
      </listitem>
    </orderedlist>  
  </sect5>
</sect4>
</sect3>

<sect3 id="rdfsupportedrequestmethodsofprotocol"><title>Request Methods</title>
<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Methods List</title>
    <tgroup align="char" charoff="50" char="." cols="3">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>Method</entry>
        <entry>Supported?</entry>
        <entry>Notes</entry>
      </row>
    </thead>
    <tbody>
      <row>
        <entry>GET</entry>
        <entry>Yes</entry>
        <entry>Short queries are sent in GET mode</entry>
      </row>
      <row>
        <entry>POST</entry>
        <entry>Yes</entry>
        <entry>Queries longer than 1900 bytes are POST-ed.</entry>
      </row>
      <row>
        <entry>DELETE</entry>
        <entry>No</entry>
        <entry></entry>
      </row>
      <row>
        <entry>PUT</entry>
        <entry>No</entry>
        <entry></entry>
      </row>
    </tbody>
  </tgroup>
</table>
</sect3>

<sect3 id="rdfsparqlclientfunctions"><title>Functions</title>
<para>The SPARQL client can be invoked by three similar functions:</para>

<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>Functions List</title>
    <tgroup align="char" charoff="50" char="." cols="3">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>Function</entry>
        <entry>Notes</entry>
      </row>
    </thead>
    <tbody>
      <row>
        <entry>DB.DBA.SPARQL_REXEC</entry>
        <entry>Behaves like DBA.SPARQL_EVAL, but executes the query on the specified server. The procedure does not return anything. Instead, it creates a result set. </entry>
      </row>
      <row>
        <entry>DB.DBA.SPARQL_REXEC_TO_ARRAY</entry>
        <entry>Behaves like DBA.SPARQL_EXEC_TO_ARRAY (), but executes the query on the specified server. The function return a vector of rows, where every row is represented by a vector of field values.</entry>
      </row>
      <row>
        <entry>DB.DBA.SPARQL_REXEC_WITH_META</entry>
        <entry>Has no local 'SPARQL_EVAL' analog. It produces an array of result rows together with an array of result set metadata in the same format as produced by the exec () function. 
	This function can be used when the result should be passed later to exec_result_names () and exec_result () built-in functions. 
	To process a local query in similar style, an application can use the SQL built-in function exec () - a SPARQL query (with the 'SPARQL' keyword in front) can be passed to exec () instead of a plain SQL SELECT statement.</entry>
      </row>
    </tbody>
  </tgroup>
</table>

<programlisting>
create procedure DB.DBA.SPARQL_REXEC (
    in service varchar, in query varchar, in dflt_graph varchar, in named_graphs any,
    in req_hdr any, in maxrows integer, in bnode_dict any );
</programlisting>

<programlisting>
create function DB.DBA.SPARQL_REXEC_TO_ARRAY (
    in service varchar, in query varchar, in dflt_graph varchar, in named_graphs any,
    in req_hdr any, in maxrows integer, in bnode_dict any )
    returns any;
</programlisting>

<programlisting>
create procedure DB.DBA.SPARQL_REXEC_WITH_META (
    in service varchar, in query varchar, in dflt_graph varchar, in named_graphs any,
    in req_hdr any, in maxrows integer, in bnode_dict any,
    out metadata any,  -- metadata like exec () returns.
    out resultset any) -- results as 'long valmode' values.
</programlisting>

</sect3>


<sect3 id="rdfsparqlendpointexamples"><title>Examples</title>
<para>Virtuoso's SPARQL demo offers a live demonstration of Virtuoso's implementation of the
<ulink url="http://www.w3.org/TR/rdf-dawg-uc/">DAWG's SPARQL test-suite</ulink>,
a collection of SPARQL query language use cases that enable interactive and simplified testing of a triple store implementation.
If you have installed the SPARQL Demo VAD locally, it can be found at a URL similar to 'http://example.com:8080/sparql_demo/', the exact form will depend on your local configuration. Alternatively, a live 
version of the documentation is available at <ulink url="http://demo.openlinksw.com/sparql_demo">Virtuoso Demo Server</ulink>.
</para>
<sect4 id="rdfsparqlendpointexamples1"><title>Example SPARQL query issued via curl</title>
<programlisting><![CDATA[
curl -F "query=SELECT DISTINCT ?p FROM <http://demo.openlinksw.com/DAV/home/demo/rdf_sink/> WHERE {?s ?p ?o}" http://demo.openlinksw.com/sparql
]]></programlisting>
    <para>The result should be:</para>
<programlisting><![CDATA[
<?xml version="1.0" ?>
<sparql xmlns="http://www.w3.org/2005/sparql-results#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2001/sw/DataAccess/rf1/result2.xsd">
 <head>
  <variable name="p"/>
 </head>
 <results distinct="false" ordered="true">
  <result>
   <binding name="p"><uri>http://www.w3.org/1999/02/22-rdf-syntax-ns#type</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/nick</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/name</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/homepage</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/knows</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/workplaceHomepage</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/mbox</uri></binding>
  </result>
 </results>
</sparql>
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlendpointexamples2"><title>Other Examples of SPARQL query issued via curl</title>
<para><emphasis>Further example SPARQL queries:</emphasis></para>
<programlisting><![CDATA[
curl -F "query=SELECT DISTINCT ?Concept FROM <http://dbpedia.org> WHERE {?s a ?Concept} LIMIT 10" http://dbpedia.org/sparql
]]></programlisting>
<programlisting><![CDATA[
curl -F "query=SELECT DISTINCT ?Concept FROM <http://example.com/dataspace/person/kidehen> WHERE {?s a ?Concept} LIMIT 10" http://demo.openlinksw.com/sparql
]]></programlisting>
<programlisting><![CDATA[
curl -F "query=SELECT DISTINCT ?Concept FROM <http://data.openlinksw.com/oplweb/product_family/virtuoso> WHERE {?s a ?Concept} LIMIT 10" http://demo.openlinksw.com/sparql
]]></programlisting>
<programlisting><![CDATA[
curl -F "query=SELECT DISTINCT ?Concept FROM <http://openlinksw.com/dataspace/organization/openlink> WHERE {?s a ?Concept} LIMIT 10" http://demo.openlinksw.com/sparql
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlendpointexamples3"><title>Example with curl and SPARQL-WebID endpoint</title>
<programlisting><![CDATA[
$ curl -H "Accept: text/rdf+n3"  --cert test.pem -k https://demo.openlinksw.com/dataspace/person/demo
Enter PEM pass phrase: *****
@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:    <https://demo.openlinksw.com/dataspace/demo/socialnetwork/demo%27s%20AddressBook/1046#> .
@prefix foaf:   <http://xmlns.com/foaf/0.1/> .
ns1:this        rdf:type        foaf:Person .
@prefix ns3:    <http://www.pipian.com/rdf/tami/juliette.n3#> .
ns3:juliette    rdf:type        foaf:Document .
@prefix ns4:    <https://demo.openlinksw.com/dataspace/person/> .
ns4:demo        rdf:type        foaf:PersonalProfileDocument .
@prefix ns5:    <https://demo.openlinksw.com/dataspace/person/demo#> .
@prefix geo:    <http://www.w3.org/2003/01/geo/wgs84_pos#> .
ns5:based_near  rdf:type        geo:Point .
@prefix ns7:    <https://demo.openlinksw.com/dataspace/demo/socialnetwork/demo%27s%20AddressBook/1042#> .
ns7:this        rdf:type        foaf:Person .
ns5:this        rdf:type        foaf:Person .
@prefix ns8:    <https://demo.openlinksw.com/dataspace/person/demo/online_account/> .
@prefix sioc:   <http://rdfs.org/sioc/ns#> .
ns8:demo        rdf:type        sioc:User .
@prefix ns10:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/myAddressBook/1001#> .
ns10:this       rdf:type        foaf:Person .
@prefix ns11:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/demo%27s%20AddressBook/1045#> .
ns11:this       rdf:type        foaf:Person .
@prefix ns12:   <https://demo.openlinksw.com/dataspace/demo#> .
ns12:this       rdf:type        sioc:User .
ns5:org rdf:type        foaf:Organization .
@prefix ns13:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/demo%27s%20AddressBook/1048#> .
ns13:this       rdf:type        foaf:Person .
@prefix ns14:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/myAddressBook/1001#this#> .
ns14:org        rdf:type        foaf:Organization .
@prefix ns15:   <https://demo.openlinksw.com/dataspace/person/imitko#> .
ns15:this       rdf:type        foaf:Person .
@prefix ns16:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/myAddressBook/1049#> .
ns16:this       rdf:type        foaf:Person .
@prefix ns17:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/myAddressBook/1000#> .
ns17:this       rdf:type        foaf:Person .
ns8:MySpace     rdf:type        sioc:User .
@prefix ns18:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/demo%27s%20AddressBook/1044#> .
ns18:this       rdf:type        foaf:Person .
@prefix dc:     <http://purl.org/dc/elements/1.1/> .
ns4:demo        dc:title        "demo demo's FOAF file" .
ns14:org        dc:title        "OpenLink" .
ns5:org dc:title        "OpenLink" .
ns18:this       foaf:name       "Kingsley Idehen" .
ns13:this       foaf:name       "Juliette" .
ns17:this       foaf:name       "Kingsley Idehen" .
ns5:this        foaf:name       "demo demo" .
ns15:this       foaf:name       "Mitko Iliev" .
ns10:this       foaf:name       "test test12" .
@prefix rdfs:   <http://www.w3.org/2000/01/rdf-schema#> .
ns5:this        rdfs:seeAlso    ns4:demo .
ns15:this       rdfs:seeAlso    ns4:imitko .
ns4:demo        foaf:maker      ns5:this .
ns15:this       foaf:nick       "imitko" .
ns7:this        foaf:nick       "Orri Erling" .
ns13:this       foaf:nick       "Juliette" .
ns10:this       foaf:nick       "test1" .
ns5:this        foaf:nick       "demo" .
ns18:this       foaf:nick       "Kingsley" .
ns17:this       foaf:nick       "Kingsley" .
ns16:this       foaf:nick       "test2" .
ns1:this        foaf:nick       "TEST" .
ns11:this       foaf:nick       "TEST" .
ns5:this        foaf:holdsAccount       ns8:demo ,
                ns8:MySpace ,
                ns12:this .
@prefix ns21:   <http://example.com/dataspace/person/imitko#> .
ns5:this        foaf:knows      ns21:this ,
                ns17:this ,
                ns16:this ,
                ns3:juliette ,
                ns10:this ,
                ns7:this .
@prefix ns22:   <http://example.com/dataspace/person/kidehen#> .
ns5:this        foaf:knows      ns22:this ,
                ns18:this ,
                ns11:this ,
                ns1:this .
@prefix ns23:   <http://bblfish.net/people/henry/card#me\u0020> .
ns5:this        foaf:knows      ns23: ,
                ns13:this ,
                ns15:this ;
        foaf:firstName  "demo" ;
        foaf:family_name        "demo" ;
        foaf:gender     "male" ;
        foaf:icqChatID  "125968" ;
        foaf:msnChatID  "45demo78" ;
        foaf:aimChatID  "demo1234" ;
        foaf:yahooChatID        "demo678" ;
        foaf:based_near ns5:based_near .
@prefix ns24:   <http://www.openlinksw.com> .
ns5:this        foaf:workplaceHomepage  ns24: .
ns5:org foaf:homepage   ns24: .
ns5:this        foaf:homepage   ns24: .
ns14:org        foaf:homepage   ns24: .
ns4:demo        foaf:primaryTopic       ns5:this .
ns5:based_near  geo:lat "47.333332" ;
        geo:long        "13.333333" .
@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:    <https://demo.openlinksw.com/dataspace/demo#> .
@prefix foaf:   <http://xmlns.com/foaf/0.1/> .
ns1:this        rdf:type        foaf:OnlineAccount .
@prefix ns3:    <https://demo.openlinksw.com/dataspace/person/demo/online_account/> .
ns3:MySpace     rdf:type        foaf:OnlineAccount .
ns3:demo        rdf:type        foaf:OnlineAccount .
@prefix ns4:    <https://demo.openlinksw.com/dataspace/person/demo#> .
ns4:this        foaf:holdsAccount       ns3:MySpace ,
                ns1:this ,
                ns3:demo .
@prefix vcard:  <http://www.w3.org/2001/vcard-rdf/3.0#> .
ns4:this        vcard:ADR       ns4:addr .
ns4:addr        vcard:Country   "United States" ;
        vcard:Locality  "New York" ;
        vcard:Region    "Nebraska" .
@prefix ns6:    <http://myspace.com> .
ns3:MySpace     foaf:accountServiceHomepage     ns6: .
@prefix ns7:    <skype:demo?> .
ns3:demo        foaf:accountServiceHomepage     ns7:chat ;
        foaf:accountName        "demo" .
ns3:MySpace     foaf:accountName        "MySpace" .
@prefix ns8:    <http://vocab.org/bio/0.1/> .
ns4:this        ns8:olb "this is short resume of user Demo." .
@prefix ns9:    <https://demo.openlinksw.com/dataspace/> .
ns4:this        foaf:openid     ns9:demo ;
        ns8:keywords    "demo, openlinksw, virtuoso, weblog, rdf" .
@prefix foaf:   <http://xmlns.com/foaf/0.1/> .
@prefix ns1:    <https://demo.openlinksw.com/dataspace/demo/subscriptions/> .
@prefix ns2:    <https://demo.openlinksw.com/dataspace/person/demo#> .
ns1:DemoFeeds   foaf:maker      ns2:this .
@prefix ns3:    <https://demo.openlinksw.com/dataspace/demo/community/> .
ns3:demoCommunity       foaf:maker      ns2:this .
@prefix ns4:    <https://demo.openlinksw.com/dataspace/demo/eCRM/demo%27s%20eCRM> .
ns4:    foaf:maker      ns2:this .
@prefix ns5:    <https://demo.openlinksw.com/dataspace/demo/calendar/> .
ns5:mycalendar  foaf:maker      ns2:this .
@prefix ns6:    <https://demo.openlinksw.com/dataspace/demo/photos/> .
ns6:MyGallery   foaf:maker      ns2:this .
@prefix ns7:    <https://demo.openlinksw.com/dataspace/demo/briefcase/> .
ns7:mybriefcase foaf:maker      ns2:this .
@prefix ns8:    <https://demo.openlinksw.com/dataspace/demo/wiki/> .
ns8:ESBWiki     foaf:maker      ns2:this .
@prefix ns9:    <https://demo.openlinksw.com/dataspace/demo/bookmark/> .
ns9:mybookmarks foaf:maker      ns2:this .
@prefix ns10:   <https://demo.openlinksw.com/dataspace/demo/weblog/> .
ns10:myblog     foaf:maker      ns2:this .
@prefix ns11:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/demo%27s%20AddressBook> .
ns11:   foaf:maker      ns2:this .
@prefix ns12:   <https://demo.openlinksw.com/dataspace/demo/community/demo%27s%20Community> .
ns12:   foaf:maker      ns2:this .
ns8:mywiki      foaf:maker      ns2:this .
@prefix ns13:   <https://demo.openlinksw.com/dataspace/demo/eCRM/demo%20demo%27s%20eCRM> .
ns13:   foaf:maker      ns2:this .
@prefix ns14:   <https://demo.openlinksw.com/dataspace/demo/polls/> .
ns14:mypolls    foaf:maker      ns2:this .
@prefix ns15:   <https://demo.openlinksw.com/dataspace/demo/socialnetwork/> .
ns15:myAddressBook      foaf:maker      ns2:this .
ns3:SP2 foaf:maker      ns2:this .
ns2:this        foaf:made       ns11: ,
                ns4: ,
                ns3:demoCommunity ,
                ns12: ,
                ns15:myAddressBook ,
                ns10:myblog ,
                ns9:mybookmarks ,
                ns7:mybriefcase ,
                ns5:mycalendar ,
                ns14:mypolls ,
                ns8:mywiki ,
                ns1:DemoFeeds ,
                ns8:ESBWiki ,
                ns6:MyGallery ,
                ns3:SP2 ,
                ns13: .
@prefix rdfs:   <http://www.w3.org/2000/01/rdf-schema#> .
ns9:mybookmarks rdfs:label      "demo demo's Bookmarks" .
ns15:myAddressBook      rdfs:label      "demo demo's AddressBook" .
ns4:    rdfs:label      "demo demo's eCRM" .
ns12:   rdfs:label      "demo's Community" .
ns14:mypolls    rdfs:label      "demo demo's Polls" .
ns13:   rdfs:label      "demo demo's eCRM Description" .
ns8:mywiki      rdfs:label      "demo demo's Wiki" .
ns7:mybriefcase rdfs:label      "demo demo's Briefcase" .
ns1:DemoFeeds   rdfs:label      "demo demo's Feeds" .
ns10:myblog     rdfs:label      "demo's Weblog" .
ns5:mycalendar  rdfs:label      "demo demo's Calendar" .
ns11:   rdfs:label      "demo demo's AddressBook" .
ns6:MyGallery   rdfs:label      "demo demo's Gallery" .
ns8:ESBWiki     rdfs:label      "demo demo's Wiki" .
ns3:demoCommunity       rdfs:label      "demo demo's Community" .
ns3:SP2 rdfs:label      "demo demo's Community" .
@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:    <https://demo.openlinksw.com/dataspace/person/demo#> .
@prefix ns2:    <http://www.w3.org/ns/auth/rsa#> .
ns1:cert        rdf:type        ns2:RSAPublicKey .
@prefix dc:     <http://purl.org/dc/elements/1.1/> .
@prefix ns4:    <https://demo.openlinksw.com/dataspace/person/demo/projects#ods%20project> .
ns4:    dc:title        "ods project" .
@prefix foaf:   <http://xmlns.com/foaf/0.1/> .
ns4:    foaf:maker      ns1:this .
ns1:this        foaf:made       ns4: .
@prefix ns6:    <http://www.w3.org/ns/auth/cert#> .
ns1:cert        ns6:identity    ns1:this ;
        ns2:modulus     ns1:cert_mod .
ns1:cert_mod    ns6:hex "b8edefa13092d05e85257d6be0aca54218091278583f1d18759c4bced0007948fa6e920018abc3c30b8885d303ec2e679f3a7c15036d38452ddd9ebfcbb41
e1bd08dca66b7737b744fd9e441ebefa425311363711714cd0fe3b334a79ce50be9eb3443193bcbf2f1486481e775382f1a1792a2a8438543ca6f478c3b13c5db2a7f9a12a9a5aed5ec498
6be0169a1859d027170812a28914d158fb76a5933f11777a06c8db64d10f7c02900c4bb4bbf2d24c0e34c6ca135fdb5e05241bc029196ceef13a2006f07d1800f17762c0cfe05b3dac3042
09e1b7a3973122e850e96fcd0396544f82f0b11a46f0d868ba0f3d8efd957e7ef224871905a06c3c5d85ac9" .
ns1:cert        ns2:public_exponent     ns1:cert_exp .
ns1:cert_exp    ns6:decimal     "65537" .
@prefix rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:    <https://demo.openlinksw.com/dataspace/person/demo#> .
@prefix ns2:    <http://vocab.org/bio/0.1/> .
ns1:event       rdf:type        ns2:Birth .
@prefix foaf:   <http://xmlns.com/foaf/0.1/> .
@prefix ns4:    <mailto:demo@openlinksw.com> .
ns1:this        foaf:mbox       ns4: ;
        foaf:birthday   "01-01" .
@prefix dc:     <http://purl.org/dc/elements/1.1/> .
ns1:event       dc:date "1968-01-01" .
ns1:this        ns2:event       ns1:event .
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlendpointexamples4"><title>Example with curl and SPARQL-OAuth endpoint</title>
<para>Note: this is just an example as token had expired already. You can go to <link linkend="sparqloauthendpoint">this</link> section to see how to interact with our Virtuoso UI.</para>
<programlisting><![CDATA[
$ curl "http://demo.openlinksw.com/oauth/sparql.vsp?debug=on&default-graph-uri=&format=text%2Fhtml&oauth_consumer_key=27f105a327f5f23163e0636f78901
8dacdd70bb5&oauth_nonce=a14d43339fcb2638&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1242106643&oauth_token=42e2af4d9264ef42521c1010aff99f60a8
ee95a2&oauth_version=1.0&query=select%20distinct%20%3FURI%20%3FObjectType%20where%20%7B%3FURI%20a%20%3FObjectType%7D%20limit%2050&oauth_signature=C
w9yJ2saU1vgHuFxWcughai5cZY%3D"
<table class="sparql" border="1">
  <tr>
    <th>URI</th>
    <th>ObjectType</th>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-nonblank</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-nonblank-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarchar</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarchar-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarbinary</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarbinary-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-uri</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-uri-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-uri</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-uri-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-doubleprecision</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-doubleprecision-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-date</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-date-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-datetime</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-datetime-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#multipart-uri</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#multipart-uri-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#multipart-uri-fn-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#multipart-literal-fn-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-uri-fn</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-uri-fn-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-uri-fn</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-uri-fn-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-literal-fn</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-literal-fn-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-literal-fn</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-integer-literal-fn-nullable</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.w3.org/1999/02/22-rdf-syntax-ns#type</td>
    <td>http://www.w3.org/1999/02/22-rdf-syntax-ns#Property</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-nullable-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-nonblank-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-iid-nonblank-nullable-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#default-nullable-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-nullable-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarchar-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarchar-nullable-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarbinary-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-longvarbinary-nullable-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
  <tr>
    <td>http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-uri-SuperFormats</td>
    <td>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</td>
  </tr>
</table>
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlendpointexamples5"><title>Example with CONSTRUCT</title>
<para>Go to the sparql endpoint UI: i.e. go to http://host:port/sparql</para>
<para>For the Default Graph URI enter: http://www.w3.org/2001/sw/DataAccess/proto-tests/data/construct/simple-data.rdf</para>
<para>Select "Retrieve remote RDF data for all missing source graphs".</para>
<para>For the query text enter:</para>
<programlisting>
SELECT * WHERE {?s ?p ?o}
</programlisting>
<para>Click the "Run Query" button.</para>
<para>The query results, shown below, are cached locally ( network resources being fetched ). The remote RDF data is saved in the local RDF quad store as  graph http://www.w3.org/2001/sw/DataAccess/proto-tests/data/construct/simple-data.rdf</para>
<programlisting>
s  	                                  p  	                                           o
http://www.example/jose/foaf.rdf#jose 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/nick 	           Jo
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/name 	           Jose Jimen~ez
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/knows 	           http://www.example/jose/foaf.rdf#juan
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/homepage 	           http://www.example/jose/
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/workplaceHomepage 	   http://www.corp.example/
http://www.example/jose/foaf.rdf#kendall  http://xmlns.com/foaf/0.1/knows                  http://www.example/jose/foaf.rdf#edd
http://www.example/jose/foaf.rdf#julia 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#julia 	  http://xmlns.com/foaf/0.1/mbox 	           mailto:julia@mail.example
http://www.example/jose/foaf.rdf#juan 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#juan 	  http://xmlns.com/foaf/0.1/mbox 	           mailto:juan@mail.example
</programlisting>
<para>Now let's take the CONSTRUCT query:</para>
<programlisting><![CDATA[
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX myfoaf: <http://www.example/jose/foaf.rdf#>
CONSTRUCT
  { myfoaf:jose foaf:depiction <http://www.example/jose/jose.jpg>.
    myfoaf:jose foaf:schoolHomepage <http://www.edu.example/>.
    ?s ?p ?o.
  }
FROM <http://www.w3.org/2001/sw/DataAccess/proto-tests/data/construct/simple-data.rdf>
WHERE
  {
    ?s ?p ?o. myfoaf:jose foaf:nick "Jo".
    FILTER ( ! (?s = myfoaf:kendall && ?p = foaf:knows && ?o = myfoaf:edd )
    && ! ( ?s = myfoaf:julia && ?p = foaf:mbox && ?o = <mailto:julia@mail.example> )
    && ! ( ?s = myfoaf:julia && ?p = rdf:type && ?o = foaf:Person))
  }
]]></programlisting>
<para>From an HTTP client, issue the GET command with the above query added as a URL-encoded parameter value:</para>
<programlisting><![CDATA[
GET -e -s http://host:port/sparql/?query=PREFIX+rdf%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23%3E%0D%0APREFIX+foaf%3A+%3Chttp%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2F%3E%0D%0APREFIX+myfoaf%3A+%3Chttp%3A%2F%2Fwww.example%2Fjose%2Ffoaf.rdf%23%3E%0D%0A%0D%0ACONSTRUCT+%7B+myfoaf%3Ajose+foaf%3Adepiction+%3Chttp%3A%2F%2Fwww.example%2Fjose%2Fjose.jpg%3E.%0D%0A++++++++++++myfoaf%3Ajose+foaf%3AschoolHomepage+%3Chttp%3A%2F%2Fwww.edu.example%2F%3E.%0D%0A++++++++++++%3Fs+%3Fp+%3Fo.%7D%0D%0AFROM+%3Chttp%3A%2F%2Fwww.w3.org%2F2001%2Fsw%2FDataAccess%2Fproto-tests%2Fdata%2Fconstruct%2Fsimple-data.rdf%3E%0D%0AWHERE+%7B+%3Fs+%3Fp+%3Fo.+myfoaf%3Ajose+foaf%3Anick+%22Jo%22.%0D%0A+++++++FILTER+%28+%21+%28%3Fs+%3D+myfoaf%3Akendall+%26%26+%3Fp+%3D+foaf%3Aknows+%26%26+%3Fo+%3D+myfoaf%3Aedd+%29%0D%0A++++++++++++++%26%26+%21+%28+%3Fs+%3D+myfoaf%3Ajulia+%26%26+%3Fp+%3D+foaf%3Ambox+%26%26+%3Fo+%3D+%3Cmailto%3Ajulia%40mail.example%3E+%29%0D%0A++++++++++%26%26+%21+%28+%3Fs+%3D+myfoaf%3Ajulia+%26%26+%3Fp+%3D+rdf%3Atype+%26%26+%3Fo+%3D+foaf%3APerson%29%29%0D%0A%7D%0D%0A&format=application%2Frdf%2Bxml
]]></programlisting>
<para>The request response will be similar to:</para>
<programlisting><![CDATA[
200 OK
Connection: close
Date: Fri, 28 Dec 2007 10:06:14 GMT
Accept-Ranges: bytes
Server: Virtuoso/05.00.3023 (Win32) i686-generic-win-32  VDB
Content-Length: 2073
Content-Type: application/rdf+xml; charset=UTF-8
Client-Date: Fri, 28 Dec 2007 10:06:14 GMT
Client-Peer: 83.176.40.177:port
Client-Response-Num: 1

<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#juan"><ns0pred:mbox xmlns:ns0pred="http://xmlns.com/foaf/0.1/" rdf:resource="mailto:juan@mail.example"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:schoolHomepage xmlns:ns0pred="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.edu.example/"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#" rdf:resource="http://xmlns.com/foaf/0.1/Person"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:homepage xmlns:ns0pred="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.example/jose/"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#juan"><ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#" rdf:resource="http://xmlns.com/foaf/0.1/Person"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:workplaceHomepage xmlns:ns0pred="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.corp.example/"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:nick xmlns:ns0pred="http://xmlns.com/foaf/0.1/">Jo</ns0pred:nick></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:depiction xmlns:ns0pred="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.example/jose/jose.jpg"/></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:name xmlns:ns0pred="http://xmlns.com/foaf/0.1/">Jose Jime?+ez</ns0pred:name></rdf:Description>
<rdf:Description rdf:about="http://www.example/jose/foaf.rdf#jose"><ns0pred:knows xmlns:ns0pred="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.example/jose/foaf.rdf#juan"/></rdf:Description>
</rdf:RDF>
Done
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlendpointexamples6"><title>Example with extraction part of literal as variable</title>
<para>The following example shows how to extract a part of a literal as a variable for
use in a numeric comparison using SPARQL</para>
<para>Suppose there are the following triples inserted:</para>
<programlisting><![CDATA[
SQL>SPARQL INSERT INTO GRAPH <http://mygraph.com> {  <:a>
                                                     <:p>
                                                     "123 abc" };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 30 msec.
SQL>SPARQL INSERT INTO GRAPH <http://mygraph.com> {  <:a>
                                                     <:p>
                                                     "234 abc" };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 0 msec.
]]></programlisting>
<para>In order to extract the numeric part, and then do a numeric (&lt;.&gt;,=), you can use
atoi (), atol or atof in the filter:</para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://mygraph.com>
WHERE
  {
    ?s ?p ?o . filter (bif:atoi (?o) > 130)
  };
s        p         o
VARCHAR  VARCHAR   VARCHAR
___________________________________

:a       :p        234 abc

1 Rows. -- 10 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlendpointexamples7"><title>Example how to define rule</title>
See details <link linkend="rdfsparqlruleexamples">here</link> how to define rule context that is
initialized from the contents of a given graph.
</sect4>
</sect3>

<sect3 id="rdfsparqlendpointimplnotes"><title>Implementation Notes</title>
<para>This service has been implemented using <ulink url="http://docs.openlinksw.com/virtuoso">Virtuoso Server</ulink>.</para>
</sect3>

<sect3 id="rdftables"><title>Virtuoso 'Semantic Bank' End Point</title>
    <para>
	<emphasis>What is Piggy Bank?</emphasis>
  </para>
    <para>
	Piggy Bank is an extension to the Firefox Web browser that turns it into a Semantic Web browser, letting you make use of existing information on the Web in more useful and flexible ways not offered by the original Web sites.
  </para>
    <para></para>
    <para>
	<emphasis>What is Semantic Bank?</emphasis>
  </para>
    <para>
	Semantic Bank is the server companion of Piggy Bank that lets you persist, share and publish data collected by individuals, groups or communities. Here is a screen shot of one in action:
  </para>
    <para>
	<emphasis>What can I do with this?</emphasis>
  </para>
    <para>
	A Semantic Bank allows you to:
       </para>
    <itemizedlist mark="bullet" spacing="compact">
    <listitem>
      Persist your information remotely on a server - This is useful, for example, if you want to share data between two of your computers or to avoid losing it due to mistakes or failure.
    </listitem>
    <listitem>
	Share information with other people - The ability to tag resources creates a powerful serendipitous categorization (as proven by things like del.icio.us or Flickr).
    </listitem>
    <listitem>
	Lets you publish your information - Both in the "pure" RDF form (for those who know how to make use of it) or to regular web pages, with the usual Longwell faceted browsing view of it
    </listitem>
    </itemizedlist>
    <para>
	<emphasis>How can I help?</emphasis>
       </para>
    <para>
	Semantic Bank is Open Source software and built around the spirit of open participation and collaboration.
       </para>
    <para>
	There are several ways you can help:
       </para>
    <itemizedlist mark="bullet" spacing="compact">
      <listitem>Install a Semantic Bank and let us know about it, so that we can update the list of available Semantic Banks.</listitem>
      <listitem>Subscribe to our mailing lists to show your interest and give us feedback</listitem>
      <listitem>Report problems and ask for new features through our issue tracking system.</listitem>
      <listitem>Send us patches or fixes to the code</listitem>
    </itemizedlist>
    <para>
	<emphasis>Licensing and Legal Issues</emphasis>
       </para>
    <para>
	Semantic Bank is open source software and is licensed under the BSD license.
       </para>
    <para>
	<emphasis>Note</emphasis>, however, that this software ships with libraries that are not released under the same license; that we interpret their licensing terms to be compatible with ours and that we are redistributing them unmodified. For more information on the licensing terms of the libraries Semantic Bank depends on, please refer to the source code.
    </para>
    <para>
	<emphasis>Download location:</emphasis>
    </para>
    <para>
	<ulink url="http://simile.mit.edu/dist/semantic-bank/">"http://simile.mit.edu/dist/semantic-bank/</ulink>
    </para>
    <para>
	<emphasis>The Virtuoso Semantic Bank End Point</emphasis>
    </para>
    <para>
	Before you can publish, you must register with one or more Semantic Banks:
    </para>
    <itemizedlist mark="bullet" spacing="compact">
      <listitem>Invoke the menu command Tools &gt; Piggy Bank &gt; My Semantic Bank Accounts ...</listitem>
      <listitem>Click Add... in the Semantic Bank Accounts dialog box.</listitem>
      <listitem>In the popup dialog box, type in the URL to the Virtuoso Semantic Bank you want to register with. Example: http://server_name:server_port/bank</listitem>
      <listitem>Enter the account of a valid Virtuoso DAV user. (Note: currently we do not use encryption during authentication; do not use your precious password here.)</listitem>
      <listitem>Click OK, wait for the account to be registered, and then dismiss the Semantic Bank Accounts dialog box.</listitem>
      <listitem>To publish an item, just click the corresponding Publish button (much like how you save the item). To publish all the items being viewed, click the Publish All button.</listitem>
    </itemizedlist>
    <para>
	<emphasis>What is the graph name used by Virtuoso for the triples from PiggyBank?</emphasis>
    </para>
    <para>
	http://simile.org/piggybank/&lt;piggybank-generated-name&gt;
    </para>
    <para>
	The piggybank-generated-name is a Virtuoso DAV user ID.
       </para>
</sect3>
<sect3 id="rdfsparqlexnpointnorthwindexample"><title>Making Linked Data Views Dereferenceable - Northwind Example</title>
<para>Consider an application that makes some relational data available for SPARQL requests, as described in the <link linkend="rdfviewnorthwindexample1">first part of the Northwind Linked Data View example</link>. This may be sufficient for some clients but the IRIs of the described subjects are not dereferenceable.
This means that external SPARQL processors cannot retrieve that data using the Virtuoso Sponger or the like. It also means that if some external resources refer to the IRI of 
some Northwind subject and a user browses that resource then he cannot look at the application's data by clicking on the subject link.</para>
<para>To make RDF access complete, applications can do the following:</para>
<orderedlist>
<listitem>Create a virtual directory</listitem>
<listitem>Instruct the server how to prepare RDF resources on demand</listitem>
<listitem>Configure rendering of RDF resources for non-RDF clients (including Web search engines)</listitem>
<listitem>Make the used ontology available</listitem>
<listitem>Provide an index or sitemap page to help users who try to browse published data but do not know the proper URLs</listitem>
</orderedlist>
<para>The following sequence of operations demonstrates how to implement the listed features without writing any special web pages.
All requests (except the application-specific index/sitemap) will be handled by existing web service endpoints.</para>
<para>As a precaution, we erase any URL rewriting rule lists created by this example that may be in the database following a previous run of the script.</para>
<programlisting><![CDATA[
DB.DBA.URLREWRITE_DROP_RULELIST ('demo_nw_rule_list1', 1)
;
]]></programlisting>
<para>Do the same for individual rewrite rules:</para>
<programlisting><![CDATA[
DB.DBA.URLREWRITE_DROP_RULE ('demo_nw_rule1', 1)
;
DB.DBA.URLREWRITE_DROP_RULE ('demo_nw_rule2', 1)
;
DB.DBA.URLREWRITE_DROP_RULE ('demo_nw_rule3', 1)
;
DB.DBA.URLREWRITE_DROP_RULE ('demo_nw_rule4', 1)
;
]]></programlisting>
<para>As a sanity check we ensure that there are no other similarly named rules:</para>
<programlisting><![CDATA[
SQL>SELECT signal ('WEIRD', sprintf ('Rewrite rule "%s" found', URR_RULE))
FROM DB.DBA.URL_REWRITE_RULE WHERE URR_RULE like 'demo_nw%'
;

]]></programlisting>
<para>Next we create URI rewrite rules based on regular expressions by calling <link linkend="fn_urlrewrite_create_regex_rule"><function>DB.DBA.URLREWRITE_CREATE_REGEX_RULE</function></link>, so the same path will be redirected to different places depending on the MIME types the client can accept.</para>
<para>
For a given input path, that is a URI identifying a particular Linked Data entity, the rewrite rule below generates an N3 or RDF/XML representation of the entity using a CONSTRUCT
query. (Note: In the regular expression identifying the Accept: MIME types this rule applies to, i.e. in rdf.n3 and rdf.xml, each period (.) replaces a literal character
because some SPARQL web clients published before the relevant W3C recommendations produce slightly incorrect &quot;Accept:&quot; strings.)
</para>
<programlisting><![CDATA[
SQL>DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
    'demo_nw_rule2',
    1,
    '(/[^#]*)',
    vector('path'),
    1,
    '/sparql?query=CONSTRUCT+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}+FROM+%%3Chttp%%3A//^{URIQADefaultHost}^/Northwind%%3E+WHERE+{+%%3Chttp%%3A//^{URIQADefaultHost}^%U%%23this%%3E+%%3Fp+%%3Fo+}&format=%U',
    vector('path', 'path', '*accept*'),
    null,
    '(text/rdf.n3)|(application/rdf.xml)',
    0,
    null
    );
]]></programlisting>
<note><para>The request URL for the SPARQL web service looks terrible because it is URL-encoded; the sprintf format string for it is even worse! The easiest way of composing 
encoded strings of this sort is to use the Conductor UI for configuring the rewrite rules. Alternatively open the SPARQL endpoint page (assuming it supports a UI for entering queries, if no query string is specified), type in the desired CONSTRUCT or DESCRIBE statement into the web form (using some sample URI), execute it, cut the URL of the page with results
from the address line of the browser window, paste it into the script and then replace the host name with
<emphasis>^{URIQADefaultHost}^</emphasis>, every percent with double percent, the parts of the sample IRI to be substituted with <emphasis>%U</emphasis>; finally adjust the vector of replacement parameters so that its length is equal to the number of <emphasis>%U</emphasis> or other format specifiers in the template.</para></note>
<para>The next rule redirects to the RDF browser service to display a description of the subject URI and let the user explore related subjects.</para>
<programlisting><![CDATA[
SQL>DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
    'demo_nw_rule1',
    1,
    '(/[^#]*)',
    vector('path'),
    1,
    '/rdfbrowser/index.html?uri=http%%3A//^{URIQADefaultHost}^%U%%23this',
    vector('path'),
    null,
    '(text/html)|(\\*/\\*)',
    0,
    303
    );
]]></programlisting>
<para>This next rule removes any trailing slash from the input path. Note that <emphasis>\x24</emphasis> is the hex character code for the 
end-of-line pattern <emphasis>$</emphasis>. It is written escaped because the dollar sign indicates the beginning of macro in ISQL.</para>
<programlisting><![CDATA[
SQL>DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
    'demo_nw_rule3',
    1,
    '(/[^#]*)/\x24',
    vector('path'),
    1,
    '%s',
    vector('path'),
    null,
    null,
    0,
    null
    );
]]></programlisting>
<para>To configure the server to furnish the ontology underpinning the example Northwind Linked Data View, the procedure LOAD_NW_ONTOLOGY_FROM_DAV, listed
below, takes the ontology described in file /DAV/VAD/demo/sql/nw.owl and loads it into graph http://demo.openlinksw.com/schemas/NorthwindOntology/1.0/
in the local quad store. A rewrite rule is then created to query this graph when the input path identifies entities from this ontology.
</para>
<programlisting><![CDATA[
SQL>create procedure DB.DBA.LOAD_NW_ONTOLOGY_FROM_DAV()
{
  declare content1, urihost varchar;
  SELECT cast (RES_CONTENT as varchar) INTO content1 from WS.WS.SYS_DAV_RES WHERE RES_FULL_PATH = '/DAV/VAD/demo/sql/nw.owl';
  DB.DBA.RDF_LOAD_RDFXML (content1, 'http://demo.openlinksw.com/schemas/northwind#', 'http://demo.openlinksw.com/schemas/NorthwindOntology/1.0/');
  urihost := cfg_item_value(virtuoso_ini_path(), 'URIQA','DefaultHost');
  if (urihost = 'demo.openlinksw.com')
  {
    DB.DBA.VHOST_REMOVE (lpath=>'/schemas/northwind');
    DB.DBA.VHOST_DEFINE (lpath=>'/schemas/northwind', ppath=>'/DAV/VAD/demo/sql/nw.owl', vsp_user=>'dba', is_dav=>1, is_brws=>0);
    DB.DBA.VHOST_REMOVE (lpath=>'/schemas/northwind#');
    DB.DBA.VHOST_DEFINE (lpath=>'/schemas/northwind#', ppath=>'/DAV/VAD/demo/sql/nw.owl', vsp_user=>'dba', is_dav=>1, is_brws=>0);
  }
};

DB.DBA.LOAD_NW_ONTOLOGY_FROM_DAV();

drop procedure DB.DBA.LOAD_NW_ONTOLOGY_FROM_DAV;

DB.DBA.URLREWRITE_CREATE_REGEX_RULE (
    'demo_nw_rule4',
    1,
    '/schemas/northwind#(.*)',
    vector('path'),
    1,
    '/sparql?query=DESCRIBE%20%3Chttp%3A//demo.openlinksw.com/schemas/northwind%23%U%3E%20FROM%20%3Chttp%3A//demo.openlinksw.com/schemas/NorthwindOntology/1.0/%3E',
    vector('path'),
    null,
    '(text/rdf.n3)|(application/rdf.xml)',
    0,
    null
    );
]]></programlisting>
<para>Next we define virtual directory <emphasis>/Northwind</emphasis> and associate with this a rulelist containing the URL rewriting rules defined 
above. Requests matching the rewriting rules should then be properly redirected to produce the requested data. Attempts to access the virtual directory 
root will execute the application's default VSP page, namely <emphasis>sfront.vspx</emphasis>.</para>
<programlisting><![CDATA[
SQL>DB.DBA.URLREWRITE_CREATE_RULELIST (
    'demo_nw_rule_list1',
    1,
    vector (
                'demo_nw_rule1',
                'demo_nw_rule2',
                'demo_nw_rule3',
                'demo_nw_rule4'
          ));

VHOST_REMOVE (lpath=>'/Northwind');
DB.DBA.VHOST_DEFINE (lpath=>'/Northwind', ppath=>'/DAV/home/demo/', vsp_user=>'dba', is_dav=>1, def_page=>'sfront.vspx',
          is_brws=>0, opts=>vector ('url_rewrite', 'demo_nw_rule_list1'));
]]></programlisting>
<para>Finally, to register the namespace prefix <emphasis>northwind</emphasis> as persistent we execute:</para>
<programlisting><![CDATA[
SQL>DB.DBA.XML_SET_NS_DECL ('northwind', 'http://demo.openlinksw.com/schemas/northwind#', 2);
]]></programlisting>
</sect3>
<sect3 id="rdfproxyservice"><title>Sponger Proxy URI Service</title>
<para>
In certain cases, such as Ajax applications, it's prohibited to issue HTTP requests to a server other than the original server.
In other cases it is necessary to transform the content of a target to an RDF format. To this end Virtuoso Server provides a Sponger Proxy URI Service. 
This service takes as an argument a target URL and may return the target's content "as is" or the Sponger may try to transform the content and return 
an RDF representation of the target. When transforming to RDF, the RDF format (RDF/XML, N3, TURTLE etc) of the output can be forced by a URL parameter 
or by content negotiation.
</para>
<para>
When the cartridges_dav.vad package is installed, Virtuoso reserves the path '/about/[id|html|data|rdf]/http/' for the RDF
proxy service. In the current implementation, Virtuoso defines virtual directories for HTTP requests that
come to the port specified as 'ServerPort' in the '[HTTPServer]' section of Virtuoso configuration file
and refer to the above path string. So, if the Virtuoso installation on host example.com listens for HTTP
requests on port 8080, client applications should use the 'service endpoint' string equal to
'http://example.com:8080/about/[id|html|data|rdf]/http/'.
</para>
<para>
If the cartridges_dav.vad VAD package is not installed, then the path '/proxy/rdf/' is used for the Sponger Proxy URI Service.
</para>
<para>
The old pattern for the Sponger Proxy URI Service, '/proxy/', is now deprecated.
</para>
<para>
<emphasis>Note:</emphasis> If you do not have the cartridges package installed, in order for the Sponger Proxy URI Service to work correctly,
you must grant the SPARQL_UPDATE role to user SPARQL and grant execute permission on procedure RDF_SPONGE_UP.
</para>
<para>
To enable SPARQL_UPDATE using the Conductor UI:
</para>
<orderedlist>
  <listitem>Go to the Virtuoso Administration Conductor i.e. http://host:port/conductor</listitem>
  <listitem>Login as dba user</listitem>
  <listitem>Go to System Admin->User Accounts->Roles</listitem>
    <figure id="rl1" float="1">
      <title>Conductor UI</title>
      <graphic fileref="ui/cn1.png"/>
    </figure>
  <listitem>Click the link "Edit" for "SPARQL_UPDATE</listitem>
  <listitem>Select from the list of available user/groups "SPARQL" and click the ">>" button so to add it to the right-positioned list.</listitem>
    <figure id="rl2" float="1">
      <title>Conductor UI</title>
      <graphic fileref="ui/cn2.png"/>
    </figure>
  <listitem>Click the button "Update".</listitem>
</orderedlist>
    <para>
       To grant execute permission on RDF_SPONGE_UP:
    </para>
<programlisting><![CDATA[
grant execute on DB.DBA.RDF_SPONGE_UP to "SPARQL";
]]></programlisting>
    <para>
	When invoked with a URL of the form http://host:port/proxy?..., the Sponger Proxy URI Service accepts the following query string parameters:
    </para>
    <itemizedlist mark="bullet" spacing="compact">
	<listitem><emphasis>force</emphasis> - if 'rdf' is specified, the Sponger will try to extract RDF data from the target and return it</listitem>
	<listitem><emphasis>header</emphasis> - HTTP headers to be sent to the target</listitem>
	<listitem><emphasis>output-format</emphasis> - if 'force=rdf' is given, designates the desired output MIME type of the RDF data. The default is 'rdf+xml'. Other supported MIME types are 'n3', 'turtle' or 'ttl'.</listitem>
    </itemizedlist>
    <para>
	When RDF data is requested and 'output-format' is not specified, the result will be serialized with a MIME type determined by the request 'Accept' headers i.e. the proxy service will do content negotiation.
    </para>
    <para>Example: RDF file with URL: http://www.w3.org/People/Berners-Lee/card</para>
<programlisting><![CDATA[
-- Access the url in order to view the result in HTML format:
http://host:port/about/html/http/www.w3.org/People/Berners-Lee/card
-- Access the url in order to view the result in RDF:
http://host:port/about/rdf/http://www.w3.org/People/Berners-Lee/card
-- or use the following proxy invocation style:
http://host:port/proxy/rdf/http://www.w3.org/People/Berners-Lee/card
-- or this one:
http://host:port/proxy?url=http://www.w3.org/People/Berners-Lee/card&force=rdf
]]></programlisting>
	<para>Note: It is not permitted, when using the style http://host:port/proxy/rdf, to pass URL query string parameters to the proxy. </para>
<para>Now go to the SPARQL endpoint, i.e. http://host:port/sparql</para>
<para>For the 'Default Graph URI' enter the URL of the RDF file:  http://www.w3.org/People/Berners-Lee/card</para>
<para>For 'Query' enter:</para>
<programlisting><![CDATA[
SELECT *
WHERE
  {
    ?s ?p ?o
  }
]]></programlisting>
<para>Query result:</para>
<programlisting><![CDATA[
s  	                                        p  	                                           o
http://www.w3.org/People/Berners-Lee/card 	http://www.w3.org/1999/02/22-rdf-syntax-ns#type    http://xmlns.com/foaf/0.1/PersonalProfileDocument
http://www.w3.org/People/Berners-Lee/card 	http://purl.org/dc/elements/1.1/title 	           Tim Berners-Lee's FOAF file
http://www.w3.org/People/Berners-Lee/card 	http://creativecommons.org/ns#license 	           http://creativecommons.org/licenses/by-nc/3.0/
http://www.w3.org/People/Berners-Lee/card 	http://xmlns.com/foaf/0.1/maker 	           http://www.w3.org/People/Berners-Lee/card#i
etc ...
]]></programlisting>
</sect3>
<sect3 id="sparqliniservice"><title>SPARQL INI service</title>
    <para>
	The <link linkend="ini_SPARQL">[SPARQL] section</link> of the virtuoso.ini configuration file sets parameters and limits for the SPARQL query web service.
	The values contained in the [SPARQL] section can be exposed in RDF form via the URL pattern http://cname/sparql?ini
    </para>
    <para>Example: http://demo.openlinksw.com/sparql?ini</para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:MaxQueryCostEstimationTime xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">1000</ns0pred:MaxQueryCostEstimationTime></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:ExternalXsltSource xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">1</ns0pred:ExternalXsltSource></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:DefaultQuery xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">SELECT ?Subject ?Concept WHERE {?Subject a ?Concept}</ns0pred:DefaultQuery></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:ResultSetMaxRows xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">100000</ns0pred:ResultSetMaxRows></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:MaxQueryExecutionTime xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">30</ns0pred:MaxQueryExecutionTime></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:ExternalQuerySource xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">1</ns0pred:ExternalQuerySource></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:DefaultGraph xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">http://demo.openlinksw.com/dataspace/person/demo</ns0pred:DefaultGraph></rdf:Description>
<rdf:Description rdf:about="http://www.openlinksw.com/schemas/virtini#SPARQL"><ns0pred:PingService xmlns:ns0pred="http://www.openlinksw.com/schemas/virtini#">http://rpc.pingthesemanticweb.com/</ns0pred:PingService></rdf:Description>
</rdf:RDF>
]]></programlisting>
</sect3>
<sect3 id="sparqlexcel"><title>SPARQL Endpoint with Excel MIME Type Output Option</title>
<para>The SPARQL endpoint offers an Excel MIME type output option.</para>
<para>From http://cname:host/sparql, select "Spreadsheet" for the "Display Results As:" option and click the "Run Query" button.</para>
<figure id="sparqlexcel1" float="1">
      <title>SPARQL Endpoint with Excel MIME type output</title>
      <graphic fileref="ui/Excel1.png"/>
    </figure>
<para>The resulting query string contains a format parameter value of <emphasis>"application/vnd.ms-excel"</emphasis>. For example,
<ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=http%3A%2F%2Fdemo.openlinksw.com%2Fdataspace%2Fperson%2Fdemo&amp;should-sponge=&amp;query=select+*%0D%0Awhere+%7B%3Fs+%3Fp+%3Fo%7D%0D%0Alimit+10&amp;format=application%2Fvnd.ms-excel&amp;debug=on">A URL such as this one</ulink> will be generated, and can be opened directly with Excel.</para>
<figure id="sparqlexcel2" float="1">
      <title>SPARQL Endpoint with Excel MIME type output</title>
      <graphic fileref="ui/Excel2.png"/>
    </figure>
</sect3>
<sect3 id="sparqljson"><title>SPARQL Endpoint with RDF+JSON Output: SPARQL UI Example</title>
<para>The SPARQL endpoint also offers a RDF+JSON output option.</para>
<para>From http://cname:host/sparql select "JSON" for "Display Results As:"  and click the "Run Query" button.</para>
<figure id="sparqljson1" float="1">
      <title>SPARQL Endpoint with RDF+JSON output</title>
      <graphic fileref="ui/JSON1.png"/>
    </figure>
<para>As result URL containing as parameter the format <emphasis>application/sparql-results+json</emphasis> will be generated and the content should look like:</para>
<figure id="sparqljson2" float="1">
      <title>SPARQL Endpoint with JSON+RDF</title>
      <graphic fileref="ui/JSON2.png"/>
    </figure>
</sect3>
<sect3 id="sparqljsonp"><title>SPARQL Endpoint with JSON/P Output Option: Curl Example</title>
<para>The SPARQL endpoint also offers a  JSON/P output option.</para> 
<para>The SPARQL endpoint accepts a 'callback' URL parameter and in this case 
when parameter 'format' is 'json', then it will produce JSON/P output.</para>
<programlisting><![CDATA[
$ curl "http://lod.openlinksw.com/sparql?query=select+*+where+\{+%3Fx+a+%3Fz+.+\}+limit+10&format=json&debug=on&callback=func"
func(

{ "head": { "link": [], "vars": ["x", "z"] },
  "results": { "distinct": false, "ordered": true, "bindings": [
    { "x": { "type": "bnode", "value": "nodeID://b196899188" }  , "z": { "type": "uri", "value": "http://www.w3.org/2000/10/swap

    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2005/04/sparqlette/#profile" } , "z": { "type": "uri", "value":
services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/#b-profile" }   , "z": { "type": "uri",
aml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/#a-profile" }   , "z": { "type": "uri",
aml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/index.rdf#a-profile" }  , "z": { "type":
://www.daml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/index.rdf#b-profile" }  , "z": { "type":
://www.daml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2006/06/blogmatrix/#profile" } , "z": { "type": "uri", "value":
services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/#b-profile" }   , "z": { "type": "uri",
aml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/#a-profile" }   , "z": { "type": "uri",
aml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }},
    { "x": { "type": "uri", "value": "http://www.wasab.dk/morten/2003/12/nearestAirport/#b-profile" }   , "z": { "type": "uri",
aml.org/services/owl-s/1.1/Service.owl#ServiceProfile" }} ] } })
]]></programlisting>
</sect3>
</sect2>
<sect2 id="sparqldebug"><title>Troubleshooting SPARQL Queries</title>
<para>A short SPARQL query can be compiled into a long SQL statement,
especially if data comes from many quad map patterns. A moderately sized
application with 50 tables and 10 columns per table may create
thousands of quad map patterns for subjects spanning hundreds of different
types. An attempt to &quot;select everything&quot; from Linked Data View of
that complexity may easily create 5000 lines of SQL code. Thus it is
to be expected that some queries will be rejected even if the same
queries would work fine if the RDF data were held as physical quads in default storage, rather than 
synthesized through an Linked Data View.
</para>

<para>In addition, the SQL compiler catches typos efficiently, signalling an error if a table or column
name is unknown, efficiently catching typos. SPARQL uses IRIs that are
long and sometimes unreadable, but there is no &quot;closed
world&quot; schema of the data so a typo in an IRI is not an error; it
is simply some other IRI. So a typo in an IRI or in a namespace prefix
causes missing bindings of some triple patterns of the query and an
incomplete result, but usually no errors are reported. A typo in graph
or predicate IRI may cause the SPARQL compiler to generate code that
accesses default (quad) storage instead of a relational source or generate
empty code that accesses nothing.</para>

<para>The SQL compiler does not signal casting errors when it runs the
statement generated from SPARQL, because the generated SQL code
contains <emphasis>option (QUIETCAST)</emphasis>. This means that
mismatches between expected and actual datatypes of values stay
invisible and may cause rounding errors (e.g. integer division instead
of floating-point) and even empty joins (due to join conditions that
silently return NULL instead of returning a comparison error).</para>

<para>In other words, SPARQL queries are so laconic that there is no
room for details that let the compiler distinguish between intent
and a bug. This masks query complexity, misuse of names and
type mismatches. One may make debugging easier by making queries
longer.</para>

<para>Two very helpful debugging tools are automatic void variable
recognition and plain old code inspection. &quot;Automatic&quot; means
&quot;cheap&quot; so the very first step of debugging is to ensure
that every triple pattern of the query may in principle return
something.  This helps in finding typos when the query gets data from
Linked Data Views. It also helps when a query tries to join two disjoint
sorts of subjects. If the <emphasis>define sql:signal-void-variables
1</emphasis> directive is placed in the preamble of the SPARQL query,
the compiler will signal an error if it finds any triple pattern that cannot 
bind variables or any variable that is proved to be always
unbound. This is especially useful when data are supposed to come from an
<emphasis>option (exclusive)</emphasis> or <emphasis>option (soft
exclusive)</emphasis> quad map. Without one of these options, the SPARQL
compiler will usually bind variables using &quot;physical quads&quot;;
the table of physical quads may contain any rows that match any given
triple pattern; thus many errors will remain undiscovered. If the name
of a quad map pattern is known then it is possible to force the SPARQL
compiler to use only that quad map for the whole query or a
part of the query. This is possible by using the following syntax:</para>

<programlisting><![CDATA[
QUAD MAP quad-map-name { group-pattern }
]]></programlisting>
<para>If some triple pattern inside <emphasis>group-pattern</emphasis> cannot be bound using <emphasis>quad-map-name</emphasis> or one of its descendants then <emphasis>define sql:signal-void-variables 1</emphasis> will force the compiler to signal the error.</para>

<note><para>Although it is technically possible to use <emphasis>QUAD
MAP</emphasis> to improve the performance of a query that tries to
access redundant Linked Data Views, it is much better to achieve the same
effect by providing a more restrictive query or by changing/extending
the Linked Data View. If an application relies on this trick then interoperable third-party SPARQL
clients may experience problems because they cannot use Virtuoso-specific
extensions.</para></note>

<para>
If the automated query checking gives nothing, function <function>sparql_to_sql_text</function> can be used in order to get the SQL text generated from the given query. 
Its only argument is the text of the SPARQL query to compile (without any leading SPARQL keyword or semicolon at the end). The returned value is the SQL text. The output may be long but it is the most authoritative source of diagnostic data.
</para>
<para>
When called from ISQL or an ODBC client, the return value of <function>sparql_to_sql_text</function> may be transferred as a BLOB so ISQL requires the &quot;set blobs on&quot; instruction to avoid data truncation. Even better, the SQL text can be saved to a file:
</para>
<programlisting><![CDATA[
string_to_file ('debug.sql', sparql_to_sql_text ('SELECT * WHERE { graph ?g { ?s a ?type }}'), -2);
]]></programlisting>
<para>(The -2 is to overwrite the previous version of the file, as this function may be called many times).</para>
<note><para>
When passing the query text to sparql_to_sql_text, if the query contains single quotes, each embedded single quote must be doubled up.
Use double quotes in SPARQL queries to avoid this inconvenience.
</para>
</note>
<para>As an example, let's find out why the query</para>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX northwind: <http://demo.openlinksw.com/schemas/northwind#>
SELECT DISTINCT ?emp
FROM <http://myhost.example.com/Northwind>
WHERE {
    ?order1 northwind:has_salesrep ?emp ; northwind:shipCountry ?country1 .
    ?order2 northwind:has_salesrep ?emp ; northwind:shipCountry ?country2 .
    filter (?country1 != ?country2) }
]]></programlisting>
<para>is much slower than a similar SQL statement. The call of <function>sparql_to_sql_text</function> returns the equivalent SQL statement:</para>
<programlisting><![CDATA[
SELECT DISTINCT sprintf_iri ( 'http://myhost.example.com/Northwind/Employee/%U%U%d#this' ,
    /*retval[*/ "s-6-1-t0"."b067b7d~FirstName~0" /* emp */ /*]retval*/ ,
    /*retval[*/  "s-6-1-t0"."b067b7d~FirstName~1" /*]retval*/ ,
    /*retval[*/  "s-6-1-t0"."b067b7d~FirstName~2" /*]retval*/ ) AS /*tmpl*/ "emp"
FROM (SELECT "s-6-1-t0-int~orders"."OrderID" AS /*tmpl*/ "20ffecc~OrderID",
         "s-6-1-t0-int~employees"."FirstName" AS /*as-name-N*/ "b067b7d~FirstName~0",
         "s-6-1-t0-int~employees"."LastName" AS /*as-name-N*/ "b067b7d~FirstName~1",
         "s-6-1-t0-int~employees"."EmployeeID" AS /*as-name-N*/ "b067b7d~FirstName~2"
         FROM Demo.demo.Employees AS "s-6-1-t0-int~employees", Demo.demo.Orders AS "s-6-1-t0-int~orders"
         WHERE /* inter-alias join cond */
       "s-6-1-t0-int~orders".EmployeeID = "s-6-1-t0-int~employees".EmployeeID) AS "s-6-1-t0",
    (SELECT "s-6-1-t1-int~orders"."OrderID" AS /*tmpl*/ "20ffecc~OrderID",
        "s-6-1-t1-int~orders"."ShipCountry" AS /*tmpl*/ "e45a7f~ShipCountry"
        FROM Demo.demo.Orders AS "s-6-1-t1-int~orders") AS "s-6-1-t1",
    (SELECT "s-6-1-t2-int~orders"."OrderID" AS /*tmpl*/ "20ffecc~OrderID",
        "s-6-1-t2-int~employees"."FirstName" AS /*as-name-N*/ "b067b7d~FirstName~0",
	"s-6-1-t2-int~employees"."LastName" AS /*as-name-N*/ "b067b7d~FirstName~1",
	"s-6-1-t2-int~employees"."EmployeeID" AS /*as-name-N*/ "b067b7d~FirstName~2"
	FROM Demo.demo.Employees AS "s-6-1-t2-int~employees", Demo.demo.Orders AS "s-6-1-t2-int~orders"
    WHERE /* inter-alias join cond */
       "s-6-1-t2-int~orders".EmployeeID = "s-6-1-t2-int~employees".EmployeeID) AS "s-6-1-t2",
    (SELECT "s-6-1-t3-int~orders"."OrderID" AS /*tmpl*/ "20ffecc~OrderID",
        "s-6-1-t3-int~orders"."ShipCountry" AS /*tmpl*/ "e45a7f~ShipCountry"
    FROM Demo.demo.Orders AS "s-6-1-t3-int~orders") AS "s-6-1-t3"
WHERE /* two fields belong to same equiv */
    /*retval[*/  "s-6-1-t0"."20ffecc~OrderID" /* order1 */ /*]retval*/  =
    /*retval[*/  "s-6-1-t1"."20ffecc~OrderID" /* order1 */ /*]retval*/
    AND /* two fields belong to same equiv */
    sprintf_iri ( 'http://myhost.example.com/Northwind/Employee/%U%U%d#this' ,
        /*retval[*/  "s-6-1-t0"."b067b7d~FirstName~0" /* emp */ /*]retval*/ ,
	/*retval[*/  "s-6-1-t0"."b067b7d~FirstName~1" /*]retval*/ ,
	/*retval[*/  "s-6-1-t0"."b067b7d~FirstName~2" /*]retval*/ ) =
    sprintf_iri ( 'http://myhost.example.com/Northwind/Employee/%U%U%d#this' ,
        /*retval[*/  "s-6-1-t2"."b067b7d~FirstName~0" /* emp */ /*]retval*/ ,
	/*retval[*/  "s-6-1-t2"."b067b7d~FirstName~1" /*]retval*/ ,
	/*retval[*/  "s-6-1-t2"."b067b7d~FirstName~2" /*]retval*/ )
    AND /* two fields belong to same equiv */
    /*retval[*/  "s-6-1-t2"."20ffecc~OrderID" /* order2 */ /*]retval*/  =
    /*retval[*/  "s-6-1-t3"."20ffecc~OrderID" /* order2 */ /*]retval*/
    AND /* filter */
   ( /*retval[*/  "s-6-1-t1"."e45a7f~ShipCountry" /* country1 */ /*]retval*/  <>
        /*retval[*/  "s-6-1-t3"."e45a7f~ShipCountry" /* country2 */ /*]retval*/ )
OPTION (QUIETCAST)
]]></programlisting>
<para>The query is next to unreadable but some comments split it into meaningful expressions. Every triple (or list of similar triples) becomes a subquery that returns fields needed to build the values of bound variables. The fields are printed wrapped by comments like

<emphasis>/*retval[*/ expression /* original variable name */
/*]retval*/</emphasis>. Names like<emphasis>"s-6-1-t0"</emphasis>
contain the source line number where a group pattern begins (6) and the serial
number of the triple (0). Comment <emphasis>/* inter-alias join cond
*/</emphasis> means that the expression which follows is the condition as
written in the declaration of the quad map pattern. Comment
<emphasis>/* filter */</emphasis> precedes expressions for FILTER
expressions in the source SPARQL. The word &quot;equiv&quot; means
&quot;equivalence class&quot;, i.e. a group of occurrences of variables
in the source query such that all occurrences are bound to the same
value. E.g. when a name repeats in many triples of a group, all its
occurrences form an equivalence class. In some cases the compiler can
prove that two variables are always equal even if the names differ - these
variables are also placed into an &quot;equiv&quot;.</para>

<para>Looking at this query, you may notice equalities like
<emphasis>sprintf_iri (...) = sprintf_iri (...)</emphasis>. That is
sub-optimal because it indicates that no index will be used to optimize the join
and that there will be one function call per row. When the variable
<emphasis>?emp</emphasis> appears in two different triples, it means
that the value of the variable is the same in both triples. The query
compares IRIs instead of comparing the arguments of <link
linkend="fn_sprintf_iri"><function>sprintf_iri</function></link>
because the format string is not proven to be a bijection. Indeed it
cannot be a bijection for <emphasis>arbitrary</emphasis> strings, but the database must reflect the 
real world. If it is assumed that the real names of persons never start with a
digit, within the <emphasis>%d%U</emphasis> format fragment, the digits will always
be distinguishable from the name; so the IRI class can be declared as a
bijection even if it is not true for arbitrary strings. The script
can then include &quot;suspicious&quot; <emphasis>option (bijection)</emphasis> as
follows:</para>
<programlisting><![CDATA[
create iri class sample:Employee "http://example.com/Employee/%d%U#this"
  (in employee_id integer not null, in employee_lastname varchar not null)
  option (bijection) .
]]></programlisting>
<para>Unfortunately, attempts to use the same trick with the declaration from the Northwind example will fail:</para>
<programlisting><![CDATA[
create iri class northwind:Employee "http://^{URIQADefaultHost}^/Northwind/Employee/%U%U%d#this"
  (in employee_firstname varchar not null, in employee_lastname varchar not null, in employee_id integer not null)
  option (bijection) .
]]></programlisting>
<para>Bijection will allow the parsing, but it will never give the proper result, because the first <emphasis>%U</emphasis> will read the whole concatenation of <emphasis>%U%U%d</emphasis>, leaving nothing before the<emphasis>#this</emphasis> for the second <emphasis>%U</emphasis> (this is an error) and 
leaving nothing for the <emphasis>%d</emphasis> (that is an explicit parse error, becauses the integer field cannot be empty).</para>.
<para>The string parser will process the string from left to right so it will be unable to parse the string.
The compiler might sometimes report an error if it can prove that the format string is not appropriate for bijection.</para>
<para>The correct way of improving the Northwind example is to enable reliable bijection by adding strong delimiters:</para>
<programlisting><![CDATA[
create iri class northwind:Employee "http://^{URIQADefaultHost}^/Northwind/Employee/%U/%U/%d#this"
  (in employee_firstname varchar not null, in employee_lastname varchar not null, in employee_id integer not null)
  option (bijection) .
]]></programlisting>
<para>After running the updated script, the query contains three comparisons of fields that were arguments of <function>sprintf_iri</function> in the previous version.</para>
<para><emphasis>Example for casting string as IRI type</emphasis></para>
<programlisting><![CDATA[
create function DB.DBA.RDF_DF_GRANTEE_ID_URI (in id integer)
{
  declare isrole integer;
  isrole := coalesce ((SELECT top 1 U_IS_ROLE FROM DB.DBA.SYS_USERS WHERE U_ID = id));
  if (isrole is null)
    return NULL;
  else if (isrole)
    return sprintf ('http://%s/sys/group?id=%d', registry_get ('URIQADefaultHost'), id);
  else
    return sprintf ('http://%s/sys/user?id=%d', registry_get ('URIQADefaultHost'), id);
}
;

grant execute on DB.DBA.RDF_DF_GRANTEE_ID_URI to SPARQL_SELECT
;

create function DB.DBA.RDF_DF_GRANTEE_ID_URI_INVERSE (in id_iri varchar)
{
  declare parts any;
  parts := sprintf_inverse (id_iri, sprintf ('http://%s/sys/user?id=%%d', registry_get ('URIQADefaultHost')), 1);
  if (parts is not null)
    {
      if (exists (SELECT TOP 1 1 FROM DB.DBA.SYS_USERS WHERE U_ID = parts[0] and not U_IS_ROLE))
        return parts[0];
    }
  parts := sprintf_inverse (id_iri, sprintf ('http://%s/sys/group?id=%%d', registry_get ('URIQADefaultHost')), 1);
  if (parts is not null)
    {
      if (exists (SELECT TOP 1 1 FROM DB.DBA.SYS_USERS WHERE U_ID = parts[0] and U_IS_ROLE))
        return parts[0];
    }
  return NULL;
}
;

grant execute on DB.DBA.RDF_DF_GRANTEE_ID_URI_INVERSE to SPARQL_SELECT
;


create iri class oplsioc:grantee_iri using
  function DB.DBA.RDF_DF_GRANTEE_ID_URI (in id integer) returns varchar ,
  function DB.DBA.RDF_DF_GRANTEE_ID_URI_INVERSE (in id_iri varchar) returns integer
  option ( bijection ,
    returns	"http://^{URIQADefaultHost}^/sys/group?id=%d"
    union	"http://^{URIQADefaultHost}^/sys/user?id=%d" ) .

]]></programlisting>
</sect2>

<sect2 id="rdfsparqlinline"><title>SPARQL Inline in SQL</title>
<para>Virtuoso extends the SQL 92 syntax with SPARQL queries and subqueries. Instead of writing a SQL SELECT query or subquery, one can write the SPARQL keyword and a SPARQL query after the keyword.</para>
<programlisting>
SQL>SPARQL SELECT DISTINCT ?p WHERE { graph ?g { ?s ?p ?o } };
p
varchar
----------
http://example.org/ns#b
http://example.org/ns#d
http://xmlns.com/foaf/0.1/name
http://xmlns.com/foaf/0.1/mbox
...


SQL>SELECT distinct subseq ("p", strchr ("p", '#')) as fragment
  FROM (SPARQL SELECT DISTINCT ?p WHERE { graph ?g { ?s ?p ?o } } ) as all_predicates
  WHERE "p" like '%#%';
fragment
varchar
----------
#query
#data
#name
#comment
...
</programlisting>
<para>Note that names of variables returned from SPARQL are always case-sensitive and no case mode rules apply to them.
Depending on the CaseMode parameter in the Virtuoso configuration file, double quotes should be used if necessary to refer to them in surrounding SQL code.
</para>
<para>
It is possible to pass parameters to a SPARQL query via a Virtuoso-specific syntax extension.
<emphasis>??</emphasis> or <emphasis>$?</emphasis> indicates a positional parameter similar to <emphasis>?</emphasis> in plain SQL. <emphasis>??</emphasis> can be used in graph patterns or anywhere in place of a SPARQL variable.
The value of a parameter should be passed in SQL form, i.e. this should be a number or a untyped string.
An IRI ID can be passed in all cases where an absolute IRI can, except the obvious case of when the variable is an argument of a function that requires string.
If the parameter is used in the 'graph', 'subject' or 'object' position of the SPARQL pattern, the string parameter is converted into an IRI automatically. In other cases an IRI string is indistinguishable from a string literal, so it is necessary to call the built-in SPARQL function <emphasis>iri()</emphasis> , e.g. <emphasis>iri (??)</emphasis>.
Using this notation, any dynamic SQL client (whether ODBC, JDBC or some other) can execute parameterized SPARQL queries, binding parameters just as with dynamic SQL.
</para>
<programlisting><![CDATA[
SQL> create function param_passing_demo ()
  {
  declare stat, msg varchar;
  declare mdata, rset any;
  exec ('SPARQL SELECT ?s WHERE { graph ?g { ?s ?? ?? }}',
    stat, msg,
    vector ( /* Vector of two parameters */
      'http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#int1',
      4 ),
    10, /* Max no of rows */
    mdata, /* Variable to get metadata */
    rset ); /* Variable to get result-set */
  if (length (rset) = 0)
    signal ('23000',
      'No data found, try demo database with installed Virtuoso tutorials');
  return rset[0][0];
  }

SQL> SELECT param_passing_demo ();
callret
VARCHAR
_______________________________________________________________________________

http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#four

1 Rows. -- 00000 msec.
]]></programlisting>
<para>Another example:</para>
<programlisting><![CDATA[
INSERT INTO GRAPH <http://example.com/Northwind>
{ `iri($?)` <http://example.com/schemas/northwind#has_province> "Valencia" };
]]></programlisting>
<para>An inline SPARQL query can refer to SQL variables that are in scope in the SQL query or stored procedure containing it.
Virtuoso extends the SPARQL syntax with a special notation to this effect. A reference to SQL variable X can be written as <emphasis>?:X</emphasis> or <emphasis>$:X</emphasis>.
A reference to column <emphasis>C</emphasis> of a table or a sub-select with alias <emphasis>T</emphasis> can be written as <emphasis>?:T.C</emphasis> or <emphasis>$:T.C</emphasis>.
Both notations can be used in any place where a variable name is allowed, except the 'AS' clause described below.
</para>
<para>A column of a result set of a SPARQL SELECT can be used in SQL code inside a for statement just like any column from a SQL select.
</para>
<para>SQL rules about double-quoted names are applicable to variables that are passed to a SPARQL query or selected from one.
If a variable name contains unusual characters or should not be normalized according to SQL conventions then the
name should use double quotes for escaping. e.g., the notation <emphasis>?:"OrderLine"</emphasis> will always refer to variable or column
titled <emphasis>OrderLine</emphasis> whereas <emphasis>?:OrderLine</emphasis> can be converted to <emphasis>ORDERLINE</emphasis> or <emphasis>orderline</emphasis>.
</para>
<para>It is safer to avoid using variable names that conflict with column names of RDF system tables, esp. <emphasis>G</emphasis>, <emphasis>S</emphasis>, <emphasis>P</emphasis> and <emphasis>O</emphasis>.
These names are not reserved now but they may cause subtle bugs when an incorrect SPARQL subquery is compiled into SQL code that refers to identically named table columns.
Some of these names may be rejected as syntax errors by future Virtuoso versions.
</para>
<programlisting><![CDATA[
SQL> create procedure sql_vars_demo ()
{
#pragma prefix sort0: <http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#>
  declare RES varchar;
  declare obj integer;
  result_names (RES);
  obj := 4;
  for (SPARQL SELECT ?subj WHERE { graph ?g { ?subj sort0:int1 ?:obj } } ) do
    result ("subj");
}

SQL> sql_vars_demo ();
RES
VARCHAR
_______________________________________________________________________________

http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#four

1 Rows. -- 00000 msec.
]]></programlisting>
<para>The example also demonstrates the Virtuoso/PL pragma line for procedure-wide declarations of namespace prefixes.
This makes the code more readable and eliminates duplicate declarations of namespace prefixes when the procedure
contains many SPARQL fragments that refer to a common set of namespaces.
</para>
<para>A SPARQL ASK query can be used as an argument of the SQL EXISTS predicate.
</para>
<programlisting><![CDATA[
create function sparql_ask_demo () returns varchar
{
  if (exists (sparql ask where { graph ?g { ?s ?p 4}}))
    return 'YES';
  else
    return 'NO';
}

SQL> SELECT sparql_ask_demo ();
_______________________________________________________________________________

YES
]]>
</programlisting>
<sect3 id="rdfcontrollingsparqloutputtypes"><title>Controlling SPARQL Output Data Types</title>
<para>The compilation of a SPARQL query may depend on an environment that is usually provided by the SPARQL protocol and which includes the default graph URI. Environment settings that come from the SPARQL protocol may override settings in the text of a SPARQL query. To let an application configure the environment for a query, SPARQL's syntax has been extended with the 'define' clause:</para>
<programlisting>
define parameter-qname parameter-value
</programlisting>
<para>Examples of supported parameters are <emphasis>output:valmode</emphasis> and <emphasis>output:format</emphasis></para>
<para>
<emphasis>output:valmode</emphasis> specifies which data types (i.e. representation) should be used for values in the result set. The default is "SQLVAL",
meaning that a query returns result set values in SQL format and behaves as a typical SQL select - IRIs and string literals
are returned as strings, making the output compatible with ODBC and the standard SQL routines. To compose triple vectors in Virtuoso PL code, an
application may need data in long format. A valmode of "LONG" means that IRIs are returned as IRI_IDs and string literals may be returned as special "RDF boxes"
even if they are actually plain strings. This may cause problems if these new datatypes are not known to the data recipient or if IRIs come from RDF
Views (in which case IRI_IDs are created on the fly and 'pollute' the database), but it can result in fewer data conversions and thus better speed if used
properly. "AUTO" disables all types of conversion for the result set, so the latter can comprise a mix of values across "SQLVAL" and "LONG" value modes, as well as
some internal representations. It is better to avoid using this mode in user applications because the output may change from version to version.
</para>
<para>If the query contains a</para>
<programlisting>
define output:valmode 'LONG'
</programlisting>
<para>clause then all returned values are in long format. e.g., the following query returns IRI_ID's instead of IRI strings.</para>
<programlisting>
SQL>SPARQL define output:valmode 'LONG' SELECT distinct ?p WHERE { graph ?g { ?s ?p ?o } };
p
----------
#i1000001
#i1000003
#i1000005
#i1000006
...
</programlisting>
<para><emphasis>output:format</emphasis> instructs the SPARQL compiler that the result of the query should be serialized into an RDF document -
that document will be returned as a single column of a single row result set.
<emphasis>output:format</emphasis> is especially useful if a SPARQL CONSTRUCT or SPARQL DESCRIBE query is executed directly via an ODBC or JDBC database connection
and the client cannot receive the resulting dictionary of triples (there's no way to transfer such an object via ODBC).
Using this option, the client can receive the document that contains the whole result set of a SELECT or the dictionary of triples of a CONSTRUCT/DESCRIBE, and parse it locally.
</para>
<para>
Supported values for <emphasis>output:format</emphasis> are <emphasis>RDF/XML</emphasis> and <emphasis>TURTLE</emphasis> (or <emphasis>TTL</emphasis>).
If both <emphasis>output:valmode</emphasis> and <emphasis>output:format</emphasis> are specified, <emphasis>output:format</emphasis> has higher priority,
raising an error if <emphasis>output:valmode</emphasis> is set to a value other than <emphasis>LONG</emphasis>.
</para>
<para>
When a SPARQL query is compiled, the compiler checks whether the result set is to be sent to a remote ODBC/JDBC client or used in some other way.
The compiler will automatically set <emphasis>output:format</emphasis> to <emphasis>TURTLE</emphasis> if compiling for execution by an SQL client.
</para>
<para>
The example below demonstrates how different values of <emphasis>output:format</emphasis> affect the result of SPARQL SELECT.
Note 10 rows and 4 columns in the first result, and single LONG VARCHAR in the others.
When using the ISQL client, use the 'set blobs on;' directive if fetching long texts to avoid receiving a 'data truncated' warning.
</para>
<programlisting><![CDATA[
SQL> SPARQL SELECT * WHERE {graph ?g { ?s ?p ?o }} limit 10;
g                                            s                    p                              o
VARCHAR                                      VARCHAR              VARCHAR                        VARCHAR
______________________________________________________________________

http://local.virt/DAV/bound/manifest.rdf     nodeID://1000000000 http://example.com/test#query http://local.virt/DAV/bound/bound1.rq
. . .
http://local.virt/DAV/examples/manifest.rdf nodeID://1000000019 http://example.com/test#query http://local.virt/DAV/examples/ex11.2.3.1_1.rq

10 Rows. -- 00000 msec.

SQL> SPARQL define output:format "TTL" SELECT * WHERE {graph ?g { ?s ?p ?o }} limit 10;
callret-0
LONG VARCHAR
_______________________________________________________________________________

@prefix :rdf <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix :rs <http://www.w3.org/2005/sparql-results#> .
@prefix :xsd <http://www.w3.org/2001/XMLSchema#> .
[ rdf:type rs:results ;
  rs:result [
      rs:binding [ rs:name "g" ; rs:value <http://local.virt/DAV/bound/manifest.rdf> ] ;
      rs:binding [ rs:name "s" ; rs:value _:nodeID1000000000 ] ;
      rs:binding [ rs:name "p" ; rs:value <http://example.com/test#query> ] ;
      rs:binding [ rs:name "o" ; rs:value <http://local.virt/DAV/bound/bound1.rq> ] ;
      ] ;

. . .

  rs:result [
      rs:binding [ rs:name "g" ; rs:value <http://local.virt/DAV/examples/manifest.rdf> ] ;
      rs:binding [ rs:name "s" ; rs:value _:nodeID1000000019 ] ;
      rs:binding [ rs:name "p" ; rs:value <http://example.com/test#query> ] ;
      rs:binding [ rs:name "o" ; rs:value <http://local.virt/DAV/examples/ex11.2.3.1_1.rq> ] ;
      ] ;
    ] .

1 Rows. -- 00000 msec.

SQL> SPARQL define output:format "RDF/XML" SELECT * WHERE {graph ?g { ?s ?p ?o }} LIMIT 10;
callret-0
LONG VARCHAR
_______________________________________________________________________________

<rdf:RDF
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:rs="http://www.w3.org/2005/sparql-results#"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema#" >
   <rs:results rdf:nodeID="rset">
  <rs:result rdf:nodeID="sol206">
   <rs:binding rdf:nodeID="sol206-0" rs:name="g"><rs:value rdf:resource="http://local.virt/DAV/bound/manifest.rdf"/></rs:binding>
   <rs:binding rdf:nodeID="sol206-1" rs:name="s"><rs:value rdf:nodeID="1000000000"/></rs:binding>
   <rs:binding rdf:nodeID="sol206-2" rs:name="p"><rs:value rdf:resource="http://example.com/test#query"/></rs:binding>
   <rs:binding rdf:nodeID="sol206-3" rs:name="o"><rs:value rdf:resource="http://local.virt/DAV/bound/bound1.rq"/></rs:binding>
  </rs:result>

. . .

  <rs:result rdf:nodeID="sol5737">
   <rs:binding rdf:nodeID="sol5737-0" rs:name="g"><rs:value rdf:resource="http://local.virt/DAV/examples/manifest.rdf"/></rs:binding>
   <rs:binding rdf:nodeID="sol5737-1" rs:name="s"><rs:value rdf:nodeID="1000000019"/></rs:binding>
   <rs:binding rdf:nodeID="sol5737-2" rs:name="p"><rs:value rdf:resource="http://example.com/test#query"/></rs:binding>
   <rs:binding rdf:nodeID="sol5737-3" rs:name="o"><rs:value rdf:resource="http://local.virt/DAV/examples/ex11.2.3.1_1.rq"/></rs:binding>
  </rs:result>
 </rs:results>
</rdf:RDF>

1 Rows. -- 00000 msec.
]]></programlisting>
<para>SPARQL CONSTRUCT and SPARQL DESCRIBE results are serialized as one would expect:</para>
<programlisting><![CDATA[
SQL>SPARQL
define output:format "TTL"
CONSTRUCT { ?s ?p "004" }
WHERE
  {
    graph ?g { ?s ?p 4 }
  };
callret-0
LONG VARCHAR
_______________________________________________________________________________

<http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#four> <http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#int1> "004" .
_:b1000000913 <http://www.w3.org/2001/sw/DataAccess/tests/result-set#index> "004" .


1 Rows. -- 00000 msec.

SQL> SPARQL
define output:format "RDF/XML"
CONSTRUCT { ?s ?p "004" }
WHERE
  {
    graph ?g { ?s ?p 4 }
  };
callret-0
LONG VARCHAR
_______________________________________________________________________________

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description about="http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#four"><ns0pred:int1 xmlns:ns0pred="http://www.w3.org/2001/sw/DataAccess/tests/data/Sorting/sort-0#">004</ns0pred:int1></rdf:Description>
<rdf:Description rdf:nodeID="b1000000913"><ns0pred:index xmlns:ns0pred="http://www.w3.org/2001/sw/DataAccess/tests/result-set#">004</ns0pred:index></rdf:Description>
</rdf:RDF>

1 Rows. -- 00000 msec.
]]></programlisting>
<para>SPARQL ASK returns a non-empty result set if a match is found for the graph pattern, an empty result set otherwise. If <emphasis>output:format</emphasis> is specified then the query makes a 'boolean result' document instead:</para>
<programlisting><![CDATA[
SQL> SPARQL ASK WHERE {graph ?g { ?s ?p 4 }};
__ask_retval
INTEGER
_______________________________________________________________________________

1

1 Rows. -- 00000 msec.

SQL> SPARQL ASK WHERE {graph ?g { ?s ?p "no such" }};
__ask_retval
INTEGER
_______________________________________________________________________________


0 Rows. -- 00000 msec.

SQL> SPARQL define output:format "TTL" ASK WHERE {graph ?g { ?s ?p 4 }};
callret
VARCHAR
_______________________________________________________________________________

@prefix :rdf <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
 @prefix :rs <http://www.w3.org/2005/sparql-results#> .
[ rdf:type rs:results ; rs:boolean TRUE ]

1 Rows. -- 00000 msec.

SQL> SPARQL define output:format "RDF/XML" ASK WHERE {graph ?g { ?s ?p 4 }};
callret
VARCHAR
_______________________________________________________________________________

<rdf:RDF
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:rs="http://www.w3.org/2005/sparql-results#"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema#" >
   <rs:results rdf:nodeID="rset">
    <rs:boolean rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">1</rs:boolean></results></rdf:RDF>

1 Rows. -- 00000 msec.
]]></programlisting>
</sect3>
</sect2>
<sect2 id="rdfapi">
<title>API Functions</title>
<para>
SPARQL can be used inline wherever SQL can be used.
The only API functions that one needs to know are the ones for loading RDF data into the store.
Dynamic SQL client applications can issue SPARQL queries against Virtuoso through the regular SQL client API, ODBC, JDBC or any other supported API, simply by prefixing the SPARQL query with the SPARQL keyword. Parameters work just as with dynamic SQL.
Stored procedures can have SPARQL expressions inline and can declare cursors over SPARQL result sets.
</para>

<para>
Value conversions between SQL and SPARQL are most often automatic and
invisible. In some cases one needs to be aware of the different
SPARQL value representations (valmodes). SPARQL offers declarations
for specifying whether returned graphs are to be serialized as XML
or Turtle,  or whether these will be hash tables of
triples. See <link linkend="fn_dict_new"><function>dict_new()</function></link> and related functions for a description of the hash table SQL data type.
The use of dict's is convenient for further programmatic processing of graphs.
</para>

<para>RDF-related procedures use Virtuoso/PL vectors
and dictionaries to represent RDF triples and sets of triples.</para>
<para><emphasis>Valmode</emphasis> means the "format of values returned by an
expression", i.e. 'short', 'long' or 'SQL value'.</para>
<para><emphasis>Triple vector</emphasis> is a vector (array) of S, P and O, where all values are in
'long' formats, i.e. IRI_ID's for IRI values, numbers or datetimes for corresponding XMLSchema types, special &quot;RDF box&quot; objects if O is neither string nor IRI.</para>
<para><emphasis>Dictionary of triples</emphasis> or <emphasis>Hash table of triples</emphasis> is an
dictionary object made by the SQL function <emphasis>dict_new ()</emphasis> whose keys are
triple vectors and values are not specified; this is a good storage
format for an unordered set of distinct triples.</para>
<para><emphasis>Dictionary of blank node names</emphasis> is a dictionary used for tricky
processing of a number of TURTLE or RDF /XML descriptions of subgraphs
that come from a common graph. Imagine a situation where different
descriptions actually refer to the same blank nodes of the original graph
and, moreover, the application that generates these descriptions always
generates the same blank node id string for the same node. A reader of
descriptions can correctly join described subgraphs into one big
subgraph by filling in a dictionary that contains blank node id strings
as keys and IRI_ID's assigned to those strings as dependent data.
The sharing of the same node dictionary by all readers of an application will ensure that no blank node is duplicated.</para>
<sect3 id="rdfapidataimport"><title>Data Import</title>
<sect4 id="rdfapidataimportttlp"><title>Using TTLP</title>
<para>DB.DBA.TTLP() parses TTL (TURTLE or N3 resource) and places its triples into DB.DBA.RDF_QUAD.</para>
<programlisting>
create procedure DB.DBA.TTLP (
    in strg any,       -- text of the resource
    in base varchar,   -- base IRI to resolve relative IRIs to absolute
    in graph varchar, -- target graph IRI, parsed triples will appear in that graph.
    in flags int)   -- bitmask of flags that permit some sorts of syntax errors in resource, use 0.
</programlisting>
<para>For loading a file of any great length, it is more practical to use
the file_to_string_output function.
</para>
<para>It is important the file be accessible to the Virtuoso server. You need to have set properly set the
<emphasis>DirsAllowed</emphasis> parameter value in the section [Parameters] of the Virtuoso database INI file.
For example on Windows it could be:
</para>
<programlisting>
virtuoso.ini file:
[Parameters]
...
DirsAllowed =  .\tmp
...
</programlisting>
<para>So, in the example, the file you want to import from, should be in the tmp folder or in a subfolder.
Note that this example folder is a subfolder of the Virtuoso Server working directory.
</para>
<programlisting>
SQL> DB.DBA.TTLP (file_to_string_output ('.\tmp\data.ttl'), '', 'http://my_graph', 0);
</programlisting>
</sect4>
<sect4 id="rdfapidataimportttlpmt"><title>Using TTLP_MT</title>
<para>The DB.DBA.TTLP_MT() procedure is like DB.DBA.TTLP() but loads the file on multiple threads,
using parallel I/O and multiprocessing if available. The function does not leave a transaction log.
Hence, after a successful load, one should execute the checkpoint statement to make sure that a
server restart does not wipe out the results.
</para>
<programlisting>
create procedure DB.DBA.TTLP_MT (
    in strg any,       -- text of the resource
    in base varchar,   -- base IRI to resolve relative IRIs to absolute
    in graph varchar,  -- target graph IRI, parsed triples will appear in that graph.
    in flags int) -- flags, use 0
</programlisting>
</sect4>
<sect4 id="rdfapidataimportxmlttlpmt"><title>Using RDF_LOAD_RDFXML_MT</title>
<para>For loading large resources when transactional integrity is not important (loading of a single resource may take more than one transaction)
you can use also the <emphasis>DB.DBA.RDF_LOAD_RDFXML_MT()</emphasis> procedure:</para>
<programlisting>
create procedure DB.DBA.RDF_LOAD_RDFXML_MT (
    in strg varchar,  -- text of the resource
    in base varchar,  -- base IRI to resolve relative IRIs to absolute
    in graph varchar) -- target graph IRI, parsed triples will appear in that graph.
</programlisting>
<para>The following example demonstrates importing data from the RDF resource with URI: http://www.w3.org/People/Berners-Lee/card</para>
<programlisting><![CDATA[
SQL>create procedure MY_LOAD_FILE (in full_uri varchar, in in_resultset integer := 0)
  {
  declare REPORT varchar;
  declare graph_uri, dattext varchar;
  declare app_env any;
  app_env := null;
  whenever sqlstate '*' goto err_rep;
  if (not in_resultset)
    result_names (REPORT);
  dattext := cast (XML_URI_GET_AND_CACHE (full_uri) as varchar);
  MY_SPARQL_REPORT (sprintf ('Downloading %s: %d bytes',
      full_uri, length (dattext) ) );
  graph_uri := full_uri;
  DELETE FROM RDF_QUAD WHERE G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_uri);
  DB.DBA.RDF_LOAD_RDFXML_MT (dattext, full_uri, graph_uri);
  return graph_uri;
err_rep:
  result (sprintf ('%s: %s', __SQL_STATE, __SQL_MESSAGE));
  return graph_uri;
  }
;

Done. -- 0 msec.

SQL>create procedure MY_SPARQL_REPORT(in strg varchar)
  {
  if (__tag(strg) <> 182)
    strg := cast (strg as varchar) || sprintf (' -- not a string, tag=%d', __tag(strg));
  strg := replace (strg, 'SPARQL_DAV_DATA_URI()', '\044{SPARQL_DAV_DATA_URI()}');
  strg := replace (strg, 'SPARQL_DAV_DATA_PATH()', '\044{SPARQL_DAV_DATA_PATH()}');
  strg := replace (strg, 'SPARQL_FILE_DATA_ROOT()', '\044{SPARQL_FILE_DATA_ROOT()}');
  result (strg);
  }
;

Done. -- 0 msec.

SQL> MY_LOAD_FILE('http://www.w3.org/People/Berners-Lee/card');
REPORT
VARCHAR
_______________________________________________________________________________

Downloading http://www.w3.org/People/Berners-Lee/card: 17773 bytes

1 Rows. -- 4046 msec.

SQL>SPARQL
SELECT *
FROM <http://www.w3.org/People/Berners-Lee/card>
WHERE {?s ?p ?o} ;

s                                             p                                               o
VARCHAR                                       VARCHAR                                         VARCHAR
__________________________________________________________________________________________________________

http://bblfish.net/people/henry/card#me       http://xmlns.com/foaf/0.1/name                  Henry Story
http://www.w3.org/People/Berners-Lee/card#i   http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://xmlns.com/foaf/0.1/Person
http://www.w3.org/People/Berners-Lee/card#i   http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2000/10/swap/pim/contact#Male
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/nick                  TimBL
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/nick                  timbl
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/mbox                  mailto:timbl@w3.org
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/mbox_sha1sum          965c47c5a70db7407210cef6e4e6f5374a525c5c
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://bblfish.net/people/henry/card#me
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://hometown.aol.com/chbussler/foaf/chbussler.foaf#me
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://danbri.org/foaf#danbri
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://norman.walsh.name/knows/who#norman-walsh
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://www.aaronsw.com/about.xrdf#aaronsw
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://www.ivan-herman.net/foaf.rdf#me
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://www.w3.org/People/Berners-Lee/card#amy
http://www.w3.org/People/Berners-Lee/card#i   http://xmlns.com/foaf/0.1/knows                 http://dig.csail.mit.edu/People/RRS
..........

]]></programlisting>
</sect4>
<sect4 id="rdfapidataimportttlphash"><title>Using RDF_TTL2HASH</title>
<para>The DB.DBA.RDF_TTL2HASH() does not load TTL content, instead it returns a dictionary of triples in 'long valmode'.</para>
<programlisting>
create function DB.DBA.RDF_TTL2HASH (
    in strg any,
    in base varchar,
    in graph varchar ) returns any
</programlisting>
<para>Parameter <emphasis>flags</emphasis> is useful when the syntax of the resource is TURTLE-like, but not correct TURTLE.
By default, use zero value.
Add 1 to let string literals contain end-of-line characters.
Add 2 to suppress error messages on blank node verbs.
Add 4 to allow variables instead of blank nodes.
Add 8 to silently skip triples with literal subjects.
       </para>
     </sect4>
<sect4 id="rdfapidataimportloadrdfxml"><title>Using RDF_LOAD_RDFXML</title>
<para>The DB.DBA.RDF_LOAD_RDFXML() procedure parses RDF/XML and places its triples into DB.DBA.RDF_QUAD.</para>
       <programlisting>
create procedure DB.DBA.RDF_LOAD_RDFXML (
    in strg any,           -- text of and XML document
    in base_iri varchar,   -- base IRI to resolve relative IRIs
    in graph_iri varchar ) -- the IRI of destination graph
       </programlisting>
<para>See <link linkend="rdfsparqlrulespecifywhatindexexample">example</link></para>.
</sect4>
<sect4 id="rdfapidataimportloadrdfuri"><title>Using RDF_QUAD_URI, RDF_QUAD_URI_L and RDF_QUAD_URI_L_TYPED</title>
<para>To insert a single quad into DB.DBA.RDF_QUAD() table, use one of these procedures:</para>
       <programlisting>
-- Simple insertion of a quad where the object is a node
create procedure DB.DBA.RDF_QUAD_URI (
  in g_uri varchar, in s_uri varchar, in p_uri varchar,
  in o_uri varchar ) -- IRI string or IRI_ID

-- Simple insertion of a quad where the object is a literal value in 'SQL valmode'
create procedure DB.DBA.RDF_QUAD_URI_L (
  in g_uri varchar, in s_uri varchar, in p_uri varchar,
  in o_lit any ) -- string, number or datetime, NULL is not allowed

create procedure DB.DBA.RDF_QUAD_URI_L_TYPED (
  in g_uri varchar, in s_uri varchar, in p_uri varchar,
  in o_lit any,     -- string value of the literal
  in dt any,        -- datatype as IRI string or IRI_ID, can be NULL
  in lang varchar ) -- language as string or NULL
       </programlisting>
</sect4>
<para>Arguments g_uri, s_uri and p_uri of these three functions should be IRI strings or IRI_IDs.
All string arguments should be in UTF-8 encoding, otherwise they will be stored but are not queryable via SPARQL.</para>
</sect3>
<sect3 id="rdfapidataexport"><title>Data Export</title>
<para>These two procedures serialize a vector of triples into a session, in TURTLE or RDF/XML syntax.
In their current versions, every triple is printed in a separate top-level record (say, in an rdf:Description tag), without any pretty-printing or nesting optimization.
       </para>
<programlisting>
create procedure DB.DBA.RDF_TRIPLES_TO_TTL (
    inout triples any, -- vector of triples in 'long valmode'.
    inout ses any )    -- an output stream in server default encoding

create procedure DB.DBA.RDF_TRIPLES_TO_RDF_XML_TEXT (
    inout triples any,          -- vector of triples in 'long valmode'.
    in print_top_level integer, -- zero if only rdf:Description tags should be written,
                                -- non-zero if the rdf:RDF top-level element should also be written
    inout ses any )             -- an output stream in server default encoding
</programlisting>
</sect3>
<sect3 id="rdfapidataquery"><title>Data query</title>
<programlisting>
-- Local execution of SPARQL via SPARQL protocol, produces a result set of SQL values.
create procedure DB.DBA.SPARQL_EVAL (
    in query varchar,      -- text of SPARQL query to execute
    in dflt_graph varchar, -- default graph IRI, if not NULL then this overrides what's specified in query
    in maxrows integer )   -- limit on numbers of rows that should be returned.

-- Similar to SPARQL_EVAL, but returns a vector of vectors of SQL values.
create function DB.DBA.SPARQL_EVAL_TO_ARRAY (
    in query varchar,      -- text of SPARQL query to execute
    in dflt_graph varchar, -- default graph IRI, if not NULL then this overrides what's specified in query
    in maxrows integer )   -- limit on numbers of rows that should be returned.
returns any
</programlisting>
<programlisting>
-- Remote execution of SPARQL via SPARQL protocol, produces a result set of SQL values.
create procedure DB.DBA.SPARQL_REXEC (
    in service varchar,    -- service URI to call via HTTP
    in query varchar,      -- text of SPARQL query to execute
    in dflt_graph varchar, -- default graph IRI, if not NULL then this overrides what's specified in query
    in named_graphs any,   -- vector of named graph IRIs, if not NULL then this overrides what's specified in query
    in req_hdr any,        -- additional HTTP header lines that should be passed to the service; 'Host: ...' is most popular.
    in maxrows integer,    -- limit on numbers of rows that should be returned.
    in bnode_dict any )    -- dictionary of bnode ID references.

-- Similar to SPARQL_REXEC (), but returns a vector of vectors of SQL values.
-- All arguments are the same.
create function DB.DBA.SPARQL_REXEC_TO_ARRAY (
    in service varchar, in query varchar, in dflt_graph varchar, in named_graphs any,
    in req_hdr any, in maxrows integer, in bnode_dict any)
returns any

-- Similar to SPARQL_REXEC (), but fills in output parameters with metadata (like exec metadata) and a vector of vector
s of 'long valmode' values.
-- First seven arguments are the same.
create procedure DB.DBA.SPARQL_REXEC_WITH_META (
    in service varchar, in query varchar, in dflt_graph varchar, in named_graphs any,
    in req_hdr any, in maxrows integer, in bnode_dict any,
    out metadata any,  -- metadata like exec () returns.
    out resultset any) -- results as 'long valmode' value.
</programlisting>
<para>If the query is a CONSTRUCT or DESCRIBE then the result set consists of a single row and column, the value inside is a dictionary of triples in 'long valmode'.</para>
</sect3>
</sect2>
<sect2 id="rdfinternalfunctions"><title>Useful Internal Functions</title>
<sect3 id="rdfinternalconversion"><title>Conversion Functions for XMLSchema/RDF Data Serialization Syntax</title>
<para>These functions emulate constructor functions from XQuery Core Function Library.</para>
<programlisting>
create function DB.DBA."http://www.w3.org/2001/XMLSchema#boolean" (in strg any) returns integer
create function DB.DBA."http://www.w3.org/2001/XMLSchema#dateTime" (in strg any) returns datetime
create function DB.DBA."http://www.w3.org/2001/XMLSchema#double" (in strg varchar) returns double precision
create function DB.DBA."http://www.w3.org/2001/XMLSchema#float" (in strg varchar) returns float
create function DB.DBA."http://www.w3.org/2001/XMLSchema#integer" (in strg varchar) returns integer
</programlisting>
</sect3>
<sect3 id="rdfinternalpredicates"><title>RDF-specific Predicates</title>
<programlisting>
-- Returns 1 if string s matches pattern p, 0 otherwise
create function DB.DBA.RDF_REGEX (
    in s varchar,            -- source string to check
    in p varchar,            -- regular expression pattern string
    in coll varchar := null) -- unused for now (modes are not yet implemented)

-- Returns 1 if language identifier r matches lang pattern t
create function DB.DBA.RDF_LANGMATCHES (
  in r varchar, -- language identifies (string or NULL)
  in t varchar) -- language pattern (exact name, first two letters or '*')
</programlisting>
</sect3>
</sect2>
<sect2 id="rdfdefaultgraph"><title>Default and Named Graphs</title>
<para>Sometimes the default graph IRI is not known when the SPARQL query is composed. It can be added at the very last moment by providing the IRI in a 'define' clause as follows:</para>
<programlisting><![CDATA[
define input:default-graph-uri &lt;http://example.com&gt
]]></programlisting>
<para>Such a definition overrides the default graph URI set in query by the 'FROM ...' clause (if any).</para>
<para>The query may contain more than one <emphasis>define input:default-graph-uri</emphasis>.
The set of values of <emphasis>input:default-graph-uri</emphasis> has the highest possible priority and cannot be redefined in the rest of the text of the query by FROM clauses.</para>
<para>FROM NAMED clauses can be used multiple times in one query:</para>
<programlisting><![CDATA[
SPARQL
  SELECT ?id
  FROM NAMED <http://example.com/user1.ttl>
  OPTION (get:soft "soft", get:method "GET")
  FROM NAMED <http://example.com/user2.ttl>
  OPTION (get:soft "soft", get:method "GET")
  WHERE { GRAPH ?g { ?id a ?o } }
]]></programlisting>
<para>Similarly, <emphasis>define input:named-graph-uri &lt;http://example.com&gt;</emphasis> is a replacement for a FROM NAMED clause</para>
<para>
When Virtuoso receives a SPARQL request via HTTP, the value of the default graph can be set in the protocol using a <emphasis>default-graph-uri</emphasis> HTTP parameter.
Multiple occurrences of this parameter are allowed. This HTTP parameter is converted into <emphasis>define input:default-graph-uri</emphasis>.
There's similar support for <emphasis>named-graph-uri</emphasis> HTTP parameter.
For debugging purposes, graph names set in the protocol are sent back in the reply header as <emphasis>X-SPARQL-default-graph: ...</emphasis> and <emphasis>X-SPARQL-named-graph: ...</emphasis> header lines, one line per graph.
       </para>
<para>
A web service endpoint may provide different default configurations for different host names mentioned in HTTP requests.
This facility is configured via table <emphasis>DB.DBA.SYS_SPARQL_HOST</emphasis>.
</para>
<programlisting>
create table DB.DBA.SYS_SPARQL_HOST (
  SH_HOST	varchar not null primary key, -- host mask
  SH_GRAPH_URI varchar,                 -- default graph uri
  SH_USER_URI	varchar,      -- reserved for any use in applications
  SH_BASE_URI varchar,                  -- for future use (not used currently) to set BASE in sparql queries. Should be NULL for now.
  SH_DEFINES	long varchar,             -- additional defines for requests
  PRIMARY KEY (SH_HOST)
)
</programlisting>

<para>
When the SPARQL web service endpoint receives a request it checks the <emphasis>Host</emphasis> HTTP header line.
This line contains zero or more target host names, delimited by commas.
For every host name in the line, the service scans the <emphasis>DB.DBA.SYS_SPARQL_HOST</emphasis> table in search of a
row containing a matching host name in <emphasis>SH_HOST</emphasis>.
The <emphasis>SH_HOST</emphasis> field acts as 'pattern' argument for the SQL string operator LIKE. If a matching row is found,
the text of SPARQL request is extended.</para>

<para>
If a default graph is not explicitly set by the HTTP parameters and <emphasis>SH_GRAPH_URI</emphasis> is not null then the
default graph is set to <emphasis>SH_GRAPH_URI</emphasis>.</para>

<para>
If <emphasis>SH_DEFINES</emphasis> is not null then it is added in front of the query; so this field is a good place for the
text for any <link linkend="rdfsparqlimplementationextent">DEFINE</link> options.
See <link linkend="rdfcontrollingsparqloutputtypes">various</link> DEFINE examples usage.
</para>

<para>SH_USER_URI is for arbitrary user data and can be used in any way by the
application that is "responsible" for the declared host.</para>

<para>
The search of <emphasis>DB.DBA.SYS_SPARQL_HOST</emphasis> stops at the first found row, other possible matches are silently ignored.
</para>

<para>Example Usage:</para>
<programlisting><![CDATA[
INSERT INTO DB.DBA.SYS_SPARQL_HOST (SH_HOST, SH_GRAPH_URI, SH_USER_URI, SH_BASE_URI, SH_DEFINES) VALUES
('example.com', 'urn:example:com', 'urn:example:user', NULL, 'define input:inference "http://mygraph.com"');
]]></programlisting>
</sect2>

<sect2 id="rdfsqlfromsparql"><title>Calling SQL from SPARQL</title>
<para>A SPARQL expression can contain calls to Virtuoso/PL functions and built-in SQL functions in both the WHERE clause and in the
result set. Two namespace prefixes, <emphasis>bif</emphasis> and <emphasis>sql</emphasis> are reserved for
these purposes. When a function name starts with the <emphasis>bif:</emphasis> namespace
prefix, the rest of the name is treated as the name of a SQL BIF (Built-In
Function). When a function name starts with the <emphasis>sql:</emphasis> namespace prefix,
the rest of the name is treated as the name of a Virtuoso/PL function owned by
DBA with database qualifier DB, e.g. <emphasis>sql:example(...)</emphasis> is
converted into <emphasis>DB.DBA."example"(...)</emphasis>.</para>

<para>In both cases,
the function receives arguments in SQL format ('SQL valmode') and
also returns the result in SQL format. The SPARQL compiler will
automatically add code for format conversion into the resulting SQL
code so SQL functions can be used even if <emphasis>define output:valmode
'LONG'</emphasis> forces the use of RDF representation in the
result set.</para>
<sect3 id="rdfsqlfromsparqlex1"><title>Example with sql: namespace prefix</title>
<programlisting><![CDATA[
SQL>create procedure DB.DBA.ComposeInfo (
  in pname varchar,
  in pnick varchar := '',
  in pbox  varchar := '')
{
   declare ss varchar;
   ss := concat(pname, ' ', pnick, ' ', pbox);
   ss := rtrim (ss, ' ');
   return ss;

};
Done. -- 0 msec.

SQL>SPARQL
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT (sql:ComposeInfo (?name, ?nick, ?box))
FROM <http://www.w3.org/People/Berners-Lee/card>
WHERE
  {
    ?s rdf:type foaf:Person .
    optional{?s foaf:name ?name }.
    optional{?s foaf:nick ?nick }.
    optional{?s foaf:box ?box }.
    filter (?nick like '%TimBL%') .
  };
callret-0
VARCHAR
_______________________________________________________________________________

Timothy Berners-Lee TimBL

1 Rows. -- 30 msec.
]]></programlisting>
    <tip><title>See Also:</title>
      <itemizedlist mark="bullet">
        <listitem><link linkend="rdfsparqlgeospatexmp11">Example "Things around highly populated places"</link></listitem>
        <listitem><link linkend="virtuosospongerfacent">Virtuoso Faceted Web Service Examples</link></listitem>
        <listitem><link linkend="VirtFacetUsage6">Virtuoso Faceted Usage Statistics Examples</link></listitem>
      </itemizedlist>
    </tip>
  </sect3>
  <sect3 id="rdfsqlfromsparqlex2"><title>Example with sql: namespace prefix and bif:contains</title>
<programlisting><![CDATA[
SQL>SPARQL
SELECT DISTINCT ?cityUri ?cityName (sql:BEST_LANGMATCH (?cityName, 'en, en-gb;q=0.8, fr;q=0.7, *;q=0.1', '')) as ?bestCityName
WHERE
  {
    ?cityUri ?predicate ?value.
    ?cityUri a <http://dbpedia.org/ontology/City>.
    ?value bif:contains "London".
    OPTIONAL
      {
        ?cityUri rdfs:label ?cityName
      }
  };

cityUri                                              cityName                      bestCityName
ANY                                                  ANY 	                         ANY
______________________________________________________________________________________________________________
http://dbpedia.org/resource/Anerley	                 Anerley	                     Anerley
http://dbpedia.org/resource/Felixstowe	             Felixstowe	                   Felixstowe
http://dbpedia.org/resource/Chesham	                 Chesham	                     Chesham
http://dbpedia.org/resource/Stratford%2C_London	     Stratford, London	           Stratford, London
http://dbpedia.org/resource/Ashford%2C_Surrey	       Ashford (Surrey)	 A           shford (Surrey)
http://dbpedia.org/resource/Newmarket%2C_Suffolk	   Newmarket (Suffolk)	         Newmarket (Suffolk)
http://dbpedia.org/resource/North_Rhine-Westphalia	 Renania d'o Norte-Westfalia	 Renania d'o Norte-Westfalia
http://dbpedia.org/resource/West_Bromwich	           West Bromwich	               West Bromwich
....

]]></programlisting>
  </sect3>
  <sect3 id="rdfsqlfromsparqlex3"><title>Example with bif: namespace prefix</title>
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://www.w3.org/people#>
WHERE { ?s ?p ?o . ?o bif:contains '"Timo*"'};
s                                               p                                     o
VARCHAR                                         VARCHAR                               VARCHAR
_______________________________________________________________________________

 http://www.w3.org/People/Berners-Lee/card#i	http://xmlns.com/foaf/0.1/name	      Timothy Berners-Lee
 http://www.w3.org/People/Berners-Lee/card#i	http://xmlns.com/foaf/0.1/givenname   Timothy

2 Rows. -- 2 msec.

]]></programlisting>
    <tip><title>See Also:</title>
      <itemizedlist mark="bullet">
        <listitem><link linkend="rdfpredicatessparqlexamples">Example filtering RDF objects triples by a given predicate</link></listitem>
        <listitem><link linkend="rdfsparqlendpointexamples6">Example with extraction part of literal as variable</link></listitem>
        <listitem><link linkend="rdfsparulexamples25">Example for Usage of Expressions inside CONSTRUCT, INSERT and DELETE {...} Templates</link></listitem>
        <listitem><link linkend="rdfsparulexamples5">Example for various expressions usage</link></listitem>
        <listitem><link linkend="rdfsparulexamples8">Example for generating RDF information resource URI</link></listitem>
      </itemizedlist>
    </tip>
  </sect3>
</sect2>
<sect2 id="rdfsqlfromsparqldescribe"><title>SPARQL DESCRIBE</title>
  <para>The SPARQL specification does not define the precise output of DESCRIBE, so different
applications may need different results for the same subject. Some applications need quick generation of short
and incomplete results whereas others may need detailed reports composed from multiple sources.
  </para>
  <para>The supported option values for <emphasis>sql:describe-mode</emphasis> are:</para>
  <itemizedlist mark="bullet">
    <listitem><emphasis>(default)</emphasis> -- for subject-predicate-object triples of given IRI as subject plus
    subject-predicate-object triples of given IRI as object. No "sql:describe-mode" or define sql:describe-mode ""
    will set default.</listitem>
    <listitem><emphasis>SPO</emphasis> -- for subject-predicate-object triples of given IRI as subject;</listitem>
    <listitem><emphasis>CBD</emphasis> -- for concise bound description of given subject (i.e., SPO + CBD of each
    blank node object found by SPO, recursively);</listitem>
    <listitem><emphasis>OBJCBD</emphasis> -- like CBD but traverses from objects to subjects, not from subjects to objects;</listitem>
    <listitem>In addition, user may write his/her own "postporocessing" function that will get the result of
    describe, lists of "good" and "bad" graphs, name of storage, options and alter it in any way he/she wishes.
    The notation is e.g. define sql:describe-mode "SPO+XYZ", meaning SPO describe and call
    DB.DBA.SPARQL_DESC_POSTPROC_XYZ on top of it. See below example usage.</listitem>
  </itemizedlist>
  <para>If define <emphasis>sql:describe-mode "xxx"</emphasis> is specified then the generated SQL code will use
  the procedures named:
       </para>
<programlisting><![CDATA[
DB.DBA.SPARQL_DESC_DICT_xxx (in subj_dict any, in consts any, in graphs
any, in storage_name any, in options any)
]]></programlisting>
  <para>and
  </para>
<programlisting><![CDATA[
DB.DBA.SPARQL_DESC_DICT_xxx_PHYSICAL (in subj_dict any, in consts any,
in graphs any, in storage_name any, in options any)
]]></programlisting>
  <para>In a new blank database, only two such pairs of procedures are created.
Procedures <emphasis>DB.DBA.SPARQL_DESC_DICT_SPO</emphasis> and <emphasis>DB.DBA.SPARQL_DESC_DICT_SPO_PHYSICAL</emphasis>
are for <emphasis>sql:describe-mode "SPO"</emphasis>. This pair of procedures
searches for all triples where the input IRIs are used as subjects; they are faster than the default
routine which searches for all triples where the input IRIs are used as subjects or objects.
Similarly, <emphasis>DB.DBA.SPARQL_DESC_DICT_CBD</emphasis> and <emphasis>DB.DBA.SPARQL_DESC_DICT_CBD_PHYSICAL</emphasis>
are for <emphasis>sql:describe-mode "CBD"</emphasis>. CBD stands for Concise Bounded Description of given subject
(i.e., SPO + CBD of each blank node object found by SPO, recursively).
  </para>
  <para>In each pair, both procedures have the same semantics but the second one is used if and
only if the SPARQL compiler can prove that all subjects to process are
from physical storage <emphasis>(DB.DBA.RDF_QUAD)</emphasis>. Thus the second procedure
will not search for subjects in Linked Data Views.
  </para>
  <para>Each procedure should return a dictionary with triples as keys and
integer 1 as values. So the dictionary is filled by calls like:
  </para>
<programlisting><![CDATA[
dict_put (resulting_dict,
          vector (subj_iri_id, pred_iri_id, obj_iri_id_or_rdf_box),
          1);
]]></programlisting>
  <para>Procedure arguments are as follows:
  </para>
  <itemizedlist mark="bullet">
     <listitem><emphasis>subj_dict</emphasis> - a dictionary whose keys are IRI IDs and maybe values of
other types, esp. RDF boxes. Keys are subjects to be described, so
values other than IRI IDs should usually be ignored. Values should be
ignored.</listitem>
     <listitem><emphasis>consts</emphasis> - a vector of IRI IDs and values of other types. The items contained in the vector
are subjects to be described, as with the keys of subj_dict.</listitem>
     <listitem><emphasis>graphs</emphasis> - a vector of IRI IDs of graphs that can be used for DESCRIBE. The
vector may contain garbage, like in the two previous cases. A NULL can be
passed instead of a vector indicating that the source graphs are not specified
in the source query.</listitem>
     <listitem><emphasis>storage_name</emphasis> -  the value of "define input:storage" from the original
SPARQL query, NULL if missing.</listitem>
     <listitem><emphasis>options</emphasis> - reserved for future use and can be ignored.</listitem>
   </itemizedlist>
  <para>One should grant execute permission on both procedures to SPARQL_SELECT before referring to them in SPARQL.</para>
      <sect3 id="rdfsqlfromsparqldescribeex">
      <title>SPARQL DESCRIBE Examples</title>
      <para>Assume the following statements are executed:</para>
<programlisting><![CDATA[
__rdf_set_bnode_t_treshold();
SET blobs ON;
SET echo ON;

SPARQL PREFIX xmp: <http://example.com/xmp/>
CLEAR GRAPH xmp:good1;

SPARQL PREFIX xmp: <http://example.com/xmp/>
CLEAR GRAPH xmp:good2;

SPARQL PREFIX xmp: <http://example.com/xmp/>
CLEAR GRAPH xmp:bad1;

SPARQL PREFIX xmp: <http://example.com/xmp/>
CLEAR GRAPH xmp:bad2;

SPARQL PREFIX xmp: <http://example.com/xmp/>
INSERT IN xmp:good1
  {
    xmp:Top1 xmp:item xmp:TheSubject .
    xmp:TheSubject xmp:details
            xmp:ChildObject ,
            ( xmp:car xmp:cadr xmp:caddr ) .
  };

SPARQL PREFIX xmp: <http://example.com/xmp/>
INSERT IN xmp:good2
  {
    xmp:Top2 xmp:items [ rdf:_1 xmp:TheSubject ; rdf:_2 xmp:OtherSubject ] .
  };
]]></programlisting>
      <sect4 id="rdfsqlfromsparqldescribeexCBD">
        <title>Examples SPARQL DESCRIBE -- No Option</title>
<programlisting><![CDATA[
SPARQL
DEFINE output:format "NICE_TTL"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject FROM xmp:good1 FROM xmp:good2;

fmtaggret-NICE_TTL
LONG VARCHAR
 @prefix ns0: <http://example.com/xmp/> .
 @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
 ns0:Top1 ns0:item ns0:TheSubject .
 ns0:TheSubject ns0:details ns0:ChildObject , [ ] .
 _:vb78520012 rdf:_1 ns0:TheSubject .

]]></programlisting>
      </sect4>
    <sect4 id="rdfsqlfromsparqldescribeexSPO">
      <title>Examples SPARQL DESCRIBE Option "SPO"</title>
<para><emphasis>Example 1</emphasis></para>
<programlisting><![CDATA[
SPARQL
DEFINE output:format "NICE_TTL"
DEFINE sql:describe-mode "SPO"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject FROM xmp:good1 FROM xmp:good2;

fmtaggret-NICE_TTL
LONG VARCHAR
 @prefix ns0: <http://example.com/xmp/> .
 ns0:TheSubject ns0:details ns0:ChildObject , [ ] .
]]></programlisting>
<para><emphasis>Example 2</emphasis></para>
<programlisting><![CDATA[
SQL>set blobs on;
SQL>SPARQL
define sql:describe-mode "SPO"
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioct: <http://rdfs.org/sioc/types#>

DESCRIBE ?forum
FROM <http://demo.openlinksw.com/dataspace>
WHERE {
  ?forum rdf:type sioct:Weblog .
}
LIMIT 1;

callret-0
LONG VARCHAR
_______________________________________________________________________________

<http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/types#Weblog> ,
                <http://atomowl.org/ontologies/atomrdf#Feed> ;
        <http://rdfs.org/sioc/ns#description> "XML templates demo's Weblog" ;
        <http://rdfs.org/sioc/ns#has_space> <http://demo.openlinksw.com/dataspace/bloguser/space#this> ;
        <http://rdfs.org/sioc/ns#container_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/20> ,
                <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/21> ;
        <http://rdfs.org/sioc/ns#id> "bloguser_blog" ;
        <http://xmlns.com/foaf/0.1/maker> <http://demo.openlinksw.com/dataspace/person/bloguser#this> ;
        <http://rdfs.org/sioc/ns#link> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> ;
        <http://atomowl.org/ontologies/atomrdf#entry> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/20> ,
                <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/21> ;
        <http://atomowl.org/ontologies/atomrdf#contains> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/21> ,
                <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/20> ;
        <http://atomowl.org/ontologies/atomrdf#title> "bloguser_blog" ;
        <http://www.w3.org/2000/01/rdf-schema#label> "XML templates demo's Weblog" ;
        <http://rdfs.org/sioc/ns#scope_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog#owner> ;
        <http://rdfs.org/sioc/ns#has_owner> <http://demo.openlinksw.com/dataspace/bloguser#this> ;
        <http://www.w3.org/2000/01/rdf-schema#isDefinedBy> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/sioc.rdf> ;
        <http://purl.org/dc/elements/1.1/identifier> "62"^^<http://www.w3.org/2001/XMLSchema#integer> ;
        <http://rdfs.org/sioc/services#has_service> <http://demo.openlinksw.com/RPC2> ,
                <http://demo.openlinksw.com/mt-tb> ,
                <http://demo.openlinksw.com/Atom/bloguser-blog-0> ,
                <http://demo.openlinksw.com/GData/bloguser-blog-0> .
<http://demo.openlinksw.com/RPC2> <http://rdfs.org/sioc/services#service_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/mt-tb> <http://rdfs.org/sioc/services#service_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog#owner> <http://rdfs.org/sioc/ns#has_scope> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/20> <http://rdfs.org/sioc/ns#has_container> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> ;
        <http://atomowl.org/ontologies/atomrdf#source> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog/21> <http://rdfs.org/sioc/ns#has_container> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> ;
        <http://atomowl.org/ontologies/atomrdf#source> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/dataspace/bloguser#this> <http://rdfs.org/sioc/ns#owner_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/dataspace/bloguser/space#this> <http://rdfs.org/sioc/ns#space_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/dataspace/person/bloguser#this> <http://xmlns.com/foaf/0.1/made> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/Atom/bloguser-blog-0> <http://rdfs.org/sioc/services#service_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .
<http://demo.openlinksw.com/GData/bloguser-blog-0> <http://rdfs.org/sioc/services#service_of> <http://demo.openlinksw.com/dataspace/bloguser/weblog/bloguser_blog> .


1 Rows. -- 240 msec.
]]></programlisting>
      </sect4>
      <sect4 id="rdfsqlfromsparqldescribeexCBD">
        <title>Examples SPARQL DESCRIBE -- Option "CBD"</title>
<para><emphasis>Example 1</emphasis></para>
<programlisting><![CDATA[
SPARQL
DEFINE output:format "NICE_TTL"
DEFINE sql:describe-mode "CBD"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject FROM xmp:good1 FROM xmp:good2;

Query result:
fmtaggret-NICE_TTL
LONG VARCHAR
 @prefix ns0: <http://example.com/xmp/> .
 @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
 ns0:TheSubject ns0:details ns0:ChildObject , ( ns0:car ns0:cadr ns0:caddr ) .
]]></programlisting>
<para><emphasis>Example 2</emphasis></para>
  <programlisting><![CDATA[
SPARQL
DEFINE output:format "NICE_TTL"
DEFINE sql:describe-mode "CBD"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject  xmp:good1 from xmp:good2;
  ]]></programlisting>
<para><emphasis>Example 2</emphasis></para>
<programlisting><![CDATA[
SQL>SPARQL
DEFINE sql:describe-mode "CBD"
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
DESCRIBE ?friend
      WHERE
       {
    ?s foaf:knows ?friend  .
    ?friend foaf:nick ?nick.
    filter (?s=<http://www.advogato.org/person/rmorgan/foaf.rdf#me>)
}
;

@prefix rdf:	<http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:	<http://www.advogato.org/person/chrisd/foaf.rdf#> .
@prefix foaf:	<http://xmlns.com/foaf/0.1/> .
ns1:me	rdf:type	foaf:Person .
@prefix rdfs:	<http://www.w3.org/2000/01/rdf-schema#> .
ns1:me	rdfs:seeAlso	<http://www.advogato.org/person/chrisd/foaf.rdf> ;
	foaf:name	"Chris DiBona" ;
	foaf:nick	"chrisd" ;
	foaf:homepage	<http://www.dibona.com> ;
	foaf:mbox_sha1sum	"e8231d19ac0d11ccbdc565485054461e5d71f0d3" .
@prefix ns4:	<http://www.advogato.org/person/schoen/foaf.rdf#> .
ns1:me	foaf:knows	ns4:me .
@prefix ns5:	<http://www.advogato.org/person/jpick/foaf.rdf#> .
ns1:me	foaf:knows	ns5:me .
@prefix ns6:	<http://www.advogato.org/person/benson/foaf.rdf#> .
ns1:me	foaf:knows	ns6:me .
@prefix ns7:	<http://www.advogato.org/person/conrad/foaf.rdf#> .
ns1:me	foaf:knows	ns7:me .
@prefix ns8:	<http://www.advogato.org/person/starshine/foaf.rdf#> .
ns1:me	foaf:knows	ns8:me .
@prefix ns9:	<http://www.advogato.org/person/chip/foaf.rdf#> .
ns1:me	foaf:knows	ns9:me .
@prefix ns10:	<http://www.advogato.org/person/crackmonkey/foaf.rdf#> .
.....
]]></programlisting>
    </sect4>
      <sect4 id="rdfsqlfromsparqldescribeexOBJCBD">
        <title>Example SPARQL DESCRIBE -- Option "OBJCBD"</title>
<para><emphasis>Example 1</emphasis></para>
<programlisting><![CDATA[
SPARQL
DEFINE output:format "NICE_TTL"
DEFINE sql:describe-mode "OBJCBD"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject FROM xmp:good1 FROM xmp:good2;

Query result:
fmtaggret-NICE_TTL
LONG VARCHAR
 @prefix ns0: <http://example.com/xmp/> .
 @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
 ns0:Top1 ns0:item ns0:TheSubject .
 ns0:Top2 ns0:items [ rdf:_1 ns0:TheSubject ] .
]]></programlisting>
      </sect4>
      <sect4 id="rdfsqlfromsparqldescribeexcf">
        <title>Example SPARQL DESCRIBE -- Custom "Post Porocessing" function</title>
<programlisting><![CDATA[
create function DB.DBA.SPARQL_DESC_POSTPROC_SOURCES (inout triples any array, in good_graphs any array, in bad_graphs any array, in storage varchar, in opts any array)
{
  declare triple, val, so_dict, so_lst, addon_dict, dlst any;
  dbg_obj_princ ('DB.DBA.SPARQL_DESC_POSTPROC_SOURCES (', triples, good_graphs, bad_graphs, storage, opts, ')');
  so_dict := dict_new (dict_size (triples));
  dict_iter_rewind (triples);
  while (dict_iter_next (triples, triple, val))
    {
      declare v any;
      v := triple[0];
      dict_put (so_dict, iri_to_id_nosignal (v), 1);
      v := triple[2];
      if (isiri_id (v) or (isstring (v) and box_tag (v) = 1))
        dict_put (so_dict, iri_to_id_nosignal (v), 1);
    }
  so_lst := dict_list_keys (so_dict, 1);
  foreach (any s_itm in so_lst) do
    {
      for (sparql select distinct ?g where { graph ?g {{ `iri(?:s_itm)` ?p ?o } union { ?s ?p  `iri(?:s_itm)` }} filter (<LONG::bif:position>(?g, ?:good_graphs))}) do
        dict_put (triples, vector (s_itm, iri_to_id('Source'), "g"), 1);
    }
  return triples;
}
;

SPARQL
DEFINE output:format "NICE_TTL"
DEFINE sql:describe-mode "CBD+SOURCES"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject FROM xmp:good1 FROM xmp:good2;


SELECT sparql_to_sql_text ('
DEFINE output:format "NICE_TTL"
DEFINE sql:describe-mode "CBD+SOURCES"
PREFIX xmp: <http://example.com/xmp/>
DESCRIBE xmp:TheSubject FROM xmp:good1 FROM xmp:good2') as x long varchar;
]]></programlisting>
      </sect4>
    </sect3>
</sect2>
<sect2 id="rdfsparqlimplementatiotrans"><title>Transitivity in SPARQL</title>
  <para>Virtuoso SPARQL allows access to Virtuoso's SQL transitivity extension. Read the
<link linkend="transitivityinsQL">SQL section</link> for a definition of the options.</para>
  <para>The SPARQL syntax is slightly different from the SQL, although the option names
and meanings are the same.</para>
  <para>In SPARQL, the transitive options occur after a subquery enclosed in braces:</para>
  <para>The below produces all the IRI's that are the same as &lt;http://dbpedia.org/resource/New_York&gt;.</para>
  <programlisting><![CDATA[
SPARQL
SELECT ?syn
WHERE
  {
    {
      SELECT ?x ?syn
      WHERE
        {
          { ?x owl:sameAs ?syn }
          UNION
          { ?syn owl:sameAs ?x }
        }
    }
    OPTION ( TRANSITIVE, t_in (?x), t_out (?syn), t_distinct, t_min (0) )
    FILTER (?x = <http://dbpedia.org/resource/New_York>) .
  }
  ]]></programlisting>
  <para>In this case, we provide a binding for ?x in the filter outside of the
  transitive subquery. The subquery therefore is made to run from in to
  out. The same effect would be accomplished if we bound ?syn and
  SELECT ?x, the designations of in and out are arbitrary and for
  transitive steps that can be evaluated equally well in both directions
  this makes no difference.
  </para>
  <para>The transitive subquery in the above is </para>
  <programlisting><![CDATA[
{SELECT ?syn
 WHERE
  {
    { SELECT ?x ?syn
      WHERE
       {
         { ?x owl:sameAs ?syn }
         UNION
         { ?syn owl:sameAs ?x}
       }
    } OPTION (TRANSITIVE, t_in (?x), t_out (?syn), t_distinct, t_min (0) )
  }
} .
  ]]></programlisting>
  <para>Leaving out the option would just look for one step of owl:sameAs.
  Making it transitive will apply the subquery to all bindings it
  produces until all are visited at least once (the t_distinct modifier).
  </para>
  <para>If the transitive step consists of a single triple pattern, there is a shorthand:</para>
  <programlisting><![CDATA[
  <alice> foaf:knows ?friend option (transitive t_min (1))
  ]]></programlisting>
  <para>will bind ?friend to all directly and indirectly found foaf:known
  individuals. If t_min had been 0, Malice&gt; would have also been in the
  generated bindings.</para>
  <para>The syntax is</para>
  <programlisting><![CDATA[
  option (transitive transitivity_option[,...])

  transitivity_option ::=  t_in (<variable_list>)
  | t_out (<variable_list>)
  | t_distinct
  | t_shortest_only
  | t_no_cycles
  | t_cycles_only
  | t_min (INTNUM)
  | t_max (INTNUM)
  | t_end_flag (<variable>)
  | t_step (<variiable_or_step>)
  | t_direction INTNUM

  variable_list ::= <variable> [,...]

  variable_or_step ::= <variable> | path_id' | 'step_no'
]]></programlisting>
  <para>Unlike SQL, variable names are used instead of column numbers.
  Otherwise all the options have the same meaning.</para>
  <para>Some examples of the use of transitivity are:</para>
  <sect3 id="rdfsparqlimplementatiotransexamples"><title>Collection of Transitivity Option Demo Queries for SPARQL</title>
<sect4 id="rdfsparqlimplementatiotransexamples1"><title>Example for finding out what graphs contain owl:sameAs for "New York"</title>
  <para>To find out what graphs contain owl:sameAs for Dan York, we do</para>
<programlisting><![CDATA[
  SELECT ?g ?x count (*) as ?count
   WHERE {
           {
             SELECT ?x ?alias ?g
            WHERE {
                     { 
                       GRAPH ?g {?x owl:sameAs ?alias }
                  }
            UNION
                     {
                      GRAPH ?g {?alias owl:sameAs ?x}
                     }
                   }
           }
           OPTION ( TRANSITIVE, 
                    t_in (?x), 
                    t_out (?alias), 
                    t_distinct, 
                    t_min (1)) .
   FILTER (?x = <http://dbpedia.org/resource/New_York> ) .
         }
]]></programlisting>
  <para>Here we select all paths that start with the initial URI and pass
  through one or more sameAs statements. Each step produces a result of
  the transitive subquery. The graph where the sameAs triple was found
  is returned and used as the grouping column. In this way we see how
  many times each graph is used. Note that graphs are counted many
  times since the graphs containing immediate sameAs statements are
  counted for paths of length 1, then again as steps on paths that reach
  to their aliases and so on.</para>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples2"><title>Example for query that takes all the people known by Tim Berners-Lee, to a depth between 1 and 4 applications of the subquery</title>
  <para>This query takes all the people known by kidehen, to a depth between 1
  and 4 applications of the subquery. It then sorts them by the
  distance and the descending count of connections of each found connection. This is
  equivalent to the default connections list shown by LinkedIn.</para>
<programlisting><![CDATA[
SPARQL
  SELECT ?o ?dist ((SELECT COUNT (*) WHERE {?o foaf:knows ?xx}))
WHERE
  {
      {
        SELECT ?s ?o
        WHERE
          {
            ?s foaf:knows ?o
          }
      } OPTION ( TRANSITIVE, 
                 t_distinct, 
                 t_in(?s), 
                 t_out(?o), 
                 t_min (1), 
                 t_max (4), 
                 t_step ('step_no') as ?dist ) .
      FILTER (?s= <http://www.w3.org/People/Berners-Lee/card#i>)
    }
  ORDER BY ?dist DESC 3
  LIMIT 50
  ]]></programlisting>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples3"><title>Example for query that takes all the people known by Tim Berners-Lee, to a depth between 2 and 4 applications of the subquery</title>
  <para>This query takes all the people known by kidehen, to a depth between 2
  and 4 applications of the subquery. It then sorts them by the
  distance and the descending count of connections of each found connection. This is
  equivalent to the default connections list shown by LinkedIn.</para>
  <programlisting><![CDATA[
  SPARQL
  SELECT ?o ?dist ((SELECT COUNT (*) WHERE {?o foaf:knows ?xx}))
  WHERE
    {
      {
        SELECT ?s ?o
        WHERE
          {
            ?s foaf:knows ?o
          }
      } OPTION ( TRANSITIVE, 
                 t_distinct, 
                 t_in(?s), 
                 t_out(?o), 
                 t_min (2), 
                 t_max (4), 
                 t_step ('step_no') as ?dist) .
      FILTER (?s= <http://www.w3.org/People/Berners-Lee/card#i>)
    }
  ORDER BY ?dist DESC 3
  LIMIT 50
  ]]></programlisting>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples4"><title>Example for finding how two people know each other and what graphs are involved in the connection</title>
  <para>To find how two people know each other and what graphs are involved in the connection, we do:</para>
  <programlisting><![CDATA[
  SPARQL
  SELECT ?link ?g ?step ?path
  WHERE
    {
      {
        SELECT ?s ?o ?g
        WHERE
          {
            graph ?g {?s foaf:knows ?o }
          }
      } OPTION ( TRANSITIVE, 
                 t_distinct, 
                 t_in(?s), 
                 t_out(?o), 
                 t_no_cycles, 
                 T_shortest_only,
                 t_step (?s) as ?link, 
                 t_step ('path_id') as ?path, 
                 t_step ('step_no') as ?step, 
                 t_direction 3) .
      FILTER (?s= <http://www.w3.org/People/Berners-Lee/card#i>
      && ?o = <http://www.advogato.org/person/mparaz/foaf.rdf#me>)
  }
LIMIT 20
]]></programlisting>
  <para>This query binds both the t_in and t_out variables. The ?g is left as
  a free variable. Also, specifying ?s and the system defined constants
  step_no and path_id as with t_step, we get for each transitive step a
  row of results with the intermediate binding of ?s, the count of steps
  from the initial ?s and a distinct identifier for the individual path,
  since there can be many distinct paths that link the ?s and ?o
  specified in the filter.</para>
  <para>See the SQL transitive option section for details on the meaning of step_no and path_id.</para>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples5"><title>Example for TBox Subsumption</title>
<para>Subsumption Demo Using Transitivity Clause</para>
<para>Yago Class Hierarchy (TBox) Subsumption</para>
<para>AlphaReceptors</para>
<programlisting><![CDATA[
# all subjects with IRI: <http://dbpedia.org/class/yago/AlphaReceptor105609111>, 
# that are sub-classes of anything (hence ?y) 
# without restrictions on tree levels	
SELECT ?y
FROM <http://dbpedia.org/resource/classes/yago#>
WHERE
  {
    {
      SELECT *
      WHERE
        {
          ?x rdfs:subClassOf ?y .
        }
    }
    OPTION (TRANSITIVE, t_distinct, t_in (?x), t_out (?y) ) .
    FILTER (?x = <http://dbpedia.org/class/yago/AlphaReceptor105609111>)
  }
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples6"><title>Example for Receptors</title>
<programlisting><![CDATA[
SELECT ?x
FROM <http://dbpedia.org/resource/classes/yago#>
WHERE
  {
    {
      SELECT *
      WHERE
        {
          ?x rdfs:subClassOf ?y .
        }
    } OPTION (transitive, t_distinct, t_in (?x), t_out (?y) ) .
  FILTER (?y = <http://dbpedia.org/class/yago/Receptor105608868>)
}
]]></programlisting>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples7"><title>Inference Rule example using transitive properties from SKOS vocabulary</title>
<p>The following example demostrates the steps how to retrieve the skos ontology, add triples
for skos:broaderTransitiveinto the graph, define inference rule, and at the and
execute sparql query with inference rule and transitivity option. The queries were executed against
the LOD instance (http://lod.openlinksw.com):</p>
<orderedlist>
  <listitem>Make the Context graph, assuming you don't want to load entire SKOS vocabulary into our Quad Store:
<programlisting><![CDATA[
SQL>SPARQL
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
INSERT INTO GRAPH <urn:rules.skos> { skos:broader rdfs:subPropertyOf skos:broaderTransitive .
                                     skos:narrower rdfs:subPropertyOf skos:narrowerTransitive };
]]></programlisting>
</listitem>
  <listitem>OR Load entire SKOS ontology into Quad Store via iSQL interface (commandline or HTML based Conductor):
<programlisting><![CDATA[
SQL>DB.DBA.RDF_LOAD_RDFXML (http_get ('http://www.w3.org/2009/08/skos-reference/skos-owl1-dl.rdf'), 'no', 'urn:rules.skos');
Done.
]]></programlisting>
  </listitem>
  <listitem>Make Context Rule:
<programlisting><![CDATA[
SQL>rdfs_rule_set ('skos-trans', 'urn:rules.skos');
Done.
]]></programlisting>
  </listitem>
  <listitem>Go to SPARQL endpoint, for ex. http://lod.openlinksw.com/sparql</listitem>
  <listitem>Use inference rule pragma to set context rule for SPARQL query, i.e:
<programlisting><![CDATA[
SPARQL
DEFINE input:inference "skos-trans"
PREFIX p: <http://dbpedia.org/property/>
PREFIX dbpedia: <http://dbpedia.org/resource/>
PREFIX category: <http://dbpedia.org/resource/Category:>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX geo: <http://www.georss.org/georss/>

SELECT DISTINCT ?m ?n ?p ?d
WHERE
  {
    ?m rdfs:label ?n.
    ?m skos:subject ?c.
    ?c skos:broaderTransitive category:Churches_in_Paris OPTION (TRANSITIVE) .
    ?m p:abstract ?d.
    ?m geo:point ?p
    FILTER ( lang(?n) = "fr" )
    FILTER ( lang(?d) = "fr" )
  }
]]></programlisting>
</listitem>
  <listitem>You will get 22 rows returned from the query.
Note that for comparison, if the option (transitive) is ommitted, then only 2 rows will be returned
in our example query:
     <figure id="rdftr" float="1">
       <title>Transitive option</title>
       <graphic fileref="ui/trs1.png"/>
     </figure>
</listitem>
</orderedlist>
</sect4>
<sect4 id="rdfsparqlimplementatiotransexamples8"><title>Inference Rule example using
transitive properties from SKOS vocabulary: Variant II</title>
<p>This example shows how to find entities that are subcategories of Protestant Churches, no deeper
than 3 levels within the concept scheme hierarchy, filtered by a specific subcategory. It demonstrates
use of inference rules, sub-queries, and filter to obtain entities associated with category:
Protestant_churches combined with the use of the transitivitve closure, sets to a maximum of 3 steps down a SKOS based concept scheme hierarchy:</p>
<orderedlist>
  <listitem>Make sure the inference rule "skos-trans" is created as described in the previous <link linkend="rdfsparqlimplementatiotransexamples7">example</link></listitem>
  <listitem>Go to SPARQL endpoint, for ex. http://lod.openlinksw.com/sparql</listitem>
  <listitem>Use inference rule pragma to set context rule for SPARQL query, i.e:
<programlisting><![CDATA[
DEFINE input:inference "skos-trans"
PREFIX p: <http://dbpedia.org/property/>
PREFIX dbpedia: <http://dbpedia.org/resource/>
PREFIX category: <http://dbpedia.org/resource/Category:>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX geo: <http://www.georss.org/georss/>

SELECT DISTINCT ?c AS ?skos_broader
       ?trans AS ?skos_narrower
       ?dist AS ?skos_level
       ?m ?n ?p AS ?geo_point
WHERE
  {
    {
      SELECT ?c  ?m ?n ?p ?trans ?dist
      WHERE
        {
  	  ?m rdfs:label ?n.
  	  ?m skos:subject ?c.
  	  ?c skos:broaderTransitive category:Protestant_churches .
  	  ?c skos:broaderTransitive ?trans
                OPTION ( TRANSITIVE, 
                         t_distinct, 
                         t_in (?c), 
                         t_out (?trans),
                         t_max (3),  
                         t_step ( 'step_no' ) as ?dist ) .
  	  ?m p:abstract ?d.
  	  ?m geo:point ?p
  	  FILTER ( lang(?n) = "en" )
  	  FILTER ( lang(?d) = "en" )
        }
    }
    FILTER ( ?trans = <http://dbpedia.org/resource/Category:Churches_in_London> )
  }
ORDER BY ASC (?dist)
]]></programlisting>
</listitem>
  <listitem>You will get 22 rows returned from the query.
     <figure id="rdftr" float="1">
       <title>Transitive option</title>
       <graphic fileref="ui/trs2.png"/>
     </figure>
</listitem>
</orderedlist>
</sect4>
</sect3>
</sect2>
<sect2 id="rdfsparqlimplementatioptragmas"><title>Supported SPARQL-BI "define" pragmas</title>
<para>SPARQL-BI compiler and run-time support are not isolated from environment and some used heuristics 
	are not perfect and sometimes different use cases require different behavior within same standard. 
	These reasons are seen frequently in the industry, and the solution is well known: compiler pragmas. 
	So we allow them at the beginning of any SPARQL query in form:
</para>
<programlisting><![CDATA[
define QName value
]]></programlisting>
<sect3 id="rdfsparqlimplementatioptragmascinp"><title>Pragmas to control the input data set for 
the query</title>
<programlisting><![CDATA[
input:default-graph-uri works like "FROM" clause;
input:named-graph-uri works like "FROM NAMED" clause;
input:default-graph-exclude works like "NOT FROM" clause;
input:named-graph-exclude works like "NOT FROM NAMED" clause.
]]></programlisting>
<para>The difference is that these pragmas have higher priority and they can be used for security 
	restrictions in combination with <code>define input:freeze</code> that blocks further changes in 
	the list of source graphs. The web service endpoint (or similar non-web application) can edit 
	the incoming query by placing list of pragmas ended with <code>input:freeze</code> in front of 
	query text. Even if the intruder will try to place some graph names, it will get compilation error, 
	not an access to the data. input:freeze disables all <code>input:grab-...</code> pragmas as well.
</para>
<itemizedlist mark="bullet">
  <listitem><code>input:ifp</code>: adds IFP keyword in OPTION (QUIETCAST, ...) clause in the generated 
  SQL. The value of this define is not used ATM, an empty string is safe for future extensions.</listitem>
  <listitem><code>input:same-as</code>: works like input:ifp but adds SAME_AS keyword.</listitem>
  <listitem><code>input:storage</code>: selects quad map storage to use. The value is an IRI of storage, 
  the default value is, of course, virtrdf:DefaultQuadStorage. If the value is an empty string then only 
  quads from RDF_VIEW are used. This is a good choice for low-level admin procedures, for two reasons: 
  they will not interfere with any changes in virtrdf:DefaultQuadStorage and they will continue to work 
  even if all compiler's metadata are corrupted, including the description of virtrdf:DefaultQuadStorage 
  (define input:storage "" switches the SPARQL compiler to a small set of metadata that are built in 
  server C code and thus are very hard to corrupt by users).</listitem>
  <listitem><code>input:inference</code>: specifies the name of inference rule set to use.</listitem>
  <listitem><code>input:param (and synonyms input:params, sql:param, sql:params)</code>: declares a 
  variable name as a protocol parameter. The SPARQL query can refer to protocol parameter X via 
  variable with special syntax of name ?::X . If a query text should be made by query builder that 
  does not understand SPARQL-BI extensions then the text may contain variable ?X and define 
  input:param "X" . This does not work for positional parameters, one can not replace a reference 
  to ?::3 with ?3 and define input:param "3".</listitem>
  <listitem><code>input:grab-var</code>: Network Resource Fetch values of variable;</listitem>
  <listitem><code>input:grab-iri</code>: Network Resource Fetch the constant IRI;</listitem>
  <listitem><code>input:grab-all</code>: Network Resource Fetch all constants and variables of the query;</listitem>
  <listitem><code>input:grab-seealso (and synonym input:grab-follow-predicate)</code>: sets predicate 
  that tells where to Fetch more Network Resource data about a subject;</listitem>
  <listitem><code>input:grab-limit</code>: how many resources can be fetched;</listitem>
  <listitem><code>input:grab-depth</code>: how many iterations can be done, sponging additional data 
  on each iteration;</listitem>
  <listitem><code>input:grab-base</code>: base to resolve relative IRIs before passing to Sponger;</listitem>
  <listitem><code>input:grab-resolver</code>: IRI resolving procedure (i.e., one that turns base and 
  relative IRI to an absolute IRI);</listitem>
  <listitem><code>input:grab-destination</code>: single resource that should be filled in with results 
  of all fetchings;</listitem>
  <listitem><code>input:grab-loader</code>: a name of procedure that retrieve the resource via HTTP, 
  parse it and store it.</listitem>
</itemizedlist>
<para>All these pragmas are described in more details 
	<link linkend="rdfinputgrab">here</link>, 
	but in addition there are some experimental:
</para>
<itemizedlist mark="bullet">
  <listitem><code>input:grab-intermediate</code>: extends the set of IRIs to sponge, useful in 
  combination with input:grab-seealso. If present then for a given subject, Network Resource Fetch will retrieve 
  not only values of see-also predicates for that subject but the subject itself. The define value 
  is not used in current implementation.</listitem>
  <listitem><code>input:grab-group-destination</code>: resembles input:grab-destination but sponges will 
  create individual graphs for Network Resource Fetch results, and in additional to this common routine, a copy of each 
  Network Resource Fetch result is added to the resource specified by the value of input:grab-group-destination. 
  input:grab-destination redirects loadings, input:grab-group-destination duplicates them.</listitem>
  <listitem><code>get:soft</code>: "soft" or "replacing", depending on mode of loading source graph;
  </listitem>
  <listitem><code>get:uri</code>: an URI of web resource where the graph should come from (e.g., a local 
  mirror);</listitem>
  <listitem><code>get:method</code>: "GET" or "MGET", depending on loading the resource itself or loading 
  metadata about the resource;</listitem>
  <listitem><code>get:refresh</code>: limits the lifetime of a local cached copy of the source, the value 
  is in seconds. Should be used in combination with get:soft;</listitem>
  <listitem><code>get:proxy</code>: the proxy server to use, as "host:port" string.</listitem>
</itemizedlist>
<para>These defines are described also <link linkend="rdfinputgrab">here</link>. Note that all of them can be used in option list of "FROM ... OPTION (get:... )" extended SPARQL-BI syntax for FROM/FROM NAMED clause.
</para>
<para>Note that all of them can be used in option list of "FROM ... OPTION (get:... )" extended SPARQL-BI syntax for FROM/FROM NAMED clause.
</para>
</sect3>
<sect3 id="rdfsparqlimplementatioptragmasccg"><title>Pragmas to control code generation</title>
<itemizedlist mark="bullet">
  <listitem><code>sql:assert-user</code>: defines the user who is supposed to be the single "proper" use 
  for the query. If the compiler is launched by other user, an error is signaled. The typical use is 
  define sql:assist-user "dba". This is too weak to be a security measure, but may help in debugging 
  of security issues.</listitem>
  <listitem><code>sql:gs-app-callback</code>: application-specific callback that returns permission 
  bits of a given graph;</listitem>
  <listitem><code>sql:gs-app-uid</code>: application-specific user id to use in callback.
    <tip><title>See Also:</title>
    <para><link linkend="rdfgraphsecurityappcallb">RDF Graph Security</link></para>
  </tip>
  </listitem>
  <listitem><code>sql:globals-mode</code>: tells how to print names of global variables, supported 
  values are "XSLT" (print colon before name of global variable and "SQL" (print as usual).</listitem>
  <listitem><code>sql:log-enable</code>: value that will be passed to SPARUL procedures and there 
  it will be passed to log_enable() BIF. Thus define sql:log-enable N will result in log_enable(N, 1) 
  at the beginning of the operation and other log_enable() call will restore previous mode of 
  transaction log at exit from the procedure or at any error signalled from it.</listitem>
  <listitem><code>sql:table-option</code>: value will be added as an option to each triple in the 
  query and later it will be printed in TABLE OPTION (...) clause of source table clause. This 
  works only for SQL code for plain triples from RDF_QUAD, fragments of queries related to RDF 
  Views will remain unchanged.</listitem>
  <listitem><code>sql:select-option</code>: value will be added as an global OPTION () clause of 
  the generated SQL SELECT. This clause is always printed, it is always at least OPTION (QUIETCAST, ...). 
  The most popular use case is define sql:table-option "ORDER" to tell the SQL compiler execute joins 
  in the order of their use in the query (this can make query compilation much faster but the 
  compilation result can be terrible if you do not know precisely what you're doing and not inspected 
  execution plan of the generated SQL query).</listitem>
  <listitem><code>sql:describe-mode</code>: sets procedures that will produce the result of a DESCRIBE 
  query. The pragma is ignored for other types of SPARQL queries. In the default mode, the result 
  contains all X ?p ?o and all ?s ?p X triples for each given X. In "SPO" mode, the result contains 
  X ?p ?o triples only. In "CBD" mode, the result contains concise bound descriptions of given 
  subjects. Application developers may add more modes.</listitem>
  <listitem><code>sql:signal-void-variables</code>: the most useful debugging variable if Linked Data Views 
  are in use. It tells the SPARQL compiler to signal an error if it can prove that some variable can 
  never be bound. Usually it means error in query, like typo in IRI or totally wrong triple 
  pattern.</listitem> 
</itemizedlist>
</sect3>
<sect3 id="rdfsparqlimplementatioptragmasctr"><title>Pragmas to control the type of the result</title>
<itemizedlist mark="bullet">
  <listitem><code>output:valmode</code>: tells the compiler which SQL datatypes should be used for 
  output values. ODBC clients and the like known nothing about RDF and expect plain SQL values, 
  so the appropriate value for them is "SQLVAL" and that's the default. When a Virtuoso/PL procedure 
  is RDF-aware and keeps results for further passing to other SPARQL queries or some low-level RDF 
  routines, the value "LONG" tells the compiler to preserve RDF boxes as is and to return IRI IDs 
  instead of IRI string value. Third possible value, "AUTO", is for dirty hackers that do not want 
  any conversion of any sort at the output to read the SQL output of SPARQL front-end, find the 
  format of each column and add the needed conversions later. You will probably never need it.</listitem>
  <listitem><code>output:format</code>: tells the compiler that the query should produce a string 
  output with the serialization of the result, not a result set. There are three of them because 
  the caller, like SPARQL web service endpoint, may not know the actual type of the query that 
  should be executed. The value of output:format is used for SELECT and data manipulation queries, 
  if specified, it can also be used for CONSTRUCT, DESCRIBE or ASK, if it is specified but related 
  output:dict-format or output:scalar-format is not.</listitem>
  <listitem><code>output:scalar-format</code>: tells the compiler that the query should produce a 
  string output with the serialization of the result, not a result set. There are three of them 
  because the caller, like SPARQL web service endpoint, may not know the actual type of the query 
  that should be executed. The value of output:scalar-format is used for ASK queries only, if 
  specified.</listitem>
  <listitem><code>output:dict-format</code>: tells the compiler that the query should produce 
  a string output with the serialization of the result, not a result set. There are three of 
  them because the caller, like SPARQL web service endpoint, may not know the actual type of 
  the query that should be executed. The value of output:dict-format is used for CONSTRUCT 
  and DESCRIBE queries only, if specified.</listitem>
</itemizedlist>
</sect3>
<sect3 id="rdfsparqlimplementatioptragmassfs"><title>Supported formats that return a string session</title>
<itemizedlist mark="bullet">
  <listitem>"RDF/XML",</listitem>
  <listitem>"TURTLE" (and "TTL" is a synonym),</listitem>
  <listitem>"JSON" (canonical JSON for result sets, Talis-style JSON for CONSTRUCT and DESCRIBE),</listitem>
  <listitem>"JSON;ODATA" (oData-style JSON for CONSTRUCT and DESCRIBE, error otherwise),</listitem>
  <listitem>"RDFA;XHTML" (only for CONSTRUCT and DESCRIBE, error otherwise),</listitem>
  <listitem>"ATOM;XML" (only for CONSTRUCT and DESCRIBE as well).</listitem>
</itemizedlist>
</sect3>
<sect3 id="rdfsparqlimplementatioptragmassdfs"><title>Supported formats that do not return a string 
session to the caller</title>
<para>Supported formats that do not return a string session to the caller, but form an HTTP response 
	instead and send it directly to the client HTTP connection with an appropriate HTTP header:
</para>
<itemizedlist mark="bullet">
  <listitem>"HTTP+XML mime/type",</listitem>
  <listitem>"HTTP+TTL mime/type",</listitem>
  <listitem>"HTTP+NT mime/type". A MIME type in value will be placed in the returned header, 
  it should be separated from the starting keyword with one white space.</listitem>
</itemizedlist>
</sect3>
<sect3 id="rdfsparqlimplementatioptragmassspfs"><title>Supported Special formats</title>
<para>A special format "_JAVA_" is for SPARQL queries sent via JDBC. It changes only the output of ASK queries.</para>
<para>The "_JAVA_" and "_UDBC_" are aliases in Virtuoso Version 6.1.5. Till Virtuoso 6.1.5 the default 
	behaves as "TTL". For Virtuoso version 6.1.5 and higher it is ODBC/JDBC oriented e.g. "_UDBC_" is 
	the default format for ODBC/JDBC clients.</para>
  <para><emphasis>Note</emphasis>: If you want to revert to old TTL behaviour, you should specify it explicitly via:</para>
<programlisting><![CDATA[
define output:format "TTL"	
]]></programlisting>  
<para>Note: Pragmas output:valmode and output:format may conflict if used together, and if 
	they're not in conflict then output:valmode is redundant: the compiler knows for sure which 
	output:valmode-s are needed by various output:format-s.
</para>
<itemizedlist mark="bullet">
  <listitem><code>output:route</code>: works only for SPARUL operators and tells the SPARQL compiler 
  to generate procedure names that differ from default. As a result, the effect of operator will 
  depend on application. That is for tricks. E.g., consider an application that extracts metadata 
  from DAV resources stored in the Virtuoso and put them to RDF storage to make visible from outside. 
  When a web application has permissions and credentials to execute a SPARUL query, the changed metadata 
  can be written to the DAV resource (and after that the trigger will update them in the RDF storage), 
  transparently for all other parts of application.</listitem>
  <listitem><code>output:maxrows</code>: limits the number of rows in the returned result set. The 
  integer value is expected, the positive integer value is obviously recommended.</listitem>
</itemizedlist>
</sect3>
<sect3 id="rdfsparqlimplementatioptragmasmnotes"><title>Minor notes</title>
<para>Values of most pragmas are strings. Exceptions are:
</para>
<itemizedlist mark="bullet">
  <listitem>input:grab-depth,</listitem>
  <listitem>input:grab-limit,</listitem>
  <listitem>output:maxrows,</listitem>
  <listitem>sql:log-enable,</listitem>
  <listitem>sql:signal-void-variables</listitem>
</itemizedlist>
<para>that have integer values.
</para>
<para>Values of some pragmas a passed through the compiler to the run-time so they are seen in the 
	generated SQL code as arguments of procedures:
</para>
<itemizedlist mark="bullet">
  <listitem>get:method,</listitem>
  <listitem>get:proxy,</listitem>
  <listitem>get:query,</listitem>
  <listitem>get:refresh,</listitem>
  <listitem>get:soft,</listitem>
  <listitem>get:uri</listitem>  
</itemizedlist>
<para>so sometimes you may meet them in SQL debuggers output and the like.
</para>
</sect3>
</sect2>
<sect2 id="rdfsparqlbif"><title>Built-in bif functions</title>
<itemizedlist mark="bullet">
  <listitem><emphasis>bif:__rdf_long_from_batch_params(i_nt integer, st_value, st2_value)</emphasis>
    <itemizedlist mark="bullet">
      <listitem>For value URI, the params values should be: 1, value.stringValue(), NULL</listitem>
      <listitem>For value BNODE, the params values should be: 1, "_:"+((BNode)value).getID(), NULL</listitem>
      <listitem>For value Literal with Language!=NULL, the params values should be: 5, lit.stringValue(), lit.getLanguage()</listitem>
      <listitem>For value Literal with Datatype!=NULL, the params values should be: 4, lit.stringValue(), lit.getDatatype().toString()</listitem>
      <listitem>For value Literal with Datatype==NULL &amp;&amp; Language==NULL, the params values should be: 3, lit.stringValue(), NULL</listitem>
      <listitem>For value any value exclude above, the params values should be: 3, value.stringValue(), NULL</listitem>
      <listitem>For string value (without Datatype and Language), the params values should be: 3, value.stringValue(), NULL</listitem>
    </itemizedlist>  
    <para><emphasis>Example:</emphasis></para>
<programlisting><![CDATA[
SPARQL SELECT * 
WHERE 
  { graph ?g { `iri(??)` `iri(??)`
    `bif:__rdf_long_from_batch_params(3,value.stringValue(),NULL)` }
  }
]]></programlisting>
  </listitem>
</itemizedlist>
</sect2>
<sect2 id="rdfsparqlsoap"><title>Sending SOAP Requests to Virtuoso SPARQL Endpoint</title>
  <para>This section presents a sample scenario on how to execute a SPARQL query as a SOAP 
  	request to the Virtuoso SPARQL Endpoint.
  </para>	
  <orderedlist>
    <listitem>Assume the following sample SOAP request containing simple SPARQL query:
<programlisting><![CDATA[
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
      <query-request xmlns="http://www.w3.org/2005/09/sparql-protocol-types/#">
         <query xmlns="">SELECT DISTINCT ?z FROM virtrdf: {?x ?y ?z .} LIMIT 10</query>
      </query-request>
   </soapenv:Body>
</soapenv:Envelope>	
]]></programlisting>    	
    </listitem>
    <listitem>Save locally the content from above for ex. to file with the name "soap.xml".</listitem>
    <listitem>To pass the SOAP request to a Virtuoso SPARQL Endpoint, execute the following curl command:
<programlisting><![CDATA[
$ curl -d@soap.xml -H "Content-Type:text/xml" -H "SOAPAction: ''" http://example.com/sparql
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <soapenv:Body>
    <query-result xmlns="http://www.w3.org/2005/09/sparql-protocol-types/#">
      <sparql xmlns="http://www.w3.org/2005/sparql-results#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2001/sw/DataAccess/rf1/result2.xsd">
        <head>
         <variable name="z"/>
        </head>
        <results distinct="false" ordered="true">
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadMapFormat</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadStorage</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapFormat</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadMap</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadMapValue</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapColumn</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadMapColumn</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#array-of-QuadMapATable</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadMapATable</uri>
            </binding>
          </result>
          <result>
            <binding name="z">
             <uri>http://www.openlinksw.com/schemas/virtrdf#QuadMapFText</uri>
            </binding>
          </result>          
        </results>        
      </sparql>
    </query-result>
  </soapenv:Body>
</soapenv:Envelope>	
]]></programlisting>    	
    </listitem>
  </orderedlist>  
</sect2>
<sect2 id="rdfsparqlhashjoin"><title>Use of Hash Join With RDF</title>
  <para>For queries that touch large quantities of RDF data and have many selection confitions use of hash join is 
often desirable. For short lookup queries hash join is usually not desirable.</para>
  <para>Depending on the version, the query optimizer may of may not do hash joins with RDF. This is controlled 
  by the <emphasis>hash_join_enable</emphasis> flag.</para>
  <para>To check the flag do:</para>
<programlisting><![CDATA[
sys_stat ('hash_join_enable');
]]></programlisting>
  <itemizedlist mark="bullet">
    <listitem>0 -- means that hash joins are never made</listitem>
    <listitem>1 -- means that these are for SQL only</listitem>
    <listitem>2 -- means that these can also be used wih RDF</listitem>  
  </itemizedlist>
  <para>The flag is set in the ini file in the [Flags] section:</para>
<programlisting><![CDATA[
[Flags]
hash_join_enable = 2
]]></programlisting>
   <para>The flag may be transiently set with the SQL statement:</para>  
<programlisting><![CDATA[
__dbf_set ('hash_join_enable', 2);
]]></programlisting>
   <para>To check the effectiveness of hash joins and whether the optimizer introduces these in the first place, 
   it is most convenient to use the <emphasis>profile</emphasis> function.</para>
   <para>In the following example, we look at the star schema benchmark Q4:</para>
<programlisting><![CDATA[
SPARQL PREFIX rdfh: <http://lod2.eu/schemas/rdfh#>
SELECT SUM(?rdfh_lo_revenue) AS ?lo_revenue ?d_year ?p_brand1 
FROM <http://lod2.eu/schemas/rdfh-inst#ssb1_ttl_qb>
WHERE 
  {
    ?li a rdfh:lineorder ;
        rdfh:lo_orderdate ?lo_orderdate ;
        rdfh:lo_partkey ?lo_partkey ;
        rdfh:lo_suppkey ?lo_suppkey ;
	      rdfh:lo_revenue ?rdfh_lo_revenue .

    ?lo_orderdate rdfh:d_year ?d_year .
    ?lo_partkey rdfh:p_brand1 ?p_brand1 .
    ?lo_partkey rdfh:p_category "MFGR#12" .
    ?lo_suppkey rdfh:s_region "AMERICA" . 
  } 
GROUP BY ?d_year ?p_brand1 
ORDER BY ?d_year ?p_brand1
;
]]></programlisting>
  <para>The query aggregates rows from a large fact table and selects based on a date range, a brand and 
  the location of the supplier. To run this, it is best to put the query in a file and have profile 
  ('sparql... ') wrapped around the text. Then in isql:</para>
<programlisting><![CDATA[
SET SET BLOBS ON;
LOAD q4.sql;
]]></programlisting>
  <para>Without hash join the profile is:</para>
<programlisting><![CDATA[
{ 
time   1.9e-06% fanout         1 input         1 rows
 
Precode:
      0: __rdflit := Call __rdflit (rdflit170373)
      5: __rdflit := Call __rdflit (rdflit16802)
      10: BReturn 0
Subquery 31 
{ 
time     1e-06% fanout         1 input         1 rows
{ fork
time   0.00035% fanout         1 input         1 rows
{ fork
time       3.6% fanout     1e+06 input         1 rows
RDF_QUAD     1e+06 rows(s_18_9_t6.S, s_18_9_t6.O)
 inlined  P =  #dfh#p_brand1  G =  #nst#ssb1_ttl_qb 
time       1.7% fanout   0.03979 input     1e+06 rows
RDF_QUAD_POGS unq      0.04 rows (s_18_9_t7.S)
 P =  #dfh#p_category  ,  O = rdflit170373 ,  S = s_18_9_t6.S ,  G =  #nst#ssb1_ttl_qb 
time       2.5% fanout   180.179 input     39790 rows
 
Precode:
      0: __ro2sq := Call __ro2sq (s_18_9_t6.O)
      5: BReturn 0
RDF_QUAD_POGS   4.4e+02 rows(s_18_9_t2.S)
 P =  #dfh#lo_partkey  ,  O = k_s_18_9_t6.S G =  #nst#ssb1_ttl_qb 
time        35% fanout         1 input 7.16932e+06 rows
RDF_QUAD         1 rows(s_18_9_t3.O, s_18_9_t3.S)
 inlined  P =  #dfh#lo_suppkey  ,  S = s_18_9_t2.S G =  #nst#ssb1_ttl_qb 
time       4.5% fanout  0.201214 input 7.16932e+06 rows
RDF_QUAD_POGS unq       0.2 rows ()
 P =  #dfh#s_region  ,  O = rdflit16802 ,  S = cast ,  G =  #nst#ssb1_ttl_qb 
time        21% fanout         1 input 1.44256e+06 rows
RDF_QUAD         1 rows(s_18_9_t4.S, s_18_9_t4.O)
 inlined  P =  #dfh#lo_revenue  ,  S = k_s_18_9_t2.S G =  #nst#ssb1_ttl_qb 
time        12% fanout         1 input 1.44256e+06 rows
RDF_QUAD_POGS unq       0.8 rows (s_18_9_t0.S)
 P =  #-ns#type  ,  O =  #dfh#lineorder  ,  S = k_s_18_9_t2.S ,  G =  #nst#ssb1_ttl_qb 
time        14% fanout         1 input 1.44256e+06 rows
RDF_QUAD         1 rows(s_18_9_t1.O)
 inlined  P =  #dfh#lo_orderdate  ,  S = s_18_9_t0.S G =  #nst#ssb1_ttl_qb 
time       3.5% fanout         1 input 1.44256e+06 rows
RDF_QUAD         1 rows(s_18_9_t5.O)
 inlined  P =  #dfh#d_year  ,  S = cast G =  #nst#ssb1_ttl_qb 
time       1.9% fanout         0 input 1.44256e+06 rows
Sort (s_18_9_t5.O, s_18_9_t6.O) -> (s_18_9_t4.O, __ro2sq)
 
}
time   4.1e-05% fanout       280 input         1 rows
group by read node  
(s_18_9_t5.O, s_18_9_t6.O, aggregate, __ro2sq)
time   0.00043% fanout         0 input       280 rows
 
Precode:
      0: __ro2sq := Call __ro2sq (s_18_9_t5.O)
      5: BReturn 0
Sort (__ro2sq, __ro2sq) -> (aggregate)
 
}
time   2.9e-05% fanout       280 input         1 rows
Key from temp (aggregate, __ro2sq, __ro2sq)
 
 
After code:
      0: lo_revenue :=  := artm aggregate
      4: d_year :=  := artm __ro2sq
      8: p_brand1 :=  := artm __ro2sq
      12: BReturn 0
time   7.6e-07% fanout         0 input       280 rows
Subquery Select(lo_revenue, d_year, p_brand1)
}
 
After code:
      0: lo_revenue := Call __ro2sq (lo_revenue)
      5: d_year := Call __ro2sq (d_year)
      10: p_brand1 := Call __ro2sq (p_brand1)
      15: BReturn 0
time   6.3e-07% fanout         0 input       280 rows
Select (lo_revenue, d_year, p_brand1)
}


 5542 msec 2420% cpu, 2.11877e+07 rnd 8.13668e+06 seq   85.6039% same seg   13.6018% same pg 
Compilation: 10 msec 0 reads         0% read 0 messages         0% clw


  <para>With hash join the profile is:</para>
<programlisting><![CDATA[
{ 
time   1.4e-05% fanout         1 input         1 rows
time         7% fanout         1 input         1 rows
 
Precode:
      0: __rdflit := Call __rdflit (rdflit170373)
      5: __rdflit := Call __rdflit (rdflit16802)
      10: BReturn 0
{ hash filler
time     0.088% fanout     1e+06 input         1 rows
RDF_QUAD     1e+06 rows(s_18_9_t6.S, s_18_9_t6.O)
 inlined  P =  #dfh#p_brand1  G =  #nst#ssb1_ttl_qb 
time      0.15% fanout         0 input     1e+06 rows
Sort hf 39 (s_18_9_t6.S, s_18_9_t6.S) -> (s_18_9_t6.O)
 
}
time   0.00046% fanout         1 input         1 rows
{ hash filler
time    0.0004% fanout      2556 input         1 rows
RDF_QUAD_POGS   2.6e+03 rows(s_18_9_t5.S, s_18_9_t5.O)
 inlined  P =  #dfh#d_year  G =  #nst#ssb1_ttl_qb 
time   0.00056% fanout         0 input      2556 rows
Sort hf 56 (s_18_9_t5.S) -> (s_18_9_t5.O)
 
}
time    0.0036% fanout         1 input         1 rows
{ hash filler
time   0.00094% fanout     12068 input         1 rows
RDF_QUAD_POGS   1.2e+04 rows(s_18_9_t8.S)
 P =  #dfh#s_region  ,  O = rdflit16802 G =  #nst#ssb1_ttl_qb 
time   0.00046% fanout         0 input     12068 rows
Sort hf 69 (s_18_9_t8.S)
}
time     0.012% fanout         1 input         1 rows
{ hash filler
time    0.0026% fanout     39790 input         1 rows
RDF_QUAD_POGS     4e+04 rows(s_18_9_t7.S)
 P =  #dfh#p_category  ,  O = rdflit170373 G =  #nst#ssb1_ttl_qb 
time    0.0013% fanout         0 input     39790 rows
Sort hf 82 (s_18_9_t7.S)
}
Subquery 88 
{ 
time   1.5e-05% fanout         1 input         1 rows
{ fork
time   1.3e-05% fanout         1 input         1 rows
{ fork
time        52% fanout 7.16932e+06 input         1 rows
RDF_QUAD   1.8e+08 rows(s_18_9_t2.O, s_18_9_t2.S)
 inlined  P =  #dfh#lo_partkey  G =  #nst#ssb1_ttl_qb 
hash partition+bloom by 86 (tmp)hash join merged always card      0.04 -> ()
time       6.1% fanout         1 input 7.16932e+06 rows
 
Precode:
      0: s_18_9_t7.S :=  := artm s_18_9_t2.O
      4: BReturn 0
Hash source 82 merged into ts       0.04 rows(cast) -> ()
time         7% fanout  0.201214 input 7.16932e+06 rows
RDF_QUAD         1 rows(s_18_9_t3.O, s_18_9_t3.S)
 inlined  P =  #dfh#lo_suppkey  ,  S = s_18_9_t2.S G =  #nst#ssb1_ttl_qb 
hash partition+bloom by 73 (tmp)hash join merged always card       0.2 -> ()
time    0.0018% fanout         1 input 1.44256e+06 rows
Hash source 69 merged into ts        0.2 rows(cast) -> ()
time       2.3% fanout         1 input 1.44256e+06 rows
RDF_QUAD_POGS unq       0.8 rows (s_18_9_t0.S)
 P =  #-ns#type  ,  O =  #dfh#lineorder  ,  S = k_s_18_9_t2.S ,  G =  #nst#ssb1_ttl_qb 
time       2.3% fanout         1 input 1.44256e+06 rows
RDF_QUAD         1 rows(s_18_9_t1.O, s_18_9_t1.S)
 inlined  P =  #dfh#lo_orderdate  ,  S = s_18_9_t0.S G =  #nst#ssb1_ttl_qb 
hash partition+bloom by 60 ()
time      0.38% fanout         1 input 1.44256e+06 rows
Hash source 56           1 rows(cast) -> (s_18_9_t5.O)
time       2.2% fanout         1 input 1.44256e+06 rows
RDF_QUAD         1 rows(s_18_9_t4.O)
 inlined  P =  #dfh#lo_revenue  ,  S = k_s_18_9_t0.S G =  #nst#ssb1_ttl_qb 
time        20% fanout         1 input 1.44256e+06 rows
Hash source 39         1.6 rows(k_s_18_9_t2.O, k_s_18_9_t7.S) -> (s_18_9_t6.O)
time      0.86% fanout         0 input 1.44256e+06 rows
Sort (set_no, s_18_9_t5.O, s_18_9_t6.O) -> (s_18_9_t4.O)
 
}
time   0.00023% fanout       280 input         1 rows
group by read node  
(gb_set_no, s_18_9_t5.O, s_18_9_t6.O, aggregate)
time       0.1% fanout         0 input       280 rows
 
Precode:
      0: __ro2sq := Call __ro2sq (s_18_9_t6.O)
      5: __ro2sq := Call __ro2sq (s_18_9_t5.O)
      10: BReturn 0
Sort (__ro2sq, __ro2sq) -> (aggregate)
 
}
time    0.0002% fanout       280 input         1 rows
Key from temp (aggregate, __ro2sq, __ro2sq)
 
 
After code:
      0: lo_revenue :=  := artm aggregate
      4: d_year :=  := artm __ro2sq
      8: p_brand1 :=  := artm __ro2sq
      12: BReturn 0
time   5.3e-06% fanout         0 input       280 rows
Subquery Select(lo_revenue, d_year, p_brand1)
}
 
After code:
      0: lo_revenue := Call __ro2sq (lo_revenue)
      5: d_year := Call __ro2sq (d_year)
      10: p_brand1 := Call __ro2sq (p_brand1)
      15: BReturn 0
time   5.5e-06% fanout         0 input       280 rows
Select (lo_revenue, d_year, p_brand1)
}


 3101 msec 993% cpu, 1.14967e+07 rnd 1.81041e+08 seq   99.5619% same seg  0.417643% same pg 
Compilation: 23 msec 0 reads         0% read 0 messages         0% clw
]]></programlisting>
  <para>These are runs on warm cache on a dataset of scale factor 30, about 3 bm triples.</para>
   <para>We notice that the hash based plan completes faster and has a lower CPU percentage. This is to be 
   expected since hash joins are specially useful for joins between a large table and a smaller one.</para>
   <para>The index based plan does 21M random index lookups whereas the hash based one only 11M. We also note 
   that the index access pattern is more local with the hash plan, with 99% of lookups hitting the same segment 
   as the previous, against only 85%.</para>
   <para>These numbers are in the summary at the bottom of each profile:</para>
   <itemizedlist mark="bullet">
     <listitem>rnd -- means index access</listitem>
     <listitem>seq -- means rows retrieved in sequential scan, same seg is the percentage of index accesses that hit the same segment as the previous access. </listitem>
   </itemizedlist>
   <para>The index based plan starts with the smallest selection, in this case the days parts with the given 
   brand. From this it joins to the lineorder and gets the supplier. It fetches the region of the supplier and 
   leaves out the ones not in America. </para>
   <para>The hash based plan makes a hash table of all the parts with the brand, all the suppliers in America and 
   all the days in the time dimension. It then scans lineorder and first drops the rows whose part is not in the 
   hash, then the rows where the supplier is not in the hash, then gets the year of each date. This last operation 
   does not drop any rows but is still done by hash because there are relatively few days and the day to year 
   translation is done a very large number of times.</para>
   <para>The number of rows in and out of each operator is given after the time percent, above the operator. 
   Fanout is the number of rows of output per one row of input. </para>
   <para>Given the long-running queries of any workload, you can perform this same comparison to determine if hash 
   join is useful in the case at hand. Looking at the real time and CPU% is usually enough.</para>
   <itemizedlist mark="bullet">
     <listitem>Using the sql:select-option pragma: One can specify the hash join is not to be used:
<programlisting><![CDATA[
define sql:select-option "loop" 
]]></programlisting>     
       <para>which will exclude use of hash join in the specific query.</para>  
     </listitem>
     <listitem>Using the table_option construct: Can be used for selecting the join type for any triple pattern:
<programlisting><![CDATA[
{ 
  ?lo rdfh:lo_suppkey ?supp . 
  ?supp rdfh:s_region  "AMERICA" option (table_option "hash")
}  
]]></programlisting>
      <para>would have the effect of building a hash from the suppliers in America. </para>
     </listitem>
   </itemizedlist> 
   <para>You may experiment with these options and look at the profile output for each.</para>
   <para>For some analytics workloads enabling hash join may give a factor of 2 or 3 more performance. 
   For lookup workloads there may be no gain.</para>
  <para>Sometimes a hash join may be used when an index lookupp would be better, thus in some cases it makes 
  sense to turn off hash joins either per query or globally.</para>
</sect2>
</sect1>


<sect1 id="sparqlextensions"><title>Extensions</title>
<sect2 id="rdfsparqlrulefulltext"><title>Using Full Text Search in SPARQL</title>
<para>Virtuoso's triple store supports optional full text indexing of RDF object values since version 5.0.
It is possible to declare that objects of triples with a given predicate or graph get indexed.
The graphs and triples may be enumerated or a wildcard may be used.
</para>
<para>The triples for which a full text index entry exists can be found using the <emphasis>bif:contains</emphasis>
or related filters and predicates.
</para>
<para>For example, the query:
</para>
<programlisting>
SQL>SELECT *
FROM &lt;people&gt;
WHERE
  {
    ?s foaf:Name ?name . ?name bif:contains "'rich*'".
  }
</programlisting>
<para>would match all subjects whose <emphasis>foaf:Name</emphasis> contain a word Rich.
This would match Richard, Richie etc.
</para>
<para>Note that words and phrases should be enclosed in quotes if they contain spaces or
other non-alphanumeric chars.
</para>
<para>If the <emphasis>bif:contains</emphasis> or related predicate is applied to an object that is not
a string or is not the object of an indexed triple, no match will be found.
</para>
<para>The syntax for text patterns is identical to the syntax for the SQL contains predicate.
</para>
<para>The SPARQL/SQL optimizer determines whether the text pattern will be used to drive the query or whether it
will filter results after other conditions are applied first. In contrast to <emphasis>bif:contains</emphasis>,
regexp matching never drives the query or makes use of an index, thus in practice regexps are checked after other conditions.
</para>
<sect3 id="rdfsparqlrulespecifywhatindex"><title>Specifying What to Index</title>
<para>Whether the object of a given triple is indexed in the text index depends on indexing rules. If at least one
indexing rule matches the triple, the object gets indexed if the object is a string. An indexing rule specifies
a graph and a predicate. Either may be an IRI or NULL, in which case it matches all IRI's.
</para>
<para>Rules also have a 'reason', which can be used to group rules into application-specific sets. A triple will stop
being indexed only after all rules mandating its indexing are removed. When an application requires indexing a
certain set of triples, rules are added for that purpose. These rules are tagged with the name of the application
as their reason. When an application no longer requires indexing, the rules belonging to this application can be
removed. This will not turn off indexing if another application still needs certain triples to stay indexed.
</para>
<para>Indexing is enabled/disabled for specific graph/predicate combinations with:
</para>
<programlisting>
create function DB.DBA.RDF_OBJ_FT_RULE_ADD
  (in rule_g varchar, in rule_p varchar, in reason varchar) returns integer
</programlisting>
<programlisting>
create function DB.DBA.RDF_OBJ_FT_RULE_DEL
  (in rule_g varchar, in rule_p varchar, in reason varchar) returns integer
</programlisting>
<emphasis>Example:</emphasis>
<para>The first function adds a rule. The first two arguments are the text representation of the IRI's for the graph
and predicate. If NULL is given then all graph's or predicates match. Specifying both as NULL means that all
string valued objects will be added to a text index.
       </para>
<emphasis>Example:</emphasis>
<programlisting><![CDATA[
DB.DBA.RDF_OBJ_FT_RULE_ADD (null, null, 'All');	
]]></programlisting>
<para>The second function reverses the effect of the first. Only a rule that has actually been added can be deleted.
Thus one cannot say that all except a certain enumerated set should be indexed.
</para>
<programlisting><![CDATA[
DB.DBA.RDF_OBJ_FT_RULE_DEL (null, null, 'All');
]]></programlisting>
<para>The reason argument is an arbitrary string identifying the application that needs this rule.
Two applications can add the same rule.
Removing one of them will still keep the rule in effect.
If an object is indexed by more than one rule, the index data remain free from duplicates, neither index size nor speed is affected.
</para>
<para>
If <emphasis>DB.DBA.RDF_OBJ_FT_RULE_ADD</emphasis> detects that <emphasis>DB.DBA.RDF_QUAD</emphasis> contains quads whose graphs and/or predicates match to the new rule but which have not been indexed before then these quads are indexed automatically.
However the function <emphasis>DB.DBA.RDF_OBJ_FT_RULE_DEL</emphasis> does not remove indexing data about related objects.
Thus the presence of indexing data about an object does not imply that it is necessarily used in some quad that matches to some rule.
</para>
<para>The above functions return one if the rule is added or deleted and zero if the call was redundant (the rule has been added before or there's no rule to delete).
</para>
<sect4 id="rdfsparqlrulespecifywhatindexexample"><title>Example</title>
<programlisting><![CDATA[

-- We load Tim Berners-Lee's FOAF file into a graph called 'people'.

SQL>DB.DBA.RDF_LOAD_RDFXML (http_get ('http://www.w3.org/People/Berners-Lee/card#i'), 'no', 'http://www.w3.org/people#');
Done. -- 172 msec.

-- We check how many triples we got.

SQL>SPARQL SELECT COUNT (*) FROM <http://www.w3.org/people#> WHERE {?s ?p ?o};
callret-0
INTEGER
 266
No. of rows in result: 1

-- We check the GRAPH: <http://www.w3.org/people#> for objects like "Tim":

SQL>SPARQL
SELECT *
FROM <http://www.w3.org/people#>
WHERE
  {
    ?s ?p ?o . FILTER (?o LIKE '%Tim%')
  };
s                                               p                                           o
VARCHAR                                         VARCHAR                                     VARCHAR
_______________________________________________________________________________

http://www.w3.org/People/Berners-Lee/card#i     http://xmlns.com/foaf/0.1/name              Timothy Berners-Lee
http://www.w3.org/People/Berners-Lee/card#i     http://xmlns.com/foaf/0.1/nick              TimBL
http://www.w3.org/People/Berners-Lee/card#i     http://www.w3.org/2002/07/owl#sameAs        http://www4.wiwiss.fu-berlin.de/bookmashup/persons/Tim+Berners-Lee
http://www.w3.org/People/Berners-Lee/card#i     http://xmlns.com/foaf/0.1/knows             http://dbpedia.org/resource/Tim_Bray
http://www.w3.org/People/Berners-Lee/card#i     http://www.w3.org/2000/01/rdf-schema#label  Tim Berners-Lee
http://www.w3.org/People/Berners-Lee/card#i     http://xmlns.com/foaf/0.1/givenname         Timothy
http://dbpedia.org/resource/Tim_Bray            http://xmlns.com/foaf/0.1/name              Tim Bray
no                                              http://purl.org/dc/elements/1.1/title       Tim Berners-Lee's FOAF file

8 Rows. -- 230 msec.


-- We specify that all string objects in the graph 'people' should be text indexed.

SQL>DB.DBA.RDF_OBJ_FT_RULE_ADD('http://www.w3.org/people#', null, 'people');
Done. -- 130 msec.

-- We update the text index.

SQL>DB.DBA.VT_INC_INDEX_DB_DBA_RDF_OBJ ();
Done. -- 140 msec.

-- See impact of the index  by querying the subjects and predicates
-- of all triples in the GRAPH: <http://www.w3.org/people#>,
-- where the object is a string which contains a word beginning with "TIM".

SQL>SPARQL SELECT * FROM <http://www.w3.org/people#> WHERE { ?s ?p ?o . ?o bif:contains '"Timo*"'};
s                                               p                                     o
VARCHAR                                         VARCHAR                               VARCHAR
_______________________________________________________________________________

 http://www.w3.org/People/Berners-Lee/card#i	http://xmlns.com/foaf/0.1/name	      Timothy Berners-Lee
 http://www.w3.org/People/Berners-Lee/card#i	http://xmlns.com/foaf/0.1/givenname   Timothy

2 Rows. -- 2 msec.
]]></programlisting>
<para>
The query below is identical with that above but uses a different syntax.
The filter syntax is more flexible in that it allows passing extra options to the <emphasis>contains</emphasis> predicate. These may be useful in the future.
</para>
<programlisting><![CDATA[
SQL>SPARQL SELECT * FROM <people> WHERE { ?s ?p ?o . FILTER (bif:contains(?o,  '"Timo*"')) };
]]></programlisting>
</sect4>
<note><title>Note:</title><para>It is advisable to upgrade to the latest version of Virtuoso before adding free-text rules for the first time.
This is especially the case if large amounts of text are to be indexed. The reason is that the free-text index on RDF may be changed in future versions
and automatic upgrading of an existing index data into the new format may take much more time than indexing from scratch.</para></note>
<para>The table <emphasis>DB.DBA.RDF_OBJ_FT_RULES</emphasis> stores list of free-text index configuration rules.
</para>
<programlisting><![CDATA[
create table DB.DBA.RDF_OBJ_FT_RULES (
  ROFR_G varchar not null,       -- specific graph IRI or NULL for "all graphs"
  ROFR_P varchar not null,       -- specific predicate IRI or NULL for "all predicates"
  ROFR_REASON varchar not null,  -- identification string of a creator, preferably human-readable
  primary key (ROFR_G, ROFR_P, ROFR_REASON) );
]]></programlisting>
<para>
Applications may read from this table but they should not write to it directly.
Duplications in the rules do not affect the speed of free-text index operations because the content of the table is cached in memory in a special form.
Unlike the use of configuration functions, directly writing to the table will not update the in-memory cache.
</para>
<para>
The table is convenient to search for rules added by a given application.
If a unique identification string is used during installation of an application when rules are added then it's easy to remove those rules
as part of any uninstall routine.
</para>
   </sect3>
<sect3 id="rdfsparqlruletimeindexing"><title>Time of Indexing</title>
<para>The triple store's text index is in manual batch mode by default. This means that changes in triples are periodically
reflected in the text index but are not maintained in strict synchrony. This is much more efficient than keeping the
indices in constant synchrony. This setting may be altered with the <emphasis>db.dba.vt_batch_update</emphasis> stored procedure.
</para>
<para>To force synchronization of the RDF text index, use:
</para>
<programlisting>
DB.DBA.VT_INC_INDEX_DB_DBA_RDF_OBJ ();
</programlisting>
<para>To set the text index to follow the triples in real time, use:
</para>
<programlisting>
DB.DBA.VT_BATCH_UPDATE ('DB.DBA.RDF_OBJ', 'OFF', null);
</programlisting>
<para>To set the text index to be updated every 10 minutes, use:
</para>
<programlisting>
DB.DBA.VT_BATCH_UPDATE ('DB.DBA.RDF_OBJ', 'ON', 10);
</programlisting>
<para>To make the update always manual, specify NULL as the last argument above.
</para>
<para>
One problem related to free-text indexing of <emphasis>DB.DBA.RDF_QUAD</emphasis> is that some applications (e.g. those that import billions of triples) may set off triggers.
This will make free-text index data incomplete.
Calling procedure <emphasis>DB.DBA.RDF_OBJ_FT_RECOVER ()</emphasis> will insert all missing free-text index items by dropping and re-inserting every existing free-text index rule.
</para>
</sect3>
<sect3 id="rdfviewsandfreetext"><title>Free-Text Indexes on Linked Data Views</title>
<para>
If an <emphasis>O</emphasis> field of a quad map pattern gets its value from a database column that has a free text index then this index can be used in SPARQL for efficient text searching. As a variation of this facility, the free-text index of another table may be used.
</para>
<para>
If a statement of a quad map pattern declaration starts with a declaration of table aliases, the table alias declaration may include the name of a table column that should have a text index.
For example, consider the possibility of using a free-text index on the content of DAV resources stored in the DAV system tables of Virtuoso:
</para>
<programlisting><![CDATA[
prefix mydav: <...>
create quad storage mydav:metadata
FROM WS.WS.SYS_DAV_RES as dav_resource text literal RES_CONTENT
...
  {
    ...
    mydav:resource-iri (dav_resource.RES_FULL_PATH)
        a mydav:resource ;
        mydav:resource-content dav_resource.RES_CONTENT ;
        mydav:resource-mime-type dav_resource.RESTYPE ;
    ...
  }
]]></programlisting>
<para>
The clause <emphasis>text literal RES_CONTENT</emphasis> grants the SPARQL compiler permission to use a free-text index for objects that are literals composed from column <emphasis>dav_resource.RES_CONTENT</emphasis>. This clause also allows choosing between <emphasis>text literal</emphasis> (supports only the <emphasis>contains()</emphasis> predicate) and <emphasis>text xml literal</emphasis> (supports both <emphasis>contains()</emphasis> and <emphasis>xcontains()</emphasis>) text indexes.
It is important to understand that the free-text index will produce results using raw relational data.
If a literal class transformation changes the text stored in the column then these changes are ignored by free-text search.
e.g. if a transformation concatenates a word to the value of the column, but the free-text search will not find this word.
</para>
<para>
The free-text index may be used in a more sophisticated way. Consider a built-in table <emphasis>DB.DBA.RDF_QUAD</emphasis> that does not have a free-text index.
Moreover, the table does not contain the full values of all objects; the <emphasis>O</emphasis> column contains &quot;short enough&quot; values inlined, but long and special values are represented by links to the <emphasis>DB.DBA.RDF_OBJ</emphasis> table.
The RDF_OBJ table, however, has free-text index that can be used.
The full declaration of the built-in default mapping for default storage could be written this way:
   </para>
<programlisting><![CDATA[
-- Important! Do not try to execute on live system
-- without first changing the quad storage and quad map pattern names!

SPARQL
create virtrdf:DefaultQuadMap as
graph rdfdf:default-iid-nonblank (DB.DBA.RDF_QUAD.G)
subject rdfdf:default-iid (DB.DBA.RDF_QUAD.S)
predicate rdfdf:default-iid-nonblank (DB.DBA.RDF_QUAD.P)
object rdfdf:default (DB.DBA.RDF_QUAD.O)

create quad storage virtrdf:DefaultQuadStorage
FROM DB.DBA.RDF_QUAD as physical_quad
FROM DB.DBA.RDF_OBJ as physical_obj text xml literal RO_DIGEST of (physical_quad.O)
WHERE (^{physical_quad.}^.O = ^{physical_obj.}^.RO_DIGEST)
  {
    create virtrdf:DefaultQuadMap as
      graph rdfdf:default-iid-nonblank (physical_quad.G)
      subject rdfdf:default-iid (physical_quad.S)
      predicate rdfdf:default-iid-nonblank (physical_quad.P)
      object rdfdf:default (physical_quad.O) .
 }
;
]]></programlisting>
<para>
The reference to the free-text index is extended by clause <emphasis> of (physical_quad.O)</emphasis>.
This means that the free-text on <emphasis>DB.DBA.RDF_OBJ.RO_DIGEST</emphasis> will be used when the object value comes from <emphasis>physical_quad.O</emphasis> as if <emphasis>physical_quad.O</emphasis> were indexed itself.
If a SPARQL query invokes <emphasis>virtrdf:DefaultQuadMap</emphasis> but contains no free-text criteria then only <emphasis>DB.DBA.RDF_QUAD</emphasis> appears in the final SQL statement and no join with <emphasis>DB.DBA.RDF_OBJ</emphasis> is made.
Adding a free-text predicate will add <emphasis>DB.DBA.RDF_OBJ</emphasis> to the list of source tables and a join condition for <emphasis>DB.DBA.RDF_QUAD.O</emphasis> and <emphasis>DB.DBA.RDF_OBJ.RO_DIGEST</emphasis>; and it will add <emphasis>contains (RO_DIGEST, ...)</emphasis> predicate, rather than <emphasis>contains (O, ...)</emphasis>.
As a result, &quot;you pay only for what you use&quot;: adding free-text index to the declaration does not add tables to the query unless the index is actually used.
</para>
<para>
Boolean functions <function>bif:contains</function> and <function>bif:xcontains</function> are used 
for objects that come from Linked Data Views as well as for regular &quot;physical&quot; triples.
Every function takes two arguments and returns a boolean value. The first argument is an local variable.
The argument variable should be used as an object field in the group pattern where the filter condition is placed.
Moreover, the occurrence of the variable in an object field should be placed <emphasis>before</emphasis> the filter.
If there are many occurrences of the variable in object fields then the free-text search is associated with the rightmost occurrence that is still to the left of the filter.
The triple pattern that contains the rightmost occurrence is called the &quot;intake&quot; of the free-text search.
When the SPARQL compiler chooses the appropriate quad map patterns that may generate data matching the intake triple pattern, it skips quad map patterns that have no declared free-text indexes, because nothing can be found by free-text search in data that have no free-text index.
Every quad map pattern that has a free-text pattern will ultimately produce an invocation of the SQL <link linkend="containspredicate">contains</link> or <link linkend="xcontainspredicate">xcontains</link> predicate, so the final result of a free-text search may be a union of free-text searches from different quad map patterns.
</para>
<para>
The described logic is important only in very complicated cases, whereas simple queries are self-evident:
   </para>
<programlisting><![CDATA[
SELECT * FROM <my-dav-graph>
WHERE {
    ?resource a mydav:resource ;
        mydav:resource-content ?text .
    FILTER (bif:contains (?text, "hello and world")) }
]]></programlisting>
<para>
or, more succinctly,
   </para>
<programlisting><![CDATA[
SELECT * FROM <my-dav-graph>
WHERE {
    ?resource a mydav:resource ;
        mydav:resource-content ?text .
    ?text bif:contains "hello and world" . }
]]></programlisting>
</sect3>
<sect3 id="rdfsparqlrulescoreexmp"><title>Example Using Score</title>
<programlisting><![CDATA[
SQL>
SPARQL
SELECT *
WHERE
  {
    ?s ?p ?o .
    ?o bif:contains 'NEW AND YORK'
    OPTION (score ?sc) .
  }
ORDER BY DESC (?sc)
LIMIT 10

s                                                                        p                                               o                                                sc
ANY                                                                      ANY                                             ANY 	                                          ANY
______________________________________________________________________________________________________________________________________________________________________________

http://dbpedia.org/resource/New_York%2C_New_York_%28disambiguation%29	 http://www.w3.org/2000/01/rdf-schema#comment	 New York, New York, New York kentini........     88
http://dbpedia.org/resource/New_York%2C_New_York_%28disambiguation%29	 http://dbpedia.org/property/abstract	         New York, New York, New York kentinin re....     88
http://newyorkjobs.wordpress.com/2006/07/10/new-york-jobs-71006	         http://purl.org/dc/elements/1.1/description	 York Marketing Jobs New York Retail Jobs....     84
http://dbpedia.org/resource/Global_Communication	                 http://dbpedia.org/property/contenu	         A - New York, New York (Headfuq Mix) B1 ....     84
http://dbpedia.org/resource/New_York_%28disambiguation%29	         http://www.w3.org/2000/01/rdf-schema#comment	 New York a^?? New York amerikai vA~?ros ....     76
http://dbpedia.org/resource/New_York_%28disambiguation%29	         http://dbpedia.org/property/abstract	         New York a^?? New York amerikai vA~?ros ....     76
http://dbpedia.org/resource/New_York_%28disambiguation%29	         http://www.w3.org/2000/01/rdf-schema#comment	 New York ima lahko naslednje pomene: New ...     74
http://dbpedia.org/resource/New_York_%28disambiguation%29	         http://dbpedia.org/property/abstract	         New York ima lahko naslednje pomene: New ...     74
http://dbpedia.org/resource/New_York_College	                         http://www.w3.org/2000/01/rdf-schema#comment	 There are several colleges of New York t ...     72
http://dbpedia.org/resource/New_York_College	                         http://dbpedia.org/property/abstract	         There are several colleges of New York t ...     72
No. of rows in result: 10

]]></programlisting>
</sect3>
</sect2>
<sect2 id="rdfsparul"><title>SPARUL -- an Update Language For RDF Graphs</title>
     <sect3 id="rdfsparulintro"><title>Introduction</title>
       <para>Starting with version 5.0, Virtuoso supports the
<ulink url="http://jena.hpl.hp.com/~afs/SPARQL-Update.html">SPARQL/Update</ulink> extension to SPARQL.
This is sufficient for most of routine data manipulation operations. If the <emphasis>SPARQL_UPDATE</emphasis>
role is granted to user <emphasis>SPARQL</emphasis> user then data manipulation statements may be
executed via the SPARQL web service endpoint as well as by data querying.
       </para>
     </sect3>
     <sect3 id="rdfsparulfunc"><title>Manage RDF Storage</title>
       <para>Two functions allow the user to alter RDF storage by inserting or deleting all
triples listed in some vector. Both functions receive the IRI of the graph that should be altered and
a vector of triples that should be added or removed. The graph IRI can be either an IRI ID or a string.
The third optional argument controls the transactional behavior - the parameter value is
passed to the <link linkend="fn_log_enable"><function>log_enable</function></link> function.
The return values of these functions are not defined and should not be used by applications.
       </para>
       <programlisting>
create function DB.DBA.RDF_INSERT_TRIPLES (in graph_iri any, in triples any, in log_mode integer := null)
create function DB.DBA.RDF_DELETE_TRIPLES (in graph_iri any, in triples any, in log_mode integer := null)
       </programlisting>
       <para>Simple operations may be faster if written as low-level SQL code instead of using SPARUL.
The use of SPARQL DELETE is unnecessary in cases where the better alternative is for the application to delete from RDF_QUAD
using simple SQL filters like:
       </para>
       <programlisting>
DELETE FROM DB.DBA.RDF_QUAD
WHERE G = DB.DBA.RDF_MAKE_IID_OF_QNAME (
    'http://local.virt/DAV/sparql_demo/data/data-xml/source-simple2/source-data-01.rdf' );
       </programlisting>
       <para>On the other hand, simple filters does not work when the search criteria refer to triples
that are affected by the modification. Consider a function that deletes all triples whose subjects
are nodes of type 'http://xmlns.com/foaf/0.1/Person'. Type information is stored in triples that
will be deleted, so the simplest function is something like this:
       </para>
<programlisting><![CDATA[
create procedure DELETE_PERSONAL_DATA (in foaf_graph varchar)
{
  declare pdata_dict, pdata_array any;
-- Step 1: select everything that should be deleted
  pdata_dict := ((
      sparql construct { ?s ?p ?o }
      WHERE { graph ?:foaf_graph {
              ?s ?p ?o . ?s rdf:type <http://xmlns.com/foaf/0.1/Person>
            } }
      ));
-- Step 2: delete all found triples
  pdata_array := dict_list_keys (pdata_dict, 1);
  RDF_DELETE_TRIPLES (foaf_graph, pdata_array);
};

DELETE_PERSONAL_DATA (
  'http://local.virt/DAV/sparql_demo/data/data-xml/source-simple2/source-data-01.rdf' );
]]></programlisting>
       <para>From Virtuoso 5.0 onwards, applications can use SPARUL to do the same in a more convenient way:
       </para>
<programlisting><![CDATA[
create procedure DELETE_PERSONAL_DATA (in foaf_graph varchar)
  {
  sparql delete { ?s ?p ?o }
      WHERE { graph ?:foaf_graph {
              ?s ?p ?o . ?s rdf:type <http://xmlns.com/foaf/0.1/Person>
            } }
};
]]></programlisting>
     </sect3>
     <sect3 id="rdfsparulexamples"><title>Examples</title>
     <sect4 id="rdfsparulexamples1"><title>Example for changing the graph</title>
       <para>The graph to be changed may be specified by an option preceding of query, instead
of being specified in the 'insert into graph' clause.
       </para>
<programlisting><![CDATA[
SQL>SPARQL DEFINE input:default-graph-uri <http://mygraph.com>
INSERT INTO <http://mygraph.com> { <http://example.com/dataspace/Kingsley#this> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/ns#User> };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 20 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples2"><title>Example for delete graph equivalence</title>
       <para>The following two statements are equivalent but the latter may work faster, especially
if there are many Linked Data Views in the system or if the graph in question contains triples from Linked Data Views.
Note that neither of these two statements affects data coming from Linked Data Views.
</para>
<programlisting><![CDATA[
SQL> SPARQL DELETE FROM GRAPH <http://mygraph.com> { ?s ?p ?o } FROM <http://mygraph> WHERE { ?s ?p ?o };
callret-0
VARCHAR
_______________________________________________________________________________

Delete from <http://mygraph.com>, 1 triples -- done

1 Rows. -- 10 msec.

SQL> SPARQL CLEAR GRAPH <http://mygraph.com>;
callret-0
VARCHAR
__________________________________________________________

Clear <http://mygraph.com> -- done

1 Rows. -- 10 msec.
]]></programlisting>
   <para>DROP GRAPH is equivalent of CLEAR GRAPH. It requires initially the graph to be 
   	created explicitly.</para>
   <para>Note: All SPARQL commands should work via SPARUL ( i.e. executed from the /sparql endpoint) 
   	as soon as "SPARQL" user has "SPARQL_UPDATE" privilege.</para>
   <para>Assume the following sequence of commands to be executed from the /sparql endpoint:</para>
   <itemizedlist mark="bullet">
     <listitem>Create explicitly a graph:
<programlisting><![CDATA[
CREATE GRAPH <qq>
	
callret-0
Create graph <qq> -- done
]]></programlisting>     	
     </listitem>
     <listitem>If you don't know whether the graph is created explicitly or not, use <emphasis>DROP SILENT GRAPH</emphasis>:
<programlisting><![CDATA[
DROP SILENT	GRAPH <qq> 
	
callret-0	
Drop silent graph <qq> -- done	
]]></programlisting>     	
     </listitem>
     <listitem>If you know the graph is created explicitly, use <emphasis>DROP GRAPH</emphasis>:
<programlisting><![CDATA[
DROP GRAPH <qq> 	
	
callret-0
Drop graph <qq> -- done	
]]></programlisting>     	
     </listitem>
   </itemizedlist>
</sect4>
<sect4 id="rdfsparulexamples3"><title>Example for deleting all triples for given subject</title>
<para>The following statement deletes all records with &lt;http://example.com/dataspace/Kingsley#this&gt; as the subject:</para>
<programlisting><![CDATA[
SQL>SPARQL
DELETE FROM GRAPH <http://mygraph.com> { ?s ?p ?o }
 FROM <http://mygraph.com>
WHERE { ?s ?p ?o . filter ( ?s = <http://example.com/dataspace/Kingsley#this>) };
callret-0
VARCHAR
_______________________________________________________________________________

Delete from <http://mygraph.com>, 1 triples -- done

1 Rows. -- 10 msec.
]]></programlisting>
<para>Alternatively, the statement can be written in this way:</para>
<programlisting><![CDATA[
SQL>SPARQL
DELETE FROM GRAPH <http://mygraph.com> { <http://example.com/dataspace/Kingsley#this> ?p ?o }
 FROM <http://mygraph.com>
WHERE { <http://example.com/dataspace/Kingsley#this> ?p ?o };
callret-0
VARCHAR
_______________________________________________________________________________

Delete from <http://mygraph.com>, 1 triples -- done

1 Rows. -- 10 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples4"><title>Example for INSERT statements equivalent</title>
       <para>Keywords 'insert in' and 'insert into' are interchangeable in Virtuoso for backward
compatibility, but the SPARUL specification lists only 'insert into'. For example,
the statements below are equivalent:
</para>
<programlisting><![CDATA[
SQL>SPARQL INSERT INTO GRAPH <http://mygraph.com> {  <http://example.com/dataspace/Kingsley#this>
                                                     <http://rdfs.org/sioc/ns#id>
                                                     <Kingsley> };
callret-0
VARCHAR
______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 0 msec.
SQL>SPARQL INSERT INTO GRAPH <http://mygraph.com> {  <http://example.com/dataspace/Caroline#this>
                                                     <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
                                                     <http://rdfs.org/sioc/ns#User> };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 0 msec.

-- and

SQL>SPARQL INSERT IN GRAPH <http://mygraph.com> {  <http://example.com/dataspace/Kingsley#this>
                                                   <http://rdfs.org/sioc/ns#id>
                                                   <Kingsley> };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 10 msec.
SQL>SPARQL INSERT IN GRAPH <http://mygraph.com> {  <http://example.com/dataspace/Caroline#this>
                                                   <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
                                                   <http://rdfs.org/sioc/ns#User> };
callret-0
VARCHAR
________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done

1 Rows. -- 0 msec.

]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples5"><title>Example for various expressions usage</title>
       <para>It is possible to use various expressions to calculate fields of new triples. This is
very convenient, even if not a part of the original specification.
       </para>
<programlisting><![CDATA[
SQL>SPARQL INSERT INTO GRAPH <http://mygraph.com> { ?s <http://rdfs.org/sioc/ns#id> `iri (bif:concat (str (?o), "Idehen"))` }
WHERE { ?s <http://rdfs.org/sioc/ns#id> ?o };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 4 triples -- done

1 Rows. -- 0 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples6"><title>Example for operator IN usage</title>
<para>The example shows how to find which predicate/object pairs the following
subjects have in common and count the occurances:</para>
<programlisting><![CDATA[
http://dbpedia.org/resource/Climate_change
http://dbpedia.org/resource/Disaster_risk_reduction
http://dbpedia.org/resource/Tanzania
http://dbpedia.org/resource/Capacity_building
http://dbpedia.org/resource/Poverty
http://dbpedia.org/resource/Construction
http://dbpedia.org/resource/Vulnerability
http://dbpedia.org/resource/Mount_Kilimanjaro
http://dbpedia.org/resource/Social_vulnerability
]]></programlisting>
<para>The following query returns the desired results:</para>
<programlisting><![CDATA[
SPARQL
SELECT ?s1 ?s2 COUNT (1)
WHERE
  {
    ?s1 ?p ?o .
    FILTER (?s1 IN (<http://dbpedia.org/resource/Climate_change>,
    <http://dbpedia.org/resource/Disaster_risk_reduction>,
    <http://dbpedia.org/resource/Tanzania>,
    <http://dbpedia.org/resource/Capacity_building>,
    <http://dbpedia.org/resource/Poverty>,
    <http://dbpedia.org/resource/Construction>,
    <http://dbpedia.org/resource/Vulnerability>,
    <http://dbpedia.org/resource/Mount_Kilimanjaro>,
    <http://dbpedia.org/resource/Social_vulnerability> ))
    ?s2 ?p ?o .
    FILTER (?s2 IN (<http://dbpedia.org/resource/Climate_change>,
    <http://dbpedia.org/resource/Disaster_risk_reduction>,
    <http://dbpedia.org/resource/Tanzania>,
    <http://dbpedia.org/resource/Capacity_building>,
    <http://dbpedia.org/resource/Poverty>,
    <http://dbpedia.org/resource/Construction>,
    <http://dbpedia.org/resource/Vulnerability>,
    <http://dbpedia.org/resource/Mount_Kilimanjaro>,
    <http://dbpedia.org/resource/Social_vulnerability> ))
    FILTER (?s1 != ?s2)
    FILTER (str(?s1) < str (?s2))
  }
LIMIT 20
]]></programlisting>
<para>The result of executing the query:</para>
<programlisting><![CDATA[
s1  	                                         s2  	                                              callret-2
http://dbpedia.org/resource/Climate_change 	 http://dbpedia.org/resource/Tanzania 	              2
http://dbpedia.org/resource/Social_vulnerability http://dbpedia.org/resource/Vulnerability 	      1
http://dbpedia.org/resource/Mount_Kilimanjaro 	 http://dbpedia.org/resource/Poverty 	              1
http://dbpedia.org/resource/Mount_Kilimanjaro 	 http://dbpedia.org/resource/Tanzania 	              3
http://dbpedia.org/resource/Capacity_building 	 http://dbpedia.org/resource/Disaster_risk_reduction  1
http://dbpedia.org/resource/Poverty 	         http://dbpedia.org/resource/Tanzania 	              1
]]></programlisting>
<para>You can also find live demo query results <ulink url="http://dbpedia.org/sparql?default-graph-uri=http%3A%2F%2Fdbpedia.org&amp;should-sponge=&amp;query=SELECT+%0D%0A%3Fs1+%3Fs2+count+%281%29+where+{%0D%0A%3Fs1+%3Fp+%3Fo+.%0D%0Afilter+%28%3Fs1+in+%28%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FClimate_change%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDisaster_risk_reduction%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FTanzania%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FCapacity_building%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FPoverty%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FConstruction%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FVulnerability%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FMount_Kilimanjaro%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FSocial_vulnerability%3E+%29%29%0D%0A%3Fs2+%3Fp+%3Fo+.%0D%0Afilter+%28%3Fs2+in+%28%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FClimate_change%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FDisaster_risk_reduction%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FTanzania%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FCapacity_building%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FPoverty%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FConstruction%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FVulnerability%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FMount_Kilimanjaro%3E%2C%0D%0A%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FSocial_vulnerability%3E+%29%29%0D%0Afilter+%28%3Fs1+!%3D+%3Fs2%29%0D%0Afilter+%28str%28%3Fs1%29+%3C+str+%28%3Fs2%29%29%0D%0A}+limit+20&amp;format=html&amp;debug=on&amp;timeout=">here</ulink></para>
  <tip><title>See Also:</title>
    <para><link linkend="rdfsparulexamples18">Example usage of IN operator for retrieving all triples for each entity.</link></para>
  </tip>
</sect4>
<sect4 id="rdfsparulexamples7"><title>Example for Modify used as Update</title>
       <para>'Modify graph' may be used as a form of 'update' operation.
       </para>
<programlisting><![CDATA[
-- update a graph with insert scoped on the same graph	
SQL>SPARQL 
MODIFY GRAPH <http://mygraph.com> 
DELETE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o } 
INSERT { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type1> ?o } 
FROM <http://mygraph.com> 	
WHERE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o };

-- update a graph with insert scoped on another graph	
SQL>SPARQL 
MODIFY GRAPH <http://mygraph.com> 
DELETE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o } 
INSERT { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type1> ?o } 
FROM <http://example.com>
WHERE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o };
	

-- update a graph with insert scoped over the whole RDF Quad Store
SQL>SPARQL 
MODIFY GRAPH <http://mygraph.com> 
DELETE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o } 
INSERT { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type1> ?o } 
WHERE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o };
	
-- update a graph with delete of particular tripple
SQL>SPARQL
DELETE FROM GRAPH <http://mygraph.com> { <http://example.com/dataspace/Caroline#this> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type1> <http://rdfs.org/sioc/ns#User> };

]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples8"><title>Example for generating RDF information resource URI</title>
       <para>The RDF information resource URI can be generated via a string expression.</para>
<orderedlist>
  <listitem>Suppose there is a sample file kidehen.n3:
<programlisting><![CDATA[
<http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/ns#User> .
<http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this> <http://www.w3.org/2000/01/rdf-schema#label> 	"Kingsley" .
<http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this> <http://rdfs.org/sioc/ns#creator_of> <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300> .
]]></programlisting>
</listitem>
  <listitem>Using Conductor UI go to Web Application Server and create folder, for ex. with name "n3_collection"</listitem>
  <listitem>Upload the "n3_collection" folder the file kidehen.n3</listitem>
  <listitem>Click properties for the folder "n3_collection" and set to be public readable in the Permissions section</listitem>
  <listitem>Check "Apply changes to all subfolders and resources".</listitem>
  <listitem>Finally Click "Update"</listitem>
     <figure id="gnr1" float="1">
       <title>Generating RDF information resource URI</title>
       <graphic fileref="ui/exmp1.png"/>
     </figure>
  <listitem>To clear the graph execute:
<programlisting><![CDATA[
SQL>SPARQL CLEAR GRAPH <http://mygraph.com>;
callret-0
VARCHAR
_____________________________________________________________________

Clear <http://mygraph.com> -- done

1 Rows. -- 10 msec.
]]></programlisting></listitem>
   <listitem>To load the kidehen.n3 file execute:
<programlisting><![CDATA[
SQL>SPARQL
load bif:concat ("http://", bif:registry_get("URIQADefaultHost"), "/DAV/n3_collection/kidehen.n3")
INTO GRAPH <http://mygraph.com>;
callret-0
VARCHAR
_______________________________________________________________________________

Load <http://example.com/DAV/n3_collection/kidehen.n3> into graph <http://mygraph.com> -- done

1 Rows. -- 30 msec.
]]></programlisting>
  </listitem>
  <listitem>In order to check the new inserted triples execute:
<programlisting><![CDATA[
SQL>SPARQL
SELECT *
FROM <http://mygraph.com>
WHERE
{
    ?s ?p ?o
}
;
s                                                                  p                                                   o
VARCHAR                                                            VARCHAR                                             VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this    http://www.w3.org/1999/02/22-rdf-syntax-ns#type     http://rdfs.org/sioc/ns#User
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this    http://rdfs.org/sioc/ns#creator_of                  http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this    http://www.w3.org/2000/01/rdf-schema#label          Kingsley

3 Rows. -- 10 msec.
]]></programlisting>
  </listitem>
</orderedlist>
</sect4>
<sect4 id="rdfsparulexamples9"><title>Example for operations over a web service endpoint</title>
       <para>Several operations can be sent to a web service endpoint as a single statement and
executed in sequence.
</para>
<programlisting><![CDATA[
SQL>SPARQL
INSERT IN GRAPH <http://mygraph.com> { <http://example.com/dataspace/Kingsley#this>
                                       <http://rdfs.org/sioc/ns#id>
                                       <Kingsley> }
INSERT INTO GRAPH <http://mygraph.com> { <http://example.com/dataspace/Caroline#this>
                                         <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
                                         <http://rdfs.org/sioc/ns#User> }
INSERT INTO GRAPH <http://mygraph.com> { ?s <http://rdfs.org/sioc/ns#id> `iri (bif:concat (str (?o), "Idehen"))` }
WHERE { ?s <http://rdfs.org/sioc/ns#id> ?o };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://mygraph.com>, 1 triples -- done
Insert into <http://mygraph.com>, 1 triples -- done
Insert into <http://mygraph.com>, 8 triples -- done
Commit -- done


1 Rows. -- 10 msec.

SQL>SPARQL
MODIFY GRAPH <http://mygraph.com>
DELETE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o }
INSERT { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type1> ?o }
FROM <http://mygraph.com>	
WHERE { ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ?o };

SQL>DELETE FROM GRAPH <http://mygraph.com> { <http://example.com/dataspace/Caroline#this> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type1> <http://rdfs.org/sioc/ns#User> };

SQL>SPARQL
load bif:concat ("http://", bif:registry_get("URIQADefaultHost"), "/DAV/n3_collection/kidehen.n3") INTO GRAPH <http://mygraph.com>;
]]></programlisting>
  <tip><title>See Also:</title>
    <para><link linkend="rdfsparulexamples7">Examples for Modify used as Update</link></para>
  </tip>
</sect4>
<sect4 id="rdfsparulexamples10"><title>Example for Dropping graph</title>
<para>When handling very large RDF data collections (e.g. 600 million triples ) loaded into Virtuoso
server as a single graph, the fastest operation to drop the graph is:</para>
<programlisting><![CDATA[
SQL>SPARQL CLEAR GRAPH <http://mygraph.com>;
callret-0
VARCHAR
______________________________________________________________________________

Clear <http://mygraph.com> -- done

1 Rows. -- 10 msec.
]]></programlisting>
<para>The operation can be speeded up by executing log_enable (0) or even log_enable (2) beforehand,
and log_enable(1) after it completes.
</para>
</sect4>
<sect4 id="rdfsparulexamples11"><title>Example for testing Graph Equality</title>
<para>The procedure below keeps simple cases of graphs with bnodes:</para>
<orderedlist>
  <listitem>First it compares all triples without bnodes</listitem>
  <listitem>Then it iteratively establishes equivalences between bnodes that
are directly and unambiguously connected to equivalent vertexes by identical predicates.</listitem>
</orderedlist>
<programlisting><![CDATA[
-- Fast Approximate RDF Graph Equivalence Test
-- (C) 2009-2018 OpenLink Software
-- License: GNU General Public License (only version 2 of the license).
-- No warranty, even implied warranty

-- This compares the content of triple dictionaries \c dict1 and \c dict2,
-- returns NULL if no difference found (with bnode equivalence in mind),
-- returns description of a difference otherwise.
-- The function is experimental (note suffix _EXP), so no accurate QA is made.
-- Some version of the function may be inserted later in OpenLink Virtuoso Server under some different name.
create function DB.DBA.RDF_TRIPLE_DICTS_DIFFER_EXP (
  in dict1 any, --- Triple dictionary, traditional, (vectors of S, P, O are keys, any non-nulls are values)
  in dict2 any, --- Second triple dictionary, like to \c dict1
  in accuracy integer,	--- Accuracy, 0 if no bnodes expected, 1 if "convenient" trees with intermediate bnodes expected, 2 and more are not yet implemented
  in equiv_map any := null, --- If specified then it contain mapping from IRI_IDs of bnodes of \c dict1 to equivalent IRI_IDs of bnodes of \c dict1.
-- It can be extended during the run so use dict_duplicate() before call if needed.
  in equiv_rev any := null --- If specified then it is an inverted dictionary of \c equiv_map (this time \c dict2 bnodes are keys and \c dict1 bnodes are values)
  )
{
  declare dict_size1, dict_size2 integer;
  declare old_dirt_level, dirt_level integer;
  declare ctr, tailctr, sp_made_new_equiv integer;
  declare array1, array2, dict2_sp, dict1_op, dict2_op, array1_op any;
  dict_size1 := dict_size (dict1);
  dict_size2 := dict_size (dict2);
  dict2 := dict_duplicate (dict2);
  if (dict_size1 <> dict_size2)
    return 'Sizes differ';
  if (equiv_map is null)
    {
      equiv_map := dict_new (dict_size1);
      equiv_rev := dict_new (dict_size1);
    }
  old_dirt_level := dict_size1 - dict_size (equiv_map);
  array1 := dict_list_keys (dict1, 0);
next_loop:
-- Step 1: removing triples with all three items matched
  ctr := dict_size1-1;
  while (ctr >= 0)
    {
      declare s_in_1, o_in_1, s_in_2, o_in_2, triple_in_2 any;
      s_in_1 := array1[ctr][0];
      o_in_1 := array1[ctr][2];
      if (is_bnode_iri_id (s_in_1))
        {
          s_in_2 := dict_get (equiv_map, s_in_1, null);
          if (s_in_2 is null)
            goto next_full_eq_check;
        }
      else
        s_in_2 := s_in_1;
      if (is_bnode_iri_id (o_in_1))
        {
          o_in_2 := dict_get (equiv_map, o_in_1, null);
          if (o_in_2 is null)
            goto next_full_eq_check;
        }
      else
        o_in_2 := o_in_1;
      triple_in_2 := vector (s_in_2, array1[ctr][1], o_in_2);
      if (dict_get (dict2, triple_in_2, null) is null)
        return vector (array1[ctr], ' is in first, ', triple_in_2, ' is missing in second');
      dict_remove (dict2, triple_in_2);
      if (ctr < dict_size1-1)
        array1[ctr] := array1[dict_size1-1];
      dict_size1 := dict_size1-1;
next_full_eq_check:
      ctr := ctr-1;
    }
-- Step 1 end, garbage truncated:
  if ((0 = dict_size1) or (0 = accuracy))
    return null;
  if (dict_size1 < length (array1))
    array1 := subseq (array1, 0, dict_size1);
  if (dict_size (dict2) <> dict_size1)
    signal ('OBLOM', 'Internal error: sizes of graphs suddenly differ');
-- Step 2: establishing equivs between not-yet-coupled bnodes that are values of functional predicates of coupled subjects
  sp_made_new_equiv := 0;
  dict2_sp := dict_new (dict_size1);
  array2 := dict_list_keys (dict2, 0);
  for (ctr := dict_size1-1; ctr >= 0; ctr := ctr-1)
    {
      declare sp2, o2, prev_uniq_o2 any;
      sp2 := vector (array2[ctr][0], array2[ctr][1]);
      prev_uniq_o2 := dict_get (dict2_sp, sp2, null);
      if (prev_uniq_o2 is null)
        {
          o2 := array2[ctr][2];
          if (is_bnode_iri_id (o2))
            dict_put (dict2_sp, sp2, o2);
          else
            dict_put (dict2_sp, sp2, #i0);
        }
      else if (prev_uniq_o2 <> #i0)
        dict_put (dict2_sp, sp2, #i0);
    }
  rowvector_subj_sort (array1, 0, 1);
  rowvector_subj_sort (array1, 1, 1);
  rowvector_subj_sort (array2, 1, 1);
  ctr := 0;
  while (ctr < dict_size1)
    {
      declare s_in_1, o_in_1, s_in_2, o_in_2, o_in_dict2_sp, o_in_dict2_sp_in_1 any;
      tailctr := ctr+1;
      if (array1[ctr][1] <> array2[ctr][1])
        {
          if (array1[ctr][1] > array2[ctr][1])
            return vector ('Cardinality of predicate ', array2[ctr][1], ' is greater in second than in first');
          else
            return vector ('Cardinality of predicate ', array1[ctr][1], ' is greater in first than in second');
        }
      while ((tailctr < dict_size1) and
        (array1[tailctr][0] = array1[ctr][0]) and
        (array1[tailctr][1] = array1[ctr][1]) )
        tailctr := tailctr+1;
      if ((tailctr - ctr) > 1)
        goto next_sp_check;
      o_in_1 := array1[ctr][2];
      if (not is_bnode_iri_id (o_in_1))
        goto next_sp_check;
      o_in_2 := dict_get (equiv_map, o_in_1, null);
      if (o_in_2 is not null)
        goto next_sp_check;
      s_in_1 := array1[ctr][0];
      if (is_bnode_iri_id (s_in_1))
        {
          s_in_2 := dict_get (equiv_map, s_in_1, null);
          if (s_in_2 is null)
            goto next_sp_check;
        }
      else
        s_in_2 := s_in_1;
      o_in_dict2_sp := dict_get (dict2_sp, vector (s_in_2, array1[ctr][1]), null);
      if (o_in_dict2_sp is null)
        return vector (vector (s_in_1, array1[ctr][1], o_in_1), ' is unique SP in first, ', vector (s_in_2, array1[ctr][1]), ' is missing SP in second');
      if (o_in_dict2_sp = #i0)
        return vector (vector (s_in_1, array1[ctr][1], o_in_1), ' is unique SP in first, ', vector (s_in_2, array1[ctr][1]), ' is not unique SP-to-bnode in second');
      o_in_dict2_sp_in_1 := dict_get (equiv_rev, o_in_dict2_sp, null);
      if (o_in_dict2_sp_in_1 is not null)
  {
          if (o_in_dict2_sp_in_1 = o_in_1)
            goto next_sp_check;
          return vector (vector (s_in_1, array1[ctr][1], o_in_1), ' is unique SP in first, ', vector (s_in_2, array1[ctr][1], o_in_dict2_sp), ' is unique SP in second but ', o_in_dict2_sp, ' rev-equiv to ', o_in_dict2_sp_in_1);
        }
      dict_put (equiv_map, o_in_1, o_in_dict2_sp);
      dict_put (equiv_rev, o_in_dict2_sp, o_in_1);
      sp_made_new_equiv := sp_made_new_equiv + 1;
next_sp_check:
      ctr := tailctr;
    }
  dict_list_keys (dict2_sp, 2);
-- Step 2 end
  if (sp_made_new_equiv * 10 > dict_size1)
    goto next_loop; -- If dictionary is noticeably extended then it's worth to remove more triples before continue.
-- Step 3: establishing equivs between not-yet-coupled bnodes that are subjects of inverse functional properties with coupled objects.
  dict1_op := dict_new (dict_size1);
  for (ctr := dict_size1-1; ctr >= 0; ctr := ctr-1)
    {
      declare op1, s1, prev_uniq_s1 any;
      op1 := vector (array1[ctr][2], array1[ctr][1]);
      prev_uniq_s1 := dict_get (dict1_op, op1, null);
      if (prev_uniq_s1 is null)
        {
          s1 := array1[ctr][0];
          if (is_bnode_iri_id (s1))
            dict_put (dict1_op, op1, s1);
          else
            dict_put (dict1_op, op1, #i0);
        }
      else if (prev_uniq_s1 <> #i0)
        dict_put (dict1_op, op1, #i0);
    }
  array1_op := dict_to_vector (dict1_op, 2);
  dict2_op := dict_new (dict_size1);
  for (ctr := dict_size1-1; ctr >= 0; ctr := ctr-1)
    {
      declare op2, s2, prev_uniq_s2 any;
      op2 := vector (array2[ctr][2], array2[ctr][1]);
      prev_uniq_s2 := dict_get (dict2_op, op2, null);
      if (prev_uniq_s2 is null)
        {
          s2 := array2[ctr][0];
          if (is_bnode_iri_id (s2))
            dict_put (dict2_op, op2, s2);
          else
            dict_put (dict2_op, op2, #i0);
        }
      else if (prev_uniq_s2 <> #i0)
        dict_put (dict2_op, op2, #i0);
    }
  ctr := length (array1_op) - 2;
  while (ctr >= 0)
    {
      declare o_in_1, s_in_1, o_in_2, s_in_2, s_in_dict2_op, s_in_dict2_op_in_1 any;
      s_in_1 := array1_op[ctr+1];
      if (not is_bnode_iri_id (s_in_1))
        goto next_op_check;
      s_in_2 := dict_get (equiv_map, s_in_1, null);
      if (s_in_2 is not null)
        goto next_op_check;
      o_in_1 := array1_op[ctr][0];
      if (is_bnode_iri_id (o_in_1))
        {
          o_in_2 := dict_get (equiv_map, o_in_1, null);
          if (o_in_2 is null)
            goto next_op_check;
        }
      else
        o_in_2 := o_in_1;
      s_in_dict2_op := dict_get (dict2_op, vector (o_in_2, array1_op[ctr][1]), null);
      if (s_in_dict2_op is null)
        return vector (vector (s_in_1, array1_op[ctr][1], o_in_1), ' is unique OP in first, ', vector (o_in_2, array1_op[ctr][1]), ' is missing OP in second');
      if (s_in_dict2_op = #i0)
        return vector (vector (s_in_1, array1_op[ctr][1], o_in_1), ' is unique OP in first, ', vector (o_in_2, array1_op[ctr][1]), ' is not unique OP-to-bnode in second');
      s_in_dict2_op_in_1 := dict_get (equiv_rev, s_in_dict2_op, null);
      if (s_in_dict2_op_in_1 is not null)
        {
          if (s_in_dict2_op_in_1 = s_in_1)
            goto next_op_check;
          return vector (vector (s_in_1, array1_op[ctr][1], o_in_1), ' is unique OP in first, ', vector (s_in_dict2_op, array1[ctr][1], o_in_2), ' is unique OP in second but ', s_in_dict2_op, ' rev-equiv to ', s_in_dict2_op_in_1);
        }
      dict_put (equiv_map, s_in_1, s_in_dict2_op);
      dict_put (equiv_rev, s_in_dict2_op, s_in_1);
next_op_check:
      ctr := ctr - 2;
    }
  dict_list_keys (dict2_op, 2);
-- Step 3 end
  dirt_level := dict_size1 - dict_size (equiv_map);
  if (dirt_level >= old_dirt_level)
    return vector (vector (array1[0][0], array1[0][1], array1[0][2]), ' has no matches in second with the requested accuracy');
  old_dirt_level := dirt_level;
  goto next_loop;
}
;

create function DB.DBA.RDF_GRAPHS_DIFFER_EXP (in g1_uri varchar, in g2_uri varchar, in accuracy integer)
{
  return DB.DBA.RDF_TRIPLE_DICTS_DIFFER_EXP (
    (sparql define output:valmode "LONG" construct { ?s ?p ?o } where { graph `iri(?:g1_uri)` { ?s ?p ?o }}),
    (sparql define output:valmode "LONG" construct { ?s ?p ?o } where { graph `iri(?:g2_uri)` { ?s ?p ?o }}),
    accuracy );
}
;

-- The rest of file contains some minimal tests.

set verbose off;
set banner off;
set types off;

create function DB.DBA.DICT_EXTEND_WITH_KEYS (in dict any, in keys any)
  {
  if (dict is null)
    dict := dict_new (length (keys));
  foreach (any k in keys) do
    dict_put (dict, k, 1);
  return dict;
}
;

create function DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP (in title varchar, in should_differ integer, in v1 any, in v2 any, in accuracy integer)
{
  declare d1, d2, eqm, eqr, differ_status any;
  d1 := DB.DBA.DICT_EXTEND_WITH_KEYS (null, v1);
  d2 := DB.DBA.DICT_EXTEND_WITH_KEYS (null, v2);
  eqm := dict_new (10);
  eqr := dict_new (10);
  dbg_obj_princ ('===== ' || title);
  differ_status := DB.DBA.RDF_TRIPLE_DICTS_DIFFER_EXP (d1, d2, accuracy, eqm, eqr);
  dbg_obj_princ ('Result: ', differ_status);
  if (0 < dict_size (eqm))
  dbg_obj_princ ('Equivalence map: ', dict_to_vector (eqm, 0));
  dbg_obj_princ ('Equivalence rev: ', dict_to_vector (eqr, 0));
  return sprintf ('%s: %s',
    case when (case when should_differ then equ (0, isnull (differ_status)) else isnull (differ_status) end) then 'PASSED' else '***FAILED' end,
    title );
}
;

create function DB.DBA.TEST_RDF_GRAPHS_DIFFER_EXP (in title varchar, in should_differ integer, in g1_uri varchar, in g2_uri varchar, in accuracy integer)
{
  declare differ_status any;
  differ_status := DB.DBA.RDF_GRAPHS_DIFFER_EXP (g1_uri, g2_uri, accuracy);
  dbg_obj_princ ('Result: ', differ_status);
  return sprintf ('%s: %s',
    case when (case when should_differ then equ (0, isnull (differ_status)) else isnull (differ_status) end) then 'PASSED' else '***FAILED' end,
    title );
}
;

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'Identical graphs', 0,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i200, 1) ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i200, 1) ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'Sizes differ', 1,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i200, 1) ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i200, 1),
    vector (#i101, #i201, #i301) ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'Cardinality of a pred differ', 1,
  vector (
    vector (#i100, #i200, #ib300),
    vector (#i101, #i200, #ib302),
    vector (#i103, #i201, #ib304),
    vector (#ib109, #i200, #ib109) ),
  vector (
    vector (#i100, #i200, #ib301),
    vector (#i101, #i200, #ib303),
    vector (#i103, #i201, #ib305),
    vector (#ib109, #i201, #ib109) ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'Bnodes in O with unique SP (equiv)', 0,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib301),
    vector (#i101, #i201, #ib301),
    vector (#i102, #i202, #ib303),
    vector (#ib303, #i204, #i306),
    vector (#ib303, #i205, #ib305),
    vector (#i100, #i200, 1) ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib302),
    vector (#i101, #i201, #ib302),
    vector (#i102, #i202, #ib304),
    vector (#ib304, #i204, #i306),
    vector (#ib304, #i205, #ib306),
    vector (#i100, #i200, 1) ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'Bnodes in O with unique SP (diff 1)', 1,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib301),
    vector (#i102, #i202, #ib303),
    vector (#ib303, #i204, #i306),
    vector (#ib303, #i205, #ib305),
    vector (#i100, #i200, 1) ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib302),
    vector (#i102, #i202, #ib304),
    vector (#ib304, #i204, #i306),
    vector (#ib304, #i205, #i306),
    vector (#i100, #i200, 1) ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'Bnodes in O with unique SP (diff 2)', 1,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib301),
    vector (#i102, #i202, #ib303),
    vector (#ib303, #i204, #i306),
    vector (#ib303, #i205, #ib305),
    vector (#i100, #i200, 1) ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib302),
    vector (#i102, #i202, #ib304),
    vector (#ib304, #i204, #i306),
    vector (#ib304, #i205, #ib304),
    vector (#i100, #i200, 1) ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'foaf-like-mix (equiv)', 0,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib301),
    vector (#i100, #i201, #ib303),
    vector (#i100, #i201, #ib305),
    vector (#i100, #i201, #ib307),
    vector (#ib301, #i202, 'Anna'),
    vector (#ib303, #i202, 'Anna'),
    vector (#ib305, #i202, 'Brigit'),
    vector (#ib307, #i202, 'Clara'),
    vector (#ib301, #i203, 'ann@ex.com'),
    vector (#ib303, #i203, 'ann@am.com'),
    vector (#ib305, #i203, 'root@ple.com'),
    vector (#ib307, #i203, 'root@ple.com') ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib302),
    vector (#i100, #i201, #ib304),
    vector (#i100, #i201, #ib306),
    vector (#i100, #i201, #ib308),
    vector (#ib302, #i202, 'Anna'),
    vector (#ib304, #i202, 'Anna'),
    vector (#ib306, #i202, 'Brigit'),
    vector (#ib308, #i202, 'Clara'),
    vector (#ib302, #i203, 'ann@ex.com'),
    vector (#ib304, #i203, 'ann@am.com'),
    vector (#ib306, #i203, 'root@ple.com'),
    vector (#ib308, #i203, 'root@ple.com') ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'foaf-like-mix (swapped names)', 1,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib301),
    vector (#i100, #i201, #ib303),
    vector (#i100, #i201, #ib305),
    vector (#i100, #i201, #ib307),
    vector (#ib301, #i202, 'Anna'),
    vector (#ib303, #i202, 'Anna'),
    vector (#ib305, #i202, 'Brigit'),
    vector (#ib307, #i202, 'Clara'),
    vector (#ib301, #i203, 'ann@ex.com'),
    vector (#ib303, #i203, 'ann@am.com'),
    vector (#ib305, #i203, 'root@ple.com'),
    vector (#ib307, #i203, 'root@ple.com') ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib302),
    vector (#i100, #i201, #ib304),
    vector (#i100, #i201, #ib306),
    vector (#i100, #i201, #ib308),
    vector (#ib302, #i202, 'Anna'),
    vector (#ib304, #i202, 'Brigit'),
    vector (#ib306, #i202, 'Anna'),
    vector (#ib308, #i202, 'Clara'),
    vector (#ib302, #i203, 'ann@ex.com'),
    vector (#ib304, #i203, 'ann@am.com'),
    vector (#ib306, #i203, 'root@ple.com'),
    vector (#ib308, #i203, 'root@ple.com') ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'foaf-like-mix (swapped names)', 1,
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib301),
    vector (#i100, #i201, #ib303),
    vector (#i100, #i201, #ib305),
    vector (#i100, #i201, #ib307),
    vector (#ib301, #i202, 'Anna'),
    vector (#ib303, #i202, 'Anna'),
    vector (#ib305, #i202, 'Brigit'),
    vector (#ib307, #i202, 'Clara'),
    vector (#ib301, #i203, 'ann@ex.com'),
    vector (#ib303, #i203, 'ann@am.com'),
    vector (#ib305, #i203, 'root@ple.com'),
    vector (#ib307, #i203, 'root@ple.com') ),
  vector (
    vector (#i100, #i200, #i300),
    vector (#i100, #i201, #ib302),
    vector (#i100, #i201, #ib304),
    vector (#i100, #i201, #ib306),
    vector (#i100, #i201, #ib308),
    vector (#ib302, #i202, 'Anna'),
    vector (#ib304, #i202, 'Brigit'),
    vector (#ib306, #i202, 'Anna'),
    vector (#ib308, #i202, 'Clara'),
    vector (#ib302, #i203, 'ann@ex.com'),
    vector (#ib304, #i203, 'ann@am.com'),
    vector (#ib306, #i203, 'root@ple.com'),
    vector (#ib308, #i203, 'root@ple.com') ),
  100
);

select DB.DBA.TEST_RDF_TRIPLE_DICTS_DIFFER_EXP ( 'bnodes only (equiv that can not be proven)', 1,
  vector (
    vector (#ib101, #i200, #ib103),
    vector (#ib103, #i201, #ib101) ),
  vector (
    vector (#ib102, #i200, #ib104),
    vector (#ib104, #i201, #ib102) ),
  100
);

sparql clear graph <http://GraphCmp/One>;

TTLP ('@prefix foaf: <http://i-dont-remember-it> .
_:me
	a foaf:Person ;
	foaf:knows	[ foaf:nick "oerling" ; foaf:title "Mr." ; foaf:sha1 "abra" ] ;
	foaf:knows	[ foaf:nick "kidehen" ; foaf:title "Mr." ; foaf:sha1 "bra" ] ;
	foaf:knows	[ foaf:nick "aldo" ; foaf:title "Mr." ; foaf:sha1 "cada" ] .',
'', 'http://GraphCmp/One' );

sparql clear graph <http://GraphCmp/Two>;
TTLP ('@prefix foaf: <http://i-dont-remember-it> .
_:iv
	foaf:knows	[ foaf:title "Mr." ; foaf:sha1 "cada" ; foaf:nick "aldo" ] ;
	foaf:knows	[ foaf:sha1 "bra" ; foaf:title "Mr." ; foaf:nick "kidehen" ] ;
	foaf:knows	[ foaf:nick "oerling" ; foaf:sha1 "abra" ; foaf:title "Mr." ] ;
	a foaf:Person .',
'', 'http://GraphCmp/Two' );

select DB.DBA.TEST_RDF_GRAPHS_DIFFER_EXP ( 'nonexisting graphs (equiv, of course)', 0,
  'http://GraphCmp/NoSuch', 'http://GraphCmp/NoSuch',
  100 );

select DB.DBA.TEST_RDF_GRAPHS_DIFFER_EXP ( 'throughout test on foafs (equiv)', 0,
  'http://GraphCmp/One', 'http://GraphCmp/Two',
  100 );

]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples12"><title>Example for Adding triples to graph</title>
<programlisting><![CDATA[
SQL>SPARQL 
INSERT INTO GRAPH <http://BookStore.com>
{ <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/title>  "SPARQL and RDF" .
  <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/date> <1999-01-01T00:00:00>.
  <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/title> "Design notes" .
  <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/date> <2001-01-01T00:00:00>.
  <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/title> "Fundamentals of Compiler Design" .
  <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/date> <2002-01-01T00:00:00>. };
callret-0
VARCHAR
_________________________________________________________________

Insert into <http://BookStore.com>, 6 triples -- done

1 Rows. -- 0 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples13"><title>Example for Updating triples from graph</title>
<para>A SPARQL/Update request that contains a triple to be deleted and a triple to be added (used here to correct a book title).
</para>
<programlisting><![CDATA[
SQL>SPARQL
MODIFY GRAPH <http://BookStore.com>
DELETE
 { <http://www.w3.org/People/Connolly/#me>  <http://purl.org/dc/elements/1.1/title>  "Fundamentals of Compiler Design" }
INSERT
 { <http://www.w3.org/People/Connolly/#me>  <http://purl.org/dc/elements/1.1/title>  "Fundamentals" };
callret-0
VARCHAR
_______________________________________________________________________________

Modify <http://BookStore.com>, delete 1 and insert 1 triples -- done

1 Rows. -- 20 msec.
]]></programlisting>
  <tip><title>See Also:</title>
    <para><link linkend="rdfsparulexamples7">Examples for Modify used as Update</link></para>
  </tip>
</sect4>
<sect4 id="rdfsparulexamples14"><title>Example for Deleting triples from graph</title>
<para>The example below has a request to delete all records of old books (dated before year 2000)
</para>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX dc:  <http://purl.org/dc/elements/1.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
DELETE FROM GRAPH <http://BookStore.com> { ?book ?p ?v }
WHERE
  { GRAPH  <http://BookStore.com>
   { ?book dc:date ?date
     FILTER ( xsd:dateTime(?date) < xsd:dateTime("2000-01-01T00:00:00")).
    ?book ?p ?v.
   }
  };
_______________________________________________________________________________

Delete from <http://BookStore.com>, 6 triples -- done

1 Rows. -- 10 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples15"><title>Example for Copying triples from one graph to another</title>
<para>The next snippet copies records from one named graph to another based on a pattern:
</para>
<programlisting><![CDATA[
SQL>SPARQL clear graph <http://BookStore.com>;
SQL>SPARQL clear graph <http://NewBookStore.com>;
SQL>SPARQL
insert in graph <http://BookStore.com>
  {
    <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/date> <1999-04-01T00:00:00> .
    <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/date> <1998-05-03T00:00:00> .
    <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/date> <2001-02-08T00:00:00>
  };
SQL>SPARQL
PREFIX dc:  <http://purl.org/dc/elements/1.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
INSERT INTO GRAPH <http://NewBookStore.com> { ?book ?p ?v }
WHERE
  { GRAPH  <http://BookStore.com>
   { ?book dc:date ?date
     FILTER ( xsd:dateTime(?date) > xsd:dateTime("2000-01-01T00:00:00")).
     ?book ?p ?v.
   }
  };
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://NewBookStore.com>, 6 triples -- done

1 Rows. -- 30 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples16"><title>Example for Moving triples from one graph to another</title>
<para>This example moves records from one named graph to another named graph based on a pattern:
</para>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX dc:  <http://purl.org/dc/elements/1.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>

INSERT INTO GRAPH <http://NewBookStore.com>
 { ?book ?p ?v }
WHERE
  { GRAPH  <http://BookStore.com>
     { ?book dc:date ?date .
       FILTER ( xsd:dateTime(?date) > xsd:dateTime("2000-01-01T00:00:00")).
       ?book ?p ?v.
     }
  };
_______________________________________________________________________________

Insert into <http://NewBookStore.com>, 6 triples -- done

1 Rows. -- 10 msec.

SQL>SPARQL
PREFIX dc:  <http://purl.org/dc/elements/1.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
DELETE FROM GRAPH <http://BookStore.com>
 { ?book ?p ?v }
WHERE
  { GRAPH  <http://BookStore.com>
      { ?book dc:date ?date .
        FILTER ( xsd:dateTime(?date) > xsd:dateTime("2000-01-01T00:00:00")).
        ?book ?p ?v.
      }
  };
_______________________________________________________________________________

Delete from <http://BookStore.com>, 3 triples -- done

1 Rows. -- 10 msec.
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples17"><title>Example for BBC SPARQL Collection</title>
<programlisting><![CDATA[
## All programmes related to James Bond:
PREFIX po: <http://purl.org/ontology/po/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?uri ?label
WHERE
  {
    ?uri po:category
      <http://www.bbc.co.uk/programmes/people/bmFtZS9ib25kLCBqYW1lcyAobm8gcXVhbGlmaWVyKQ#person> ;
    rdfs:label ?label.
   }
]]></programlisting>

<programlisting><![CDATA[
## Find all Eastenders broadcasta after 2009-01-01,
## along with the broadcast version & type
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX tl: <http://purl.org/NET/c4dm/timeline.owl#>
PREFIX po: <http://purl.org/ontology/po/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?version_type ?broadcast_start
WHERE
  {
    <http://www.bbc.co.uk/programmes/b006m86d#programme> po:episode ?episode .
    ?episode po:version ?version .
    ?version a ?version_type .
    ?broadcast po:broadcast_of ?version .
    ?broadcast event:time ?time .
    ?time tl:start ?broadcast_start .
    FILTER ( (?version_type != <http://purl.org/ontology/po/Version>)
    && (?broadcast_start > "2009-01-01T00:00:00Z"^^xsd:dateTime) )
  }
]]></programlisting>

<programlisting><![CDATA[
## Find all programmes that featured both the Foo Fighters and Al Green
PREFIX po: <http://purl.org/ontology/po/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX mo: <http://purl.org/ontology/mo/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX event: <http://purl.org/NET/c4dm/event.owl#>
PREFIX tl: <http://purl.org/NET/c4dm/timeline.owl#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
SELECT DISTINCT ?programme ?label
WHERE
  {
    ?event1 po:track ?track1 .
    ?track1 foaf:maker ?maker1 .
    ?maker1 owl:sameAs
       <http://www.bbc.co.uk/music/artists/67f66c07-6e61-4026-ade5-7e782fad3a5d#artist> .
    ?event2 po:track ?track2 .
    ?track2 foaf:maker ?maker2 .
    ?maker2 owl:sameAs
       <http://www.bbc.co.uk/music/artists/fb7272ba-f130-4f0a-934d-6eeea4c18c9a#artist> .
    ?event1 event:time ?t1 .
    ?event2 event:time ?t2 .
    ?t1 tl:timeline ?tl .
    ?t2 tl:timeline ?tl .
    ?version po:time ?t .
    ?t tl:timeline ?tl .
    ?programme po:version ?version .
    ?programme rdfs:label ?label .
  }
]]></programlisting>

<programlisting><![CDATA[
## Get short synopsis' of EastEnders episodes
PREFIX po: <http://purl.org/ontology/po/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
SELECT ?t ?o
WHERE
{
    <http://www.bbc.co.uk/programmes/b006m86d#programme> po:episode ?e .
    ?e a po:Episode .
    ?e po:short_synopsis ?o .
    ?e dc:title ?t
  }
]]></programlisting>

<programlisting><![CDATA[
## Get short synopsis' of EastEnders episodes (with graph)
PREFIX po: <http://purl.org/ontology/po/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
SELECT ?g ?t ?o
WHERE
  {
    graph ?g
      {
         <http://www.bbc.co.uk/programmes/b006m86d#programme> po:episode ?e .
         ?e a po:Episode .
         ?e po:short_synopsis ?o .
         ?e dc:title ?t
      }
  }
]]></programlisting>

<programlisting><![CDATA[
## Get reviews where John Paul Jones' has been involved

PREFIX mo: <http://purl.org/ontology/mo/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX rev: <http://purl.org/stuff/rev#>
PREFIX po: <http://purl.org/ontology/po/>
SELECT DISTINCT ?r_name, ?rev
WHERE
  {
    {
      <http://www.bbc.co.uk/music/artists/4490113a-3880-4f5b-a39b-105bfceaed04#artist> foaf:made ?r1 .
      ?r1 a mo:Record .
      ?r1 dc:title ?r_name .
      ?r1 rev:hasReview ?rev
    }
    UNION
    {
      <http://www.bbc.co.uk/music/artists/4490113a-3880-4f5b-a39b-105bfceaed04#artist> mo:member_of ?b1 .
      ?b1 foaf:made ?r1 .
      ?r1 a mo:Record .
      ?r1 dc:title ?r_name .
      ?r1 rev:hasReview ?rev
    }
  }
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples18"><title>Example usage of IN operator for retrieving all triples for each entity</title>
<para>To retrieve all triples for each entity for a given list of entities uris, one might use the following syntax:</para>
<programlisting><![CDATA[
SELECT ?p ?o
WHERE
  {
    ?s ?p ?o .
    FILTER ( ?s IN (<someGraph#entity1>, <someGraph#entity2>, ...<someGraph#entityN> ) )
  }
]]></programlisting>
<para>So to demonstrate this feature, execute the following query:
</para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT DISTINCT ?p ?o
WHERE
  {
    ?s ?p ?o .
    FILTER ( ?s IN (<http://dbpedia.org/resource/Climate_change>, <http://dbpedia.org/resource/Social_vulnerability> ) )
  }
LIMIT 100

p    	                                                o
ANY                                                     ANY
_______________________________________________________________________________

http://www.w3.org/1999/02/22-rdf-syntax-ns#type 	http://s.zemanta.com/ns#Target
http://s.zemanta.com/ns#title 	                        Climate change
http://s.zemanta.com/ns#targetType 	                http://s.zemanta.com/targets#rdf

3 Rows. -- 10 msec.
]]></programlisting>
    <tip><title>See Also:</title>
    <para><link linkend="rdfsparulexamples6">Example usage of IN operator.</link></para>
    </tip>
</sect4>
<sect4 id="rdfsparulexamples19"><title>Example for extending SPARQL via SQL for Full Text search: Variant I</title>
<para>To find all albums looked up by album name, one might use the following syntax:</para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT ?s ?o ?an ( bif:search_excerpt ( bif:vector ( 'In', 'Your' ) , ?o ) ) 
WHERE
  {
    ?s rdf:type mo:Record .
    ?s foaf:maker ?a .
    ?a foaf:name ?an .
    ?s dc:title ?o .
    FILTER ( bif:contains ( ?o, '"in your"' ) ) 
      }
LIMIT 10;


http://musicbrainz.org/music/record/30f13688-b9ca-4fa5-9430-f918e2df6fc4  China in Your Hand  	          Fusion  China             <b>in</b> <b>Your</b> Hand.
http://musicbrainz.org/music/record/421ad738-2582-4512-b41e-0bc541433fbc 	China in Your Hand 	            T'Pau 	China             <b>in</b> <b>Your</b> Hand.
http://musicbrainz.org/music/record/01acff2a-8316-4d4b-af93-97289e164379 	China in Your Hand 	            T'Pau 	China             <b>in</b> <b>Your</b> Hand.
http://musicbrainz.org/music/record/4fe99b06-ac73-40dd-8be7-bdaefb014981 	China in Your Hand 	            T'Pau 	China             <b>in</b> <b>Your</b> Hand.
http://musicbrainz.org/music/record/ac1cb011-6040-4515-baf2-59551a9884ac 	In Your Hands 	                Stella One Eleven 	      <b>In</b> <b>Your</b> Hands.
http://dbtune.org/magnatune/album/mercy-inbedinst 	                      In Your Bed - instrumental mix 	Mercy Machine 	          <b>In</b> <b>Your</b> Bed mix.
http://musicbrainz.org/music/record/a09ae12e-3694-4f68-bf25-f6ff4f790962 	A Word in Your Ear 	Alfie 	    A Word <b>in</b>          <b>Your</b> Ear.
http://dbtune.org/magnatune/album/mercy-inbedremix 	                      In Your Bed - the remixes 	    Mercy Machine 	          <b>In</b> <b>Your</b> Bed the remixes.
http://musicbrainz.org/music/record/176b6626-2a25-42a7-8f1d-df98bec092b4 	Smoke Gets in Your Eyes 	      The Platters 	Smoke Gets  <b>in</b> <b>Your</b> Eyes.
http://musicbrainz.org/music/record/e617d90e-4f86-425c-ab97-efdf4a8a452b 	Smoke Gets in Your Eyes 	      The Platters 	Smoke Gets  <b>in</b> <b>Your</b> Eyes.
]]></programlisting>
<para>Note that the query will not show anything when there are triples like:</para>
<programlisting><![CDATA[
<x> <y> "In" 
<z> <q> "Your"		
]]></programlisting>

</sect4>
<sect4 id="rdfsparulexamples20"><title>Example for extending SPARQL via SQL for Full Text search: Variant II</title>
<para>To get movies from DBpedia, where the query can contain terms from the title, 
one might use the following syntax:</para>
<programlisting><![CDATA[
SQL>SPARQL
 SELECT ?s ?an ?dn ?o( bif:search_excerpt ( bif:vector ( 'Broken', 'Flowers' ) , ?o ) ) 
 WHERE
  {
    ?s rdf:type dbpedia-owl:Film .
    ?s dbpprop:name ?o .
    FILTER ( bif:contains ( ?o, '"broken flowers"' ) )
    OPTIONAL { ?s dbpprop:starring ?starring .}
    OPTIONAL { ?s dbpprop:director ?director . }
    OPTIONAL { ?starring dbpprop:name ?an . }
    OPTIONAL { ?director dbpprop:name ?dn . }
  };


http://dbpedia.org/resource/Broken_Flowers  Tilda Swinton  	Jim Jarmusch  	Broken Flowers  	          <b>Broken</b> <b>Flowers</b>.
http://dbpedia.org/resource/Broken_Flowers 	Swinton, Tilda 	Jim Jarmusch 	  Broken Flowers 	            <b>Broken</b> <b>Flowers</b>.
....
http://dbpedia.org/resource/Broken_Flowers  Bill Murray  	  Jim Jarmusch  	Music from Broken Flowers  	Music from <b>Broken</b> <b>Flowers</b>.
....
]]></programlisting>
<para>Note that the query will not show anything when there are triples like:</para>
<programlisting><![CDATA[
<x> <y> "Broken" 
<z> <q> "Flowers"		
]]></programlisting>
</sect4>
<sect4 id="rdfsparulexamples21"><title>Example for date manipulation of xsd types within SPARQL</title>
<para>This example shows usage of dateTime column truncation to date only
and performs a group by on this column:
  </para>
<programlisting><![CDATA[
-- prepare the data by inserting triples in a graph:
SQL>SPARQL
INSERT INTO GRAPH <http://BookStore.com>
  {
    <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/title>  "SPARQL and RDF" .
    <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/date> <1999-01-01T00:00:00>.
    <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/title> "Design notes" .
    <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/date> <2001-01-01T00:00:00>.
    <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/title> "Fundamentals of Compiler Design" .
    <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/date> <2002-01-01T00:00:00>.
    <http://www.ivan-herman.net/foaf.rdf#me> <http://purl.org/dc/elements/1.1/title>  "RDF Store" .
    <http://www.ivan-herman.net/foaf.rdf#me> <http://purl.org/dc/elements/1.1/date> <2001-03-05T00:00:00>.
    <http://bblfish.net/people/henry/card#me> <http://purl.org/dc/elements/1.1/title> "Design RDF notes" .
    <http://bblfish.net/people/henry/card#me> <http://purl.org/dc/elements/1.1/date> <2001-01-01T00:00:00>.
    <http://hometown.aol.com/chbussler/foaf/chbussler.foaf#me> <http://purl.org/dc/elements/1.1/title> "RDF Fundamentals" .
    <http://hometown.aol.com/chbussler/foaf/chbussler.foaf#me> <http://purl.org/dc/elements/1.1/date> <2002-01-01T00:00:00>. 
  };

_______________________________________________________

Insert into <http://BookStore.com>, 12 triples -- done

-- Find Count of Group by Dates
SQL>SPARQL 
SELECT (xsd:date(bif:subseq(str(?a_dt), 0, 10))), count(*)
FROM <http://BookStore.com> 
WHERE 
  { 
    ?s <http://purl.org/dc/elements/1.1/date> ?a_dt 
  }
GROUP BY (xsd:date(bif:subseq(str(?a_dt), 0, 10)));

callret-0                                         callret-1
VARCHAR                                           VARCHAR
__________________________________________________
1999-01-01                                        1
2001-01-01                                        2
2002-01-01                                        2
2001-03-05                                        1

4 Rows. -- 15 msec.
SQL>	
]]></programlisting>
     </sect4>
     <sect4 id="rdfsparulexamples22"><title>Example for executing INSERT/DELETE (SPARUL) statements 
     against a WebID protected SPARQL endpoint</title>
       <para>The following sample scenario demonstrates how to perform INSERT/DELETE (SPARUL) statements 
       	against a protected SPARQL Endpoint by setting WebID Protocol ACLs using the Virtuoso 
       	Authentication Server UI:</para>
       	<orderedlist>
          <listitem>Obtain a WebID:
            <orderedlist>
              <listitem>Download and install the <ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/ods_framework_dav.vad">ods_framework_dav.vad</ulink>.
                <itemizedlist mark="bullet">
                  <listitem>Note: an existing ODS DataSpace user instance can also be used, for example at 
                  	<ulink url="http://id.myopenlink.net/ods/">http://id.myopenlink.net/ods/</ulink>
                  </listitem>
                </itemizedlist>
              </listitem>
              <listitem>Register an ODS Data Space user, for example with name "demo".</listitem>
              <listitem>The generated WebID will be for example:
<programlisting><![CDATA[
http://id.myopenlink.net/dataspace/person/demo#this 	
]]></programlisting>              	
              </listitem>
              <listitem><ulink url="http://ods.openlinksw.com/wiki/ODS/ODSGenerateX509Certificate">Generate a Personal HTTP based Identifier</ulink> 
              for the "demo" user and then bind the personal Identifier to an X.509 Certificate, thereby 
              giving assigning the user a WebID.
              </listitem>	
            </orderedlist>          	
          </listitem>
          <listitem>Download and install the 
          	<ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> 
          	package, if not already installed.
          </listitem>
          <listitem>Go to http://&lt;cname&gt;:&lt;port&gt;/conductor, where &lt;cname&gt;:&lt;port&gt; 
          	are replaced by your local server values.</listitem>
          <listitem>Go to System Admin -> Linked Data -> Access Control -> SPARQL-WebID
            <figure id="si1" float="1">
              <title>Conductor SPARQL-WebID</title>
              <graphic fileref="ui/si1.png"/>
            </figure>          	
          </listitem>
          <listitem>In the displayed form:
            <orderedlist>
              <listitem>Enter the Web ID for the user registered above, for example:
<programlisting><![CDATA[
http://id.myopenlink.net/dataspace/person/demo#this	
]]></programlisting>              	
              </listitem>
              <listitem>Select "SPARQL Role": "<emphasis>UPDATE</emphasis>".
                <figure id="si2" float="1">
                  <title>Conductor SPARQL-WebID</title>
                  <graphic fileref="ui/si2.png"/>
                </figure>                  
              </listitem>
            </orderedlist>          	
          </listitem>
          <listitem>Click the "Register" button.</listitem>
          <listitem>The WebID Protocol ACL will be created:
            <figure id="si3" float="1">
              <title>Conductor SPARQL-WebID</title>
              <graphic fileref="ui/si3.png"/>
            </figure>          	          	
          </listitem>
          <listitem>Go to the SPARQL-WebID endpoint, https://&lt;cname&gt;:&lt;port&gt;/sparql-webid, 
          	where &lt;cname&gt;:&lt;port&gt; are replaced by your local server values.
          </listitem>
          <listitem>Select the user's certificate:
            <figure id="si4" float="1">
              <title>Conductor SPARQL-WebID</title>
              <graphic fileref="ui/si4.png"/>
            </figure>          	          	          	
          </listitem>
          <listitem>The SPARQL Query UI will be displayed:
            <figure id="si5" float="1">
              <title>Conductor SPARQL-WebID</title>
              <graphic fileref="ui/si5.png"/>
            </figure>          	          	          	          	
          </listitem>
          <listitem>Execute the query:
<programlisting><![CDATA[
INSERT INTO GRAPH <http://example.com> {
  <s1> <p1> <o1> .
  <s2> <p2> <o2> .
  <s3> <p3> <o3> 
}	
]]></programlisting> 
            <figure id="si6" float="1">
              <title>Conductor SPARQL-WebID</title>
              <graphic fileref="ui/si6.png"/>
            </figure>          	
            <figure id="si7" float="1">
              <title>Conductor SPARQL-WebID</title>
              <graphic fileref="ui/si7.png"/>
            </figure>          	            
          </listitem>
        </orderedlist>
       <para>Note: If the SPARQL Role "Sponge" is set instead, in order to be able to execute 
       	DELETE/INSERT statements over the protected SPARQL Endpoint, the following grants need to be 
       	performed for the user, associated with the WebID ACL Role:</para>	
<programlisting><![CDATA[
grant execute on DB.DBA.SPARQL_INSERT_DICT_CONTENT to "demo";
grant execute on DB.DBA.SPARQL_DELETE_DICT_CONTENT to "demo";	
]]></programlisting>       	
     </sect4>
     <sect4 id="rdfsparulexamples23"><title>Example usage of deleting Triple Patterns that are Not Scoped to a Named Graph</title>
     <para>Presuming this triple exists in one or more graphs in the store:</para>
<programlisting><![CDATA[
{ 
  <http://kingsley.idehen.net/dataspace/person/kidehen#this> 
    <http://xmlns.com/foaf/0.1/knows> 
      <http://id.myopenlink.net/dataspace/person/KingsleyUyiIdehen#this> 
}
]]></programlisting>     
    <para>The SQL query below will delete that triple from all graphs in the store:</para>
<programlisting><![CDATA[
DELETE 
  FROM DB.DBA.RDF_QUAD 
 WHERE p = iri_to_id
             ('http://xmlns.com/foaf/0.1/knows') 
   AND s = iri_to_id 
             ('http://kingsley.idehen.net/dataspace/person/kidehen#this') 
   AND o = iri_to_id 
             ('http://id.myopenlink.net/dataspace/person/KingsleyUyiIdehen#this')
;	
]]></programlisting>    
    <para>According to 
    	<ulink url="http://www.w3.org/TR/2010/WD-sparql11-update-20100126/#t414">SPARQL 1.1 Update</ulink>, 
    	the FROM clause which scopes the query to a single graph is optional. Thus, the SQL query above can 
    	be rewritten to the SPARQL query below, again deleting the matching triple from all graphs in the 
    	store:
    </para>
<programlisting><![CDATA[
DELETE 
  { 
    GRAPH ?g 
      { 
        <http://kingsley.idehen.net/dataspace/person/kidehen#this> 
          <http://xmlns.com/foaf/0.1/knows> 
            <http://id.myopenlink.net/dataspace/person/KingsleyUyiIdehen#this> 
      } 
  }
WHERE 
  { 
    GRAPH ?g 
      { 
        <http://kingsley.idehen.net/dataspace/person/kidehen#this> 
          <http://xmlns.com/foaf/0.1/knows> 
            <http://id.myopenlink.net/dataspace/person/KingsleyUyiIdehen#this> 
      } 
  }	
]]></programlisting>
       </sect4>
       <sect4 id="rdfsparulexamples24"><title>Example usage of deleting triples containing blank nodes</title>
         <para>There are two ways to delete a particular blank node:</para>
         <orderedlist>
           <listitem>To refer to it via some properties or:</listitem>
           <listitem>To convert it to it's internal "serial number", a long integer, and back.</listitem>
         </orderedlist>       
         <para>Assume the following sample scenario:</para>
         <orderedlist>
           <listitem>Clear the graph:
<programlisting><![CDATA[
SPARQL CLEAR GRAPH <http://sample/>;

Done. -- 4 msec.	
]]></programlisting>
           </listitem>
           <listitem>Insert three blank nodes with two related triples each:
<programlisting><![CDATA[
SPARQL 
  INSERT IN GRAPH <http://sample/> 
    { 
      [] <p> <o1a> , <o1b> . 
      [] <p> <o2a> , <o2b> . 
      [] <p> <o3a> , <o3b> 
    }

Done. -- 15 msec.	
]]></programlisting>
           </listitem>
           <listitem>Delete one pair of triples:
<programlisting><![CDATA[
SPARQL WITH <http://sample/> 
  DELETE { ?s ?p ?o } 
  WHERE 
    { 
      ?s ?p ?o ; 
      <p> <o1a> .
    }

Done. -- 7 msec.	
]]></programlisting>
           </listitem>
           <listitem>Ensure that we still have two bnodes, two triple per bnode:	
<programlisting><![CDATA[
SPARQL 
  SELECT * 
  FROM <http://sample/> 
  WHERE 
    { 
      ?s ?p ?o 
    }
s               p       o
VARCHAR         VARCHAR VARCHAR
________________

nodeID://b10006 p       o3a
nodeID://b10006 p       o3b
nodeID://b10007 p       o2a
nodeID://b10007 p       o2b

4 Rows. -- 4 msec.
]]></programlisting>
           </listitem>
           <listitem>
<programlisting><![CDATA[
]]></programlisting>
           </listitem>
           <listitem>Each bnode, as well as any "named" node, is identified internally as an integer:
<programlisting><![CDATA[
SPARQL 
  SELECT (<LONG::bif:iri_id_num>(?s)) AS ?s_num, ?p, ?o 
  FROM <http://sample/> 
  WHERE 
    { 
      ?s ?p ?o 
    };
s_num                p       o
INTEGER              VARCHAR VARCHAR
_____________________________

4611686018427397910  p       o3a
4611686018427397910  p       o3b
4611686018427397911  p       o2a
4611686018427397911  p       o2b

4 Rows. -- 5 msec.	
]]></programlisting>
           </listitem>
           <listitem>The integer can be converted back to internal identifier. Say, here we try to 
           	delete a triple that does not exist (even if the ID integer is valid):
<programlisting><![CDATA[
SPARQL 
  DELETE FROM <http://sample/>
    { 
      `bif:iri_id_from_num(4611686018427397911)` <p> <o3a> 
    };

Done. -- 5 msec.	
]]></programlisting>
           </listitem>
           <listitem>Should have no effect, because the "46..11" IRI has &lt;o2a&gt; and &lt;o2b&gt;, 
           	and was not requested &lt;o3a&gt;:
<programlisting><![CDATA[
SPARQL 
  SELECT * 
  FROM <http://sample/> 
  WHERE 
    { 
      ?s ?p ?o 
    };
s               p        o
VARCHAR         VARCHAR  VARCHAR
________________

nodeID://b10006 p        o3a
nodeID://b10006 p        o3b
nodeID://b10007 p        o2a
nodeID://b10007 p        o2b

4 Rows. -- 5 msec.	
]]></programlisting>
           </listitem>
           <listitem>Now let's try to delete a triple that does actually exist. Note the use of 
           	backquotes to insert an expression into template:
<programlisting><![CDATA[
SPARQL 
  DELETE FROM <http://sample/>
    { 
      `bif:iri_id_from_num(4611686018427397911)` <p> <o2a> 
    };

Done. -- 4 msec.	
]]></programlisting>
           </listitem>
           <listitem>So there's an effect:
<programlisting><![CDATA[
SPARQL 
  SELECT * 
  FROM <http://sample/> 
  WHERE 
    { 
      ?s ?p ?o 
    };
s                p        o
VARCHAR          VARCHAR  VARCHAR
_________________

nodeID://b10006  p        o3a
nodeID://b10006  p        o3b
nodeID://b10007  p        o2b

3 Rows. -- 2 msec.	
]]></programlisting>
           </listitem>
           <listitem>Now delete everything related to <code>nodeID://b10006</code> subject:
<programlisting><![CDATA[
SPARQL 
  WITH <http://sample/> 
  DELETE 
    { 
      ?s ?p ?o 
    } 
  WHERE 
    { 
      ?s ?p ?o .
      FILTER (?s = bif:iri_id_from_num(4611686018427397910)) 
    };

Done. -- 18 msec.	
]]></programlisting>
           </listitem>
           <listitem>Three minus two gives one triple remaining:
<programlisting><![CDATA[
SQL> SPARQL 
  SELECT * 
  FROM <http://sample/> 
  WHERE 
    { 
      ?s ?p ?o 
    };
s                p        o
VARCHAR          VARCHAR  VARCHAR
_________________

nodeID://b10007  p        o2b

1 Rows. -- 4 msec.	
]]></programlisting>
           </listitem>
         </orderedlist>       
         <para><emphasis>Note</emphasis>: IDs of bnodes will vary from server to server and even from 
         run to run on the same server, so the application should identify bnodes by properties 
         before doing <code>bif:iri_id_XXX</code> tricks.
         </para>
       </sect4>
       <sect4 id="rdfsparulexamples25"><title>Example usage of expressions inside CONSTRUCT, INSERT and DELETE {...} Templates</title>
         <para>When one wants to use expressions inside CONSTRUCT {...}, INSERT {...} or DELETE {...} 
         	construction templates, the expressions should be in back-quotes i.e:
         </para>	
<programlisting><![CDATA[
`expression`
]]></programlisting>
           <sect5 id="rdfsparulexamples25what"><title>What?</title>
             <para>How to construct RDF triples via SPARQL CONSTRUCT queries that include expressions.</para>
           </sect5>           
           <sect5 id="rdfsparulexamples25why"><title>Why?</title>
             <para>The are times when you need to post-process existing RDF triples en route to creating enhanced data views. For instance, enhancing the literal values associated with annotation properties such as rdfs:label and rdfs:comment .</para>
           </sect5>           
           <sect5 id="rdfsparulexamples25how"><title>Why?</title>
             <para>Here some SPARQL 1.1 Update Language examples showcasing how this is achieved using Virtuoso.</para>
             <sect6 id="rdfsparulexamples25howex1"><title>Example usage of expression inside CONSTRUCT</title>             
<programlisting><![CDATA[
CONSTRUCT 
  {
    ?inst  rdfs:label `bif:concat ( ?inst_label, 
                                    " Instance with up to ", 
                                    str(?core_val), 
                                    " logical processor cores and " , 
                                    str(?sess_val) , " concurrent ODBC sessions from licensed host" )`
  }
FROM <http://uda.openlinksw.com/pricing/>
WHERE 
  {
    ?inst a gr:Individual, oplweb:ProductLicense ;
                          rdfs:label ?inst_label ;
           oplweb:hasMaximumProcessorCores ?core ;
                        oplweb:hasSessions ?sess .
    
    ?core a gr:QuantitativeValueInteger ;
        gr:hasMaxValueInteger ?core_val .
             
    ?sess a gr:QuantitativeValueInteger ;
                  gr:hasValue ?sess_val .
}	
]]></programlisting>     
               <para>See <ulink url="http://uriburner.com/c/MBGD7Y">live results</ulink> of the query.</para>        
             </sect6>             
             <sect6 id="rdfsparulexamples25howex2"><title>Example usage of expression inside INSERT</title>             
<programlisting><![CDATA[
SPARQL 
INSERT INTO GRAPH <urn:mygraph> 
  {
    ?inst  rdfs:label `bif:concat ( ?inst_label, 
                                    " Instance with up to ", 
                                    str(?core_val), 
                                    " logical processor cores and " , 
                                    str(?sess_val) , 
                                    " concurrent ODBC sessions from licensed host" )`
  }
FROM <http://uda.openlinksw.com/pricing/>
WHERE 
  {
    ?inst a gr:Individual, oplweb:ProductLicense ;
                          rdfs:label ?inst_label ;
           oplweb:hasMaximumProcessorCores ?core ;
                        oplweb:hasSessions ?sess .
    
    ?core a gr:QuantitativeValueInteger ;
        gr:hasMaxValueInteger ?core_val .
             
    ?sess a gr:QuantitativeValueInteger ;
                  gr:hasValue ?sess_val .
};

Done. -- 406 msec.


SQL> SPARQL 
SELECT ?label 
FROM <urn:mygraph> 
WHERE 
  {
    ?inst  rdfs:label ?label
  };


label
VARCHAR
_______________________________________________________________________________

ODBC Driver (Single-Tier Lite Express Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
ODBC Driver (Single-Tier Lite Express Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
JDBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
OLEDB Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host
ADO.NET Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor cores and 5 concurrent ODBC sessions from licensed host

9 Rows. -- 31 msec.	
]]></programlisting>             
             </sect6>             
             <sect6 id="rdfsparulexamples25howex3"><title>Example usage of expression inside DELETE</title>             
<programlisting><![CDATA[
SPARQL 
DELETE FROM GRAPH <urn:mygraph> 
{ ?inst rdfs:label `bif:concat ( "JDBC Driver (Single-Tier Lite Edition) Instance with up to ", str(?core_val), " logical processor cores and " , str(?sess_val) , " concurrent ODBC sessions from licensed host" )` } 
FROM <http://uda.openlinksw.com/pricing/> 
WHERE 
{
    ?inst a gr:Individual, oplweb:ProductLicense ;
                          rdfs:label ?inst_label ;
           oplweb:hasMaximumProcessorCores ?core ;
                        oplweb:hasSessions ?sess .
    filter (regex(?inst_label,"JDBC Driver")) .                     
    
    ?core a gr:QuantitativeValueInteger ;
        gr:hasMaxValueInteger ?core_val .
             
    ?sess a gr:QuantitativeValueInteger ;
                  gr:hasValue ?sess_val .
}
;


Done. -- 32 msec.


SQL> SPARQL 
SELECT ?label 
FROM <urn:mygraph> 
WHERE 
  {
    ?inst  rdfs:label ?label
  };


label
VARCHAR
_______________________________________________________________________________

ODBC Driver (Single-Tier Lite Express Edition) Instance with up to 16 logical ...
ODBC Driver (Single-Tier Lite Express Edition) Instance with up to 16 logical ...
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor ...
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor ...
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor ...
ODBC Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor ...
OLEDB Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor ...
ADO.NET Driver (Single-Tier Lite Edition) Instance with up to 16 logical processor ...

8 Rows. -- 16 msec.	
]]></programlisting>             
             </sect6>                                       
           </sect5>           
       </sect4>
   </sect3>
   </sect2>
<sect2 id="sparqlbi"><title>Business Intelligence Extensions for SPARQL</title>
<para>
Virtuoso extends SPARQL with expressions in results, subqueries, aggregates and grouping.
These extensions allow a straightforward translation of arbitrary SQL queries to SPARQL.
This extension is called  "SPARQL BI", because the primary objective is to match needs of Business Intelligence.
The extended features apply equally to querying physical quads or relational tables mapped through Linked Data Views.
  </para>
<note><para>In this section, many examples use the TPC-H namespace. You may test them on your local demo database.
They use data from the TPC-H dataset that is mapped into a graph with an IRI of the form
http://example.com/tpch. When testing, you should replace the fake host name &quot;example.com&quot; with the host name of your own installation
verbatim, that is as specified in the &quot;DefaultHost&quot; parameter in the [URIQA] section of the Virtuoso configuration file.</para></note>
<sect3 id="rdfsparqlaggregate"><title>Aggregates in SPARQL</title>
<para>Virtuoso extends SPARQL with SQL like aggregate and "group by" functionality. This functionality is
also available by embedding SPARQL text inside SQL, but the SPARQL extension syntax has the benefit
of also working over the SPARQL protocol and of looking more SPARQL-like.
</para>
<para>The supported aggregates are <emphasis>COUNT</emphasis>, <emphasis>MIN</emphasis>,
<emphasis>MAX</emphasis>, <emphasis>AVG</emphasis> and <emphasis>SUM</emphasis>. These can take an
optional <emphasis>DISTINCT</emphasis> keyword. These are permitted only in the selection part of a
select query. If a selection list consists of a mix of variables and aggregates, the non-aggregate
selected items are considered to be grouping columns and a <emphasis>GROUP BY</emphasis> over them is implicitly added
at the end of the generated SQL query. Virtuoso also supports explicit syntax for
<emphasis>GROUP BY</emphasis>, <emphasis>ORDER BY</emphasis>, <emphasis>LIMIT</emphasis> and <emphasis>OFFSET</emphasis>.
There is no explicit syntax for <emphasis>HAVING</emphasis> in Virtuoso SPARQL.
</para>
<para>If a selection consists of aggregates exclusively, the result set has one row with the values
of the aggregates. If there are aggregates and variables in the selection, the result set has as many
rows as there are distinct combinations of the variables; the aggregates are then calculated over each
such distinct combination, as if there were a SQL GROUP BY over all non-aggregates.
The implicit grouping pays attention to all subexpressions in the return list; say, if a result column expression is <code>(?x * max (?y))</code> then
<code>?y</code> is aggregated and <code>?x</code> is not so it is grouped by ?x.
This also means that if a result column expression is <code>(bif:year (?shipdate))</code> then a group is made for each distinct <code>?shipdate</code>,
i.e. up to 366 groups for each distinct year.
If you need one group per year, write explicit <code>GROUP BY (bif:year (?shipdate))</code>.

</para>
<para>With the count aggregate the argument may be either <emphasis>*</emphasis>, meaning counting all rows, or a variable
name, meaning counting all the rows where this variable is bound. If there is no implicit <emphasis>GROUP BY</emphasis>,
there can be an optional <emphasis>DISTINCT</emphasis> keyword before the variable that is the argument of an aggregate.
  </para>
<para>There is a special syntax for counting distinct combinations of selected variables. This is:</para>
<programlisting><![CDATA[
SELECT COUNT DISTINCT ?v1 ... ?vn
  FROM ....
]]></programlisting>
<para>User-defined aggregate functions are not supported in current version of the SPARQL compiler.</para>
<sect4 id="rdfsparqlaggregatepathexpressions"><title>Path Expressions</title>
<para>Virtuoso has support for paths consisting of dereferencing properties in SPARQL. Virtuoso allows
simple paths in expressions and has a separate feature for transitivity:</para>

  <itemizedlist mark="bullet">
  <listitem>S+&gt;P: for "one or many values of P of S"</listitem>
  <listitem>S*&gt;P: for "zero or many values of P of S", so *&gt; may form a LEFT OUTER JOIN whereas +&gt; forms an INNER JOIN.</listitem>
  <listitem>S|&gt;P: is reserved for potential "single value of P of S or an error if there are many values"</listitem>
   </itemizedlist>
<para>If this property is set (for example by an Linked Data View) then +&gt; should be used.</para>

<para><emphasis>Simple Example</emphasis></para>
  <programlisting><![CDATA[
SELECT ?f+>foaf:name ?f|>foaf:mbox WHERE { ?x foaf:name "Alice" . ?x foaf:knows ?f . FILTER (?f+>foaf:name = "John") }
  ]]></programlisting>
<para>means:</para>
  <programlisting><![CDATA[
SELECT ?fname ?mbox
 WHERE
  {
    ?x foaf:knows ?f .
    ?x foaf:knows ?f .
    OPTIONAL {?f foaf:mbox ?mbox} .
    ?f foaf:name ?fname .
    ?x foaf:name "Alice" .
    ?x foaf:knows ?f2 .
    ?f2 foaf:name "John" .
  }
  ]]></programlisting>
<para><emphasis>Other Examples</emphasis></para>
  <programlisting><![CDATA[
SPARQL
DEFINE sql:signal-void-variables 1
PREFIX tpcd: <http://www.openlinksw.com/schemas/tpcd#>
PREFIX oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT
  ?l+>tpcd:returnflag,
  ?l+>tpcd:linestatus,
  sum(?l+>tpcd:linequantity) as ?sum_qty,
  sum(?l+>tpcd:lineextendedprice) as ?sum_base_price,
  sum(?l+>tpcd:lineextendedprice*(1 - ?l+>tpcd:linediscount)) as ?sum_disc_price,
  sum(?l+>tpcd:lineextendedprice*(1 - ?l+>tpcd:linediscount)*(1+?l+>tpcd:linetax)) as ?sum_charge,
  avg(?l+>tpcd:linequantity) as ?avg_qty,
  avg(?l+>tpcd:lineextendedprice) as ?avg_price,
  avg(?l+>tpcd:linediscount) as ?avg_disc,
  count(1) as ?count_order
FROM <http://example.com/tpcd>
WHERE {
    ?l a tpcd:lineitem .
    FILTER (?l+>tpcd:shipdate <= bif:dateadd ("day", -90, '1998-12-01'^^xsd:date)) }
ORDER BY ?l+>tpcd:returnflag ?l+>tpcd:linestatus

  ]]></programlisting>
  <programlisting><![CDATA[
SPARQL
DEFINE sql:signal-void-variables 1
PREFIX tpcd: <http://www.openlinksw.com/schemas/tpcd#>
SELECT
  ?supp+>tpcd:acctbal,
  ?supp+>tpcd:name,
  ?supp+>tpcd:has_nation+>tpcd:name as ?nation_name,
  ?part+>tpcd:partkey,
  ?part+>tpcd:mfgr,
  ?supp+>tpcd:address,
  ?supp+>tpcd:phone,
  ?supp+>tpcd:comment
FROM <http://example.com/tpcd>
   WHERE {
  ?ps a tpcd:partsupp; tpcd:has_supplier ?supp; tpcd:has_part ?part .
  ?supp+>tpcd:has_nation+>tpcd:has_region tpcd:name 'EUROPE' .
  ?part tpcd:size 15 .
  ?ps tpcd:supplycost ?minsc .
  { SELECT ?part min(?ps+>tpcd:supplycost) as ?minsc
            WHERE {
        ?ps a tpcd:partsupp; tpcd:has_part ?part; tpcd:has_supplier ?ms .
        ?ms+>tpcd:has_nation+>tpcd:has_region tpcd:name 'EUROPE' .
      } }
    FILTER (?part+>tpcd:type like '%BRASS') }
ORDER BY
  desc (?supp+>tpcd:acctbal)
  ?supp+>tpcd:has_nation+>tpcd:name
  ?supp+>tpcd:name
  ?part+>tpcd:partkey

  ]]></programlisting>
</sect4>
<sect4 id="rdfsparqlaggregateexamples"><title>Examples</title>
<sect5 id="rdfsparqlaggregateexamples1"><title>Example for count of physical triples in http://mygraph.com</title>
<programlisting><![CDATA[
SPARQL
SELECT COUNT (*)
  FROM <http://mygraph.com>
 WHERE {?s ?p ?o}
]]></programlisting>
<para><emphasis>Example for count of O's for each distinct P</emphasis></para>
<programlisting><![CDATA[
SPARQL define input:inference "http://mygraph.com"
SELECT ?p COUNT (?o)
  FROM <http://mygraph.com>
 WHERE {?s ?p ?o}
]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples2"><title>Example for count of triples, including inferred triples and the count of
distinct O values</title>
<programlisting><![CDATA[
SPARQL define input:inference "http://mygraph.com"
SELECT COUNT (?p) COUNT (?o) COUNT (DISTINCT ?o)
 FROM <http://mygraph.com>
WHERE {?s ?p ?o}
]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples3"><title>Example for get number of distinct bindings of ?s ?p ?o</title>
<programlisting><![CDATA[
SPARQL define input:inference "http://mygraph.com"
SELECT count distinct ?s ?p ?o
  FROM <http://mygraph.com>
 WHERE {?s ?p ?o}
]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples4"><title>Example for get counts and total prices of ordered items, grouped by item status</title>
  <programlisting><![CDATA[
  SPARQL
prefix tpch: <http://www.openlinksw.com/schemas/tpch#>
SELECT ?status count(*) sum(?extendedprice)
FROM <http://localhost.localdomain:8310/tpch>
WHERE {
    ?l a tpch:lineitem ;
      tpch:lineextendedprice ?extendedprice ;
      tpch:linestatus ?status .
    }
  ]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples5"><title>Example for get counts and total prices of ordered items, grouped by item status</title>
   <para><emphasis>Example: A dataset of people, some duplicated</emphasis></para>
   <para>Suppose there is a dataset with many people, some of them sharing the same name. To list them we would, ideally, execute the query:
   </para>
  <programlisting><![CDATA[
  SPARQL
SELECT DISTINCT
 (?name) ?person ?mail
 WHERE {
   ?person rdf:type foaf:Person .
   ?person foaf:name ?name .
   ?person foaf:mbox_sha1sum ?mail
    }
  ]]></programlisting>
   <para>Unfortunately, the facility to apply DISTINCT to a part of the result set row (i.e. to ?name) does not currently exist.
   (Although the above form is permitted, it's interpreted as being identical to 'SELECT DISTINCT ?name, ?person, ?mail WHERE ...')
   If there's demand for such a feature then we may introduce an aggregate called, say, SPECIMEN, that will return the very first of the aggregated values. e.g.:
   </para>
<programlisting><![CDATA[
SPARQL
SELECT ?name (specimen(?person)) (specimen(?mail))
      WHERE
        {
    ?person rdf:type foaf:Person .
    ?person foaf:name ?name .
    ?person foaf:mbox_sha1sum ?mail
  }
]]></programlisting>
   <para>As a workaround to this limitation, the MIN aggregate can be used, provided duplicates are few and there's no requirement
   that ?person should correspond to ?mail
(i.e. the result should contain some person node and some mail node but they don't have to be connected by foaf:mbox_sha1sum):
   </para>
<programlisting><![CDATA[
SPARQL
SELECT ?name (min(?person)) (min(?mail))
      WHERE
        {
    ?person rdf:type foaf:Person .
    ?person foaf:name ?name .
    ?person foaf:mbox_sha1sum ?mail
}
]]></programlisting>
   <para>Otherwise, a complicated query is needed:</para>
<programlisting><![CDATA[
SPARQL
SELECT
 ?name
 ((SELECT (min (?person3))
     WHERE {
         ?person3 rdf:type foaf:Person .
         ?person3 foaf:name ?name .
         ?person3 foaf:mbox_sha1sum ?mail } )) as ?person
 ?mail
 WHERE {
     { SELECT distinct ?name
       WHERE {
           ?person1 rdf:type foaf:Person .
           ?person1 foaf:name ?name .
           ?person1 foaf:mbox_sha1sum ?mail1 } }
     { SELECT ?name (min(?mail2)) as ?mail
       WHERE {
           ?person2 rdf:type foaf:Person .
           ?person2 foaf:name ?name .
           ?person2 foaf:mbox_sha1sum ?mail2 } }
 }
]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples6"><title>Example quering dbpedia</title>
<para>The following example demonstrate how to query dbpedia. Suppose there is local onotlogy,
which has a datatype property hasLocation with a string containing city names. The query below finds
which of those cities are in dbpedia:</para>
<programlisting><![CDATA[
SPARQL
PREFIX dbpprop: <http://dbpedia.org/property/>
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX vocab:<http://myexample.com/localOntology.rdf>
PREFIX dbpedia: <http://dbpedia.org/>
PREFIX dbpres: <http://dbpedia.org/resource/>
SELECT ?city
WHERE
  {
    ?sub :location ?city .
    FILTER(bif:exists(( ASK { ?subdb a dbo:City . ?subdb dbpprop:officialName ?city })))
  }
]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples7"><title>Example for MAX with HAVING and GROUP BY</title>
<programlisting><![CDATA[
## Example "Find which town or city in
## the UK has the largest proportion of students.

PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>
PREFIX dbpedia-owl-uni: <http://dbpedia.org/ontology/University/>
PREFIX dbpedia-owl-inst: <http://dbpedia.org/ontology/EducationalInstitution/>

SELECT ?town COUNT(?uni)
       ?pgrad ?ugrad
       MAX(?population)
       ( ((?pgrad+?ugrad)/ MAX(?population))*100 ) AS ?percentage
WHERE
  {
    ?uni dbpedia-owl-inst:country dbpedia:United_Kingdom ;
                         dbpedia-owl-uni:postgrad ?pgrad ;
                        dbpedia-owl-uni:undergrad ?ugrad ;
                             dbpedia-owl-inst:city ?town .
    OPTIONAL
    {
          ?town dbpedia-owl:populationTotal ?population .
          FILTER (?population > 0 )
    }
  }
GROUP BY ?town ?pgrad ?ugrad
HAVING ( ( ( (?pgrad+?ugrad)/ MAX(?population) )*100 ) > 0 )
ORDER BY DESC 6
]]></programlisting>
</sect5>
<sect5 id="rdfsparqlaggregateexamples8"><title>Example Aggregating Distance Values Over Years</title>
<para>The following example demonstrate how to aggregate Distance Values Over Years:</para>
<para>First we insert some data in a graph with name for ex. &lt;urn:dates:distances&gt;:</para>
<programlisting><![CDATA[
SQL> SPARQL INSERT INTO GRAPH <urn:dates:distances>
  {
    <:a1> <http://purl.org/dc/elements/1.1/date> <2010-12-23T00:00:00> .
    <:a1> <http://linkedgeodata.org/vocabulary#distance> <0.955218675> .    
    <:a2> <http://purl.org/dc/elements/1.1/date> <2010-12-24T00:00:00> .
    <:a2> <http://linkedgeodata.org/vocabulary#distance> <0.798155989> .
    <:a3> <http://purl.org/dc/elements/1.1/date> <2010-12-25T00:00:00> .
    <:a3> <http://linkedgeodata.org/vocabulary#distance> <0.064686628> .
    <:a4> <http://purl.org/dc/elements/1.1/date> <2010-12-26T00:00:00> .
    <:a4> <http://linkedgeodata.org/vocabulary#distance> <0.279800332> .
    <:a5> <http://purl.org/dc/elements/1.1/date> <2010-12-27T00:00:00> .
    <:a5> <http://linkedgeodata.org/vocabulary#distance> <0.651255995> .
    <:a6> <http://purl.org/dc/elements/1.1/date> <2010-12-28T00:00:00> .
    <:a6> <http://linkedgeodata.org/vocabulary#distance> <0.094410557> .
    <:a7> <http://purl.org/dc/elements/1.1/date> <2010-12-29T00:00:00> .
    <:a7> <http://linkedgeodata.org/vocabulary#distance> <0.43461913> .
    <:a8> <http://purl.org/dc/elements/1.1/date> <2010-12-30T00:00:00> .
    <:a8> <http://linkedgeodata.org/vocabulary#distance> <0.264862918> .
    <:a9> <http://purl.org/dc/elements/1.1/date> <2010-12-31T00:00:00> .
    <:a9> <http://linkedgeodata.org/vocabulary#distance> <0.770588658> .
    <:a10> <http://purl.org/dc/elements/1.1/date> <2011-01-01T00:00:00> .
    <:a10> <http://linkedgeodata.org/vocabulary#distance> <0.900997627> .
    <:a11> <http://purl.org/dc/elements/1.1/date> <2011-01-02T00:00:00> .
    <:a11> <http://linkedgeodata.org/vocabulary#distance> <0.324972375> .
    <:a12> <http://purl.org/dc/elements/1.1/date> <2011-01-03T00:00:00> .
    <:a12> <http://linkedgeodata.org/vocabulary#distance> <0.937221226> .
    <:a13> <http://purl.org/dc/elements/1.1/date> <2011-01-04T00:00:00> .
    <:a13> <http://linkedgeodata.org/vocabulary#distance> <0.269511925> .
    <:a14> <http://purl.org/dc/elements/1.1/date> <2011-01-05T00:00:00> .
    <:a14> <http://linkedgeodata.org/vocabulary#distance> <0.726014538> .
    <:a15> <http://purl.org/dc/elements/1.1/date> <2011-01-06T00:00:00> .
    <:a15> <http://linkedgeodata.org/vocabulary#distance> <0.843581439> .
    <:a16> <http://purl.org/dc/elements/1.1/date> <2011-01-07T00:00:00> .
    <:a16> <http://linkedgeodata.org/vocabulary#distance> <0.835685559> .
    <:a17> <http://purl.org/dc/elements/1.1/date> <2011-01-08T00:00:00> .
    <:a17> <http://linkedgeodata.org/vocabulary#distance> <0.673213742> .
    <:a18> <http://purl.org/dc/elements/1.1/date> <2011-01-09T00:00:00> .
    <:a18> <http://linkedgeodata.org/vocabulary#distance> <0.055026879> .
    <:a19> <http://purl.org/dc/elements/1.1/date> <2011-01-10T00:00:00> .
    <:a19> <http://linkedgeodata.org/vocabulary#distance> <0.987475424> .
    <:a20> <http://purl.org/dc/elements/1.1/date> <2011-01-11T00:00:00> .
    <:a20> <http://linkedgeodata.org/vocabulary#distance> <0.167315598> .
    <:a21> <http://purl.org/dc/elements/1.1/date> <2011-01-12T00:00:00> .
    <:a21> <http://linkedgeodata.org/vocabulary#distance> <0.545317103> .
    <:a22> <http://purl.org/dc/elements/1.1/date> <2011-01-13T00:00:00> .
    <:a22> <http://linkedgeodata.org/vocabulary#distance> <0.75137005> .
    <:a23> <http://purl.org/dc/elements/1.1/date> <2011-01-14T00:00:00> .
    <:a23> <http://linkedgeodata.org/vocabulary#distance> <0.123649985> .
    <:a24> <http://purl.org/dc/elements/1.1/date> <2011-01-15T00:00:00> .
    <:a24> <http://linkedgeodata.org/vocabulary#distance> <0.750214251> .
  };	
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <urn:dates:distances>, 48 (or less) triples -- done

1 Rows. -- 94 msec.
]]></programlisting>
<para>Then we execute the following query:</para>
<programlisting><![CDATA[
SQL> SPARQL 
PREFIX dst: <http://linkedgeodata.org/vocabulary#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
SELECT (bif:year( bif:stringdate(?sdate)) AS ?syear) (bif:sum( bif:number(?dist)) AS ?distance)
FROM <urn:dates:distances>
WHERE
  {
    ?row dc:date ?sdate .
    ?row dst:distance ?dist
  }
GROUP BY (bif:year(bif:stringdate(?sdate)))
ORDER BY ASC(bif:year(bif:stringdate(?sdate)));

syear                                distance
VARCHAR                              VARCHAR
________________________________________________

2010                                 4.313598882
2011                                 8.891567721

2 Rows. -- 31 msec.
]]></programlisting>
</sect5>	
</sect4>
<sect4 id="rdfsparqlaggregatenote"><title>Note on Aggregates and Inference</title>
<para>Inferencing is added to a SPARQL query only for those variables whose value is actually used. Thus,
</para>
<programlisting><![CDATA[
SELECT COUNT (*)
 FROM <http://mygraph.com>
WHERE {?s ?p ?o}
]]></programlisting>
<para>will not return inferred values since s, p, and o are not actually used. In contrast,
</para>
<programlisting><![CDATA[
SPARQL
SELECT COUNT (?s) COUNT (?p) COUNT (?o)
 FROM <http://mygraph.com>
WHERE {?s ?p ?o}
]]></programlisting>
<para>will also return all the inferred triples.
</para>
<para>
Note: This difference in behaviour may lead to confusion and will, therefore, likely be altered in the future.
</para>
</sect4>
</sect3>
<sect3 id="rdfsparqlarrowop"><title>Pointer Operator (<emphasis>+&gt;</emphasis> and <emphasis>*&gt;</emphasis>)</title>
<para>
When expressions occur in result sets, many variables are often introduced only for the purpose of passing a value from a triple pattern to the result expression.
This is inconvenient because many triple patterns are trivial. The presence of large numbers of variable names masks &quot;interesting&quot; variables that are used in more than once in pattern and which establish logical relationships between different parts of the query.
As a solution we introduce pointer operators.</para>
<para>
The <emphasis>+&gt;</emphasis> (pointer) operator allows referring to a property value without naming it as a variable and explicitly writing a triple pattern. We can shorten the example above to:</para>
<programlisting><![CDATA[SPARQL
prefix tpch: <http://www.openlinksw.com/schemas/tpch#>
SELECT ?l+>tpch:linestatus count(*) sum(?l+>tpch:lineextendedprice)
FROM <http://localhost.localdomain:8310/tpch>
WHERE { ?l a tpch:lineitem }]]></programlisting>
<para>
The <emphasis>?subject+&gt;propertyname</emphasis> notation is equivalent to having a triple pattern <emphasis>?subject propertyname ?aux_var</emphasis> binding an auxiliary variable to the mentioned property of the subject, within the group pattern enclosing the reference.
For a SELECT, the enclosing group pattern is considered to be the top level pattern of the where clause or, in the event of a union, the top level of each term of the union.
Each distinct pointer adds exactly one triple pattern to the enclosing group pattern. Multiple uses of <emphasis>+&gt;</emphasis> with the same arguments do not each add a triple pattern.
(Having multiple copies of an identical pattern might lead to changes in cardinality if multiple input graphs were being considered.
If a lineitem had multiple discounts or extended prices, then we would get the cartesian product of both.)
</para>
<para>
If a property referenced via <emphasis>+&gt;</emphasis> is absent, the variable on the left side of the operator is not bound in the enclosing group pattern because it should be bound in all triple patterns where it appears as a field, including implicitly added patterns.
</para>
<para>
The <emphasis>?subject*&gt;propertyname</emphasis> notation is introduced in order to access optional property values. It adds an OPTIONAL group <emphasis>OPTIONAL { ?subject propertyname ?aux_var }</emphasis>, not a plain triple pattern, so the binding of ?subject is not changed even if the object variable is not bound. If the property is set for all subjects in question then the results of <emphasis>*&gt;</emphasis> and <emphasis>+&gt;</emphasis> are the same. All other things being equal, the <emphasis>+&gt;</emphasis> operator produces better SQL code than <emphasis>*&gt;</emphasis> so use <emphasis>*&gt;</emphasis> only when it is really needed.</para>
</sect3>
<sect3 id="rdfsparqlnesting"><title>Subqueries in SPARQL</title>
<para>
Pure SPARQL does not allow binding a value that is not retrieved through a triple pattern.
We lift this restriction by allowing expressions in the result set and providing names for result columns.
We also allow a SPARQL SELECT statement to appear in another SPARQL statement in any place where a group pattern may appear.
The names of the result columns form the names of the variables bound, using values from the returned rows.
This resembles derived tables in SQL.</para>
<para>
For instance, the following statement finds the prices of the 1000 order lines with the biggest discounts:</para>

<programlisting><![CDATA[SPARQL
define sql:signal-void-variables 1
prefix tpch: <http://www.openlinksw.com/schemas/tpch#>
SELECT ?line ?discount (?extendedprice * (1 - ?discount)) as ?finalprice
FROM <http://localhost.localdomain:8310/tpch>
WHERE
  {
    ?line a tpch:lineitem ;
    tpch:lineextendedprice ?extendedprice ;
    tpch:linediscount ?discount .
  }
ORDER BY DESC (?extendedprice * ?discount)
LIMIT 1000]]></programlisting>

<para>
After ensuring that this query works correctly, we can use it to answer more complex questions.
Imagine that we want to find out how big the customers are who have received the biggest discounts.</para>

<programlisting><![CDATA[SPARQL
define sql:signal-void-variables 1
prefix tpch: <http://www.openlinksw.com/schemas/tpch#>
SELECT ?cust sum(?extendedprice2 * (1 - ?discount2)) max (?bigdiscount)
FROM <http://localhost.localdomain:8310/tpch>
WHERE
  {
    {
      SELECT ?line (?extendedprice * ?discount) as ?bigdiscount
      WHERE {
        ?line a tpch:lineitem ;
          tpch:lineextendedprice ?extendedprice ;
          tpch:linediscount ?discount . }
      ORDER BY DESC (?extendedprice * ?discount)
      LIMIT 1000
    }
    ?line tpch:has_order ?order .
    ?order tpch:has_customer ?cust .
    ?cust tpch:customer_of ?order2 .
    ?order2 tpch:order_of ?line2 .
    ?line2 tpch:lineextendedprice ?extendedprice2 ;
      tpch:linediscount ?discount2 .
  }
ORDER BY (SUM(?extendedprice2 * (1 - ?discount2)) / MAX (?bigdiscount))]]></programlisting>

<para>
The inner select finds the 1000 biggest (in absolute value) discounts and their order lines. For each line we find orders of it, and the customer. For each customer found, we find all the orders he made and all the lines of each of the orders (variable ?line2).</para>
<para>
Note that the inner select does not contain FROM clauses. It is not required because the inner select inherits the access permissions of all the outer queries. It is also important to note that the internal variable bindings of the subquery are not visible in the outer query; only the result set variables are bound. Similarly, variables bound in the outer query are not accessible to the subquery.</para>
<para>
Note also the declaration <emphasis>define sql:signal-void-variables 1</emphasis> that forces the SPARQL compiler to signal errors if some variables cannot be bound due to misspelt names or attempts to make joins across disjoint domains. These diagnostics are especially important when the query is long.</para>
</sect3>
<sect3 id="rdfsparqlbackq"><title>Expressions in Triple Patterns</title>
<para>In addition to expressions in filters and result sets, Virtuoso allows the use of expressions in triples of a
CONSTRUCT pattern or WHERE pattern - an expression can be used instead of a constant or a variable name for a subject, predicate or object.
When used in this context, the expression is surrounded by backquotes.</para>

<para><emphasis>Example: With a WHERE Clause:</emphasis></para>
<para>The following example returns all the distinct 'fragment' parts of all subjects in all graphs that have some predicate whose value is equal to 2+2.</para>
<programlisting><![CDATA[
SQL>SPARQL SELECT distinct (bif:subseq (?s, bif:strchr (?s, '#')))
   WHERE {
     graph ?g {
       ?s ?p `2+2` .
       FILTER (! bif:isnull (bif:strchr (?s, '#') ) )
     } };

callret
VARCHAR
----------
#four
]]></programlisting>
<para>Inside a WHERE part, every expression in a triple pattern is replaced with new variable and a filter expression is added to the enclosing group. The new filter is an equality of the new variable and the expression. Hence the sample above is identical to:</para>
<programlisting><![CDATA[
SPARQL
SELECT distinct (bif:subseq (?s, bif:strchr (?s, '#')))
   WHERE {
     graph ?g {
       ?s ?p ?newvariable .
       FILTER (! bif:isnull (bif:strchr (?s, '#') ) )
       FILTER (?newvariable = (2+2)) .
     } }
]]></programlisting>
<para><emphasis>Example: With CONSTRUCT</emphasis></para>
<programlisting><![CDATA[
CONSTRUCT {
  <http://bio2rdf.org/interpro:IPR000181>
<http://bio2rdf.org/ns/bio2rdf#hasLinkCount>
`(SELECT (count(?s)) as ?countS
   WHERE { ?s ?p <http://bio2rdf.org/interpro:IPR000181> })` }
WHERE { ?s1 ?p1 ?o1 } limit 1
]]></programlisting>
<para>The result should be:</para>
<programlisting><![CDATA[
<http://bio2rdf.org/interpro:IPR000181> <http://bio2rdf.org/ns/bio2rdf#hasLinkCount> "0"^^<http://www.w3.org/2001/XMLSchema#integer> .
]]></programlisting>
<para><emphasis>Example: Inserting into a graph using an expression</emphasis></para>
<programlisting><![CDATA[
SQL>SPARQL insert into graph <http://MyNewGraph.com/> {
<http://bio2rdf.org/interpro:IPR000181>
<http://bio2rdf.org/ns/bio2rdf#hasLinkCount>
  `(SELECT (count(?s)) as ?countS
    WHERE { ?s ?p <http://bio2rdf.org/interpro:IPR000181> })` }
 WHERE { ?s1 ?p1 ?o1 } limit 1  ;

callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://MyNewGraph.com/>, 1 triples -- done

1 Rows. -- 30 msec.
]]></programlisting>
</sect3>
</sect2>
</sect1>

<sect1 id="rdfgraphsecurity"><title>RDF Graphs Security</title>
<sect2 id="rdfgraphsecuritygroups"><title>RDF Graph Groups</title>
<para>In some cases, the data-set of a SPARQL query is not known at compile time. It is possible to pass
IRIs of source graphs via parameters, but the method is not perfect as:</para>
<itemizedlist mark="bullet">
  <listitem>not all protocols are suitable for parameter passing, and no one is an interoperable standard</listitem>
  <listitem>passing list of IRIs as a parameter will usually require the use of Virtuoso-specific functions
in the text of SPARQL query, that's bad for some query builders.</listitem>
  <listitem>lack of knowledge about actual graphs may damage query optimization</listitem>
</itemizedlist>
<para>It would be nice to create named lists of graphs and a clause like "SELECT from all graph names of the specified list".
<emphasis>"Graph groups"</emphasis> serve for this purpose. That is Virtuoso-specific SPARQL extension that let create a named list of IRIs such that
if name of the list is used in <emphasis>FROM</emphasis> clause like <emphasis>IRI</emphasis> of default
graph then it is equivalent to list of <emphasis>FROM</emphasis> clauses, one clause for each item of the
list.</para>
<para>Internally, descriptions of graph groups are kept in two tables:</para>
<emphasis>Table of graph groups:</emphasis>
<programlisting><![CDATA[
create table DB.DBA.RDF_GRAPH_GROUP (
  RGG_IID IRI_ID not null primary key,	-- IRI ID of RGG_IRI field
  RGG_IRI varchar not null,		-- Name of the group
  RGG_MEMBER_PATTERN varchar,		-- Member IRI pattern
  RGG_COMMENT varchar			-- Comment
  )
create index RDF_GRAPH_GROUP_IRI on DB.DBA.RDF_GRAPH_GROUP (RGG_IRI)
;
]]></programlisting>
<para><emphasis>Table of contents of groups:</emphasis></para>

<programlisting><![CDATA[
create table DB.DBA.RDF_GRAPH_GROUP_MEMBER (
  RGGM_GROUP_IID IRI_ID not null,	-- IRI_ID of the group
  RGGM_MEMBER_IID IRI_ID not null,	-- IRI_ID of the group member
  primary key (RGGM_GROUP_IID, RGGM_MEMBER_IID)
  )
;
]]></programlisting>

<para>Fields <emphasis>RGG_MEMBER_PATTERN</emphasis> and <emphasis>RGG_COMMENT</emphasis> are not used by system internals but applications may wish to write their data there for future reference.
<emphasis>RGG_COMMENT</emphasis> is supposed to be human-readable description of the group and <emphasis>RGG_MEMBER_PATTERN</emphasis> may be useful for functions that automatically add
IRIs of a given graph to all graph groups such that the graph IRI string match <emphasis>RGG_MEMBER_PATTERN</emphasis> regexp pattern.
</para>
<para>A dictionary of all groups and their members is cached in memory for fast access.
Due to this reason, applications may read these tables and modify <emphasis>RGG_MEMBER_PATTERN</emphasis> and <emphasis>RGG_COMMENT</emphasis> if needed but not change other fields directly.
The following API procedures makes changes in a safe way:
</para>
<programlisting><![CDATA[
DB.DBA.RDF_GRAPH_GROUP_CREATE (
  in group_iri varchar,
  in quiet integer,
  in member_pattern varchar := null,
  in comment varchar := null)
]]></programlisting>

<para>That creates a new empty graph group. An error is signaled if the group exists already and quiet
parameter is zero.</para>

<programlisting><![CDATA[
DB.DBA.RDF_GRAPH_GROUP_INS (in group_iri varchar, in memb_iri varchar)
DB.DBA.RDF_GRAPH_GROUP_DEL (in group_iri varchar, in memb_iri varchar)
]]></programlisting>


<para>These two are to add or remove member to an existing group. Double insert or removal of not a member
will not signal errors, but missing group will.be signaled.</para>

<programlisting><![CDATA[
DB.DBA.RDF_GRAPH_GROUP_DROP (
  in group_iri varchar,
  in quiet integer)
]]></programlisting>

<para>That removes graph group. An error is signaled if the group did not exist before the call and quiet
parameter is zero.</para>

<para>Graph groups are <emphasis>"macro-expanded"</emphasis> only in FROM clauses and have no effect on
FROM NAMED or on GRAPH &lt;IRI&gt; {...} . Technically, it is not prohibited to use an IRI as both plain
graph IRI and graph group IRI in one storage but this is confusing and is not recommended.</para>

<para>Graph groups can not be members of other graph groups, i.e. the IRI of a graph group can appear in the
list of members of some group but it will be treated as plain graph IRI and will not cause recursive
expansion of groups.</para>
</sect2>
<sect2 id="rdfgraphsecuritynotfrom"><title>NOT FROM and NOT FROM NAMED Clauses</title>
<para>In addition to standard FROM and FROM NAMED clauses, Virtuoso extends SPARQL with NOT FROM and NOT
FROM NAMED clauses of "opposite" meaning.</para>

<programlisting><![CDATA[
SELECT ... NOT FROM <x> ... WHERE {...}
]]></programlisting>

<para>means "SELECT FROM other graphs, but not from the given one".
This is especially useful because NOT FROM supports graph groups (NOT FROM NAMED supports only plain graphs).
So if</para>

<programlisting><![CDATA[
<http://example.com/users/private>
]]></programlisting>

<para>is a graph group of all graphs with confidential data about users then</para>

<programlisting><![CDATA[
SELECT * NOT FROM <http://example.com/users/private> WHERE {...}
]]></programlisting>

<para>will be restricted only to insecure data.</para>

<para>NOT FROM overrides any FROM and NOT FROM NAMED overrides any FROM NAMED, the order of clauses in the
query text is not important.</para>

<para>The SPARQL web service endpoint configuration string may contain pragmas
<emphasis>input:default-graph-exclude</emphasis> and <emphasis>input:named-graph-exclude</emphasis> that
become equivalent to NOT FROM and NOT FROM NAMED clauses like <emphasis>input:default-graph-uri</emphasis>
and <emphasis>input:named-graph-uri</emphasis> mimics FROM and FROM NAMED.</para>
</sect2>
  <sect2 id="rdfgraphsecuritylevel"><title>Graph-Level Security</title>
    <para>Virtuoso supports graph-level security for "physical" RDF storage. That is somewhat similar to table access
    	permissions in SQL. However, the difference between SPARQL and SQL data models results in totally different style
    	of security administration. In SQL, when new application is installed it comes with its own set of tables and every
    	query in its code explicitly specifies tables in use. Security restrictions of two applications interfere only if
    	applications knows each other and are supposedly designed to cooperate. It is possible to write an application that
    	will get list of available tables and retrieve data from any given table but that is a special case and it usually
    	requires DBA privileges.</para>
    <para>In SPARQL, data of different applications shares one table and the query language allows to select data of all
    	applications at once. This feature makes SPARQL convenient for cross-application data integration. At the same time,
    	that become a giant security hole if any sensitive data are stored.</para>
    <para>A blind copying SQL security model to SPARQL domain would result in significant loss of performance or weak
    	security or even both problems at the same time. That is why SPARQL model is made much more restrictive, even if
    	it becomes inconvenient for some administration tasks.</para>
    <para>Graph-level security does not replace traditional SQL security. A user should become member of appropriate
    	group (<code>SPARQL_SELECT</code>, <code>SPARQL_SPONGE</code> or <code>SPARQL_UPDATE</code>) in order to start
    	using its graph-level privileges.</para>
  </sect2>
  <sect2 id="rdfgraphsecuritylevelrow"><title>Graph-Level Security and SQL</title>
    <para>SPARQL-level graph security is sufficient for SPARQL client operating over HTTP. It is not sufficient for SQL
    	clients due to the fact that graph level security is baked into the SPARQL compiler, not by an SQL compiler.</para>
    <para>The Virtuoso SPARQL compiler analyzes the graph-level permissions of a user (an identity principal named using
    	an identifier e.g., WebID or NetID). For each triple pattern or graph group pattern the compiler adds an implicit
    	FILTER () that ensures that appropriate privileges are granted on target named graphs to a given user. Ultimately,
    	these FILTERs becomes part of the generated SQL code processed against the RDF_QUAD and related RDF data management
    	system tables.</para>
    <para>SQL users accessing Virtuoso via ODBC, JDBC, ADO.NET, and OLE-DB connections have the ability to execute arbitrary
    	SQL code via stored procedures, subject to SQL level privileges on target Tables and Views which provides a point of
    	vulnerability to the RDF system tables (RDF_QUAD and others). To close this vulnerability, the SQL compiler restricts
    	SQL connection access, in regards to RDF system tables, to members of <code>the SPARQL_SELECT_RAW</code> group.</para>
    <para><i><emphasis>Note:</emphasis> <code>SPARQL_SELECT_RAW</code> group is a feature applicable to Virtuoso 7.5 or higher.</i></para>
    <sect3 id="rdfgraphsecuritylevelrowex"><title>Graph-Level Security and SQL Usage Example</title>
      <para>The following example demonstrates how to grant <code>SPARQL_SELECT_RAW</code> to a Virtuoso SQL user:</para>
<programlisting><![CDATA[
SQL> DB.DBA.USER_CREATE ('John', 'John');
Done. -- 0 msec.
SQL> GRANT SPARQL_SELECT to "John";
Done. -- 0 msec.
SQL> GRANT SPARQL_SELECT_RAW to "John";
Done. -- 0 msec.
]]></programlisting>
    </sect3>
</sect2>
<sect2 id="rdfgraphsecurityunddefperm"><title>Understanding Default Permissions</title>

<para>In relational database, default permissions are trivial. DBA is usually the only account that can
access any table for both read and write. Making some table public or private does not affect applications
that do not refer that table in the code. Tables are always created before making security restrictions on
them.</para>

<para>Chances are very low that an application will unintentionally create some table and fill in with
confidential data. There are no unauthenticated users, any client has some user ID and no one user is
"default user" so permissions of any two users are always independent.</para>

<para>SPARQL access can be anonymous and graphs can be created during routine data manipulation.
For anonymous user, only public resources are available. Thus "default permissions" on some or all graphs are
actually permissions of "nobody" user, (the numeric ID of this user can be obtained by http_nobody_uid()
function call). As a consequence, there's a strong need in "default permission" for a user, this is the only
way to specify what to do with all graphs that does not exist now it might exist in some future.
</para>

<para>An attempt to make default permissions wider than specific is always potential security hole in SPARQL,
so this is strictly prohibited.</para>

<para>Four sorts of access are specified by four bits of an integer "permission bit-mask", plain old UNIX
style:</para>

<itemizedlist mark="bullet">
  <listitem>Bit 1 permits read access.</listitem>
  <listitem>Bit 2 permits write access via SPARUL and it's basically useless without bit 1 set.</listitem>
  <listitem>Bit 4 permits write access via "RDF sponge" methods and it's basically useless
without bits 1 and 2 set.</listitem>
  <listitem>Bit 8 allows to obtain list of members of graph group; an IRI can be used as graph IRI and as
graph group IRI at the same time so bit 8 can be freely combined with any of bits 1, 2 or 4.</listitem>
</itemizedlist>

<para>Note that obtaining the list of members of a graph group does not grant any access permissions to
triples from member graphs. It is quite safe to mix secure and public graphs in one graph group.</para>

<para>When a SPARQL query should check whether a given user have permission to access a given graph then
the order of checks is as follows:</para>

<orderedlist>
  <listitem>permissions of the user on the specific graph;</listitem>
  <listitem>default permissions of the user on all graphs;</listitem>
  <listitem>public permissions on the specific graph;</listitem>
  <listitem>public permissions on all graphs</listitem>
</orderedlist>

<para>If no one above mentioned permission is set then the access is "read/write/sponge/list".</para>
<para>For "nobody" user, steps 3 and 4 become exact copies of steps 1 and 2 so they are skipped.</para>
</sect2>
<sect2 id="rdfgraphsecurityintconfsec"><title>Initial Configuration of SPARQL Security</title>
<para>It is convenient to configure the RDF storage security by adding restrictions in the order inverse
to the order of checks:</para>

<orderedlist>
  <listitem>Step 1: Set public permissions on all graphs to the most restricted level of any application
that will be installed. So if any single graph will be unreadable for public, then public permissions on
all graphs should be set to 0 or 8.</listitem>
  <listitem>Step 2: Public permissions on "insecure" graphs should be set. So if the database contains
DBpedia or WordNet or some other data of Linking Open Data project then public permissions for that graphs
may be set to 1.</listitem>
  <listitem>Step3: Configure trusted users, such as administrative DBA-like accounts, and to specify their
permissions on all graphs.</listitem>
  <listitem>Step 4: Some additional right can be granted to some specific users on some specific graphs.</listitem>
</orderedlist>

<para>Note that there's no need to permit something to DBA itself, because DBA's default permissions are
set automatically.</para>
<sect3 id="rdfgraphsecurityintconfsecuser"><title>Configuring New User</title>
<orderedlist>
  <listitem>Step 1: Grant SPARQL_SELECT, SPARQL_SPONGE or SPARQL_UPDATE to the user.</listitem>
  <listitem>Step 2: Set user's permissions on all graphs.</listitem>
  <listitem>Step 3: Grant rights on some specific graphs.</listitem>
</orderedlist>
</sect3>
<sect3 id="rdfgraphsecurityintex"><title>Example: Blogs and Resource Sharing</title>

  <para>The following example demonstrates usage of the following functions:</para>

  <itemizedlist mark="bullet">
    <listitem><link linkend="fn_rdf_default_user_perms_set"><function>DB.DBA.RDF_DEFAULT_USER_PERMS_SET (uname, permissions, set_private)</function></link> ;</listitem>
    <listitem><link linkend="fn_rdf_default_user_perms_del"><function>DB.DBA.RDF_DEFAULT_USER_PERMS_DEL (uname, set_private)</function></link> ;</listitem>
    <listitem><link linkend="fn_rdf_graph_user_perms_set"><function>DB.DBA.RDF_GRAPH_USER_PERMS_SET (graph_iri, uname, permissions)</function></link> ;</listitem>  
    <listitem><link linkend="fn_rdf_graph_user_perms_del"><function>DB.DBA.RDF_GRAPH_USER_PERMS_DEL (graph_iri, uname)</function></link> ;</listitem>  
    <listitem><link linkend="fn_rdf_all_user_perms_del"><function>DB.DBA.RDF_ALL_USER_PERMS_DEL (uname, uid)</function></link> ;</listitem>  
    <listitem></listitem>
  </itemizedlist>
  
<para>Consider a "groupware" application that let users create personal resources with access policies.</para>

<programlisting><![CDATA[
-- First, create few users, in alphabetical order.
SQL> DB.DBA.USER_CREATE ('Anna', 'Anna');
Done. -- 0 msec.
SQL> DB.DBA.USER_CREATE ('Brad', 'Brad');
Done. -- 0 msec.
SQL> DB.DBA.USER_CREATE ('Carl', 'Carl');
Done. -- 16 msec.
SQL> GRANT SPARQL_UPDATE to "Anna";
Done. -- 0 msec.
SQL> GRANT SPARQL_UPDATE to "Brad";
Done. -- 0 msec.
SQL> GRANT SPARQL_UPDATE to "Carl";
Done. -- 0 msec.

-- At least some data are supposed to be confidential, thus the whole storage becomes confidential.
SQL> DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('nobody', 0);
Done. -- 16 msec.

-- Moreover, no one of created users have access to all graphs (even for reading).
SQL> DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Anna', 0);
Done. -- 0 msec.
SQL> DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Brad', 0);
Done. -- 0 msec.
SQL> DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Carl', 0);
Done. -- 0 msec.


-- Add Anna's and Brad's private graph to the group http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs:
SQL> DB.DBA.RDF_GRAPH_GROUP_INS ('http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs', 'http://example.com/Anna/private');
SQL> DB.DBA.RDF_GRAPH_GROUP_INS ('http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs', 'http://example.com/Brad/private');

-- Anna's graphs:
--insert simple data in Anna's personal system graph:
SQL> SPARQL INSERT IN <http://example.com/Anna/system> { <Anna-system> a <secret> };   
Done. -- 31 msec.

--insert simple data in Anna's private graph:
SQL> SPARQL INSERT IN <http://example.com/Anna/private> { <Anna-private> a <secret> };
Done. -- 0 msec.


-- Anna can only read her personal system data graph.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Anna/system', 'Anna', 1);
Done. -- 0 msec

-- Anna can read and write her private data graph.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Anna/private', 'Anna', 3);
Done. -- 0 msec

-- Brad's graphs:
-- insert simple data in Brad's personal system graph:
SQL> SPARQL INSERT IN <http://example.com/Brad/system> { <Brad-system> a <secret> };   
Done. -- 0 msec

-- insert simple data in Brad's private graph:
SQL> SPARQL INSERT IN <http://example.com/Brad/private> { <Brad-private> a <secret> };
Done. -- 0 msec

-- Brad can only read his personal system data graph.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Brad/system', 'Brad', 1);
Done. -- 0 msec

-- Brad can read and write his private data graph.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Brad/private', 'Brad', 3);
Done. -- 0 msec


-- Friends graphs:
SQL> SPARQL INSERT IN <http://example.com/Anna/friends> { <Anna-friends> foaf:knows 'Brad' };   
Done. -- 14 msec
SQL> SPARQL INSERT IN <http://example.com/Brad/friends> { <Brad-friends> foaf:knows 'Anna' };   
Done. -- 15 msec

-- Anna and Brad are friends and can read each others notes for friends.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Anna/friends', 'Anna', 3);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Anna/friends', 'Brad', 1);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Brad/friends', 'Brad', 3);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Brad/friends', 'Anna', 1);

-- BubbleSortingServicesInc graph 
SQL> SPARQL INSERT IN <http://example.com/BubbleSortingServicesInc> { <BubbleSortingServicesInc-info> a <info> };   
Done. -- 31 msec

-- Brad and Carl share write access to graph of his company.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/BubbleSortingServicesInc', 'Brad', 3);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/BubbleSortingServicesInc', 'Carl', 3);

-- Anna's blog
SQL> SPARQL INSERT IN <http://example.com/Anna/blog> { <Anna-blog> a <my-blog> };   

-- Anna writes a blog for public.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Anna/blog', 'Anna', 3);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Anna/blog', 'nobody', 1);

-- DBpedia is public read and local discussion wiki is readable and writable.
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://dbpedia.org/', 'nobody', 1);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/wiki', 'nobody', 3);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/public', 'nobody', 3);

-- Graph groups have its own security.
SQL> DB.DBA.RDF_GRAPH_GROUP_CREATE ('http://example.com/Personal', 1);
SQL> DB.DBA.RDF_GRAPH_GROUP_INS ('http://example.com/Personal', 'http://example.com/Anna/system');
SQL> DB.DBA.RDF_GRAPH_GROUP_INS ('http://example.com/Personal', 'http://example.com/Anna/private');
SQL> DB.DBA.RDF_GRAPH_GROUP_INS ('http://example.com/Personal', 'http://example.com/Brad/system');
SQL> DB.DBA.RDF_GRAPH_GROUP_INS ('http://example.com/Personal', 'http://example.com/Brad/private');
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Personal', 'Anna', 8);
SQL> DB.DBA.RDF_GRAPH_USER_PERMS_SET ('http://example.com/Personal', 'Brad', 8);

-- See as dba user what is in the <http://example.com/Personal> graph:
SQL> SPARQL 
SELECT *
FROM <http://example.com/Personal>
WHERE { ?s ?p ?o } ;
s               p                                                  o
VARCHAR         VARCHAR                                            VARCHAR
_______________________________________________________________________________

Anna-system     http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret
Anna-private    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret
Brad-system     http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret
Brad-private    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret

4 Rows. -- 32 msec.

-- See as Anna user what is in the <http://example.com/Personal> graph:
SQL> reconnect Anna;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> SPARQL 
SELECT *
FROM <http://example.com/Personal>
WHERE { ?s ?p ?o };
s               p                                                  o
VARCHAR         VARCHAR                                            VARCHAR
_______________________________________________________________________________

Anna-system     http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret
Anna-private    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret


-- See as Brad user what is in the <http://example.com/Personal> graph:
SQL> reconnect Brad;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> SPARQL SELECT *
FROM <http://example.com/Personal>
WHERE { ?s ?p ?o };
s               p                                                  o
VARCHAR         VARCHAR                                            VARCHAR
_______________________________________________________________________________

Brad-system     http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret
Brad-private    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret

-- See as Anna user what is in Brad's friends graph <http://example.com/Brad/friends>:
SQL> reconnect Anna;
Connected to OpenLink Virtuoso

SQL> SPARQL SELECT *
FROM <http://example.com/Brad/friends>
WHERE { ?s ?p ?o };
s              p                                 o
VARCHAR        VARCHAR                           VARCHAR
_________________________________________________________

Brad-friends   http://xmlns.com/foaf/0.1/knows   Anna
1 Rows. -- 0 msec.

-- Remove Anna's read permissions on Brad's notes:
SQL> reconnect dba;
SQL> RDF_GRAPH_USER_PERMS_DEL('http://example.com/Brad/friends','Anna');

-- See again as Anna user what is in Brad's friends graph <http://example.com/Brad/friends>:
SQL> reconnect Anna;
Connected to OpenLink Virtuoso

SQL> SPARQL SELECT *
FROM <http://example.com/Brad/friends>
WHERE { ?s ?p ?o };
s              p                                 o
VARCHAR        VARCHAR                           VARCHAR
_________________________________________________________

0 Rows. -- 0 msec.

SQL> SPARQL 
SELECT *
FROM <http://example.com/Anna/blog>
WHERE { ?s ?p ?o } ;

s             p                                                o
VARCHAR       VARCHAR                                          VARCHAR
_______________________________________________________________________________

Anna-blog     http://www.w3.org/1999/02/22-rdf-syntax-ns#type  my-blog

1 Rows. -- 16 msec.

SQL> SPARQL 
SELECT *
FROM <http://example.com/Anna/friends>
WHERE { ?s ?p ?o } ;

s             p                                o
VARCHAR       VARCHAR                          VARCHAR
_______________________________________________________________________________

Anna-friends  http://xmlns.com/foaf/0.1/knows  Brad
1 Rows. -- 16 msec.


-- Remove all the setting of Brad's permissions, both default permissions and 
-- permissions on specific graphs.
-- Note: 142 is example id of Brads U_ID in DB.DBA.SYS_USERS table

SQL> reconnect dba;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> DB.DBA.RDF_ALL_USER_PERMS_DEL('Brad',142);
Done. -- 0 msec.

SQL> reconnect Brad;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> SPARQL 
SELECT *
FROM <http://example.com/Anna/friends>
WHERE { ?s ?p ?o } ;
s          p          o
VARCHAR    VARCHAR    VARCHAR
_______________________________________________________________________________

0 Rows. -- 16 msec.


-- Check what Carl can see --
SQL> reconnect Carl;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> SPARQL 
SELECT *
FROM <http://example.com/BubbleSortingServicesInc>
WHERE { ?s ?p ?o } ;

s                              p                                                o
VARCHAR                        VARCHAR                                          VARCHAR
_______________________________________________________________________________

BubbleSortingServicesInc-info  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  info

1 Rows. -- 0 msec.


SQL> SPARQL 
SELECT *
FROM <http://example.com/Anna/private>
WHERE { ?s ?p ?o } ;
s               p                                                  o
VARCHAR         VARCHAR                                            VARCHAR
_______________________________________________________________________________

0 Rows. -- 16 msec.


-- let Carl read everything:
SQL> reconnect dba;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Carl', 1, 1);
Done. -- 0 msec

SQL> reconnect Carl;
Enter password for Carl :
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> SPARQL 
SELECT *
FROM <http://example.com/Anna/private>
WHERE { ?s ?p ?o } ;
s               p                                                  o
VARCHAR         VARCHAR                                            VARCHAR
_______________________________________________________________________________

Anna-private    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    secret

1 Rows. -- 16 msec.

-- Remove Carl's default permissions:
SQL> reconnect dba;
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> DB.DBA.RDF_DEFAULT_USER_PERMS_DEL('Carl', 1);
Done. -- 0 msec.

SQL> reconnect Carl;
Enter password for Carl :
Connected to OpenLink Virtuoso
Driver: 06.04.3132 OpenLink Virtuoso ODBC Driver

SQL> SPARQL 
SELECT *
FROM <http://example.com/Anna/private>
WHERE { ?s ?p ?o } ;
s               p                                                  o
VARCHAR         VARCHAR                                            VARCHAR
_______________________________________________________________________________

0 Rows. -- 16 msec.
]]></programlisting>
</sect3>
</sect2>
<sect2 id="rdfgraphsecurityappcallb"><title>Application Callbacks for Graph Level Security</title>
<para>In some cases, different applications should provide different security for different users. Two
SPARQL pragmas are provided for this purpose:</para>

<itemizedlist mark="bullet">
  <listitem>Pragma sql:gs-app-callback is to specify Virtuoso/PL callback function that return permission bits for given graph.</listitem>
  <listitem>Pragma sql:gs-app-uid is to specify application-specific user ID that is some string that is passed to the callback "as is".</listitem>
</itemizedlist>

<para>The name of callback is always DB.DBA.SPARQL_GS_APP_CALLBACK_nnn, where nnn is value of
sql:gs-app-callback.</para>

<para>The callback is called only if the application has access to the graph in question so it may restrict
the caller's account but not grant more permissions.</para>
<sect3 id="rdfgraphsecurityappcallbex"><title>Example</title>
<para>Let user of application get full access to graphs whose IRIs contain user's name in path.
In addition, let all of them permission to use all graph groups and let the "moderator" user read everything.</para>

<programlisting><![CDATA[
reconnect "dba";

create function DB.DBA.SPARQL_GS_APP_CALLBACK_TEST (in g_iid IRI_ID, in app_uid varchar) returns integer
{
  declare g_uri varchar;
-- A fake IRI ID #i0 is used to mention account's default permissions for all graphs.
  if (#i0 = g_iid)
    {
      if ('moderator' = app_uid)
        return 9; -- Moderator can read and list everything.
      return 8; -- Other users can list everything.
    }
  g_uri := id_to_iri (g_iid);
  if (strstr (g_uri, '/' || app_uid || '/'))
    return 15; -- User has full access to "his" graph.
  return 8; -- User can list any given graph group.
}
;

SPARQL
define sql:gs-app-callback "TEST"
define sql:gs-app-uid "Anna"
SELECT ?g ?s WHERE { ?s <p> ?o }
;
]]></programlisting>
</sect3>
</sect2>
<sect2 id="rdfgraphsecurityspongeprivate"><title>Graph-level security and sponging</title>
<para>In some cases the sponged data contains private information for instances cartridges like Facebook, etc. To
perform private sponging, Virtuoso offers <emphasis>get:private</emphasis> pragma:</para>
<programlisting><![CDATA[
define get:private ""
or
define get:private <graph_group_IRI>
]]></programlisting>
  <para>When used for sponging graph X, it adjusts graph-level security of graph X (and of graph_group_IRI, if
  specified) so that X becomes a privately accessible graph of the user who sponges the X and if graph_group_IRI
  is specified then X becomes accessible to users that can access graph_group_IRI with permissions like permissions
  they have on graph_group_IRI.</para>
  <para>The exact rules are as following:</para>
  <itemizedlist mark="bullet">
    <listitem>If graph is virtrdf: then an error is signaled.</listitem>
    <listitem>If graph name is an IRI of handshaked web service endpoint or "public IRI" of a handshaked web
    service endpoint then an error is signaled.</listitem>
    <listitem>If access is public by default even for private graphs then an error is signaled and sponging is
    not tried.</listitem>
    <listitem>If default is "no access" but someone (other than current user) has specifically granted read access
    to the graph in question AND current user is not dba AND current user has no bit 32 permission on this graph
    then an error is signaled.</listitem>
    <listitem>If read access is public by default for world and disabled for private graphs then the graph to be
    sponged is added to the group of private graphs.</listitem>
    <listitem>If current user is not DBA, current user gets granted read+write+sponge+admin access to the graph to
    be sponged. In addition, current user gets special permission bit 32, indicating that the graph is made by
    private sponge of this specific user.</listitem>
    <listitem>If the value of get:private is an IRI then:</listitem>
      <itemizedlist mark="bullet">
        <listitem>The IRI is supposed to be an IRI of "plain" graph group, error is signaled in case of nonexising
        graph group, group of private graphs or group of graphs to be replicated.</listitem>
        <listitem>The graph is added to that group</listitem>
        <listitem>Each non-dba user that can get list of files of the group will get permissions for the loaded
        graph equal to permissions they have on graph group minus "list" permission.</listitem>
      </itemizedlist>
  </itemizedlist>
  <sect3 id="rdfgraphsecurityspongeprivateconfdb"><title>Example Performing Sponging on a entirely confidential database using get:private pragma</title>
    <para>The following example demonstrates how private sponging using get:private pragma works for entirely
    confidential database.</para>
    <para><emphasis>Note</emphasis>: Please take in mind that the steps from below will change the security of any existing database, thus the example scenario should be performed on a empty db.</para>
    <orderedlist mark="bullet">
     <listitem>Create few users in alphabetical order:
<programlisting><![CDATA[
DB.DBA.USER_CREATE ('Anna', 'Anna');
DB.DBA.USER_CREATE ('Brad', 'Brad');
DB.DBA.USER_CREATE ('Carl', 'Carl');
]]></programlisting>
     </listitem>
     <listitem>Set to Anna, Brad and Carl SPARQL SELECT, UPDATE and SPONGE permissions:
<programlisting><![CDATA[
grant SPARQL_SELECT to "Anna";
grant SPARQL_SELECT to "Brad";
grant SPARQL_SELECT to "Carl";

grant SPARQL_UPDATE to "Anna";
grant SPARQL_UPDATE to "Brad";
grant SPARQL_UPDATE to "Carl";

grant SPARQL_SPONGE to "Anna";
grant SPARQL_SPONGE to "Brad";
grant SPARQL_SPONGE to "Carl";
]]></programlisting>
     </listitem>
     <listitem>Set specific privileges to given graphs for specifics users: Catering for the fact that some
     datasets are supposed to be confidential, thus the whole quad storage is set to confidential.
     Then specific privileges can be assigned to specific graphs for specific users:
<programlisting><![CDATA[
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('nobody', 0);
]]></programlisting>
     </listitem>
     <listitem>Set specific privileges: assuming for users Anna, Brad and Carl none of these individual has any
     kind of global access to graphs:
<programlisting><![CDATA[
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Anna', 0);
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Brad', 0);
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Carl', 0);
]]></programlisting>
</listitem>
     <listitem>Assuming the following four sorts of access that are specified by four bits of an integer
     "permission bit-mask", following plain old UNIX style:
     <itemizedlist mark="bullet">
        <listitem>Bit 1 permits read access. </listitem>
        <listitem>Bit 2 permits write access via SPARUL and is basically useless without bit 1 set. </listitem>
        <listitem>Bit 4 permits write access via "RDF Network Resource Fetch" methods and is basically useless
        without bits 1 and 2 set. </listitem>
        <listitem>Bit 8 allows retrieval of the list of members of a graph group. An IRI can be used as a graph
        IRI and as a graph group IRI at the same time, so bit 8 can be freely combined with any of bits 1, 2 or
        4. </listitem>
        <listitem>In the statements from below should be considered:
          <itemizedlist mark="bullet">
            <listitem>"15 = 8+4+2+1" -- i.e. combining all the four sorts of access FROM above </listitem>
            <listitem>"9 = 8 + 1" -- i.e. read access + access to retrieve the list of members for a given
            graph group</listitem>
</itemizedlist>
        </listitem>
      </itemizedlist>
<programlisting><![CDATA[
-- Create Graph Group for Anna and set privileges:
DB.DBA.RDF_GRAPH_GROUP_CREATE ('urn:Anna:Sponged:Data', 1);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Anna:Sponged:Data', 'Anna', 15);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Anna:Sponged:Data', 'Brad', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Anna:Sponged:Data', 'Carl', 9);

-- Create Graph Group for Brad and set privileges:
DB.DBA.RDF_GRAPH_GROUP_CREATE ('urn:Brad:Sponged:Data', 1);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Brad:Sponged:Data', 'Anna', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Brad:Sponged:Data', 'Brad', 15);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Brad:Sponged:Data', 'Carl', 9);

-- Create Graph Group for Carl and set privileges:
DB.DBA.RDF_GRAPH_GROUP_CREATE ('urn:Carl:Sponged:Data', 1);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Carl:Sponged:Data', 'Anna', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Carl:Sponged:Data', 'Brad', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Carl:Sponged:Data', 'Carl', 15);
]]></programlisting>
     </listitem>
     <listitem>Examples with invalid graph group names:
       <orderedlist>
         <listitem>Example with Non-existing Graph Group:
       <programlisting><![CDATA[
-- An error for non-existing Graph group <http://nosuch/> will be raised.

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <http://nosuch/>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
       ]]></programlisting>
       </listitem>
       <listitem>Example with "virtrdf:PrivateGraphs" graph group which is reserved for system usage:
       <programlisting><![CDATA[
-- An error for attempt to add a graph to special
-- graph group <http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs> will be raised.

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private virtrdf:PrivateGraphs
  SELECT * FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
       ]]></programlisting>
         </listitem>
         <listitem>Example with "virtrdf:rdf_repl_graph_group" graph group which is reserved for system usage:
<programlisting><![CDATA[

-- An error for attempt to add a graph to special
-- graph group <http://www.openlinksw.com/schemas/virtrdf#rdf_repl_graph_group> will be raised.

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private virtrdf:rdf_repl_graph_group
  SELECT * FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
       </orderedlist>
     </listitem>
     <listitem>Examples to check Anna's sponging permissions on different graph groups:
       <orderedlist>
         <listitem>Example for adding graph to Anna's graph group &lt;urn:Anna:Sponged:Data&gt;:
<programlisting><![CDATA[

-- No error will be raised as Anna has the efficient rights for graph group <urn:Anna:Sponged:Data>

reconnect "Anna";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Anna:Sponged:Data>
  SELECT *
  FROM <http://example.com/anna/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Brad's graph group &lt;urn:Brad:Sponged:Data&gt;:
<programlisting><![CDATA[

-- An error will be raised because "Anna" has not enough rights on that group

reconnect "Anna";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Brad:Sponged:Data>
  SELECT * FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Carl's graph group &lt;urn:Carl:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Anna" has not enough rights on that group

reconnect "Anna";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
       </listitem>
       </orderedlist>
     </listitem>
     <listitem>Examples check Brad's sponging permissions on different graph groups:
       <orderedlist>
         <listitem>Example for adding graph to Anna's graph group &lt;urn:Anna:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Brad" has not enough rights on that group

reconnect "Brad";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Anna:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Brad's graph group &lt;urn:Brad:Sponged:Data&gt;:
<programlisting><![CDATA[
-- No error will be raised as Brad has the efficient rights for graph group <urn:Brad:Sponged:Data>

reconnect "Brad";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Brad:Sponged:Data>
  SELECT *
  FROM <http://example.com/brad/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Carl's graph group &lt;urn:Carl:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Brad" has not enough rights on that group

reconnect "Brad";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
       </listitem>
       </orderedlist>
     </listitem>
     <listitem>Examples check Carl's sponging permissions on different graph groups:
       <orderedlist>
         <listitem>Example for adding graph to Anna's graph group &lt;urn:Anna:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Carl" has not enough rights on that group

reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Anna:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Brad's graph group &lt;urn:Brad:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be rased because "Carl" has not enough rights on that group

reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Brad:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Carl's graph group &lt;urn:Carl:Sponged:Data&gt;:
<programlisting><![CDATA[
-- No error will be raised as Carl has the efficient rights for graph group <urn:Brad:Sponged:Data>

reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://example.com/carl/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
       </listitem>
       </orderedlist>
     </listitem>
     <listitem>User Carl performs private sponging:
<programlisting><![CDATA[
reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://www.openlinksw.com/data/turtle/products.ttl>
  WHERE
    { ?s ?p ?o };

-- Should return for ex. 365 rows.
SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT COUNT(*)
  FROM <http://www.openlinksw.com/data/turtle/products.ttl>
  WHERE
    { ?s ?p ?o };


SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM NAMED <http://www.openlinksw.com/data/turtle/software.ttl>
  FROM NAMED <http://www.openlinksw.com/data/turtle/licenses.ttl>
  WHERE
    {
      graph ?g
        { ?s ?p ?o
      }
    };


-- Should return for ex. 1317 rows.
SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT COUNT(*)
  FROM NAMED <http://www.openlinksw.com/data/turtle/software.ttl>
  FROM NAMED <http://www.openlinksw.com/data/turtle/licenses.ttl>
  WHERE
    {
      graph ?g
        { ?s ?p ?o
      }
    };
]]></programlisting>
     </listitem>
     <listitem>Viewing Graph Groups shows Carl's graph group &lt;urn:Carl:Sponged:Data&gt; contains total 4 graphs:
<programlisting><![CDATA[
SQL> SELECT id_to_iri (RGGM_GROUP_IID), id_to_iri(RGGM_MEMBER_IID)
     FROM DB.DBA.RDF_GRAPH_GROUP_MEMBER
     ORDER BY 1,2;


id_to_iri                id_to_iri__1
VARCHAR                  VARCHAR
__________________________________________________________

....
urn:Anna:Sponged:Data    http://example.com/anna/
urn:Brad:Sponged:Data    http://example.com/brad/
urn:Carl:Sponged:Data    http://example.com/carl/
urn:Carl:Sponged:Data    http://www.openlinksw.com/data/turtle/licenses.ttl
urn:Carl:Sponged:Data    http://www.openlinksw.com/data/turtle/products.ttl
urn:Carl:Sponged:Data    http://www.openlinksw.com/data/turtle/software.ttl
]]></programlisting>
     </listitem>
    </orderedlist>
  </sect3>
  <sect3 id="rdfgraphsecurityspongeprivategraphs"><title>Example Performing Sponging with Private Graphs Using get:private pragma</title>
    <para>The following example demonstrates how private sponging using get:private pragma works for database
    with private graphs.</para>
    <orderedlist mark="bullet">
     <listitem>Create few users in alphabetical order:
<programlisting><![CDATA[
DB.DBA.USER_CREATE ('Anna', 'Anna');
DB.DBA.USER_CREATE ('Brad', 'Brad');
DB.DBA.USER_CREATE ('Carl', 'Carl');
]]></programlisting>
     </listitem>
     <listitem>Set to Anna, Brad and Carl SPARQL SELECT, UPDATE and SPONGE permissions:
<programlisting><![CDATA[
grant SPARQL_SELECT to "Anna";
grant SPARQL_SELECT to "Brad";
grant SPARQL_SELECT to "Carl";

grant SPARQL_UPDATE to "Anna";
grant SPARQL_UPDATE to "Brad";
grant SPARQL_UPDATE to "Carl";

grant SPARQL_SPONGE to "Anna";
grant SPARQL_SPONGE to "Brad";
grant SPARQL_SPONGE to "Carl";
]]></programlisting>
     </listitem>
     <listitem>Set specific privileges: Setup assuming 3 users: Anna, Brad and Carl where each of these
     individual users has read access to graphs:
<programlisting><![CDATA[
-- Close any public access to "private" graphs
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('nobody', 0, 1);

-- Set Read Only for public on graphs not listed as "private".
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('nobody', 1);

DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Anna', 0, 1);
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Brad', 0, 1);
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Carl', 0, 1);

DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Anna', 1);
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Brad', 1);
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('Carl', 1);
]]></programlisting>
     </listitem>
     <listitem>Assuming the following four sorts of access that are specified by four bits of an integer
     "permission bit-mask", following plain old UNIX style:
     <itemizedlist mark="bullet">
        <listitem>Bit 1 permits read access. </listitem>
        <listitem>Bit 2 permits write access via SPARUL and is basically useless without bit 1 set. </listitem>
        <listitem>Bit 4 permits write access via "RDF Network Resource Fetch" methods and is basically useless
        without bits 1 and 2 set. </listitem>
        <listitem>Bit 8 allows retrieval of the list of members of a graph group. An IRI can be used as a graph
        IRI and as a graph group IRI at the same time, so bit 8 can be freely combined with any of bits 1, 2 or
        4. </listitem>
        <listitem>In the statements from below should be considered:
          <itemizedlist mark="bullet">
            <listitem>"15 = 8+4+2+1" -- i.e. combining all the four sorts of access FROM above </listitem>
            <listitem>"9 = 8 + 1" -- i.e. read access + access to retrieve the list of members for a given
            graph group</listitem>
          </itemizedlist>
        </listitem>
      </itemizedlist>
<programlisting><![CDATA[
-- Create Graph Group for Anna and set privileges:
DB.DBA.RDF_GRAPH_GROUP_CREATE ('urn:Anna:Sponged:Data', 1);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Anna:Sponged:Data', 'Anna', 15);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Anna:Sponged:Data', 'Brad', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Anna:Sponged:Data', 'Carl', 9);

-- Create Graph Group for Brad and set privileges:
DB.DBA.RDF_GRAPH_GROUP_CREATE ('urn:Brad:Sponged:Data', 1);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Brad:Sponged:Data', 'Anna', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Brad:Sponged:Data', 'Brad', 15);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Brad:Sponged:Data', 'Carl', 9);

-- Create Graph Group for Carl and set privileges:
DB.DBA.RDF_GRAPH_GROUP_CREATE ('urn:Carl:Sponged:Data', 1);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Carl:Sponged:Data', 'Anna', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Carl:Sponged:Data', 'Brad', 9);
DB.DBA.RDF_GRAPH_USER_PERMS_SET ('urn:Carl:Sponged:Data', 'Carl', 15);


-- Set Anna's, Brad's and Carl's graphs by inserting them into the <b>virtrdf:PrivateGraphs</b> graph group:
DB.DBA.RDF_GRAPH_GROUP_INS ('http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs', 'http://example.com/anna/');
DB.DBA.RDF_GRAPH_GROUP_INS ('http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs', 'http://example.com/brad/');
DB.DBA.RDF_GRAPH_GROUP_INS ('http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs', 'http://example.com/carl/');
]]></programlisting>
     </listitem>
     <listitem>Examples with invalid graph group names:
       <orderedlist>
         <listitem>Example with Non-existing Graph Group:
       <programlisting><![CDATA[
-- An error for non-existing Graph group <http://nosuch/> will be raised.

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <http://nosuch/>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
       ]]></programlisting>
       </listitem>
       <listitem>Example with "virtrdf:PrivateGraphs" graph group which is reserved for system usage:
       <programlisting><![CDATA[
-- An error for attempt to add a graph to special
-- graph group <http://www.openlinksw.com/schemas/virtrdf#PrivateGraphs> will be raised.

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private virtrdf:PrivateGraphs
  SELECT * FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
       ]]></programlisting>
         </listitem>
         <listitem>Example with "virtrdf:rdf_repl_graph_group" graph group which is reserved for system usage:
<programlisting><![CDATA[

-- An error for attempt to add a graph to special
-- graph group <http://www.openlinksw.com/schemas/virtrdf#rdf_repl_graph_group> will be raised.

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private virtrdf:rdf_repl_graph_group
  SELECT * FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
       </orderedlist>
     </listitem>
     <listitem>Examples to check Anna's sponging permissions on different graph groups:
       <orderedlist>
         <listitem>Example for adding graph to Anna's graph group &lt;urn:Anna:Sponged:Data&gt;:
<programlisting><![CDATA[

-- No error will be raised as Anna has the efficient rights for graph group <urn:Anna:Sponged:Data>

reconnect "Anna";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Anna:Sponged:Data>
  SELECT *
  FROM <http://example.com/anna/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Brad's graph group &lt;urn:Brad:Sponged:Data&gt;:
<programlisting><![CDATA[

-- An error will be raised because "Anna" has not enough rights on that group

reconnect "Anna";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Brad:Sponged:Data>
  SELECT * FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Carl's graph group &lt;urn:Carl:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Anna" has not enough rights on that group

reconnect "Anna";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
       </listitem>
       </orderedlist>
     </listitem>
     <listitem>Examples check Brad's sponging permissions on different graph groups:
       <orderedlist>
         <listitem>Example for adding graph to Anna's graph group &lt;urn:Anna:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Brad" has not enough rights on that group

reconnect "Brad";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Anna:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Brad's graph group &lt;urn:Brad:Sponged:Data&gt;:
<programlisting><![CDATA[
-- No error will be raised as Brad has the efficient rights for graph group <urn:Brad:Sponged:Data>

reconnect "Brad";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Brad:Sponged:Data>
  SELECT *
  FROM <http://example.com/brad/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Carl's graph group &lt;urn:Carl:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Brad" has not enough rights on that group

reconnect "Brad";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
       </listitem>
       </orderedlist>
     </listitem>
     <listitem>Examples check Carl's sponging permissions on different graph groups:
       <orderedlist>
         <listitem>Example for adding graph to Anna's graph group &lt;urn:Anna:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be raised because "Carl" has not enough rights on that group

reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Anna:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Brad's graph group &lt;urn:Brad:Sponged:Data&gt;:
<programlisting><![CDATA[
-- An error will be rased because "Carl" has not enough rights on that group

reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Brad:Sponged:Data>
  SELECT *
  FROM <http://example.com/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
         </listitem>
         <listitem>Example for adding graph to Carl's graph group &lt;urn:Carl:Sponged:Data&gt;:
<programlisting><![CDATA[
-- No error will be raised as Carl has the efficient rights for graph group <urn:Brad:Sponged:Data>

reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://example.com/carl/>
  WHERE
    { ?s ?p ?o };
]]></programlisting>
       </listitem>
       </orderedlist>
     </listitem>
     <listitem>User Carl performs private sponging:
<programlisting><![CDATA[
reconnect "Carl";

SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM <http://www.openlinksw.com/data/turtle/products.ttl>
  WHERE
    { ?s ?p ?o };

-- Should return for ex. 365 rows.
SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT COUNT(*)
  FROM <http://www.openlinksw.com/data/turtle/products.ttl>
  WHERE
    { ?s ?p ?o };


SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT *
  FROM NAMED <http://www.openlinksw.com/data/turtle/software.ttl>
  FROM NAMED <http://www.openlinksw.com/data/turtle/licenses.ttl>
  WHERE
    {
      graph ?g
        { ?s ?p ?o
      }
    };


-- Should return for ex. 1317 rows.
SPARQL
  DEFINE get:soft "replacing"
  DEFINE get:private <urn:Carl:Sponged:Data>
  SELECT COUNT(*)
  FROM NAMED <http://www.openlinksw.com/data/turtle/software.ttl>
  FROM NAMED <http://www.openlinksw.com/data/turtle/licenses.ttl>
  WHERE
    {
      graph ?g
        { ?s ?p ?o
      }
    };
]]></programlisting>
     </listitem>
     <listitem>User Anna reads Carl's data:
<programlisting><![CDATA[
SQL> reconnect "Anna";
SQL> SPARQL
  SELECT COUNT(*)
  FROM <http://www.openlinksw.com/data/turtle/products.ttl>
  WHERE
    { ?s ?p ?o };


callret-0
INTEGER
_______________________________________________________________________________

365

1 Rows. -- 15 msec.

]]></programlisting>
     </listitem>
    </orderedlist>
  </sect3>
</sect2>
</sect1>

<sect1 id="rdfviewsrdbms">
	<title>Linked Data Views over RDBMS Data Source</title>
<para>
Linked Data Views map relational data into RDF and allow customizing RDF representation of locally stored RDF data.
To let SPARQL clients access relational data as well as physical RDF graphs in a single query, we introduce a declarative Meta Schema Language for mapping SQL Data to RDF Ontologies.
As a result, all types of clients can efficiently access all data stored on the server.
The mapping functionality dynamically generates RDF Data Sets for popular ontologies such as SIOC, SKOS, FOAF, and ATOM/OWL without disruption to the existing database infrastructure of Web 1.0 or Web 2.0 solutions.
Linked Data Views are also suitable for declaring custom representation for RDF triples, e.g. property tables, where one row holds many single-valued properties.
</para>
<sect2 id="rdfviewsintro"><title>Introduction</title>
<para>
The Virtuoso Linked Data Views meta schema is a built-in feature of Virtuoso's SPARQL to SQL translator.
It recognizes triple patterns that refer to graphs for which an alternate representation is declared and translates these into SQL accordingly.
The main purpose of this is evaluating SPARQL queries against existing relational databases.
There exists previous work from many parties for rendering relational data as RDF and opening it to SPARQL access.
We can mention D2RQ, SPASQL, Squirrel RDF, DBLP and others.
The Virtuoso effort differs from these mainly in the following:
</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>Integration with a triple store.
Virtuoso can process a query for which some triple patterns will go to local or remote relational data and some to local physical RDF triples.
</listitem><listitem>SPARQL query can be used in any place where SQL can.
Database connectivity protocols are neutral to the syntax of queries they transmit, thus any SQL client, e.g. JDBC, ODBC or XMLA application, can send SPARQL queries and fetch result sets.
Moreover, a SQL query may contain SPARQL subqueries and SPARQL expressions may use SQL built-in functions and stored procedures.
</listitem><listitem>Integration with SQL.
Since SPARQL and SQL share the same run time and query optimizer, the query compilation decisions are always made with the best knowledge of the data and its location.
This is especially important when mixing triples and relational data or when dealing with relational data distributed across many outside databases.
</listitem><listitem>No limits on SPARQL.
It remains possible to make queries with unspecified graph or predicate against mapped relational data, even though these may sometimes be inefficient.
</listitem><listitem>Coverage of the whole relational model.
Multi-part keys etc. are supported in all places.
</listitem>
</itemizedlist>
</sect2>
<sect2 id="rdfviewrationale"><title>Rationale</title>
<para>
Since most of the data that is of likely use for the emerging semantic web is stored in relational databases, the argument for exposing this to SPARQL access is clear.
We note that historically, SQL access to relational data has essentially never been given to the public outside of the organization.
If programmatic access to corporate IS has been available to partners or the public, it has been through dynamic web pages or more recently web services.
There are reasons of performance, security, maintainability and so forth for this.
</para><para>
The culture of the emerging semantic web is however taking a different turn.
Since RDF and OWL offer a mergeable and queryable model for heterogeneous data, it is more meaningful and maintainable to expose selected data for outside query than it would be with SQL.
Advances in hardware make this also less of a performance issue than it would have been in the client-server database era.
</para><para>
In the context of Virtuoso, since Virtuoso is originally a virtual/federated database, incorporating SPARQL to relational mapping is an evident extension of the product's mission as a multi-protocol, multi-platform connector between information systems.
</para>
</sect2>
<sect2 id="rdfviewquadmapatternsvalueandiriclasses"><title>Quad Map Patterns, Values and IRI Classes</title>
<para>
In the simplest sense, any relational schema can be rendered into RDF by converting all primary keys and foreign keys into IRI's, assigning a predicate IRI to each column, and an rdf:type predicate for each row linking it to a RDF class IRI corresponding to the table.
Then a triple with the primary key IRI as subject, the column IRI as predicate and the column's value as object is considered to exist for each column that is neither part of a primary or foreign key.
</para><para>
Strictly equating a subject value to a row and each column to a predicate is often good but is too restrictive for the general case.
</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>Multiple triples with the same subject and predicate can exist.
</listitem><listitem>A single subject can get single-valued properties from multiple tables or in some cases stored procedures.
</listitem><listitem>An IRI value of a subject or other field of a triple can be composed from more than one SQL value, these values may reside in different columns, maybe in different joined tables.
</listitem><listitem>Some table rows should be excluded from mapping.
</listitem></itemizedlist>
<para>
Thus in the most common case the RDF meta schema should consist of independent transformations; the domain of each transformation is a result-set of some SQL <emphasis>SELECT</emphasis> statement and range is a set of triples.
The <emphasis>SELECT</emphasis> that produce the domain is quite simple: it does not use aggregate functions, joins and sorting, only inner joins and <emphasis>WHERE</emphasis> conditions.
There is no need to support outer joins in the RDF meta schema because NULLs are usually bad inputs for functions that produce IRIs.
In the rare cases when NULLs are OK for functions, outer joins can be encapsulated in SQL views.
The range of mapping can be described by a SPARQL triple pattern: a pattern field is a variable if it depends on table columns, otherwise it is a constant.
Values of variables in the pattern may have additional restrictions on datatypes, when datatypes of columns are known.
</para><para>
This common case of an RDF meta schema is implemented in Virtuoso, with one adjustment.
Virtuoso stores quads, not triples, using the graph field (G) to indicate that a triple belongs to some particular application or resource.
A SPARQL query may use quads from different graphs without large difference between G and the other three fields of a quad.
E.g., variable <emphasis>?g</emphasis> in expression <emphasis>GRAPH ?g {...}</emphasis> can be unbound.
SPARQL has special syntax for &quot;graph group patterns&quot; that is convenient for sets of triple patterns with a common graph, but it also has shorthands for common subject and predicate, so the difference is no more than in syntax.
There is only one feature that is specific for graphs but not for other fields: the SPARQL compiler can create restrictions on graphs according to <emphasis>FROM</emphasis> and <emphasis>FROM NAMED</emphasis> clauses.
</para><para>
Virtuoso Linked Data Views should offer the same flexibility with the graphs as SPARQL addressing physical triples.
A transformation cannot always be identified by the graph used for ranges because graph may be composed from SQL data. The key element of the meta schema is a &quot;<emphasis>quad map pattern</emphasis>&quot;.
A simple quad map pattern fully defines one particular transformation from one set of relational columns into triples that match one SPARQL graph pattern.
The main part of quad map pattern is four declarations of &quot;<emphasis>quad map values</emphasis>&quot;, each declaration specifies how to calculate the value of the corresponding triple field from the SQL data.
The pattern also lists boolean SQL expressions that should be used to filter out unwanted rows of source data (and to join multiple tables if source columns belong to different tables).
There are also quad map patterns that group together similar quad patterns but do not specify any real transformation or even prevent unwanted transformations from being used, they are described in &quot;Grouping Map Patterns&quot; below.
</para><para>
Quad map values refer to schema elements of two further types: &quot;IRI classes&quot; and &quot;literal classes&quot;.
</para>
<note><para>In SQL, adding a new view can not break anything. This is because SQL lacks the ability of querying &quot;everything&quot; so data sources are always specified. This is not true for SPARQL, so please treat <emphasis>any</emphasis> metadata manipulation as potentially destructive operation. If an RDF storage is supposed to be used by more than one application then these applications should be tested together, not one after other, and they should be installed/upgraded on live database in the very same order as they were installed/upgraded on instrumental machine during testing. Always remember that these applications share RDF tables so they may interfere.</para></note>
<sect3 id="rdfviewiriclasses"><title>IRI Classes</title>
<para>
An IRI class declares that a column or set of columns gets converted into a IRI in a certain way.
The conversion of this sort can be declared revertible (bijection) so an IRI can be parsed into original SQL values; this is useful when some equality of an IRI constant and a calculated IRI can be replaced with an equality of a parse result of a constant and an SQL column that is index criteria or simply faster.
In addition, the SPARQL optimizer will eliminate redundant conversions if one IRI class is explicitly declared as a subclass of another.
The most flexible declaration for conversion consists of specifying functions that assemble and disassemble from IRI into its constituent parts.
This is overkill for typical conversions so it is possible to specify only one sprintf-style format string such that <emphasis>sprintf()</emphasis> SQL function will print an IRI using this format and <emphasis>sprintf_inverse()</emphasis> will be able to parse it back.
</para><para>The use of <emphasis>sprintf_inverse()</emphasis> assumes that the format does not contain fragments like <emphasis>'%s%s'</emphasis> that make it impossible to separate parts of IRI from each other.
</para><para>
In the following, we shall map the Virtuoso users and user roles system tables into the SIOC ontology.
</para>
<programlisting><![CDATA[
create iri class oplsioc:user_iri "http://myhost/sys/user?id=%d"
  (in uid integer not null) .
create iri class oplsioc:group_iri "http://myhost/sys/group?id=%d"
  (in gid integer not null) .
create iri class oplsioc:membership_iri
  "http://myhost/sys/membership?super=%d&sub=%d"
  (in super integer not null, in sub integer not null) .
create iri class oplsioc:dav_iri "http://myhost%s"
  (in path varchar) .
]]></programlisting>
<para>
These IRI classes are used for mapping data from the <emphasis>DB.DBA.SYS_USERS</emphasis> and <emphasis>DB.DBA.SYS_ROLE_GRANTS</emphasis> system tables that are defined in Virtuoso as follows:
</para>
<programlisting><![CDATA[
create table DB.DBA.SYS_USERS (
  U_ID                integer not null unique,
  U_NAME              char (128) not null primary key,
  U_IS_ROLE           integer default 0,
  U_FULL_NAME         char (128),
  U_E_MAIL            char (128) default &quot;,
  U_ACCOUNT_DISABLED  integer default 1,
  U_DAV_ENABLE        integer default 0,
  U_SQL_ENABLE        integer default 1,
  U_HOME              varchar (128),
. . .
 );
]]></programlisting>
<para>
Single record in <emphasis>DB.DBA.SYS_USERS</emphasis> corresponds to a plain user or a group (role).
Users and roles are collectively named &quot;grantees&quot;. Thus a role may be granted to another role or to a user account.
A role grant may be direct (explicit) or assigned by recursion.
</para>
<programlisting><![CDATA[
create table SYS_ROLE_GRANTS (
  GI_SUPER   integer,
  GI_SUB     integer,
  GI_DIRECT  integer default 1,
. . .
  primary key (GI_SUPER, GI_SUB, GI_DIRECT));
]]></programlisting>
<para>One IRI class usually corresponds to one ontology class, because similar things are usually called similarly.
One may wish to use identifiers of ontology classes as identifiers of related IRI classes, to not remember double number of names, e.g. <emphasis>create IRI class mybank:XpressXfer</emphasis> for subjects that will have <emphasis>rdf:type</emphasis> property <emphasis>mybank:XpressXfer</emphasis> made by mapping. That is technically possible but proven to become inconvenient and misleading as application evolves. While RDF types tend to persist, IRI classes may change over time or same subject may get more than one name via more than one IRI class, say, for exports to different systems. It is found to be more convenient to compose names of IRI classes by adding some common prefixes or suffixes to RDF classes (or to table names), say, write <emphasis>create IRI class mybank:XpressXfer_iri</emphasis>.</para>
</sect3>
<sect3 id="rdfviewliteralclasses"><title>Literal Classes</title>
<para>
A &quot;literal class&quot; declares that a column or set of columns gets converted into a literal instead of an IRI.
More precisely, the result of conversion can be <emphasis>IRI_ID</emphasis> so it represents an IRI, but in current version of Virtuoso this is supported only for some internal built-in literal classes, not for classes declared by the user.
So for user-defined literal class the result of the conversion is an RDF literal even if it is a string representation of a valid IRI.
</para><para>
In any case, a literal class can be used only in quad map values of O fields, because Virtuoso does not support literal values as subjects.
</para><para>
A special case of literal class is the identity class that converts a value from <emphasis>varchar</emphasis> column into an untyped literal and value from column of any other SQL datatype into a typed literal with type from XMLSchema set, i.e. <emphasis>xsd:integer</emphasis>, <emphasis>xsd:dateTime</emphasis> and so on.
Columns of types <emphasis>ANY</emphasis> and <emphasis>IRI_ID</emphasis> are not supported.
</para><para>
The SPARQL optimizer knows that RDF literal types are pairwise disjoint so literal classes that produce literals of different types are known to be pairwise disjoint.
The optimizer will replace a join on two disjoint literal classes with an empty statement, to simplify the resulting query.
</para>
</sect3>
<sect3 id="rdfviewsimplequadmappatterns"><title>Simple Quad Map Patterns</title>
<para>
The following declaration of quad map pattern is self-explanatory. The line for <emphasis>object</emphasis> uses identity literal class so there's no need to specify its name.
</para>
<programlisting><![CDATA[
graph      <http://myhost/sys>
subject    oplsioc:user_iri (DB.DBA.SYS_USERS.U_ID)
predicate  foaf:email
object     DB.DBA.SYS_USERS.U_E_MAIL
]]></programlisting>
<para>
The description language also supports SPARQL-style notation that contains less keywords and eliminates duplicate graphs, subjects and predicates.
The following add two patterns with constant graph IRI <emphasis>&lt;http://myhost/sys&gt;</emphasis> and subjects are made from column <emphasis>DB.DBA.SYS_USERS.U_ID</emphasis> by <emphasis>oplsioc:user_iri</emphasis>.
</para>
<programlisting><![CDATA[
graph <http://myhost/sys>
  {
    oplsioc:user_iri (DB.DBA.SYS_USERS.U_ID)
      a sioc:user ;
      oplsioc:name DB.DBA.SYS_USERS.U_FULL_NAME .
  }
]]></programlisting>
</sect3>
<sect3 id="rdfviewassigningnamestoquadmappatterns"><title>Assigning Names To Quad Map Patterns</title>
<para>
In real applications, quad map patterns should be named, for schema manipulation and keeping debug info readable.
Thus it is much better to rewrite the previous example as
</para>
<programlisting><![CDATA[
create virtrdf:SysUsers as graph <http://myhost/sys>
  {
    oplsioc:user_iri (DB.DBA.SYS_USERS.U_ID)
      a sioc:user
          as virtrdf:SysUserType-User;
      oplsioc:name DB.DBA.SYS_USERS.U_FULL_NAME
          as virtrdf:SysUsersFullName .
  }
]]></programlisting>
<para>
Using these names, one may later write, say, <emphasis>drop quad map virtrdf:SysUserType-User</emphasis>.
</para><para>
One name, <emphasis>virtrdf:DefaultQuadMap</emphasis> is reserved.
It is an internal quad map pattern used to access &quot;native-form&quot; quads from <emphasis>DB.DBA.RDF_QUAD</emphasis>:
</para>
<programlisting><![CDATA[
create virtrdf:DefaultQuadMap as
graph rdfdf:default-iid-nonblank (DB.DBA.RDF_QUAD.G)
subject rdfdf:default-iid (DB.DBA.RDF_QUAD.S)
predicate rdfdf:default-iid-nonblank (DB.DBA.RDF_QUAD.P)
object rdfdf:default (DB.DBA.RDF_QUAD.O)
]]></programlisting>
<para>
IRI classes from <emphasis>rdfdf:...</emphasis> namespace are also reserved.
</para>
</sect3>
<sect3 id="rdfviewgroupingmappatterns"><title>Grouping Map Patterns</title>
<para>
The previous example actually contains three map patterns, not two.
The name <emphasis>virtrdf:SysUsers</emphasis> refers to a &quot;<emphasis>group map pattern</emphasis>&quot; that does not define any real transformation of relational data into RDF but helps organize quad map patterns into a tree.
Group may contain both quad map patterns and other groups.
A group can be manipulated as a whole, e.g. <emphasis>drop quad map virtrdf:SysUsers</emphasis> will remove all three map patterns.
</para>
</sect3>
</sect2>

<sect2 id="rdfviewconfiguringrdfstorages"><title>Configuring RDF Storages</title>
<para>
&quot;<emphasis>Quad Storage</emphasis>&quot; is a named set of quad map patterns.
The declaration <emphasis>define input:storage storage-name</emphasis> states that a SPARQL query will be executed using only quad patterns of the given quad storage.
Declarations of IRI classes, literal classes and quad patterns are shared between all quad storages of an RDF meta schema but every quad storage contains only a subset of all available quad patterns.
Two quad storages are always defined:
</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>A <emphasis>virtrdf:default</emphasis> one usually consists of everything (all user-relational mappings plus <emphasis>virtrdf:DefaultQuadMap</emphasis> for &quot;native-form&quot; quads from <emphasis>DB.DBA.RDF_QUAD</emphasis>)
</listitem><listitem>A <emphasis>virtrdf:empty</emphasis> storage refers solely to <emphasis>DB.DBA.RDF_QUAD</emphasis> and can not be altered.
</listitem></itemizedlist>
<para>
Three statements for manipulating storages are
</para>
<itemizedlist mark="number" spacing="compact">
<listitem><emphasis>create quad storage storage-name { quad-map-decls } .</emphasis>
</listitem><listitem><emphasis>alter quad storage storage-name { quad-map-decls-or-drops } .</emphasis>
</listitem><listitem><emphasis>drop quad storage storage-name . </emphasis>
</listitem></itemizedlist>
<para>
A map pattern can be created only as a part of <emphasis>create quad storage</emphasis> or <emphasis>alter quad storage</emphasis> statement, so initially it is used by exactly one storage.
It can be imported to some other storage using directive <emphasis>create map-id using storage source-storage</emphasis>. E.g., declarations of many storages create <emphasis>virtrdf:DefaultQuadMap</emphasis> using storage <emphasis>virtrdf:DefaultQuadStorage</emphasis>.
</para><para>
Only a &quot;top-level&quot; quad map pattern (standalone or a whole group with descendants) can be imported, member of a group can not.
The import directive also can not be a part of some group declaration.
</para><para>
The directive <emphasis>drop quad map map-name</emphasis> removes a map from one storage when it appears inside <emphasis>alter quad storage</emphasis> statement.
Otherwise it removes the map from all storages.
There exists garbage collection for quad map patterns, so any unused map is immediately deleted.
A group is deleted with all its descendants.
</para>
</sect2>
<sect2 id="rdfviewtranslationofpatterns"><title>Translation Of SPARQL Triple Patterns To Quad Map Patterns</title>
<para>
When a SPARQL query is compiled into SQL using a quad storage, every triple pattern should become a subquery that retrieves data from relational tables.
This subquery is an <emphasis>UNION ALL</emphasis> of joins generated from appropriate quad map patterns.
The complete SQL query is composed from these basic subqueries.
Thus the first operation of the SQL generation for a triple pattern is searching for quad map patterns that may in principle produce triples that match the triple pattern.
</para><para>
The more restrictions contained in the triple pattern the fewer quad map patterns will be used.
A triple pattern <emphasis>graph ?g { ?s ?p ?o }</emphasis> is common enough to invoke all data transformations of the storage.
A triple pattern <emphasis>graph &lt;g&gt; { ?s &lt;p&gt; &lt;o&gt; }</emphasis> will usually intersect with the range of only one quad map.
Sometimes it is possible to prove that the storage can not contain any data that matches the given triple pattern, hence zero number of members of <emphasis>UNION ALL</emphasis> will result in constantly empty result-set.
</para>
<para>The search for quad maps for a given pair of triple pattern and quad map storage is quite simple.
The storage is treated as a tree of map patterns where quad map patterns are leafs, grouping patterns are inner nodes and the whole storage is also treated as a grouping pattern that specify no fields and contains all top-level map patterns of the storage.
</para>
<para>
The tree is traversed from the root, left to right, non-leaf vertex are checked before their children.
The check of a vertex consists of up to four field checks, for G, S, P and O.
Every field check compares the field definition in the vertex and the corresponding field in the triple pattern, G and G, S and S and so on.
Note that a non-leaf vertex defines less than four of its fields, e.g., the root vertex does not define any of its fields and top-level <emphasis>graph map { ... }</emphasis> defines only graph.
Checks are performed only for defined fields and return one of three values: &quot;failed&quot;, &quot;passed&quot;, &quot;full match&quot;, according to the following rules:
</para>
<table><title>Matching Triple Field and Vertex Field</title>
<tgroup cols="3">
<thead><row>
<entry>Field of vertex</entry><entry>Field in triple pattern</entry><entry>Result</entry>
</row></thead>
<tbody>
<row><entry>constant</entry><entry>same constant</entry><entry>full match</entry></row>
<row><entry>constant</entry><entry>different constant</entry><entry>failed</entry></row>
<row><entry>constant</entry><entry>variable of same type</entry><entry>passed</entry></row>
<row><entry>constant</entry><entry>variable of different type</entry><entry>failed</entry></row>
<row><entry>quad map value</entry><entry>constant of same type</entry><entry>full match</entry></row>
<row><entry>quad map value</entry><entry>constant of different type</entry><entry>failed</entry></row>
<row><entry>quad map value of type X</entry><entry>variable, X or subtype of X</entry><entry>full match</entry></row>
<row><entry>quad map value of type X</entry><entry>variable, supertype of X</entry><entry>passed</entry></row>
<row><entry>quad map value of type X</entry><entry>variable, type does not intersect with X</entry><entry>failed</entry></row>
</tbody>
</tgroup>
</table>
<para>
If any of the checks fails, the vertex and all its children are excluded from the rest of processing.
Otherwise, if all four fields are defined for the quad map pattern, the map is added to the list of matching map patterns.
The difference between &quot;passed&quot; and &quot;full match&quot; is significant only if the map is declared with <emphasis>option (exclusive)</emphasis>
If all performed checks return &quot;full match&quot; and <emphasis>option (exclusive)</emphasis> is set then the traverse of the tree is stopped as soon as all children of the vertex are traversed.
The most typical use of this option is when the application developer is sure that all triples of a graph belong to his application and they come from his own quad map patterns, not from <emphasis>DB.DBA.RDF_QUAD</emphasis>.
This is to prevent the SPARQL compiler from generating redundant subqueries accessing <emphasis>DB.DBA.RDF_QUAD</emphasis>.
The declaration may look like
</para>
<programlisting><![CDATA[
create quad storage <mystorage>
  {
    graph <mygraph> option (exclusive) { . . . }
    create virtrdf:DefaultQuadMap
      using storage virtrdf:DefaultQuadStorage .
  }
]]></programlisting>
<para>
Exclusive patterns make the order of declarations important, because an exclusive declaration may &quot;throw a shadow&quot; on declarations after it.
Consider a database that have a special table RDF_TYPE that caches all RDF types of all subjects in all graphs.
Consider two declarations: all triples from graph <emphasis>&lt;http://myhost/sys&gt;</emphasis> and all triples with <emphasis>rdf:type</emphasis> predicate, both exclusive:
</para>
<programlisting><![CDATA[
graph <http://myhost/sys> option (exclusive)
  {
    . . . # mapping of DB.DBA.SYS_USERS as in previous examples.
  }
graph rdfdf:default-iid-nonblank (DB.DBA.RDF_TYPE.G)
subject rdfdf:default-iid (DB.DBA.RDF_TYPE.S)
predicate rdf:type
object rdfdf:default (DB.DBA.RDF_TYPE.O)
option (exclusive)
]]></programlisting>
<para>
The order of these declarations dictates that triple pattern
</para>
<programlisting><![CDATA[
graph <http://myhost/sys> {?s rdf:type ?o}
]]></programlisting>
<para>
is compiled using only quad map patterns of the graph declaration, ignoring second declaration (and of course ignoring default mapping rule, if any).
An explicit <emphasis>option (order N)</emphasis> at the end of quad map pattern will tweak the priority.
By default, order will grow from 1000 for the first declaration in the statement to 1999 for the last, explicit configuration is especially useful to make order persistent to <emphasis>alter storage</emphasis> statements.
</para>
<para>
The <emphasis>option (exclusive)</emphasis> trick is ugly, low-level and prone to cause compilation errors after altering storage declarations.
When misused, it is as bad as &quot;red cut&quot; in PROLOG, but one must use this trick to build scalable storages.
</para>
<para>The <emphasis>option (exclusive)</emphasis> helps the SPARQL compiler to prepare better SQL queries, but sometimes it is &quot;too exclusive&quot;. For instance, if a grouping quad map pattern specify only quad map value for graph and no other fields then making it exclusive prohibits the use of all declarations of the storage after that one. Sometimes it is better to notify compiler that quads made by the given quad map pattern are supposed to be different from all quads made by declarations listed after the given one.</para>

<para>Consider an application that exports users&apos; personal data
as graphs whose IRIs looks like
<emphasis>http://www.example.com/DAV/home/</emphasis>username<emphasis>/RDF/personal/</emphasis>;
the application makes a query and a triple pattern is proven to be
restrictive enough to filter out all quads that are not similar to
quads generated by the given quad map pattern (say, the graph is
constant
<emphasis>http://www.example.com/DAV/home/JohnSmith/RDF/personal/</emphasis>). The
application do not hope to find any quads that match the pattern but
made by other applications, because graphs named like in the pattern
are supposed to be solely for this single purpose; if, say,
DB.DBA.RDF_QUAD occasionally contains some quads with graph equal to
<emphasis>http://www.example.com/DAV/home/JohnSmith/RDF/personal/</emphasis>
then they can be ignored.</para>

<para>Under this circumstances, the quad map pattern may have <emphasis>option (soft exclusive)</emphasis>. That grants a permission to the compiler to ignore rest of storage as soon as it is proven that the triple pattern can not access quads that does not match the pattern. So if that is proven then the pattern is exclusive and it makes the query faster; when unsure, the compiler work like there is no option at all.</para>


<note><para>The <emphasis>option (exclusive)</emphasis> can be used as
a security measure, <emphasis>option (soft exclusive)</emphasis> can
not. Say, if an financial application exports its data as a single
graph <emphasis>http://www.example.com/front-office/cash/</emphasis>
using <emphasis>exclusive</emphasis> then the query that explicitly
refers to that graph will never access any quads written by the
attacker into DB.DBA.RDF_QUAD using same graph IRI. The use of
<emphasis>soft exclusive</emphasis> gives no such protection. From the
compiler's perspective, the <emphasis>option (soft
exclusive)</emphasis> is a hint that may be ignored, not an
unambiguous order.</para></note>

<para>
There is one exception from the rules described above.
This exception is for <emphasis>virtrdf:DefaultQuadStorage</emphasis> only.
If a graph variable of a quad map pattern is not bound and no source graph specified by <emphasis>FROM</emphasis> clauses then quad maps for specific constant graphs are ignored.
In other words, if a default quad storage contains quad maps for specific graphs then the query in that storage should explicitly specify the graph in order to use a map for graph.
This rule will not work if the default quad map is removed from the <emphasis>virtrdf:DefaultQuadStorage</emphasis>.
This rule relates to the default storage itself, not to the containing patterns; copying some or all patterns into other storage will not reproduce there this special effect.
</para>
<para>So for example the query from below returns results when graph is specified i.e. 
when no graph is referenced, then run over physical store only is performed:</para>
<programlisting><![CDATA[
SQL>SPARQL 
SELECT * 
WHERE
    {
    <http://localhost:8990/Demo/categories/CategoryID/1#this>  ?p ?o 
  };
p        o
VARCHAR  VARCHAR
_______________________________________________________________________________

0 Rows. -- 0 msec.

SQL>SPARQL 
SELECT * 
WHERE 
  { 
    GRAPH ?g 
      {
        <http://localhost:8990/Demo/categories/CategoryID/1#this>  ?p ?o 
      }
  };
g                              p                                                     o
VARCHAR                        VARCHAR                                               VARCHAR
___________________________________________________________________________________________________________________________________

http://localhost:8990/Demo#    http://www.w3.org/1999/02/22-rdf-syntax-ns#type       http://localhost:8990/schemas/Demo/Categories
http://localhost:8990/Demo#    http://localhost:8990/schemas/Demo/categoryid         1
http://localhost:8990/Demo#    http://localhost:8990/schemas/Demo/categoryname       ...
...

]]></programlisting>
    </sect2>
<sect2 id="rdfviewdescribingsourcerelationaltables"><title>Describing Source Relational Tables</title>
<para>Quad map patterns of an application usually share a common set of source tables and quad map values of one pattern usually share either a single table or very small number of joined tables.
Join and filtering conditions are also usually repeated in different patterns.
It is not necessary to type table descriptions multiple times, they are declare once in the beginning of storage declaration statement and shared between all quad map declarations inside the statement.
Names of aliases can be used instead of table names in quad map values.
</para>
<programlisting><![CDATA[
FROM DB.DBA.SYS_USERS as user WHERE (^{user.}^.U_IS_ROLE = 0)
FROM DB.DBA.SYS_USERS as group WHERE (^{group.}^.U_IS_ROLE = 1)
FROM DB.DBA.SYS_USERS as account
FROM user as active_user
  WHERE (^{active_user.}^.U_ACCOUNT_DISABLED = 0)
FROM DB.DBA.SYS_ROLE_GRANTS as grant
  WHERE (^{grant.}^.GI_SUPER = ^{account.}^.U_ID)
  WHERE (^{grant.}^.GI_SUB = ^{group.}^.U_ID)
  WHERE (^{grant.}^.GI_SUPER = ^{user.}^.U_ID)
]]></programlisting>
<para>
This declares five distinct aliases for two distinct tables, and six filtering conditions.
Every condition is an SQL expression with placeholders where a reference to the table should be printed.
The SPARQL compiler will not try to parse texts of these expressions (except dummy search for placeholders), so any logical expressions are acceptable.
When a quad map pattern declaration refers to some aliases, the <emphasis>WHERE</emphasis> clause of the generated SQL code will contain a conjunction of all distinct texts of &quot;relevant&quot; conditions.
A condition is relevant if every alias inside the condition is used in some quad map value of the map pattern, either directly or via clause like <emphasis>from user as active_user</emphasis>.
(<emphasis>user</emphasis> is a &quot;<emphasis>base alias</emphasis>&quot; for <emphasis>active_user</emphasis>).
</para><para>
Consider a group of four declarations.
</para>
<programlisting><![CDATA[
graph <http://myhost/sys>
  {
    oplsioc:user_iri (active_user.U_ID)
        a oplsioc:active-user .
    oplsioc:membership_iri (grant.GI_SUPER, grant.GI_SUB).
        oplsioc:is_direct
            grant.GI_DIRECT ;
        oplsioc:member-e-mail
            active_user.U_E_MAIL
               where (^{active_user.}^.U_E_MAIL like 'mailto:%').
    ldap:account-ref (account.U_NAME)
        ldap:belongs-to
            ldap:account-ref (group.U_NAME) option (using grant).
  }
]]></programlisting>
<para>
The first declaration will extend <emphasis>&lt;http://myhost/sys&gt;</emphasis> graph with one imaginary triples <emphasis>{ user a oplsioc:active-user }</emphasis> for every account record that is not a role and not disabled.
The second declaration deals with membership records.
A membership is a pair of a grantee (&quot;super&quot;) and a granted role (&quot;sub&quot;) stored as a row in <emphasis>DB.DBA.SYS_ROLE_GRANTS</emphasis>).
</para><para>
The second declaration states that every membership has <emphasis>oplsioc:is_direct</emphasis> property with value from <emphasis>GI_DIRECT</emphasis> column of that table (roles may be granted to other roles and users, so permissions are &quot;direct&quot; or &quot;recursive&quot;).
</para><para>
The third declaration declares <emphasis>oplsioc:member-e-mail</emphasis> property of memberships.
The value is a literal string from <emphasis>DB.DBA.SYS_USERS.U_E_MAIL</emphasis>, if the grantee is active (not disabled) and is not a role and its e-mail address starts with <emphasis>'mailto:'</emphasis>.
The join between <emphasis>DB.DBA.SYS_ROLE_GRANTS</emphasis> and <emphasis>DB.DBA.SYS_USERS</emphasis> is made by equality <emphasis>(GI_SUPER = U_ID)</emphasis> because the alias <emphasis>active_user</emphasis> in the declaration &quot;inherits&quot; all conditions specified for <emphasis>user</emphasis>.
In addition, the SPARQL compiler will add one more condition to check if the <emphasis>U_E_MAIL</emphasis> is not null because the NULL value is not a valid object and it knows that <emphasis>U_E_MAIL</emphasis> is not declared as <emphasis>NOT NULL</emphasis>.
</para><para>
The last declaration contains an <emphasis>option</emphasis> clause.
As usual, this indicates that the basic functionality is good for many tasks but not for all.
In this declaration, the <emphasis>ldap:belongs-to</emphasis> property establishes a relation between grantee (subject) and a granted role (object).
Both subject and object IRIs are based on account name, <emphasis>DB.DBA.SYS_USERS.U_NAME</emphasis>, so the quad map pattern contains two references to different aliases of <emphasis>DB.DBA.SYS_USERS</emphasis> but no alias for <emphasis>DB.DBA.SYS_ROLE_GRANTS</emphasis>.
Hence the declaration could produce a triple for every row of the Cartesian product of the <emphasis>DB.DBA.SYS_USERS</emphasis>.
To fix the problem, <emphasis>option (using alias-name)</emphasis> tells the compiler to process the alias-name as if it's used in some quad map value of the pattern.
</para><para>
It is an error to use an alias only in <emphasis>where</emphasis> clause of the quad map pattern but neither in values or in <emphasis>option (using alias-name)</emphasis>.
To detect more typos, an alias used in quad map values can not appear in <emphasis>option (using alias-name)</emphasis> clause.
</para>
</sect2>
<sect2 id="rdfviewiriusingfunction"><title>Function-Based IRI Classes</title>
<para>Most of IRI classes can be declared by a sprintf format string, but sophisticated cases may require calculations, not only printing the string. <emphasis>create IRI class using function</emphasis> allows the application transform relational values to IRIs by any custom routines.</para>
<para>
Let us extend the previous example about users and groups by a new class for grantees. Both users and groups are grantees and we have defined two IRI classes for them. Classes <emphasis>oplsioc:user_iri</emphasis> and <emphasis>oplsioc:group_iri</emphasis> work fine for quad maps of <emphasis>U_ID</emphasis> if and only if the value of <emphasis>U_IS_ROLE</emphasis> is accordingly restricted to FALSE or TRUE, otherwise one may occasionally generate, say, user IRI for a group.
To create and parse IRIs that correspond to any U_IDs, two functions should be created:
</para>
<programlisting><![CDATA[
create function DB.DBA.GRANTEE_URI (in id integer)
returns varchar
{
  declare isrole integer;
  isrole := coalesce ((SELECT top 1 U_IS_ROLE
      FROM DB.DBA.SYS_USERS WHERE U_ID = id ) );
  if (isrole is null)
    return NULL;
  else if (isrole)
    return sprintf ('http://%s/sys/group?id=%d', id);
  else
    return sprintf ('http://%s/sys/user?id=%d', id);
};
]]></programlisting>
<programlisting><![CDATA[
create function DB.DBA.GRANTEE_URI_INVERSE (in id_iri varchar)
returns integer
{
  declare parts any;
  parts := sprintf_inverse (id_iri,
      'http://myhost/sys/user?id=%d', 1 );
  if (parts is not null)
    {
      if (exists (SELECT top 1 1 FROM DB.DBA.SYS_USERS
          WHERE U_ID = parts[0] and not U_IS_ROLE ) )
        return parts[0];
    }
  parts := sprintf_inverse (id_iri,
      'http://myhost/sys/group?id=%d', 1 );
  if (parts is not null)
    {
      if (exists (SELECT top 1 1 FROM DB.DBA.SYS_USERS
          WHERE U_ID = parts[0] and U_IS_ROLE ) )
        return parts[0];
   }
  return NULL;
  };
]]></programlisting>
<para>These functions may be more useful if the SPARQL web service endpoint is allowed to use them:</para>
<programlisting><![CDATA[
grant execute on DB.DBA.GRANTEE_URI to "SPARQL";
grant execute on DB.DBA.GRANTEE_URI_INVERSE to "SPARQL";
]]></programlisting>
<para>
The next declaration creates an IRI class based on these two functions:
</para>
<programlisting><![CDATA[
create iri class oplsioc:grantee_iri using
  function DB.DBA.GRANTEE_URI (in id integer)
    returns varchar,
  function DB.DBA.GRANTEE_URI_INVERSE (in id_iri varchar)
    returns integer .
]]></programlisting>
<para>
In common case, IRI class declaration contains an N-array function that composes IRIs and N inverse functions that gets an IRI as an argument and extracts the Nth SQL value.
IRI composing function should silently return NULL on incorrect arguments instead of error signal.
Inverse functions should return NULL if the argument has an incorrect type or value.
</para>
<para>
It is possible to specify only composing function without any of inverse functions. However <emphasis>option (bijection)</emphasis> can not be used in that case, obviously.
</para>
</sect2>
<sect2 id="rdfconnvarsiniriclasses"><title>Connection Variables in IRI Classes</title>
<para>Writing function-based IRI class is overkill when the IRI can in principle be made by a <link linkend="fn_sprintf_iri"><function>sprintf_iri</function></link> but the format should contain some context-specific data, such as host name used for the <link linked="rdfdynamiclocal">dynamic renaming of local IRIs</link>.
Format strings offer a special syntax for that cases.
<emphasis>%{varname}U</emphasis> acts as <emphasis>%U</emphasis> but the function <link linkend="fn_sprintf"><function>sprintf</function></link> will take the value from client connection variable <emphasis>varname</emphasis>, not from list of arguments.
Similarly, <link linkend="fn_sprintf_inverse"><function>sprintf_inverse</function></link> will not return fragment that match to <emphasis>%{varname}U</emphasis> in the vector of other fragments; instead it will get the value from connection environment and ensure that it matches the fragment of input; mismatch between printed and actual value of variable will means that the whole string do not match the format.</para>
<para>SPARQL optimizer knows about this formatting feature and sometimes it makes more deductions from occurrence of <emphasis>%{varname}U</emphasis> than from occurrence of plain <emphasis>%U</emphasis>, so this notation may be used in <emphasis>option ( returns ...)</emphasis> when appropriate.
Of course, the optimizer has no access to the actual value of connection variable because it may vary from run to run or may change between the compilation and the run, but the value is supposed to be persistent during any single query run so <emphasis>%{myvariable}U</emphasis> in one place is equal to <emphasis>%{myvariable}U</emphasis> in other.</para>
<para>Connection variables are set by <link linkend="fn_connection_set"><function>connection_set</function></link> and some of them have default values that are used if not overridden by application:</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem><emphasis>URIQADefaultHost</emphasis> is for default host as it is specified in Virtuoso configuration file.
Note, however, that it will be escaped when printed so if it contains colon and port number then the colon is escaped.
In addition, there are special variables that match dynamic renaming of local IRIs more accurately.</listitem>
<listitem><emphasis>WSHost</emphasis> is for host and port as it is used by current client connection for dynamic renaming.
The colon before port will be escaped.</listitem>
<listitem><emphasis>WSHostName</emphasis> is for host name only, without port, as it is used by current client connection for dynamic renaming.</listitem>
<listitem><emphasis>WSHostPort</emphasis> is for port part of host IRI. That is string, not integer. The only real use of the variable is in formats like <emphasis>http://%{WSHostName}U:%{WSHostPort}U/...</emphasis>.</listitem>
</itemizedlist>
<para>It is inconvenient to write different format strings for
different cases. Two most common policies are different host names
for default HTTP port of a publicly available service and different
non-default ports for one or more host names of an intranet
installation; these two approaches are almost never used in a mix. So
declaration of IRI classes may use shorthand
<emphasis>^{DynamicLocalFormat}^</emphasis> in format strings that is
expanded either to <emphasis>http://%{WSHost}U</emphasis> or to
<emphasis>http://%{WSHostName}U:%{WSHostPort}U/...</emphasis>,
depending on absence or presence of port number in the value of
<emphasis>DefaultHost</emphasis> parameter of
<emphasis>URIQA</emphasis> section of configuration file.</para>
<note><para><emphasis>^{DynamicLocalFormat}^</emphasis> is for IRI class declarations only and is not expanded in any other place, so it is useful sometimes to create an IRI class with empty argument list in order to get &quot;almost constant&quot; IRIs calculated without writing special procedures.</para></note>
</sect2>
<sect2 id="rdfviewbijandreturns"><title>Lookup Optimization -- BIJECTION and RETURNS Options</title>
<para>
There is one subtle problem with IRI class declarations.
To get benefit from a relational index, SPARQL optimizer should compose equality between table column and some known SQL value, not between return value of IRI class and a known composed IRI.
In addition, redundant calculations of IRIs takes time.
To enable this optimization, an IRI class declaration should end with <emphasis>option (bijection)</emphasis> clause. For some simple format strings the compiler may recognize the bijection automatically but an explicit declaration is always a good idea.
</para>
<note><title>Note:</title>
<para>
See also: <ulink url="http://en.wikipedia.org/wiki/One-to-one_correspondence">Wikipedia - Bijection</ulink>.
In mathematics, a bijection, or a bijective function is a function f from a set X to a set Y such that,
for every y in Y, there is exactly one x in X such that f(x) = y.
</para>
<para>
Alternatively, f is bijective if it is a one-to-one correspondence between those sets; i.e.,
both one-to-one (injective) and onto (surjective).
</para>
</note>
<para>
The SPARQL compiler may produce big amounts of SQL code when the query contains equality of two calculated IRIs and these IRIs may come from many different IRI classes.
It is possible to provide hints that will let the compiler check if two IRI classes form disjoint sets of possible IRI values. The more disjoint sets are found the less possible combinations remain so the resulting SQL query will contain fewer unions of joins.
The SPARQL compiler can prove some properties of sprintf format strings. E.g., it can prove that set of all strings printed by &quot;http://example.com/item%d&quot; and the set of strings printed by &quot;http://example.com/item%d/&quot; are disjoint.
It can prove some more complicated statements about unions and intersections of sets of strings.
The IRI or literal class declaration may contain <emphasis>option (returns ...)</emphasis> clause that will specify one or more sprintf patterns that cover the set of generated values.
Consider a better version of IRI class declaration listed above:
</para>
<programlisting><![CDATA[
create iri class oplsioc:grantee_iri using
  function DB.DBA.GRANTEE_URI (in id integer)
    returns varchar,
  function DB.DBA.GRANTEE_URI_INVERSE (in id_iri varchar)
    returns integer
  option ( bijection,
    returns "http://myhost/sys/group?id=%d"
    union   "http://myhost/sys/user?id=%d" ) .
]]></programlisting>
<para>
It is very important to keep IRI classes easily distinguishable by the text of IRI string and easy to parse.
</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>Format <emphasis>%U</emphasis> is better than <emphasis>%s</emphasis>, especially in the middle of IRI, because the <emphasis>%U</emphasis> fragment can not contain characters like &quot;/&quot; or &quot;=&quot;; one may prove that <emphasis>/%U/</emphasis> and <emphasis>/abra%d/cadabra/</emphasis> are disjoint but <emphasis>/%s/</emphasis> and <emphasis>/abra%d/cadabra/</emphasis> are not disjoint.
</listitem><listitem>It is better when the variable part like <emphasis>%U</emphasis> or <emphasis>%d</emphasis> is placed between characters that may not occur in the <emphasis>%U</emphasis> or <emphasis>%d</emphasis> output, i.e. <emphasis>%U</emphasis> is placed between &quot;/&quot;, &quot;&amp;&quot; or &quot;=&quot; and <emphasis>%d</emphasis> is placed between non-digits; <emphasis>order_line_%d</emphasis> is better than <emphasis>order-line-%d</emphasis> because minus may be part of <emphasis>%d</emphasis> output.
</listitem><listitem>End-of-line is treated as a special character, so placing <emphasis>%U</emphasis> or <emphasis>%d</emphasis> between &quot;/&quot; and end of line is as good as placing it between two &quot;/&quot;.
</listitem></itemizedlist>
<para>
In some cases <emphasis>option (returns ...)</emphasis> can be used for IRI classes that are declared using sprintf format, but actual data have more specific format.
Consider a literal class declaration that is used to output strings and the application knows that all these strings are ISBN numbers:
</para>
<programlisting><![CDATA[
create literal class example:isbn_ref "%s" (in isbn varchar not null)
  option ( bijection, returns "%u-%u-%u-%u" union "%u-%u-%u-X" )
]]></programlisting>
<para>
Sometimes interoperability restrictions will force you to violate these rules but please try to follow them as often as possible.
</para>
</sect2>
<sect2 id="rdfviewsubclasses"><title>Join Optimization -- Declaring IRI Subclasses</title>
<para>
Additional problem appears when the equality is between two IRIs of two different IRI classes.
Even if both of them are bijections, the compiler does not know if these IRI classes behave identically on the intersection of their domains.
To let the optimizer know this fact, one IRI class can be explicitly declared as a subclass of another:
</para>
<programlisting><![CDATA[
make oplsioc:user_iri subclass of oplsioc:grantee_iri .
make oplsioc:group_iri subclass of oplsioc:grantee_iri .
]]></programlisting>
<para>
The SPARQL compiler can not check the validity of a subclass declaration.
The developer should carefully test functions to ensure that transformations are really subclasses, as well as to ensure that functions of an IRI class declarations are really inverse to each other.
</para><para>
When declaring that a table's primary key is converted into a IRI according to one IRI class, one usually declares that all foreign keys referring to this class also get converted into an IRI as per this same class, or subclass of same class.
</para><para>
Subclasses can be declared for literal classes as well as for IRI classes, but this case is rare. The reason is that most of literals are made by identity literal classes that are disjoint to each other even if values may be equal in SQL sense, such as <emphasis>"2"</emphasis> of type <emphasis>xsd:integer</emphasis> and <emphasis>"2.0"</emphasis> of type <emphasis>xsd:double</emphasis>.
</para>
    </sect2>
<sect2 id="rdfmetadatarecovery"><title>RDF Metadata Maintenance and Recovery</title>
   <para>
This section refers to checking and backing up Linked Data View and storage declarations only. The checks and backup/restore do not affect physical quads, relational schema or tables or data therein. For general backup and restore, see server administration.
     To detect and fix automatically most popular sorts of RDF metadata corruption use <link linkend="fn_rdf_audit_metadata"><function>DB.DBA.RDF_AUDIT_METADATA</function></link>.
It is also possible to backup RDF data by
    <link linkend="fn_rdf_backup_metadata"><function>DB.DBA.RDF_BACKUP_METADATA</function></link>
and restore the saved state later by using
    <link linkend="fn_rdf_restore_metadata"><function>DB.DBA.RDF_RESTORE_METADATA</function></link>.
It is convenient to make a backup before any modification of quad storages, quad map patterns or IRI classes, especially during debugging new Linked Data Views.
   </para>
<note><para>In SQL, adding a new view can not break anything. This is because SQL lacks the ability of querying &quot;everything&quot; so data sources are always specified. This is not true for SPARQL, so please treat <emphasis>any</emphasis> metadata manipulation as potentially destructive operation. If an RDF storage is supposed to be used by more than one application then these applications should be tested together, not one after other, and they should be installed/upgraded on live database in the very same order as they were installed/upgraded on instrumental machine during testing. Always remember that these applications share RDF tables so they may interfere.</para></note>
</sect2>
<sect2 id="splitrdfview"><title>Split Linked Data View</title>
<para>Linked Data View can be created by two or more "sparql alter storage" statements. In each statement
can be created one quad map that contains mappings for half or a third of all tables. Quad maps
created should have distinct names but may mention same graph. The important fact is that if the
Linked Data View in question is exclusive for a graph then only the last quad map should be exclusive but
all previous should not have this option. This is because if a map is exclusive on a graph the rest
of maps on that graph will be silently ignored.</para>
<para>The example below shows a sample part of the Virtuoso eCRM Views code,
where the Linked Data View is split in two parts: with quad map virtrdf:ecrmDemo1 and with
quad map virtrdf:ecrmDemo2:</para>
<programlisting><![CDATA[
SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
drop quad map virtrdf:ecrmDemo1 .
;

SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
drop quad map virtrdf:ecrmDemo2 .
;

...

SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
alter quad storage virtrdf:DefaultQuadStorage
FROM eCRM.DBA.SFA_SALES_QUOTA_VIEW2 as sales_quotas
FROM eCRM.DBA.SFA_COMPANIES_VIEW2 as companies
FROM eCRM.DBA.SFA_COMPANIES as companies_table text literal companies_table.DESCRIPTION of (companies.DESCRIPTION)
FROM eCRM.DBA.SFA_CONTACTS_VIEW2 as contacts
FROM eCRM.DBA.SFA_CONTACTS as contacts_table text literal contacts_table.NAME_FIRST of (contacts.NAME_FIRST)
FROM eCRM.DBA.SFA_EMPLOYMENTS_VIEW2 as employments
FROM eCRM.DBA.SFA_LEADS_VIEW2 as leads
FROM eCRM.DBA.SFA_LEADS as leads_table text literal leads_table.SUBJECT of (leads.SUBJECT)
FROM eCRM.DBA.SFA_OPPORTUNITIES_VIEW2 as opportunities
FROM eCRM.DBA.SFA_OPPORTUNITIES as opportunities_table text literal opportunities_table.OPPORTUNITY_NAME of (opportunities.OPPORTUNITY_NAME)
FROM eCRM.DBA.SFA_ACTIVITIES as activities
FROM eCRM.DBA.SFA_MAIL_MESSAGES as messages
FROM eCRM.DBA.SFA_DOCUMENTS_VIEW2 as documents
FROM eCRM.DBA.SFA_INFLUENCERS_VIEW2 as influencers
FROM eCRM.DBA.SFA_TEAMS_VIEW2 as teams
FROM eCRM.DBA.SFA_NOTES_VIEW2 as notes
FROM eCRM.DBA.SFA_NOTES as notes_table text literal notes_table.DESCRIPTION of (notes.DESCRIPTION)
FROM eCRM.DBA.SFA_COMPETITORS_VIEW2 as competitors
FROM eCRM.DBA.SFA_ISSUES_VIEW2 as issues
FROM eCRM.DBA.SFA_CUSTOM_FIELD_DEFS_VIEW2 as custom_field_defs
FROM eCRM.DBA.SFA_CUSTOM_FIELDS_VIEW2 as custom_fields
FROM eCRM.DBA.SFA_CASES_VIEW2 as cases
FROM eCRM.DBA.SFA_CASES as cases_table text literal cases_table.SUMMARY of (cases.SUMMARY)
FROM eCRM.DBA.SFA_ORDERS_VIEW2 as orders
FROM eCRM.DBA.SFA_ORDERS as orders_table text literal orders_table.EMAIL of (orders.EMAIL)
FROM eCRM.DBA.SFA_ORDER_ITEMS_VIEW2 as order_items
FROM eCRM.DBA.PM_CATEGORIES_VIEW2 as categories
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTE_DEFS_VIEW2 as product_attribute_defs
FROM eCRM.DBA.PM_PRODUCTS_VIEW2 as products
FROM eCRM.DBA.PM_PRODUCTS as products_table text literal products_table.DESCRIPTION of (products.DESCRIPTION)
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTES_VIEW2 as product_attributes
FROM eCRM.DBA.PM_CATALOGS_VIEW2 as catalogs
FROM eCRM.DBA.PM_CATALOG_PRODUCTS_VIEW2 as catalog_products
FROM eCRM.DBA.XSYS_MODULES as modules
FROM eCRM.DBA.XSYS_REGISTRY as registries
FROM eCRM.DBA.XSYS_ORGANIZATIONS_DATA as organizations_data
FROM eCRM.DBA.XSYS_MESSAGES as xsysmessages
FROM eCRM.DBA.XSYS_COUNTRIES_VIEW2 as countries
FROM eCRM.DBA.XSYS_PROVINCES_VIEW2 as provinces
FROM eCRM.DBA.XSYS_TIMEZONES as timezones
FROM eCRM.DBA.XSYS_MIME_TYPES as mimetypes
FROM eCRM.DBA.XSYS_MIME_EXTENSIONS as mimeexts
FROM eCRM.DBA.XSYS_CNAMES as cnames
FROM eCRM.DBA.XSYS_QUOTAS as quotas
FROM eCRM.DBA.XSYS_ROLES as roles
FROM eCRM.DBA.XSYS_ACCOUNTS as accounts
FROM eCRM.DBA.XSYS_USERDATA as userdatas
FROM eCRM.DBA.XSYS_GROUPDATA as groupdatas
FROM eCRM.DBA.XSYS_MEMBERS as members
FROM eCRM.DBA.XSYS_SESSIONS_DATA as sessionsdatas
FROM eCRM.DBA.XSYS_SESSION_DATA as sessiondatas
FROM eCRM.DBA.XSYS_LIST_MEMBERS_DEFS as list_members_defs
FROM eCRM.DBA.XSYS_CLASSES as classes
FROM eCRM.DBA.XSYS_ORG_CLASSES as org_classes
FROM eCRM.DBA.XSYS_CLASS_METHODS as class_methods
FROM eCRM.DBA.XSYS_CLASS_VIEWS as class_views
FROM eCRM.DBA.XSYS_ROLE_PRIVILEGES as role_priveleges
FROM eCRM.DBA.XSYS_USER_PRIVILEGES as user_priveleges
FROM eCRM.DBA.XSYS_HISTORY as history
FROM eCRM.DBA.XSYS_USERS as xsys_users
FROM eCRM.DBA.AP_PROCESSES_VIEW2 as ap_processes
FROM eCRM.DBA.AP_RULES_VIEW2 as ap_rules
FROM eCRM.DBA.AP_QUEUE as ap_queues
WHERE (^{companies.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{contacts.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{products.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{orders.}^.SHIP_COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads_table.}^.FREETEXT_ID = ^{leads.}^.FREETEXT_ID)
WHERE (^{contacts_table.}^.FREETEXT_ID = ^{contacts.}^.FREETEXT_ID)
WHERE (^{companies_table.}^.FREETEXT_ID = ^{companies.}^.FREETEXT_ID)
WHERE (^{opportunities_table.}^.FREETEXT_ID = ^{opportunities.}^.FREETEXT_ID)
WHERE (^{cases_table.}^.FREETEXT_ID = ^{cases.}^.FREETEXT_ID)
WHERE (^{notes_table.}^.FREETEXT_ID = ^{notes.}^.FREETEXT_ID)
WHERE (^{orders_table.}^.FREETEXT_ID = ^{orders.}^.FREETEXT_ID)
WHERE (^{products_table.}^.FREETEXT_ID = ^{products.}^.FREETEXT_ID)
    {
        create virtrdf:ecrmDemo1 as graph iri ("http://^{URIQADefaultHost}^/ecrm") option (order 1501)
      {
            ecrm:Country (countries.COUNTRY_NAME)
                a ecrm:Country
                    as virtrdf:Country-Countrys2 ;
                a geo:SpatialThing
                    as virtrdf:Country-Countrys ;
                owl:sameAs ecrm:dbpedia_iri (countries.COUNTRY_NAME) ;
                ecrm:countryID countries.COUNTRY_ID
                        as virtrdf:Country-COUNTRY_ID ;
                ecrm:countryID3 countries.COUNTRY_ID3
                        as virtrdf:Country-COUNTRY_ID3 ;
                ecrm:isoCode countries.ISO_CODE
                        as virtrdf:Country-ISO_CODE ;
                ecrm:countryName countries.COUNTRY_NAME
                        as virtrdf:Country-COUNTRY_NAME .

            ecrm:Country (countries.COUNTRY_NAME)
                        ecrm:has_province
            ecrm:Province (provinces.COUNTRY_ID, provinces.PROVINCE_NAME) where
                        (^{provinces.}^.COUNTRY_ID = ^{countries.}^.COUNTRY_ID) as virtrdf:ecrmCountry-has_province .

...
  } .
} .
;
SPARQL
prefix ecrm: <http://demo.openlinksw.com/schemas/ecrm#>
prefix oplsioc: <http://www.openlinksw.com/schemas/oplsioc#>
prefix sioc: <http://rdfs.org/sioc/ns#>
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
prefix cal: <http://www.w3.org/2002/12/cal/ical#>
prefix product: <http://www.swop-project.eu/ontologies/pmo/product.owl#>
prefix owl: <http://www.w3.org/2002/07/owl#>
alter quad storage virtrdf:DefaultQuadStorage
FROM eCRM.DBA.SFA_SALES_QUOTA_VIEW2 as sales_quotas
FROM eCRM.DBA.SFA_COMPANIES_VIEW2 as companies
FROM eCRM.DBA.SFA_COMPANIES as companies_table text literal companies_table.DESCRIPTION of (companies.DESCRIPTION)
FROM eCRM.DBA.SFA_CONTACTS_VIEW2 as contacts
FROM eCRM.DBA.SFA_CONTACTS as contacts_table text literal contacts_table.NAME_FIRST of (contacts.NAME_FIRST)
FROM eCRM.DBA.SFA_EMPLOYMENTS_VIEW2 as employments
FROM eCRM.DBA.SFA_LEADS_VIEW2 as leads
FROM eCRM.DBA.SFA_LEADS as leads_table text literal leads_table.SUBJECT of (leads.SUBJECT)
FROM eCRM.DBA.SFA_OPPORTUNITIES_VIEW2 as opportunities
FROM eCRM.DBA.SFA_OPPORTUNITIES as opportunities_table text literal opportunities_table.OPPORTUNITY_NAME of (opportunities.OPPORTUNITY_NAME)
FROM eCRM.DBA.SFA_ACTIVITIES as activities
FROM eCRM.DBA.SFA_MAIL_MESSAGES as messages
FROM eCRM.DBA.SFA_DOCUMENTS_VIEW2 as documents
FROM eCRM.DBA.SFA_INFLUENCERS_VIEW2 as influencers
FROM eCRM.DBA.SFA_TEAMS_VIEW2 as teams
FROM eCRM.DBA.SFA_NOTES_VIEW2 as notes
FROM eCRM.DBA.SFA_NOTES as notes_table text literal notes_table.DESCRIPTION of (notes.DESCRIPTION)
FROM eCRM.DBA.SFA_COMPETITORS_VIEW2 as competitors
FROM eCRM.DBA.SFA_ISSUES_VIEW2 as issues
FROM eCRM.DBA.SFA_CUSTOM_FIELD_DEFS_VIEW2 as custom_field_defs
FROM eCRM.DBA.SFA_CUSTOM_FIELDS_VIEW2 as custom_fields
FROM eCRM.DBA.SFA_CASES_VIEW2 as cases
FROM eCRM.DBA.SFA_CASES as cases_table text literal cases_table.SUMMARY of (cases.SUMMARY)
FROM eCRM.DBA.SFA_ORDERS_VIEW2 as orders
FROM eCRM.DBA.SFA_ORDERS as orders_table text literal orders_table.EMAIL of (orders.EMAIL)
FROM eCRM.DBA.SFA_ORDER_ITEMS_VIEW2 as order_items
FROM eCRM.DBA.PM_CATEGORIES_VIEW2 as categories
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTE_DEFS_VIEW2 as product_attribute_defs
FROM eCRM.DBA.PM_PRODUCTS_VIEW2 as products
FROM eCRM.DBA.PM_PRODUCTS as products_table text literal products_table.DESCRIPTION of (products.DESCRIPTION)
FROM eCRM.DBA.PM_PRODUCT_ATTRIBUTES_VIEW2 as product_attributes
FROM eCRM.DBA.PM_CATALOGS_VIEW2 as catalogs
FROM eCRM.DBA.PM_CATALOG_PRODUCTS_VIEW2 as catalog_products
FROM eCRM.DBA.XSYS_MODULES as modules
FROM eCRM.DBA.XSYS_REGISTRY as registries
FROM eCRM.DBA.XSYS_ORGANIZATIONS_DATA as organizations_data
FROM eCRM.DBA.XSYS_MESSAGES as xsysmessages
FROM eCRM.DBA.XSYS_COUNTRIES_VIEW2 as countries
FROM eCRM.DBA.XSYS_PROVINCES_VIEW2 as provinces
FROM eCRM.DBA.XSYS_TIMEZONES as timezones
FROM eCRM.DBA.XSYS_MIME_TYPES as mimetypes
FROM eCRM.DBA.XSYS_MIME_EXTENSIONS as mimeexts
FROM eCRM.DBA.XSYS_CNAMES as cnames
FROM eCRM.DBA.XSYS_QUOTAS as quotas
FROM eCRM.DBA.XSYS_ROLES as roles
FROM eCRM.DBA.XSYS_ACCOUNTS as accounts
FROM eCRM.DBA.XSYS_USERDATA as userdatas
FROM eCRM.DBA.XSYS_GROUPDATA as groupdatas
FROM eCRM.DBA.XSYS_MEMBERS as members
FROM eCRM.DBA.XSYS_SESSIONS_DATA as sessionsdatas
FROM eCRM.DBA.XSYS_SESSION_DATA as sessiondatas
FROM eCRM.DBA.XSYS_LIST_MEMBERS_DEFS as list_members_defs
FROM eCRM.DBA.XSYS_CLASSES as classes
FROM eCRM.DBA.XSYS_ORG_CLASSES as org_classes
FROM eCRM.DBA.XSYS_CLASS_METHODS as class_methods
FROM eCRM.DBA.XSYS_CLASS_VIEWS as class_views
FROM eCRM.DBA.XSYS_ROLE_PRIVILEGES as role_priveleges
FROM eCRM.DBA.XSYS_USER_PRIVILEGES as user_priveleges
FROM eCRM.DBA.XSYS_HISTORY as history
FROM eCRM.DBA.XSYS_USERS as xsys_users
FROM eCRM.DBA.AP_PROCESSES_VIEW2 as ap_processes
FROM eCRM.DBA.AP_RULES_VIEW2 as ap_rules
FROM eCRM.DBA.AP_QUEUE as ap_queues
WHERE (^{companies.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{contacts.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{products.}^.COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{orders.}^.SHIP_COUNTRY_NAME = ^{countries.}^.COUNTRY_NAME)
WHERE (^{leads_table.}^.FREETEXT_ID = ^{leads.}^.FREETEXT_ID)
WHERE (^{contacts_table.}^.FREETEXT_ID = ^{contacts.}^.FREETEXT_ID)
WHERE (^{companies_table.}^.FREETEXT_ID = ^{companies.}^.FREETEXT_ID)
WHERE (^{opportunities_table.}^.FREETEXT_ID = ^{opportunities.}^.FREETEXT_ID)
WHERE (^{cases_table.}^.FREETEXT_ID = ^{cases.}^.FREETEXT_ID)
WHERE (^{notes_table.}^.FREETEXT_ID = ^{notes.}^.FREETEXT_ID)
WHERE (^{orders_table.}^.FREETEXT_ID = ^{orders.}^.FREETEXT_ID)
WHERE (^{products_table.}^.FREETEXT_ID = ^{products.}^.FREETEXT_ID)
{
        create virtrdf:ecrmDemo2 as graph iri ("http://^{URIQADefaultHost}^/ecrm") option (exclusive, order 1502)
  {
            ecrm:Order (orders.ORG_ID, orders.ORDER_ID)
                a ecrm:Order
                    as virtrdf:Order-Orders ;
                ecrm:has_ecrm_organization ecrm:OrganizationsData(orders.ORG_ID, organizations_data.DNS_ZONE) where (^{orders.}^.ORG_ID = ^{organizations_data.}^.ORG_ID)
                        as virtrdf:Order-ORG_ID ;
                ecrm:owner ecrm:XSys_User(orders.ORG_ID, xsys_users.ACCOUNT_NAME, orders.OWNER_ID)
                        where (^{orders.}^.OWNER_ID = ^{xsys_users.}^.ACCOUNT_ID and ^{orders.}^.ORG_ID = ^{xsys_users.}^.ORG_ID)
                        as virtrdf:Order-OWNER_ID ;
                ecrm:FREETEXT_ID orders.FREETEXT_ID
                        as virtrdf:Order-FREETEXT_ID ;
                ecrm:has_company ecrm:Company(orders.COMPANY_NAME, orders.COMPANY_ID, orders.ORG_ID)
                        as virtrdf:Order-COMPANY_ID ;
                ecrm:companyName orders.COMPANY_NAME
                        as virtrdf:Order-COMPANY_NAME ;
                ecrm:has_contact ecrm:Contact(contacts.NAME_FIRST, contacts.NAME_MIDDLE, contacts.NAME_LAST, orders.CONTACT_ID, orders.ORG_ID)
                        where (^{orders.}^.CONTACT_ID = ^{contacts.}^.CONTACT_ID and ^{orders.}^.ORG_ID = ^{contacts.}^.ORG_ID)
                        as virtrdf:Order-CONTACT_ID ;
                ecrm:contactName orders.CONTACT_NAME
                        as virtrdf:Order-CONTACT_NAME ;
                ecrm:orderNo orders.ORDER_NO
                        as virtrdf:Order-ORDER_NO ;
                ecrm:shipFirstName orders.SHIP_FNAME
                        as virtrdf:Order-SHIP_FNAME ;
                ecrm:shipSecondName orders.SHIP_SNAME
                        as virtrdf:Order-SHIP_SNAME ;
                ecrm:phoneNumber orders.PHONE_NUMBER
                        as virtrdf:Order-PHONE_NUMBER ;
                ecrm:phoneExtension orders.PHONE_EXTENSION
                        as virtrdf:Order-PHONE_EXTENSION ;
                ecrm:email orders.EMAIL
                        as virtrdf:Order-EMAIL ;
                ecrm:shipCountry ecrm:Country(orders.SHIP_COUNTRY_NAME)
                        as virtrdf:Order-SHIP_COUNTRY_NAME ;
                ecrm:shipCountryCode ecrm:Country (countries.COUNTRY_NAME) where  (^{countries.}^.COUNTRY_NAME = ^{orders.}^.SHIP_COUNTRY_NAME)
                        as virtrdf:Order-SHIP_COUNTRY_CODE ;
                ecrm:shipProvince orders.SHIP_PROVINCE
                        as virtrdf:Order-SHIP_PROVINCE ;
                ecrm:shipCity orders.SHIP_CITY
                        as virtrdf:Order-SHIP_CITY ;
                ecrm:dbpedia_shipCity ecrm:dbpedia_iri (orders.SHIP_CITY)
                        as virtrdf:Order-SHIP_dbpedia_CITY ;
                ecrm:shipPostalCode orders.SHIP_POSTAL_CODE
                        as virtrdf:Order-SHIP_POSTAL_CODE ;
                ecrm:shipAddress1 orders.SHIP_ADDRESS1
                        as virtrdf:Order-SHIP_ADDRESS1 ;
                ecrm:shipAddress2 orders.SHIP_ADDRESS2
                        as virtrdf:Order-SHIP_ADDRESS2 ;
                ecrm:salesRep orders.SALESREP
                        as virtrdf:Order-SALESREP ;
                ecrm:orderDate orders.ORDER_DATE
                        as virtrdf:Order-ORDER_DATE ;
                ecrm:orderValue orders.ORDER_VALUE
                        as virtrdf:Order-ORDER_VALUE ;
                ecrm:refund orders.REFUND
                        as virtrdf:Order-REFUND ;
                ecrm:year orders.YEAR
                        as virtrdf:Order-YEAR ;
                ecrm:month orders.MONTH
                        as virtrdf:Order-MONTH ;
                ecrm:quarter orders.QUARTER
                        as virtrdf:Order-QUARTER ;
                ecrm:financialYear orders.FINANCIAL_YEAR
                        as virtrdf:Order-FINANCIAL_YEAR ;
                ecrm:CONTACT_REL_ID orders.CONTACT_REL_ID
                        as virtrdf:Order-CONTACT_REL_ID ;
                ecrm:COMPANY_REL_ID orders.COMPANY_REL_ID
                        as virtrdf:Order-COMPANY_REL_ID .

...
        } .
} .
;

]]></programlisting>
</sect2>
<sect2 id="rdfviewsrcur"><title>Linked Data Views and recursive FK relationships</title>
  <para>Here is sample example of a script to include an additional table alias for a table:</para>
<programlisting><![CDATA[
alter quad storage virtrdf:DefaultQuadStorage
  :
  FROM isports_rdf.prs10_isports_rdf.VRef_Call      as Ref_Call_tbl
  FROM isports_rdf.prs10_isports_rdf.VRef_Call      as Ref_Call_tbl_1
  :
{
  :
  refcall:ref-call_iri (Ref_Call_tbl.Call_Num) a refcall:Ref-Call as
virtrdf:ref-call_pk ;
  :
  refcall:has_parent  refcall:ref-call_iri (Ref_Call_tbl_1.Call_Num)
where  ( ^{Ref_Call_tbl.}^.Parent    = ^{Ref_Call_tbl_1.}^.Call_Num )   as
virtrdf:Ref-Call_has_parent .
]]></programlisting>
  <para>This demonstrates the way to self-join the table VRef_Call with itself. Like in SQL,
are needed two different aliases for one table if you want to join it with itself.
  </para>
    </sect2>
</sect1>
<sect1 id="rdfrdfviewgnr"><title>Automated Generation of Linked Data Views over Relational Data Sources</title>
<sect2 id="rdfrdfviewgnrintro"><title>Introduction</title>
<para>Virtuoso offers from Conductor UI an HTML based Wizard interface for dynamically generating &amp;
publishing RDF based Linked Data from ODBC or JDBC accessible relational data sources. Basically,
a mechanism for building RDF based Linked Data views over relational data sources.
</para>
<para>The proliferation of relational databases across enterprises and behind Web sites, makes them a
vital data source for the burgeoning Linked Data Web. Thus, the process of publishing Linked Data from
these sources needs to be as unobtrusive as possible. Naturally, a balance has to be struck between
unobtrusive generation of Linked Data and traditional relational database management system (RDBMS)
virtues such as:
</para>
<itemizedlist mark="bullet">
  <listitem>Scalability</listitem>
  <listitem>Security</listitem>
  <listitem>Analytical Expressivity of SQL</listitem>
  <listitem>Separation of Data Access and Data Storage via ODBC, JDBC, ADO.NET CLIs.</listitem>
</itemizedlist>
<para>The following steps must be taken to publish RDF-based Linked Data:
</para>
<orderedlist>
  <listitem>Identifying ODBC or JDBC data sources that host the data you seek to publish
(assuming the data isn't Virtuoso RDBMS hosted -- in which case, skip ahead to step #3).</listitem>
  <listitem>Attach/Link TABLEs or VIEWs from the external data sources into Virtuoso via their Data Source Names (DSNs).</listitem>
  <listitem>Identify the internal or external TABLEs or VIEWs that hold the data you wish to publish.</listitem>
  <listitem>Configure Endpoints and Re-write Rules to disambiguate data object (resource) identity and description through HTTP-based content negotiation.</listitem>
  <listitem>Expose the Data Source Ontology and associated Instance Data in Linked Data form through those Endpoints and Re-write Rules.</listitem>
</orderedlist>
<para>
These steps may be largely automated (the "One-Click" Deployment below), or performed manually ("Using the Conductor's HTML-based Wizard" further down).
</para>
</sect2>
<sect2 id="rdfrdfviewgnroneclick"><title>One Click Linked Data Generation &amp; Deployment</title>
<para>The following steps provide a one-click guide for publishing ODBC- or JDBC-accessible RDBMS data in RDF Linked Data form, using the "Generate &amp; Publish" Conductor feature.
</para>
    <orderedlist>
  <listitem>Go to http://&lt;cname&gt;:port/conductor ;</listitem>
  <listitem>Log in as user dba (or another user with DBA privileges);</listitem>
  <listitem>Follow menu path Linked Data -&gt; Views;
    <figure id="rd1" float="1">
      <title>Linked Data Views</title>
      <graphic fileref="ui/rd1.png"/>
    </figure>
</listitem>
  <listitem>In the form presented, perform the following steps:
<orderedlist>
  <listitem>Select the Database Name Qualifier (e.g., "Demo")
that exposes the Tables / Views for this exercise </listitem>
  <listitem>Enter the Base URL to which your URL rewrite rules will be bound
(e.g. http://&lt;cname&gt;:8890/Demo)</listitem>
  <listitem>Select specific Tables containing the data to be published (e.g. Demo.demo.Orders and Demo.demo.Products)</listitem>
  <listitem>Click the "Generate &amp; Publish" button</listitem>
    </orderedlist>
    <figure id="rd2" float="1">
      <title>Linked Data Views Generate and Publish</title>
      <graphic fileref="ui/rd2.png"/>
    </figure>
</listitem>
  <listitem>Virtuoso will perform the entire process of ontology generation, instance data generation, and
linked data deployment (re-write rules generation and application).</listitem>
  <listitem>Error messages will be presented if the Wizard encounters problems. If there are no error
messages, your Linked Data View declarations and Linked Data publishing activities will have completed successfully.
    <figure id="rd13" float="1">
      <title>Linked Data View declarations and Linked Data publishing activities</title>
      <graphic fileref="ui/rd13.png"/>
    </figure>
      </listitem>
  <listitem>Optionally, you could also perform one of the following tasks:
<orderedlist>
  <listitem>Save Data Mappings: when clicked, offers to save the generated Definitions to local file system</listitem>
  <listitem>Save Ontology Mappings: when clicked, offers to save the generated Ontology to local file system</listitem>
  <listitem>Click on the "Cancel" should you want to return to the initial Linked Data View Generation form.</listitem>
</orderedlist>
  </listitem>
</orderedlist>
</sect2>
<sect2 id="rdfrdfviewgnrwizzard"><title>Manual Linked Data Generation &amp; Deployment using the Conductor's HTML-based wizard</title>
<para>The following step-by guide will lead you through manually publishing ODBC- or JDBC-accessible RDBMS
data in RDF Linked Data form, using the Conductor's HTML-based wizard:
</para>
<orderedlist>
  <listitem>Go to http://&lt;cname&gt;:port/conductor</listitem>
  <listitem>Log in as user dba (or another user with DBA privileges)</listitem>
  <listitem>Follow menu path Linked Data -&gt; Views
    <figure id="rd1" float="1">
      <title>Linked Data Views</title>
      <graphic fileref="ui/rd1.png"/>
    </figure>
</listitem>
  <listitem>In the form presented, perform the following steps:
    <orderedlist>
      <listitem>Select the Database Name Qualifier (e.g., "Demo") that exposes the Tables / Views for this
exercise</listitem>
      <listitem>Enter the Base URL to which your URL rewrite rules will be bound (e.g. http://&lt;cname&gt;:8890/Demo)</listitem>
      <listitem>Select specific Tables containing the data to be published (e.g., Demo.demo.Orders and Demo.demo.Products)</listitem>
      <listitem>Click the "Generate via Wizard" button
        <figure id="rd2" float="1">
          <title>Generate via Wizard</title>
          <graphic fileref="ui/rd2.png"/>
        </figure>
        </listitem>
    </orderedlist>
</listitem>
  <listitem>At this point, you are presented with the option to edit your column selection. Select the
"Edit" link, for example, for table Demo.demo.Products.
    <figure id="rd3" float="1">
      <title>Column Selection</title>
      <graphic fileref="ui/rd3.png"/>
    </figure>
</listitem>
  <listitem>For images or other binary data in MIME formats to be revealed as anything other than generic "binary objects", you must map large
varbinary types to the appropriate MIME types like image/gif. To do so, select the Edit link for Binding/MIME Type of the relevant table columns.
You can:
    <orderedlist>
      <listitem>Leave the Binding/MIME Type literal; or</listitem>
      <listitem>Set to skip, such that the column will not be used in RDF generation; or </listitem>
      <listitem>Select the binary object value in order for the column to be referenced as binary.</listitem>
    </orderedlist>
    <figure id="rd14" float="1">
      <title>Binding/MIME Types</title>
      <graphic fileref="ui/rd14.png"/>
    </figure>
</listitem>
  <listitem>After finishing with your changes click the Save button, or cancel the changes and go back by
clicking the Cancel button.</listitem>
  <listitem>Make sure you click the "Next" button.</listitem>
  <listitem>At this point, the Linked Data View Definition form will let you Select Generation Targets options:
    <orderedlist>
      <listitem>Data Source Ontology Mappings</listitem>
      <listitem>Instance Data View Mappings</listitem>
      <listitem>VoID statistic</listitem>
    </orderedlist>
    <figure id="rd15" float="1">
      <title>Generation Targets options</title>
      <graphic fileref="ui/rd15.png"/>
    </figure>
</listitem>
  <listitem>Make sure you click the "Next" button.</listitem>
  <listitem>Based on your selections in the prior form, the Linked Data View Definition Deployment Options form will be offered:
    <orderedlist>
      <listitem>Data Source Ontology Rules</listitem>
      <listitem>Instance Data Rules
</listitem>
    </orderedlist>
    <figure id="rd4" float="1">
      <title>Generation Targets options</title>
      <graphic fileref="ui/rd4.png"/>
    </figure>
</listitem>
  <listitem>Select the desired option(s) and click the "Prepare to Execute" button which unveils a generated
Instance Data and/or Ontology form.
    <figure id="rd5" float="1">
      <title>Instance Data and/or Ontology</title>
      <graphic fileref="ui/rd5.png"/>
    </figure>
</listitem>
  <listitem>Click the Execute button and Virtuoso will:
    <orderedlist>
      <listitem>Apply the generated declarations (instance data and ontology) to your Virtuoso instance</listitem>
      <listitem>Publish / Deploy declarations that expose the Wizard-generated Rewrite Rules and associated endpoints.
        <figure id="rd6" float="1">
          <title>Publishing / Deployment declarations</title>
          <graphic fileref="ui/rd6.png"/>
        </figure>
</listitem>
    </orderedlist>
</listitem>
  <listitem>Optionally, you can also perform one of the following tasks:
    <orderedlist>
      <listitem>Save Data Mappings: when clicked, offers to save the generated Definitions to local file
system</listitem>
      <listitem>Save Ontology Mappings: when clicked, offers to save the generated Ontology to local file
system</listitem>
      <listitem>Export as WebDAV resource: exports the selected objects/items as a WebDAV resource:
       <orderedlist>
          <listitem>Click "Browse"</listitem>
          <listitem>Enter a WebDAV resource and click the "Select" button.</listitem>
        </orderedlist>
        <figure id="rd7" float="1">
          <title>WebDAV resource</title>
          <graphic fileref="ui/rd7.png"/>
        </figure>
</listitem>
    </orderedlist>
    <para>Note, the WebDAV resource path value will be shown in the WebDAV location field.</para>
</listitem>
  <listitem>Then click the "Save Data Mappings" or "Save Ontology Mappings" button, to complete the option task of saving your generated (or edited) view declarations.
    <figure id="rd8" float="1">
      <title>WebDAV resource</title>
      <graphic fileref="ui/rd8.png"/>
    </figure>
</listitem>
  <listitem>Error messages will be presented if the Wizard encounters problems. If there are no error 
  	messages, your Linked Data View declarations and Linked Data publishing activities will have completed successfully.
    <figure id="rd9" float="1">
      <title>Linked Data View declarations and Linked Data publishing activities Finish</title>
      <graphic fileref="ui/rd9.png"/>
    </figure>
</listitem>
  <listitem>Click on Cancel to return to the initial Linked Data View Generation form.</listitem>
</orderedlist>
</sect2>
<sect2 id="rdfrdfviewgnrisql"><title>Manual Linked Data Generation &amp; Deployment using iSQL command-line</title>
To generate data synchronization triggers and/or actual RDF quad store data from transient Linked Data views, one can use the
Virtuoso built-in <link linkend="fn_rdf_view_sync_to_physical"><function>RDF_VIEW_SYNC_TO_PHYSICAL()</function></link> function calling it
from the iSQL command-line. See <link linkend="fn_rdf_view_sync_to_physical"><function>more details and usage examples.</function></link>
</sect2>
</sect1>

<sect1 id="r2rml"><title>Virtuoso R2RML Support</title>
  <sect2 id="r2rmlwhat"><title>What is R2RML?</title>
    <para><ulink url="http://www.w3.org/TR/r2rml/">R2RML</ulink> is a language for expressing customized 
    mappings from relational databases to RDF data sets. Such mappings provide the ability to view existing 
    relational data in the RDF data model, expressed in a structure and target vocabulary of the mapping 
    author's choice.</para>
    <para>R2RML mappings are themselves RDF graphs written in Turtle syntax.</para>    
  </sect2>
  <sect2 id="r2rmlwhy"><title>Why use it?</title>
    <para>As a W3C working draft, R2RML is becoming the generic standard adopted by most vendors of tools 
    	mapping relational data to RDF, enabling the interoperability of R2RML scripts, whether created with 
    	such tools or by hand.</para>  
  </sect2>
  <sect2 id="r2rmlhow"><title>How do I use it with Virtuoso?</title>  
    <para>Virtuoso has its own previously-developed proprietary equivalent of R2RML called 
    	<link linkend="rdfviewsrdbms">Linked Data Views</link>, which uses Virtuoso's 
    	<ulink url="http://virtuoso.openlinksw.com/whitepapers/relational%20rdf%20views%20mapping.html">Meta Schema Mapping Language</ulink> to map relational data to RDF.</para>  
    <para>R2RML support is achieved by the inclusion of a simple translator which basically translates R2RML syntax to Virtuoso's own Linked Data Views syntax, which can then be executed to create the Linked Data Views themselves.</para>  
    <sect3 id="r2rmlhowinst"><title>Install R2RML VAD package</title>  
      <para>First you will need to ensure you have the R2RML VAD package 
      	(<ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/rdb2rdf_dav.vad">rdb2rdf_dav.vad</ulink>) 
      	installed.
      </para>	
    </sect3>  
    <sect3 id="r2rmlhowtest"><title>Test with simple test script (basic.sql)</title>  
      <para>Having installed the R2RML VAD package, to test R2RML functionality, the easiest way is by executing a basic.sql script via the command line isql tool:</para>
<programlisting><![CDATA[

CREATE TABLE "R2RML"."TEST"."PRODUCT"(
  id integer primary key,
  name VARCHAR(100)
);

INSERT SOFT "R2RML"."TEST"."PRODUCT" VALUES (1, 'Virtuoso');

SPARQL CLEAR GRAPH <http://temp/product>;
SPARQL CLEAR GRAPH <http://example.com/>;



DB.DBA.TTLP ('
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix exa: <http://example.com/ns#> .
@prefix product: <http://example.com/product#> .

<http://example.com/ns#TriplesMap1>
    a rr:TriplesMap;

    rr:logicalTable 
    [ 
      rr:tableSchema "R2RML"; 
      rr:tableOwner "TEST"; 
      rr:tableName  "PRODUCT" 
    ];

    rr:subjectMap 
    [ 
      rr:template "http://example.com/product/{id}";
      rr:class exa:product;
      rr:graph <http://example.com/>;
    ];

    rr:predicateObjectMap
    [
      rr:predicate product:id;
      rr:objectMap [ rr:column "id" ];
    ];

    rr:predicateObjectMap
    [
      rr:predicate product:name;
      rr:objectMap [ rr:column "name" ];
    ];
.
', 'http://temp/product', 'http://temp/product' )
;



--select DB.DBA.R2RML_TEST ('http://temp/product');

--DB.DBA.OVL_VALIDATE ('http://temp/product', 'http://www.w3.org/ns/r2rml#OVL');

-- Running the validation in order to find error in name of R2RML description graph
--DB.DBA.OVL_VALIDATE ('http://temp/product-nosuch', 'http://www.w3.org/ns/r2rml#OVL');

-- Running the validation in order to find error in name of R2RML metadata graph
--DB.DBA.OVL_VALIDATE ('http://temp/product', 'http://www.w3.org/ns/r2rml#OVL-nosuch');

--select DB.DBA.R2RML_EXECUTE ('http://temp/product');

exec ('sparql ' || DB.DBA.R2RML_MAKE_QM_FROM_G ('http://temp/product'));

--sparql select distinct ?g where { graph ?g { ?s a ?t }};

SPARQL
SELECT * FROM <http://example.com/>
WHERE {?s ?p ?o .};	
]]></programlisting>
      <orderedlist>
        <listitem>First, copy basic.sql into:
<programlisting><![CDATA[
<VIRTUOSO_INSTALL>/bin/
]]></programlisting>        	
        </listitem>
        <listitem>Next, open Unix session or Windows Command Prompt and execute:
<programlisting><![CDATA[

cd <OPENLINK_INSTALL>/bin

./isql     (Unix)
isql.exe   (Windows)

OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL>	
]]></programlisting>        	
        </listitem>
        <listitem>Then, within isql execute:
<programlisting><![CDATA[
SQL> load basic.sql;	
]]></programlisting>        	
        </listitem>
        <listitem>Execution should finish with a simple SPARQL query that will return Linked Data for 
        	the test table created at the start of the script:
<programlisting><![CDATA[
s                               p                                                  o
VARCHAR                         VARCHAR                                            VARCHAR
________________________________________________________________________________________________________________

http://example.com/product/1    http://example.com/product#id                      1
http://example.com/product/1    http://example.com/product#name                    Virtuoso
http://example.com/product/1    http://www.w3.org/1999/02/22-rdf-syntax-ns#type    http://example.com/ns#product
3 Rows. -- 0 msec.	
]]></programlisting>        	
        </listitem>        
      </orderedlist>      
      <para><emphasis>Note</emphasis>: Subsequent executions of basic.sql will return an error since the test table will already exist. However, the remainder of the script will execute fine.</para>  
    </sect3>     
    <sect3 id="r2rmlhowexam"><title>Examining basic.sql</title>  
      <orderedlist>
        <listitem>We start by creating and populating the test table:
<programlisting><![CDATA[
CREATE TABLE "R2RML"."TEST"."PRODUCT"
  (
    id   INTEGER PRIMARY KEY ,
    name VARCHAR(100)
  );

INSERT SOFT "R2RML"."TEST"."PRODUCT" 
  VALUES 
    (
      1, 'Virtuoso'
    );	
]]></programlisting>        	
        </listitem>
        <listitem>Next we clear any graphs (temporary or permanent) that are to be used during this process:
<programlisting><![CDATA[
SPARQL CLEAR GRAPH <http://temp/product> ;
SPARQL CLEAR GRAPH <http://example.com/> ;	
]]></programlisting>        	
        </listitem>
        <listitem>Next we use the <link linkend="fn_ttlp"><function>DB.DBA.TTLP()</function></link> 
        procedure to insert the R2RML into a temporary graph, &lt;http://temp/product&gt;:
<programlisting><![CDATA[        
DB.DBA.TTLP 
  (
    ' @prefix       rr:  <http://www.w3.org/ns/r2rml#>    .
      @prefix      exa:  <http://example.com/ns#>         .
      @prefix  product:  <http://example.com/product#>    .

      <http://example.com/ns#TriplesMap1>
          a                              rr:TriplesMap  ;

          rr:logicalTable 
            [ 
              rr:tableSchema  "R2RML"    ; 
              rr:tableOwner   "TEST"     ; 
              rr:tableName    "PRODUCT" 
            ];

          rr:subjectMap 
            [ 
              rr:template  "http://example.com/product/{id}" ;
              rr:class     exa:product                       ;
              rr:graph     <http://example.com/>
            ];

          rr:predicateObjectMap
            [
              rr:predicate  product:id  ;
              rr:objectMap  
                [ 
                  rr:column  "id" 
                ];
            ];

          rr:predicateObjectMap
            [
              rr:predicate  product:name  ;
              rr:objectMap 
                [ 
                  rr:column "name" 
                ];
            ];
         .
    ', 
    'http://temp/product',
    'http://temp/product' 
  );        	
]]></programlisting>       
        </listitem>
        <listitem>Next, there is a series of commented out lines that can be used for sanity checking:
<programlisting><![CDATA[
--SELECT DB.DBA.R2RML_TEST ('http://temp/product');

--DB.DBA.OVL_VALIDATE ('http://temp/product', 'http://www.w3.org/ns/r2rml#OVL');

-- Running the validation in order to find error in name of R2RML description graph
--DB.DBA.OVL_VALIDATE ('http://temp/product-nosuch', 'http://www.w3.org/ns/r2rml#OVL');

-- Running the validation in order to find error in name of R2RML metadata graph
--DB.DBA.OVL_VALIDATE ('http://temp/product', 'http://www.w3.org/ns/r2rml#OVL-nosuch');

--SELECT DB.DBA.R2RML_EXECUTE ('http://temp/product');	
]]></programlisting>        	
        </listitem>
        <listitem>Next, DB.DBA.R2RML_MAKE_QM_FROM_G() is used to perform the conversion from R2RML into 
        	Virtuoso's own Linked Data Views script. The output is then prepended with the keyword 'SPARQL' 
        	and a space, and executed using <link linkend="fn_exec"><function>exec()</function></link> :
<programlisting><![CDATA[
EXEC ('SPARQL ' || DB.DBA.R2RML_MAKE_QM_FROM_G ('http://temp/product'));	
]]></programlisting>        	
          <para><emphasis>Note</emphasis>: The final triples are placed in the graph defined in the R2RML script itself (&lt;http://example.com/&gt;)</para>
          <para>Alternatively, the destination graph can be specified as an optional second parameter of DB.DBA.R2RML_MAKE_QM_FROM_G():</para>
<programlisting><![CDATA[
DB.DBA.R2RML_MAKE_QM_FROM_G
  (
    (
      IN g VARCHAR 
   [, IN target_graph VARCHAR := NULL]
    )
  )
	
]]></programlisting>
        </listitem>
        <listitem>Finally, a simple SPARQL statement is executed to prove data is returned:
<programlisting><![CDATA[
SPARQL
  SELECT * 
  FROM <http://example.com/>
  WHERE {?s ?p ?o .};	
]]></programlisting>
        </listitem>
      </orderedlist>        
    </sect3>     
  </sect2>  
  <sect2 id="r2rmlknlim"><title>Known Limitations</title>  
    <para><emphasis>rr:sqlQuery</emphasis> is not currently supported, due to limitations in 
    the optimizer used for Virtuoso's native implementation of Linked Data Views.</para>
  </sect2>      
  <sect2 id="r2rmlgenlviewisql"><title>Generating an R2RML based Linked Data View from iSQL command-line</title>
    <para>Using Virtuoso you can programmatically generate Linked Data Views atop Relational Data Sources, 
    	using R2RML via the built-in function: <emphasis>R2RML_GENERATE_LINKED_VIEW</emphasis> function. 
    	In order to use this function, you need to have the 
    	<ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/rdb2rdf_dav.vad">rdb2rdf_dav.vad</ulink> 
    	package installed.
    </para>  
<programlisting><![CDATA[
R2RML_GENERATE_LINKED_VIEW 
  (
    in source varchar, 
    in destination_graph varchar, 
    in graph_type int default 0, 
    in clear_source_graph int default 1
  )	
]]></programlisting>    
    <para>Here is detailed description of the funcion's parameter:</para>
    <itemizedlist mark="bullet">
      <listitem><emphasis>source</emphasis>: The source R2RMLdocument URI. Acceptable schemes include: file:, dav:, http: and https:. These are also acceptable as source graph URI;</listitem>
      <listitem><emphasis>destination graph</emphasis>: This is a default graph name (an IRI) applicable to either virtual or physical graph.</listitem>
      <listitem><emphasis>graph_type</emphasis>: 0 - virtual; 1 - physical graph which sets the actual graph type;</listitem>
      <listitem><emphasis>clear_source_graph</emphasis>: Determines if existing R2RML source graphs (those holding view declarations) are replaced as part of processing pipeline.</listitem>
    </itemizedlist>
    <para><emphasis>Note</emphasis>: The R2RML mapping script may have a triples like:</para>
<programlisting><![CDATA[
[] rr:graph <graph_name> 	
]]></programlisting>    
    <para>and in this case they take precedence and virtual graph would be defined as in the R2RML. 
    	If so, then if destination graph is specified as physical, all virtual graphs found in the 
    	R2RML would go in the destination_graph.</para>
    <sect3 id="r2rmlgenlviewisqlex"><title>Usage Example</title>	
      <orderedlist>
        <listitem>Ensure the R2RML VAD package <ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/rdb2rdf_dav.vad">rdb2rdf_dav.vad</ulink> is installed.</listitem>
        <listitem>To clear out existing mappings execute:
<programlisting><![CDATA[
SQL> SELECT RDF_VIEW_DROP_STMT_BY_GRAPH ('http://example.com');
VARCHAR
_______________________________________________________________________________

SPARQL drop silent quad map <http://demo.openlinksw.com/r2rmldemo.n3> .;


1 Rows. -- 16 msec.

SQL> SPARQL DROP SILENT QUAD MAP <http://demo.openlinksw.com/r2rmldemo.n3> ;

STATE    MESSAGE
VARCHAR  VARCHAR
_______________________________________________________________________________

00000    Quad map <http://demo.openlinksw.com/r2rmldemo.n3> is no longer used in storage <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadStorage
>
00000    Quad map <http://demo.openlinksw.com/r2rmldemo.n3> is deleted
00000    Transaction committed, SPARQL compiler re-configured
00000    2 RDF metadata manipulation operations done

4 Rows. -- 406 msec.


SQL> SPARQL CLEAR <http://demo.openlinksw.com/r2rmldemo.n3>;

callret-0
VARCHAR
_______________________________________________________________________________

Clear <http://demo.openlinksw.com/r2rmldemo.n3> -- done

1 Rows. -- 15 msec.


SQL> DROP TABLE "R2RML"."TEST"."PRODUCT" ;

Done. -- 0 msec.

SQL> CREATE TABLE "R2RML"."TEST"."PRODUCT"
(
  "id" INTEGER,
  "name" VARCHAR(100),
  PRIMARY KEY ("id")
);
Done. -- 16 msec.	
]]></programlisting>        	
        </listitem>
        <listitem>Insert sample data into a Table by executing:
<programlisting><![CDATA[
SQL> INSERT SOFT "R2RML"."TEST"."PRODUCT" VALUES(1, 'Virtuoso');
Done. -- 0 msec.	
]]></programlisting>        	
        </listitem>
        <listitem>Locate or create your R2RML mapping document, for example: .n3 file with the following content:
<programlisting><![CDATA[
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix exa: <http://example.com/ns#> .
@prefix product: <http://example.com/product#> .

<http://example.com/ns#TriplesMap1>
    a rr:TriplesMap;

    rr:logicalTable 
    [ 
      rr:tableSchema "R2RML"; 
      rr:tableOwner "TEST"; 
      rr:tableName  "PRODUCT" 
    ];

    rr:subjectMap 
    [ 
      rr:template "http://example.com/product/{id}";
      rr:class exa:product;
    ];

    rr:predicateObjectMap
    [
      rr:predicate product:id;
      rr:objectMap [ rr:column "id" ];
    ];

    rr:predicateObjectMap
    [
      rr:predicate product:name;
      rr:objectMap [ rr:column "name" ];
    ];
.	
]]></programlisting>       	
        </listitem>
        <listitem>Generate a Linked Data View from the R2RML document that applies to the sample data (created earlier) by executing the statement:
<programlisting><![CDATA[
SQL> DB.DBA.R2RML_GENERATE_LINKED_VIEW('http://demo.openlinksw.com/r2rmldemo.n3', 'http://example.com', 0);
STATE    MESSAGE
VARCHAR  VARCHAR
_______________________________________________________________________________

00000    IRI class <r2rml:virt02-8513ca7e0ce41d2e38f0c750fd552139> has been defined (inherited from rdfdf:sql-integer-uri-nullable)
00000    Literal class <r2rml:virt02-daca9ceddea29d53dbbdb6bd0f3dee68> has been defined (inherited from rdfdf:sql-integer-literal-nullable)
00000    Quad storage <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadStorage> is flagged as being edited
00000    Quad map <http://demo.openlinksw.com/r2rmldemo.n3> has been created and added to the <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadSt
orage>
00000    Quad map <sys:qm-1be5dbd931459cf9e2df2338428f418d> has been created and added to the <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadSt
orage>
00000    Quad map <sys:qm-c5f81d7126efa3e7a93f7e903fd5fa93> has been created and added to the <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadSt
orage>
00000    Quad map <sys:qm-25c4599111b9f07fbd8fc60ce0b42eaf> has been created and added to the <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadSt
orage>
00000    Quad storage <http://www.openlinksw.com/schemas/virtrdf#DefaultQuadStorage> is unflagged and can be edited by other transactions
00000    Transaction committed, SPARQL compiler re-configured
00000    9 RDF metadata manipulation operations done

10 Rows. -- 1109 msec.
SQL>	
]]></programlisting>        	
        </listitem>
        <listitem>Verify successful creation of the Linked Data View by executing the following SPARQL query via iSQL or Conductor interface: 
<programlisting><![CDATA[
SQL> SPARQL 
SELECT * 
FROM <http://example.com> 
WHERE {?s ?p ?o} ;

s                                p                                                   o
VARCHAR                          VARCHAR                                             VARCHAR
_______________________________________________________________________________

http://example.com/product/1     http://example.com/product#id                       1
http://example.com/product/1     http://example.com/product#name                     Virtuoso
http://example.com/product/1     http://www.w3.org/1999/02/22-rdf-syntax-ns#type     http://example.com/ns#product

3 Rows. -- 15 msec.	
]]></programlisting>        	
        </listitem>
      </orderedlist>      
    </sect3>
  </sect2>  
  <sect2 id="r2rmlcondwiz"><title>Virtuoso Conductor R2RML Import Wizard</title>  
    <para>The Virtuoso Conductor can be used for importing existing R2RML scripts into Virtuoso and generate the necessary RDF Linked Data Views for Virtuoso hosting and deployment.</para>  
    <sect3 id="r2rmlcondwizex"><title>Usage Example</title>  
      <orderedlist>
        <listitem>Ensure the R2RML <ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/rdb2rdf_dav.vad">rdb2rdf_dav.vad</ulink> 
        and latest Conductor <ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> VAD packages are installed.
        </listitem>
        <listitem>Create a test table with sample data: 
<programlisting><![CDATA[
SQL> CREATE TABLE "R2RML"."TEST"."PRODUCT"
(
  "id" INTEGER,
  "name" VARCHAR(100),
  PRIMARY KEY ("id")
);
Done. -- 16 msec.
SQL> INSERT SOFT "R2RML"."TEST"."PRODUCT" VALUES(1, 'Virtuoso');
Done. -- 0 msec.
SQL> INSERT SOFT "R2RML"."TEST"."PRODUCT" VALUES(2, 'UDA');
Done. -- 0 msec.
SQL>	
]]></programlisting>        	
        </listitem>
        <listitem>Grant select privileges on the "R2RML"."TEST"."PRODUCT" table to the SPARQL user to enable execution via SPARQL endpoint: 
<programlisting><![CDATA[
SQL> GRANT SELECT ON R2RML.TEST.PRODUCT  TO "SPARQL", "SPARQL_UPDATE"
Done. -- 1 msec.	
]]></programlisting>        	
        </listitem>
        <listitem>Create the following R2RML mapping script for the "R2RML"."TEST"."PRODUCT" table: 
<programlisting><![CDATA[
$ cat demo.n3
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix exa: <http://example.com/ns#> .
@prefix product: <http://example.com/product#> .

<http://example.com/ns#TriplesMap1>
    a rr:TriplesMap;

    rr:logicalTable 
    [ 
      rr:tableSchema "R2RML"; 
      rr:tableOwner "TEST"; 
      rr:tableName  "PRODUCT" 
    ];

    rr:subjectMap 
    [ 
      rr:template "http://example.com/product/{id}";
      rr:class exa:product;
    ];

    rr:predicateObjectMap
    [
      rr:predicate product:id;
      rr:objectMap [ rr:column "id" ];
    ];

    rr:predicateObjectMap
    [
      rr:predicate product:name;
      rr:objectMap [ rr:column "name" ];
    ];
.
$	
]]></programlisting>        	
        </listitem>
        <listitem>Got to the Linked Data -&gt; R2RML tab of the Virtuoso Conductor:
          <figure id="VirtConductorR2RMLImport01" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport01.png"/>
          </figure>        	
        </listitem>
        <listitem>Select the Choose File button and select the R2RML file to load: 
          <figure id="VirtConductorR2RMLImport02" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport02.png"/>
          </figure>           	
        </listitem>
        <listitem>Select the Validate button to verify the R2RML mapping script: 
          <figure id="VirtConductorR2RMLImport03" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport03.png"/>
          </figure>         	
        </listitem>
        <listitem>Select the Generate button to generate the RDF Linked Data Views mappings for the R2RML mapping script:
          <figure id="VirtConductorR2RMLImport04" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport04.png"/>
          </figure>         	
        </listitem>
        <listitem>Select the Execute button to create the RDF Linked Data Views mapping the the Quad Store:
          <figure id="VirtConductorR2RMLImport05" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport05.png"/>
          </figure>         	        	
        </listitem>        
        <listitem>The RDF Linked Data View creation is complete and status is displayed: 
          <figure id="VirtConductorR2RMLImport06" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport06.png"/>
          </figure>         	
        </listitem>
        <listitem>The Default Graph Name (transient) specified http://demo.openlinksw.com/r2rml# can 
        	now be used to run a SPARQL query against the created Linked Data View. If the Generate 
        	<link linkend="rdb2rdftriggers">RDB2RDF triggers</link> and Enable Data Syncs with Physical 
        	Quad Store check boxes are selected the Physical Graph Name (persistent) specified 
        	urn:demo.openlinksw.com/r2rml# can be used to run a SPARQL query against the materialized 
        	triples in the Quad Store. 
          <figure id="VirtConductorR2RMLImport07" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport07.png"/>
          </figure>         	
        </listitem>
        <listitem>The results set for the Linked Data View graph are displayed:
          <figure id="VirtConductorR2RMLImport08" float="1">
            <title>Conductor R2RML Import Wizard</title>
            <graphic fileref="ui/VirtConductorR2RMLImport08.png"/>
          </figure>           	
        </listitem>
      </orderedlist>    
    </sect3>    
  </sect2>    
  <sect2 id="r2rmlgentransperslviewrs"><title>Generate Transient and/or Persistent Linked Data Views atop Remote Relational Data Sources Using Conductor</title>  
    <para>This section describes how you can generate R2RML Scripts from Linked Data Views, using the Virtuoso Conductor ODBC or JDBC accessible.</para>  
    <orderedlist>
        <listitem>Ensure you have installed Conductor <ulink url="http://opldownload.s3.amazonaws.com/uda/vad-packages/6.3/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> VAD package with version 1.32.38 or higher.
        </listitem>
        <listitem>Go to http://&lt;cname&gt;[:&lt;port&gt;]/conductor.       	
        </listitem>
        <listitem>Enter dba credentials.       	
        </listitem>
        <listitem>Go to Linked Data -&gt; Views:
          <figure id="r0" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r0.png"/>
          </figure>        	
        </listitem>
        <listitem>Select Qualifier Demo:
          <figure id="r00" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r00.png"/>
          </figure>           	
        </listitem>
        <listitem>Select table(s) by hatching the check-box to the left of the table name; for example, select the following tables from the Northwind DB: Categories, Customers, Employees, Order_Details, Orders, Products .
          <figure id="r1g" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r1g.png"/>
          </figure>         	
        </listitem>
        <listitem>Click Generate via Wizard:
          <figure id="r2g" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r2g.png"/>
          </figure>         	
        </listitem>
        <listitem>Click Prepare to Execute.</listitem> 
        <listitem>The R2RML script for the selected table(s) will be generated and displayed in the R2RML Graph text-area:
          <figure id="r3g" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r3g.png"/>
          </figure> 
          <figure id="r4g" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r4g.png"/>
          </figure> 
          <figure id="r5g" float="1">
            <title>Generating Transient and/or Persistent Linked Data Views</title>
            <graphic fileref="ui/r5g.png"/>
          </figure>                               	        	
        </listitem>        
        <listitem>As result the following R2RML script should be generated for the Northwind DB collection:
<programlisting><![CDATA[
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix Demo: <http://demo.openlinksw.com/schemas/Demo/> .
@prefix demo-stat: <http://demo.openlinksw.com/Demo/stat#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix void: <http://rdfs.org/ns/void#> .
@prefix scovo: <http://purl.org/NET/scovo#> .
@prefix aowl: <http://bblfish.net/work/atom-owl/2006-06-06/> .


<#TriplesMapCategories> a rr:TriplesMap; rr:logicalTable [ rr:tableSchema "Demo" ; rr:tableOwner "demo" ; rr:tableName "Categories" ]; 
rr:subjectMap [ rr:termtype "IRI"  ; rr:template "http://demo.openlinksw.com/Demo/categories/{CategoryID}"; rr:class Demo:Categories; ];
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:categoryid ] ; rr:objectMap [ rr:column "CategoryID" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:categoryname ] ; rr:objectMap [ rr:column "CategoryName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:description ] ; rr:objectMap [ rr:column "Description" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:picture ] ; rr:objectMap [ rr:column "Picture" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:categories_of_products ] ; rr:objectMap [ rr:parentTriplesMap <#TriplesMapProducts>;  rr:joinCondition [ rr:child "CategoryID" ; rr:parent "CategoryID" ] ; ]; ] .

<#TriplesMapCustomers> a rr:TriplesMap; rr:logicalTable [ rr:tableSchema "Demo" ; rr:tableOwner "demo" ; rr:tableName "Customers" ]; 
rr:subjectMap [ rr:termtype "IRI"  ; rr:template "http://demo.openlinksw.com/Demo/customers/{CustomerID}"; rr:class Demo:Customers; ];
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:customerid ] ; rr:objectMap [ rr:column "CustomerID" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:companyname ] ; rr:objectMap [ rr:column "CompanyName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:contactname ] ; rr:objectMap [ rr:column "ContactName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:contacttitle ] ; rr:objectMap [ rr:column "ContactTitle" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:address ] ; rr:objectMap [ rr:column "Address" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:city ] ; rr:objectMap [ rr:column "City" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:region ] ; rr:objectMap [ rr:column "Region" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:postalcode ] ; rr:objectMap [ rr:column "PostalCode" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:country ] ; rr:objectMap [ rr:column "Country" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:countrycode ] ; rr:objectMap [ rr:column "CountryCode" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:phone ] ; rr:objectMap [ rr:column "Phone" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:fax ] ; rr:objectMap [ rr:column "Fax" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:customers_of_orders ] ; rr:objectMap [ rr:parentTriplesMap <#TriplesMapOrders>;  rr:joinCondition [ rr:child "CustomerID" ; rr:parent "CustomerID" ] ; ]; ] .

<#TriplesMapEmployees> a rr:TriplesMap; rr:logicalTable [ rr:tableSchema "Demo" ; rr:tableOwner "demo" ; rr:tableName "Employees" ]; 
rr:subjectMap [ rr:termtype "IRI"  ; rr:template "http://demo.openlinksw.com/Demo/employees/{EmployeeID}"; rr:class Demo:Employees; ];
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:employeeid ] ; rr:objectMap [ rr:column "EmployeeID" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:lastname ] ; rr:objectMap [ rr:column "LastName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:firstname ] ; rr:objectMap [ rr:column "FirstName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:title ] ; rr:objectMap [ rr:column "Title" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:titleofcourtesy ] ; rr:objectMap [ rr:column "TitleOfCourtesy" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:birthdate ] ; rr:objectMap [ rr:column "BirthDate" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:hiredate ] ; rr:objectMap [ rr:column "HireDate" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:address ] ; rr:objectMap [ rr:column "Address" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:city ] ; rr:objectMap [ rr:column "City" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:region ] ; rr:objectMap [ rr:column "Region" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:postalcode ] ; rr:objectMap [ rr:column "PostalCode" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:country ] ; rr:objectMap [ rr:column "Country" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:countrycode ] ; rr:objectMap [ rr:column "CountryCode" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:homephone ] ; rr:objectMap [ rr:column "HomePhone" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:extension ] ; rr:objectMap [ rr:column "Extension" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:photo ] ; rr:objectMap [ rr:column "Photo" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:notes ] ; rr:objectMap [ rr:column "Notes" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:reportsto ] ; rr:objectMap [ rr:column "ReportsTo" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:employees_of_orders ] ; rr:objectMap [ rr:parentTriplesMap <#TriplesMapOrders>;  rr:joinCondition [ rr:child "EmployeeID" ; rr:parent "EmployeeID" ] ; ]; ] .

<#TriplesMapOrder_Details> a rr:TriplesMap; rr:logicalTable [ rr:tableSchema "Demo" ; rr:tableOwner "demo" ; rr:tableName "Order_Details" ]; 
rr:subjectMap [ rr:termtype "IRI"  ; rr:template "http://demo.openlinksw.com/Demo/order_details/{OrderID}/{ProductID}"; rr:class Demo:Order_Details; ];
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:unitprice ] ; rr:objectMap [ rr:column "UnitPrice" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:quantity ] ; rr:objectMap [ rr:column "Quantity" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:discount ] ; rr:objectMap [ rr:column "Discount" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:order_details_has_orders ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/orders/{OrderID}" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:order_details_has_products ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/products/{ProductID}" ]; ] .

<#TriplesMapOrders> a rr:TriplesMap; rr:logicalTable [ rr:tableSchema "Demo" ; rr:tableOwner "demo" ; rr:tableName "Orders" ]; 
rr:subjectMap [ rr:termtype "IRI"  ; rr:template "http://demo.openlinksw.com/Demo/orders/{OrderID}"; rr:class Demo:Orders; ];
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:orderid ] ; rr:objectMap [ rr:column "OrderID" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:orderdate ] ; rr:objectMap [ rr:column "OrderDate" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:requireddate ] ; rr:objectMap [ rr:column "RequiredDate" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shippeddate ] ; rr:objectMap [ rr:column "ShippedDate" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:freight ] ; rr:objectMap [ rr:column "Freight" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shipname ] ; rr:objectMap [ rr:column "ShipName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shipaddress ] ; rr:objectMap [ rr:column "ShipAddress" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shipcity ] ; rr:objectMap [ rr:column "ShipCity" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shipregion ] ; rr:objectMap [ rr:column "ShipRegion" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shippostalcode ] ; rr:objectMap [ rr:column "ShipPostalCode" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shipcountry ] ; rr:objectMap [ rr:column "ShipCountry" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:shipcountrycode ] ; rr:objectMap [ rr:column "ShipCountryCode" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:rowguid ] ; rr:objectMap [ rr:column "ROWGUID" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:orders_has_customers ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/customers/{CustomerID}" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:orders_has_employees ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/employees/{EmployeeID}" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:orders_has_shippers ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/shippers/{ShipVia}" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:orders_of_order_details ] ; rr:objectMap [ rr:parentTriplesMap <#TriplesMapOrder_Details>;  rr:joinCondition [ rr:child "OrderID" ; rr:parent "OrderID" ] ; ]; ] .

<#TriplesMapProducts> a rr:TriplesMap; rr:logicalTable [ rr:tableSchema "Demo" ; rr:tableOwner "demo" ; rr:tableName "Products" ]; 
rr:subjectMap [ rr:termtype "IRI"  ; rr:template "http://demo.openlinksw.com/Demo/products/{ProductID}"; rr:class Demo:Products; ];
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:productid ] ; rr:objectMap [ rr:column "ProductID" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:productname ] ; rr:objectMap [ rr:column "ProductName" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:quantityperunit ] ; rr:objectMap [ rr:column "QuantityPerUnit" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:unitprice ] ; rr:objectMap [ rr:column "UnitPrice" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:unitsinstock ] ; rr:objectMap [ rr:column "UnitsInStock" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:unitsonorder ] ; rr:objectMap [ rr:column "UnitsOnOrder" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:reorderlevel ] ; rr:objectMap [ rr:column "ReorderLevel" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:discontinued ] ; rr:objectMap [ rr:column "Discontinued" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:products_has_categories ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/categories/{CategoryID}" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:products_has_suppliers ] ; rr:objectMap [ rr:termtype "IRI" ; rr:template "http://demo.openlinksw.com/Demo/suppliers/{SupplierID}" ]; ] ;
rr:predicateObjectMap [ rr:predicateMap [ rr:constant Demo:products_of_order_details ] ; rr:objectMap [ rr:parentTriplesMap <#TriplesMapOrder_Details>;  rr:joinCondition [ rr:child "ProductID" ; rr:parent "ProductID" ] ; ]; ] .	
]]></programlisting>              	
        </listitem>
      </orderedlist>  
  </sect2>      
  
</sect1>

&rdfviewssamples;

<sect1 id="rdfinsertmethods"><title>RDF Insert Methods in Virtuoso</title>
    <sect2 id="rdfinsertmethodsapifunct"><title>Using API functions</title>
      <itemizedlist mark="bullet">
        <listitem><link linkend="rdfapidataimportttlp">Using the DB.DBA.TTLP() function</link>
          <itemizedlist mark="bullet">
            <listitem>Note: use this function for loading Turtle</listitem>
           </itemizedlist>
</listitem>
        <listitem><link linkend="rdfapidataimportttlpmt">Using the DB.DBA.TTLP_MT() function</link>
          <itemizedlist mark="bullet">
            <listitem>Note: use this function for loading triples from file on multiple threads</listitem>
           </itemizedlist>
</listitem>
        <listitem><link linkend="rdfapidataimportxmlttlpmt">Using the DB.DBA.RDF_LOAD_RDFXML_MT() function</link>
          <itemizedlist mark="bullet">
            <listitem>Note: Use this function for loading large resources when transactional integrity is not important (loading of a single resource may take more than one transaction)</listitem>
           </itemizedlist>
</listitem>
        <listitem><link linkend="rdfapidataimportttlphash">Using the DB.DBA.RDF_TTL2HASH() function</link>
    <itemizedlist mark="bullet">
            <listitem>Note: use this function to get dictionary of triples in 'long valmode'.</listitem>
    </itemizedlist>
</listitem>
        <listitem><link linkend="rdfapidataimportloadrdfxml">Using the DB.DBA.RDF_LOAD_RDFXML() function</link>
          <itemizedlist mark="bullet">
            <listitem>For loading RDF/XML, the best way is to split the data to be loaded into
multiple streams and load these in parallel using this function.</listitem>
           </itemizedlist>
</listitem>
      </itemizedlist>
      <para>See <link linkend="rdfperfloading">more details</link> for loading Performance Tuning specifics.</para>
    </sect2>
    <sect2 id="rdfinsertmethodshttppost"><title>SPARQL endpoint REST API</title>
	<para>With POST can be accomplished SPARQL Insert/Update etc.</para>
	<para>The result is in the rdf_quad.</para>
	<para>With GET Methods you can get the triples which are saved.</para>
        <para><emphasis>Examples:</emphasis></para>
        <para><emphasis>Example 1:</emphasis></para>
        <para>Create a DAV collection xx for user demo with password demo.</para>
        <para>Execute the following command:</para>
<programlisting><![CDATA[
curl -i -d "INSERT {<http://demo.openlinksw.com/DAV/home/demo_about.rdf>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/ns#User>}" -u "demo:demo"
-H "Content-Type: application/sparql-query" http://example.com/DAV/xx/yy
]]></programlisting>
        <para>The response should be:</para>
<programlisting><![CDATA[
HTTP/1.1 201 Created
Server: Virtuoso/05.00.3023 (Win32) i686-generic-win-32  VDB
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
Date: Fri, 28 Dec 2007 12:50:12 GMT
Accept-Ranges: bytes
MS-Author-Via: SPARQL
Content-Length: 0
]]></programlisting>
        <para>The result in the DAV/xx location will be a new WebDAV resource with name "yy" containing the inserted RDF:</para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<rdf:Description
rdf:about="http://demo.openlinksw.com/DAV/home/demo_about.rdf">
<ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
rdf:resource="http://rdfs.org/sioc/ns#User"/>
</rdf:Description>
</rdf:RDF>
]]></programlisting>
        <para><emphasis>Example 2:</emphasis></para>
        <para>Create a DAV collection, for ex. with name "test" for user ( for ex. demo).</para>
        <para>Execute the following command:</para>
<programlisting><![CDATA[
curl -i -d "INSERT IN GRAPH <http://mygraph.com>
{ <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  <http://rdfs.org/sioc/ns#User> .
  <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
  <http://www.w3.org/2000/01/rdf-schema#label>
  <Kingsley Uyi Idehen> .
  <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
  <http://rdfs.org/sioc/ns#creator_of>

<http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300>
  } " -u "demo:demo" -H "Content-Type: application/sparql-query" http://example.com/DAV/home/demo/test/myrq
]]></programlisting>
        <para>As result the response will be:</para>
<programlisting><![CDATA[
HTTP/1.1 201 Created
Server: Virtuoso/05.00.3023 (Win32) i686-generic-win-32  VDB
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
Date: Thu, 20 Dec 2007 16:25:25 GMT
Accept-Ranges: bytes
MS-Author-Via: SPARQL
Content-Length: 0
]]></programlisting>
        <para>Now let's check the inserted triples. Go to the sparql endpoint, i.e. http://example.com/sparql and:</para>
<orderedlist>
<listitem>Enter for Default Graph URI:
<programlisting><![CDATA[
http://mygraph.com
]]></programlisting>
   </listitem>
<listitem>Enter in the Query area:
<programlisting><![CDATA[
SELECT * WHERE {?s ?p ?o}
]]></programlisting>
   </listitem>
<listitem>Click the button "Run Query"</listitem>
<listitem>As result will be shown the inserted triples:
<programlisting><![CDATA[
s  	                                                          p                                                 o
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this   http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://rdfs.org/sioc/ns#User
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this   http://www.w3.org/2000/01/rdf-schema#label 	      Kingsley
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this   http://rdfs.org/sioc/ns#creator_of                http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300
]]></programlisting>
   </listitem>
</orderedlist>
</sect2>
     <sect2 id="rdfinsertmethodshttpput"><title>HTTP PUT using Content-Type: application/rdf+xml</title>
	<para>The URI in a PUT request identifies the entity enclosed with the request. Therefore using HTTP PUT is a more useful and meaningful command than using POST (which is more about submitting data to a script).</para>
        <para><emphasis>Example:</emphasis></para>
        <para>Suppose there is myfoaf.rdf file with the following content:</para>
<programlisting><![CDATA[
<rdf:RDF xmlns="http://www.example/jose/foaf.rdf#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:log="http://www.w3.org/2000/10/swap/log#"
    xmlns:myfoaf="http://www.example/jose/foaf.rdf#"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#jose">
        <foaf:homepage rdf:resource="http://www.example/jose/"/>
        <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#juan"/>
        <foaf:name>Jose Jimen~ez</foaf:name>
        <foaf:nick>Jo</foaf:nick>
        <foaf:workplaceHomepage rdf:resource="http://www.corp.example/"/>
    </foaf:Person>

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#juan">
        <foaf:mbox rdf:resource="mailto:juan@mail.example"/>
    </foaf:Person>

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#julia">
        <foaf:mbox rdf:resource="mailto:julia@mail.example"/>
    </foaf:Person>

    <rdf:Description rdf:about="http://www.example/jose/foaf.rdf#kendall">
        <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#edd"/>
    </rdf:Description>
</rdf:RDF>
]]></programlisting>
        <para>Now let's upload the myfoaf.rdf file to destination server demo.openlinksw.com for user demo:</para>
<programlisting><![CDATA[
curl -T myfoaf.rdf http://demo.openlinksw.com/DAV/home/demo/rdf_sink/myfoaf.rdf -u demo:demo
]]></programlisting>
        <para>As result the response should be:</para>
<programlisting><![CDATA[
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML>
<HEAD>
  <TITLE>201 Created</TITLE>
</HEAD>
<BODY>
  <H1>Created</H1>
  Resource /DAV/home/demo/rdf_sink/ myfoaf.rdf has been created.
</BODY>
</HTML>
]]></programlisting>
    <para>Then you can execute:</para>
<programlisting><![CDATA[
curl -F "query=SELECT DISTINCT ?p FROM <http://demo.openlinksw.com/DAV/home/demo/rdf_sink/> WHERE {?s ?p ?o}" http://demo.openlinksw.com/sparql
]]></programlisting>
    <para>The result should be:</para>
<programlisting><![CDATA[
<?xml version="1.0" ?>
<sparql xmlns="http://www.w3.org/2005/sparql-results#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2001/sw/DataAccess/rf1/result2.xsd">
 <head>
  <variable name="p"/>
 </head>
 <results distinct="false" ordered="true">
  <result>
   <binding name="p"><uri>http://www.w3.org/1999/02/22-rdf-syntax-ns#type</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/nick</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/name</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/homepage</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/knows</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/workplaceHomepage</uri></binding>
  </result>
  <result>
   <binding name="p"><uri>http://xmlns.com/foaf/0.1/mbox</uri></binding>
  </result>
 </results>
</sparql>
]]></programlisting>
<para>Other examples with curl:</para>
<programlisting><![CDATA[
curl -F "query=SELECT distinct ?Concept FROM <http://dbpedia.org> WHERE {?s a ?Concept} limit 10" http://dbpedia.org/sparql
]]></programlisting>
<programlisting><![CDATA[
curl -F "query=SELECT distinct ?Concept FROM <http://example.com/dataspace/person/kidehen> WHERE {?s a ?Concept} limit 10" http://demo.openlinksw.com/sparql
]]></programlisting>
<programlisting><![CDATA[
curl -F "query=SELECT distinct ?Concept FROM <http://data.openlinksw.com/oplweb/product_family/virtuoso> WHERE {?s a ?Concept} limit 10" http://demo.openlinksw.com/sparql
]]></programlisting>
<programlisting><![CDATA[
curl -F "query=SELECT distinct ?Concept FROM <http://openlinksw.com/dataspace/organization/openlink> WHERE {?s a ?Concept} limit 10" http://demo.openlinksw.com/sparql
]]></programlisting>
    </sect2>
     <sect2 id="rdfinsertmethodsload"><title>SPARQL Insert using LOAD</title>
        <para>SPARQL INSERT operation can be done using the LOAD features:</para>
<programlisting><![CDATA[        
	
SPARQL INSERT INTO <..> { .... } [[FROM ...] { ... }]
	
SPARQL LOAD <x> [INTO <y>]

-- <ResourceURL> will be the Graph IRI of the loaded data:
SPARQL LOAD <ResourceURL> 
	
]]></programlisting> 
        <para>Examples:</para>               
<orderedlist>
   <listitem>Load from ISQL:
<programlisting><![CDATA[
SPARQL insert in graph <http://mygraph.com>
    {
  <http://example.com/dataspace/Kingsley#this>
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  <http://rdfs.org/sioc/ns#User> .

  <http://example.com/dataspace/Kingsley#this>
  <http://rdfs.org/sioc/ns#id>
  <Kingsley> .

  <http://example.com/dataspace/Caroline#this>
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  <http://rdfs.org/sioc/ns#User> .

 <http://example.com/dataspace/Caroline#this>
  <http://rdfs.org/sioc/ns#id>
  <Caroline> .

   <http://example.com/dataspace/Matt#this>
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  <http://rdfs.org/sioc/ns#User> .

  <http://example.com/dataspace/Matt#this>
  <http://rdfs.org/sioc/ns#id>
  <Matt> .

   <http://example.com/dataspace/demo#this>
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
  <http://rdfs.org/sioc/ns#User> .

  <http://example.com/dataspace/demo#this>
  <http://rdfs.org/sioc/ns#id>
  <demo> .};
]]></programlisting>
   </listitem>
   <listitem>Load from .rq file:
     <orderedlist>
       <listitem>Create DAV collection which is visible to public, for ex: http://example.com/DAV/tmp</listitem>
       <listitem>Upload to the DAV collection the following file for ex. with name listall.rq and with the following content:
<programlisting><![CDATA[
SPARQL
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
SELECT ?x ?p ?o
FROM <http://mygraph.com>
WHERE
       {
    ?x rdf:type sioc:User .
    ?x ?p ?o.
    ?x sioc:id ?id .
    FILTER REGEX(str(?id), "^King")
  }
ORDER BY ?x
]]></programlisting>
       </listitem>
       <listitem>Execute from ISQL the following command:
<programlisting><![CDATA[
SQL>SPARQL
load bif:concat ("http://", bif:registry_get("URIQADefaultHost"), "/DAV/tmp/listall.rq") into graph <http://myNewGraph.com>;
]]></programlisting>
        <para>As result should be shown:</para>
<programlisting><![CDATA[
callret-0
VARCHAR
_______________________________________________________________________________

Load <http://example.com/DAV/tmp/listall.rq> into graph <http://myNewGraph.com> -- done

1 Rows. -- 321 msec.
]]></programlisting>
       </listitem>       
     </orderedlist>	
   </listitem>
   <listitem>Load from Resource URL:
<programlisting><![CDATA[   	
SQL> SPARQL LOAD <http://www.w3.org/People/Berners-Lee/card#i>;
callret-0
VARCHAR
_______________________________________________________________________________

Load <http://www.w3.org/People/Berners-Lee/card#i> into graph <http://www.w3.org/People/Berners-Lee/card#i> -- done

1 Rows. -- 703 msec.
SQL>

]]></programlisting>	
   </listitem>
</orderedlist>
    </sect2>
     <sect2 id="rdfindertmethodsparqlendpoint"><title>SPARQL Insert via /sparql endpoint</title>
        <para>SPARQL INSERT operation can be sent to a web service endpoint as a single statement and executed in sequence.</para>
        <para><emphasis>Example:</emphasis></para>
        <para>Using the Virtuoso ISQL tool or using the /sparql UI at http://host:port/sparql, execute the following:</para>
<orderedlist>
<listitem>Insert into graph http://BookStore.com 3 triples:
<programlisting><![CDATA[
SQL>SPARQL insert in graph <http://BookStore.com>
{ <http://www.dajobe.org/foaf.rdf#i> <http://purl.org/dc/elements/1.1/date> <1999-04-01T00:00:00> .
  <http://www.w3.org/People/Berners-Lee/card#i> <http://purl.org/dc/elements/1.1/date> <1998-05-03T00:00:00> .
  <http://www.w3.org/People/Connolly/#me> <http://purl.org/dc/elements/1.1/date> <2001-02-08T00:00:00> };
]]></programlisting>
   </listitem>
<listitem>As result will be shown the message:
<programlisting><![CDATA[
SQL>Insert into <http://BookStore.com>
3 triples -- done
]]></programlisting>
   </listitem>
<listitem>Next we will select all triples from the graph http://BookStore.com:
<programlisting><![CDATA[
SQL>SPARQL SELECT * FROM <http://BookStore.com> WHERE {?s ?p ?o};
]]></programlisting>
   </listitem>
<listitem>As result will be shown:
<programlisting><![CDATA[
s                                              p                                       o
VARCHAR                                        VARCHAR                                 VARCHAR
_______________________________________________________________________________

http://www.w3.org/People/Berners-Lee/card#i    http://purl.org/dc/elements/1.1/date    1998-05-03T00:00:00
http://www.w3.org/People/Connolly/#me          http://purl.org/dc/elements/1.1/date    2001-02-08T00:00:00
http://www.dajobe.org/foaf.rdf#i               http://purl.org/dc/elements/1.1/date    1999-04-01T00:00:00

3 Rows. -- 0 msec.

]]></programlisting>
   </listitem>
<listitem>Now let's insert into graph another http://NewBookStore.com graph's values:
<programlisting><![CDATA[
SQL>SPARQL
PREFIX dc:  <http://purl.org/dc/elements/1.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
INSERT INTO GRAPH <http://NewBookStore.com> { ?book ?p ?v }
WHERE
  { GRAPH  <http://BookStore.com>
   { ?book dc:date ?date
     FILTER ( xsd:dateTime(?date) < xsd:dateTime("2000-01-01T00:00:00")).
     ?book ?p ?v.
   }
  };
]]></programlisting>
   </listitem>
<listitem>As result will be shown:
<programlisting><![CDATA[
callret-0
VARCHAR
_______________________________________________________________________________

Insert into <http://NewBookStore.com>, 2 triples -- done
]]></programlisting>
   </listitem>
<listitem>Finally we will check the triples from the graph NewBookStore.com:
<programlisting><![CDATA[
SQL> SPARQL
SELECT *
FROM <http://NewBookStore.com>
WHERE {?s ?p ?o};
]]></programlisting>
   </listitem>
<listitem>As result will be shown:
<programlisting><![CDATA[
s                                             p                                      o
VARCHAR                                       VARCHAR                                VARCHAR
_______________________________________________________________________________

http://www.w3.org/People/Berners-Lee/card#i   http://purl.org/dc/elements/1.1/date   1998-05-03T00:00:00
http://www.dajobe.org/foaf.rdf#i              http://purl.org/dc/elements/1.1/date   1999-04-01T00:00:00

2 Rows. -- 10 msec.
]]></programlisting>
   </listitem>
</orderedlist>
</sect2>
     <sect2 id="rdfinsertmethodsparqlqueryandodswiki"><title>SPARQL Insert via SPARQL endpoint REST API and ODS wiki</title>
        <para>With HTTP Post and ODS wiki can be written an rdf document and respectively to be performed over it INSERT/UPDATE action.</para>
        <para>You can write to a file using SIOC terms for ODS-Wiki</para>
        <para>You can check with sparql the inserted / updated triples in the Quad Store.</para>
        <para><emphasis>Example:</emphasis></para>
        <para>Suppose there is ODS user test3 with ODS password 1, which has testWiki wiki instance.</para>
        <para>Execute the following:</para>
<programlisting><![CDATA[
curl -i -d "INSERT {<http://example.com/dataspace/test3/wiki/testWiki> <http://atomowl.org/ontologies/atomrdf#contains> <http://example.com/dataspace/test3/wiki/testWiki/MyTest> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#has_container> <http://example.com/dataspace/test3/wiki/testWiki> . <http://example.com/dataspace/test3/wiki/testWiki> <http://atomowl.org/ontologies/atomrdf#entry> <http://example.com/dataspace/test3/wiki/testWiki/MyTest> . <http://example.com/dataspace/test3/wiki/testWiki> <http://rdfs.org/sioc/ns#container_of> <http://example.com/dataspace/test3/wiki/testWiki/MyTest> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#topic>  <http://example.com/dataspace/test3/wiki/testWiki> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://atomowl.org/ontologies/atomrdf#source> <http://example.com/dataspace/test3/wiki/testWiki> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://rdfs.org/sioc/types#Comment> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://atomowl.org/ontologies/atomrdf#Entry> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/2000/01/rdf-schema#label> 'MyTest' . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://atomowl.org/ontologies/atomrdf#Link> . <http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#content> <test>}" -u "test3:1" -H "Content-Type: application/sparql-query" http://example.com/DAV/home/test3/wiki/testWiki/MyTest
]]></programlisting>
        <para>As result we should have 2 files created:</para>
<itemizedlist>
<listitem>In the user DAV folder "DAV/home/test3/wiki/testWiki/" will be created a file "MyTest" with type "application/sparql-query". You can view the content of this file from the Conductor UI or from the user's Briefcase UI, path "DAV/home/test3/wiki/testWiki". Its content will be:
<programlisting><![CDATA[
<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki"><ns0pred:entry xmlns:ns0pred="http://atomowl.org/ontologies/atomrdf#" rdf:resource="http://example.com/dataspace/test3/wiki/testWiki/MyTest"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:label xmlns:ns0pred="http://www.w3.org/2000/01/rdf-schema#">MyTest</ns0pred:label></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#" rdf:resource="http://atomowl.org/ontologies/atomrdf#Link"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#" rdf:resource="http://rdfs.org/sioc/types#Comment"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:type xmlns:ns0pred="http://www.w3.org/1999/02/22-rdf-syntax-ns#" rdf:resource="http://atomowl.org/ontologies/atomrdf#Entry"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:has_container xmlns:ns0pred="http://rdfs.org/sioc/ns#" rdf:resource="http://example.com/dataspace/test3/wiki/testWiki"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki"><ns0pred:container_of xmlns:ns0pred="http://rdfs.org/sioc/ns#" rdf:resource="http://example.com/dataspace/test3/wiki/testWiki/MyTest"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki"><ns0pred:contains xmlns:ns0pred="http://atomowl.org/ontologies/atomrdf#" rdf:resource="http://example.com/dataspace/test3/wiki/testWiki/MyTest"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:content xmlns:ns0pred="http://rdfs.org/sioc/ns#">test</ns0pred:content></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:topic xmlns:ns0pred="http://rdfs.org/sioc/ns#" rdf:resource="http://example.com/dataspace/test3/wiki/testWiki"/></rdf:Description>
<rdf:Description rdf:about="http://example.com/dataspace/test3/wiki/testWiki/MyTest"><ns0pred:source xmlns:ns0pred="http://atomowl.org/ontologies/atomrdf#" rdf:resource="http://example.com/dataspace/test3/wiki/testWiki"/></rdf:Description>
</rdf:RDF>
]]></programlisting>
   </listitem>
<listitem>To the user's wiki instance will be added a new WikiWord "MyTest" with content the value of the SIOC term attribute "content":
<programlisting><![CDATA[
<http://example.com/dataspace/test3/wiki/testWiki/MyTest> <http://rdfs.org/sioc/ns#content> <test>
i.e. the content will be "test".
]]></programlisting>
   </listitem>
</itemizedlist>
        <para>Now let's check what data was inserted in the Quad Store:</para>
<orderedlist>
<listitem>Go to the sparql endpoint, i.e. for ex. to http://example.com/sparql</listitem>
<listitem>Enter for Default Graph URI:
<programlisting><![CDATA[
http://example.com/DAV/home/test3/wiki/testWiki/MyTest
]]></programlisting>
   </listitem>
<listitem>Enter for Query text:
<programlisting><![CDATA[
SELECT * WHERE {?s ?p ?o}
]]></programlisting>
   </listitem>
<listitem>Click the "Run Query" button.</listitem>
<listitem>As result will be shown the inserted triples:
<programlisting><![CDATA[
s  	                                                    p  	                                             o
http://example.com/dataspace/test3/wiki/testWiki 	    http://rdfs.org/sioc/ns#container_of 	     http://example.com/dataspace/test3/wiki/testWiki/MyTest
http://example.com/dataspace/test3/wiki/testWiki 	    http://atomowl.org/ontologies/atomrdf#entry      http://example.com/dataspace/test3/wiki/testWiki/MyTest
http://example.com/dataspace/test3/wiki/testWiki 	    http://atomowl.org/ontologies/atomrdf#contains   http://example.com/dataspace/test3/wiki/testWiki/MyTest
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://rdfs.org/sioc/types#Comment
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://atomowl.org/ontologies/atomrdf#Entry
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://atomowl.org/ontologies/atomrdf#Link
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://www.w3.org/2000/01/rdf-schema#label 	     MyTest
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://rdfs.org/sioc/ns#has_container 	     http://example.com/dataspace/test3/wiki/testWiki
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://rdfs.org/sioc/ns#content 	             test
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://rdfs.org/sioc/ns#topic 	             http://example.com/dataspace/test3/wiki/testWiki
http://example.com/dataspace/test3/wiki/testWiki/MyTest  http://atomowl.org/ontologies/atomrdf#source     http://example.com/dataspace/test3/wiki/testWiki
]]></programlisting>
   </listitem>
</orderedlist>
    </sect2>
    <sect2 id="rdfinsertmethodwebdav">
      <title>Using WebDAV</title>
      <para> Example using WebDAV (mount folder to DAV and dump; if this is the rdf_sink
the Quad Store is updated automatically, or you can load from DAV manually to quad store)</para>
      <para><emphasis>Example:</emphasis></para>
      <para><emphasis>Example 1: Using ODS Briefcase</emphasis></para>
      <orderedlist>
        <listitem>Go to your ods location, for ex. http://example.com/ods</listitem>
        <listitem>Register user, for ex. user test1</listitem>
        <listitem>Login if not already in ods</listitem>
        <listitem>Go to ODS -> Briefcase</listitem>
        <listitem>Go to ODS -> Briefcase</listitem>
        <listitem>Click the "New folder" icon from the Main Briefcase horizontal navigation</listitem>
        <listitem>Enter for name for ex. "mytest" and click the "Create" button.
          <figure id="uc6" float="1">
	    <title>Using Briefcase UI</title>
	    <graphic fileref="ui/uc6.png"/>
     </figure>
   </listitem>
        <listitem>Go to folder "mytest" and click the click the "Upload" icon from the Main Briefcase horizontal navigation</listitem>
        <listitem>Enter for name for ex. "mytest" and click the "Create" button.
          <figure id="uc7" float="1">
	    <title>Using Briefcase UI</title>
	    <graphic fileref="ui/uc7.png"/>
     </figure>
</listitem>
        <listitem>In the shown form set:
          <itemizedlist>
            <listitem>Destination: RDF Store</listitem>
            <listitem>RDF graph name for ex. with the value: http://example.com/DAV/home/test2/mytest/</listitem>
            <listitem>Select URL or File. For ex. you can select the following file with name jose.rdf:
<programlisting><![CDATA[
<rdf:RDF xmlns="http://www.example/jose/foaf.rdf#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:log="http://www.w3.org/2000/10/swap/log#"
    xmlns:myfoaf="http://www.example/jose/foaf.rdf#"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#jose">
        <foaf:homepage rdf:resource="http://www.example/jose/"/>
        <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#juan"/>
        <foaf:name>Jose Jimen~ez</foaf:name>
        <foaf:nick>Jo</foaf:nick>
        <foaf:workplaceHomepage rdf:resource="http://www.corp.example/"/>
    </foaf:Person>

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#juan">
        <foaf:mbox rdf:resource="mailto:juan@mail.example"/>
    </foaf:Person>

    <foaf:Person rdf:about="http://www.example/jose/foaf.rdf#julia">
        <foaf:mbox rdf:resource="mailto:julia@mail.example"/>
    </foaf:Person>

    <rdf:Description rdf:about="http://www.example/jose/foaf.rdf#kendall">

        <foaf:knows rdf:resource="http://www.example/jose/foaf.rdf#edd"/>
    </rdf:Description>
</rdf:RDF>
]]></programlisting>
            </listitem>
</itemizedlist>
</listitem>
        <listitem>You can also perform the steps from above by uploading the file in the rdf_sink
folder i.e. in Briefcase it will be with this path: DAV/home/test2/rdf_sink and respectively the "RDF graph name"
will have this value: http://host:port/DAV/home/username/rdf_sink/</listitem>
      </orderedlist>
      <para>Execute from ISQL or from the SPARQL endpoint the following query:</para>
<programlisting><![CDATA[
SELECT * FROM <http://example.com/DAV/home/test2/mytest/>
WHERE {?s ?p ?o}
]]></programlisting>
      <para>As result should be shown:</para>
<programlisting><![CDATA[
s  	                                  p  	                                            o
http://www.example/jose/foaf.rdf#jose 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/nick 	            Jo
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/name 	            Jose Jimen~ez
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/knows 	            http://www.example/jose/foaf.rdf#juan
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/homepage 	            http://www.example/jose/
http://www.example/jose/foaf.rdf#jose 	  http://xmlns.com/foaf/0.1/workplaceHomepage 	    http://www.corp.example/
http://www.example/jose/foaf.rdf#kendall  http://xmlns.com/foaf/0.1/knows 	            http://www.example/jose/foaf.rdf#edd
http://www.example/jose/foaf.rdf#julia 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#julia 	  http://xmlns.com/foaf/0.1/mbox 	            mailto:julia@mail.example
http://www.example/jose/foaf.rdf#juan 	  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://xmlns.com/foaf/0.1/Person
http://www.example/jose/foaf.rdf#juan 	  http://xmlns.com/foaf/0.1/mbox 	            mailto:juan@mail.example
]]></programlisting>
      <para><emphasis>Example 2: Using Conductor UI</emphasis></para>
      <orderedlist>
        <listitem>Go to Conductor UI, for ex. at http://example.com/conductor</listitem>
        <listitem>Login as dba user</listitem>
        <listitem>Go to Linked Data -&gt; Quad Store Upload
          <figure id="uc1" float="1">
	    <title>Quad Store Upload</title>
	    <graphic fileref="ui/uc1.png"/>
          </figure>
  </listitem>
        <listitem>In the shown form click the "Browse" button in order to select a file, for ex. 
        	the file jose.rdf and set the "RDF IRI*"
          <figure id="uc2" float="1">
	    <title>Quad Store Upload</title>
	    <graphic fileref="ui/uc2.png"/>
          </figure>
  </listitem>
         <listitem>Click the "Upload" button.
          <figure id="uc3" float="1">
	    <title>Quad Store Upload</title>
	    <graphic fileref="ui/uc3.png"/>
          </figure>         	
         	</listitem>
      </orderedlist>
    </sect2>
    <sect2 id="rdfinsertmethodvirtuosocrawler">
      <title>Using Virtuoso Crawler</title>
      <para>Using Virtuoso Crawler (which includes the Sponger options so you crawl
non-RDF but get RDF and this can go to the Quad Store).</para>
      <para><emphasis>Example:</emphasis></para>
      <orderedlist>
        <listitem>Go to Conductor UI. For ex. at http://example.com/conductor :
          <figure id="rdfinsertwebdav11" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert11.png"/>
          </figure>
        </listitem>
        <listitem>Enter admin user credentials:
          <figure id="rdfinsertwebdav12" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert12.png"/>
          </figure>
        </listitem>
        <listitem>Go to tab Web Application Server:
          <figure id="rdfinsertwebdav13" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert13.png"/>
          </figure>
        </listitem>
        <listitem>Go to tab Content Imports:
          <figure id="rdfinsertwebdav14" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert14.png"/>
          </figure>
        </listitem>
        <listitem>Click the "New Target" button:
          <figure id="rdfinsertwebdav15" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert15.png"/>
          </figure>
        </listitem>
        <listitem>In the shown form set respectively:
          <orderedlist>
            <listitem>"Crawl Job Name": Tim Berners-Lee's electronic Business Card ;</listitem>
            <listitem>"Data Source Address (URL)": http://www.w3.org/People/Berners-Lee/ ;</listitem>
            <listitem>"Local WebDAV Identifier" for ex.: /DAV/home/demo/my-crawling/ ;</listitem>
            <listitem>Choose from the list "Local resources owner": demo ;</listitem>
            <listitem>Leave checked by default the check-box "Store documents locally". -- Note: if "Store document locally" is not checked, then in this case no documents will be save as DAV resource and the specified DAV folder from above will not be used ;</listitem>
            <listitem>Check the check-box with label "Store metadata" ; </listitem>
            <listitem>Specify which cartridges to be involved by hatching their check-box ;</listitem>
        <listitem>Note: when selected "Convert Link", then all HREFs in the local stored content will be relative.</listitem>
          </orderedlist>
          <figure id="rdfinsertwebdav16" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert16.png"/>
          </figure>
          <figure id="rdfinsertwebdav17" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert17.png"/>
          </figure>
          <figure id="rdfinsertwebdav18" float="1">
	    <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert18.png"/>
          </figure>
        </listitem>
        <listitem>Click the button "Create":
          <figure id="rdfinsertwebdav19" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert19.png"/>
          </figure>
        </listitem>
        <listitem>Click the button "Import Queues":
          <figure id="rdfinsertwebdav20" float="1">
	          <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert20.png"/>
          </figure>
        </listitem>
        <listitem>For "Robot target" with label "Tim Berners-Lee's electronic Business Card"
click "Run".</listitem>
        <listitem>As result should be shown the number of the pages retrieved.
      <figure id="rdfinsertwebdav2" float="1">
        <title>Using Virtuoso Crawler</title>
	<graphic fileref="ui/rdfinsert2.png"/>
      </figure>
        </listitem>
        <listitem>To view the crawled data, go for instance to public SPARQL Endpoint ex. http://host:port/sparql:
          <figure id="rdfinsertwebdav21" float="1">
            <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert21.png"/>
          </figure>
        </listitem>
        <listitem>Set respectively a Default Graph URI: &lt;http://www.w3.org/People/Berners-Lee/&gt; and execute the following query:
<programlisting><![CDATA[
SELECT *
FROM <http://www.w3.org/People/Berners-Lee/>
WHERE
  {
    ?s ?p ?o
  }
LIMIT 10
]]></programlisting>
          <figure id="rdfinsertwebdav22" float="1">
            <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert22.png"/>
          </figure>
          <figure id="rdfinsertwebdav23" float="1">
            <title>Using Virtuoso Crawler</title>
	          <graphic fileref="ui/rdfinsert23.png"/>
          </figure>
        </listitem>
      </orderedlist>      
    <para><emphasis>Example: Use of schedular to interface Virtuoso Quad Store with PTSW using the following program:</emphasis></para>
<programlisting><![CDATA[


create procedure PTSW_CRAWL ()
{
  declare xt, xp any;
  declare content, headers any;

  content := http_get ('http://pingthesemanticweb.com/export/', headers);
  xt := xtree_doc (content);
  xp := xpath_eval ('//rdfdocument/@url', xt, 0);
  foreach (any x in xp) do
    {
      x := cast (x as varchar);
      dbg_obj_print (x);
      {
	declare exit handler for sqlstate '*' {
	  log_message (sprintf ('PTSW crawler can not load : %s', x));
	};
        sparql load ?:x into graph ?:x;
	update DB.DBA.SYS_HTTP_SPONGE set HS_LOCAL_IRI = x, HS_EXPIRATION = null WHERE HS_LOCAL_IRI = 'destMD5=' || md5 (x) || '&graphMD5=' || md5 (x);
	commit work;
      }
    }
}
;

insert soft SYS_SCHEDULED_EVENT (SE_SQL, SE_START, SE_INTERVAL, SE_NAME)
	values ('DB.DBA.PTSW_CRAWL ()', cast (stringtime ('0:0') as DATETIME), 60, 'PTSW Crawling');
]]></programlisting>
      <tip><title>See Also:</title>
        <para><link linkend="contentcrawlerrdf">Other Methods to Set Up the Content Crawler for RDF gathering.</link></para>
      </tip>
    </sect2>
    <sect2 id="rdfinsertmethodsparqlqueryandsponger">
      <title>Using SPARQL Query and Sponger (i.e. we Fetch the Network Resources in the FROM Clause or values for the graph-uri parameter in SPARQL protocol URLs)</title>
      <para><emphasis>Example:</emphasis></para>
      <para>Execute the following query: </para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
 OPTION (get:soft "soft", get:method "GET")
WHERE { GRAPH ?g { ?id a ?o } }
limit 10;
]]></programlisting>
      <para>As result will be shown the retrieved triples:</para>
<programlisting><![CDATA[
id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/612
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/612
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/610
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/610
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/856
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/856

10 Rows. -- 20 msec.
]]></programlisting>
    </sect2>
    <sect2 id="rdfinsertmethodplapis">
      <title>Using Virtuoso PL APIs</title>
      <sect3 id="rdfinsertmethodplapissimpleexample"><title>Basic Sponger Cartridge Example</title>
      <para>In the example script we implement a basic mapper which maps a text/plain mime type
to an imaginary ontology, which extends the class Document from FOAF with properties 'txt:UniqueWords'
and 'txt:Chars', where the prefix 'txt:' we specify as 'urn:txt:v0.0:'.</para>
<programlisting><![CDATA[
use DB;

create procedure DB.DBA.RDF_LOAD_TXT_META
	(
	 in graph_iri varchar,
	 in new_origin_uri varchar,
	 in dest varchar,
         inout ret_body any,
	 inout aq any,
	 inout ps any,
	 inout ser_key any
	 )
{
  declare words, chars int;
  declare vtb, arr, subj, ses, str any;
  declare ses any;
  -- if any error we just say nothing can be done
  declare exit handler for sqlstate '*'
    {
      return 0;
    };
  subj := coalesce (dest, new_origin_uri);
  vtb := vt_batch ();
  chars := length (ret_body);
  -- using the text index procedures we get a list of words
  vt_batch_feed (vtb, ret_body, 1);
  arr := vt_batch_strings_array (vtb);
  -- the list has 'word' and positions array , so we must divide by 2
  words := length (arr) / 2;
  ses := string_output ();
  -- we compose a N3 literal
  http (sprintf ('<%s> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Document> .\n', subj), ses);
  http (sprintf ('<%s> <urn:txt:v0.0:UniqueWords> "%d" .\n', subj, words), ses);
  http (sprintf ('<%s> <urn:txt:v0.0:Chars> "%d" .\n', subj, chars), ses);
  str := string_output_string (ses);
  -- we push the N3 text into the local store
  DB.DBA.TTLP (str, new_origin_uri, subj);
  return 1;
}
;

--
DELETE FROM DB.DBA.SYS_RDF_MAPPERS WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';

INSERT SOFT DB.DBA.SYS_RDF_MAPPERS (RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION)
    VALUES ('(text/plain)', 'MIME', 'DB.DBA.RDF_LOAD_TXT_META', null, 'Text Files (demo)');

-- here we set order to some large number so don't break existing mappers
update DB.DBA.SYS_RDF_MAPPERS set RM_ID = 2000 WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';
]]></programlisting>

<orderedlist>
  <listitem>Paste the whole of this code into Conductor's iSQL interface and execute it to 
  	define and register the cartridge.</listitem>
  <listitem>Create a simple text document with a .txt extension. For ex. with name: summary.txt</listitem>
  <listitem>The .txt file must now be made Web accessible. A simple way to do this is to expose it as a WebDAV 
  	resource using Virtuoso's built-in WebDAV support:
    <orderedlist>
      <listitem>Log in to Virtuoso's ODS Briefcase application;</listitem>
      <listitem>Navigate to your Public folder;</listitem>
      <listitem>Upload your text document, ensuring that the file extension is .txt, the MIME type is set to text/plain and the file permissions are rw-r--r--.</listitem>
      <listitem>As result the file would be Web accessible via the URL http://cname/DAV/home/username/Public/summary.txt .</listitem>
      <listitem>Note: you can also check our <ulink url="http://demo.openlinksw.com:8890/tutorial/hosting/ho_s_30/WebCalendar/tools/summary.txt">live demo</ulink>.</listitem>
    </orderedlist>
  </listitem>
  <listitem>To test the mapper we just use /sparql endpoint with option 'Retrieve remote
RDF data for all missing source graphs' to execute (for ex.):
<programlisting><![CDATA[
SELECT *
FROM <http://cname/DAV/home/username/Public/summary.txt>
WHERE {?s ?p ?o}
]]></programlisting>
  </listitem>
        <listitem>Click the "Run Query" button.</listitem>
  <listitem>As result should be shown the found triples, for ex.:
<programlisting><![CDATA[
s  	                                                                    p  	                                            o
http://cname/DAV/home/username/Public/summary.txt  http://www.w3.org/1999/02/22-rdf-syntax-ns#type  http://xmlns.com/foaf/0.1/Document
http://cname/DAV/home/username/Public/summary.txt  urn:txt:v0.0:UniqueWords 	                      47
http://cname/DAV/home/username/Public/summary.txt  urn:txt:v0.0:Chars 	                            625
]]></programlisting>
  </listitem>
</orderedlist>
      <para><emphasis>Important: Setting Sponger Permissions</emphasis></para>
      <para>In order to allow the Sponger to update the local RDF quad store with triples
constituting the fetched Network Resource structured data, the role "SPARQL_SPONGE" must be granted to the
account "SPARQL", i.e., to the owner account of /sparql web service endpoint.
This should normally be the case. If not, you must manually grant this
permission. As with most Virtuoso DBA tasks, the Conductor provides the simplest means of
doing this.</para>
  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem>The <link linkend="fn_rdf_load_rdfxml">DB.DBA.RDF_LOAD_RDFXML</link> function to
parse the content of RDF/XML text.</listitem>
      <listitem>The <link linkend="fn_ttlp_mt">DB.DBA.TTLP_MT</link> function to
parse TTL (TURTLE or N3 resource).</listitem>
      <listitem>The <link linkend="fn_gz_file_open">gz_file_open</link> function to
retrieve content of a gzipped file and example for loading gzipped N3 and Turtle files.</listitem>
    </itemizedlist>
  </tip>
      </sect3>
    </sect2>
    <sect2 id="rdfinsertmethodsimilerdfbankapi">
      <title>Using SIMILE RDF Bank API</title>
      <para>Virtuoso implements the HTTP-based Semantic Bank API that enables client
applications to post to its RDF Triple Store. This method offers an alternative to
using Virtuoso/PL functions or WebDAV uploads as the triples-insertion mechanism.</para>
      <para><emphasis>Example:</emphasis></para>
      <para>From your machine go to Firefox->Tools->PiggyBank->My Semantic Bank Accounts</para>
      <para>Add in the shown form:</para>
      <itemizedlist>
        <listitem>For bank: address: http://demo.openlinksw.com/bank</listitem>
        <listitem>For account id: demo</listitem>
        <listitem>For password: demo</listitem>
      </itemizedlist>
      <para>Go to http://demo.openlinksw.com/ods</para>
      <para>Log in as user demo, password: demo</para>
      <para>Go to the Weblog tab from the main ODS Navigation</para>
      <para>Click on weblog instance name, for ex. "demo's Weblog".</para>
      <para>When the weblog home page is loaded, click Alt + P.</para>
      <para>As result is shown the "My PiggyBank" page with all the collected information
presented in items.</para>
      <para>For several of the items add Tags from the form "Tag" shown for each of them.</para>
      <para>As result should be shown the message "Last updated: [here goes the date value].</para>
      <para>You can also click "Save" and "Publish" for these items.</para>
      <para>Go to http://demo.openlinksw.com/sparql</para>
      <para>Enter for the "Default Graph URI" field: http://simile.org/piggybank/demo</para>
      <para>Enter for the "Query text" text-area:</para>
<programlisting><![CDATA[
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix sioc: <http://rdfs.org/sioc/ns#>
SELECT *
FROM <http://simile.org/piggybank/demo>
WHERE {?s ?p  ?o}
]]></programlisting>
      <para>Click "Run Query".</para>
      <para>As results are shown the found results.</para>
    </sect2>
    <sect2 id="rdfinsertmethodrdfnet">
      <title>Using RDF NET</title>
      <para><emphasis>Example:</emphasis></para>
      <para>Execute the following query:</para>
<programlisting><![CDATA[
SQL> SELECT DB.DBA.HTTP_RDF_NET ('sparql load
"http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com"
into graph <http://www.openlinksw.com/>');
]]></programlisting>
      <para>As result should be shown:</para>
<programlisting><![CDATA[
callret
VARCHAR
_______________________________________________________

<?xml version="1.0" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:vcard="http://www.w3.org/2001/vcard-rdf/3.0#"
xmlns="http://example.org/book/" xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:ns="http://example.org/ns#">
<rdf:Description>
<callret-0>Load <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com>
into graph <http://www.openlinksw.com/> -- done</callret-0>
</rdf:Description>
</rdf:RDF>

1 Rows. -- 1982 msec.
]]></programlisting>
    </sect2>
    <sect2 id="rdfinsertmethodproxy">
      <title>Using the RDF Proxy (Sponger) Service</title>
    <para>Triples can be inserted also using the Sponger Proxy URI Service. For more information 
    	and examples see <link linkend="rdfproxyservice">here</link>.</para>
    </sect2>
</sect1>


<sect1 id="virtuososponger"><title>RDFizer Middleware (Sponger)</title>
    <sect2 id="virtuosospongerintro"><title>What Is The Sponger?</title>
<para>The Virtuoso Sponger is the Linked Data middleware component of Virtuoso that generates 
	Linked Data from a variety of data sources, supporting a wide variety of data representation 
	and serialization formats. The sponger is transparently integrated into Virtuoso's SPARQL Query 
	Processor where it delivers URI de-referencing within SPARQL query patterns, across disparate 
	data spaces. It also delivers configurable smart HTTP caching services. Optionally, it can be 
	used by the <link linkend="contentcrawlerrdf">Virtuoso Content Crawler</link> to periodically 
	populate and replenish data within the native RDF Quad Store. 
</para>
<para>The sponger is a fully fledged HTTP proxy service that is also directly accessible via 
	SOAP or REST interfaces.
</para>
<para>As depicted below, OpenLink's broad portfolio of Linked-Data-aware products supports a number 
	of routes for creating or consuming Linked Data. The Sponger provides a key platform for developers 
	to generate quality data meshes from unstructured or semi-structured data sources.
</para>
<figure id="virtuosospongerdiagram" float="1">
		<title>OpenLink Linked Data generation options</title>
		<graphic fileref="Sponger_LinkedDataGenOptions_2014_v3.png"/>
        </figure>
<para>
Architecturally, the Sponger is comprised of a number of Cartridges two types of cartridges: Extractor 
and Meta Cartridges. Extractor Cartridges focus on data extraction and transformation services while 
the Meta Cartridges provide lookups and joins across other linked data spaces and Web 2.0 APIs. Both 
cartridge types are themselves comprised of a data extractors and RDF Schema/Ontology Mapper 
components.</para>
<para>Cartridges are is highly customizable. Custom cartridges can be developed using any language 
	supported by the Virtuoso Server Extensions API enabling structured linked data generation from 
	resource types not available in the default Sponger Cartridge collection bundled -- as part of 
	the Virtuoso <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.3/virtuoso/cartridges_dav.vad">Cartridges VAD package</ulink>.
</para>
    <figure id="virtuosospongerdiagram" float="1">
  	  <title>Virtuoso metadata extraction &amp; RDF structured data generation</title>
		  <graphic fileref="linked_data_gen_opts4.png"/>
		<title>Virtuoso metadata extraction &amp; RDF structured data generation</title>
		<graphic fileref="linked_data_gen_opts4.png"/>
    </figure>
  </sect2>  
  
	<sect2 id="virtuosospongerimp"><title>Why is it Important?</title>
<para>	
A majority of the worlds data naturally resides in non RDF form at the current time. The Sponger
delivers middleware that accelerates the bootstrap of the Semantic Data Web by generating RDF
from non RDF data sources, unobtrusively. This "Swiss army knife" for on-the-fly Linked Data generation 
provides a bridge between the traditional Document Web and the Linked Data Web ("Data Web").
</para>
<para>Sponging data from non-RDF Web sources and converting it to RDF exposes the data in a 
canonical form for querying and inference, and enables fast and easy construction of linked 
data driven mesh-ups as opposed to code driven Web 2.0 mash-ups.
</para>
<para>The RDF extraction and instance data generation products that offer functionality 
demonstrated by the Sponger are also commonly referred to as RDFizers.
</para>
</sect2>

<sect2 id="virtuosospongerworkpr"><title>How Does It Work?</title>
<para>
When an RDF aware client requests data from a network accessible resource via the Sponger
the following events occur:
</para>
<itemizedlist>
<listitem>A requests in made for data in RDF form, and if RDF is returned nothing further happens</listitem>
<listitem>If RDF isn't returned, then the Sponger passes the data through a Metadata Extraction
Pipeline process (using Metadata Extractors)</listitem>
<listitem>The extracted data is transformed to RDF via a Mapping Pipeline process (RDF is extracted by way
of Ontology matching and mapping) that results in RDF Entities (instance data) generation</listitem>
<listitem>RDF Entities are returned to the client</listitem>
</itemizedlist>
<para>The imported data forms a local cache and its invalidation rules conform to those of traditional
HTTP clients (Web Browsers). Thus, expiration time is determined based on subsequent data fetches of
the same resource (note: the first data load will record the 'expires' header) with current time
compared to expiration time stored in the local cache. If HTTP 'expires' header data isn't returned
by the source data server, then the "Sponger" will derive it's own invalidation time frame by
evaluating the 'date' header and 'last-modified' HTTP headers. Irrespective of path taken,
local cache invalidation is driven by an assessment of current time relative to recorded expiration time.
</para>
<para>To manage the cache expiration, set the <emphasis>MinExpiration</emphasis> parameter in 
your Virtuoso.ini file.</para>
<para>Read full description of the parameter in the <link linkend="ini_SPARQL">[SPARQL] ini section</link>.</para>
<para>Designed with a pluggable architecture, the Sponger's core functionality is provided by Cartridges.
Each cartridge includes Data Extractors which extract data from one or more data sources, and Ontology
Mappers which map the extracted data to one or more ontologies/schemas, and route to producing RDF
Linked Data.
</para>
<para>The Schema Mappers are typically XSLT (e.g. GRDDL and other OpenLink Mapping Schemas) or
Virtuoso PL based. The Metadata Extractors may be developed in Virtuoso PL, C/C++, Java, or any
other language that can be integrated into the Virtuoso via it's server extensions APIs.
</para>
<para>
The Sponger also includes a pluggable name resolution mechanism that enables the development of
Custom Resolvers for naming schemes (e.g. URNs) associated with protocols beyond HTTP.
Examples of custom resolvers include:
</para>
<itemizedlist>
<listitem><ulink url="http://dbpedia.org/resource/LSID">LSID</ulink></listitem>
<listitem><ulink url="http://dbpedia.org/resource/DOI">DOI</ulink></listitem>
</itemizedlist>
</sect2>

<sect2 id="virtuosospongerinstall"><title>Installation Steps</title>
<orderedlist>
  <listitem>Download the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.3/virtuoso/cartridges_dav.vad">Cartridges VAD package</ulink>.</listitem>
  <listitem>Install the cartridges_dav.vad package by using the Conductor UI or by using iSQL:
<programlisting><![CDATA[
SQL> DB.DBA.VAD_INSTALL('tmp/cartridges_dav.vad',0);
SQL_STATE  SQL_MESSAGE
VARCHAR  VARCHAR
_______________________________________________________________________________

00000    No errors detected
00000    Installation of "Linked Data Cartridges" is complete.
00000    Now making a final checkpoint.
00000    Final checkpoint is made.
00000    SUCCESS


6 Rows. -- 1078 msec.
]]></programlisting>  	
  </listitem>
  <listitem><link linkend="virtuosospongercreatecustcartrrgst">Cartridge Configuration</link>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtuosospongercartridgesextr">Extractor Cartridges</link></listitem>
      <listitem><link linkend="virtuosospongercartridgesmeta">Meta Cartridges</link></listitem>
</itemizedlist>
  </listitem>
</orderedlist>
<para>
	<tip>
		<title>See Also: <link linkend="virtuosospongerrdfmapperspackage">Cartridges Package content description.</link></title>
  </tip>
</para>  
</sect2>


<sect2 id="virtuosospongerusage"><title>Using The Sponger</title>
    <para>The Sponger can be invoked via the following mechanisms:</para>
    <orderedlist>
      <listitem><link linkend="virtuosospongerusageprocessor">Virtuoso SPARQL query processor</link></listitem>
      <listitem><link linkend="virtuosospongerusageproxy">RDF Proxy Service</link></listitem>
      <listitem><link linkend="virtuosospongerusageclapp">OpenLink RDF client applications</link></listitem>
      <listitem><link linkend="virtuosospongerusagebrief">ODS-Briefcase (Virtuoso WebDAV)</link> - a component of the OpenLink Data Spaces distributed collaborative application platform</listitem>
      <listitem><link linkend="virtuosospongerusagedirect">Directly via Virtuoso PL</link></listitem>
    </orderedlist>
    
    <sect3 id="virtuosospongerusageprocessor"><title>SPARQL Query Processor IRI Dereferencing</title>
    <para>The Sponger is transparently integrated into the Virtuoso SPARQL query processor, where it supports
IRI dereferencing.
      </para>
      <para>Virtuoso extends the SPARQL Query Language such that it is possible to download RDF resources 
      	from a given IRI, parse, and then store the resulting triples in a graph, with all three operations 
      	performed during the SPARQL query-execution process. The IRI/URI of the graph used to store the 
      	triples is usually equal to the URL where the resources are downloaded from, consequently the 
      	feature is known as "IRI/URI dereferencing". If a SPARQL query instructs the SPARQL processor 
      	to retrieve the target graph into local storage, then the SPARQL sponger will be invoked.
      </para>
      <para>The SPARQL extensions for IRI dereferencing are described below. Essentially these enable 
      	downloading and local storage of selected triples either from one or more named graphs, or 
      	based on a proximity search from a starting URI for entities matching the select criteria and 
      	also related by the specified predicates, up to a given depth. For full details see 
      	section <link linkend="rdfiridereferencing">Linked Data - IRI Dereferencing</link>.
      </para>
      <para>Note: For brevity, any reference to URI/IRIs above or in subsequent sections implies 
      an HTTP URI/IRI, where IRI is an internationalized URI. Similarly, in the context of the 
      Sponger, the term IRI in the Virtuoso reference documentation should be taken to mean an HTTP IRI.	
      </para> 
      <sect4 id="virtuosospongerusageprocessor"><title>SPARQL Extensions for IRI Dereferencing of FROM Clauses</title>
        <para>In addition to the "define get:..." SPARQL extensions for IRI dereferencing in FROM 
        	clauses, Virtuoso supports dereferencing SPARQL IRIs used in the WHERE clause (graph patterns) 
        	of a SPARQL query via a set of "define input:grab-..." pragmas.
        </para>	
        <para>Consider an RDF resource which describes a member of a contact list, user1, and also 
        	contains statements about other users, user2 anduser3 , known to him. Resource user3 in turn 
        	contains statements about user4 and so on. If all the data relating to these users were 
        	loaded into Virtuoso's RDF database, the query to retrieve the details of all the users 
        	could be quite simple. e.g.:         	
        </para>	
<programlisting><![CDATA[
sparql
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
select ?id ?firstname ?nick
where
  {
    graph ?g
      {
        ?id rdf:type foaf:Person .
        ?id foaf:firstName ?firstname .
        ?id foaf:knows ?fn .
        ?fn foaf:nick ?nick .
      }
   }
limit 10;	
]]></programlisting>        
        <para>But what if some or all of these resources were not present in Virtuoso's quad store? 
        	The highly distributed nature of the Linked Data Web makes it highly likely that these 
        	interlinked resources would be spread across several data spaces. Virtuoso's 'input:grab-...' 
        	extensions to SPARQL enable IRI dereferencing in such a way that all appropriate Network resources 
        	are loaded, i.e. "being fetched", during query execution, even if some of the Network resources are not 
        	known beforehand. For any particular resource matched, and if necessary downloaded, by the 
        	query, it is possible to download related resources via a designated predicate path(s) to 
        	a specifiable depth i.e. number of 'hops', distance, or degrees of separation (i.e compute 
        	Transitive Closures in SPARQL).
        </para>
        <para>Using Virtuoso's 'input:grab-' pragmas to enable sponging, the above query might be 
        	recast to: 
        </para>	
<programlisting><![CDATA[
sparql
define input:grab-var "?more"
define input:grab-depth 10
define input:grab-limit 100
define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>

select ?id ?firstname ?nick
where {
    graph ?g {
               ?id rdf:type foaf:Person .
               ?id foaf:firstName ?firstname .
               ?id foaf:knows ?fn .
               ?fn foaf:nick ?nick .
               optional { ?id rdfs:SeeAlso ?more }
            }
}
limit 10;	
]]></programlisting>        
        <para>Another example showing a designated predicate traversal path via the input:grab-seealso 
        	extension is: 
        </para>	
<programlisting><![CDATA[
sparql
define input:grab-iri <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
define input:grab-var "id"
define input:grab-depth 10
define input:grab-limit 100
define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
define input:grab-seealso <foaf:maker>
prefix foaf: <http://xmlns.com/foaf/0.1/>

select ?id
where
  {
    graph ?g
      {
        ?id a foaf:Person .
      }
  }
limit 10;	
]]></programlisting>        
        <para>A list of the input:grab pragmas is given below:</para>
          <itemizedlist mark="bullet">        
            <listitem>input:grab-var specifies the name of the SPARQL variable whose values should be used as IRIs of resources that should be downloaded.</listitem>
            <listitem>input:grab-iri specifies an IRI that should be retrieved before executing the rest of the query, if it is not in the quad store already. (This pragma can be included multiple times).</listitem>
            <listitem>input:grab-seealso (or its synonym input:grab-follow-predicate) specifies a predicate IRI to be used when traversing a graph. (This pragma can be included multiple times).</listitem>
            <listitem>input:grab-limit sets the maximum number of resources (graph subject or object nodes) to be retrieved from a target graph.</listitem>
            <listitem>input:grab-depth sets the maximum 'degrees of separation' or links (predicates) between nodes in the target graph.</listitem>
            <listitem>input:grab-all "yes" instructs the SPARQL processor to dereference everything related to the query. All variables and literal IRIs in the query become values for input:grab-var and input:grab-iri. The resulting performance may be very bad.</listitem>
            <listitem>input:grab-base specifies the base IRI to use when converting relative IRIs to absolute. (Default: empty string).</listitem>
            <listitem>input:grab-destination overrides the default IRI dereferencing and forces all retrieved triples to be stored in the specified graph.</listitem>
            <listitem>input:grab-loader identifies the procedure used to retrieve each resource via HTTP, parse and store it. (Default: DB.DBA.RDF_SPONGE_UP)</listitem>
            <listitem>input:grab-resolver identifies the procedure that resolves IRIs and determines the HTTP method of retrieval. (Default: DB.DBA.RDF_GRAB_RESOLVER_DEFAULT)</listitem>
          </itemizedlist>
      </sect4>
      <sect4 id="virtuosospongerusageprocessorex"><title>SPARQL Processor Usage Example</title>
        <para>Network Resource Fetch can be performed directly from within the SPARQL processor.</para>
        <para>After logging into Virtuoso's Conductor interface, the following query can be 
        issued from the Interactive SQL (iSQL) panel: 
        </para> 	
<programlisting><![CDATA[
sparql
define get:uri "http://www.ivan-herman.net/foaf.html"
define get:soft "soft"
select * from <http://mygraph> where {?s ?p ?o}
]]></programlisting>        
        <para>Here the sparql keyword invokes the SPARQL processor from the SQL interface and the 
        	RDF data fetched from page http://www.ivan-herman.net/foaf.html is loaded into the local 
        	RDF quad store as graph http://mygraph .
        </para>	
        <para>The new graph can then be queried using the basic SPARQL client normally available 
        	in a default Virtuoso installation at http://example.com/sparql/. e.g.:
        </para>		
<programlisting><![CDATA[
select * from <http://mygraph> where {?s ?p ?o}	
]]></programlisting>        
      </sect4>      
    </sect3>   
    <sect3 id="virtuosospongerusageproxy"><title>RDF Proxy Service</title>
    <para>The Sponger's functionality is also exposed via an in-built REST style Web service. This web
service takes a target URL and either returns the content "as is" or tries to transform (by sponging)
to RDF. Thus, the proxy service can be used as a 'pipe' for RDF browsers to browse non-RDF sources.</para>
      <para>When the cartridges package is installed, Virtuoso reserves the path '/about/[id|data|rdf|html]/http/' for
Sponger Proxy URI Service. For example, if a Virtuoso installation on host example.com listens for HTTP
requests on port 8080 then client applications should use a 'service endpoint' string equal to
'http://example.com:8080/about/[id|data|rdf|html]/http/'. If the cartridges package is not installed, then
the service uses the path '/proxy/rdf/'.</para>
    <para>Note: The old Sponger Proxy URI Service pattern '/proxy/' is now deprecated.</para>
      <sect4 id="virtuosospongerusageproxyex1"><title>Example 1</title>
    <para>The following URLs return information about musician John Cale, gleaned from the MusicBrainz
music metadatabase, rendered as RDF or HTML respectively. (The Network Resource fetched data is available in the HTML
rendering through the foaf:primaryTopic property.)</para>
    <itemizedlist mark="bullet">
      <listitem>http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/artist/72c090b6-a68e-4cb9-b330-85278681a714.html</listitem>
      <listitem>http://demo.openlinksw.com/about/html/http/musicbrainz.org/artist/72c090b6-a68e-4cb9-b330-85278681a714.html</listitem>
    </itemizedlist>
      </sect4>
      <sect4 id="virtuosospongerusageproxyex2"><title>Example 2</title>
        <para>The file http://www.ivan-herman.net/foaf.html contains a short profile of the W3C Semantic 
        	Web Activity Lead Ivan Herman. This XHTML file contains RDF embedded as RDFa. Running the file 
        	through the Sponger via Virtuoso's RDF proxy service extracts the embedded FOAF data as pure 
        	RDF, as can be seen by executing:
        </para>
<programlisting><![CDATA[
$ curl -L -H "Accept:application/rdf+xml" http://linkeddata.uriburner.com/about/id/entity/http/www.ivan-herman.net/foaf.html
<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
  <rdf:Description rdf:about="http://linkeddata.uriburner.com/about/id/http/www.ivan-herman.net/foaf.html#Person1Stat"><scovo:dimension xmlns:scovo="http://purl.org/NET/scovo#" rdf:resource="http://rdfs.org/ns/void#numberOfResources"/></rdf:Description>
  <rdf:Description rdf:nodeID="b145981159"><rdf:rest rdf:nodeID="b145981158"/></rdf:Description>
  <rdf:Description rdf:about="http://linkeddata.uriburner.com/about/id/entity/http/www.mendeley.com/profiles/ivan-herman"><foaf:accountName xmlns:foaf="http://xmlns.com/foaf/0.1/">ivan-herman</foaf:accountName></rdf:Description>	
  etc ..
  <rdf:Description rdf:nodeID="b145981130"><http-voc:elementName xmlns:http-voc="http://www.w3.org/2006/http#">text/html</http-voc:elementName></rdf:Description>
</rdf:RDF>
]]></programlisting>
        <para>(linkeddata.uriburner.com hosts a public Virtuoso instance.) Though this example 
        	demonstrates the action of the /about/id/entity/ service quite transparently, it is a basic and 
        	unwieldy way to view RDF. As described earlier, the OpenLink Data Explorer uses the same 
        	proxy service to provide a more polished means to extract and view fetched RDF data.
        </para>	
      </sect4>
      <sect4 id="virtuosospongerusageproxyurlist"><title>Usage of the Sponger Middleware via REST patterns</title>
        <para>Delegation and proxies are part of the Internet and Web's federated architecture. Thus, 
        	developers of RESTful applications benefit immensely from the ability to leverage Sponger 
        	functionality via delegation to it as a proxy.</para>
        <para>The following table presents list of the supported URL parameters:</para>
        <table>
          <tgroup cols="4">
            <thead>
              <row>
                <entry>Parameter</entry><entry>Value</entry><entry>Description</entry><entry>Example</entry>
              </row>
            </thead>
            <tbody>
              <row>
                <entry><emphasis>refresh</emphasis></entry>
                <entry>clean</entry>
                <entry><emphasis>Usage</emphasis>: for overwriting. <br/>The 'clean' usage explicitly clears the graph i.e. will cause the Sponger to drop cache even if it is marked to be in the fly.<br/>Thus, if fetched cache by some reason is left in some inconsistent state like shutdown during Network Resource fetching, then 'clean' is required as it doesn't check cache state.<br/><emphasis>Note</emphasis>: must be used with caution as other threads may be doing fetching of network resources at same time.</entry>
                <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/kidehen?@Lookup@=&amp;refresh=clean">Explicitly clear the graph</ulink></entry>
              </row>
              <row>
                <entry><emphasis>sponger:get</emphasis></entry>
                <entry>add</entry>
                <entry><emphasis>Usage</emphasis>: Add new triples to named graphs, progressively. This is the default value for the parameter sponger:get. May be used together with refresh=&lt;seconds&gt; to overwrite the expiration in the cache.</entry>
                <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/kidehen?sponger:get=add&amp;refresh=10">Add new triples and refresh on every 10 seconds</ulink></entry>
              </row>            
              <row>
                <entry><emphasis>sponger:get</emphasis></entry>
                <entry>soft</entry>
                <entry><emphasis>Usage</emphasis>: Network Resource Fetch data subject to cache invalidation mode and associated rules of instance. May be used together with refresh=&lt;seconds&gt; to overwrite the expiration in the cache.</entry>
                <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/kidehen?sponger:get=soft&amp;refresh=10">Network Resource Fetch data with option <emphasis>soft</emphasis> and refresh on every 10 seconds</ulink></entry>
              </row>            
              <row>
                <entry><emphasis>sponger:get</emphasis></entry>
                <entry>replace</entry>
                <entry><emphasis>Usage</emphasis>: Replace subject to cache invalidation mode and rules, but coverage includes non fetched triples if such exist in a given named graph. may be used together with refresh=&lt;seconds&gt; to overwrite the expiration in the cache.</entry>
                <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/kidehen?sponger:get=replace&amp;refresh=10">Replace data and refresh on every 10 seconds</ulink></entry>
              </row>               
            </tbody>
          </tgroup>
        </table>
      </sect4>
    </sect3>
    <sect3 id="virtuosospongerusageclapp"><title>OpenLink RDF Client Applications</title>
      <para>OpenLink currently provides two main RDF client applications:</para>
      <itemizedlist mark="bullet">
        <listitem><ulink url="http://ode.openlinksw.com/">OpenLink Data Explorer</ulink> (ODE)</listitem>
        <listitem><ulink url="http://demo.openlinksw.com/isparql">iSPARQL</ulink></listitem>
      </itemizedlist>
      <para>ODE is a Linked Data explorer packaged as a Firefox plugin (support for other browsers 
      	is planned). iSPARQL is an interactive AJAX-based SPARQL query builder with support for 
      	SPARQL QBE, bundled as part of the 
      	<ulink url="http://oat.openlinksw.com/">OpenLink Ajax Toolkit</ulink> (OAT). Both RIA 
      	clients utilise sponging extensively.</para>
      <para>The ODE plugin is dual faceted - RDF data can be viewed and explored natively, through its 
      	integral RDF browser, or, as described above, rendered as HTML through ODE's 'View Page Metadata' 
      	option. The screenshots below show ODE's RDF browser being launched through the 'View Linked 
      	Data Sources' popup menu.
      </para>
      <figure id="ODEBrowser" float="1">
        <title>Launching ODE's RDF browser</title>
        <graphic fileref="twitter_home.png"/>
      </figure>
      <para>The RDF browser then displays RDF data fetched via the Crunchbase cartridge.</para>
      <figure id="ODEBrowserRDF" float="1">
        <title>ODE RDF browser displaying Crunchbase network resource fetched data</title>
        <graphic fileref="twitter_ode_rdf.png"/>
      </figure>      
      <para>iSPARQL directs queries to the configured SPARQL endpoint. When targetting a Virtuoso 
      	/sparql service, Virtuoso specific sponging options can be enabled through the 'Preferences' 
      	dialog box.
      </para>	
      <para>The iSPARQL sponger settings are appended to SPARQL queries through the 'should-sponge' 
      	query parameter. These are translated to IRI dereferencing pragmas on the server as follows: 
      </para>	
      <table>
<tgroup cols="3">
<thead>
  <row>
    <entry>iSPARQL sponging setting</entry><entry>/sparql endpoint: "should-sponge" query parameter value</entry><entry>SPARQL processor directives</entry>
  </row>
</thead>
<tbody>
<row><entry>Use only local data</entry><entry>N/A</entry><entry>N/A</entry></row>
<row><entry>Retrieve remote RDF data for all missing source graphs</entry><entry>soft</entry><entry>define get:soft "soft"</entry></row>
<row><entry>Retrieve all missing remote RDF data that might be useful</entry><entry>grab-all</entry><entry>	define input:grab-all "yes" define input:grab-depth 5 <br/>define input:grab-limit 100</entry></row>
<row><entry>Retrieve all missing remote RDF data that might be useful including seeAlso references</entry><entry>grab-seealso</entry><entry>define input:grab-all "yes" define input:grab-depth 5 define input:grab-limit 200<br/>define input:grab-seealso &lt;http://www.w3.org.2000/01/rdf-schema#seeAlso&gt;<br/>define input:grab-seealso &lt;http://xmlns.com/foaf/0.1/seeAlso&gt;</entry></row>
<row><entry>Try to download all referenced resources</entry><entry>grab-everything</entry><entry>
define input:grab-all "yes"<br/>define input:grab-intermediate "yes"<br/>define input:grab-depth 5<br/>define input:grab-limit 500<br/>define input:grab-seealso &lt;http://www.w3.org.2000/01/rdf-schema#seeAlso&gt;<br/>define input:grab-seealso &lt;http://xmlns.com/foaf/0.1/seeAlso&gt;<br/></entry></row>
</tbody>
</tgroup>
</table>

</sect3>
    <sect3 id="virtuosospongerusagebrief"><title>ODS-Briefcase (Virtuoso WebDAV)</title>
      <para>ODS-Briefcase is a component of 
      	<ulink url="http://virtuoso.openlinksw.com/wiki/main/Main/OdsIndex">OpenLink Data Spaces</ulink> 
      	(ODS), a new generation distributed collaborative application platform for creating Semantic 
      	Web presence via Data Spaces derived from weblogs, wikis, feed aggregators, photo galleries, 
      	shared bookmarks, discussion forums and more. It is also a high level interface to the Virtuoso 
      	WebDAV repository.
      </para>
      <para>ODS-Briefcase offers file-sharing functionality that includes the following features:</para>
      <itemizedlist mark="bullet">
        <listitem>Web brower-based interactions</listitem>
        <listitem>Web Services (direct use of the HTTP based WebDAV protocol)</listitem>
        <listitem>SPARQL query language support - all WebDAV resources are exposed as SIOC ontology 
        	instance data (RDF data sets)</listitem>        
      </itemizedlist>      
      <para>When resources or documents are put into the ODS Briefcase and are made publicly readable 
      	(via a Unix-style +r permission or ACL setting) and the resource in question is of a supported 
      	content type, metadata is automatically extracted at file upload time.
      </para>
      <para>Note: ODS-Briefcase extracts metadata from a wide array of file formats, automatically.</para>
      <para>The extracted metadata is available in two forms, pure WebDAV and RDF (with RDF/XML or 
      	N3/Turtle serialization options), that is optionally synchronized with the underlying Virtuoso 
      	Quad Store.
      </para>                  
      <para>All public readable resources in WebDAV have their owner, creation time, update time, size 
      	and tags published, plus associated content type dependent metadata. This WebDAV metadata is 
      	also available in RDF form as a SPARQL queriable graph accessible via the SPARQL protocol 
      	endpoint using the WebDAV location as the RDF data set URI (graph or data source URI).
      </para>                  
      <para>You can also use a special RDF_Sink folder to automate the process of uploading RDF 
      	resources files into the Virtuoso Quad Store via WebDAV or raw HTTP. The properties of the special 
      	folder control whether sponging (RDFization) occurs. Of course, by default, this feature is enabled 
      	across all Virtuoso and ODS installations (with an ODS-Briefcase Data Space instance enabled).
      </para>
      <sect4 id="virtuosospongerusagebriefex"><title>Raw HTTP Example for Extracting Metadata using CURL</title>                  
<programlisting><![CDATA[
Username: demo
Password: demo
Source File: wine.rdf
Destination Folder:
http://demo.openlinksw.com/DAV/home/demo/rdf_sink/
Content Type: application/rdf+xml

$ curl -v -T wine.rdf -H content-type:application/rdf+xml http://demo.openlinksw.com/DAV/home/demo/rdf_sink/ -u demo:demo	
]]></programlisting>      
      <para>Finally, you can also get RDF data into Virtuoso's Quad Store via WebDAV using the Virtuoso 
      	Web Crawler utility (configurable via the Virtuoso Conductor UI). This feature also provides the 
      	ability to enable or disable Sponging as depicted below.
      </para>
      </sect4>   
      <sect4 id="virtuosospongerusagebriefint"><title>Sponger and ODS-Briefcase Structured Data Extractor Interrelationship</title>                                                         
        <para>As the Sponger and ODS-Briefcase both extract structured data, what is the relationship 
        	between these two facilities?
        </para>	
        <para>The principal difference between the two is that the Sponger is anRDF data crawler &amp; 
        	generator, whereas Briefcase's structured data extractor is a WebDAV resourcefilter. The 
        	Briefcase structured data extractor is aimed at providing RDF data from WebDAV resources. 
        	Thus, if none of the available Sponger cartridges are able to extract metadata and produce 
        	RDF structured data, the Sponger calls upon the Briefcase extractor as the last resort in 
        	the RDF structured data generation pipeline.
        </para>
        <figure id="cninp1" float="1">
          <title>Conductor's content import configuration panel</title>
          <graphic fileref="fig2_top.png"/>
        </figure>
        <figure id="cninp2" float="1">
          <title>Conductor's content import configuration panel</title>
          <graphic fileref="fig2_bottom.png"/>
        </figure>
        <figure id="cninp2" float="1">
          <title>Conductor's content import configuration panel</title>
          <graphic fileref="fig2_bottom2.png"/>
        </figure>  
        <figure id="cninp2" float="1">
          <title>Conductor's content import configuration panel</title>
          <graphic fileref="fig2_bottom3.png"/>
        </figure>
       </sect4> 
    </sect3>
    <sect3 id="virtuosospongerusagedirect"><title>Directly via Virtuoso PL</title>
    <para>Sponger cartridges are invoked through a cartridge hook which provides a Virtuoso PL entry point 
    	to the packaged functionality. Should you wish to utilize the Sponger from your own Virtuoso PL 
    	procedures, you can do so by calling these hook routines directly. Full details of the hook 
    	function prototype and how to define your own cartridges are presented <link linkend="virtuosospongercreatecustcartran">here</link>.</para>
    </sect3>
</sect2>

<sect2 id="virtuosospongerconsm"><title>Consuming the Generated RDF Structured Data</title>
  <para>The generated RDF-based structured data (RDF) can be consumed in a number of ways, depending on 
  	whether or not the data is persisted in Virtuoso's RDF Quad Store.
  </para>
  <para>If the data is persisted, it can be queried through the Virtuoso SPARQL endpoint associated with 
  	any Virtuoso instance: /sparql. The RDF is exposed in a graph typically identified using a URL matching 
  	the source resource URL from which the RDF data was generated. Naturally, any SQL query can also access 
  	this, since SPARQL can be freely intermixed with SQL via Virtuoso's SPASQL (SPARQL inside SQL) 
  	functionality. RDF data is also accessible through Virtuoso's implementation of the URIQA protocol.
  </para>
  <para>If not persisted, as is the case with the RDF Proxy Service, the data can be consumed by an RDF 
  	aware Web client, e.g. an RDF browser such as the OpenLink Data Explorer (ODE).
  </para>
 </sect2>


   <sect2 id="virtuosospongercartridgesextractorusecases"><title>RDF Cartridges Use Cases</title>
<para>This section contains examples of Web resources which can be transformed by RDF Cartridges.
It also states where additional setup for given cartridges is needed i.e. keys account names etc.
</para>
<para><emphasis>Service based:</emphasis></para>
<itemizedlist>
<listitem>amazon
<programlisting><![CDATA[
needs: api key
example: http://www.amazon.com/gp/product/0553383043
]]></programlisting>
</listitem>
<listitem>ebay
<programlisting><![CDATA[
needs: account, api-key
example: http://cgi.ebay.com/RARE-DAY-IN-FAIRY-LAND-ELEPHANT-FOLIO-20-FULL-COLOR_W0QQitemZ140209597189QQihZ004QQcategoryZ29223QQssPageNameZWDVWQQrdZ1QQcmdZViewItem
]]></programlisting>
</listitem>
<listitem>flickr
needs: api-key
example: http://farm1.static.flickr.com/212/496684670_7122c831ed.jpg
<programlisting><![CDATA[
]]></programlisting>
</listitem>
<listitem>mbz
<programlisting><![CDATA[
example: http://musicbrainz.org/release/37e955d4-a53c-45aa-a812-1b23b88dbc13.html
]]></programlisting>
</listitem>
<listitem>mql (freebase)
<programlisting><![CDATA[
example: http://www.freebase.com/view/en/beta_ursae_majoris
]]></programlisting>
</listitem>
<listitem>facebook
<programlisting><![CDATA[
needs: api-key, secret, persistent-session-id
example: http://www.facebook.com/profile.php?id=841100003
]]></programlisting>
</listitem>
<listitem>yahoo-stock
<programlisting><![CDATA[
example: http://finance.yahoo.com/q?s=AAPL
]]></programlisting>
</listitem>
<listitem>yahoo-traffic
<programlisting><![CDATA[
example: http://local.yahooapis.com/MapsService/V1/trafficData?appid=YahooDemo&street=701+First+Street&city=Sunnyvale&state=CA
]]></programlisting>
</listitem>
<listitem>Bugzilla
<programlisting><![CDATA[
example: https://bugzilla.mozilla.org/show_bug.cgi?id=251714
]]></programlisting>
</listitem>
<listitem>SVG</listitem>
<listitem>OO document
<programlisting><![CDATA[
needs: unzip plugin
]]></programlisting>
</listitem>
<listitem>Wikipedia
<programlisting><![CDATA[
needs: php plugin & dbpedia extractor
example: http://wikipedia.org/wiki/London
]]></programlisting>
</listitem>
<listitem>Opencalais</listitem>
<listitem>iCalendar</listitem>
</itemizedlist>
<para><emphasis>GRDDL</emphasis></para>
  <itemizedlist>
<listitem>Google Base (google)
<programlisting><![CDATA[
example: http://www.google.com/base/feeds/snippets/17891817243016304554
]]></programlisting>
</listitem>
<listitem>eRDF</listitem>
<listitem>RDFa</listitem>
<listitem>hCard</listitem>
<listitem>hCalendar</listitem>
<listitem>hReview</listitem>
<listitem>relLicense</listitem>
<listitem>XBRL</listitem>
<listitem>HR-XML</listitem>
<listitem>DC</listitem>
<listitem>geoURL</listitem>
<listitem>Ning</listitem>
<listitem>XFN</listitem>
<listitem>xFolk</listitem>
</itemizedlist>
<para><emphasis>URN handlers</emphasis></para>
<table colsep="1" frame="all" rowsep="0" shortentry="0" tocentry="1" tabstyle="decimalstyle" orient="land" pgwide="0">
    <title>URN handlers List</title>
    <tgroup align="char" charoff="50" char="." cols="6">
    <colspec align="left" colnum="1" colsep="0" colwidth="20pc"/>
    <thead>
      <row>
        <entry>URN handler</entry>
        <entry>Sample URI</entry>
        <entry>Resource Description</entry>
        <entry>Linked Data View</entry>        
        <entry>Linked Data Graph</entry>        
        <entry>Needs</entry>                
      </row>
    </thead>
    <tbody>
      <row>
        <entry>LSID</entry>
        <entry>urn:lsid:ubio.org:namebank:12292</entry>
        <entry><ulink url="http://demo.openlinksw.com/about/html/urn:lsid:ubio.org:namebank:12292">HTML Representation</ulink></entry>
        <entry><ulink url="http://demo.openlinksw.com/describe/?url=urn:lsid:ubio.org:namebank:12292">Linked Data View</ulink></entry>
        <entry><ulink url="http://demo.openlinksw.com/ode/?uri=urn:lsid:ubio.org:namebank:12292">Data Explorer View</ulink></entry>                                
        <entry>none</entry>        
      </row>
      <row>
        <entry>DOI</entry>
        <entry>doi:10.1038/35057062</entry>
        <entry><ulink url="http://demo.openlinksw.com/about/html/doi:10.1038/35057062">HTML Representation</ulink></entry>
        <entry><ulink url="http://demo.openlinksw.com/describe/?url=doi:10.1038/35057062">Linked Data View</ulink></entry>
        <entry><ulink url="http://demo.openlinksw.com/ode/?uri=doi:10.1038/35057062">Data Explorer View</ulink></entry>                                
        <entry>Needs hslookup plugin, relevant html, pdf, xml etc. mappers enabled.</entry>        
      </row>
      <row>
        <entry>OAI</entry>
        <entry>oai:dcmi.ischool.washington.edu:article/8</entry>
        <entry><ulink url="http://demo.openlinksw.com/about/html/oai:dcmi.ischool.washington.edu:article/8">HTML Representation</ulink></entry>
        <entry><ulink url="http://demo.openlinksw.com/describe/?url=oai:dcmi.ischool.washington.edu:article/8">Linked Data View</ulink></entry>
        <entry><ulink url="http://demo.openlinksw.com/ode/?uri=oai:dcmi.ischool.washington.edu:article/8">Data Explorer View</ulink></entry>                                
        <entry>none</entry>        
      </row>
      
    </tbody>
  </tgroup>
</table>
<sect3 id="virtuosospongerrdfmappers"><title>SPARQL IRI Dereferencing</title>
<para>The Virtuoso SPARQL engine (called for brevity just SPARQL below) supports IRI Dereferencing,
however it understands only RDF data, that is it can retrieve only files containing RDF/XML, turtle
or N3 serialized RDF data, if format is unknown it will try mapping with built-in WebDAV metadata
extractor. In order to extend this feature with dereferencing web or file resources which naturally
don't have RDF data (like PDF, JPEG files for example) is provided a special mechanism in SPARQL
engine. This mechanism is called RDF mappers for translation of non-RDF data files to RDF.</para>
<para>In order to instruct the SPARQL to call a RDF mapper it needs to be registered and it will
be called for a given URL or MIME type pattern. In other words, when unknown for SPARQL format is
received during URL dereferencing process, it will look into a special registry (a table) to match
either the MIME type or IRI using a regular expression, if match is found the mapper function will
be called.</para>
<sect4 id="virtuosospongerproxy"><title>Sponger Proxy service</title>
<para>Sponger functionality is also exposed via Virtuoso's "/proxy/rdf/" endpoint, as an in-built
REST style Web service available in any Virtuoso standard installation. This web service takes
a target URL and either returns the content "as is" or tries to transform (by sponging) to RDF.
Thus, the proxy service can be used as a 'pipe' for RDF browsers to browse non-RDF sources.
</para>
<para>For more information see <link linkend="rdfproxyservice">RDF Sponger Proxy service</link></para>
</sect4>
<sect4 id="virtuosospongercache"><title>Cache Invalidation</title>
<para>To clear cache on all values of HS_LOCAL_IRI of the SYS_HTTP_SPONGE table use:</para>
<programlisting><![CDATA[
SPARQL clear graph <A-Named-Graph>;
]]></programlisting>
</sect4>
</sect3>
</sect2>

<sect2 id="virtuosospongerarch"><title>Cartridge Architecture</title>
  <sect3 id="virtuosospongerarchcr"><title>What is a Cartridge?</title>
    <para>See full description <link linkend="virtuosospongeroverviewcartarch">here</link></para>
  </sect3>
  <sect3 id="virtuosospongercartridgesextr"><title>Extractor Cartridges</title>
    <para>An Extractor Cartridge processes a Resource of a given format, extracting RDF according to 
    	rules appropriate to that format. External data does not come into play; only the content of 
    	the Resource fed to the Sponger.
    </para>	
    <sect4 id="virtuosospongercartridgesstand"><title>Supported Standard Non-RDF Data Formats</title>
      <para>These Cartridges handle open formats - typically community-developed, openly-documented, 
      	and freely-licensed data structures.</para>
<table>
<tgroup cols="4">
<thead>
  <row>
    <entry>Cartridge</entry><entry>Sample URI</entry><entry>Resource Description</entry><entry>Linked Data Graph</entry>
  </row>
</thead>
<tbody>
<row><entry>AB Meta</entry><entry><ulink url="http://abmeta.org/album2.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/abmeta.org/album2.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://abmeta.org/album2.html">Data Explorer View</ulink> </entry></row>
<row><entry>Atom</entry><entry><ulink url="http://www.openlinksw.com/blog/~kidehen/gems/atom.xml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.openlinksw.com/blog/~kidehen/gems/atom.xml">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.openlinksw.com/blog/~kidehen/gems/atom.xml">Data Explorer View</ulink> </entry></row>
<row><entry>CSV</entry><entry><ulink url="http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">Data Explorer View</ulink> </entry></row>
<row><entry>DC</entry><entry><ulink url="http://dublincore.org/2008/01/14/dcterms.rdf">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/dublincore.org/2008/01/14/dcterms.rdf">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://dublincore.org/2008/01/14/dcterms.rdf">Data Explorer View</ulink> </entry></row>
<row><entry>eRDF</entry><entry><ulink url="http://www.w3.org/2001/sw/grddl-wg/doc29/hotel-data.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.w3.org/2001/sw/grddl-wg/doc29/hotel-data.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.w3.org/2001/sw/grddl-wg/doc29/hotel-data.html">Data Explorer View</ulink> </entry></row>
<row><entry>hAudio</entry><entry><ulink url="http://alpha.libre.fm/user/weborganics">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/alpha.libre.fm/user/weborganics">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://alpha.libre.fm/user/weborganics">Data Explorer View</ulink> </entry></row>
<row><entry>hCalendar</entry><entry><ulink url="http://www.maine.gov/portal/government/calendar.shtml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.maine.gov/portal/government/calendar.shtml">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.maine.gov/portal/government/calendar.shtml">Data Explorer View</ulink></entry></row>
<row><entry>hCard</entry><entry><ulink url="http://dbaron.org/">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/dbaron.org/">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://dbaron.org/">Data Explorer View</ulink></entry></row>
<row><entry>hListing</entry><entry><ulink url="http://feedback.ebay.com/ws/eBayISAPI.dll?ViewFeedback2&amp;userid=manganos&amp;ftab=AllFeedback&amp;frm=284&amp;iid=5863912155">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/feedback.ebay.com/ws/eBayISAPI.dll?ViewFeedback2&amp;userid=manganos&amp;ftab=AllFeedback&amp;frm=284&amp;iid=5863912155">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://feedback.ebay.com/ws/eBayISAPI.dll?ViewFeedback2&amp;userid=manganos&amp;ftab=AllFeedback&amp;frm=284&amp;iid=5863912155">Data Explorer View</ulink></entry></row>
<row><entry>hNews</entry><entry><ulink url="http://www.semissourian.com/story/1620555.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http://www.semissourian.com/story/1620555.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.semissourian.com/story/1620555.html">Data Explorer View</ulink></entry></row>
<row><entry>hProduct</entry><entry><ulink url="http://www.roger-junca.com/Roti-de-magret-du-Sud-ouest-produit-89.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.roger-junca.com/Roti-de-magret-du-Sud-ouest-produit-89.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.roger-junca.com/Roti-de-magret-du-Sud-ouest-produit-89.html">Data Explorer View</ulink></entry></row>
<row><entry>HR-XML</entry><entry><ulink url="http://ns.hr-xml.org/2_5/HR-XML-2_5/SEP/ResumeExample.xml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/ns.hr-xml.org/2_5/HR-XML-2_5/SEP/ResumeExample.xml">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://ns.hr-xml.org/2_5/HR-XML-2_5/SEP/ResumeExample.xml">Data Explorer View</ulink></entry></row>
<row><entry>hRecipe</entry><entry><ulink url="http://www.bbc.co.uk/food/recipes/beef_tacos_with_salsa_67130%01hrecipe">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.bbc.co.uk/food/recipes/beef_tacos_with_salsa_67130%01hrecipe">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.bbc.co.uk/food/recipes/beef_tacos_with_salsa_67130">Data Explorer View</ulink></entry></row>
<row><entry>hResume</entry><entry><ulink url="http://brev.name/resume/brev-resume.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/brev.name/resume/brev-resume.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://brev.name/resume/brev-resume.html">Data Explorer View</ulink></entry></row>
<row><entry>hReview</entry><entry><ulink url="http://www.concertbuzz.net/genres/classic-rock/jethro-tull.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.concertbuzz.net/genres/classic-rock/jethro-tull.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.concertbuzz.net/genres/classic-rock/jethro-tull.html">Data Explorer View</ulink></entry></row>
<row><entry>HTTP in RDF</entry><entry><ulink url="http://linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">Data Explorer View</ulink></entry></row>
<row><entry>iCalendar</entry><entry><ulink url="http://www.mozilla.org/projects/calendar/caldata/AustrianHolidays.ics">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.mozilla.org/projects/calendar/caldata/AustrianHolidays.ics">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.mozilla.org/projects/calendar/caldata/AustrianHolidays.ics">Data Explorer View</ulink></entry></row>
<row><entry>Microsoft Word 2003 XML Document</entry><entry><ulink url="http://ec2-174-129-156-25.compute-1.amazonaws.com/DAV/office_tests/word2003xml.xml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/ec2-174-129-156-25.compute-1.amazonaws.com/DAV/office_tests/word2003xml.xml">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://ec2-174-129-156-25.compute-1.amazonaws.com/DAV/office_tests/word2003xml.xml">Data Explorer View</ulink></entry></row>
<row><entry>Microsoft XML Spreadsheet 2003</entry><entry><ulink url="http://mathewpeet.org/thesis/programs/spreadsheet/XRDCALC-LATTICEPARAM.xls">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/mathewpeet.org/thesis/programs/spreadsheet/XRDCALC-LATTICEPARAM.xls">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://mathewpeet.org/thesis/programs/spreadsheet/XRDCALC-LATTICEPARAM.xls">Data Explorer View</ulink></entry></row>
<row><entry>Microsoft Documents</entry><entry><ulink url="http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">Data Explorer View</ulink></entry></row>
<row><entry>OData</entry><entry><ulink url="http://services.odata.org/Northwind/Northwind.svc/">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http://services.odata.org/Northwind/Northwind.svc/">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://services.odata.org/Northwind/Northwind.svc/">Data Explorer View</ulink></entry></row>
<row><entry>OO document</entry><entry><ulink url="http://www.pitonyak.org/AndrewMacro.odt">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.pitonyak.org/AndrewMacro.odt">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.pitonyak.org/AndrewMacro.odt">Data Explorer View</ulink></entry></row>
<row><entry>OPML</entry><entry><ulink url="http://news.bbc.co.uk/rss/feeds.opml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/news.bbc.co.uk/rss/feeds.opml">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://news.bbc.co.uk/rss/feeds.opml">Data Explorer View</ulink></entry></row>
<row><entry>PPTX</entry><entry><ulink url="http://download.microsoft.com/download/4/D/E/4DE0D83D-7845-4FD1-9A8E-12F532EC81BC/Keynote.pptx">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/download.microsoft.com/download/4/D/E/4DE0D83D-7845-4FD1-9A8E-12F532EC81BC/Keynote.pptx">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://download.microsoft.com/download/4/D/E/4DE0D83D-7845-4FD1-9A8E-12F532EC81BC/Keynote.pptx">Data Explorer View</ulink></entry></row>
<row><entry>RDFa</entry><entry><ulink url="http://virtuoso.openlinksw.com/presentations/Creating_Deploying_Exploiting_Linked_Data2/Creating_Deploying_Exploiting_Linked_Data2_TimBL_v3.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/virtuoso.openlinksw.com/presentations/Creating_Deploying_Exploiting_Linked_Data2/Creating_Deploying_Exploiting_Linked_Data2_TimBL_v3.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://virtuoso.openlinksw.com/presentations/Creating_Deploying_Exploiting_Linked_Data2/Creating_Deploying_Exploiting_Linked_Data2_TimBL_v3.html">Data Explorer View</ulink></entry></row>
<row><entry>RSS</entry><entry><ulink url="http://microformats.org/feed/">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/microformats.org/feed/">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://microformats.org/feed/">Data Explorer View</ulink></entry></row>
<row><entry>Slidy</entry><entry><ulink url="http://slideshow.rubyforge.org/microformats.html">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/slideshow.rubyforge.org/microformats.html">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://slideshow.rubyforge.org/microformats.html">Data Explorer View</ulink></entry></row>
<row><entry>vCalendar</entry><entry><ulink url="http://upcoming.org/event/130719/">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/upcoming.org/event/130719/">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://upcoming.org/event/130719/">Data Explorer View</ulink></entry></row>
<row><entry>vCard</entry><entry><ulink url="http://tech.yahoo.com/pr/apple-ipod-video-30gb-black-mp3-player/1992981873">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/tech.yahoo.com/pr/apple-ipod-video-30gb-black-mp3-player/1992981873">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://tech.yahoo.com/pr/apple-ipod-video-30gb-black-mp3-player/1992981873">Data Explorer View</ulink></entry></row>
<row><entry>WebDAV Metadata</entry><entry><ulink url="http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/rdf/http://data.london.gov.uk/datafiles/business-economy/employment-projections-sector-2009.csv">Data Explorer View</ulink></entry></row>
<row><entry>XBRL</entry><entry><ulink url="http://www.sec.gov/Archives/edgar/data/51143/000110465908059468/ibm-20080429.xml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http://www.sec.gov/Archives/edgar/data/51143/000110465908059468/ibm-20080429.xml">HTML Representation</ulink> </entry><entry>  <ulink url="http://www.sec.gov/Archives/edgar/data/51143/000110465908059468/ibm-20080429.xml">Data Explorer View</ulink></entry></row>
<row><entry>XFN Profile</entry><entry><ulink url="http://www.molly.com/people.php">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.molly.com/people.php">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.molly.com/people.php">Data Explorer View</ulink></entry></row>
<row><entry>XFN Profile2</entry><entry><ulink url="http://www.molly.com/people.php">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.molly.com/people.php">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.molly.com/people.php">Data Explorer View</ulink></entry></row>
<row><entry>xHTML</entry><entry><ulink url="http://virtuoso.openlinksw.com/Whitepapers/html/AIW_Virtuoso_SOA.htm">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/virtuoso.openlinksw.com/Whitepapers/html/AIW_Virtuoso_SOA.htm">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://virtuoso.openlinksw.com/Whitepapers/html/AIW_Virtuoso_SOA.htm">Data Explorer View</ulink></entry></row>
<row><entry>XHTML</entry><entry><ulink url="http://www.lespetitescases.net/semantique-et-xhtml">example</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/about/html/http/www.lespetitescases.net/semantique-et-xhtml">HTML Representation</ulink> </entry><entry>  <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.lespetitescases.net/semantique-et-xhtml">Data Explorer View</ulink></entry></row>
</tbody>
</tgroup>
</table>
    </sect4>
    <sect4 id="virtuosospongercartridgesnonstand"><title>Supported Vendor-specific Non-RDF Data Formats</title>
<para>These Cartridges handle closed formats - typically proprietary; sometimes undocumented; possibly 
	licensed to no-one except the format originator. Sometimes data may not be parsed as desired or 
	expected, as many of these Cartridges have required reverse-engineering of the data format 
	in question.</para>
<table>
<tgroup cols="4">
<thead>
  <row>
              <entry>Cartridge</entry><entry>Needs</entry><entry>Sample URI</entry><entry>Resource Description</entry><entry>Linked Data Graph</entry>
  </row>
</thead>
<tbody>
            <row><entry>Amazon</entry><entry>API Key</entry><entry><ulink url="http://www.amazon.com/gp/product/0553383043">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.amazon.com/gp/product/0553383043">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.amazon.com/gp/product/0553383043">Data Explorer View</ulink>  </entry></row>
            <row><entry>BestBuy</entry><entry> API Key </entry><entry><ulink url="http://www.bestbuy.com/site/KOSS+-+Pro+DJ100+Professional+Over-the-Ear+Headphones+-+Black/9745789.p?skuId=9745789&amp;id=1218165774336">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.bestbuy.com/site/KOSS+-+Pro+DJ100+Professional+Over-the-Ear+Headphones+-+Black/9745789.p?skuId=9745789&amp;id=1218165774336">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.bestbuy.com/site/KOSS+-+Pro+DJ100+Professional+Over-the-Ear+Headphones+-+Black/9745789.p?skuId=9745789&amp;id=1218165774336">Data Explorer View</ulink>  </entry></row>
            <row><entry>Bing</entry><entry> none </entry><entry><ulink url="http://www.bing.com/community/members/livesearch/default.aspx">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.bing.com/community/members/livesearch/default.aspx">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.bing.com/community/members/livesearch/default.aspx">Data Explorer View</ulink>  </entry></row>
            <row><entry>Bugzillas</entry><entry> none</entry><entry><ulink url="https://bugzilla.mozilla.org/show_bug.cgi?id=251714">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/https://bugzilla.mozilla.org/show_bug.cgi?id=251714">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=https://bugzilla.mozilla.org/show_bug.cgi?id=251714">Data Explorer View</ulink>  </entry></row>
            <row><entry>CNET</entry><entry> API Key </entry><entry><ulink url="http://shopper.cnet.com/samsung-moment-sprint/4014-6452_9-33775546.html">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/linkeddata.uriburner.com/about/rdf/http://shopper.cnet.com/samsung-moment-sprint/4014-6452_9-33775546.html">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/rdf/http://shopper.cnet.com/samsung-moment-sprint/4014-6452_9-33775546.html">Data Explorer View</ulink>  </entry></row>
            <row><entry>CrunchBase</entry><entry> none </entry><entry><ulink url="http://linkeddata.uriburner.com/about/rdf/http://www.crunchbase.com/company/radar-music-videos">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/linkeddata.uriburner.com/about/rdf/http://www.crunchbase.com/company/radar-music-videos">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/rdf/http://www.crunchbase.com/company/radar-music-videos">Data Explorer View</ulink>  </entry></row>
            <row><entry>Delicious</entry><entry> none </entry><entry><ulink url="http://delicious.com/popular/blog">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/delicious.com/popular/blog">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://delicious.com/popular/blogI">Data Explorer View</ulink>  </entry></row>
            <row><entry>Digg</entry><entry> none </entry><entry><ulink url="http://digg.com/general_sciences/At_Last-Stem_Cells_without_Side_Effects_">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/digg.com/general_sciences/At_Last-Stem_Cells_without_Side_Effects_">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://digg.com/general_sciences/At_Last-Stem_Cells_without_Side_Effects_">Data Explorer View</ulink>  </entry></row>
            <row><entry>Discogs</entry><entry> php plugin, DBpedia Extractor </entry><entry><ulink url="http://www.discogs.com/Todd-Barton-and...ic-Poetry-Of-The-Kesh/release/2053074">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.amazon.com/o/ASIN/0553383043">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.amazon.com/o/ASIN/0553383043">Data Explorer View</ulink>  </entry></row>
            <row><entry>Disqus</entry><entry> API Key, API Account </entry><entry><ulink url="http://www.scripting.com/stories/2009/02/01/wheresYourData.html#disqus_thread">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.scripting.com/stories/2009/02/01/wheresYourData.html#disqus_thread">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.scripting.com/stories/2009/02/01/wheresYourData.html#disqus_thread">Data Explorer View</ulink>  </entry></row>
            <row><entry>DOI</entry><entry> hslookup plugin; relevant html-, pdf-, xml-, etc., -mappers enabled </entry><entry><ulink url="http://doi.ieeecomputersociety.org/10.1109/MIC.2008.16">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/doi.ieeecomputersociety.org/10.1109/MIC.2008.16">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://doi.ieeecomputersociety.org/10.1109/MIC.2008.16">Data Explorer View</ulink>  </entry></row>
            <row><entry>Dublin Core</entry><entry> none </entry><entry><ulink url="http://dublincore.org/documents/2002/07/31/dcmes-xml/dcmes-xml-dtd.shtml">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/dublincore.org/documents/2002/07/31/dcmes-xml/dcmes-xml-dtd.shtml">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://dublincore.org/documents/2002/07/31/dcmes-xml/dcmes-xml-dtd.shtml">Data Explorer View</ulink>  </entry></row>
            <row><entry>eBay</entry><entry> account, API Key </entry><entry><ulink url="http://cell-phones.shop.ebay.com/Cell-Phones-Smartphones-/3312/i.html?LH_TopRatedSellers=1&amp;LH_ItemCondition=1&amp;Brand=Apple%2520iPhone&amp;LH_Price=100..%40c&amp;_nkw=%22iphone+4%22&amp;_trkparms=65%253A15%257C66%253A2%257C39%253A1&amp;_dmd=1&amp;_dmpt=Cell_Phones&amp;_mPrRngCbx=1&amp;_sc=1&amp;_sticky=1&amp;_trksid=p3907.m551&amp;_sop=12&amp;_sc=1%26clkid%3D6198561558861688578&amp;_qi=RTM733776">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/cell-phones.shop.ebay.com/Cell-Phones-Smartphones-/3312/i.html?LH_TopRatedSellers=1&amp;LH_ItemCondition=1&amp;Brand=Apple%2520iPhone&amp;LH_Price=100..%40c&amp;_nkw=%22iphone+4%22&amp;_trkparms=65%253A15%257C66%253A2%257C39%253A1&amp;_dmd=1&amp;_dmpt=Cell_Phones&amp;_mPrRngCbx=1&amp;_sc=1&amp;_sticky=1&amp;_trksid=p3907.m551&amp;_sop=12&amp;_sc=1%26clkid%3D6198561558861688578&amp;_qi=RTM733776">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://cell-phones.shop.ebay.com/Cell-Phones-Smartphones-/3312/i.html?LH_TopRatedSellers=1&amp;LH_ItemCondition=1&amp;Brand=Apple%2520iPhone&amp;LH_Price=100..%40c&amp;_nkw=%22iphone+4%22&amp;_trkparms=65%253A15%257C66%253A2%257C39%253A1&amp;_dmd=1&amp;_dmpt=Cell_Phones&amp;_mPrRngCbx=1&amp;_sc=1&amp;_sticky=1&amp;_trksid=p3907.m551&amp;_sop=12&amp;_sc=1%26clkid%3D6198561558861688578&amp;_qi=RTM733776">Data Explorer View</ulink>  </entry></row>
            <row><entry>Evri</entry><entry> none </entry><entry><ulink url="http://www.evri.com/organization/general-motors-0x4938f">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.evri.com/organization/general-motors-0x4938f">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.evri.com/organization/general-motors-0x4938f">Data Explorer View</ulink>  </entry></row>
            <row><entry>Facebook</entry><entry> API key and secret, OAuth token <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtCartConfigFacebook">See details</ulink></entry><entry><ulink url="http://www.facebook.com/people/Richard-Max-Koster/1319314117">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.facebook.com/people/Richard-Max-Koster/1319314117">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.facebook.com/people/Richard-Max-Koster/1319314117">Data Explorer View</ulink>  </entry></row>
            <row><entry>Flickr</entry><entry> API Key </entry><entry><ulink url="http://farm1.static.flickr.com/212/496684670_7122c831ed.jpg">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.google.com/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.amazon.com/o/ASIN/0553383043">Data Explorer View</ulink>  </entry></row>
            <row><entry>Freebase</entry><entry> none </entry><entry><ulink url="http://www.freebase.com/view/location">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.freebase.com/view/location">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.freebase.com/view/location">Data Explorer View</ulink>  </entry></row>
            <row><entry>Geonames</entry><entry> none </entry><entry><ulink url="http://ws.geonames.org/search?q=london&amp;amp;maxRows=10&amp;amp;type=rdf">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/ws.geonames.org/search?q=london&amp;amp;maxRows=10&amp;amp;type=rdf">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://ws.geonames.org/search?q=london&amp;amp;maxRows=10&amp;amp;type=rdf">Data Explorer View</ulink>  </entry></row>
            <row><entry>geoURL</entry><entry> none </entry><entry><ulink url="http://geourl.org/near?p=wiki.worldflicks.org/saanen.html">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/geourl.org/near?p=wiki.worldflicks.org/saanen.html">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://geourl.org/near?p=wiki.worldflicks.org/saanen.html">Data Explorer View</ulink>  </entry></row>
            <row><entry>Get Satisfaction</entry><entry> none </entry><entry><ulink url="http://getsatisfaction.com/mozilla/topics/ubiquity_mostly_fails_on_mac_ppc">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/getsatisfaction.com/mozilla/topics/ubiquity_mostly_fails_on_mac_ppc">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://getsatisfaction.com/mozilla/topics/ubiquity_mostly_fails_on_mac_ppc">Data Explorer View</ulink>  </entry></row>
            <row><entry>Google+</entry><entry>API key <ulink url="http://edit-wiki.usnet.private/dataspace/dav/wiki/VOS/VirtCartConfigGooglePlus">See details</ulink></entry><entry><ulink url="https://plus.google.com/106795562240032292110">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/https/plus.google.com/106795562240032292110">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=https://plus.google.com/106795562240032292110">Data Explorer View</ulink>  </entry></row>            
            <row><entry>Google Base</entry><entry> none </entry><entry><ulink url="http://www.google.com/base/feeds/snippets">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.google.com/base/feeds/snippets/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.google.com/base/feeds/snippets">Data Explorer View</ulink>  </entry></row>
            <row><entry>Google Book</entry><entry> none  </entry><entry><ulink url="http://books.google.com/books?id=SRadJIuhVjAC&amp;dq=GOOGLE&amp;ie=ISO-8859-1&amp;source=gbs_gdata">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://books.google.com/books?id=SRadJIuhVjAC&amp;dq=GOOGLE&amp;ie=ISO-8859-1&amp;source=gbs_gdata">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://books.google.com/books?id=SRadJIuhVjAC&amp;dq=GOOGLE&amp;ie=ISO-8859-1&amp;source=gbs_gdata">Data Explorer View</ulink>  </entry></row>
            <row><entry>Google Document</entry><entry> none </entry><entry><ulink url="http://docs.google.com/Present?docid=dc7jvc6m_1056g8fs54hp">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/docs.google.com/Present?docid=dc7jvc6m_1056g8fs54hp">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://docs.google.com/Present?docid=dc7jvc6m_1056g8fs54hp">Data Explorer View</ulink>  </entry></row>
            <row><entry>Google Social Graph</entry><entry> none </entry><entry><ulink url="http://socialgraph.apis.google.com/lookup?q=http://example.com/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/socialgraph.apis.google.com/lookup?q=http://example.com/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://socialgraph.apis.google.com/lookup?q=http://example.com/">Data Explorer View</ulink>  </entry></row>
            <row><entry>Google Spreadsheet</entry><entry> none </entry><entry><ulink url="http://spreadsheets.google.com/pub?key=p9pdwsai2hDMsLkXsoM05KQ&amp;gid=1">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/spreadsheets.google.com/pub?key=p9pdwsai2hDMsLkXsoM05KQ&amp;gid=1">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://spreadsheets.google.com/pub?key=p9pdwsai2hDMsLkXsoM05KQ&amp;gid=1">Data Explorer View</ulink>  </entry></row>
            <row><entry>Hoovers</entry><entry> none </entry><entry><ulink url="http://www.hoovers.com/openlink/--ID__104304--/free-co-factsheet.xhtml">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.hoovers.com/openlink/--ID__104304--/free-co-factsheet.xhtml">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.hoovers.com/openlink/--ID__104304--/free-co-factsheet.xhtml">Data Explorer View</ulink>  </entry></row>
            <row><entry>ISBN</entry><entry> API Key </entry><entry><ulink url="http://isbndb.com/search-all.html?kw=King+Stephen">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/isbndb.com/search-all.html?kw=King+Stephen">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://isbndb.com/search-all.html?kw=King+Stephen">Data Explorer View</ulink>  </entry></row>
            <row><entry>LastFM</entry><entry> API Key </entry><entry><ulink url="http://www.last.fm/music/+noredirect/Teddy+Pendegrass">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.last.fm/music/+noredirect/Teddy+Pendegrass">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.last.fm/music/+noredirect/Teddy+Pendegrass">Data Explorer View</ulink>  </entry></row>
            <row><entry>LibraryThing</entry><entry> API Key </entry><entry><ulink url="http://www.librarything.com/work/1060">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.librarything.com/work/1060">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.librarything.com/work/1060">Data Explorer View</ulink>  </entry></row>
            <row><entry>LinkedIn</entry><entry> API key and secret, OAuth token <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtCartConfigLinkedIn">See details</ulink> </entry><entry><ulink url="http://www.linkedin.com/in/kidehen">example</ulink></entry><entry><ulink url="http://uriburner.com/about/html/http/uriburner.com/about/id/entity/http/www.linkedin.com/in/kidehen">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.linkedin.com/in/kidehen">Data Explorer View</ulink>  </entry></row>
            <row><entry>LSID</entry><entry> none </entry><entry><ulink url="http://lsid.tdwg.org/urn:lsid:ubio.org:namebank:11815">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/lsid.tdwg.org/urn:lsid:ubio.org:namebank:11815">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://lsid.tdwg.org/urn:lsid:ubio.org:namebank:11815">Data Explorer View</ulink>  </entry></row>
            <row><entry>Meetup</entry><entry> API Key </entry><entry><ulink url="http://www.meetup.com/Nonpracticing-Lawyers-Social-Support-Group/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.meetup.com/Nonpracticing-Lawyers-Social-Support-Group/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.meetup.com/Nonpracticing-Lawyers-Social-Support-Group/">Data Explorer View</ulink>  </entry></row>
            <row><entry>MusicBrainz</entry><entry> none </entry><entry><ulink url="http://musicbrainz.org/release/37e955d4-a53c-45aa-a812-1b23b88dbc13.html">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/musicbrainz.org/release/37e955d4-a53c-45aa-a812-1b23b88dbc13.html">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://musicbrainz.org/release/37e955d4-a53c-45aa-a812-1b23b88dbc13.html">Data Explorer View</ulink>  </entry></row>
            <row><entry>Ning Metadata</entry><entry> none </entry><entry><ulink url="http://www.ning.com/search?q=new+york+city&amp;sf=rs&amp;rs=1">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.ning.com/search?q=new+york+city&amp;sf=rs&amp;rs=1">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.ning.com/search?q=new+york+city&amp;sf=rs&amp;rs=1">Data Explorer View</ulink>  </entry></row>
            <row><entry>OAI</entry><entry> none </entry><entry><ulink url="http://news.cnet.com/IBM%2C-screensaver-to-tackle-smallpox/2100-1008_3-983374.html">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/news.cnet.com/IBM%2C-screensaver-to-tackle-smallpox/2100-1008_3-983374.html">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://news.cnet.com/IBM%2C-screensaver-to-tackle-smallpox/2100-1008_3-983374.html">Data Explorer View</ulink>  </entry></row>
            <row><entry>Open Social</entry><entry> none </entry><entry><ulink url="http://socialgraph.apis.google.com/otherme?pretty=1&amp;amp;q=www.openlinksw.com/blog/~kidehen/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/socialgraph.apis.google.com/otherme?pretty=1&amp;amp;q=www.openlinksw.com/blog/~kidehen/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://socialgraph.apis.google.com/otherme?pretty=1&amp;amp;q=www.openlinksw.com/blog/~kidehen/">Data Explorer View</ulink>  </entry></row>
            <row><entry>OpenLibrary</entry><entry> none </entry><entry><ulink url="http://openlibrary.org/b/OL1230137M">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/openlibrary.org/b/OL1230137M">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://openlibrary.org/b/OL1230137M">Data Explorer View</ulink>  </entry></row>
            <row><entry>OpenStreetMap</entry><entry> none </entry><entry><ulink url="http://openstreetmap.org/?lat=57.6569&amp;amp;lon=11.8886&amp;amp;zoom=14&amp;amp;layers=B000FTF">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/openstreetmap.org/?lat=57.6569&amp;amp;lon=11.8886&amp;amp;zoom=14&amp;amp;layers=B000FTF">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://openstreetmap.org/?lat=57.6569&amp;amp;lon=11.8886&amp;amp;zoom=14&amp;amp;layers=B000FTF">Data Explorer View</ulink>  </entry></row>
            <row><entry>oReilly</entry><entry> none </entry><entry><ulink url="http://oreilly.com/catalog/9780596523206/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/oreilly.com/catalog/9780596523206/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://oreilly.com/catalog/9780596523206/">Data Explorer View</ulink>  </entry></row>
            <row><entry>Picasa</entry><entry> none </entry><entry><ulink url="http://picasaweb.google.com/lh/explore#utm_medium=embed&amp;utm_source=pwalogin">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://picasaweb.google.com/lh/explore#utm_medium=embed&amp;utm_source=pwalogin">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://picasaweb.google.com/lh/explore#utm_medium=embed&amp;utm_source=pwalogin">Data Explorer View</ulink>  </entry></row>
            <row><entry>Radio Pop</entry><entry> none </entry><entry><ulink url="http://www.radiopop.co.uk/users/tristanf/friends.xml">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.radiopop.co.uk/users/tristanf/friends.xml">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.radiopop.co.uk/users/tristanf/friends.xml">Data Explorer View</ulink>  </entry></row>
            <row><entry>relLicense</entry><entry> none </entry><entry><ulink url="http://creativecommons.org/licenses/by-nc/3.0/us/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/creativecommons.org/licenses/by-nc/3.0/us/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://creativecommons.org/licenses/by-nc/3.0/us/">Data Explorer View</ulink>  </entry></row>
            <row><entry>Revyu</entry><entry> none </entry><entry><ulink url="http://revyu.com/people/smonroe/about/html">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/revyu.com/people/smonroe/about/html">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://revyu.com/people/smonroe/about/html">Data Explorer View</ulink>  </entry></row>
            <row><entry>Rhapsody</entry><entry> none </entry><entry><ulink url="http://mp3.rhapsody.com/playlistdetail?playlistId=ply.25288413">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/mp3.rhapsody.com/playlistdetail?playlistId=ply.25288413">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://mp3.rhapsody.com/playlistdetail?playlistId=ply.25288413">Data Explorer View</ulink>  </entry></row>
            <row><entry>SalesForce.com</entry><entry>API Key,user login</entry><entry><ulink url="http://linkeddata.uriburner.com/about/rdf/https://na6.salesforce.com/0038000000ViF6k#this">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/linkeddata.uriburner.com/about/rdf/https://na6.salesforce.com/0038000000ViF6k#this">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/rdf/https://na6.salesforce.com/0038000000ViF6k#this">Data Explorer View</ulink>  </entry></row>
            <row><entry>SlideShare</entry><entry> API Key, SharedSecret </entry><entry><ulink url="http://www.slideshare.net/rumito/rdf-views-of-sql-data-power-point-presentation-1-173180/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.slideshare.net/rumito/rdf-views-of-sql-data-power-point-presentation-1-173180/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.slideshare.net/rumito/rdf-views-of-sql-data-power-point-presentation-1-173180/">Data Explorer View</ulink>  </entry></row>
            <row><entry>SlideSix</entry><entry> none </entry><entry><ulink url="http://slidesix.com/view/ESWC2008SPARQLBIOpenLink">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/slidesix.com/view/ESWC2008SPARQLBIOpenLink">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://slidesix.com/view/ESWC2008SPARQLBIOpenLink">Data Explorer View</ulink>  </entry></row>
            <row><entry>SVG</entry><entry> none </entry><entry><ulink url="http://www.schepers.cc/semweb/rdf-diagram.svg">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.schepers.cc/semweb/rdf-diagram.svg">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.schepers.cc/semweb/rdf-diagram.svg">Data Explorer View</ulink>  </entry></row>
            <row><entry>Tesco</entry><entry> none </entry><entry><ulink url="http://www.tesco.com/superstore/xpi/6/xpi54427686.htm">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.tesco.com/superstore/xpi/6/xpi54427686.htm">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.tesco.com/superstore/xpi/6/xpi54427686.htm">Data Explorer View</ulink>  </entry></row>
            <row><entry>Tumblr</entry><entry> none </entry><entry><ulink url="http://cupcakesoftheday.tumblr.com/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://cupcakesoftheday.tumblr.com/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://cupcakesoftheday.tumblr.com/">Data Explorer View</ulink>  </entry></row>
            <row><entry>TWFY (theyworkforyou)</entry><entry>	API Key  </entry><entry><ulink url="http://www.theyworkforyou.com/mp/diane_abbott/hackney_north_and_stoke_newington">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.theyworkforyou.com/mp/diane_abbott/hackney_north_and_stoke_newington">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.theyworkforyou.com/mp/diane_abbott/hackney_north_and_stoke_newington">Data Explorer View</ulink>  </entry></row>
            <row><entry>Twitter</entry><entry> API key and secret, OAuth token <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtCartConfigTwitter">See details</ulink> </entry><entry><ulink url="http://twitter.com/kidehen">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://twitter.com/kidehen">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://twitter.com/kidehen">Data Explorer View</ulink>  </entry></row>
            <row><entry>Ustream</entry><entry> none </entry><entry><ulink url="http://www.ustream.tv/recorded/4848859/highlight/55618">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.ustream.tv/recorded/4848859/highlight/55618">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.ustream.tv/recorded/4848859/highlight/55618">Data Explorer View</ulink>  </entry></row>
            <row><entry>Web Resource CC (Licenses) </entry><entry> none </entry><entry><ulink url="http://creativecommons.org/licenses/by-nc-nd/2.5/">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/creativecommons.org/licenses/by-nc-nd/2.5/">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://creativecommons.org/licenses/by-nc-nd/2.5/">Data Explorer View</ulink>  </entry></row>
            <row><entry>Wikipedia</entry><entry> none </entry><entry><ulink url="http://en.wikipedia.org/wiki/House_MD">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://en.wikipedia.org/wiki/House_MD">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://en.wikipedia.org/wiki/House_MD">Data Explorer View</ulink>  </entry></row>
            <row><entry>xFolk</entry><entry> none </entry><entry><ulink url="http://blogmarks.net/marks/tag/hdd">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/blogmarks.net/marks/tag/hdd">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://blogmarks.net/marks/tag/hdd">Data Explorer View</ulink>  </entry></row>
            <row><entry>Yahoo! Finance</entry><entry> none </entry><entry><ulink url="http://finance.yahoo.com/q?s=AAPL">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/finance.yahoo.com/q?s=AAPL">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://finance.yahoo.com/q?s=AAPL">Data Explorer View</ulink>  </entry></row>
            <row><entry>Yahoo! SearchMonkey</entry><entry> none </entry><entry><ulink url="http://tech.groups.yahoo.com/group/ysearchboss/message/1983">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/tech.groups.yahoo.com/group/ysearchboss/message/1983">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://tech.groups.yahoo.com/group/ysearchboss/message/1983">Data Explorer View</ulink>  </entry></row>
            <row><entry>Yahoo! Traffic Data</entry><entry> none </entry><entry><ulink url="http://local.yahooapis.com/MapsService/rss/trafficData.xml?appid=YdnDemo&amp;street=701+First+Avenue&amp;city=Sunnyvale&amp;state=CA">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/local.yahooapis.com/MapsService/rss/trafficData.xml?appid=YdnDemo&amp;street=701+First+Avenue&amp;city=Sunnyvale&amp;state=CA">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://local.yahooapis.com/MapsService/rss/trafficData.xml?appid=YdnDemo&amp;street=701+First+Avenue&amp;city=Sunnyvale&amp;state=CA">Data Explorer View</ulink>  </entry></row>
            <row><entry>Yahoo! Weather</entry><entry> none </entry><entry><ulink url="http://weather.yahooapis.com/forecastrss?p=94089">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/weather.yahooapis.com/forecastrss?p=94089">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://weather.yahooapis.com/forecastrss?p=94089">Data Explorer View</ulink>  </entry></row>
            <row><entry>Yelp</entry><entry> none </entry><entry><ulink url="http://api.yelp.com/business_review_search?term=yelp&amp;tl_lat=37.9&amp;tl_long=-122.5&amp;br_lat=37.788022&amp;br_long=-122.399797&amp;limit=3&amp;ywsid=XXXXXXXXXXXXXXXXXX">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/api.yelp.com/business_review_search?term=yelp&amp;tl_lat=37.9&amp;tl_long=-122.5&amp;br_lat=37.788022&amp;br_long=-122.399797&amp;limit=3&amp;ywsid=XXXXXXXXXXXXXXXXXX">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://api.yelp.com/business_review_search?term=yelp&amp;tl_lat=37.9&amp;tl_long=-122.5&amp;br_lat=37.788022&amp;br_long=-122.399797&amp;limit=3&amp;ywsid=XXXXXXXXXXXXXXXXXX">Data Explorer View</ulink>  </entry></row>
            <row><entry>Youtube</entry><entry> none </entry><entry><ulink url="http://www.youtube.com/watch?v=rOsPKjbMvxY">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/www.youtube.com/watch?v=rOsPKjbMvxY">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.youtube.com/watch?v=rOsPKjbMvxY">Data Explorer View</ulink>  </entry></row>
            <row><entry>Zillow</entry><entry> none </entry><entry><ulink url="http://www.zillow.com/homedetails/26-George-Rd-Maynard-MA-01754/57086765_zpid">example</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.zillow.com/homedetails/26-George-Rd-Maynard-MA-01754/57086765_zpid">HTML Representation</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.zillow.com/homedetails/26-George-Rd-Maynard-MA-01754/57086765_zpid">Data Explorer View</ulink>  </entry></row>
</tbody>
</tgroup>
</table>
</sect4>
  </sect3>
  <sect3 id="virtuosospongercartridgesmeta"><title>Meta Cartridges</title>
    <para>A Meta Cartridge submits a Resource to a third-party Web Service for processing. Returned RDF 
    	supplements the RDF generated by Extractor and other Meta Cartridges. Locally generated RDF may 
    	also be submitted to the third-party services, instead-of or in-addition-to the original 
    	Resource itself.
    </para>	
    <para>Default Sponger behavior is for all installed Meta Cartridges to be brought to bear on all 
    	submitted Resources:
    </para>	
<table>
<tgroup cols="5">
<thead>
  <row>
    <entry>Cartridge</entry><entry>Needs</entry><entry>Sample URI</entry><entry>Resource Description</entry><entry>Linked Data Graph</entry><entry>Notes</entry>
  </row>
</thead>
<tbody>
<row><entry>Alchemy</entry><entry>API Key</entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/jira.codehaus.org/browse/CONTINUUM-884%01alchemy_entity_1">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/jira.codehaus.org/browse/CONTINUUM-884">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://jira.codehaus.org/browse/CONTINUUM-884">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Amazon Search for products</entry><entry>API Key, secret</entry><entry><ulink url="http://www.amazon.com/o/ASIN/0553383043#this">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.amazon.com/o/ASIN/0553383043">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.amazon.com/o/ASIN/0553383043">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>BBC Links</entry><entry></entry><entry><ulink url="http://news.bbc.co.uk">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://twitter.com/bbcnews">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://twitter.com/bbcnews">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>BestBuy Search for products</entry><entry>API Key</entry><entry><ulink url="http://www.bestbuy.com/site/KOSS+-+Pro+DJ100+Professional+Over-the-Ear+Headphones+-+Black/9745789.p?skuId=9745789&amp;id=1218165774336">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.bestbuy.com/site/KOSS+-+Pro+DJ100+Professional+Over-the-Ear+Headphones+-+Black/9745789.p?skuId=9745789&amp;id=1218165774336">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.bestbuy.com/site/KOSS+-+Pro+DJ100+Professional+Over-the-Ear+Headphones+-+Black/9745789.p?skuId=9745789&amp;id=1218165774336">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Bing</entry><entry>API Key</entry><entry><ulink url="http://www.bing.com/videos/watch/video/cap-reattached-to-leaking-bp-well/6kn9out">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/twitter.com/bing">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://twitter.com/bing">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Bit.ly</entry><entry></entry><entry><ulink url="http://bit.ly/cGwRgc">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.springerlink.com/content/528h15634q332p02/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.springerlink.com/content/528h15634q332p02/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>CNET Search for products</entry><entry>API Key</entry><entry><ulink url="http://www.xomreviews.com/shopper.cnet.com">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.xomreviews.com/shopper.cnet.com">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.xomreviews.com/shopper.cnet.com">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Collecta</entry><entry></entry><entry><ulink url="http://www.crunchbase.com/company/microsoft">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.crunchbase.com/company/microsoft">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.crunchbase.com/company/microsoft">Data Explorer View</ulink> </entry><entry>Check rdfs:seeAlso <ulink url="http://localhost:9301/about/html/http/social.msdn.microsoft.com/Forums/en-US/wpf/thread/108fbffc-c849-4d60-ab3e-00c425342aaf">links</ulink> found for microsoft.</entry></row>
<row><entry>Crunchbase</entry><entry></entry><entry><ulink url="http://www.crunchbase.com/company/radar-music-videos">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://techcrunch.com/2010/05/02/rapportive/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://techcrunch.com/2010/05/02/rapportive/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Dapper Search</entry><entry></entry><entry><ulink url="http://www.dapper.net/dapplications/Blotter/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.alexa.com/siteinfo/dapper.net">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.alexa.com/siteinfo/dapper.net">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>DBpedia Meta</entry><entry></entry><entry><ulink url="http://dbpedia.org/page/TweetDeck">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.programmableweb.com/api/dbpedia">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.programmableweb.com/api/dbpedia">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Delicious Meta</entry><entry>User Login</entry><entry><ulink url="http://delicious.com/tag/crunchbase">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.crunchbase.com/company/google">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/id/entity/http/www.crunchbase.com/company/google">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Digg.com</entry><entry></entry><entry><ulink url="http://google.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://google.com/">Data Explorer View</ulink> </entry><entry>Check rdfs:seeAlso the <ulink url="http://mashable.com/2010/07/12/google-app-inventor/">links</ulink> from Digg.com .</entry></row>
<row><entry>Discogs</entry><entry>API Key, php plugin, DBpedia Extractor</entry><entry><ulink url="http://www.discogs.com/Todd-Barton-and...ic-Poetry-Of-The-Kesh/release/2053074">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.amazon.com/o/ASIN/0553383043">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.amazon.com/o/ASIN/0553383043">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Document Links</entry><entry></entry><entry>example </entry><entry> HTML Representation </entry><entry> Data Explorer View </entry><entry></entry></row>
<row><entry>eBay Search for products</entry><entry>account, API Key</entry><entry><ulink url="http://shop.ebay.com/stores/__W0QQLHQ5fSellerWithStoreZ1QQ_nkwZtoys?_rdc=1">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/wordpress.xdroop.com/space/HotWheels/Buying+Hotwheels+On+eBay">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://wordpress.xdroop.com/space/HotWheels/Buying+Hotwheels+On+eBay">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Evri Meta</entry><entry></entry><entry><ulink url="http://www.evri.com/organization/apple-store-0x60ee5">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.evri.com/organization/apple-store-0x60ee5">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.evri.com/organization/apple-store-0x60ee5">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Facebook</entry><entry>API Key, secret, persistent-session-id.</entry><entry> <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtCartConfigFacebook">See details</ulink></entry><entry><ulink url="http://www.facebook.com/people/Richard-Max-Koster/1319314117">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.facebook.com/people/Richard-Max-Koster/1319314117">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.facebook.com/people/Richard-Max-Koster/1319314117">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Flickr Search for photos</entry><entry>API Key</entry><entry><ulink url="http://farm5.static.flickr.com/4034/4406875198_7e562051cc.jpg">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/id/entity/http/www.google.com/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>FOAF-Search</entry><entry></entry><entry><ulink url="http://www.slideshare.net/kidehen">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.slideshare.net/kidehen/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.slideshare.net/kidehen">Data Explorer View</ulink> </entry><entry>Check rdfs:seeAlso at: <ulink url="http://identi.ca/user/14092">link1</ulink>, <ulink url="http://semantictweet.com/kidehen#me">link2</ulink>, <ulink url="http://myopenlink.net:8890/dataspace/person/kidehen#this">link3</ulink></entry></row>
<row><entry>Foursquare</entry><entry></entry><entry><ulink url="http://foursquare.com/scobleizer">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/foursquare.com/scobleizer">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://foursquare.com/scobleizer">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Freebase NYTC</entry><entry>API Key</entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/rdf.freebase.com/ns/en.john_kerry">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/html/http://www.freebase.com/view/en/john_kerry">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/html/http://www.freebase.com/view/en/john_kerry">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Freebase NYTCF</entry><entry>API Key</entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/rdf.freebase.com/ns/en.john_kerry">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/html/http://www.freebase.com/view/en/john_kerry">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/html/http://www.freebase.com/view/en/john_kerry">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>FriendFeed</entry><entry></entry><entry><ulink url="http://friendfeed.com/informationweek/0f212311/web-link-shrinkage-powers-spam-surge">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.informationweek.com/news/security/vulnerabilities/showArticle.jhtml?articleID=218401098">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.informationweek.com/news/security/vulnerabilities/showArticle.jhtml?articleID=218401098">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Geonames Meta</entry><entry></entry><entry><ulink url="http://ws.geonames.org/search?q=london&amp;maxRows=10&amp;type=rdf">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/ws.geonames.org/search?q=london&amp;maxRows=10&amp;type=rdf">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://ws.geonames.org/search?q=london&amp;maxRows=10&amp;type=rdf">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Geopoints</entry><entry></entry><entry><ulink url="http://www.geopoint.nl/en/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.mit.edu/~6.170/api/ps2/GeoPoint.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.mit.edu/~6.170/api/ps2/GeoPoint.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Gowalla</entry><entry></entry><entry><ulink url="http://gowalla.com/spots/7163409">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://gowalla.com/spots/7163409">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://gowalla.com/spots/7163409">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Get Glue Meta</entry><entry>User Login</entry><entry><ulink url="http://getglue.com">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://blog.adaptiveblue.com/?p=4746">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://blog.adaptiveblue.com/?p=4746">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Get Glue</entry><entry></entry><entry><ulink url="http://getglue.com/smartlinks">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.readwriteweb.com/archives/get_glue_on_your_iphone.php">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.readwriteweb.com/archives/get_glue_on_your_iphone.php">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Google Buzz</entry><entry></entry><entry> <ulink url="http://google.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://google.com/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Google Book</entry><entry></entry><entry><ulink url="http://www.google.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.google.com/">Data Explorer View</ulink> </entry><entry>Check rdfs:seeAlso links like <ulink url="http://books.google.com/*">this one</ulink></entry></row>
<row><entry>Google Plus</entry><entry></entry><entry><ulink url="https://plus.google.com/112399767740508618350">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/https://plus.google.com/112399767740508618350">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=https://plus.google.com/112399767740508618350">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Google Places</entry><entry></entry><entry><ulink url="http://maps.google.com/maps?cid=9095091124955696692">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://maps.google.com/maps?cid=9095091124955696692">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://maps.google.com/maps?cid=9095091124955696692">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Google Search</entry><entry></entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.google.com/finance?q=NYSE:MON%01Recognition1">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.google.com/finance?q=NYSE:MON">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.google.com/finance?q=NYSE:MON">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Google Social Graph</entry><entry></entry><entry><ulink url="http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com/about.rdf">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.openlinksw.com/blog/~kidehen/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.openlinksw.com/blog/~kidehen/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Guardian</entry><entry>API Key</entry><entry><ulink url="http://www.guardian.co.uk/politics/2010/mar/15/gordon-brown-continue-as-labour-leader">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.alexa.com/siteinfo/guardian.co.uk">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.alexa.com/siteinfo/guardian.co.uk">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Hoovers</entry><entry>API Key</entry><entry><ulink url="http://www.hoovers.com/openlink/--ID__104304--/free-co-factsheet.xhtml">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.hoovers.com/company/OpenLink_Software_Inc/rfcyfci-1.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.hoovers.com/company/OpenLink_Software_Inc/rfcyfci-1.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Jigsaw (company)</entry><entry></entry><entry><ulink url="http://google.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://google.com/">Data Explorer View</ulink> </entry><entry>Check the c:location <ulink url="proxy:entity/http/www.google.com/%23address">link</ulink></entry></row>
<row><entry>Jigsaw (person)</entry><entry></entry><entry><ulink url="http://www.crunchbase.com/person/todd-sharp">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.crunchbase.com/person/todd-sharp">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.crunchbase.com/person/todd-sharp">Data Explorer View</ulink> </entry><entry>Check several Jigsaw search seeAlso <ulink url="http://www.jigsaw.com/scid5307274/todd_sharp.xhtml">link</ulink>.</entry></row>
<row><entry>Journalisted</entry><entry></entry><entry><ulink url="http://journalisted.com/walter-gammie">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/whois.domaintools.com/journalisted.com">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://whois.domaintools.com/journalisted.com">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Last.FM</entry><entry>API Key</entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.last.fm/music/Rod+Stewart/Greatest+Hits">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.last.fm/music/Rod+Stewart/Greatest+Hits">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.last.fm/music/Rod+Stewart/Greatest+Hits">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>LinkedIn</entry><entry>API Key and Session Key;</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtLinkedInMeta">See details</ulink></entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.jigsaw.com/scid34931678/vinod_khosla.xhtml">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.jigsaw.com/scid34931678/vinod_khosla.xhtml">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.jigsaw.com/scid34931678/vinod_khosla.xhtml">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Local Search</entry><entry></entry><entry>example </entry><entry> HTML Representation </entry><entry> Data Explorer View </entry><entry></entry></row>
<row><entry>LOD</entry><entry></entry><entry><ulink url="http://lod.openlinksw.com/b3s/search.vsp?q=1">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/lod.openlinksw.com/b3s/search.vsp?q=1">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://lod.openlinksw.com/b3s/search.vsp?q=1">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>MIME Type</entry><entry></entry><entry>example </entry><entry> HTML Representation </entry><entry> Data Explorer View </entry><entry></entry></row>
<row><entry>New York Times</entry><entry>API Key</entry><entry><ulink url="http://www.nytimes.com/2005/03/17/technology/circuits/17stat.html?ex=1268715600&amp;amp;en=6840507aca561cf8&amp;amp;ei=5090&amp;amp;partner=rssuserland">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/download.cnet.com/New-York-Times-Crosswords-Monthly-Subscription/3000-2111_4-10918973.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://download.cnet.com/New-York-Times-Crosswords-Monthly-Subscription/3000-2111_4-10918973.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>NPR Meta</entry><entry>API Key</entry><entry><ulink url="http://www.crunchbase.com/company/google">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.crunchbase.com/company/google">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.crunchbase.com/company/google">Data Explorer View</ulink> </entry><entry>Check rdfs:seeAlso links like: <ulink url="http://www.npr.org/templates/rss/podcast.php?id=510236">link1</ulink>; <ulink url="http://www.npr.org/templates/rss/podcast.php?id=510235">link2</ulink>; <ulink url="http://www.npr.org/templates/rss/podcast.php?id=510213">link3</ulink>.</entry></row>
<row><entry>NYT: The Article Search</entry><entry></entry><entry><ulink url="http://twitter.com/bbcnews">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/bbcnews">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://twitter.com/bbcnews">Data Explorer View</ulink> </entry><entry>Check the rdfs:seeAlso: <ulink url="proxy:entity/http/twitter.com/bbcnews%23nytimes_1">link</ulink>. </entry></row>
<row><entry>NYT: The TimesTags</entry><entry></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/rdf.freebase.com/ns/en.john_kerry">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/html/http://www.freebase.com/view/en/john_kerry">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/html/http://www.freebase.com/view/en/john_kerry">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>OpenCalais</entry><entry>any html page</entry><entry><ulink url="http://d.opencalais.com/er/company/ralg-tr1r/7cf71bb9-2969-35d3-8fa1-3723bdb5958c">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/news.cnet.com/8301-13505_3-9805771-16.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://news.cnet.com/8301-13505_3-9805771-16.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Oreilly Search for products</entry><entry></entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/oreilly.com/catalog/9780596007973">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/oreilly.com/catalog/9780596007973/?@Lookup@=17&amp;refresh=0">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/id/entity/http/oreilly.com/catalog/9780596007973/?@Lookup@=17&amp;refresh=0">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Primal</entry><entry></entry><entry><ulink url="http://www.google.com">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.google.com">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.google.com">Data Explorer View</ulink> </entry><entry>Check the set of sioc:topic and scot:hasScot.</entry></row>
<row><entry>ProgrammableWeb</entry><entry></entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.productwiki.com/samsung-un46c6400%01Product">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.productwiki.com/samsung-un46c6400%01Product">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/id/entity/http/www.productwiki.com/samsung-un46c6400%01Product">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Provenance</entry><entry></entry><entry><ulink url="http://www.amazon.com/Catch-22-Joseph-Heller/dp/0684833395/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.amazon.com/Catch-22-Joseph-Heller/dp/0684833395/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.amazon.com/Catch-22-Joseph-Heller/dp/0684833395/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Punkt</entry><entry></entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.youtube.com/watch?v=DrZvn1qckIs">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/www.youtube.com/watch?v=DrZvn1qckIs">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/id/entity/http/www.youtube.com/watch?v=DrZvn1qckIs">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>RapLeaf</entry><entry></entry><entry><ulink url="http://www.facebook.com/people/Roman-Korolenko/1465877213">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.crunchbase.com/service-provider/ivan-pr">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.crunchbase.com/service-provider/ivan-pr">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>SameAs.org</entry><entry></entry><entry><ulink url="http://sameas.org/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.ckan.net/package/sameas">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.ckan.net/package/sameas">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Sindice</entry><entry></entry><entry><ulink url="http://sindice.com/developers/inspector?doReasoning=true&amp;amp;q=tim+berners+lee&amp;amp;qt=term&amp;amp;url=http%3A%2F%2Fwww.nylon.gr%2Ftag%2Ftim-berners-lee%2F">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.killerstartups.com/Web20/sindice-com-the-semantic-web-index">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.killerstartups.com/Web20/sindice-com-the-semantic-web-index">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>SimpleGeo</entry><entry></entry><entry><ulink url="https://simplegeo.com/SG_0ODZdCvKJeB22EyTOyiLxO_37.772357_-122.405783@1294081369/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/https://simplegeo.com/SG_0ODZdCvKJeB22EyTOyiLxO_37.772357_-122.405783@1294081369/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=https://simplegeo.com/SG_0ODZdCvKJeB22EyTOyiLxO_37.772357_-122.405783@1294081369/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Technorati</entry><entry>API Key</entry><entry><ulink url="http://support.technorati.com/support/siteguide/tags">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/download.cnet.com/Technorati-Tag/3000-12565_4-10658084.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://download.cnet.com/Technorati-Tag/3000-12565_4-10658084.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Tesco Product Search</entry><entry>User Login, DeveloperKey, ApplicationKey</entry><entry><ulink url="http://www.tesco.com/groceries/Product/Details/?id=252330678">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.tesco.com/groceries/Product/Details/?id=252330678">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.tesco.com/groceries/Product/Details/?id=252330678">Data Explorer View</ulink> </entry><entry>Check set of Tesco rdfs:seeAlso links like <ulink url="tesco:groceries/Product/Details/?id=252990345">this one</ulink>.</entry></row>
<row><entry>Topsy</entry><entry></entry><entry><ulink url="http://google.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://google.com/">Data Explorer View</ulink> </entry><entry>Check the <ulink url="http://www1.folha.uol.com.br/folha/informatica/ult124u675304.shtml">rdfs:seeAlso</ulink> from topsy.com.</entry></row>
<row><entry>TrueKnowledge</entry><entry></entry><entry><ulink url="http://www.microsoft.com">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.microsoft.com">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.microsoft.com">Data Explorer View</ulink> </entry><entry>Check set of rdfs:seeAlso links like: <ulink url="http://www.microsoft.com/">link1</ulink>; <ulink url="http://www.microsoft.com/worldwide">link2</ulink>; <ulink url="http://en.wikipedia.org/wiki/Microsoft">link3</ulink>.</entry></row>
<row><entry>Tweetme</entry><entry></entry><entry><ulink url="http://google.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/google.com/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://google.com/">Data Explorer View</ulink> </entry><entry>Check the <ulink url="http://tweetmeme.com/story/12017895">rdfs:seeAlso</ulink> link.</entry></row>
<row><entry>Twitter Meta</entry><entry>User Login</entry><entry><ulink url="http://twitter.com/Techmeme/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.techmeme.com/081001/p43">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.techmeme.com/081001/p43">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>uClassify</entry><entry></entry><entry><ulink url="http://linkeddata.uriburner.com/about/id/entity/http/www.openlinksw.com/blog/~kidehen/%01Russian">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/html/http://www.openlinksw.com/blog/~kidehen/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/html/http://www.openlinksw.com/blog/~kidehen/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Uclassify</entry><entry></entry><entry><ulink url="http://www.crunchbase.com/company/google">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.crunchbase.com/company/google">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.crunchbase.com/company/google">Data Explorer View</ulink> </entry><entry>Check diff langs uc:class: <ulink url="proxy:entity/http/www.crunchbase.com/company/google%23Arabic">link</ulink></entry></row>
<row><entry>UMBEL</entry><entry>min-score, max-results</entry><entry><ulink url="http://fgiasson.com/blog/index.php/2008/08/29/umbel-as-a-coherent-framework-to-support-ontology-development/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/umbel.zitgist.com/api_philosophy.php">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http/umbel.zitgist.com/api_philosophy.php">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>USA Today Best-Selling Books</entry><entry></entry><entry><ulink url="http://api.usatoday.com/open/bestsellers/books/titles?title=HARRY+POTTER&amp;api_key=8ge2bvrna8x27wvc84qhz3e7&amp;encoding=xml">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://api.usatoday.com/open/bestsellers/books/titles?title=HARRY+POTTER&amp;api_key=8ge2bvrna8x27wvc84qhz3e7&amp;encoding=xml">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://api.usatoday.com/open/bestsellers/books/titles?title=HARRY+POTTER&amp;api_key=8ge2bvrna8x27wvc84qhz3e7&amp;encoding=xml">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Ustream</entry><entry></entry><entry><ulink url="http://www.ustream.tv/recorded/4848859/highlight/55618">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.ustream.tv/recorded/4848859/highlight/55618">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.ustream.tv/recorded/4848859/highlight/55618">Data Explorer View</ulink> </entry><entry>Check rdfs:seeAlso links like: <ulink url="http://www.ustream.tv/recorded/7039249/highlight/75027">this one</ulink>.</entry></row>
<row><entry>Virtuoso Faceted Web Service</entry><entry></entry><entry> example </entry><entry> HTML Representation </entry><entry> Data Explorer View </entry><entry></entry></row>
<row><entry>voID Statistics</entry><entry></entry><entry>example </entry><entry> HTML Representation </entry><entry> Data Explorer View </entry><entry></entry></row>
<row><entry>whoisi?</entry><entry></entry><entry><ulink url="http://whoisi.com/api/getPersonForURL?app=example&amp;url=http://boatcovers.bestdealsoffer.com/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>World Bank</entry><entry>API Key</entry><entry><ulink url="http://linkeddata.uriburner.com/about/html/http/web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTEDUCATION/0,,contentMDK:20263110~menuPK:531742~pagePK:210058~piPK:210062~theSitePK:282386,00.html">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.lib.berkeley.edu/doemoff/govinfo/intl/gov_ibrd.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.lib.berkeley.edu/doemoff/govinfo/intl/gov_ibrd.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>WorldCat Basic Search</entry><entry></entry><entry><ulink url="http://www.crunchbase.com/company/yahoo">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.crunchbase.com/company/yahoo">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.crunchbase.com/company/yahoo">Data Explorer View</ulink> </entry><entry>Check seeAlso links like <ulink url="http://worldcat.org/oclc/613205142">this one</ulink>.</entry></row>
<row><entry>xISBN</entry><entry>API Key</entry><entry><ulink url="http://isbndb.com/d/book/a_concise_history_of_italy_a02.html">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://isbndb.com/d/book/a_concise_history_of_italy_a02.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://isbndb.com/d/book/a_concise_history_of_italy_a02.html">Data Explorer View</ulink> </entry><entry>Check set of owl:sameAs links: <ulink url="http://purl.org/NET/book/isbn/0670236489#book">link1</ulink>; <ulink url="http://purl.org/NET/book/isbn/0500450102#book">link2</ulink>.</entry></row>
<row><entry>XRD</entry><entry></entry><entry><ulink url="http://s2.googleusercontent.com/webfinger/?q=john@gmail.com">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/hueniverse.com/xrd/">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://hueniverse.com/xrd/">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Yahoo BOSS</entry><entry>API Key</entry><entry><ulink url="http://boss.yahooapis.com/ysearch/web/v1/UNITED+STATES?appid=cxgCRk3V34ElK7Amd8ieyy9eELzD7QrpAHTzOv6MeEBz56JUUgGi1x2vf_.pE.c-&amp;format=xml&amp;view=searchmonkey_rdf">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/kidehen?@Lookup@=10&amp;refresh=0">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://linkeddata.uriburner.com/about/html/http://linkeddata.uriburner.com/about/id/entity/http/twitter.com/kidehen?@Lookup@=10&amp;refresh=0">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Yahoo Geocode</entry><entry>API Key</entry><entry><ulink url="http://local.yahooapis.com">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http://www.programmableweb.com/api/yahoo-local-search">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.programmableweb.com/api/yahoo-local-search">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Yelp Search for business</entry><entry>API Key</entry><entry><ulink url="http://www.smallbusinesssem.com/inc-magazine-goes-deep-on-yelp/2669/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.readwriteweb.com/archives/yelp_assembles_small_business_advisory_board.php">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://www.readwriteweb.com/archives/yelp_assembles_small_business_advisory_board.php">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Zemanta</entry><entry>API Key, min-score, max-results</entry><entry><ulink url="http://developer.zemanta.com/wiki/helloworld/ruby">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/kobesearch.cpan.org/htdocs/Net-Zemanta/Net/Zemanta/Suggest.pm.html">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http://kobesearch.cpan.org/htdocs/Net-Zemanta/Net/Zemanta/Suggest.pm.html">Data Explorer View</ulink> </entry><entry></entry></row>
<row><entry>Zillow</entry><entry>API Key</entry><entry><ulink url="http://www.zillow.com/homedetails/26-George-Rd-Maynard-MA-01754/57086765_zpid/">example</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/about/html/http/www.crunchbase.com/company/zillow">HTML Representation</ulink> </entry><entry> <ulink url="http://linkeddata.uriburner.com/ode/?uri=http/www.crunchbase.com/company/zillow">Data Explorer View</ulink> </entry><entry></entry></row>
</tbody>
</tgroup>
</table>
    <sect4 id="virtuosospongercartridgetypesmetarest"><title>Meta Cartridge Usage via REST Request</title>
<para>Description.vsp underlies the /about/html/ page, and accepts the parameters described below.</para> 
<table>
<tgroup cols="4">
<thead>
  <row>
    <entry>Parameter</entry><entry>Value</entry><entry>Description</entry><entry>Example</entry>
  </row>
</thead>
<tbody>
<row>
  <entry><emphasis>@Lookup@</emphasis></entry>
  <entry> </entry>
  <entry>The type of lookup</entry>
  <entry> </entry>
</row>
<row>
  <entry></entry>
  <entry>No Value</entry>
  <entry>When value is not given (i.e., <emphasis>@Lookup@=</emphasis>), all will work as if the parameter were not present. %BR% The "Lookup" name is chosen to distinguish between parameters belonging to the URL being processed, and parameters for the Sponger.</entry>
  <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.apple.com/?@Lookup@=&amp;refresh=0">Refresh the graph with all current cartridges, either type</ulink></entry>
</row>  
<row>
  <entry></entry>
  <entry><emphasis>0</emphasis></entry>
  <entry>NLP meta only</entry>
  <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.apple.com/?@Lookup@=0&amp;refresh=0">Execute only NLP meta extraction</ulink></entry>
</row>  
<row>
  <entry></entry>
  <entry><emphasis>-2</emphasis></entry>
  <entry>Keywords-based only</entry>
  <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.apple.com/?@Lookup@=-2&amp;refresh=0">Execute only keywords-based meta extraction</ulink></entry>
</row>  
<row>
  <entry></entry>
  <entry><emphasis>x,y...</emphasis></entry>
  <entry>A list of meta cartridges to be executed, by their unique IDs. The ID column can be found in <emphasis>Conductor -&amp;gt; Linked Data -&amp;gt; Sponger -&amp;gt; Meta Cartridges</emphasis></entry>
  <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.apple.com/?@Lookup@=19,22&amp;refresh=0">Execute only CNET (ID=19) and NYT: The TimesTags (ID=22) meta cartridges</ulink></entry>
</row>  
<row>
  <entry><emphasis>refresh=0,1,2 etc.</emphasis></entry>
  <entry></entry>
  <entry><emphasis>Usage</emphasis>: for cache invalidation. When used 1 or larger number (n), adds 
  <emphasis>get:refresh "N"</emphasis> (explicit refresh interval in seconds) as a directive to Sponger. 
  A refresh of zero ("0") seconds will make a new graph on the next lookup with the 
  '<emphasis>@Lookup@</emphasis>' parameter value.</entry>
  <entry><ulink url="http://linkeddata.uriburner.com/about/html/http://www.apple.com/?@Lookup@=&amp;refresh=0">Refresh the graph with all current cartridges</ulink></entry>
</row>  
<row>
  <entry><emphasis>refresh=clean</emphasis></entry>
  <entry></entry>
  <entry><emphasis>Usage</emphasis>: for overwriting. The 'clean' usage explicitly clears the graph i.e. 
  will cause the Sponger to drop cache even if it is marked to be in the fly. Thus, if 
  network resource fetched cache by some reason is left in some inconsistent state like shutdown during the fetching, then 'clean' is required 
  as it doesn't check cache state. <emphasis>Note</emphasis>: must be used with caution as other threads 
  may be doing Network Resource Fetch at same time.</entry>
  <entry></entry>
</row>  
</tbody>
</tgroup>
</table>
    </sect4>  
    <sect4 id="virtuosospongercartridgetypesmetarestexamples"><title>Meta Cartridges Parametrized Examples</title>
<para>All examples in the table below start from the same Resource, 
	http://www.news.com, and submit it to the Sponger for processing with the single listed Meta Cartridge.</para>

<para>It can be informative to start by seeing what the results would be 
	<ulink url="http://linkeddata.uriburner.com/about/html/http/www.news.com/?@Lookup@=99999&amp;refresh=0">with no Meta Cartridges at all</ulink>.</para>  

<para>If you have a lot of time to spare, you may want to see what the results would be 
	<ulink url="http://linkeddata.uriburner.com/about/html/http/www.news.com/?@Lookup@=&amp;refresh=0">with all Meta Cartridges combined</ulink>. 
	As may be obvious, this must wait for each of the above services to respond, so it may take quite 
	some time to return.</para>

<table>
<tgroup cols="3">
<thead>
  <row>
    <entry>Cartridge</entry><entry>URL Pattern</entry><entry>Example</entry>
  </row>
</thead>
<tbody>
<row><entry>Alchemy</entry><entry>@Lookup@=8&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL1">cURL example</ulink></entry></row> 
<row><entry>Amazon Search for products</entry><entry>@Lookup@=13&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL2">cURL example</ulink></entry></row> 
<row><entry>BBC</entry><entry>@Lookup@=1665&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL3">cURL example</ulink></entry></row> 
<row><entry>BestBuy Search for products</entry><entry>@Lookup@=14&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL4">cURL example</ulink></entry></row> 
<row><entry>Bing</entry><entry>@Lookup@=11&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL5">cURL example</ulink></entry></row> 
<row><entry>Bit.ly</entry><entry>@Lookup@=915&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL6">cURL example</ulink></entry></row> 
<row><entry>CNET</entry><entry>@Lookup@=19&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL7">cURL example</ulink></entry></row> 
<row><entry>Crunchbase</entry><entry>@Lookup@=839&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL8">cURL example</ulink></entry></row> 
<row><entry>Dapper</entry><entry>@Lookup@=243&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL9">cURL example</ulink></entry></row> 
<row><entry>DBpedia</entry><entry>@Lookup@=26&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL10">cURL example</ulink></entry></row> 
<row><entry>Delicious Meta</entry><entry>@Lookup@=23&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL11">cURL example</ulink></entry></row> 
<row><entry>Discogs</entry><entry>@Lookup@=840&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL12">cURL example</ulink></entry></row> 
<row><entry>Document Links</entry><entry>@Lookup@=34&amp;refresh=0</entry><entry>   <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL13">cURL example</ulink></entry></row> 
<row><entry>eBay</entry><entry>@Lookup@=18&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL14">cURL example</ulink></entry></row>  
<row><entry>Evri Meta</entry><entry>@Lookup@=3966&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL15">cURL example</ulink></entry></row>  
<row><entry>Flickr Search for photos</entry><entry>@Lookup@=16&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL16">cURL example</ulink></entry></row>  
<row><entry>Freebase NYTC</entry><entry>@Lookup@=5&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL17">cURL example</ulink></entry></row>  
<row><entry>Freebase NYTCF</entry><entry>@Lookup@=4&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL18">cURL example</ulink></entry></row>  
<row><entry>Geonames Meta</entry><entry>@Lookup@=24&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL19">cURL example</ulink></entry></row>  
<row><entry>Geopoints</entry><entry>@Lookup@=3731&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL20">cURL example</ulink></entry></row>  
<row><entry>Get Glue Meta</entry><entry>@Lookup@=25&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL21">cURL example</ulink></entry></row>  
<row><entry>Google Search</entry><entry>@Lookup@=1382&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL22">cURL example</ulink></entry></row>  
<row><entry>Google Social Graph</entry><entry>@Lookup@=30&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL23">cURL example</ulink></entry></row>  
<row><entry>Guardian</entry><entry>@Lookup@=28&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL24">cURL example</ulink></entry></row>  
<row><entry>Hoovers</entry><entry>@Lookup@=2&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL25">cURL example</ulink></entry></row>  
<row><entry>Journalisted</entry><entry>@Lookup@=3174&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL26">cURL example</ulink></entry></row>  
<row><entry>Local Search</entry><entry>@Lookup@=15&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL27">cURL example</ulink></entry></row>  
<row><entry>LOD</entry><entry>@Lookup@=21&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL28">cURL example</ulink></entry></row>  
<row><entry>MIME Type</entry><entry>@Lookup@=1029&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL29">cURL example</ulink></entry></row>  
<row><entry>New York Times</entry><entry>@Lookup@=22&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL30">cURL example</ulink></entry></row>  
<row><entry>NPR Meta</entry><entry>@Lookup@=29&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL31">cURL example</ulink></entry></row>  
<row><entry>NYT: The Article Search</entry><entry>@Lookup@=9&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL32">cURL example</ulink></entry></row>  
<row><entry>NYT: The TimesTags</entry><entry>@Lookup@=22&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL33">cURL example</ulink></entry></row>  
<row><entry>OpenCalais</entry><entry>@Lookup@=1&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL34">cURL example</ulink></entry></row>  
<row><entry>Oreilly Search for products</entry><entry>@Lookup@=17&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL35">cURL example</ulink></entry></row>  
<row><entry>RapLeaf</entry><entry>@Lookup@=2745&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL36">cURL example</ulink></entry></row>  
<row><entry>SameAs.org</entry><entry>@Lookup@=3257&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL37">cURL example</ulink></entry></row>  
<row><entry>Sindice</entry><entry>@Lookup@=12&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL38">cURL example</ulink></entry></row>  
<row><entry>Technorati</entry><entry>@Lookup@=27&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL39">cURL example</ulink></entry></row>  
<row><entry>Tesco</entry><entry>@Lookup@=31&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL40">cURL example</ulink></entry></row>  
<row><entry>TrueKnowledge</entry><entry>@Lookup@=3967&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL41">cURL example</ulink></entry></row>  
<row><entry>Twitter</entry><entry>@Lookup@=4020&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL42">cURL example</ulink></entry></row>  
<row><entry>uClassify</entry><entry>@Lookup@=3086&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL43">cURL example</ulink></entry></row>  
<row><entry>UMBEL</entry><entry>@Lookup@=6&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL44">cURL example</ulink></entry></row>  
<row><entry>Ustream</entry><entry>@Lookup@=3902&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL45">cURL example</ulink></entry></row>  
<row><entry>Virtuoso Faceted Web Service</entry><entry>@Lookup@=21&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL46">cURL example</ulink></entry></row>  
<row><entry>voID Statistics</entry><entry>@Lookup@=35&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL47">cURL example</ulink></entry></row>  
<row><entry>whoisi?</entry><entry>@Lookup@=3052&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL48">cURL example</ulink></entry></row>  
<row><entry>World Bank</entry><entry>@Lookup@=3&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL49">cURL example</ulink></entry></row>  
<row><entry>XRD</entry><entry>@Lookup@=3650&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL50">cURL example</ulink></entry></row>  
<row><entry>Yahoo BOSS</entry><entry>@Lookup@=10&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL51">cURL example</ulink></entry></row>  
<row><entry>Yahoo Geocode</entry><entry>@Lookup@=2855&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL52">cURL example</ulink></entry></row>  
<row><entry>Yelp Search for business</entry><entry>@Lookup@=20&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL53">cURL example</ulink></entry></row>  
<row><entry>Zemanta</entry><entry>@Lookup@=7&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL54">cURL example</ulink></entry></row>  
<row><entry>Zillow</entry><entry>@Lookup@=32&amp;refresh=0</entry><entry><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamplescURL55">cURL example</ulink></entry></row>  
</tbody>
</tgroup>
</table>
</sect4>
</sect3>
 </sect2>
<sect2 id="rdfspongerprogrammerguide"><title>Sponger Programmers Guide</title>
  <para>The Sponger forms part of the extensible RDF framework built into Virtuoso Universal Server. A
key component of the Sponger's pluggable architecture is its support for Sponger Cartridges, which
themselves are comprised of an Entity Extractor and an Ontology Mapper. Virtuoso bundles numerous
pre-written cartridges for RDF data extraction from a wide range of data sources. However, developers
are free to develop their own custom cartridges. This programmer's guide describes how.</para>
  <para>The guide is a companion to the <ulink url="http://virtuoso.openlinksw.com/Whitepapers/pdf/sponger_whitepaper_10102007.pdf">Virtuoso Sponger</ulink> whitepaper. The latter describes the Sponger in depth, its architecture, configuration, use and integration with other Virtuoso facilities such as the Open Data Services (ODS) application framework. This guide focuses solely on custom cartridge development.</para>
      <sect3 id="virtuosospongeroverviewxmlset">
      <title>Configuration of CURIEs used by the Sponger</title>
      <para>For configuring CURIEs used by the Sponger which is exposed via Sponger
clients such as "description.vsp" - the VSP based information resource description utility,
you can use the <link linkend="fn_xml_set_ns_decl"><function>xml_set_ns_decl</function></link> function.</para>
      <para>Here is sample example to add curie pattern:</para>
<programlisting><![CDATA[
-- Example link: http://linkeddata.uriburner.com/about/rdf/http://twitter.com/guykawasaki/status/1144945513#this
XML_SET_NS_DECL ('uriburner',
                 'http://linkeddata.uriburner.com/about/rdf/http://',
                 2);
]]></programlisting>
    </sect3>
    <sect3 id="virtuosospongeroverviewcartarch">
      <title>Cartridge Architecture</title>
      <para>The Sponger is comprised of cartridges which are themselves comprised of an entity extractor
and an ontology mapper. Entities extracted from non-RDF resources are used as the basis for generating
structured data by mapping them to a suitable ontology. A cartridge is invoked through its cartridge hook,
a Virtuoso/PL procedure entry point and binding to the cartridge's entity extractor and ontology mapper.</para>
      <para><emphasis>Entity Extractor</emphasis></para>
      <para>When an RDF aware client requests data from a network accessible resource via the Sponger the
following events occur:</para>
      <itemizedlist mark="bullet">
        <listitem>A request is made for data in RDF form (explicitly via HTTP Accept Headers), and if
RDF is returned nothing further happens. </listitem>
        <listitem>If RDF isn't returned, the Sponger passes the data through a <emphasis>Entity Extraction Pipeline</emphasis>
(using Entity Extractors). </listitem>
        <listitem>The extracted data is transformed into RDF via a <emphasis>Mapping Pipeline</emphasis>. RDF instance data is generated by way of ontology matching and mapping. </listitem>
        <listitem>RDF instance data (aka. RDF Structured Linked Data) are returned to the client.</listitem>
      </itemizedlist>
      <para><emphasis>Extraction Pipeline</emphasis></para>
      <para>Depending on the file or format type detected at ingest, the Sponger applies the appropriate
entity extractor. Detection occurs at the time of content negotiation instigated by the retrieval user
agent. The normal extraction pipeline processing is as follows:</para>
      <itemizedlist mark="bullet">
        <listitem>The Sponger tries to get RDF data (including N3 or Turtle) directly from the dereferenced
URL. If it finds some, it returns it, otherwise, it continues.</listitem>
        <listitem>If the URL refers to a HTML file, the Sponger tries to find "link" elements referring to
RDF documents. If it finds one or more of them, it adds their triples into a temporary RDF graph and
continues its processing.</listitem>
        <listitem>The Sponger then scans for microformats or GRDDL. If either is found, RDF triples are
generated and added to a temporary RDF graph before continuing.</listitem>
        <listitem>If the Sponger finds eRDF or RDFa data in the HTML file, it extracts it from the HTML
file and inserts it into the RDF graph before continuing.</listitem>
        <listitem>If the Sponger finds it is talking with a web service such as Google Base, it maps
the API of the web service with an ontology, creates triples from that mapping and includes the triples
into the temporary RDF graph.</listitem>
        <listitem>The next fallback is scanning of the HTML header for different Web 2.0 types or RSS
1.1, RSS 2.0, Atom, etc. </listitem>
        <listitem>Failing those tests, the scan then uses standard Web 1.0 rules to search in the
header tags for metadata (typically Dublin Core) and transform them to RDF and again add them to the
temporary graph. Other HTTP response header data may also be transformed to RDF.</listitem>
        <listitem>If nothing has been retrieved at this point, the ODS-Briefcase metadata
extractor is tried.</listitem>
        <listitem>Finally, if nothing is found, the Sponger will return an empty graph.</listitem>
</itemizedlist>
      <para><emphasis>Ontology Mapper</emphasis></para>
      <para>Sponger ontology mappers peform the the task of generating RDF instance data from
extracted entities (non-RDF) using ontologies associated with a given data source type. They are
typically XSLT (using GRDDL or an in-built Virtuoso mapping scheme) or Virtuoso/PL based. Virtuoso
comes preconfigured with a large range of ontology mappers contained in one or more Sponger
cartridges.</para>
      <para><emphasis>Cartridge Registry</emphasis></para>
      <para>To be recognized by the SPARQL engine, a Sponger cartridge must be registered in
the Cartridge Registry by adding a record to the table DB.DBA.SYS_RDF_MAPPERS, either manually
via DML, or more easily through Conductor, Virtuoso's browser-based administration console,
which provides a UI for adding your own cartridges. (Sponger configuration using Conductor is
described in detail later.) The SYS_RDF_MAPPERS table definition is as follows:</para>
<programlisting><![CDATA[
create table "DB"."DBA"."SYS_RDF_MAPPERS"
(
"RM_ID" INTEGER IDENTITY,  -- cartridge ID. Determines the order of the cartridge's invocation in the Sponger processing chain
"RM_PATTERN" VARCHAR,  -- a REGEX pattern to match the resource URL or MIME type
"RM_TYPE" VARCHAR,  -- which property of the current resource to match: "MIME" or "URL"
"RM_HOOK" VARCHAR,  -- fully qualified Virtuoso/PL function name
"RM_KEY" LONG VARCHAR,  -- API specific key to use
"RM_DESCRIPTION" LONG VARCHAR,  -- cartridge description (free text)
"RM_ENABLED" INTEGER,  -- a 0 or 1 integer flag to exclude or include the cartridge from the Sponger processing chain
"RM_OPTIONS" ANY,  -- cartridge specific options
"RM_PID" INTEGER IDENTITY,
PRIMARY KEY ("RM_PATTERN", "RM_TYPE")
);
]]></programlisting>
</sect3>
    <sect3 id="virtuosospongeroverviewcartinvo">
      <title>Cartridge Invocation</title>
      <para>The Virtuoso SPARQL processor supports IRI dereferencing via the Sponger. If a SPARQL query
references non-default graph URIs, the Sponger goes out (via HTTP) to Fetch the Network Resource data source URIs and
inserts the extracted RDF data into the local RDF quad store. The Sponger invokes the appropriate
cartridge for the data source type to produce RDF instance data. If none of the registered cartridges
are capable of handling the received content type, the Sponger will attempt to obtain RDF instance
data via the in-built WebDAV metadata extractor.</para>
      <para>Sponger cartridges are invoked as follows:</para>
      <para>When the SPARQL processor dereferences a URI, it plays the role of an HTTP user agent
(client) that makes a content type specific request to an HTTP server via the HTTP request's Accept
headers. The following then occurs:</para>
      <itemizedlist mark="bullet">
        <listitem>If the content type returned is RDF then no further transformation is needed and
the process stops. For instance, when consuming an (X)HTML document with a GRDDL profile, the profile
URI points to a data provider that simply returns RDF instance data.</listitem>
        <listitem>If the content type is not RDF (i.e. application/rdf+xml or text/rdf+n3 ), for
instance 'text/plain', the Sponger looks in the Cartridge Registry iterating over every record for
which the RM_ENABLED flag is true, with the look-up sequence ordered on the RM_ID column values.
For each record, the processor tries matching the content type or URL against the RM_PATTERN value
and, if there is match, the function specified in RM_HOOK column is called. If the function doesn't
exist, or signals an error, the SPARQL processor looks at next record.
          <itemizedlist mark="bullet">
            <listitem>If the hook returns zero, the next cartridge is tried. (A cartridge function
can return zero if it believes a subsequent cartridge in the chain is capable of extracting more
RDF data.)</listitem>
            <listitem>If the result returned by the hook is negative, the Sponger is instructed
that no RDF was generated and the process stops. </listitem>
            <listitem>If the hook result is positive, the Sponger is informed that structured
data was retrieved and the process stops.</listitem>
          </itemizedlist>
        </listitem>
        <listitem>If none of the cartridges match the source data signature (content type or URL),
the ODS-Briefcase WebDAV metadata extractor and RDF generator is called.</listitem>
      </itemizedlist>
      <para><emphasis>Meta-Cartridges</emphasis></para>
      <para>The above describes the RDF generation process for 'primary' Sponger cartridges. Virtuoso
also supports another cartridge type - a 'meta-cartridge'. Meta-cartridges act as post-processors in
the cartridge pipeline, augmenting entity descriptions in an RDF graph with additional information
gleaned from 'lookup' data sources and web services. Meta-cartridges are described in more detail in
a later section.</para>
      <figure id="spong1" float="1">
        <title>Meta-Cartridges</title>
        <graphic fileref="ui/spong1.png"/>
      </figure>
</sect3>

    <sect3 id="virtuosospongercatrbundled">
      <title>Cartridges Bundled with Virtuoso</title>
      <sect4 id="virtuosospongercatrbundledrdfvad">
        <title>Cartridges VAD</title>
        <para>Virtuoso supplies a number of prewritten cartridges for extracting RDF data from a
variety of popular Web resources and file types. The cartridges are bundled as part of the cartridges_dav 
VAD (Virtuoso Application Distribution).</para>
        <para>To see which cartridges are available, look at the 'Linked Data' screen in 
        	Conductor. This can be reached through the Linked Data -&gt; Sponger -&gt; Extractor 
        	Cartridges and Meta Cartridges menu items.</para>
        <figure id="spong2" float="1">
          <title>RDF Cartridges</title>
          <graphic fileref="ui/spong2.png"/>
        </figure>
        <para>To check which version of the cartridges VAD is installed, or to upgrade it,
refer to Conductor's 'VAD Packages' screen, reachable through the 'System Admin' > 'Packages'
menu items.</para>
        <para>The latest VADs for the closed source releases of Virtuoso can be
<ulink url="http://download.openlinksw.com/download/">downloaded</ulink> from the downloads area on the OpenLink website. Select either the 'DBMS (WebDAV) Hosted' or
'File System Hosted' product format from the 'Distributed Collaborative Applications' section,
depending on whether you want the Virtuoso application to use WebDAV or native filesystem storage.
VADs for Virtuoso Open Source edition (VOS) are available for
<ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload">download</ulink> from
the VOS Wiki.</para>
</sect4>
      <sect4 id="virtuosospongercatrbundledexample">
        <title>Example Source Code</title>
        <para>For developers wanting example cartridge code, the most authoritative reference is
the cartridges VAD source code itself. This is included as part of the VOS distribution. After
downloading and unpacking the sources, the script used to create the cartridges, and the associated
stylesheets can be found in:</para>
        <itemizedlist mark="bullet">
          <listitem>&lt;vos root>/binsrc/rdf_mappers/rdf_mappers.sql</listitem>
          <listitem>&lt;vos root>/binsrc/rdf_mappers/xslt/*.xsl</listitem>
        </itemizedlist>
        <para>Alternatively, you can look at the actual cartridge implementations installed in your
Virtuoso instance by inspecting the cartridge hook function used by a particular cartridge. This is
easily identified from the 'Cartridge name' field of Conductor's 'RDF Cartridges' screen, after
selecting the cartridge of interest. The hook function code can be viewed from the 'Schema Objects'
screen under the 'Database' menu, by locating the function in the 'DB' > 'Procedures' folder.
Stylesheets used by the cartridges are installed in the WebDAV folder DAV/VAD/cartridges/xslt.
This can be explored using Conductor's WebDAV interface. The actual rdf_mappers.sql file installed
with your system can also be found in the DAV/VAD/cartridges folder.</para>
</sect4>
    </sect3>
    <sect3 id="virtuosospongercatrbundledcusgtomcart">
      <title>Custom Cartridge</title>
      <para>Virtuoso comes well supplied with a variety of Sponger cartridges and GRDDL filters.
When then is it necessary to write your own cartridge?</para>
      <para>In the main, writing a new cartridge should only be necessary to generate RDF from
a REST-style Web service not supported by an existing cartridge, or to customize the output from
an existing cartridge to your own requirements. Apart from these circumstances, the existing
Sponger infrastructure should meet most of your needs. This is particularly the case for
document resources.</para>
     <sect4 id="virtuosospongerdocres">
       <title>Document Resources</title>
       <para>We use the term document resource to identify content which is not being returned
from a Web service. Normally it can broadly be conceived as some form of document, be it a text
based entity or some form of file, for instance an image file.</para>
       <para>In these cases, the document either contains RDF, which can be extracted directly,
or it holds metadata in a supported format which can be transformed to RDF using an existing
filter.</para>
       <para>The following cases should all be covered by the existing Sponger cartridges:</para>
         <itemizedlist mark="bullet">
           <listitem>embedded or linked RDF</listitem>
           <listitem>RDFa, eRDF and other popular microformats extractable directly or via GRDDL</listitem>
           <listitem>popular syndication formats (RSS 2.0 , Atom, OPML , OCS , XBEL)</listitem>
         </itemizedlist>
</sect4>
     <sect4 id="virtuosospongergrddl">
       <title>GRDDL</title>
       <para>GRDDL (Gleaning Resource Descriptions from Dialects of Languages) is mechanism for deriving
RDF data from XML documents and in particular XHTML pages. Document authors may associate transformation
algorithms, typically expressed in XSLT, with their documents to transform embedded metadata into RDF.</para>
       <para>The cartridges VAD installs a number of GRDDL filters for transforming popular microformats
(such as RDFa, eRDF or hCalendar) into RDF. The available filters can be viewed, or configured, in
Conductor's 'GRDDL Filters for XHTML' screen. Navigate to the 'RDF Cartridges' screen using the
'RDF' > 'RDF Cartridges' menu items, then SELECT the 'GRDDL Mappings' tab to display the 'GRDDL Filters
for XHTML' screen. GRDDL filters are held in the WebDAV folder /DAV/VAD/rdf_cartridges/xslt/ alongside
other XSLT templates. The Conductor interface allows you to add new GRDDL filters should you so wish.</para>
       <para>For an introduction to GRDDL, try the <ulink url="http://www.w3.org/TR/grddl-primer/">GRDDL Primer</ulink>. To underline GRDDL's utility, the primer
includes an example of transforming Excel spreadsheet data, saved as XML, into RDF.</para>
       <para>A comprehensive <ulink url="http://esw.w3.org/topic/CustomRdfDialects">list of stylesheets</ulink>
for transforming HTML and non-HTML XML dialects is maintained
on the ESW Wiki. The list covers a range of microformats, syndication formats and feedlists.</para>
     </sect4>
     <para>To see which Web Services are already catered for, view the list of cartridges in Conductor's 'RDF Cartridges' screen.</para>
    </sect3>
    <sect3 id="virtuosospongercreatecustcartr">
      <title>Creating Custom Cartridges</title>
      <para>The Sponger is fully extensible by virtue of its pluggable cartridge architecture. New data formats can
be fetched by creating new cartridges. While OpenLink is active in adding cartridges for new data sources,
you are free to develop your own custom cartridges. Entity extractors can be built using Virtuoso PL,
C/C++, Java or any other external language supported by Virtuoso's Server Extension API. Of course,
Virtuoso's own entity extractors are written in Virtuoso PL.</para>
      <sect4 id="virtuosospongercreatecustcartran">
        <title>The Anatomy of a Cartridge</title>
        <para><emphasis>Cartridge Hook Prototype</emphasis></para>
        <para>Every Virtuoso PL hook function used to plug a custom Sponger cartridge into the Virtuoso
SPARQL engine must have a parameter list with the following parameters (the names of the parameters are
not important, but their order and presence are):</para>
        <itemizedlist mark="bullet">
          <listitem><emphasis>in graph_iri varchar</emphasis>: the IRI of the graph being retrieved/crawled</listitem>
          <listitem><emphasis>in new_origin_uri varchar</emphasis>: the URL of the document being retrieved</listitem>
          <listitem><emphasis>in dest varchar</emphasis>: the destination/target graph IRI</listitem>
          <listitem><emphasis>inout content any</emphasis>: the content of the retrieved document</listitem>
          <listitem><emphasis>inout async_queue any</emphasis>: if the PingService initialization parameter
has been configured in the [SPARQL] section of the virtuoso.ini file, this is a pre-allocated asynchronous
queue to be used to call the ping service</listitem>
          <listitem><emphasis>inout ping_service any</emphasis>: the URL of a ping service, as assigned
to the PingService parameter in the [SPARQL] section of the virtuoso.ini configuration file.
PingTheSemanticWeb is an <link linkend="virtuosospongerefping">example</link> of a such a service.</listitem>
          <listitem><emphasis>inout api_key any</emphasis>: a string value specific to a given cartridge,
contained in the RC_KEY column of the DB.DBA.SYS_RDF_CARTRIDGES table. The value can be a single
string or a serialized array of strings providing cartridge specific data.</listitem>
          <listitem><emphasis>inout opts any</emphasis>: cartridge specific options held in a Virtuoso/PL
vector which acts as an array of key-value pairs.</listitem>
        </itemizedlist>
        <para><emphasis>Return Value</emphasis></para>
        <para>If the hook procedure returns zero the next cartridge will be tried. If the result is negative
the sponging process stops, instructing the SPARQL engine that nothing was retrieved. If the result is
positive the process stops, this time instructing the SPARQL engine
that RDF data was successfully retrieved.</para>
        <para>If your cartridge should need to test whether other cartridges are configured to handle a
particular data source, the following extract taken from the RDF_LOAD_CALAIS hook procedure illustrates
how you might do this:</para>
<programlisting><![CDATA[
if (xd is not null)
       {
  -- Sponging successful. Load network resource data being fetched in the Virtuoso Quad Store:
  DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
  flag := 1;
  }

declare ord any;
ord := (SELECT RM_ID FROM DB.DBA.SYS_RDF_MAPPERS WHERE
	  RM_HOOK = 'DB.DBA.RDF_LOAD_CALAIS');
for SELECT RM_PATTERN FROM DB.DBA.SYS_RDF_MAPPERS WHERE
  RM_ID > ord and RM_TYPE = 'URL' and RM_ENABLED = 1 ORDER BY RM_ID do
  {
  if (regexp_match (RM_PATTERN, new_origin_uri) is not null)
    -- try next candidate cartridge
    flag := 0;
  }
return flag;
]]></programlisting>
        <para><emphasis>Specifying the Target Graph</emphasis></para>
        <para>Two cartridge hook function parameters contain graph IRIs, graph_iri and dest. graph_iri
identifies an input graph being crawled. dest holds the IRI specified in any input:grab-destination
pragma defined to control the SPARQL processor's IRI dereferencing. The pragma overrides the default
behaviour and forces all retrieved triples to be stored in a single graph, irrespective of their graph
of origin.</para>
        <para>So, under some circumstances depending on how the Sponger has been invoked and whether
it is being used to crawl an existing RDF graph, or derive RDF data from a non-RDF data source,
dest may be null.</para>
        <para>Consequently, when loading network resource being fetched as RDF data into the quad store, cartridges typically
specify the graph to receive the data using the coalesce function which returns the first non-null
parameter. e.g.</para>
<programlisting><![CDATA[
DB.DBA.RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
]]></programlisting>
        <para>Here xd is an RDF/XML string holding the fetched RDF.</para>
        <para><emphasis>Specifying &amp; Retrieving Cartridge Specific Options</emphasis></para>
        <para>The hook function prototype allows cartridge specific data to be passed to a cartridge
through the RM_OPTIONS parameter, a Virtuoso/PL vector which acts as a heterogeneous array.</para>
        <para>In the following example, two options are passed, 'add-html-meta' and 'get-feeds'
with both values set to 'no'.</para>
<programlisting><![CDATA[
insert soft DB.DBA.SYS_RDF_MAPPERS (
  RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION, RM_OPTIONS
)
values (
  '(text/html)|(text/xml)|(application/xml)|(application/rdf.xml)',
  'MIME', 'DB.DBA.RDF_LOAD_HTML_RESPONSE', null, 'xHTML',
  vector ('add-html-meta', 'no', 'get-feeds', 'no')
);
]]></programlisting>
        <para>The RM_OPTIONS vector can be handled as an array of key-value pairs using the
get_keyword function. get_keyword performs a case sensitive search for the given keyword at
every even index of the given array. It returns the element following the keyword, i.e.
the keyword value.</para>
        <para>Using get_keyword, any options passed to the cartridge can be retrieved
using an approach similar to that below:</para>
<programlisting><![CDATA[
create procedure DB.DBA.RDF_LOAD_HTML_RESPONSE (
  in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
  inout ret_body any, inout aq any, inout ps any, inout _key any,
  inout opts any )
  {
  declare get_feeds, add_html_meta;
  ...
  get_feeds := add_html_meta := 0;
  if (isarray (opts) and 0 = mod (length(opts), 2))
  {
    if (get_keyword ('get-feeds', opts) = 'yes')
      get_feeds := 1;
    if (get_keyword ('add-html-meta', opts) = 'yes')
      add_html_meta := 1;
  }
  ...
]]></programlisting>
        <para><emphasis>XSLT - The Fulchrum</emphasis></para>
        <para>XSLT is the fulchrum of all OpenLink supplied cartridges. It provides the most convenient
means of converting structured data extracted from web content by a cartridge's Entity Extractor into RDF.</para>
        <para><emphasis>Virtuoso's XML Infrastructure &amp; Tools</emphasis></para>
        <para>Virtuoso's XML support and XSLT support are covered in detail in the on-line documentation.
Virtuoso includes a highly capable XML parser and supports XPath, XQuery, XSLT and XML Schema validation.</para>
        <para>Virtuoso supports extraction of XML documents from SQL datasets. A SQL long varchar, long xml
or xmltype column in a database table can contain XML data as text or in a binary serialized format.
 A string representing a well-formed XML entity can be converted into an entity object representing
the root node.</para>
        <para>While Sponger cartridges will not normally concern themselves with handling XML extracted
from SQL data, the ability to convert a string into an in-memory XML document is used extensively.
The function xtree_doc(string) converts a string into such a document and returns a reference to the
document's root. This document together with an appropriate stylesheet forms the input for
the transformation of the extracted entities to RDF using XSLT. The input string to xtree_doc
generally contains structured content derived from a web service.</para>
        <para><emphasis>Virtuoso XSLT Support</emphasis></para>
        <para>Virtuoso implements XSLT 1.0 transformations as SQL callable functions. The xslt() Virtuoso/PL function
applies a given stylesheet to a given source XML document and returns the transformed document. Virtuoso
provides a way to extend the abilities of the XSLT processor by creating user defined XPath functions.
The functions xpf_extension() and xpf_extension_remove() allow addition and removal of XPath extension
functions.</para>
        <para><emphasis>General Cartridge Pipeline</emphasis></para>
        <para>The broad pipeline outlined here reflects the steps common to most cartridges:</para>
          <itemizedlist mark="bullet">
            <listitem>Redirect from the requested URL to a Web service which returns XML</listitem>
            <listitem>Stream the content into an in-memory XML document</listitem>
            <listitem>Convert it to the required RDF/XML, expressed in the chosen ontology, using XSLT</listitem>
            <listitem>Encode the RDF/XML as UTF-8</listitem>
            <listitem>Load the RDF/XML into the quad store</listitem>
          </itemizedlist>
        <para>The <ulink url="http://musicbrainz.org/">MusicBrainz</ulink> cartridge typifies this approach. MusicBrainz is a community music
metadatabase which captures information about artists, their recorded works, and the relationships
between them. Artists always have a unique ID, so the URL
http://musicbrainz.org/artist/4d5447d7-c61c-4120-ba1b-d7f471d385b9.html takes you directly to entries
for John Lennon.</para>
        <para>If you were to look at this page in your browser, you would see that the information about
the artist contains no RDF data. However, the cartridge is configured to intercept requests to URLs of
the form http://musicbrainz.org/([^/]*)/([^.]*) and redirect to the cartridge to Fetch all the
available information on the given artist, release, track or label.</para>
        <para>The cartridge extracts entities by redirecting to the MusicBrainz XML Web Service using
as the basis for the initial query the item ID, e.g. an artist or label ID, extracted from the original
URL. Stripped to its essentials, the core of the cartridge is:</para>
<programlisting><![CDATA[
webservice_uri := sprintf ('http://musicbrainz.org/ws/1/%s/%s?type=xml&inc=%U',
					kind, id, inc);
content := RDF_HTTP_URL_GET (webservice_uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (content);
...
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_cartridges_path_') || 'xslt/mbz2rdf.xsl', xt);
...
xd := serialize_to_UTF8_xml (xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
]]></programlisting>
        <para>In the above outline, RDF_HTTP_URL_GET sends a query to the MusicBrainz web service,
using query parameters appropriate for the original request, and retrieves the response using
Network Resource Fetch.</para>
        <para>The returned XML is parsed into an in-memory parse tree by xtree_doc. Virtuoso/PL
function RDF_MAPPER_XSLT is a simple wrapper around the function xslt which sets the current user
to dba before returning an XML document transformed by an XSLT stylesheet, in this case mbz2rdf.xsl.
Function serialize_to_UTF8_xml changes the character set of the in-memory XML document to UTF8.
Finally, RM_RDF_LOAD_RDFXML is a wrapper around RDF_LOAD_RDFXML which parses the content of an
RDF/XML string into a sequence of RDF triples and loads them into the quad store. XSLT stylesheets
are usually held in the DAV/VAD/cartridges/xslt folder of Virtuoso's WebDAV store.
registry_get('cartridges_path') returns the Cartridges VAD path, 'DAV/VAD/cartridges', from the
Virtuoso registry.</para>
        <para><emphasis>Error Handling with Exit Handlers</emphasis></para>
        <para>Virtuoso condition handlers determine the behaviour of a Virtuoso/PL procedure when a
condition occurs. You can declare one or more condition handlers in a Virtuoso/PL procedure for
general SQL conditions or specific SQLSTATE values. If a statement in your procedure raises an
SQLEXCEPTION condition and you declared a handler for the specific SQLSTATE or SQLEXCEPTION
condition the server passes control to that handler. If a statement in your Virtuoso/PL procedure
raises an SQLEXCEPTION condition, and you have not declared a handler for the specific SQLSTATE
or the SQLEXCEPTION condition, the server passes the exception to the calling procedure (if any).
If the procedure call is at the top-level, then the exception is signaled to the calling client.</para>
        <para>A number of different condition handler types can be declared (see the <ulink url="http://docs.openlinksw.com/virtuoso/handlingplcondit.html">Virtuoso reference documentation</ulink>
for more details.) Of these, exit handlers are probably all you will need. An example is shown below
which handles any SQLSTATE. Commented out is a debug statement which outputs the message describing
the SQLSTATE.</para>
<programlisting><![CDATA[
create procedure DB.DBA.RDF_LOAD_SOCIALGRAPH (in graph_iri varchar, ...)
  {
  declare qr, path, hdr any;
  ...
  declare exit handler for sqlstate '*'
{
    -- dbg_printf ('%s', __SQL_MESSAGE);
    return 0;
};
  ...
  -- data extraction and mapping successful
  return 1;
}
]]></programlisting>
        <para>Exit handlers are used extensively in the Virtuoso supplied cartridges. They are useful
for ensuring graceful failure when trying to convert content which may not conform to your expectations.
The RDF_LOAD_FEED_SIOC procedure (which is used internally by several cartridges) shown below uses this
approach:</para>
<programlisting><![CDATA[
-- /* convert the feed in rss 1.0 format to sioc */
create procedure DB.DBA.RDF_LOAD_FEED_SIOC (in content any, in iri varchar, in graph_iri varchar, in is_disc int := '')
    {
  declare xt, xd any;
  declare exit handler for sqlstate '*'
    {
      goto no_sioc;
};
  xt := xtree_doc (content);
  xd := DB.DBA.RDF_MAPPER_XSLT (
      registry_get ('_cartridges_path_') || 'xslt/feed2sioc.xsl', xt,
      vector ('base', graph_iri, 'isDiscussion', is_disc));
  xd := serialize_to_UTF8_xml (xd);
  DB.DBA.RM_RDF_LOAD_RDFXML (xd, iri, graph_iri);
  return 1;
no_sioc:
  return 0;
}
]]></programlisting>
        <para><emphasis>Loading RDF into the Quad Store</emphasis></para>
        <para><emphasis>RDF_LOAD_RDFXML &amp; TTLP</emphasis></para>
        <para>The two main Virtuoso/PL functions used by the cartridges for loading RDF data into the
Virtuoso quad store are DB.DBA.TTLP and DB.DBA.RDF_LOAD_RDFXML. Multithreaded versions of these functions,
DB.DBA.TTLP_MT and DB.DBA.RDF_LOAD_RDFXML_MT, are also available.</para>
        <para>RDF_LOAD_RDFXML parses the content of an RDF/XML string as a sequence of RDF triples and
loads then into the quad store. TTLP parses TTL (Turtle or N3) and places its triples into quad
storage. Ordinarily, cartridges use RDF_LOAD_RDFXML. However there may be occasions where you want
to insert statements written as TTL, rather than RDF/XML, in which case you should use TTLP.</para>
  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="rdfinsertmethodsapifunct">Loading RDF using API functions</link></listitem>
    </itemizedlist>
  </tip>
        <para><emphasis>Attribution</emphasis></para>
        <para>Many of the OpenLink supplied cartridges actually use RM_RDF_LOAD_RDFXML to load data
into the quad store. This is a thin wrapper around RDF_LOAD_RDFXML which includes in the generated
graph an indication of the external ontologies being used. The attribution takes the form:</para>
<programlisting><![CDATA[
<ontologyURI> a opl:DataSource .
<spongedResourceURI> rdfs:isDefinedBy <ontologyURI> .
<ontologyURI> opl:hasNamespacePrefix "<ontologyPrefix>" .
]]></programlisting>
        <para>where prefix opl: denotes the ontology http://www.openlinksw.com/schema/attribution#.</para>
        <para><emphasis>Deleting Existing Graphs</emphasis></para>
        <para>Before loading network resource fetched RDF data into a graph, you may want to delete any existing graph
with the same URI. To do so, select the 'RDF' > 'List of Graphs' menu commands in Conductor, then use
the 'Delete' command for the appropriate graph. Alternatively, you can use one of the following SQL
commands:</para>
<programlisting><![CDATA[
SPARQL CLEAR GRAPH
-- or
DELETE FROM DB.DBA.RDF_QUAD WHERE G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_iri)
]]></programlisting>
        <para><emphasis>Proxy Service Data Expiration</emphasis></para>
        <para>When the Proxy Service is invoked by a user agent, the Sponger records the expiry date
of the imported data in the table DB.DBA.SYS_HTTP_SPONGE. The data invalidation rules conform to those
of traditional HTTP clients (Web browsers). The data expiration time is determined based on subsequent
data fetches of the same resource. The first data retrieval records the 'expires' header. On subsequent
fetches, the current time is compared to the expiration time stored in the local cache. If HTTP 'expires'
header data isn't returned by the source data server, the Sponger will derive its own expiration time by
evaluating the 'date' header and 'last-modified' HTTP headers.</para>
      </sect4>
      <sect4 id="virtuosospongercreatecustcartrontolg">
        <title>Ontology Mapping</title>
        <para>After extracting entities from a web resource and converting them to an in-memory XML
document, the entities must be transformed to the target ontology using XSLT and an appropriate stylesheet.
A typical call sequence would be:</para>
<programlisting><![CDATA[
xt := xtree_doc (content);
...
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_cartridges_path_') || 'xslt/mbz2rdf.xsl', xt);
]]></programlisting>
        <para>Because of the wide variation in the data mapped by cartridges, it is not possible to present
a typical XSL stylesheet outline. The Examples section presented later includes detailed extracts from the
MusicBrainz? cartridge's stylesheet which provide a good example of how to map to an ontology. Rather than
attempting to be an XSLT tutorial, the material which follows offers some general guidelines.</para>
        <para><emphasis>Passing Parameters to the XSLT Processor</emphasis></para>
        <para>Virtuoso's XSLT processor will accept default values for global parameters from the optional
third argument of the xslt() function. This argument, if specified, must be a vector of parameter names
and values of the form vector(name1, value1,... nameN, valueN), where name1 ... nameN must be of type
varchar, and value1 ... valueN may be of any Virtuoso datatype, but may not be null.</para>
        <para>This extract from the Crunchbase cartridge shows how parameters may be passed to the XSLT
processor. The function RDF_MAPPER_XSLT (in xslt varchar, inout xt any, in params any := null) passes
the parameters vector directly to xslt().</para>
<programlisting><![CDATA[
xt := DB.DBA.RDF_MAPPER_XSLT (
registry_get ('_cartridges_path_') || 'xslt/crunchbase2rdf.xsl', xt,
vector ('baseUri', coalesce (dest, graph_iri), 'base', base, 'suffix', suffix)
);
]]></programlisting>
        <para>The corresponding stylesheet crunchbase2rdf.xsl retrieves the parameters baseUri, base and suffix as follows:</para>
<programlisting><![CDATA[
...
<xsl:output method="xml" indent="yes" />
  <xsl:variable name="ns">http://www.crunchbase.com/</xsl:variable>
  <xsl:param name="baseUri" />
  <xsl:param name="base"/>
  <xsl:param name="suffix"/>
  <xsl:template name="space-name">
...
]]></programlisting>
        <para><emphasis>An RDF Description Template</emphasis></para>
        <para><emphasis>Defining A Generic Resource Description Wrapper</emphasis></para>
        <para>Many of the OpenLink cartridges create a resource description formed to a common "wrapper"
template which describes the relationship between the (usually) non-RDF source network resource being fetched 
and the RDF description generated by the Sponger. The wrapper is appropriate for resources which can
broadly be conceived as documents. It provides a generic minimal description of the source document,
but also links to the much more detailed description provided by the Sponger. So, instead of just
emitting a resource description, the Sponger factors the container into the generated graph constituting
the RDF description.</para>
        <para>The template is depicted below:</para>
        <figure id="spong3" float="1">
          <title>Template</title>
          <graphic fileref="ui/spong3.png"/>
        </figure>
        <para>To generate an RDF description corresponding to the wrapper template, a stylesheet containing
the following block of instructions is used. This extract is taken from the eBay cartridge's stylesheet,
ebay2rdf.xsl. Many of the OpenLink cartridges follow a similar pattern.</para>
<programlisting><![CDATA[
    <xsl:param name="baseUri"/>
...
    <xsl:variable name="resourceURL">
	<xsl:value-of select="$baseUri"/>
    </xsl:variable>
...
    <xsl:template match="/">
	<rdf:RDF>
	    <rdf:Description rdf:about="{$resourceURL}">
		<rdf:type rdf:resource="Document"/>
		<rdf:type rdf:resource="Document"/>
		<rdf:type rdf:resource="Container"/>
		<sioc:container_of rdf:resource="{vi:proxyIRI ($resourceURL)}"/>
		<foaf:primaryTopic rdf:resource="{vi:proxyIRI ($resourceURL)}"/>
		<dcterms:subject rdf:resource="{vi:proxyIRI ($resourceURL)}"/>
	    </rdf:Description>
	    <rdf:Description rdf:about="{vi:proxyIRI ($resourceURL)}">
		<rdf:type rdf:resource="Item"/>
		<sioc:has_container rdf:resource="{$resourceURL}"/>
		<xsl:apply-templates/>
	    </rdf:Description>
	</rdf:RDF>
    </xsl:template>
...
]]></programlisting>
        <para><emphasis>Using SIOC as a Generic Container Model</emphasis></para>
        <para>The generic resource description wrapper just described uses SIOC to establish the
container/contained relationship between the source resource and the generated graph. Although the most
important classes for the generic wrapper are obviously Container and Item, SIOC provides a generic data
model of containers, items, item types, and associations between items which can be combined with other
 vocabularies such as FOAF and Dublin Core.</para>
        <para>SIOC defines a number of other classes, such as User, UserGroup, Role, Site, Forum and
Post. A separate SIOC types module (T-SIOC) extends the SIOC Core ontology by defining subclasses and
subproperties of SIOC terms. Subclasses include: AddressBook, BookmarkFolder, Briefcase, EventCalendar,
ImageGallery, Wiki, Weblog, BlogPost, Wiki plus many others.</para>
        <para>OpenLink Data Spaces (ODS) uses SIOC extensively as a data space "glue" ontology to
describe the base data and containment hierarchy of all the items managed by ODS applications
(Data Spaces). For example, ODS-Weblog is an application of type sioc:Forum. Each ODS-Weblog
application instance contains blogs of type sioct:Weblog. Each blog is a sioc:container_of posts
of type sioc:Post.</para>
        <para>Generally, when deciding how to describe resources handled by your own custom cartridge,
SIOC provides a useful framework for the description which complements the SIOC-based container model
adopted throughout the ODS framework.</para>
        <para><emphasis>Naming Conventions for Sponger Generated Descriptions</emphasis></para>
        <para>As can be seen from the stylesheet extract just shown, the URI of the resource
description generated by the Sponger to describe the network resource being fetched, is given by the
function {vi:proxyIRI ($resourceURL)} where resourceURL is the URL of the original network resource
being fetched. proxyIRI is an XPath extension function defined in rdf_mappers.sql as</para>
<programlisting><![CDATA[
xpf_extension ('http://www.openlinksw.com/virtuoso/xslt/:proxyIRI', 'DB.DBA.RDF_SPONGE_PROXY_IRI');
]]></programlisting>
        <para>which maps to the Virtuoso/PL procedure DB.DBA.RDF_SPONGE_PROXY_IRI. This procedure in
turn generates a resource description URI which typically takes the form:
http://&lt;hostName:port&gt;/about/html/http/&lt;resourceURL&gt;#this</para>
    </sect4>
      <sect4 id="virtuosospongercreatecustcartrrgst">
        <title>Registering &amp; Configuring Cartridges</title>
        <para>Once you have developed a cartridge, you must register it in the Cartridge Registry
to have the SPARQL processor recognize and use it. You should have compiled your cartridge hook
function first by issuing a "create procedure DB.DBA.RDF_LOAD_xxx ..." command through one of
Virtuoso's SQL interfaces. You can create the required Cartridge Registry entry either by adding
a row to the SYS_REF_MAPPERS table directly using SQL, or by using the Conductor UI.</para>
        <sect5 id="virtuosospongercreatecustcartrrgstis">
        <title>Using SQLs</title>
        <para>If you choose register your cartridge using SQL, possibly as part of a Virtuoso/PL
script, the required SQL will typically mirror one of the following INSERT commands.</para>
        <para>Below, a cartridge for OpenCalais is being installed which will be tried when
the MIME type of the network resource data being fetched is one of text/plain, text/xml or text/html.
(The definition of the SYS_RDF_MAPPERS table was introduced earlier in section
'Cartridge Registry'.)</para>
<programlisting><![CDATA[
insert soft DB.DBA.SYS_RDF_MAPPERS (
  RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION, RM_ENABLED)
values (
  '(text/plain)|(text/xml)|(text/html)', 'MIME', 'DB.DBA.RDF_LOAD_CALAIS',
  null, 'Opencalais', 1);
]]></programlisting>
        <para>As an alternative to matching on the content's MIME type,
candidate cartridges to be tried in the conversion pipeline can be identified by matching the
data source URL against a URL pattern stored in the cartridge's entry in the Cartridge Registry.</para>
<programlisting><![CDATA[
insert soft DB.DBA.SYS_RDF_MAPPERS (
  RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION, RM_OPTIONS)
values (
  '(http://api.crunchbase.com/v/1/.*)|(http://www.crunchbase.com/.*)', 'URL',
  'DB.DBA.RDF_LOAD_CRUNCHBASE', null, 'CrunchBase', null);
]]></programlisting>
        <para>The value of RM_ID to set depends on where in the cartridge invocation order you want
to position a particular cartridge. RM_ID should be set lower than 10028 to ensure the cartridge
is tried before the ODS-Briefcase (WebDAV) metadata extractor, which is always the last mapper to
be tried if no preceding cartridge has been successful.</para>
<programlisting><![CDATA[
UPDATE DB.DBA.SYS_RDF_MAPPERS
SET RM_ID = 1000
WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_BIN_DOCUMENT';
]]></programlisting>
        </sect5>
        <sect5 id="virtuosospongercreatecustcartrrgstcn">
        <title>Using Conductor</title>
        <para>Cartridges can be added manually using the 'Add' panel of the 'RDF Cartridges' screen.</para>
        <figure id="spong4" float="1">
          <title>RDF Cartridges</title>
          <graphic fileref="ui/spong4.png"/>
        </figure>
        </sect5>
        <sect5 id="virtuosospongercreatecustcartrrgstit">
        <title>Installing Stylesheets</title>
        <para>Although you could place your cartridge stylesheet in any folder configured to be accessible
by Virtuoso, the simplest option is to upload them to the DAV/VAD/cartridges/xslt folder using the
WebDAV browser accessible from the Conductor UI.</para>
        <figure id="spong6" float="1">
          <title>WebDAV browser</title>
          <graphic fileref="ui/spong6.png"/>
        </figure>
        <para>Should you wish to locate your stylesheets elsewhere, ensure
that the DirsAllowed setting in the virtuoso.ini file is configured appropriately.</para>
        </sect5>
        <sect5 id="virtuosospongercreatecustcartrrgstap">
        <title>Setting API Key</title>        
         <para>Some Cartridges require and API account and/or API Key to be 
provided for accessing the required service. This can be done from the Linked Data -&gt; Sponger tab of the 
Conductor by selecting the cartridge from the list provided, entering the API Account and API Key 
in the dialog at the bottom of the page and click update to save, as indicated in the screenshot 
below:</para>
     <figure id="catr1" float="1">
       <title>Registering API Key</title>
       <graphic fileref="ui/cartrapikey.png"/>
     </figure>
     <para>For example, for the service Flickr developers must register
to obtain a key. See http://developer.yahoo.com/flickr/. In order
to cater for services which require an application key, the Cartridge Registry
SYS_RDF_MAPPERS table includes an RM_KEY column to store any key required for a
particular service. This value is passed to the service's cartridge through the
_key parameter of the cartridge hook function.</para>
         <para>Alternatively a cartridge can store a key value in the virtuoso.ini
configuration file and retrieve it in the hook function.</para>       
         </sect5>
         <sect5 id="virtuosospongercreatecustcartrrgstflickr">
           <title>Flickr Cartridge</title> 
           <para>This example shows an extract from the Flickr cartridge hook function
DB.DBA.RDF_LOAD_FLICKR_IMG and the use of an API key. Also, commented out, is a call to
cfg_item_value() which illustrates how the API key could instead be stored and retrieved
from the SPARQL section of the virtuoso.ini file.
         </para>
<programlisting><![CDATA[
create procedure DB.DBA.RDF_LOAD_FLICKR_IMG (
in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
inout _ret_body any, inout aq any, inout ps any, inout _key any,
inout opts any )
{
declare xd, xt, url, tmp, api_key, img_id, hdr, exif any;
declare exit handler for sqlstate '*'
{
 return 0;
};
tmp := sprintf_inverse (new_origin_uri,
  'http://farm%s.static.flickr.com/%s/%s_%s.%s', 0);
img_id := tmp[2];
api_key := _key;
--cfg_item_value (virtuoso_ini_path (), 'SPARQL', 'FlickrAPIkey');
if (tmp is null or length (tmp) <> 5 or not isstring (api_key))
  return 0;
url :=  sprintf('http://api.flickr.com/services/rest/?method=flickr.photos.getInfo&photo_id=%s&api_key=%s',img_id, api_key);
tmp := http_get (url, hdr);
]]></programlisting>
        </sect5>
    </sect4>
      <sect4 id="virtuosospongercreatecustcartrexmp">
        <title>MusicBrainz Example: A Music Metadatabase</title>
        <para>To illustrate some of the material presented so far, we'll delve
deeper into the <ulink url="http://musicbrainz.org/">MusicBrainz</ulink> cartridge mentioned earlier.</para>
        <para><emphasis>MusicBrainz XML Web Service</emphasis></para>
        <para>The cartridge extracts data through the <ulink url="http://musicbrainz.org/doc/XMLWebService">MusicBrainz XML Web Service</ulink> using, as the basis
for the initial query, an item type and MBID (MusicBrainz ID) extracted from the original URI submitted
to the RDF proxy. A range of item types are supported including artist, release and track.</para>
        <para>Using the album "Imagine" by John Lennon as an example, a standard HTML description of
the album (which has an MBID of f237e6a0-4b0e-4722-8172-66f4930198bc) can be retrieved direct from
MusicBrainz using the URL:</para>
<programlisting><![CDATA[
http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html
]]></programlisting>
        <para>Alternatively, information can be extracted in XML form through the web service.
A description of the tracks on the album can be obtained with the query:</para>
<programlisting><![CDATA[
http://musicbrainz.org/ws/1/release/f237e6a0-4b0e-4722-8172-66f4930198bc?type=xml&inc=tracks
]]></programlisting>
        <para>The XML returned by the web service is shown below (only the first two tracks
are shown for brevity):</para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="UTF-8"?>
  <metadata xmlns="http://musicbrainz.org/ns/mmd-1.0#"
   xmlns:ext="http://musicbrainz.org/ns/ext-1.0#">
    <release id="f237e6a0-4b0e-4722-8172-66f4930198bc" type="Album Official" >
      <title>Imagine</title>
        <text-representation language="ENG" script="Latn"/>
        <asin>B0000457L2</asin>
        <track-list>
          <track id="b88bdafd-e675-4c6a-9681-5ea85ab99446">
            <title>Imagine</title>
            <duration>182933</duration>
          </track>
          <track id="b38ce90d-3c47-4ccd-bea2-4718c4d34b0d">
            <title>Crippled Inside</title>
            <duration>227906</duration>
          </track>
	  . . .
        </track-list>
      </release>
  </metadata>
]]></programlisting>
        <para>Although, as shown above, MusicBrainz defines its own <ulink url="http://musicbrainz.org/doc/MusicBrainzXMLMetaData">XML Metadata Format</ulink> to represent
music metadata, the MusicBrainz sponger converts the raw data to a subset of the <ulink url="http://musicontology.com/">Music Ontology</ulink>,
an RDF vocabulary which aims to provide a set of core classes and properties for describing music
on the Semantic Web. Part of the subset used is depicted in the following RDF graph (representing
in this case a John Cale album).</para>
        <figure id="spong7" float="1">
          <title>RDF graph</title>
          <graphic fileref="ui/spong7.png"/>
        </figure>
        <para>With the prefix mo: denoting the Music Ontology at http://purl.org/ontology/mo/, it can be
seen that artists are represented by instances of class mo:Artist, their albums, records etc. by instances
of class mo:Release and tracks on these releases by class mo:Track. The property foaf:made links an
artist and his/her releases. Property mo:track links a release with the tracks it contains</para>
        <para><emphasis>RDF Output</emphasis></para>
        <para>An RDF description of the album can be obtained by sponging the same URL, i.e. by submitting it to the Sponger's proxy interface using the URL:</para>
<programlisting><![CDATA[
http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html
]]></programlisting>
        <para>The extract below shows part of the (reorganized) RDF output returned by the Sponger
for "Imagine". Only the album's title track is included.</para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
 xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">

<rdf:Description
 rdf:about="http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html">
  <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Document"/>
</rdf:Description>

<rdf:Description
 rdf:about="http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html">
  <foaf:primaryTopic xmlns:foaf="http://xmlns.com/foaf/0.1/"
   rdf:resource="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this"/>
</rdf:Description>

<rdf:Description rdf:about="http://purl.org/ontology/mo/">
  <rdf:type rdf:resource="http://www.openlinksw.com/schema/attribution#DataSource"/>
</rdf:Description>
...
<rdf:Description
 rdf:about="http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html">
  <rdfs:isDefinedBy rdf:resource="http://purl.org/ontology/mo/"/>

</rdf:Description>
...
<!-- Record description -->
<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this">
  <rdf:type rdf:resource="http://purl.org/ontology/mo/Record"/>
</rdf:Description>

<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this">
  <dc:title xmlns:dc="http://purl.org/dc/elements/1.1/">Imagine</dc:title>
</rdf:Description>

<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this">
  <mo:release_status xmlns:mo="http://purl.org/ontology/mo/" rdf:resource="http://purl.org/ontology/mo/official"/>
</rdf:Description>

<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this">
  <mo:release_type xmlns:mo="http://purl.org/ontology/mo/"
   rdf:resource="http://purl.org/ontology/mo/album"/>
</rdf:Description>
<!-- Title track description -->
<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/release/f237e6a0-4b0e-4722-8172-66f4930198bc.html#this">
  <mo:track xmlns:mo="http://purl.org/ontology/mo/"
   rdf:resource="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this"/>
</rdf:Description>
<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this">
  <rdf:type rdf:resource="http://purl.org/ontology/mo/Track"/>
</rdf:Description>

<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this">
  <dc:title xmlns:dc="http://purl.org/dc/elements/1.1/">Imagine</dc:title>
</rdf:Description>

<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this">
  <mo:track_number xmlns:mo="http://purl.org/ontology/mo/">1</mo:track_number>
</rdf:Description>

<rdf:Description
 rdf:about="http://demo.openlinksw.com/about/rdf/http://musicbrainz.org/track/b88bdafd-e675-4c6a-9681-5ea85ab99446.html#this">
  <mo:duration xmlns:mo="http://purl.org/ontology/mo/" rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">182933</mo:duration>
</rdf:Description>
</rdf:RDF>
]]></programlisting>
        <para><emphasis>Cartridge Hook Function</emphasis></para>
        <para>The cartridge's hook function is listed below. It is important to note that MusicBrainz
 supports a variety of query types, each of which returns a different set of information, depending
on the item type being queried. Full details can be found on the MusicBrainz? site. The sponger
cartridge is capable of handling all the query types supported by MusicBrainz? and is intended to
be used in a drill-down scenario, as would be the case when using an RDF browser such as the
<ulink url="http://ode.openlinksw.com/">OpenLink Data Explorer (ODE)</ulink>. This example focuses primarily on the types release and track.</para>
<programlisting><![CDATA[
create procedure DB.DBA.RDF_LOAD_MBZ (
  in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
  inout _ret_body any, inout aq any, inout ps any, inout _key any,
  inout opts any)
{
  declare kind, id varchar;
  declare tmp, incs any;
  declare uri, cnt, hdr, inc, xd, xt varchar;
  tmp := regexp_parse ('http://musicbrainz.org/([^/]*)/([^\.]+)', new_origin_uri, 0);
  declare exit handler for sqlstate '*'
  {
    -- dbg_printf ('%s', __SQL_MESSAGE);
    return 0;
  };
  if (length (tmp) < 6)
    return 0;

  kind := subseq (new_origin_uri, tmp[2], tmp[3]);
  id :=   subseq (new_origin_uri, tmp[4], tmp[5]);
  incs := vector ();
  if (kind = 'artist')
    {
      inc := 'aliases artist-rels label-rels release-rels track-rels url-rels';
      incs :=
    vector (
	'sa-Album', 'sa-Single', 'sa-EP', 'sa-Compilation', 'sa-Soundtrack',
	'sa-Spokenword', 'sa-Interview', 'sa-Audiobook', 'sa-Live', 'sa-Remix', 'sa-Other'
	, 'va-Album', 'va-Single', 'va-EP', 'va-Compilation', 'va-Soundtrack',

	'va-Spokenword', 'va-Interview', 'va-Audiobook', 'va-Live', 'va-Remix', 'va-Other'
	);
    }
  else if (kind = 'release')
    inc := 'artist counts release-events discs tracks artist-rels label-rels release-rels track-rels url-rels track-level-rels labels';
  else if (kind = 'track')
    inc := 'artist releases puids artist-rels label-rels release-rels track-rels url-rels';
  else if (kind = 'label')
    inc := 'aliases artist-rels label-rels release-rels track-rels url-rels';
  else
    return 0;
  if (dest is null)
    DELETE FROM DB.DBA.RDF_QUAD WHERE G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_iri);
  DB.DBA.RDF_LOAD_MBZ_1 (graph_iri, new_origin_uri, dest, kind, id, inc);
  DB.DBA.TTLP (sprintf ('<%S> <http://xmlns.com/foaf/0.1/primaryTopic> <%S> .\n<%S> a <http://xmlns.com/foaf/0.1/Document> .',
  	new_origin_uri, DB.DBA.RDF_SPONGE_PROXY_IRI (new_origin_uri), new_origin_uri),
  	'', graph_iri);
  foreach (any inc1 in incs) do
    {
      DB.DBA.RDF_LOAD_MBZ_1 (graph_iri, new_origin_uri, dest, kind, id, inc1);
    }
  return 1;
};
]]></programlisting>
        <para>The hook function uses a subordinate procedure RDF_LOAD_MBZ_1:</para>
<programlisting><![CDATA[
create procedure DB.DBA.RDF_LOAD_MBZ_1 (in graph_iri varchar, in new_origin_uri varchar,
   in dest varchar, in kind varchar, in id varchar, in inc varchar)
{
  declare uri, cnt, xt, xd, hdr any;
  uri := sprintf ('http://musicbrainz.org/ws/1/%s/%s?type=xml&inc=%U', kind, id, inc);
  cnt := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
  xt := xtree_doc (cnt);
  xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_cartridges_path_') || 'xslt/mbz2rdf.xsl', xt,
        vector ('baseUri', new_origin_uri));
  xd := serialize_to_UTF8_xml (xd);
  DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
};
]]></programlisting>
        <para><emphasis>XSLT Stylesheet</emphasis></para>
        <para>The key sections of the MusicBrainz XSLT template relevant to this example are
listed below. Only the sections relating to an artist, his releases, or the tracks on those
releases, are shown.</para>
<programlisting><![CDATA[
<!DOCTYPE xsl:stylesheet [
<!ENTITY xsd "http://www.w3.org/2001/XMLSchema#">
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<!ENTITY rdfs "http://www.w3.org/2000/01/rdf-schema#">
<!ENTITY mo "http://purl.org/ontology/mo/">
<!ENTITY foaf "http://xmlns.com/foaf/0.1/">
<!ENTITY mmd "http://musicbrainz.org/ns/mmd-1.0#">
<!ENTITY dc "http://purl.org/dc/elements/1.1/">
]>
<xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:vi="http://www.openlinksw.com/virtuoso/xslt/"
    xmlns:rdf=""
    xmlns:rdfs=""
    xmlns:foaf=""
    xmlns:mo=""
    xmlns:mmd=""
    xmlns:dc=""
    >

    <xsl:output method="xml" indent="yes" />
    <xsl:variable name="base" select="'http://musicbrainz.org/'"/>
    <xsl:variable name="uc">ABCDEFGHIJKLMNOPQRSTUVWXYZ</xsl:variable>
    <xsl:variable name="lc">abcdefghijklmnopqrstuvwxyz</xsl:variable>
    <xsl:template match="/mmd:metadata">
	<rdf:RDF>
	    <xsl:apply-templates />
	</rdf:RDF>
    </xsl:template>

    ...

    <xsl:template match="mmd:artist[@type='Person']">
	<mo:MusicArtist rdf:about="{vi:proxyIRI (concat($base,'artist/',@id,'.html'))}">
	    <foaf:name><xsl:value-of select="mmd:name"/></foaf:name>
	    <xsl:for-each select="mmd:release-list/mmd:release|mmd:relation-list[@target-type='Release']/mmd:relation/mmd:release">
		<foaf:made rdf:resource="{vi:proxyIRI (concat($base,'release/',@id,'.html'))}"/>
	    </xsl:for-each>
	</mo:MusicArtist>
	<xsl:apply-templates />
    </xsl:template>

    <xsl:template match="mmd:release">
	<mo:Record rdf:about="{vi:proxyIRI (concat($base,'release/',@id,'.html'))}">
	    <dc:title><xsl:value-of select="mmd:title"/></dc:title>
	    <mo:release_type rdf:resource="{translate (substring-before (@type, ' '),
                                                          $uc, $lc)}"/>
	    <mo:release_status rdf:resource="{translate (substring-after (@type, ' '), $uc,
                                                  $lc)}"/>
	    <xsl:for-each select="mmd:track-list/mmd:track">
		<mo:track rdf:resource="{vi:proxyIRI (concat($base,'track/',@id,'.html'))}"/>

	    </xsl:for-each>
	</mo:Record>
	<xsl:apply-templates select="mmd:track-list/mmd:track"/>
    </xsl:template>

    <xsl:template match="mmd:track">
	<mo:Track rdf:about="{vi:proxyIRI (concat($base,'track/',@id,'.html'))}">
	    <dc:title><xsl:value-of select="mmd:title"/></dc:title>
	    <mo:track_number><xsl:value-of select="position()"/></mo:track_number>
	    <mo:duration rdf:datatype="integer">
             <xsl:value-of select="mmd:duration"/>
           </mo:duration>
	    <xsl:if test="artist[@id]">
		<foaf:maker rdf:resource="{vi:proxyIRI (concat ($base, 'artist/',
                                          artist/@id, '.html'))}"/>
	    </xsl:if>
	    <mo:musicbrainz rdf:resource="{vi:proxyIRI (concat ($base, 'track/', @id, '.html'))}"/>
	</mo:Track>
    </xsl:template>

    ...

    <xsl:template match="text()"/>
</xsl:stylesheet>
]]></programlisting>
      </sect4>
      <sect4 id="virtuosospongercreatecartentextmapcont">
        <title>Entity Extractor &amp; Mapper Component</title>

<para>
Used to extract RDF from a Web Data Source the Virtuoso Sponger Cartridge RDF Extractor consumes services from: Virtuoso PL, C/C++, Java
based RDF Extractors</para>
<para>The RDF mappers provide a way to extract metadata from non-RDF documents such as HTML pages,
images Office documents etc. and pass to SPARQL sponger (crawler which retrieve missing
source graphs). For brevity further in this article the "RDF mapper" we simply will call "mapper".
</para>
<para>The mappers consist of PL procedure (hook) and extractor, where extractor itself can be built
using PL, C or any external language supported by Virtuoso server.</para>
<para>Once the mapper is developed it must be plugged into the SPARQL engine by adding a record
in the table DB.DBA.SYS_RDF_MAPPERS.</para>
<para>If a SPARQL query instructs the SPARQL processor to retrieve target graph into local storage,
then the SPARQL sponger will be invoked. If the target graph IRI represents a dereferenceable URL
then content will be retrieved using content negotiation. The next step is the content type
to be detected:</para>
<itemizedlist>
<listitem>If RDF and no further transformation such as GRDDL is needed, then the process would stop.</listitem>
<listitem>If such as 'text/plain' and is not known to have metadata, then the SPARQL sponger will
look in the DB.DBA.SYS_RDF_MAPPERS table by order of RM_ID and for every matching URL or MIME
type pattern (depends on column RM_TYPE) will call the mapper hook.
  <itemizedlist>
    <listitem>If hook returns zero the next mapper will be tried;</listitem>
    <listitem>If result is negative the process would stop instructing the SPARQL nothing was retrieved;</listitem>
    <listitem>If result is positive the process would stop instructing the SPARQL that metadata was retrieved.</listitem>
  </itemizedlist>
</listitem>
</itemizedlist>
<sect5 id="virtuosospongercartridgesextractorpl">
  <title>Virtuoso/PL based Extractors</title>
<para><emphasis>PL hook requirements:</emphasis></para>
<para>Every PL function used to plug a mapper into SPARQL engine must have following parameters in
the same order:</para>
<itemizedlist>
<listitem>in graph_iri varchar: the graph IRI which is currently retrieved</listitem>
<listitem>in new_origin_uri varchar: the URL of the document retrieved</listitem>
<listitem>in destination varchar: the destination graph IRI</listitem>
<listitem>inout content any: the content of the document retrieved by SPARQL sponger </listitem>
<listitem>inout async_queue any: an asynchronous queue, can be used to push something to execute
on background if needed.</listitem>
<listitem>inout ping_service any: the value of [SPARQL] - PingService INI parameter, could be used
to configure a service notification such as pingthesemanticweb.com</listitem>
<listitem>inout api_key any: a plain text id single key value or serialized vector of key structure,
basically the value of RM_KEY column of the DB.DBA.SYS_RDF_MAPPERS table.</listitem>
</itemizedlist>
<para>Note: the names of the parameters are not important, but their order and presence are!</para>
<para><emphasis>Example Implementation:</emphasis></para>
<para>In the example script below we implement a basic mapper, which maps a text/plain mime type to an
imaginary ontology, which extends the class Document from FOAF with properties 'txt:UniqueWords'
and 'txt:Chars', where the prefix 'txt:' we specify as 'urn:txt:v0.0:'.</para>
<programlisting><![CDATA[
use DB;

create procedure DB.DBA.RDF_LOAD_TXT_META
 (
  in graph_iri varchar,
  in new_origin_uri varchar,
  in dest varchar,
  inout ret_body any,
  inout aq any,
  inout ps any,
  inout ser_key any
  )
{
  declare words, chars int;
  declare vtb, arr, subj, ses, str any;
  declare ses any;
  -- if any error we just say nothing can be done
  declare exit handler for sqlstate '*'
    {
      return 0;
    };
  subj := coalesce (dest, new_origin_uri);
  vtb := vt_batch ();
  chars := length (ret_body);

  -- using the text index procedures we get a list of words
  vt_batch_feed (vtb, ret_body, 1);
  arr := vt_batch_strings_array (vtb);

  -- the list has 'word' and positions array, so we must divide by 2
  words := length (arr) / 2;
  ses := string_output ();

  -- we compose a N3 literal
  http (sprintf ('<%s> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Document> .\n', subj), ses);
  http (sprintf ('<%s> <urn:txt:v0.0:UniqueWords> "%d" .\n', subj, words), ses);
  http (sprintf ('<%s> <urn:txt:v0.0:Chars> "%d" .\n', subj, chars), ses);
  str := string_output_string (ses);

  -- we push the N3 text into the local store
  DB.DBA.TTLP (str, new_origin_uri, subj);
  return 1;
};

DELETE FROM DB.DBA.SYS_RDF_MAPPERS WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';

INSERT SOFT DB.DBA.SYS_RDF_MAPPERS (RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION)
VALUES ('(text/plain)', 'MIME', 'DB.DBA.RDF_LOAD_TXT_META', null, 'Text Files (demo)');

-- here we set order to some large number so don't break existing mappers
update DB.DBA.SYS_RDF_MAPPERS
SET RM_ID = 2000
WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_TXT_META';
]]></programlisting>
<para>To test the mapper we just use /sparql endpoint with option 'Retrieve remote RDF data
for all missing source graphs' to execute:</para>
<programlisting><![CDATA[
SELECT *
FROM <URL-of-a-txt-file>
WHERE { ?s ?p ?o }
]]></programlisting>
<para>It is important that the SPARQL_UPDATE role to be granted to "SPARQL" account in order
to allow local repository update via Network Resource Fetch feature.</para>
<para><emphasis>Authentication in Sponger</emphasis></para>
<para>To enable usage of user defined authentication, there are added more parameters to the
/proxy/rdf and /sparql endpoints. So to use it, the RDF browser and iSPARQL should send following
url parameters:</para>
<itemizedlist>
<listitem>for /proxy/rdf endpoint:
<programlisting><![CDATA[
'login=<account name>'
]]></programlisting>
</listitem>
<listitem>for /sparql endpoint:
<programlisting><![CDATA[
get-login=<account name>
]]></programlisting>
</listitem>
</itemizedlist>
</sect5>
<sect5 id="virtuosospongerrdfmappersregistry"><title>Registry</title>
<para>The table DB.DBA.SYS_RDF_MAPPERS is used as registry for registering RDF mappers.</para>
<programlisting><![CDATA[
create table DB.DBA.SYS_RDF_MAPPERS (
    RM_ID integer identity,         -- mapper ID, designate order of execution
    RM_PATTERN varchar,             -- a REGEX pattern to match URL or MIME type
    RM_TYPE varchar default 'MIME', -- what property of the current resource to match: MIME or URL are supported at present
    RM_HOOK varchar,                -- fully qualified PL function name e.q. DB.DBA.MY_MAPPER_FUNCTION
    RM_KEY  long varchar,           -- API specific key to use
    RM_DESCRIPTION long varchar,    -- Mapper description, free text
    RM_ENABLED integer default 1,   -- a flag 0 or 1 integer to include or exclude the given mapper from processing chain
    primary key (RM_TYPE, RM_PATTERN))
;
]]></programlisting>
<para>The current way to register/update/unregister a mapper is just a DML statement e.g.
NSERT/UPDATE/DELETE.</para>
</sect5>
<sect5 id="virtuosospongerrdfmappersexec"><title>Execution order and processing</title>
<para>When SPARQL retrieves a resource with unknown content it will look in the mappers registry
and will loop over every record having RM_ENABLED flag true. The sequence of look-up is based on
ordering by RM_ID column. For every record it will either try matching the MIME type or URL against
RM_PATTERN value and if there is match the function specified in RM_HOOK column will be called.
If the function doesn't exists or signal an error the SPARQL will look at next record.</para>
<para>When it stops looking? It will stop if value returned by mapper function is positive or
negative number, if the return is negative processing stops with meaning no RDF was supplied,
if return is positive the meaning is that RDF data was extracted, if zero integer is returned
then SPARQL will look for next mapper. The mapper function also can return zero if it is expected
next mapper in the chain to get more RDF data.</para>
<para>If none of the mappers matches the signature (MIME type nor URL) the built-in WebDAV
metadata extractor will be called.</para>
</sect5>
<sect5 id="virtuosospongerrdfmappersextfunc"><title>Extension function</title>
<para>The mapper function is a PL stored procedure with following signature:</para>
<programlisting><![CDATA[
THE_MAPPER_FUNCTION_NAME (
        in graph_iri varchar,
        in origin_uri varchar,
        in destination_uri varchar,
        inout content varchar,
        inout async_notification_queue any,
        inout ping_service any,
        inout keys any
        )
{
   -- do processing here
   -- return -1, 0 or 1 (as explained above in Execution order and processing section)
}
;
]]></programlisting>
<para><emphasis>Parameters</emphasis></para>
<itemizedlist>
<listitem>graph_iri - the target graph IRI</listitem>
<listitem>origin_uri - the current URI of processing</listitem>
<listitem>destination_uri - get:destination value</listitem>
<listitem>content - the resource content</listitem>
<listitem>async_notification_queue - if INI parameter PingService is specified in SPARQL
section in the INI file, this is a pre-allocated asynchronous queue to be used to call
ping service</listitem>
<listitem>ping_service - the URL of the ping service configured in SPARQL section in the
INI in PingService parameter</listitem>
<listitem>keys - a string value contained in the RM_KEY column for given mapper, can be
single string or serialized array, generally can be used as mapper specific data.</listitem>
</itemizedlist>
<para><emphasis>Return value</emphasis></para>
<itemizedlist>
<listitem>0 - no data was retrieved or some next matching mapper must extract more data</listitem>
<listitem>1 - data is retrieved, stop looking for other mappers</listitem>
<listitem>-1 - no data is retrieved, stop looking for more data</listitem>
</itemizedlist>
</sect5>
<sect5 id="virtuosospongerrdfmapperspackage"><title>Cartridges package content</title>
<para>The Virtuoso supply as a <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.3/virtuoso/cartridges_dav.vad">cartridges_dav.vad</ulink> VAD package a cartridge for extracting RDF data
from certain popular Web resources and file types. It can be installed (if not already) using
VAD_INSTALL function, see the VAD chapter in documentation on how to do that.</para>
<para><emphasis>HTTP-in-RDF</emphasis></para>
<para>Maps the HTTP request response to HTTP Vocabulary in RDF, see http://www.w3.org/2006/http#.</para>
<para>This mapper is disabled by default. If it's enabled , it must be first in order of execution.</para>
<para>Also it always will return 0, which means any other mapper should push more data.</para>
<para><emphasis>HTML</emphasis></para>
<para>This mapper is composite, it looking for metadata which can specified in a HTML pages as
follows:</para>
<itemizedlist>
<listitem>Embedded/linked RDF
  <itemizedlist>
    <listitem>scan for meta in RDF
<programlisting><![CDATA[
<link rel="meta" type="application/rdf+xml"
]]></programlisting></listitem>
    <listitem>RDF embedded in xHTML (as markup or inside XML comments)</listitem>
      </itemizedlist>
</listitem>
<listitem>Micro-formats
  <itemizedlist>
    <listitem>GRDDL - GRDDL Data Views: RDF expressed in XHTML and XML: http://www.w3.org/2003/g/data-view#</listitem>
    <listitem>eLinked Data - http://purl.org/NET/erdf/profile</listitem>
    <listitem>RDFa</listitem>
    <listitem>hCard - http://www.w3.org/2006/03/hcard</listitem>
    <listitem>hCalendar - http://dannyayers.com/microformats/hcalendar-profile</listitem>
    <listitem>hReview - http://dannyayers.com/micromodels/profiles/hreview</listitem>
    <listitem>relLicense - CC license: http://web.resource.org/cc/schema.rdf</listitem>
    <listitem>Dublin Core (DCMI) - http://purl.org/dc/elements/1.1/</listitem>
    <listitem>geoURL - http://www.w3.org/2003/01/geo/wgs84_pos#</listitem>
    <listitem>Google Base - OpenLink Virtuoso specific mapping</listitem>
    <listitem>Ning Metadata </listitem>
      </itemizedlist>
</listitem>
<listitem>Feeds extraction
  <itemizedlist>
    <listitem>RSS/Linked Data - SIOC &amp; AtomOWL</listitem>
    <listitem>RSS 1.0 - RSS/RDF, SIOC &amp; AtomOWL</listitem>
    <listitem>Atom 1.0 - RSS/RDF, SIOC &amp; AtomOWL</listitem>
  </itemizedlist>
</listitem>
<listitem>xHTML metadata transformation using FOAF (foaf:Document) and Dublin Core
properties (dc:title, dc:subject etc.)</listitem>
</itemizedlist>
<para>The HTML page mapper will look for RDF data in order as listed above, it will try to extract
metadata on each step and will return positive flag if any of the above step give a RDF data. In
case where page URL matches some of other RDF mappers listed in registry it will return 0 so
next mapper to extract more data. In order to function properly, this mapper must be executed
before any other specific mappers.</para>
<para><emphasis>Flickr URLs</emphasis></para>
<para>This mapper extracts metadata of the Flickr images, using Flickr REST API. To function
properly it must have configured key. The Flickr mapper extracts metadata using: CC license,
Dublin Core, Dublin Core Metadata Terms, GeoURL, FOAF, EXIF: http://www.w3.org/2003/12/exif/ns/ ontology.
</para>
<para><emphasis>Amazon URLs</emphasis></para>
<para>This mapper extracts metadata for Amazon articles, using Amazon REST API. It needs a Amazon
API key in order to be functional.</para>
<para><emphasis>eBay URLs</emphasis></para>
<para>Implements eBay REST API for extracting metadata of eBay articles, it needs a key and user
name to be configured in order to work.</para>
<para><emphasis>Open Office (OO) documents</emphasis></para>
<para>The OO documents contains metadata which can be extracted using UNZIP, so this extractor
needs Virtuoso unzip plugin to be configured on the server.</para>
<para><emphasis>Yahoo traffic data URLs</emphasis></para>
<para>Implements transformation of the result of Yahoo traffic data to RDF.
</para>
<para><emphasis>iCal files</emphasis></para>
<para>Transform iCal files to RDF as per http://www.w3.org/2002/12/cal/ical# .</para>
<para><emphasis>Binary content, PDF, PowerPoint</emphasis></para>
<para>The unknown binary content, PDF and MS PowerPoint files can be transformed to RDF using
Aperture framework (http://aperture.sourceforge.net/). This mapper needs Virtuoso with Java hosting
support, Aperture framework and MetaExtractor.class installed on the host system in order to work.</para>
<para>The Aperture framework &amp; MetaExtractor.class must be installed on the system before to
install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.3/virtuoso/cartridges_dav.vad">Cartridges VAD package</ulink>. 
If the package is already installed, then to activate this mapper
you can just re-install the VAD.</para>
<para><emphasis>Setting-up Virtuoso with Java hosting to run Aperture framework</emphasis></para>
<itemizedlist>
<listitem>Install a Virtuoso binary which includes built-in Java hosting support (The executable name will
indicate whether the required hosting support is built in - a suitably enabled executable will include
javavm in the name, for example virtuoso-javavm-t, rather than virtuoso-t).</listitem>
<listitem>Download the Aperture framework from http://aperture.sourceforge.net.</listitem>
<listitem>Unpack the contents of the framework's lib directory into an 'aperture' subdirectory of the
Virtuoso working directory, i.e. of the directory containing the database and virtuoso.ini files.</listitem>
<listitem>Ensure the Virtuoso working directory includes a 'lib' subdirectory containing the file
MetaExtractor.class. (At the current time MetaExtractor.class in not included in the cartridges VAD.
Please contact OpenLink Technical Support to obtain a copy.)</listitem>
<listitem>In the [Parameters] section of the virtuoso.ini configuration file:
<itemizedlist>
  <listitem>Add the line (linebreaks have been inserted for clarity):
<programlisting><![CDATA[
JavaClasspath = lib:aperture/DFKIUtils2.jar:aperture/JempBox-0.2.0.jar:aperture/activation-1.0.2-upd2.jar:aperture/aduna-commons-xml-2.0.jar:
aperture/ant-compression-utils-1.7.1.jar:aperture/aperture-1.2.0.jar:aperture/aperture-examples-1.2.0.jar:aperture/aperture-test-1.2.0.jar:
aperture/applewrapper-0.2.jar:aperture/bcmail-jdk14-132.jar:aperture/bcprov-jdk14-132.jar:aperture/commons-codec-1.3.jar:aperture/commons-httpclient-3.1.jar:
aperture/commons-lang-2.3.jar:aperture/demork-2.1.jar:aperture/flickrapi-1.0.jar:aperture/fontbox-0.2.0-dev.jar:aperture/htmlparser-1.6.jar:
aperture/ical4j-1.0-beta4.jar:aperture/infsail-0.1.jar:aperture/jacob-1.10.jar:aperture/jai_codec-1.1.3.jar:aperture/jai_core-1.1.3.jar:aperture/jaudiotagger-1.0.8.jar:
aperture/jcl104-over-slf4j-1.5.0.jar:aperture/jpim-0.1-aperture-1.jar:aperture/junit-3.8.1.jar:aperture/jutf7-0.9.0.jar:aperture/mail-1.4.jar:
aperture/metadata-extractor-2.4.0-beta-1.jar:aperture/mstor-0.9.11.jar:aperture/nrlvalidator-0.1.jar:aperture/openrdf-sesame-2.2.1-onejar-osgi.jar:
aperture/osgi.core-4.0.jar:aperture/pdfbox-0.7.4-dev-20071030.jar:aperture/poi-3.0.2-FINAL-20080204.jar:aperture/poi-scratchpad-3.0.2-FINAL-20080204.jar:
aperture/rdf2go.api-4.6.2.jar:aperture/rdf2go.impl.base-4.6.2.jar:aperture/rdf2go.impl.sesame20-4.6.2.jar:aperture/rdf2go.impl.util-4.6.2.jar:
aperture/slf4j-api-1.5.0.jar:aperture/slf4j-jdk14-1.5.0.jar:aperture/unionsail-0.1.jar:aperture/winlaf-0.5.1.jar
]]></programlisting>
</listitem>
  <listitem>Ensure DirsAllowed includes directories /tmp, (or the temporary directory for the host
operating system), lib and aperture.</listitem>
          </itemizedlist>
        </listitem>
<listitem>Start the Virtuoso server with java hosting support</listitem>
<listitem>Configure the cartridge either by installing the cartridges VAD or, if the VAD is already
installed, by executing procedure DB.DBA.RDF_APERTURE_INIT.</listitem>
<listitem>During the VAD installation process, RDF_APERTURE_INIT() configures the Aperture cartridge. If
you look in the list of available cartridges under the RDF &gt; Sponger tab in Conductor, you should see
an entry for 'Binary Files'.</listitem>
      </itemizedlist>
<para>To check the cartridge has been configured, connect with Virtuoso's ISQL tool:</para>
<itemizedlist>
<listitem>Issue the command:
<programlisting><![CDATA[
SQL> SELECT udt_is_available('APERTURE.DBA.MetaExtractor');
]]></programlisting>
</listitem>
<listitem>Copy a test PDF document to the Virtuoso working directory, then execute:
<programlisting><![CDATA[
SQL> SELECT APERTURE.DBA."MetaExtractor"().getMetaFromFile ('some_pdf_in_server_working_dir.pdf', 0);

... some RDF data should be returned ...
]]></programlisting>
</listitem>
         </itemizedlist>
<para>You should now be able to Fetch all Network Resource document types supported by the Aperture framework, (using
one of the standard Sponger invocation mechanisms, for instance with a URL of the form
http://example.com/about/rdf/http://targethost/targetfile.pdf), subject to the MIME type pattern
filters configured for the cartridge in the Conductor UI. By default the Aperture cartridge is
registered to match MIME types (application/octet-stream)|(application/pdf)|(application/mspowerpoint).
To Fetch all the Network Resource MIME types Aperture is capable of handling, changed the MIME type pattern to
'application/.*'.</para>
<para>Important: The installation guidelines presented above have been verified on Mac OS X with
Aperture 1.2.0. Some adjustment may be needed for different operating systems or versions of Aperture.</para>
<para><emphasis>Examples &amp; tutorials</emphasis></para>
<para>How to write own RDF mapper? Look at Virtuoso tutorial on this subject
http://demo.openlinksw.com/tutorial/rdf/rd_s_1/rd_s_1.vsp .
</para>
</sect5>
     </sect4>
    </sect3>
      <sect3 id="virtuosospongercreatecustcartrxslt">
      <title>Meta-Cartridges</title>
        <para>So far the discussion has centered on 'primary' cartridges. However,
Virtuoso supports an alternative type of cartridge, a 'meta-cartridge'. The way a
meta-cartridge operates is essentially the same as a primary cartridge, that is it
has a cartridge hook function with the same signature and its inserts data into the
quad store through entity extraction and ontology mapping as before. Where meta-cartridges
differ from primary cartridges is in their intent and their position in the cartridge
invocation pipeline.</para>
        <para>The purpose of meta-cartridges is to enrich graphs produced by other (primary)
cartridges. They serve as general post-processors to add additional information about selected
entities in an RDF graph. For instance, a particular meta-cartridge might be designed to search
for entities of type 'umbel:Country' in a given graph, and then add additional statements about
each country it finds, where the information contained in these statements is retrieved from the
web service targeted by the meta-cartridge. One such example might be a 'World Bank' meta-cartridge
which adds information relating to a country's GDP, its exports of goods and services as a percentage
of GDP etc; retrieved using the <ulink url="http://developer.worldbank.org/docs/">World Bank web service API</ulink>.
In order to benefit from the World Bank meta-cartridge, any primary cartridge which might generate instance data relating to countries should
ensure that each country instance it handles is also described as being of rdf:type 'umbel:Country'.
Here, the <ulink url="http://wiki.umbel.org/index.php/Welcome">UMBEL</ulink> (Upper Mapping and Binding Exchange Layer) ontology is used as a data-source-agnostic
classification system. It provides a core set of 20,000+ subject concepts which act as "a fixed set of
reference points in a global knowledge space". The use of UMBEL in this way serves to decouple
meta-cartridges from primary cartridges and data source specific ontologies.</para>
        <para>Virtuoso includes two default meta-cartridges which use
UMBEL and <ulink url="http://www.opencalais.com/">OpenCalais</ulink> to augment source graphs.</para>
        <para><emphasis>Registration</emphasis></para>
        <para>Meta-cartridges must be registered in the RDF_META_CARTRIDGES table, which fulfills a role
similar to the SYS_RDF_MAPPERS table used by primary cartridges. The structure of the table, and the
meaning and use of its columns, are similar to SYS_RDF_MAPPERS. The meta-cartridge hook function
signature is identical to that for primary cartridges.</para>
        <para>The RDF_META_CARTRIDGES table definition is as follows:</para>
<programlisting><![CDATA[
create table DB.DBA.RDF_META_CARTRIDGES (
MC_ID INTEGER IDENTITY,		-- meta-cartridge ID. Determines the order of the
                           	   meta-cartridge's invocation in the Sponger
                                 processing chain
MC_SEQ INTEGER IDENTITY,
MC_HOOK VARCHAR,			-- fully qualified Virtuoso/PL function name
MC_TYPE VARCHAR,
MC_PATTERN VARCHAR, 		-- a REGEX pattern to match resource URL or
					   MIME type
MC_KEY VARCHAR,			-- API specific key to use
MC_OPTIONS ANY,			-- meta-cartridge specific options
MC_DESC LONG VARCHAR, 		-- meta-cartridge description (free text)
MC_ENABLED INTEGER		-- a 0 or 1 integer flag to exclude or include
					   meta-cartridge from Sponger processing chain
);
]]></programlisting>
        <para>(At the time of writing there is no Conductor UI for registering meta-cartridges, they must be
registered using SQL. A Conductor interface for this task will be added in due course.)</para>
       <para><emphasis>Invocation</emphasis></para>
        <para>Meta-cartridges are invoked through the post-processing hook procedure RDF_LOAD_POST_PROCESS
which is called, for every document retrieved, after RDF_LOAD_RDFXML loads fetched data into the Quad
Store.</para>
        <para>Cartridges in the meta-cartridge registry (RDF_META_CARTRIDGES) are configured to match a given
MIME type or URI pattern. Matching meta-cartridges are invoked in order of their MC_SEQ value. Ordinarily
a meta-cartridge should return 0, in which case the next meta-cartridge in the post-processing chain will
be invoked. If it returns 1 or -1, the post-processing stops and no further meta-cartridges are invoked.</para>
        <para>The order of processing by the Sponger cartridge pipeline is thus:</para>
        <orderedlist>
          <listitem>Try to get RDF in the form of TTL or RDF/XML. If RDF is retrieved if go to step 3</listitem>
          <listitem>Try generating RDF through the Sponger primary cartridges as before</listitem>
          <listitem>Post-process the RDF using meta-cartridges in order of their MC_SEQ value. If a meta-cartridge returns 1 or -1, stop the post-processing chain.</listitem>
        </orderedlist>
        <para>Notice that meta-cartridges may be invoked even if primary cartridges are not.</para>
      <sect4 id="virtuosospongercreatecustcartrexfm">
        <title>Example - A Campaign Finance Meta-Cartridge for Freebase</title>
        <para><emphasis>Note</emphasis></para>
        <para>The example which follows builds on a Freebase Sponger cartridge developed prior to the announcement of Freebase's
support for generating Linked Data through the endpoint http://rdf.freebase.com/ . The OpenLink cartridge
has since evolved to reflect these changes. A snapshot of the Freebase cartridge and stylesheet compatible
with this example can be found <link linkend="virtuosospongerfreeb">here</link>.</para>
        <para><ulink url="http://www.freebase.com/">Freebase</ulink> is an open community database of the world's information which serves facts
and statistics rather than articles. Its designers see this difference in emphasis from article-oriented
databases as beneficial for developers wanting to use Freebase facts in other websites and applications.</para>
        <para>Virtuoso includes a Freebase cartridge in the cartridges VAD. The aim of the example cartridge presented here is to provide a lightweight
meta-cartridge that is used to conditionally add triples to graphs generated by the Freebase cartridge,
if Freebase is describing a U.S. senator.</para>
        <para><emphasis>New York Times Campaign Finance (NYTCF) API</emphasis></para>
        <para>The <ulink url="http://developer.nytimes.com/docs/campaign_finance_api?authChecked=1">New York Times Campaign Finance (NYTCF) API</ulink> allows you to retrieve contribution and
expenditure data based on United States Federal Election Commission filings. You can retrieve totals
for a particular presidential candidate, see aggregates by ZIP code or state, or get details on a
particular donor.</para>
        <para>The API supports a number of query types. To keep this example from being overly long,
the meta-cartridge supports just one of these - a query for the candidate details. An example query
and the resulting output follow:</para>
        <para><emphasis>Query:</emphasis></para>
<programlisting><![CDATA[
http://api.nytimes.com/svc/elections/us/v2/president/2008/finances/candidates/obama,barack.xml?api-key=xxxx
]]></programlisting>
        <para>Result:</para>
<programlisting><![CDATA[
<result_set>
 <status>OK</status>
 <copyright>
  Copyright (c) 2008 The New York Times Company. All Rights Reserved.
 </copyright>
 <results>
  <candidate>
    <candidate_name>Obama, Barack</candidate_name>
    <committee_id>C00431445</committee_id>
    <party>D</party>
    <total_receipts>468841844</total_receipts>
    <total_disbursements>391437723.5</total_disbursements>
    <cash_on_hand>77404120</cash_on_hand>
    <net_individual_contributions>426902994</net_individual_contributions>
    <net_party_contributions>150</net_party_contributions>
    <net_pac_contributions>450</net_pac_contributions>
    <net_candidate_contributions>0</net_candidate_contributions>
    <federal_funds>0</federal_funds>
    <total_contributions_less_than_200>222694981.5</total_contributions_less_than_200>
    <total_contributions_2300>76623262</total_contributions_2300>
    <net_primary_contributions>46444638.81</net_primary_contributions>
    <net_general_contributions>30959481.19</net_general_contributions>
    <total_refunds>2058240.92</total_refunds>
    <date_coverage_from>2007-01-01</date_coverage_from>
    <date_coverage_to>2008-08-31</date_coverage_to>
  </candidate>
 </results>
</result_set>
]]></programlisting>
        <para><emphasis>Sponging Freebase</emphasis></para>
        <para><emphasis>Using OpenLink Data Explorer</emphasis></para>
        <para>The following instructions assume you have the <ulink url="http://ode.openlinksw.com/">OpenLink Data Explorer (ODE)</ulink> browser extension installed in your browser.</para>
        <para>An HTML description of Barack Obama can be obtained directly from Freebase by pasting the
 following URL into your browser: http://www.freebase.com/view/en/barack_obama</para>
        <para>To view RDF data fetched from this page, select 'Linked Data Sources' from the browser's
'View' menu. An OpenLink Data Explorer interface will load in a new tab.</para>
        <para>Clicking on the 'Barack Obama' link under the 'Person' category displayed by ODE fetches 
RDF data using the Freebase cartridge. Click the 'down arrow' adjacent to the 'Barack Obama' link to
explore the retrieved data.</para>
        <para>Assuming your Virtuoso instance is running on port 8890 on localhost, the list of data caches displayed
by ODE should include: http://example.com/about/html/http/www.freebase.com/view/en/barack_obama#this</para>
        <para>The information displayed in the rest of the page relates to the entity instance identified by this URI. The prefix http://example.com/about/html/http/ prepended to the
original URI indicates that the Sponger Proxy Service has been invoked. The Sponger creates an associated
entity instance (identified by the above URI with the #this suffix) which holds network resource information being fetched 
about the original entity.</para>
        <para><emphasis>Using the Command Line</emphasis></para>
        <para>As an alternative to ODE, you can perform Network Resource Fetch from the command line with the command:</para>
<programlisting><![CDATA[
curl -H "Accept: text/xml" "http://example.com/about/html/http/www.freebase.com/view/en/barack_obama"
]]></programlisting>
        <para>To view the results, you can use Conductor's browser-based SPARQL interface (e.g.
http://example.com/sparql) to query the resulting graph generated by the Sponger,
http://www.freebase.com/view/en/barack_obama.</para>
        <para><emphasis>Installing the Meta-Cartridge</emphasis></para>
        <para>To register the meta-cartridge, a procedure similar to the following can be used:</para>
<programlisting><![CDATA[
create procedure INSTALL_RDF_LOAD_NYTCF ()
{
  -- delete any previous NYTCF cartridge installed as a primary cartridge
  DELETE FROM SYS_RDF_MAPPERS WHERE RM_HOOK = 'DB.DBA.RDF_LOAD_NYTCF';
  -- register in the meta-cartridge post-processing chain
  INSERT SOFT DB.DBA.RDF_META_CARTRIDGES (MC_PATTERN, MC_TYPE, MC_HOOK,
    MC_KEY, MC_DESC, MC_OPTIONS)
    VALUES (
    'http://www.freebase.com/view/.*',
    'URL', 'DB.DBA.RDF_LOAD_NYTCF', '2c1d95a62e5fxxxxx', 'Freebase NYTCF',
    vector ());
};
]]></programlisting>
        <para>Looking at the list of cartridges in Conductor's 'RDF Cartridges' screen, you will
see that the Freebase cartridge is configured by default to perform Network Resource Fetch of URIs which match the pattern
"http://www.freebase.com/view/.*" The meta-cartridge is configured to match on the same URI pattern.</para>
        <para>To use the Campaign Finance API, you must register and request an API key. The script
above shows an invalid key. Replace it with your own key before executing the procedure.</para>
        <para><emphasis>NYTCF Meta-Cartridge Functions</emphasis></para>
        <para>The meta-cartridge function definitions are listed below. They can be executed by
pasting them into Conductor's iSQL interface.</para>
<programlisting><![CDATA[
-- New York Times: Campaign Finance Web Service
-- See http://developer.nytimes.com/docs/campaign_finance_api

-- DB.DBA.RDF_NYTCF_LOOKUP is in effect a lightweight lookup cartridge that is used
-- to conditionally add triples to graphs generated by the Wikipedia and
-- Freebase cartridges. These cartridges call on RDF_NYTCF_LOOKUP when
-- handling an entity of rdf:type yago:Congressman109955781. The NYTCF lookup
-- cartridge (aka a metacartridge) is used to return campaign finance data
-- for the candidate in question retrieved from the New York Times Campaign
-- Finance web service.
create procedure DB.DBA.RDF_NYTCF_LOOKUP(
  in candidate_id any, 		-- id of candidate
  in graph_iri varchar,		-- graph into which the additional campaign finance triples should be loaded
  in api_key varchar		-- NYT finance API key
)
{
  declare version, campaign_type, year any;
  declare nyt_url, hdr, tmp any;
  declare xt, xd any;

  -- Common parameters - The NYT API only supports the following values at present:
  version := 'v2';
  campaign_type := 'president';
  year := '2008';

  -- Candidate summaries
  -- nyt_url := sprintf('http://api.nytimes.com/svc/elections/us/%s/%s/%s/finances/totals.xml?api-key=%s',
  --	version, campaign_type, year, api_key);

  -- Candidate details
  nyt_url := sprintf('http://api.nytimes.com/svc/elections/us/%s/%s/%s/finances/candidates/%s.xml?api-key=%s',
  	version, campaign_type, year, candidate_id, api_key);

  tmp := http_client_ext (nyt_url, headers=>hdr, proxy=>connection_get ('sparql-get:proxy'));
  if (hdr[0] not like 'HTTP/1._ 200 %')
    signal ('22023', trim(hdr[0], '\r\n'), 'DB.DBA.RDF_LOAD_NYTCF_LOOKUP');
  xd := xtree_doc (tmp);

  -- baseUri specifies what the generated RDF description is about
  -- <rdf:Description rdf:about="{baseUri}">
  -- Example baseUri's:
  -- http://example.com/about/rdf/http://www.freebase.com/view/en/barack_obama#this
  -- http://example.com/about/rdf/http://www.freebase.com/view/en/hillary_rodham_clinton#this
  declare path any;
  declare lang, k, base_uri varchar;

  if (graph_iri like 'http://rdf.freebase.com/ns/%.%')
    base_uri := graph_iri;
  else
    {
      path := split_and_decode (graph_iri, 0, '%\0/');
      k := path [length(path) - 1];
      lang := path [length(path) - 2];

      base_uri := sprintf ('http://rdf.freebase.com/ns/%U.%U', lang, k);
    }

  xt := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_cartridges_path_') || 'xslt/nytcf2rdf.xsl', xd,
      	vector ('baseUri', base_uri));
  xd := serialize_to_UTF8_xml (xt);
  DB.DBA.RDF_LOAD_RDFXML (xd, '', graph_iri);
}
;

create procedure DB.DBA.RDF_MQL_RESOURCE_IS_SENATOR (
  in fb_graph_uri varchar	-- URI of graph containing Freebase resource
)
{
  -- Check if the resource described by Freebase is a U.S. senator. Only then does it make sense to query for campaign finance
  -- data from the NYT data space.
  --
  -- To test for senators, we start by looking for two statements in the Freebase cartridge output, similar to:
  --
  -- <rdf:Description rdf:about="http://example.com/about/rdf/http://www.freebase.com/view/en/hillary_rodham_clinton#this">
  --   <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
  --   <rdfs:seeAlso rdf:resource="http://en.wikipedia.org/wiki/Hillary_Rodham_Clinton"/>
  --   ...
  -- where the graph generated by the Sponger will be <http://www.freebase.com/view/en/hillary_rodham_clinton>
  --
  -- To test whether a resource is a senator:
  -- 1) Check whether the Freebase resource is of rdf:type foaf:Person
  -- 2) Extract the person_name from the Wikipedia URI referenced by rdfs:seeAlso
  -- 3) Use the extracted person_name to build a URI to DBpedia's description of the person.
  -- 4) Query the DBpedia description to see if the person is of rdf:type yago:Senator110578471
  declare xp, xt, tmp any;
  declare qry varchar;			-- SPARQL query
  declare qry_uri varchar;		-- query URI
  declare qry_res varchar;		-- query result
  declare dbp_resource_name varchar;	-- Equivalent resource name in DBpedia
  declare fb_resource_uri varchar; 	-- Freebase resource URI
  declare path any;
  declare lang, k varchar;

  declare exit handler for sqlstate '*' {
    return 0;
  };

  if (fb_graph_uri like 'http://rdf.freebase.com/ns/%.%')
    fb_resource_uri := fb_graph_uri;
  else
{
      path := split_and_decode (fb_graph_uri, 0, '%\0/');
      if (length (path) < 2)
	return 0;

      k := path [length(path) - 1];
      lang := path [length(path) - 2];

      fb_resource_uri := sprintf ('http://rdf.freebase.com/ns/%U.%U', lang, k);
    }

  -- 1) Check whether the Freebase resource is a politician from united_states
  {
    declare stat, msg varchar;
    declare mdata, rset any;

    qry := sprintf ('sparql ask from <%s> where { <%s> <http://rdf.freebase.com/ns/people.person.profession> <http://rdf.freebase.com/ns/en.politician> ; <http://rdf.freebase.com/ns/people.person.nationality> <http://rdf.freebase.com/ns/en.united_states> . }', fb_graph_uri, fb_resource_uri);
    exec (qry, stat, msg, vector(), 1, mdata, rset);
    if (length(rset) = 0 or rset[0][0] <> 1)
      return 0;
  }

  return 1;
}
;

create procedure DB.DBA.RDF_LOAD_NYTCF_META (in graph_iri varchar, in new_origin_uri varchar,  in dest varchar,
    inout _ret_body any, inout aq any, inout ps any, inout _key any, inout opts any)
{
  declare candidate_id, candidate_name any;
  declare api_key any;
  declare indx, tmp any;
  declare ord int;

declare exit handler for sqlstate '*'
{
 return 0;
};

  if (not DB.DBA.RDF_MQL_RESOURCE_IS_SENATOR (new_origin_uri))
    return 0;

  -- TO DO: hardcoded for now
  -- Need a mechanism to specify API key for meta-cartridges
  -- Could retrieve from virtuoso.ini?
api_key := _key;

  -- NYT API supports a candidate_id in one of two forms:
  -- candidate_id ::= {candidate_ID} | {last_name [,first_name]}
  -- first_name is optional. If included, there should be no space after the comma.
  --
  -- However, because this meta cartridge supplies additional triples for the
  -- Wikipedia or Freebase cartridges, only the second form of candidate_id is
  -- supported. i.e. We extract the candidate name, rather than a numeric
  -- candidate_ID (FEC committee ID) from the Wikipedia or Freebase URL.
  --
  -- It's assumed that the source URI includes the candidate's first name.
  -- If it is omitted, the NYT API will return information about *all* candidates
  -- with that last name - something we don't want.

  indx := strstr(graph_iri, 'www.freebase.com/view/en/');
  if (indx is not null)
  {
    -- extract candidate_id from Freebase URI
    tmp := sprintf_inverse(subseq(graph_iri, indx), 'www.freebase.com/view/en/%s', 0);
    if (length(tmp) <> 1)
  return 0;
    candidate_name := tmp[0];
  }
  else
  {
    indx := strstr(graph_iri, 'wikipedia.org/wiki/');
    if (indx is not null)
    {
      -- extract candidate_id from Wikipedia URI
      tmp := sprintf_inverse(subseq(graph_iri, indx), 'wikipedia.org/%s', 0);
      if (length(tmp) <> 1)
        return 0;
      candidate_name := tmp[0];
    }
    else
      {
	tmp := sprintf_inverse(graph_iri, 'http://%s.freebase.com/ns/%s/%s', 0);
	if (length (tmp) <> 3)
	  tmp := sprintf_inverse(graph_iri, 'http://%s.freebase.com/ns/%s.%s', 0);
	if (length (tmp) <> 3)
	  return 0;
	candidate_name := tmp[2];
      }
  }


  -- split candidate_name into its component parts
  --   candidate_name is assumed to be firstname_[middlename_]*lastname
  --   e.g. hillary_rodham_clinton (Freebase), Hillary_clinton (Wikipedia)
  {
    declare i, _end, len int;
    declare names, tmp_name varchar;

    names := vector ();
    tmp_name := candidate_name;
    len := length (tmp_name);
    while (1)
    {
      _end := strchr(tmp_name, '_');
      if (_end is not null)
      {
        names := vector_concat (names, vector(subseq(tmp_name, 0, _end)));
        tmp_name := subseq(tmp_name, _end + 1);
      }
      else
      {
        names := vector_concat(names, vector(tmp_name));
        goto done;
      }
    }
done:
    if (length(names) < 2)
      return 0;
    -- candidate_id ::= lastname,firstname
    candidate_id := sprintf('%s,%s', names[length(names)-1], names[0]);
  }

  DB.DBA.RDF_NYTCF_LOOKUP(candidate_id, coalesce (dest, graph_iri), api_key);
  return 0;
}
;
]]></programlisting>
        <para><emphasis>NYTCF Meta-Cartridge Stylesheet</emphasis></para>
        <para>The XSLT stylesheet, nyctf2rdf.xsl, used by the meta-cartridge to transform the base
Campaign Finance web service output to RDF is shown below. RDF_NYCTF_LOOKUP() assumes the stylesheet
is located alongside the other stylesheets provided by the cartridges VAD in the Virtuoso WebDAV
folder DAV/VAD/cartridges/xslt. You should create nyctf2rdf.xsl here from the following listing.
The WebDAV Browser interface in Conductor provides the easiest means to upload the stylesheet.</para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xsl:stylesheet [
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<!ENTITY nyt "http://www.nytimes.com/">
]>
<xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:vi="http://www.openlinksw.com/virtuoso/xslt/"
    xmlns:rdf=""
    xmlns:nyt=""
    >
    <xsl:output method="xml" indent="yes" />
    <xsl:template match="/result_set/status">
      <xsl:if test="text() = 'OK'">
        <xsl:apply-templates mode="ok" select="/result_set/results/candidate"/>
      </xsl:if>
    </xsl:template>

    <xsl:template match="candidate" mode="ok">
      <rdf:Description rdf:about="{vi:proxyIRI($baseUri)}">
	  <nyt:candidate_name><xsl:value-of select="candidate_name"/></nyt:candidate_name>
	  <nyt:committee_id><xsl:value-of select="committee_id"/></nyt:committee_id>
	  <nyt:party><xsl:value-of select="party"/></nyt:party>
	  <nyt:total_receipts><xsl:value-of select="total_receipts"/></nyt:total_receipts>
	  <nyt:total_disbursements>
	    <xsl:value-of select="total_disbursements"/>
	  </nyt:total_disbursements>
	  <nyt:cash_on_hand><xsl:value-of select="cash_on_hand"/></nyt:cash_on_hand>
	  <nyt:net_individual_contributions>
	    <xsl:value-of select="net_individual_contributions"/>
         </nyt:net_individual_contributions>
	  <nyt:net_party_contributions>
	    <xsl:value-of select="net_party_contributions"/>
	  </nyt:net_party_contributions>
	  <nyt:net_pac_contributions>
	    <xsl:value-of select="net_pac_contributions"/>
	  </nyt:net_pac_contributions>
	  <nyt:net_candidate_contributions>
	    <xsl:value-of select="net_candidate_contributions"/>
	  </nyt:net_candidate_contributions>
	  <nyt:federal_funds><xsl:value-of select="federal_funds"/></nyt:federal_funds>
	  <nyt:total_contributions_less_than_200>
	    <xsl:value-of select="total_contributions_less_than_200"/>
	  </nyt:total_contributions_less_than_200>
	  <nyt:total_contributions_2300>
	    <xsl:value-of select="total_contributions_2300"/>
	  </nyt:total_contributions_2300>
	  <nyt:net_primary_contributions>
	    <xsl:value-of select="net_primary_contributions"/>
	  </nyt:net_primary_contributions>
	  <nyt:net_general_contributions>
	    <xsl:value-of select="net_general_contributions"/>
	  </nyt:net_general_contributions>
	  <nyt:total_refunds><xsl:value-of select="total_refunds"/></nyt:total_refunds>
	  <nyt:date_coverage_from rdf:datatype="date">
	    <xsl:value-of select="date_coverage_from"/>
	  </nyt:date_coverage_from>
	  <nyt:date_coverage_to rdf:datatype="date">
           <xsl:value-of select="date_coverage_to"/>
          </nyt:date_coverage_to>
      </rdf:Description>
    </xsl:template>
    <xsl:template match="text()|@*"/>
</xsl:stylesheet>
]]></programlisting>
        <para>The stylesheet uses the prefix nyt: (http://www.nytimes.com) for the predicates of
the augmenting triples. This has been used purely for illustration - you may prefer to define your
own ontology for RDF data derived from New York Times APIs.</para>
        <para><emphasis>Testing the Meta-Cartridge</emphasis></para>
        <para>After creating the required Virtuoso/PL functions and installing the stylesheet, you
should be able to test the meta-cartridge by sponging a Freebase page as described earlier using
ODE or the command line. For instance:</para>
          <itemizedlist mark="bullet">
          <listitem>http://www.freebase.com/view/en/barack_obama , or </listitem>
          <listitem>http://www.freebase.com/view/en/hillary_rodham_clinton</listitem>
          </itemizedlist>
        <para>You should see campaign finance data added to the graph created by the Sponger in the form of triples with predicates starting http://www.nytimes.com/xxx, e.g. http://www.nytimes.com/net_primary_contribution.</para>
        <para><emphasis>How The Meta-Cartridge Works</emphasis></para>
        <para>The comments in the meta-cartridge code detail how the cartridge works. In brief:</para>
        <para>Given the URI of the graph being created by the Freebase cartridge,
RDF_MQL_RESOURCE_IS_SENATOR checks if the resource described by Freebase is a U.S. senator.
Only then does it make sense to query for campaign finance data from the NYTCF data space.</para>
        <para>To test for senators, the procedure starts by looking for two statements in the Freebase cartridge output similar to:</para>
<programlisting><![CDATA[
<rdf:Description rdf:about="http://example.com/about/rdf/http://www.freebase.com/view/en/barack_obama#this">
  <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
  <rdfs:seeAlso rdf:resource="http://en.wikipedia.org/wiki/Barack_Obama"/>
  ...
]]></programlisting>
        <para>where the graph generated by the Sponger will be</para>
<programlisting><![CDATA[
<http://www.freebase.com/view/en/barack_obama>
]]></programlisting>
        <para>To test whether a resource is a senator, RDF_MQL_RESOURCE_IS_SENATOR</para>
    <itemizedlist mark="bullet">
      <listitem>Checks whether the Freebase resource is of rdf:type foaf:Person</listitem>
      <listitem>Extracts the person's name from the Wikipedia URI referenced by rdfs:seeAlso</listitem>
      <listitem>Uses the extracted name to build a URI to DBpedia's description of the person. </listitem>
      <listitem>Queries the DBpedia description to see if the person is of rdf:type yago:Senator110578471 (
<ulink url="http://www.mpi-inf.mpg.de/~suchanek/downloads/yago/">YAGO</ulink> is a semantic knowledge base which provides a core set of concepts which in turn are used by DBpedia.)</listitem>
    </itemizedlist>
        <para>Only if this is the case is the RDF_NYTCF_LOOKUP routine called to query for and return campaign finance data for the candidate. The form of the query and the resulting XML output from the Campaign Finance service were presented earlier.</para>
      </sect4>
    </sect3>
    <sect3 id="virtuosospongequeue">
       <title>Sponger Queue API</title>
       <sect4 id="virtuosospongequeuefn"> 
         <title>Functions</title>	
         <itemizedlist mark="bullet">
           <listitem><emphasis>DB.DBA.RDF_SPONGER_QUEUE_ADD</emphasis>: This function is available when 
           rdf cartridges vad is installed.
<programlisting><![CDATA[
DB.DBA.RDF_SPONGER_QUEUE_ADD  (url, options);	
]]></programlisting>           
             <itemizedlist mark="bullet">
               <listitem><emphasis>url</emphasis>: the URI to perform Network Resource Fetch</listitem>
               <listitem><emphasis>options</emphasis>: an array usually typical sponger pragmas, for ex:
<programlisting><![CDATA[
vector ('get:soft',  'soft',  'refresh_free_text',  1);	
]]></programlisting>               	
               	</listitem>
             </itemizedlist>
           </listitem>
         </itemizedlist>
       </sect4> 
       <sect4 id="virtuosospongequeuerwebs"> 
         <title>REST Web service</title>	
         <para>The Sponger REST Web service has the following characteristics:</para>
         <itemizedlist mark="bullet">
           <listitem>endpoint: http://cname/about/service</listitem>
           <listitem>parameters:
             <itemizedlist mark="bullet">
               <listitem>op=add: type of operation, for now addition to the queue is supported</listitem>
               <listitem>uris=[json array]: an array of URIs to be added to the sponger queue, the format is JSON array, for example:
<programlisting><![CDATA[
 { "uris":["http://www.amazon.co.uk/Hama-Stylus-Input-Apple-iPad/dp/B003O0OM0C", "http://www.amazon.co.uk/Krusell-GAIA-Case-Apple-iPad/dp/B003QHXWWC" ] }	
]]></programlisting>               	
               </listitem>
             </itemizedlist>           	
           </listitem>
         </itemizedlist>         
         <para>The service will return a json encoded result of the number of items added, for example:</para>
<programlisting><![CDATA[
{ "result":2 }	
]]></programlisting>  
         <para>In case of error a JSON with error text will be returned and http status 500.</para>       
         <emphasis>cURL example</emphasis>
         <orderedlist>
           <listitem>Assume file.txt which contains URL encoded JSON string:
<programlisting><![CDATA[
uris=%7B%20%22uris%22%3A%5B%22http%3A%2F%2Fwww.amazon.co.uk%2FHama-Stylus-Input-Apple-iPad%2Fdp%2FB003O0OM0C%22%2C%20%22http%3A%2F%2Fwww.amazon.co.uk%2FKrusell-GAIA-Case-Apple-iPad%2Fdp%2FB003QHXWWC%22%20%5D%20%7D	
]]></programlisting>
           </listitem>
           <listitem>Execute the following command:
<programlisting><![CDATA[
curl -i -d@file.txt http://cname/about/service?op=add
HTTP/1.1 200 OK
Server: Virtuoso/06.02.3129 (Darwin) i686-apple-darwin10.0.0  VDB
Connection: Keep-Alive
Date: Thu, 05 May 2011 12:06:24 GMT
Accept-Ranges: bytes
Content-Type: applcation/json; charset="UTF-8"
Content-Length: 14

{ "result":2 }	
]]></programlisting>
           </listitem>
         </orderedlist>
       </sect4>             	
    </sect3>
    <sect3 id="virtuosospongerelatedfunc">
    <title>Virtuoso functions usage examples</title>
    <sect4 id="virtuosospongerelatedfuncstring">
      <title>String Functions</title>
        <para><emphasis><link linkend="fn_sprintf_inverse">sprintf_inverse</link></emphasis></para>
<programlisting><![CDATA[
tmp := sprintf_inverse (new_origin_uri, 'http://farm%s.static.flickr.com/%s/%s_%s.%s', 0);
img_id := tmp[2];
]]></programlisting>
        <para><emphasis><link linkend="fn_split_and_decode">split_and_decode</link></emphasis></para>
<programlisting><![CDATA[
request_hdr := headers[0];
response_hdr := headers[1];
host := http_request_header (request, 'Host');
tmp := split_and_decode (request_hdr[0], 0, '\0\0 ');

http_method := tmp[0];
url := tmp[1];
protocol_version := substring (tmp[2], 6, 8);
tmp := rtrim (response_hdr[0], '\r\n');
tmp := split_and_decode (response_hdr[0], 0, '\0\0 ');
]]></programlisting>
      </sect4>
      <sect4 id="virtuosospongerelatedfuncrurl">
      <title>Retrieving URLs</title>
        <para><emphasis><link linkend="fn_http_get">http_get</link></emphasis></para>
<programlisting><![CDATA[
url := sprintf('http://api.flickr.com/services/rest/?i"??
	method=flickr.photos.getInfo&photo_id=%s&api_key=%s', img_id, api_key);
tmp := http_get (url, hdr);
if (hdr[0] not like 'HTTP/1._ 200 %')
  signal ('22023', trim(hdr[0], '\r\n'), 'RDFXX');
xd := xtree_doc (tmp);
]]></programlisting>
        <para><emphasis>DB.DBA.RDF_HTTP_URL_GET</emphasis></para>
        <para>A wrapper around http_get. Retrieves a URL using the specified HTTP method
(defaults to GET). The function can handle proxies, redirects (up to fifteen) and HTTPS.</para>
<programlisting><![CDATA[
uri := sprintf ('http://musicbrainz.org/ws/1/%s/%s?type=xml&inc=%U',
	kind, id, inc);
cnt := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (cnt);
xd := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_cartridges_path_') || 'xslt/mbz2rdf.xsl', xt, vector ('baseUri', new_origin_uri));
]]></programlisting>
        <para><emphasis><link linkend="fn_http_request_header">http_request_header</link></emphasis></para>
<programlisting><![CDATA[
content := RDF_HTTP_URL_GET (rdf_url, new_origin_uri, hdr, 'GET',
		'Accept: application/rdf+xml, text/rdf+n3, */*');
ret_content_type := http_request_header (hdr, 'Content-Type', null, null);
]]></programlisting>
      </sect4>
      <sect4 id="virtuosospongerelatedfunchnxml">
      <title>Handling Non-XML Response Content</title>
        <para><emphasis>json_parse</emphasis>: Parses JSON content into a tree.</para>
<programlisting><![CDATA[
url := sprintf ('http://www.freebase.com/api/service/mqlread?queries=%U', qr);
  content := http_get (url, hdr);
  tree := json_parse (content);
  tree := get_keyword ('ROOT', tree);
  tree := get_keyword ('result', tree);
]]></programlisting>
      </sect4>
      <sect4 id="virtuosospongerelatedfuncwrarb">
      <title>Writing Arbitrarily Long Text</title>
        <para><emphasis><link linkend="fn_http">http</link></emphasis></para>
<programlisting><![CDATA[
-- Writing N3 to a string output stream using function http(), parsing the N3 into a graph, then loading the graph into the quad store.
ses := string_output ();
http ('@prefix opl: <http://www.openlinksw.com/schema/attribution#> .\n', ses);
http ('@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n', ses);
    ...
DB.DBA.TTLP (ses, base, graph);
DB.DBA.RDF_LOAD_RDFXML (strg, base, graph);
]]></programlisting>
        <para><emphasis><link linkend="fn_string_output">string_output</link></emphasis></para>
<programlisting><![CDATA[
ses := string_output ();
cnt := http_get (sprintf ('http://download.finance.yahoo.com/d/quotes.csv?s=%U&f=nsbavophg&e=.csv',
    symbol));
arr := rdfm_yq_parse_csv (cnt);
http ('<quote stock="NASDAQ">', ses);
foreach (any q in arr) do
  {
    http_value (q[0], 'company', ses);
    http_value (q[1], 'symbol', ses);
    ...
  }
  http ('</quote>', ses);
  content := string_output_string (ses);
  xt := xtree_doc (content);
]]></programlisting>
        <para><emphasis><link linkend="fn_string_output_string">string_output_string</link></emphasis></para>
      </sect4>
      <sect4 id="virtuosospongerelatedfuncxmlxslt">
      <title>XML &amp; XSLT</title>
        <para><emphasis><link linkend="fn_xtree_doc">xtree_doc</link></emphasis></para>
<programlisting><![CDATA[
content := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (content);
]]></programlisting>
       <para><emphasis><link linkend="fn_xpath_eval">xpath_eval</link></emphasis></para>
<programlisting><![CDATA[
profile := cast (xpath_eval ('/html/head/@profile', xt) as varchar);
]]></programlisting>
        <para><emphasis><link linkend="fn_xslt">DB.DBA.RDF_MAPPER_XSLT</link></emphasis></para>
<programlisting><![CDATA[
tmp := http_get (url);
xd := xtree_doc (tmp);
xt := DB.DBA.RDF_MAPPER_XSLT (
	registry_get ('_cartridges_path_') || 'xslt/atom2rdf.xsl',
	xd, vector ('baseUri', coalesce (dest, graph_iri)));
]]></programlisting>
      </sect4>
      <sect4 id="virtuosospongerelatedfunccharserconv">
      <title>Character Set Conversion</title>
        <para><emphasis><link linkend="fn_serialize_to_UTF8_xml">serialize_to_UTF8_xml</link></emphasis></para>
<programlisting><![CDATA[
xt := DB.DBA.RDF_MAPPER_XSLT (
	registry_get ('_cartridges_path_') || 'xslt/crunchbase2rdf.xsl',
	xt, vector ('baseUri', coalesce (dest, graph_iri), 'base', base,
	'suffix', suffix));
xd := serialize_to_UTF8_xml (xt);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
]]></programlisting>
      </sect4>
      <sect4 id="virtuosospongerelatedfuncloaddata">
      <title>Loading Data Into the Quad Store</title>
        <para><emphasis><link linkend="fn_rdf_load_rdfxml">DB.DBA.RDF_LOAD_RDFXML</link></emphasis></para>
<programlisting><![CDATA[
content := RDF_HTTP_URL_GET (uri, '', hdr, 'GET', 'Accept: */*');
xt := xtree_doc (content);
xd := DB.DBA.RDF_MAPPER_XSLT (
	registry_get ('_cartridges_path_') || 'xslt/mbz2rdf.xsl',
	xt, vector ('baseUri', new_origin_uri));
xd := serialize_to_UTF8_xml (xd);
DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
]]></programlisting>
        <para><emphasis><link linkend="fn_ttlp">DB.DBA.TTLP</link></emphasis></para>
<programlisting><![CDATA[
sess := string_output ();
	  . . .
http (sprintf ('<http://dbpedia.org/resource/%s>
	<http://xbrlontology.com/ontology/finance/stock_market#hasCompetitor>
	<http://dbpedia.org/resource/%s> .\n',
	symbol, x), sess);
http (sprintf ('<http://dbpedia.org/resource/%s>
	<http://www.w3.org/2000/01/rdf-schema#isDefinedBy>
	<http://finance.yahoo.com/q?s=%s> .\n',
	 x, x), sess);
content := string_output_string (sess);
DB.DBA.TTLP (content, new_origin_uri, coalesce (dest, graph_iri));
]]></programlisting>
  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="rdfinsertmethodsapifunct">Loading RDF using API functions</link></listitem>
    </itemizedlist>
  </tip>
      </sect4>
      <sect4 id="virtuosospongerelatedfuncdebugpoutput">
        <title>Debug Output</title>
        <para><emphasis><link linkend="fn_dbg_obj_print">dbg_obj_print</link></emphasis></para>
<programlisting><![CDATA[
dbg_obj_print ('try all grddl mappings here');
]]></programlisting>
      </sect4>
    </sect3>
    <sect3 id="virtuosospongeref">
      <title>References</title>
      <itemizedlist mark="bullet">
        <listitem>RDF Primer: http://www.w3.org/TR/2004/REC-rdf-primer-20040210/</listitem>
        <listitem>RDF/XML Syntax Specification: http://www.w3.org/TR/rdf-syntax-grammar/</listitem>
        <listitem>GRDDL Primer: http://www.w3.org/TR/grddl-primer/</listitem>
      </itemizedlist>
      <sect4 id="virtuosospongerefping">
        <title>PingTheSemanticWeb RDF Notification Service</title>
        <para><ulink url="http://www.pingthesemanticweb.com/">PingtheSemanticWeb</ulink> (PTSW) is a repository for RDF documents. The PTSW web service
archives the location of recently created or updated RDF documents on the Web. It is intended
for use by crawlers or other types of software agents which need to know when and where the
latest updated RDF documents can be found. They can request a list of recently updated documents
as a starting location to crawl the Semantic Web.</para>
        <para>You may find this service useful for publicizing your own RDF content. Content authors can
notify PTSW that an RDF document has been created or updated by pinging the service with the URL of
the document. The Sponger supports this facility through the async_queue and ping_service parameters
of the cartridge hook function, where the ping_service parameter contains the ping service URL as
configured in the SPARQL section of the virtuoso.ini file:</para>
<programlisting><![CDATA[
[SPARQL]
...
PingService = http://rpc.pingthesemanticweb.com/
...
]]></programlisting>
        <para>The configured ping service can be called using an asynchronous request and
the RDF_SW_PING procedure as illustrated below. </para>
<programlisting><![CDATA[
create procedure DB.DBA.RDF_LOAD_HTML_RESPONSE (
  in graph_iri varchar, in new_origin_uri varchar, in dest varchar,
  inout ret_body any, inout async_queue any, inout ping_service any,
  inout _key any, inout opts any )
{
  ...
  if ( ... and async_queue is not null)
    aq_request (async_queue, 'DB.DBA.RDF_SW_PING',
                vector (ping_service, new_origin_uri));
]]></programlisting>
        <para>For more details refer to section <link linkend="asyncexecmultithread">Asynchronous Execution and Multithreading in Virtuoso/PL</link></para>
      </sect4>
      <sect4 id="virtuosospongeremname">
        <title>Main Namespaces used by OpenLink Cartridges</title>
        <para>A list of the main namespaces / ontologies used by OpenLink-provided Sponger cartridges
is given below. Some of these ontologies may prove useful when creating your own cartridges.</para>
        <itemizedlist mark="bullet">
          <listitem>- http://www.openlinksw.com/virtuoso/xslt/</listitem>
          <listitem>- http://example.com/schemas/XHTML# </listitem>
          <listitem>rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns# </listitem>
          <listitem>rdfs: http://www.w3.org/2000/01/rdf-schema# </listitem>
          <listitem>dc: http://purl.org/dc/elements/1.1/</listitem>
          <listitem>dcterms: http://purl.org/dc/terms/</listitem>
          <listitem>foaf: http://xmlns.com/foaf/0.1/</listitem>
          <listitem>sioc: http://rdfs.org/sioc/ns# </listitem>
          <listitem>sioct: http://rdfs.org/sioc/types# </listitem>
          <listitem>skos: http://www.w3.org/2004/02/skos/core# </listitem>
          <listitem>bibo: http://purl.org/ontology/bibo/</listitem>
        </itemizedlist>
      </sect4>
      <sect4 id="virtuosospongerfreeb">
        <title>Freebase Cartridge &amp; Stylesheet</title>
        <para>Snapshots of the Freebase cartridge and stylesheet compatible with the meta-cartridge
example presented earlier in this document can be found below.</para>
        <para><emphasis>DB.DBA.RDF_LOAD_MQL:</emphasis></para>
<programlisting><![CDATA[
--no_c_escapes-
create procedure DB.DBA.RDF_LOAD_MQL (in graph_iri varchar, in new_origin_uri varchar,  in dest varchar,
    inout _ret_body any, inout aq any, inout ps any, inout _key any, inout opts any)
{
  declare qr, path, hdr any;
  declare tree, xt, xd, types any;
  declare k, cnt, url, sa varchar;

  hdr := null;
  sa := '';
  declare exit handler for sqlstate '*'
  {
    -- dbg_printf ('%s', __SQL_MESSAGE);
    return 0;
  };

  path := split_and_decode (new_origin_uri, 0, '%\0/');
  if (length (path) < 1)
    return 0;
  k := path [length(path) - 1];
  if (path [length(path) - 2] = 'guid')
    k := sprintf ('"id":"/guid/%s"', k);
  else
    {
    if (k like '#%')
        k := sprintf ('"id":"%s"', k);
  else
      {
	sa := DB.DBA.RDF_MQL_GET_WIKI_URI (k);
    k := sprintf ('"key":"%s"', k);
  }
  }
  qr := sprintf ('{"ROOT":{"query":[{%s, "type":[]}]}}', k);
  url := sprintf ('http://www.freebase.com/api/service/mqlread?queries=%U', qr);
  cnt := http_get (url, hdr);
  tree := json_parse (cnt);
  xt := get_keyword ('ROOT', tree);
  if (not isarray (xt))
    return 0;
  xt := get_keyword ('result', xt);
  types := vector ();
  foreach (any tp in xt) do
    {
      declare tmp any;
      tmp := get_keyword ('type', tp);
      types := vector_concat (types, tmp);
    }
  --types := get_keyword ('type', xt);
  DELETE FROM DB.DBA.RDF_QUAD WHERE g =  iri_to_id(new_origin_uri);
  foreach (any tp in types) do
{
      qr := sprintf ('{"ROOT":{"query":{%s, "type":"%s", "*":[]}}}', k, tp);
      url := sprintf ('http://www.freebase.com/api/service/mqlread?queries=%U', qr);
      cnt := http_get (url, hdr);
      --dbg_printf ('%s', cnt);
      tree := json_parse (cnt);
      xt := get_keyword ('ROOT', tree);
      xt := DB.DBA.MQL_TREE_TO_XML (tree);
      --dbg_obj_print (xt);
      xt := DB.DBA.RDF_MAPPER_XSLT (registry_get ('_cartridges_path_') || 'xslt/mql2rdf.xsl', xt,
      	vector ('baseUri', coalesce (dest, graph_iri), 'wpUri', sa));
      sa := '';
      xd := serialize_to_UTF8_xml (xt);
--      dbg_printf ('%s', xd);
  DB.DBA.RM_RDF_LOAD_RDFXML (xd, new_origin_uri, coalesce (dest, graph_iri));
    }
  return 1;
}
]]></programlisting>
        <para><emphasis>mql2rdf.xsl:</emphasis></para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="UTF-8"?>
<!--
 -
 -  $Id$
 -
 -  This file is part of the OpenLink Software Virtuoso Open-Source (VOS)
 -  project.
 -
 -  Copyright (C) 1998-2018 OpenLink Software
 -
 -  This project is free software; you can redistribute it and/or modify it
 -  under the terms of the GNU General Public License as published by the
 -  Free Software Foundation; only version 2 of the License, dated June 1991.
 -
 -  This program is distributed in the hope that it will be useful, but
 -  WITHOUT ANY WARRANTY; without even the implied warranty of
 -  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
 -  General Public License for more details.
 -
 -  You should have received a copy of the GNU General Public License along
 -  with this program; if not, write to the Free Software Foundation, Inc.,
 -  51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-->
<!DOCTYPE xsl:stylesheet [
<!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<!ENTITY bibo "http://purl.org/ontology/bibo/">
<!ENTITY xsd  "http://www.w3.org/2001/XMLSchema#">
<!ENTITY foaf "http://xmlns.com/foaf/0.1/">
<!ENTITY sioc "http://rdfs.org/sioc/ns#">
]>
<xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:vi="http://www.openlinksw.com/virtuoso/xslt/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
    xmlns:sioc=""
    xmlns:bibo=""
    xmlns:foaf=""
    xmlns:skos="http://www.w3.org/2004/02/skos/core#"
    xmlns:dcterms= "http://purl.org/dc/terms/"
    xmlns:mql="http://www.freebase.com/">

    <xsl:output method="xml" indent="yes" />

    <xsl:param name="baseUri" />
    <xsl:param name="wpUri" />

    <xsl:variable name="ns">http://www.freebase.com/</xsl:variable>

    <xsl:template match="/">
	<rdf:RDF>
	    <xsl:if test="/results/ROOT/result/*">
		<rdf:Description rdf:about="{$baseUri}">
		    <rdf:type rdf:resource="Document"/>
		    <rdf:type rdf:resource="Document"/>
		    <rdf:type rdf:resource="Container"/>
		    <sioc:container_of rdf:resource="{vi:proxyIRI($baseUri)}"/>
		    <foaf:primaryTopic rdf:resource="{vi:proxyIRI($baseUri)}"/>
		    <dcterms:subject rdf:resource="{vi:proxyIRI($baseUri)}"/>
		</rdf:Description>
		<rdf:Description rdf:about="{vi:proxyIRI($baseUri)}">
		    <rdf:type rdf:resource="Item"/>
		    <sioc:has_container rdf:resource="{$baseUri}"/>
		    <xsl:apply-templates select="/results/ROOT/result/*"/>
		    <xsl:if test="$wpUri != ''">
			<rdfs:seeAlso rdf:resource="{$wpUri}"/>
		    </xsl:if>
		</rdf:Description>
	    </xsl:if>
	</rdf:RDF>
    </xsl:template>

    <xsl:template match="*[starts-with(.,'http://') or starts-with(.,'urn:')]">
	<xsl:element namespace="{$ns}" name="{name()}">
	    <xsl:attribute name="rdf:resource">
		<xsl:value-of select="vi:proxyIRI (.)"/>
	    </xsl:attribute>
	</xsl:element>
    </xsl:template>

    <xsl:template match="*[starts-with(.,'/')]">
	<xsl:if test="local-name () = 'type' and . like '%/person'">
	    <rdf:type rdf:resource="Person"/>
	</xsl:if>
	<xsl:if test="local-name () = 'type'">
	    <sioc:topic>
		<skos:Concept rdf:about="{vi:proxyIRI (concat ($ns, 'view', .))}"/>
	    </sioc:topic>
	</xsl:if>

	<xsl:element namespace="{$ns}" name="{name()}">
	    <xsl:attribute name="rdf:resource">
		<xsl:value-of select="vi:proxyIRI(concat ($ns, 'view', .))"/>
	    </xsl:attribute>
	</xsl:element>
    </xsl:template>

    <xsl:template match="*[* and ../../*]">
	<xsl:element namespace="{$ns}" name="{name()}">
	    <xsl:attribute name="rdf:parseType">Resource</xsl:attribute>
	    <xsl:apply-templates select="@*|node()"/>
	</xsl:element>
    </xsl:template>

    <xsl:template match="*">
	<xsl:if test="* or . != ''">
		<xsl:choose>
		    <xsl:when test="name()='image'">
			<foaf:depiction rdf:resource="{vi:mql-image-by-name (.)}"/>
		    </xsl:when>
		    <xsl:otherwise>
			<xsl:element namespace="{$ns}" name="{name()}">
			    <xsl:if test="name() like 'date_%'">
				<xsl:attribute name="rdf:datatype">dateTime</xsl:attribute>
			    </xsl:if>
			    <xsl:apply-templates select="@*|node()"/>
			</xsl:element>
		    </xsl:otherwise>
		</xsl:choose>
	</xsl:if>
    </xsl:template>
</xsl:stylesheet>
]]></programlisting>
      </sect4>
    </sect3>
  <sect3 id="rdfspongerprogrammerguidepython"><title>Using Python to perform Virtuoso Sponging</title>
  <para>This section contains the generic steps to use Python language to extend the Virtuoso Sponger.</para>
    <orderedlist>
      <listitem>Build the latest Python hosting module. It will introduce a new function <code>python_exec ()</code>
<para>The parameters of python_exec are :</para>
<itemizedlist mark="bullet">
  <listitem>string containing a python code, it should define one or more functions, see remarks below</listitem>
  <listitem>string containing name of function to be called</listitem>
  <listitem>list of parameters for the function </listitem>
  </itemizedlist>
<para>For Example:</para>
<programlisting><![CDATA[
python_exec (file_to_string ('spoonge.py'), 'rdf4uri', 'http://url..', 'http://base...');
]]></programlisting>
<para>The above means call the rdf4uri ('http://url..', 'http://base...') function from spoonge.py file. 
It is importnat to know that python_exec is restricted to DBA only group and that the python source 
should not have __main__ or this to be restricted in python code to not be called . 
Any print etc. for stdout/stderr will go on server console if server is on foreground. 
Can be used for debug for example but not for real work. </para>
<para>The function is supposed to return just single string, don't try to return multiple results, 
this will not work in this revision. 
</para>
</listitem>
      <listitem>Setup the Virtuoso server INI to include python module:
<programlisting><![CDATA[
...
[Plugins]
LoadPath = ../lib
Load1    = Hosting, hosting_python.so
...
]]></programlisting>      	
      </listitem>
      <listitem>Download and install the rdflib package from http://www.rdflib.net/
Note before to build, disable Zope interface in rdflib as this not work with C-API correctly. 
Or make sure Python has no Zope interfaces installed. 
To disable the zope in rdflib, just comment out following in &lt;rdflibhome&gt;/rdflib/__init__.py:
<programlisting><![CDATA[
 36 #from rdflib.interfaces import IIdentifier, classImplements
 37 #classImplements(URIRef, IIdentifier)
 38 #classImplements(BNode, IIdentifier)
 39 #classImplements(Literal, IIdentifier)	
]]></programlisting>
<para>Then do:</para>
<programlisting><![CDATA[
perl setup.py build
perl setup.py --user install	
]]></programlisting>
      </listitem>
      <listitem>Get an example of python code for sponger like: http://www.ebusiness-unibw.org/wiki/Python4Spongers 
and make sure you disable the last lines which not suitable for calling inside Sponger:
<programlisting><![CDATA[
...
#if __name__ == '__main__':
#	rdf_xml = rdf4uri(uri='http://www.amazon.com/Apple-touch-Generation-NEWEST-MODEL/dp/B002M3SOBU/')
#	print rdf_xml
]]></programlisting>
<para>Store the python code in sponge.py in server working directory. 
Make sure this directory is allowed to read in DirsAllowed INI setting.</para>
</listitem>
      <listitem>Create a procedure and register with Sponger: 
<programlisting><![CDATA[
-- THIS IS FOR DEMO PURPOSE ONLY 

-- for demo purposes we delete all other cartridges registrations to see effect from only this cartridge
delete from DB.DBA.SYS_RDF_MAPPERS;
delete from DB.DBA.RDF_META_CARTRIDGES;

-- register cartridge 
insert soft DB.DBA.SYS_RDF_MAPPERS (RM_PATTERN, RM_TYPE, RM_HOOK, RM_KEY, RM_DESCRIPTION)
	values ('(http://.*amazon.[^/]+/[^/]+/dp/[^/]+(/.*)?)', 'URL', 'DB.DBA.RDF_LOAD_PYTHON_AMAZON_ARTICLE', null, 'Amazon articles');

-- the cartridge stored procedure itself 
create procedure DB.DBA.RDF_LOAD_PYTHON_AMAZON_ARTICLE (in graph_iri varchar, in new_origin_uri varchar,  in dest varchar,
    inout _ret_body any, inout aq any, inout ps any, inout _key any, inout opts any)
{
  declare result any;
  -- we check first python hosting is capable to run code
  if (__proc_exists ('python_exec', 2) is null)
    return 0;
  -- handle any error
  declare exit handler for sqlstate '*'
    {
      -- log the error 
      DB.DBA.RM_RDF_SPONGE_ERROR (current_proc_name (), graph_iri, dest, __SQL_MESSAGE); 	
      return 0;
    };
  -- call the python code
  result := python_exec (file_to_string ('sponge.py'), 'rdf4uri', new_origin_uri);
  -- in case of python error we will get integer zero, so we check 
  if (not isstring (result))
    return 0;
  -- for demo purpose we delete all from this graph
  delete from DB.DBA.RDF_QUAD where G = DB.DBA.RDF_MAKE_IID_OF_QNAME (graph_iri);
  -- load the results
  DB.DBA.RDF_LOAD_RDFXML (result, new_origin_uri, coalesce (dest, graph_iri), 0);
  return 1;
}
;	
]]></programlisting>      	
</listitem>
      <listitem>Test the Sponger code like this:
<programlisting><![CDATA[
sparql define get:soft "soft" select * from <http://www.amazon.com/Apple-touch-Generation-NEWEST-MODEL/dp/B002M3SOBU/> { ?s ?p ?o };	
]]></programlisting>      	
</listitem>                  
    </orderedlist>
  </sect3>
</sect2>
<sect2 id="virtuosospongernnt"><title>Sponger and Nanotations</title>
  <sect3 id="virtuosospongernnts"><title>Situation Analysis</title>
    <para>Since the advent of blogging, it has been clear to everyone that posts require augmentation in order to truly
    	function as rapid-fire meme vectors. For instance, could you imagine early blog posts without tagging?</para>
    <para>Today, we've evolved from early literal tagging (which didn't scale an iota) to wide use of <code>@handles</code> and
    <code>#hastags</code>. Basically, <code>@handles</code> are social network specific HTTP URIs that denote Agents (People,
    Organizations, and Bots) while <code>#hashtags</code> are HTTP URIs that denote Topics.</para>
    <para><emphasis>Problem:</emphasis> Looking at the picture above, in regards to productively encoding and decoding information
    via the World Wide Web medium, it should be obvious that <code>@handles</code> and <code>#hashtags</code> are basically the
    digital equivalents of nouns. And as a consequence, we are basically trying to replicate the power of natural language
    sentences without critical components such as verbs (connectors) and adjectives (classifiers).</para>
    <para><emphasis>A Solution:</emphasis> Leverage the power of an existing language, based on open standards, that already
    delivers the power of natural language without being limited by the physical constraints of paper (as a mechanism for
    sentence persistence and exchange).</para>
    <para>This is where RDF comes into play. It is an open standards based language for constructing digital sentences that pack
    	the same (or even more power) its natural language equivalents. Through the power of RDF it is possible to create
    	micro-annotations (aka. Nanotations) that are embeddable in any kind of text based documents. Naturally, the aforementioned
    	claim doesn't apply to every RDF notation, which is why RDF-Turtle is the vehicle we've chosen to unleash the full power of
    	RDF and the Semantic Web it enables when digital sentences take the form of Linked Open Data.</para>
  </sect3>
  <sect3 id="virtuosospongernntw"><title>What is Nanotation?</title>
    <para>Nanotation is a mechanism for using embedded digital sentences to enhance blog posts, forum discussion posts, tweets
    	(and other micro-blogging posts), HTML, and plain text document. In addition, it turns each of the aforementioned document
    	types into end-user oriented conduits for contributing data to public and/or private Linked Open Data clouds, on a
    	piecemeal basis -- i.e., you turn curating and publishing Linked Open Data Cloud into a productive crowd-sourced jigsaw
    	puzzle game.</para>
  </sect3>
  <sect3 id="virtuosospongernntm"><title>Why is it important?</title>
    <para>Being able to say anything, about anything, whenever, and from wherever, in a manner that's both machine and human
    	comprehensible has always sat at the very foundations of the Semantic Web Project's value proposition. Unfortunately,
    	confusion about RDF -- the powerful language that drives the notion of a global Semantic Web -- lead to a general
    	bottom-up misconception whereby most perceive it as a document content format rather that an abstract language (system
    	of signs, syntax, and semantics) exploitable using a wide variety of notations.</para>
  </sect3>
  <sect3 id="virtuosospongernnthow"><title>How do I use it?</title>
    <para>Due to the compact nature of RDF-Turtle notation, it is possible to embed RDF statements into any text based content.
    	The only requirements are as follows:</para>
    <itemizedlist mark="bullet">
      <listitem>Use the following as a marker for embedded RDF-Turtle based RDF statements:
<programlisting><![CDATA[
## Turtle Start ##
## {Trutle-based-RDF-statements}
## Turtle End ##
]]></programlisting>
      </listitem>
      <listitem>Honor the use of &lt;&gt; to indicate reference identifiers (absolute or relative) </listitem>
      <listitem>Optionally treat <code>@handle</code> as an HTTP URIs that denote Agents -- for a given data space
      (e.g., Twitter, LinkedIn, Facebook, Google+ etc..)</listitem>
      <listitem>Optionally tread <code>#hashtag</code> as an HTTP URI that denotes a Topic -- ditto .</listitem>
    </itemizedlist>
     <sect4 id="virtuosospongernnthowbasicr"><title>Basic Rules (authors and processing engines)</title>
       <para>As per natural language sentences we have the following parts:</para>
       <orderedlist>
         <listitem><emphasis>Subject</emphasis> -- statement focal point </listitem>
         <listitem><emphasis>Predicate</emphasis> -- connection, association, link </listitem>
         <listitem><emphasis>Object</emphasis> -- value of the connection, association, link.</listitem>
       </orderedlist>
       <para>Each sentence subject, predicate, object is denoted (named or referred to) using an identifier (word, phrase,
       	or term). If you want to generate Linked Data that flows across data spaces your best bet is to denote (refer to)
       	sentence subject, predicate, and object (optionally) using identifiers that function like terms -- by using HTTP URIs.</para>
       <para>Use prefixes to shorten RDF-Turtle statements:</para>
<programlisting><![CDATA[
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
]]></programlisting>
       <para>Enables statements such as:</para>
<programlisting><![CDATA[
<>
a foaf:Document .
]]></programlisting>
     </sect4>
     <sect4 id="virtuosospongernnthowex"><title>Examples</title>
<orderedlist>
  <listitem>Most basic Nanotation:
<programlisting><![CDATA[
<> a <#Document> .
<> <#topic> <#Nanotation>.
]]></programlisting>
  </listitem>
  <listitem>More sophisticated Nanotation that leverages terms from existing vocabularies:
<programlisting><![CDATA[
<> a foaf:Document .
<> foaf:topic <#Nanotation>  .
]]></programlisting>
  </listitem>
  <listitem>More sophisticated Nanotation that leverages commas and semicolons for statement
  	brevity e.g. when multiple sentences have a common Subject:
<programlisting><![CDATA[
<>
a foaf:Document;
foaf:topic <#Nanotation>.
]]></programlisting>
  </listitem>
  <listitem>When multiple sentences have a common Subject and Predicate but varying list of Objects:
<programlisting><![CDATA[
<>
a foaf:Document;
foaf:topic <#Nanotation>, <#SemanticWeb>, <#LinkedData>.
]]></programlisting>
  </listitem>
  <listitem>Incorporation of Pronouns into RDF sentence:
<programlisting><![CDATA[
<> a foaf:Document .
<> foaf:maker [a foaf:Person;
               foaf:name "Kingsley Idehen"  ] .
]]></programlisting>
  </listitem>
  <listitem>Processor (parser) hint markers that help Nanotation processors negate seriously mangled HTML content:
<programlisting><![CDATA[
## Turtle Start ##

<>
a foaf:Document;
foaf:topic <#Nanotation>, <#SemanticWeb>, <#LinkedData>.

## Turtle End ##
]]></programlisting>
  </listitem>
</orderedlist>
     </sect4>
     <sect4 id="virtuosospongernnthowprusage"><title>Nanotation Processor Usage</title>
       <para>A nanotation processer is an application or service that's capable of consuming text content enhanced with
       	RDF-Turtle based nanotations. Virtuoso's in-built Linked Data Transformation middleware (aka "Sponger") is an
       	example of an application that supports nanotation. Likewise, our URIBurner service which is a free public service
       	driven by an instance of Virtuoso with the Sponger module enabled: </para>
       <itemizedlist mark="bullet">
         <listitem>When using your own instance of Virtuoso, the Sponger service is invoked via the URL pattern:
<programlisting><![CDATA[
http://{cname-of-virtuoso-host-machine}/sponger
]]></programlisting>
         </listitem>
         <listitem>When using the public <ulink url="http://linkeddata.uriburner.com/">URIBurner service</ulink>, the Sponger
         services is invoked via the URL:
<programlisting><![CDATA[
http://linkeddata.uriburner.com/
]]></programlisting>
         </listitem>
         <listitem>Note: Either approach outlined above will lead you to an HTML page that contains an input field into which
         	you can type or paste an HTTP-accessible document URL. Alternatively you can use the following URI patterns:
<programlisting><![CDATA[
http://{cname-of-virtuoso-host-machine}/about/html/{document-http-uri}
]]></programlisting>
         </listitem>
       </itemizedlist>
       <para>In all cases, you will end up with an HTML document that includes RDF statements that describe the processed
       	document in a manner that also reveals all the embedded nanotations.</para>
     </sect4>
     <sect4 id="virtuosospongernnthownotes"><title>Virtuoso Sponger Implementation Notes</title>
       <para>The Sponger treats resources transferred over HTTP as a duality of both a container document and a primary entity.
       	When a resource is deemed to be an HTML document, the document is treated as the primary entity. Otherwise, where the
       	domain is well known, a custom extractor cartridge populates the primary entity with data arising from API calls and
       	the HTML content is regarded as secondary, relegated to the container document. For example, G+ posts are recognized
       	and the Sponger concentrates on presenting the timestamp, body, tags and links and other features of a post.</para>
       <para>When sponging an HTTP resource, multiple extractor cartridges might be brought to bear. Consequently, there may
       	be multiple triples containing the entity's content.</para>
       <para>The Turtle Sniffer is implemented as a Metacartridge, i.e it runs after all the extractor cartridges have run,
       	augmenting data in the graph. It uses SPARQL inference to collate predicates that constitute "content" for this purpose,
       	along with the HTTP request content (if any), flattening each to plain text.</para>
       <para>Currently, the list of potential content predicates is:</para>
       <itemizedlist mark="bullet">
         <listitem><code>bibo:content</code> (e.g. arising from the HTML+Variants extractor cartridge) </listitem>
         <listitem><code>bibo:abstract</code> </listitem>
         <listitem><code>oplgplus:annotation</code> (used by Google+ for text when sharing items)</listitem>
       </itemizedlist>
       <para>For each of these contents, it checks if it matches the patterns:</para>
<programlisting><![CDATA[
## Nanotation Start ## .... ## Nannotation End ##
]]></programlisting>
<programlisting><![CDATA[
## Turtle Start ## .... ## Turtle Stop ##
]]></programlisting>
<programlisting><![CDATA[
{.... } (note: only applies to tweets on Twitter)
]]></programlisting>
       <para>If a content contains one or more nanotation blocks, each block is parsed in turn as Turtle; if not,
       	it attempts to parse the content item as Turtle in entirety.</para>
       <para>The HTTP document content is only inspected in case of no triples having been extracted by prior means.</para>
       <para>Optionally (enabled by default) each triple may be reified, i.e an rdf:Statement entity created to describe its
       	subject, predicate and object, so you can identify triples arising from nanotations as entities labelled 'Embedded
       	Turtle Statement' and a number in the graph.</para>
       	<sect5 id="virtuosospongernnthownotesdom"><title>Domain-Aware Tag and User Expansion</title>
          <para>The Turtle Sniffer expands the patterns <code>#word</code> and <code>@word</code> when they appear in URI
          (&lt;&gt;) or double quotes (""), in the context of the domain of the URI being sponged.</para>
          <para>For example, a Tweet containing the nanotation: </para>
<programlisting><![CDATA[
## Nannotation Start ##
<@kidehen> foaf:name "Kingsley Idehen" ;
           foaf:knows <@openlink> ;
           scot:has_tag <#Data> .
## Nannotation End ##
]]></programlisting>
          <para>will be expanded to a Turtle string:</para>
<programlisting><![CDATA[
<https://twitter.com/kidehen> foaf:name "Kingsley Idehen" ;
  foaf:knows <https://twitter.com/openlink> ;
  scot:has_tag <https://twitter.com/hashtag/Data#this> .
]]></programlisting>
          <para>We recognize custom URI formats for users and tags in the contexts of Facebook, Twitter, G+, LinkedIn and Delicious.</para>
          <para>Note that the word must appear within quotes -- this is to avoid confusion with Turtle's @prefix directive (which is
          	not a user!) and problems that would be caused by performing similar expansions within a quoted sentence.</para>
       	</sect5>
     </sect4>
  </sect3>
  <sect3 id="virtuosospongernntliveexamples"><title>Live Examples</title>
    <itemizedlist mark="bullet">
      <listitem><ulink url="http://twitter.com/kidehen/status/493857326680776705">RDF statement about Privacy;</ulink></listitem>
      <listitem><ulink url="http://twitter.com/kidehen/status/493848344590966785">RDFs usage re. property/predicate description;</ulink></listitem>
      <listitem><ulink url="https://plus.google.com/112399767740508618350/posts/hNt1df5G7tk">owl:sameAs relation inserted into a Google+ Post;</ulink></listitem>
      <listitem><ulink url="https://www.facebook.com/kidehen/posts/10152525312840751">Heavy duty RDF statements in a Facebook post;</ulink></listitem>
      <listitem><ulink url="http://twitter.com/kidehen/status/494089069995896832">Another Facebook post that puts Ted Nelsons talk on documents and hypertext into contemporary context;</ulink></listitem>
      <listitem><ulink url="https://twitter.com/OpenLink/status/556128747297193985">A tweet reply demonstrating multiple embedded nanotation blocks and tags.</ulink></listitem>
    </itemizedlist>
  </sect3>
    <sect3 id="virtuosospongernntliveexamplesstar"><title>Nanotation generated 5-Star Linked Open Data (via URIBurner - a Nanotation Processor) Examples</title>
    <itemizedlist mark="bullet">
      <listitem><ulink url="http://bit.ly/nanotations-from-2014-07-28">Various Nanotations from 28th July 2014;</ulink></listitem>
      <listitem><ulink url="http://bit.ly/nanotations-for-2014-07-27">Various Nanotations from 27th July 2014;</ulink></listitem>
      <listitem><ulink url="http://bit.ly/utterances-on-twitter-2014-07-25-by-statement-subject">Various Nanotations from 25th July 2014;</ulink></listitem>
      <listitem><ulink url="http://linkeddata.uriburner.com/about/html/https://twitter.com/OpenLink/status/556128747297193985">A tweet reply demonstrating multiple embedded nanotation blocks and tags.</ulink></listitem>
    </itemizedlist>
  </sect3>
  <sect3 id="virtuosospongernntfctexamples"><title>Faceted Browser Nanotation Examples</title>
    <itemizedlist mark="bullet">
     <listitem>Nanotation based RDF statement that describes Nannotation:
       <figure id="rdf1" float="1">
         <title>Faceted Browser Nanotation RDF</title>
         <graphic fileref="ui/nano1.png"/>
       </figure>
     </listitem>
     <listitem>Nanotation that represents a "Hat Tip" relationship type:
       <figure id="rdf1" float="1">
         <title>Faceted Browser Nanotation Hat Tip Type</title>
         <graphic fileref="ui/nano2.png"/>
       </figure>
     </listitem>
     <listitem>Nanotation generated RDF statements aggregated by Subject:
       <figure id="rdf1" float="1">
         <title>Faceted Browser Nanotation RDF Aggregation by Subject</title>
         <graphic fileref="ui/nano3.png"/>
       </figure>
     </listitem>
     <listitem>Nanotation generated RDF statements by Subject:
       <figure id="rdf1" float="1">
         <title>Faceted Browser Nanotation Generation by Subject</title>
         <graphic fileref="ui/nano4.png"/>
       </figure>
     </listitem>
    </itemizedlist>
  </sect3>
  <sect3 id="virtuosospongernnts"><title></title>
  </sect3>
  <sect3 id="virtuosospongernnts"><title></title>
  </sect3>
</sect2>
<sect2 id="virtuosospongersampleuses"><title>Sponger Usage Examples</title>
<itemizedlist mark="bullet">
	<listitem><link linkend="virtuosospongerusageprocessorex">SPARQL Processor Usage Example</link></listitem>
	<listitem><link linkend="virtuosospongerusageproxyex2">RDF Proxy Service Example</link></listitem>
	<listitem><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtDeployingLinkedDataGuide_BrowsingNorthwindRdfView#AncMozToc2">Browsing &amp; Exploring Linked Data View Example Using ODE</ulink></listitem>
	<listitem><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtDeployingLinkedDataGuide_BrowsingNorthwindRdfView#AncMozToc3">Browsing &amp; Exploring Linked Data View Example Using iSPARQL</ulink></listitem>
	<listitem><link linkend="rdfinsertmethodplapissimpleexample">Basic Sponger Cartridge Example</link></listitem>
  <listitem><link linkend="virtuosospongerusagebriefex">HTTP Example for Extracting Metadata using CURL</link></listitem>
  <listitem><link linkend="virtuosospongercartridgetypesmetarestexamples">RESTFul Interaction Examples</link></listitem>
  <listitem><link linkend="virtuosospongercreatecustcartrrgstflickr">Flickr Cartridge Example</link></listitem>
  <listitem><link linkend="virtuosospongercreatecustcartrexmp">MusicBrainz Metadatabase Example</link></listitem>
</itemizedlist>
</sect2>	
</sect1>

<sect1 id="virtuosospongerfacetinstall">
<title>Virtuoso Faceted Browser Installation and configuration</title>
<sect2 id="virtuosospongerfacetinstallprereq">
<title>Prerequisites</title>
<para>Requires <ulink url="http://sourceforge.net/project/showfiles.php?group_id=161622&amp;package_id=319652">Virtuoso 6.0 TP1</ulink> or higher for use.
</para>
</sect2>
<sect2 id="virtuosospongerfacetinstallpreinst">
<title>Pre Installation</title>
<para><emphasis>Note</emphasis>: This step is not required for Virtuoso Release 6.1 and above builds</para>
<para>If you have an existing Virtuoso 6.x installation, and your Quad Store has greater than
10K worth of triples, please perform the following steps:
</para>
<orderedlist>
  <listitem>Run the following commands using the Virtuoso isql program before installing the Faceted
Browser VAD:
<programlisting><![CDATA[
  drop index RDF_QUAD_OPGS;
  drop index RDF_QUAD_POGS;
  drop index RDF_QUAD_GPOS;
  drop index RDF_QUAD_OGPS;

  checkpoint;

  create table R2 (G iri_id_8, S iri_id_8, P iri_id_8, O any, primary key (S, P, O, G));
  alter index R2 on R2 partition (S int (0hexffff00));

  log_enable (2);

  INSERT INTO R2 (G, S, P, O) select G, S, P, O FROM rdf_quad;

  DROP TABLE RDF_QUAD;
  ALTER TABLE r2 rename RDF_QUAD;

  checkpoint;

  create bitmap index RDF_QUAD_OPGS on RDF_QUAD (O, P, G, S) partition (O varchar (-1, 0hexffff));
  create bitmap index RDF_QUAD_POGS on RDF_QUAD (P, O, G, S) partition (O varchar (-1, 0hexffff));
  create bitmap index RDF_QUAD_GPOS on RDF_QUAD (G, P, O, S) partition (O varchar (-1, 0hexffff));

  checkpoint;
]]></programlisting>
</listitem>
</orderedlist>
<para>Note this step may take sometime depending on how many triples are already in your Quad Store.
</para>
</sect2>
<sect2 id="virtuosospongerfacetinstallvadinst">
<title>VAD Package Installation</title>
<orderedlist>
  <listitem>Download and install the <ulink url="http://download.openlinksw.com/packages/5.0/virtuoso/fct_dav.vad">Virtuoso Faceted Browser VAD</ulink>
package using the Conductor System Admin - &gt; Packages tab.
    <figure id="fctinst1" float="1">
      <title>Install the FCT package</title>
      <graphic fileref="ui/fb1.png"/>
    </figure>
</listitem>
  <listitem>The HTML interface of the Faceted Browser Engine is exposed at: <emphasis>http://&lt;cname&gt;/fct</emphasis>,
where "cname" is the hostname:portno your Virtuoso instance is running on.
    <figure id="fctinst3" float="1">
      <title>FCT HTML interface</title>
      <graphic fileref="ui/fb2.png"/>
    </figure>
</listitem>
  <listitem>The Faceted Browser Engine exposes a REST API at the endpoint:
<emphasis>http://&lt;cname&gt;/fct/service</emphasis>.
  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtuosospongerfacentuirestapi">Virtuoso APIs for Faceted REST services</link></listitem>
      <listitem><link linkend="rdfiridereferencingfacetws">Faceted Web Service and Linked Data</link></listitem>
</itemizedlist>
  </tip>
</listitem>
</orderedlist>
</sect2>
<sect2 id="virtuosospongerfacetinstallposinst">
<title>Post Installation</title>
<orderedlist>
  <listitem>Build Full Text Indexes by running the following commands using the Virtuoso
<emphasis>isql</emphasis> program:
<programlisting><![CDATA[
RDF_OBJ_FT_RULE_ADD (null, null, 'All');
VT_INC_INDEX_DB_DBA_RDF_OBJ ();
]]></programlisting>
</listitem>
  <listitem>Run the following procedure using the Virtuoso <emphasis>isql</emphasis> program to
populate label lookup tables periodically and activate the <emphasis>Label</emphasis> text box of the
<emphasis>Entity Label Lookup</emphasis> tab:
<programlisting><![CDATA[
urilbl_ac_init_db()
]]></programlisting>
</listitem>
  <listitem>Run the following procedure using the Virtuoso <emphasis>isql</emphasis> program to calculate the IRI ranks.
Note this should be run periodically as the data grows to re-rank the IRIs.
<programlisting><![CDATA[
s_rank()
]]></programlisting>
</listitem>
  <listitem>Perform Network Resource Fetch of some data to load some RDF triples in the quad store. This can easily be done using
the Virtuoso <emphasis>description.vsp</emphasis> page which provides a hypertext description of RDF Linked Data, by describing
the following page for example (or one of your choice):
<programlisting><![CDATA[
http://cname/about/html/http/news.cnet.com
]]></programlisting>
    <figure id="fctinst4" float="1">
      <title>Network Resource Fetch data</title>
      <graphic fileref="ui/fb3.png"/>
    </figure>
    <figure id="fctinst4" float="1">
      <title>Network Resource Fetch data</title>
      <graphic fileref="ui/fb4.png"/>
    </figure>
</listitem>
  <listitem>Use the Faceted Browser Search and Find User Interface to search for information on "CNET":
    <figure id="fctinst5" float="1">
      <title>Faceted Browser Search</title>
      <graphic fileref="ui/fb5.png"/>
    </figure>
</listitem>
  <listitem>Results of the following form should be returned for the network resource data being fetched.
    <figure id="fctinst6" float="1">
      <title>Faceted Browser Search Results</title>
      <graphic fileref="ui/fb6.png"/>
    </figure>
</listitem>
  <listitem>Click "Types" link shown at the right vertical Navigation</listitem>
  <listitem>Results of the classes/properties should be returned:
    <figure id="fctinst7" float="1">
      <title>Results of the classes/properties</title>
      <graphic fileref="ui/fb7.png"/>
    </figure>
</listitem>
  <listitem>To get Type description, click "Describe" link for a given type, for ex. "Person". </listitem>
  <listitem>A list of attributes and values should be presented for the given resource. Note that automatically is generated QRCode image for the described entity. 
  	<figure id="fctinst7" float="1">
      <title>Results of the classes/properties</title>
      <graphic fileref="ui/fb8.png"/>
    </figure>
</listitem>
  <listitem>Return back to the Attributes list from above by going to the "Facets" tab. </listitem>
  <listitem>To exclude a type, un-tick the checkbox associated with the type: 
    <figure id="fctinst8" float="1">
      <title>Exclude Type(s)</title>
      <graphic fileref="ui/fb9.png"/>
    </figure>
</listitem>
  <listitem>Click the Type URI link</listitem>
  <listitem>Results of excluding the Type(s) should be shown:
    <figure id="fctinst9" float="1">
      <title>Results of Excluded Type(s)</title>
      <graphic fileref="ui/fb10.png"/>
    </figure>
</listitem>
  <listitem>The Faceted Browser Web service endpoint can also be queried to obtain the same results:
<programlisting><![CDATA[
$ more cnet.xml
<?xml version="1.0"?>
<query xmlns="http://openlinksw.com/services/facets/1.0" inference="" same-as="">
  <text>CNET</text>
  <view type="text" limit="20" offset=""/>
</query>

$ curl -H "Content-Type: text/xml" -d @cnet.xml  http://cname/fct/service
<fct:facets xmlns:fct="http://openlinksw.com/services/facets/1.0/">
<fct:sparql>    SELECT ?s1 as ?c1, (bif:search_excerpt (bif:vector ('CNET'), ?o1)) as ?c2, ?sc, ?rank WHERE {{{ SELECT ?s1, (?sc * 3e-1) as ?sc, ?o1, (sql:rnk_scale (<LONG::IRI_RANK> (?s1))) as ?rank  WHERE { ?s1 ?s1textp ?o1 . ?o1 bif:contains  '"CNET"'  option (score ?sc)  . } ORDER BY DESC (?sc * 3e-1 + sql:rnk_scale (<LONG::IRI_RANK> (?s1)))  LIMIT 20  OFFSET 0 }}}</fct:sparql>
<fct:time>16</fct:time>
<fct:complete>yes</fct:complete>
<fct:timeout>0</fct:timeout>
<fct:db-activity>   131R rnd     36R seq      0P disk      0B /      0 messages</fct:db-activity>
 <fct:result type="text">
  <fct:row>
    <fct:column datatype="trank">4.5</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.com">http://news.com</fct:column>
    <fct:column>Technology News - CNET News</fct:column>
    <fct:column><span class="srch_xerpt"><b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">4.5</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml">http://news.cnet.com/2547-1_3-0-20.xml</fct:column>
    <fct:column>CNET News.com</fct:column>
    <fct:column><span class="srch_xerpt"><b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">4.5</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com">http://news.cnet.com</fct:column>
    <fct:column>Technology News - CNET News</fct:column>
    <fct:column><span class="srch_xerpt"><b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3.9</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.com">http://news.com</fct:column>
    <fct:column>Technology News - CNET News</fct:column>
    <fct:column><span class="srch_xerpt">Technology News <b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3.9</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com">http://news.cnet.com</fct:column>
    <fct:column>Technology News - CNET News</fct:column>
    <fct:column><span class="srch_xerpt">Technology News <b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.com">http://news.com</fct:column>
    <fct:column>Technology News - CNET News</fct:column>
    <fct:column><span class="srch_xerpt">Tech news and business reports by <b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml">http://news.cnet.com/2547-1_3-0-20.xml</fct:column>
    <fct:column>CNET News.com</fct:column>
    <fct:column><span class="srch_xerpt">Tech news and business reports by <b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com">http://news.cnet.com</fct:column>
    <fct:column>Technology News - CNET News</fct:column>
    <fct:column><span class="srch_xerpt">Tech news and business reports by <b>CNET</b> News.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.com#6">http://news.com#6</fct:column>
    <fct:column>There's an electric car in your future</fct:column>
    <fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml#9">http://news.cnet.com/2547-1_3-0-20.xml#9</fct:column>
    <fct:column>There's an electric car in your future</fct:column>
    <fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com#9">http://news.cnet.com#9</fct:column>
    <fct:column>There's an electric car in your future</fct:column>
    <fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.com#6">http://news.com#6</fct:column>
    <fct:column>There's an electric car in your future</fct:column>
    <fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com/2547-1_3-0-20.xml#9">http://news.cnet.com/2547-1_3-0-20.xml#9</fct:column>
    <fct:column>There's an electric car in your future</fct:column>
    <fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="trank">3</fct:column>
    <fct:column datatype="erank">5.881291583872905e-014</fct:column>
    <fct:column datatype="url" shortform="http://news.cnet.com#9">http://news.cnet.com#9</fct:column>
    <fct:column>There's an electric car in your future</fct:column>
    <fct:column><span class="srch_xerpt">... <b>CNET</b> Car Tech posts photos of electric cars expected to come out by 2011.</span></fct:column>
  </fct:row>
 </fct:result>
</fct:facets>
]]></programlisting>
</listitem>
  <listitem>Click "New search" from the Entity Relations Navigation and go to "Entity Label Lookup" tab: 
    <figure id="fctinst10" float="1">
      <title>Query Faceted Browser Web service endpoint</title>
      <graphic fileref="ui/fb11.png"/>
    </figure>
</listitem>
  <listitem>In the Label auto-complete text box of the Entity Label Lookup tab, enter the name of an rdfs label to be Described: 
    <figure id="fctinst11" float="1">
      <title>Select a URI from the list of available Labels</title>
      <graphic fileref="ui/fb12.png"/>
    </figure>
</listitem>
  <listitem>Select a URI from the list of available Labels to obtain a description of the URI:
    <figure id="fctinst11" float="1">
      <title>Select a URI from the list of available Labels</title>
      <graphic fileref="ui/fb13.png"/>
    </figure>
      <figure id="fctinst11" float="1">
      <title>Select a URI from the list of available Labels</title>
      <graphic fileref="ui/fb14.png"/>
    </figure>
  </listitem>
  <listitem>Click "Facets" and go to "Entity URI Lookup" tab:
    <figure id="fctinst12" float="1">
      <title>Enter URI</title>
      <graphic fileref="ui/fb15.png"/>
    </figure>
</listitem>
  <listitem>In the URI auto-complete text box of the Entity URI Lookup tab enter the name URI to be Described: 
    <figure id="fctinst12" float="1">
      <title>Enter URI</title>
      <graphic fileref="ui/fb16.png"/>
    </figure>
</listitem>
  <listitem>Select a URI from the list of available Labels to obtain a description of the URI:
    <figure id="fctinst13" float="1">
      <title>Obtain a description of the URI</title>
      <graphic fileref="ui/fb17.png"/>
    </figure>
        <figure id="fctinst13" float="1">
      <title>Obtain a description of the URI</title>
      <graphic fileref="ui/fb18.png"/>
    </figure>
</listitem>
  <listitem>If data is loaded into the quad store via DML functions (TTLP, RDF_LOAD_RDFXML etc.) the
following procedure needs to run from <emphasis>isql</emphasis> to build the free text indexes required each time:
<programlisting><![CDATA[
VT_INC_INDEX_DB_DBA_RDF_OBJ ()
]]></programlisting>
</listitem>
        </orderedlist>
</sect2>
<sect2 id="virtuosospongerfaceurilabels">
	<title>URI Labels</title>

	<orderedlist>
   <listitem>Go to http://cname/fct</listitem>
   <listitem>Enter a free text search pattern (for example, "Camcorder" as consumer product), and click Search: 
    <figure id="fctinst13" float="1">
      <title>URI Labels</title>
      <graphic fileref="ui/fb19.png"/>
    </figure>   	
   	</listitem>
   <listitem>Your initial query results page will display a list of literal value snippets where for each URL will be displayed a label: 
    <figure id="fctinst13" float="1">
      <title>URI Labels</title>
      <graphic fileref="ui/fb20.png"/>
    </figure>   	
   	</listitem>
   <listitem>Click for ex. on the URL link of the first row result.</listitem>
   <listitem>The product description page should be shown and a list of Attributes and Values will be presented. An URL label of the product also will be shown: "Charges Lithium Ion 800 series batteries": 
    <figure id="fctinst13" float="1">
      <title>URI Labels</title>
      <graphic fileref="ui/fb21.png"/>
    </figure>   	
    <figure id="fctinst13" float="1">
      <title>URI Labels</title>
      <graphic fileref="ui/fb22.png"/>
    </figure>   	
    <figure id="fctinst13" float="1">
      <title>URI Labels</title>
      <graphic fileref="ui/fb23.png"/>
    </figure>   	
   </listitem>
  </orderedlist>   	
</sect2>
<sect2 id="virtuosospongerfaceusagest">
<title>Usage Statistics</title>
<orderedlist>
   <listitem>Use the Faceted Browser Search and Find User Interface to search for information on "Michael Jackson":
     <figure id="VirtFacetUsage1" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb24.png"/>
     </figure>
   </listitem>
   <listitem>Results of the following form should be returned for the network resource data being fetched.
     <figure id="VirtFacetUsage2" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb25.png"/>
     </figure>
   </listitem>
   <listitem>Click the "Types" link under "Entity Relations Navigation". 
   </listitem>
   <listitem>Results about "Michael Jackson" as Type/Label/Count list should be displayed: 
     <figure id="VirtFacetUsage2" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb26.png"/>
     </figure>
   </listitem>
   <listitem>You can navigate amongst the search results pages by using the "Prev" and "Next" buttons. Click for ex. "Next": 
     <figure id="VirtFacetUsage2" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb27.png"/>
     </figure>
   </listitem>
   <listitem>Click a type link, for ex.:
<programlisting><![CDATA[
http://dbpedia.org/class/yago/Artist109812338
]]></programlisting>
   </listitem>
   <listitem>Should be shown type results and:
<programlisting><![CDATA[
Displaying Ranked Entity Names and Text summaries where:

Entity1 has any Attribute with Value "Michael Jackson" Drop.
Entity1 is a yago:Artist109812338 . Drop
]]></programlisting>
     <figure id="VirtFacetUsage9" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb28.png"/>
     </figure>
   </listitem>
   <listitem>Click the link:
<programlisting><![CDATA[
dbpedia:Michael_Jackson
]]></programlisting>
   </listitem>
   <listitem>Results about "Michael Jackson" as Attribute/Value list should be presented: 
     <figure id="VirtFacetUsage9" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb29.png"/>
     </figure>
     <figure id="VirtFacetUsage9" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb30.png"/>
     </figure>   	
     <figure id="VirtFacetUsage9" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb31.png"/>
     </figure>   	
     <figure id="VirtFacetUsage9" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb32.png"/>
     </figure>
   </listitem>
   <listitem>You can navigate amongst the search results pages by using the "First", "Prev", "Next" and "Last" buttons. Click for ex. "Last": 
     <figure id="VirtFacetUsage10" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb33.png"/>
     </figure>
     <figure id="VirtFacetUsage10" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb34.png"/>
     </figure>     
   </listitem>
   <listitem>"Metadata" tab.</listitem>
   <listitem>Results of usage statistics for "Michael Jackson" grouped in 4 tabs should be shown:
<orderedlist>
   <listitem>Referenced by Graphs: shows how many times the URI is found as subject in the relevant graph(s):
<programlisting><![CDATA[
SPARQL
SELECT ?g count (*)
where
      {
    graph ?g { <URI> ?p ?o }
  }
group by ?g
order by desc 2
limit 20
]]></programlisting>
     <figure id="VirtFacetUsage4" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb35.png"/>
     </figure>
   </listitem>
   <listitem>Source Graphs:  shows how many times the URI is found as object in the relevant graph(s):
<programlisting><![CDATA[
SPARQL
SELECT ?g count (*)
where
      {
    graph ?g { ?s ?p <URI>  }
      }
group by ?g
order by desc 2
limit 20
]]></programlisting>
     <figure id="VirtFacetUsage5" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb36.png"/>
     </figure>
   </listitem>
   <listitem>Direct co-references: shows results as subject and calculated rank, based on running transitive closure over owl:sameAs of the URI in subject or object:
<programlisting><![CDATA[
SPARQL
SELECT ?syn ( sql:rnk_scale (<LONG::IRI_RANK> (?syn)))
where
      {
    { SELECT ?s ?syn
      where
       {
         {?syn owl:sameAs ?s } union {?s owl:sameAs ?syn}
    }
  }
    option (transitive, t_distinct, t_min (0), T_in (?s), t_out (?syn)) . filter (!isliteral (?syn) &amp;&amp; ?s = <URI> )
}
order by desc 2
limit 20
]]></programlisting>
     <figure id="VirtFacetUsage6" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb37.png"/>
     </figure>
   </listitem>
   <listitem>Indirect co-references: shows expanded results for objects concur with the URI by IFP:
<programlisting><![CDATA[
SPARQL
SELECT distinct ?syn ?p ?o (sql:rnk_scale (<LONG::IRI_RANK> (?syn)))
where
  { <URI> ?p ?o .  filter (0 != (<LONG::bif:rdf_is_sub> ("b3sifp", ?p, lod:ifp_like, 3))) .
    ?syn ?p ?o .
}
order by desc 4
limit 20
]]></programlisting>
     <figure id="VirtFacetUsage7" float="1">
       <title>Usage Statistics</title>
       <graphic fileref="ui/fb38.png"/>
     </figure>
   </listitem>
</orderedlist>
   </listitem>
</orderedlist>
</sect2>
<sect2 id="virtuosospongerfacetexample"><title>Examples</title>
<para><emphasis>Faceted Browsing Sample using LOD Cloud Cache data space</emphasis></para>
<para>The following example demonstrates a simple scenario of tracking Kingsley Idehen's conversations 
across the Web, using the Virtuoso Faceted Browser hosted on LOD.</para>
<orderedlist>
   <listitem>Go to http://lod.openlinksw.com/fct/
     <figure id="fb2" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/fb2.png"/>
     </figure>
   </listitem>
   <listitem>Enter a free text search pattern (for example, "Kingsley Idehen"), and click Search
     <figure id="f1" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f1.png"/>
     </figure>
   </listitem>
   <listitem>Your initial query results page will display a list of literal value snippets from property
values associated with the query text pattern
     <figure id="f2" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f2.png"/>
     </figure>
   </listitem>
   <listitem>Using the Navigation section on the right, click on "Types", which alters the contents
of the query results area by presenting CURIE based hyperlinks for each of the Entity Types associated
with Property values that contains the query text pattern
     <figure id="f3" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f3.png"/>
     </figure>
   </listitem>
   <listitem>You can perform <b>Describe</b> for a given found type, by clicking the "Describe" link in the "Type" column. For ex, for "atom:Entry" the produced describe type page would show a list of Attributes and Values + automatically generated QRCode image:
     <figure id="f4" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f4.png"/>
     </figure>
   </listitem>
   <listitem>Click "Facets" tab to return to the Types content page from the previous step.</listitem>
   <listitem>Click on the "foaf:Person" link to narrow the result set down to Entities of this Type,
un-hatch the checkbox beside this link for Negation (filtering out) based on this Entity Type
     <figure id="f7" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f7.png"/>
     </figure>
     <itemizedlist mark="bullet">
     	 <listitem>For Negation (filtering out) based on this Entity Type un-hatch the check-box shown besides the link:
         <figure id="f5" float="1">
           <title>Faceted Navigation Example</title>
           <graphic fileref="ui/f5.png"/>
         </figure>
         <figure id="f6" float="1">
       <title>Faceted Navigation Example</title>
           <graphic fileref="ui/f6.png"/>
     </figure>
   </listitem>
   <listitem>You can filter further, by switching (pivoting) to the a Property based view, by returning
to the Navigation section and then clicking on "Properties" or "Referencing Properties" links; in either
case, you have further filtering of based on the combination of Properties and Entities where Entities
in the result-set contain values matching the query text pattern
         <figure id="f8" float="1">
           <title>Faceted Navigation Example</title>
           <graphic fileref="ui/f8.png"/>
         </figure>
       </listitem>
       <listitem></listitem>
     </itemizedlist>
   </listitem>
   <listitem>From "Entity Relations Navigation" click "Attributes".
     <figure id="f9" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f9.png"/>
     </figure>   	
   </listitem>
   <listitem>You can navigate amongst the search results pages by using the "Prev" and "Next" buttons. Click for ex. "Next": 
     <figure id="f10" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f10.png"/>
     </figure>
   </listitem>
   <listitem>From the list of Property Types, click on the "foaf:interest" link to filter further,
based on the values of this property:
     <figure id="f11" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f11.png"/>
     </figure>
   </listitem>
   <listitem>From the list of "foaf:interest" Values, click on "About:Linked Data", which filters
the result-set further to display reveal Entity Identifier Links (Generic HTTP URIs) and Labels for
each "foaf:Person" associated with the property "foaf:interest", in the LOD data space:
     <figure id="f12" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f12.png"/>
     </figure>
   </listitem>
   <listitem>Click on one of the HTTP URIs in the filtered results-set to obtain a detailed structured
   	description of a given Entity i.e. about the person Kingsley Uyi Idehen. Each listed Property is 
   	a <b>Link</b>; thus, each Property is a link to other structured Entity descriptions. Additionally, 
   	a QRCode image will be produced automatically for the given entity:
     <figure id="f13" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f13.png"/>
     </figure>
     <figure id="f14" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f14.png"/>
     </figure>
     <figure id="f15" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f15.png"/>
     </figure>  
     <figure id="f16" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f16.png"/>
     </figure>
   </listitem>
   <listitem>You can navigate amongst the search results pages by using the "First", "Prev", "Next" and "Last" buttons. Click for ex. "Last":
     <figure id="f17" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f17.png"/>
     </figure>
     <figure id="f18" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f18.png"/>
     </figure>
   </listitem>
   <listitem>Click on "<b>Metadata</b>" link to get a summary view of this Linked Data Space, "Source" 
   and "Reference" graphs are akin to saying "Table X" and "Table Y" where each table is the container 
   of Records re. RDBMS or Worksheet re. Spreadsheet.:
     <figure id="f19" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f19.png"/>
     </figure>
     <figure id="f20" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f20.png"/>
     </figure>
   </listitem>
   <listitem>"Direct" and "InDirect" coreferences show other references (Identifiers) that relate 
   	associated with Kingsley Idehen (like saying: here are his other names or his know by this name 
   	in this other place):
     <figure id="fct12" float="1">
       <title>f21 Navigation Example</title>
       <graphic fileref="ui/f21.png"/>
     </figure>
     <figure id="f22" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f22.png"/>
     </figure>
   </listitem>
   <listitem>Click on "Settings" check "owl:sameAs" and it sets a context mode for the session
(meaning: a set of rules to take place):
     <figure id="f23" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f23.png"/>
     </figure>
   </listitem>
   <listitem>Go back to the "Direct Co-reference" tab:
     <figure id="f24" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/f24.png"/>
     </figure>
   </listitem>
   <listitem>As result each link will unveil a union (combination) of all the the data associated
with all Kingsley Idehen's other Identifiers (other Names in other places), i.e., they all show the
same data.</listitem>
   <listitem>Go to "Facets" and then from "Entity Relations Navigation" click "New Search".</listitem>
   <listitem>Enter a free text search pattern  (for example, "<code>Camcorder</code>" as consumer product), 
   and click Search:
     <figure id="fb19" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/fb19.png"/>
     </figure>   
   </listitem>
   <listitem>Your initial query results page will display a list of literal value snippets where for 
   	each URI will be displayed a label:
     <figure id="fb20" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/fb20.png"/>
     </figure>   
   </listitem>   
   <listitem>Click for ex. on the URL link ofthe first row result.</listitem>
   <listitem>The product description page should be shown and a list of Attributes and Values will be 
   	presented. An URI label of the product also will be shown: "CG-800":
     <figure id="fb21" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/fb21.png"/>
     </figure> 
     <figure id="fb22" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/fb22.png"/>
     </figure>   
     <figure id="fb23" float="1">
       <title>Faceted Navigation Example</title>
       <graphic fileref="ui/fb23.png"/>
     </figure>                 
   </listitem>   
</orderedlist>
</sect2>
</sect1>
<sect1 id="virtuosospongerfacent">
<title>Virtuoso Faceted Web Service</title>
<para>The Virtuoso Faceted web service is a general purpose RDF query facility for facet based browsing.
It takes an XML description of the view desired and generates the reply as an XML tree containing the
requested data. The user agent or a local web page can use XSLT for rendering this for the end user.
The selection of facets and values is represented as an XML tree. The rationale for this is the fact
that such a representation is easier to process in an application than the SPARQL source text or a
parse tree of SPARQL and more compactly captures the specific subset of SPARQL needed for faceted
browsing. The web service returns the SPARQL source text also, thus this can serve as a basis for
and-crafted queries.</para>
<para>The top element of the tree is &lt;query&gt;, it must be in namespace
"http://openlinksw.com/services/facets/1.0/".</para>
<para>This has the following attributes:</para>
<itemizedlist mark="bullet">
  <listitem>graph="graph_iri" - default is search in all graphs but system defaults may override this</listitem>
  <listitem>timeout="no_of_msec" - default is no timeout, but system defaults may override this</listitem>
  <listitem>inference="name" where name is a name of an inference context declared with rdfs_rule_set.</listitem>
  <listitem>same-as="boolean" - If "boolean" is "yes", then owl:sameAs links will be considered in the query evaluation.</listitem>
</itemizedlist>
<para>The result is a tree of the form:</para>
<programlisting><![CDATA[
<facets xmlns="http://openlinksw.com/services/facets/1.0/">
<result><row><column datatype="..." shortform="..." xml:lang="..">...</column></row></result>
<time>msecs</time>
<complete>yes or no</complete>
<db-activity>resource use string</db-activity>
<sparql>sparql statement text</sparql>
</facets>
]]></programlisting>
<para>By convention, the first column is the subject selected by the view element, typically a URI, the second a label of the URI
and the third, if present, is either a count or a search summary.</para>
<para>The first column's text child is the text form of the value. The column element has the following attributes
qualifying this further:</para>
        <itemizedlist mark="bullet">
  <listitem>datatype - The xsd type of the value. If this is a URI, the datatype is "uri" </listitem>
  <listitem>shortform - If the value is a URI, this is an abbreviated form where known namespaces are replaced with
their prefixes and very long URI's are truncated preserving start and end. </listitem>
  <listitem>xml:lang - if the value is a language tagged string, this is the language</listitem>
        </itemizedlist>
<para>The query has the top level element &lt;query&gt;. The child elements of this represent conditions
pertaining to a single subject. A join is expressed with the property or property-of element. This has
in turn children which state conditions on a property of the first subject. property and property-of
elements can be nested to an arbitrary depth and many can occur inside one containing element. In this way,
tree-shaped structures of joins can be expressed.</para>
<para>Expressing more complex relationships, such as intermediate grouping, subqueries, arithmetic or
such requires writing the query in SPARQL. The XML format is a shorthand for easy automatic composition
of queries needed for showing facets, not a replacement for SPARQL.</para>
<para>A facet query contains a single view element. This specifies which subject of the joined
subjects is shown. Its attributes specify the manner of viewing, e.g. list of distinct values, distinct
values with occurrence counts, properties or classes of the selected subjects etc.</para>
<para>The top query element or any property or property-of element can have the following types of children:</para>
<programlisting><![CDATA[
<text property="iri">text pattern</text>
]]></programlisting>
<para>The subject has an O that matches the text pattern. If property is given, the text pattern must
occur in a value of this property. If not specified, any property will do. The value "none" for property
is the same as not specifying a property. This is restricted to occurring directly under the top level
query element.</para>
<programlisting><![CDATA[
<class iri="iri" inference="ctx_name" />
]]></programlisting>
<para>The S must be an instance of this class. If inference is specified then option (input:inference
"ctx_name" is added and applies to this pattern alone.</para>
<programlisting><![CDATA[
<property iri="iri" same_as="yes" inference="ctx_name">]]></programlisting>
<para>The child elements of this are conditions that apply to the value of this property of the S that
is in scope in the enclosing &lt;query&gt; or &lt;property&gt; element. If same_as is present, then
option (input:same-as "yes") is added to the triple pattern which specifies this property. If inference
is present, then option (input:inference "ctx_name") is added to the triple pattern for the property.</para>
<programlisting><![CDATA[
<property-of iri="iri" same_as="yes" inference="ctx_name" >
]]></programlisting>
<para>The child elements of this are conditions that apply to an S which has property "iri" whose object
is the S in scope in the enclosing &lt;query&gt; or &lt;property&gt; element. The options are otherwise
the same as with property.</para>
<programlisting><![CDATA[
<value datatype="type" xml:lang="lng" op="= | < | > | >= | <=">value </value>
]]></programlisting>
<para>When this occurs inside &lt;property&gt; or &lt;property-of&gt; this means that the property in
scope has the specified relation to the value. type and language can be used for XML typed or language
tagged literals. The "uri" type means that the value is a qualified name of a URI. If this occurs
directly under the &lt;query&gt; element, this means that the query starts with a fixed subject.
If this is so, then there must be property or propertyof elements or the view element must specify
properties or classes, list is not allowed as a view type. This is so because the query must have
at least one triple pattern.</para>
<programlisting><![CDATA[
<view type="view" limit="n" offset="n" >
]]></programlisting>
<para>This may occur once inside a &lt;query&gt; element but may occur either at top level or inside
property or property-of elements. This specifies what which subject is presented in the result set.</para>
<para>The type can be:</para>
    <itemizedlist mark="bullet">
  <listitem>"properties"
<programlisting><![CDATA[
SPARQL
SELECT ?p count (*) { ?this_s ?p ?any_o ...}
GROUP BY ?p
ORDER BY DESC 2
LIMIT l OFFSET 0
]]></programlisting>
  </listitem>
  <listitem>"properties-in"
<programlisting><![CDATA[
SPARQL
SELECT ?p count (*) { ?any_s ?p ?this_s ... }
GROUP BY ?p
ORDER BY DESC 2
LIMIT L OFFSET 0
]]></programlisting>
  </listitem>
  <listitem>"classes"
<programlisting><![CDATA[
SPARQL
SELECT ?c count (*)
WHERE { ?xx a ?c ... }
GROUP BY ?c
ORDER BY DESC 2
LIMIT l OFFSET 0
]]></programlisting>
  </listitem>
  <listitem>"text"
<programlisting><![CDATA[
SPARQL
SELECT DISTINCT ?s (bif:search_excerpt (sql:search_terms (""pattern"), ?o)) ...
LIMIT l OFFSET 0
]]></programlisting>
  </listitem>
  <listitem>"list"
<programlisting><![CDATA[
SPARQL
SELECT DISTINCT ?s long::sql:fct_label (?s) ...
LIMIT l OFFSET 0
]]></programlisting>
  </listitem>
  <listitem></listitem>
  <listitem>"list-count"
<programlisting><![CDATA[
SPARQL
SELECT ?s COUNT (*) ....
GROUP BY ?s
ORDER BY DESC 2
]]></programlisting>
  </listitem>
  <listitem>"alphabet"
<programlisting><![CDATA[
SPARQL
SELECT (sql:subseq (?s, 0, 1)) count (*) ...
GROUP BY (sql:subseq (?s, 0, 1))
ORDER BY 1
]]></programlisting>
  </listitem>
  <listitem>"geo"
<programlisting><![CDATA[
SPARQL
SELECT DISTINCT ?lat ?long ?s
WHERE ?s geo:lat ?lat . ?s geo:long ?long . ... }
]]></programlisting>
  </listitem>
  <listitem>"years"
<programlisting><![CDATA[
SPARQL
SELECT sql::year (?s) count (*) ...
GROUP BY (bif:year (?s))
ORDER BY 1
OFFSET 0 LIMIT l
]]></programlisting>
  </listitem>
  <listitem>"months"
<programlisting><![CDATA[
SPARQL
SELECT sql::round_month (?s) count (*) ...
GROUP BY (sql:round_month (?s))
ORDER BY 1 OFFSET 0 LIMIT l
]]></programlisting>
  </listitem>
  <listitem>"weeks"
<programlisting><![CDATA[
SPARQL
SELECT sql::round_week (?s) COUNT (*) ...
GROUP BY (sql:round_week (?s))
ORDER BY 1 OFFSET 0 LIMIT l
]]></programlisting>
  </listitem>
  <listitem>"describe"
<programlisting><![CDATA[
SPARQL describe ?s ... OFFSET 0 LIMIT l
]]></programlisting>
  </listitem>
    </itemizedlist>
<sect2 id="virtuosospongerfacentcust">
  <title>Customizing</title>
  <para>The following types of customization will be generally useful:</para>
  <itemizedlist mark="bullet">
    <listitem>Resource accounting and limitations, managing access and login</listitem>
    <listitem>Localization, choice of labels shown with class/property/instance URI's</listitem>
    <listitem>Adding types of views, for example timelines, map or business graphics </listitem>
    <listitem>Controlling navigation, for example choosing what type of view is initially presented when opening a given property.</listitem>
    <listitem>Page layout, captions, help texts, etc.</listitem>
  </itemizedlist>
  <para>The source code is divided in two SQL files and a number of XSLT sheets. The file facet.sql has the code for the web service. The
facet_view.sql file contains the procedures for the sample HTML interface.</para>
</sect2>
<sect2 id="virtuosospongerfacentexamples">
  <title>Examples</title>
<para>Note: in all examples the default namespace xmlns="http://openlinksw.com/services/facets/1.0/" is omitted for brevity.</para>
<para>For people called Mike:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <view type="text"/>
</query>
]]></programlisting>
<para>To open the list of people who Mike knows:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <view type="properties"/>
</query>
]]></programlisting>
<para>To show the list of subjects Mike knows:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <view type="list" />
  </property>
</query>
]]></programlisting>
<para>To show the properties of people Mike knows:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <view type="properties" />
  </property>
</query>
]]></programlisting>
<para>To show the names:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <view type="list" />
    </property>
  </property>
</query>
]]></programlisting>
<para>To specify one named Joe:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
     <property iri="foaf:name>
        <value>Joe</value>
     </property>
    <view type="properties" />
  </property>
</query>
]]></programlisting>
<para>This lists the properties of the friends of Mike that are called Joe.</para>
<para>To show the Mikes that know a Joe, one would change the shown variable in the navigation and get:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
   </property>
   <view type="text" />
</query>
]]></programlisting>
<para>This would be the search summaries of subjects with Mike in some field that know a subject with name Joe.</para>
<para>Now to specify that Mike must be a member of a discussion board:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
     <value>Joe</value>
   </property>
  </property>
  <view type="property-in" />
</query>
]]></programlisting>
<para>This lists the properties of triples whom object is Mike. Pick sioc:member_of</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
  </property>
  <property-of iri="sioc:member_of>
    <view type="list" />
  </property-of>
</query>
]]></programlisting>
<para>This would show things where Mike is a member. To specify that the thing must be a forum:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
  </property>
  <property-of iri="sioc:member_of>
    <view type="classes" />
  </property-of>
</query>
]]></programlisting>
<para>This shows classes of things where Mike is a member Clicking on sioc:Forum gives:</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
  </property>
  <property-of iri="sioc:member_of>
    <class iri="sioc:Forum" />
    <view type="classes"/>
  </property-of>
</query>
]]></programlisting>
<para>The view stays with classes, but now scoped
to the classes of things where Mike is a member that are instances of sioc:Forum.</para>
<para>To go look at the list of Mikes with the added
restriction, click the shown variable in the navigation and set it to s1.</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
  </property>
  <property-of iri="sioc:member_of>
   <class iri="sioc:Forum" />
  </property-of>
  <view type="list"/>
</query>
]]></programlisting>
<para>To say that Joe must also have a geekCode, One clicks the shown variable and sets it to s2 and the view to properties.</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
    <view type="properties"/>
  </property>
  <property-of iri="sioc:member_of>
    <class iri="sioc:Forum" />
   </property-of>
</query>
]]></programlisting>
<para>Pick geekCode</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
    <property iri="geekCode">
      <view type="list"/>
    </property>
  </property>
  <property-of iri="sioc:member_of>
    <class iri="sioc:Forum" />
  </property-of>
</query>
]]></programlisting>
<para>We specify no restriction on the geekCode. Click the shown variable to take the focus back to Mike.</para>
<programlisting><![CDATA[
<query>
  <text>Mike</text>
  <property iri="foaf:knows>
    <property iri="foaf:name>
      <value>Joe</value>
    </property>
    <property iri="geekCode"></property>
  </property>
  <property-of iri="sioc:member_of>
    <class iri="sioc:Forum" />
  </property-of>
  <view type="text"/>
</query>
]]></programlisting>
</sect2>
<sect2 id="virtuosospongerfacentui">
  <title>WebService Interface</title>
<sect3 id="virtuosospongerfacentuirest">
  <title>REST interface</title>
  <para>The Virtuoso Faceted web service provide following REST interface:</para>
  <para>Service description:</para>
      <itemizedlist mark="bullet">
  <listitem>Endpoint: http://&lt;cname&gt;/fct/service for ex. http://lod.openlinksw.com/fct/service </listitem>
  <listitem>HTTP method: POST</listitem>
  <listitem>Content-Type: MUST be 'text/xml'</listitem>
  <listitem>The entity body must be XML document with top element 'query' as described above.</listitem>
  <listitem>The request response namespace MUST be "http://openlinksw.com/services/facets/1.0"</listitem>
      </itemizedlist>
  <para>Error conditions:</para>
  <para>The all error conditions are reported via 'Error explanation'</para>
  <para>Files:</para>
  <para>The facet_svc.sql contains web service code and virtual directory mapping, and it uses
fct_req.xsl &amp; fct_resp.xsl as request &amp; response filters.</para>
  <para>Example:</para>
  <para>Using CURL program</para>
<programlisting><![CDATA[
curl -H "Content-Type: text/xml" -d @post.xml  http://lod.openlinksw.com/fct/service
]]></programlisting>
  <para>Where 'post.xml' document contains query document:</para>
<programlisting><![CDATA[
<?xml version="1.0"?>
<query xmlns="http://openlinksw.com/services/facets/1.0" inference="" same-as="">
  <text> Seattle Mariners traveled all the way to Japan to watch</text>
  <view type="text" limit="20" offset=""/>
</query>
]]></programlisting>
  <para>Produces following response:</para>
<programlisting><![CDATA[
<fct:facets xmlns:fct="http://openlinksw.com/services/facets/1.0/">
<fct:sparql>   SELECT distinct ?s1 as ?c1, (bif:search_excerpt (bif:vector ('THE', 'MARINERS', 'WAY', 'SEATTLE', 'WATCH', 'ALL', 'TO', 'JAPAN', 'TRAVELED'), ?o1)) as ?c2  WHERE { ?s1 ?s1textp ?o1 . FILTER (bif:contains (?o1, '(THE AND MARINERS AND WAY AND SEATTLE AND WATCH AND ALL AND TO AND JAPAN AND TRAVELED)')) . } LIMIT 20  OFFSET 0 </fct:sparql>
<fct:time>116</fct:time>
<fct:complete>yes</fct:complete>
<fct:db-activity>   134R rnd  9.488KR seq      0P disk  8.966MB /    602 messages</fct:db-activity>
 <fct:result>
  <fct:row>
    <fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html></fct:column>
    <fct:column />
    <fct:column><span class="srch_xerpt">... While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b> Ichiro... for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without a doubt... leading <b>the</b> Dominican <b>to</b> its...</span></fct:column>
  </fct:row>
  <fct:row>
    <fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html></fct:column>
    <fct:column />
    <fct:column><span class="srch_xerpt">Orlando While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b>... perform for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without...</span></fct:column>
  </fct:row>
 </fct:result>
</fct:facets>
]]></programlisting>
</sect3>
<sect3 id="virtuosospongerfacentuirestapi">
  <title>Virtuoso APIs for Faceted REST services</title>
<para>The Virtuoso APIs for FCT REST services are Virtuoso Stored Procedures that enable faceted browsing
over Linked Data hosted in the RDF Quad Store. This also includes Linked Data that is progressively
added to the Quad Store via URI de-referencing.
</para>
<para>They enable the use Virtuoso's VSP/VSPX technology to produce (X)HTML-based Linked Data explorer
pages that are endowed with high-performance (in-process) faceted browsing capability.
</para>
<para>You can use this API with Virtuoso SQL calls that provide data to your VSP/VSPX, ASP.NET, PHP,
etc., -based interfaces using ODBC, JDBC, ADO.NET, or XMLA connectivity (SPASQL) to Virtuoso.
</para>
  <sect4 id="virtuosospongerfacentuirestapidef">
    <title>API Definition</title>
<programlisting><![CDATA[
CREATE PROCEDURE
fct_exec
  (
    IN  tree     ANY ,
    IN  timeout  INT
  )
{
  DECLARE  start_time,
           view3,
           inx,
           n_rows      INT     ;
  DECLARE  sqls,
           msg,
           qr,
           qr2,
           act,
           query       VARCHAR ;
  DECLARE  md,
           res,
           results,
           more        ANY     ;
  DECLARE  tmp         ANY     ;
  DECLARE  offs,
           lim         INT     ;

  SET result_timeout = _min
                         (
                           timeout,
                           ATOI
                             (
                               registry_get ('fct_timeout_max')
                             )
                         )
  ;

  offs := xpath_eval ('//query/view/@offset', tree);
  lim := xpath_eval ('//query/view/@limit', tree);

  -- db_activity ();

  results := vector (null, null, null);
  more := vector ();

  IF
    (
      xpath_eval
        (
          '//query[@view3="yes"]//view[@type="text"]',
          tree
        )
      IS NOT NULL
    )
      {
      more := VECTOR ('classes', 'properties');
  }

  sqls := '00000';
  qr := fct_query
          (
            xpath_eval ('//query', tree, 1)
          )
  ;
  query := qr;
--  dbg_obj_print (qr);
  qr2 := fct_xml_wrap (tree, qr);
  start_time := msec_time ();

  dbg_printf('query: %s', qr2);

  EXEC
    (
      qr2,
      sqls,
      msg,
      vector (),
      0,
      md,
      res
    )
  ;
  n_rows := row_count ();
  act := db_activity ();
  SET result_timeout = 0;
  IF (
       sqls <> '00000'
       AND
       sqls <> 'S1TAT'
     )
    SIGNAL (sqls, msg);
  IF (
       NOT ISARRAY (res)
       OR
       0 = length (res)
       OR
       NOT ISARRAY (res[0])
       OR
       0 = length (res[0])
     )
    results[0] := xtree_doc ('<result/>');
  ELSE
    results[0] := res[0][0];

  inx := 1;

  FOREACH (VARCHAR tp IN more) DO
    {
      tree := XMLUpdate (
                          tree,
                          '/query/view/@type',
                          tp,
                          '/query/view/@limit',
                          '40',
                          '/query/view/@offset',
                          '0'
                        )
      ;
      qr := fct_query (xpath_eval ('//query', tree, 1));
      qr2 := fct_xml_wrap (tree, qr);
      sqls := '00000';
      SET result_timeout = _min (
                                  timeout,
                                  ATOI
                                    (
                                      registry_get ('fct_timeout_max')
                                    )
                                )
      ;
      EXEC (
             qr2,
             sqls,
             msg,
             vector (),
             0,
             md,
             res
           );
      n_rows := row_count ();
      act := db_activity ();
      SET result_timeout = 0;
      IF ( sqls <> '00000'
           AND
           sqls <> 'S1TAT'
         )
    SIGNAL (sqls, msg);
      IF (
           ISARRAY (res)
           AND
           LENGTH (res)
           AND
           ISARRAY (res[0])
           AND
           LENGTH (res[0])
         )
    {
      tmp := res[0][0];
      tmp := XMLUpdate (tmp, '/result/@type', tp);
      results[inx] := tmp;
    }
      inx := inx + 1;
    }



  res := XMLELEMENT
           (
             "facets",
             XMLELEMENT
               ( "sparql", query ),
             XMLELEMENT
               ( "time", msec_time () - start_time ),
             XMLELEMENT
               (
                 "complete",
                 CASE WHEN sqls = 'S1TAT'
                      THEN 'no'
                      ELSE 'yes'
                  END
                ),
             XMLELEMENT
               (
                 "timeout",
                 _min
                   (
                     timeout * 2,
                     ATOI
                       (
                         registry_get
                           ( 'fct_timeout_max' )
                       )
                   )
               ),
             XMLELEMENT
               ("db-activity", act),
             XMLELEMENT
               ("processed", n_rows),
             XMLELEMENT
               (
                 "view",
                 XMLATTRIBUTES
                   (
                     offs AS "offset",
                     lim AS "limit"
                   )
               ),
             results[0],
             results[1],
             results[2]
           );

---- for debugging:
--string_to_file ('ret.xml', serialize_to_UTF8_xml (res), -2);
--  dbg_obj_print (res);

  RETURN res;
}
;
]]></programlisting>
   </sect4>
   <sect4 id="virtuosospongerfacentuirestapiexmp">
     <title>Example</title>
<para>
The following example shows how to use the fct_exec APi in vsp page to perform a "text" search for the
word "Mike" assuming this exists in your Virtuoso RDF store (if not amend the query in the fct_example.vsp
code sample below to search for text known to exist).
</para>
<orderedlist>
  <listitem>The service can be used in the following sample fct_example.vsp:
<programlisting><![CDATA[
<?vsp

declare txt, reply, tree any;
declare timeout int;

tree := xtree_doc ('
  <query>
    <text>Mike</text>
    <view type="text"/>
  </query>
');

timeout := 3000;
reply := fct_exec (tree, timeout);

txt := string_output ();

http_value (xslt ('virt://WS.WS.SYS_DAV_RES.RES_FULL_PATH.RES_CONTENT:/DAV/fct_example.xsl',
                 reply,
   	         vector ()),
	         null, txt);

http (txt);
?>
]]></programlisting>
</listitem>
  <listitem>The xsl:
<programlisting><![CDATA[
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="html" encoding="ISO-8859-1"/>
  <xsl:variable name="rowcnt" select="count(/facets/result/row)"/>
  <xsl:template match="facets">
    <div id="res">
      <xsl:if test="/facets/complete = 'yes' and /facets/processed = 0 and $rowcnt = 0">
        <div class="empty_result">
          Nothing found.
        </div>
      </xsl:if>
      <xsl:for-each select="/facets/result">
        <xsl:call-template name="render-result"/>
      </xsl:for-each>
    </div>
    <!-- #res -->
    </xsl:template>
  <xsl:template name="render-result">
    <table class="result" border="1">
      <thead>
        <tr>
          <th>Entity</th>
          <th>Title</th>
          <th>Text excerpt</th>
        </tr>
      </thead>
      <tbody>
        <xsl:for-each select="row">
          <tr>
            <td class="rnk">
              <xsl:for-each select="column[@datatype='trank' or @datatype='erank']">
		<xsl:choose>
                  <xsl:when test="./@datatype='trank'">Text Rank:</xsl:when>
                  <xsl:when test="./@datatype='erank'">Entity Rank:</xsl:when>
                </xsl:choose>
                <xsl:value-of select="."/>
                <br/>
              </xsl:for-each>
            </td>
            <xsl:for-each select="column">
              <xsl:choose>
                <xsl:when test="'url' = ./@datatype">
                  <td>
                    <a>
                      <xsl:attribute name="href">http://lod.openlinksw.com/describe/?url=<xsl:value-of select="urlify (.)"/></xsl:attribute>
                      <xsl:attribute name="title"><xsl:value-of select="."/></xsl:attribute>
                      <xsl:choose>
                        <xsl:when test="'' != ./@shortform">
                          <xsl:value-of select="./@shortform"/>
		    </xsl:when>
                        <xsl:when test="'erank' = ./@datatype or 'trank' = ./@datatype">rank</xsl:when>
		    <xsl:otherwise>
                          <xsl:value-of select="."/>
		    </xsl:otherwise>
		</xsl:choose>
                    </a>
                  </td>
                </xsl:when>
                <xsl:when test="'erank' = ./@datatype or 'trank' = ./@datatype"/>
                <xsl:when test="'srch_xerpt' = ./span/@class">
                  <td>
                    <xsl:value-of select="."/>
                  </td>
                </xsl:when>
                <xsl:otherwise/>
              </xsl:choose>
            </xsl:for-each>
          </tr>
        </xsl:for-each>
      </tbody>
    </table>
  </xsl:template>
  <xsl:template match="@* | node()">
    <xsl:copy>
      <xsl:apply-templates select="@* | node()"/>
    </xsl:copy>
    </xsl:template>
</xsl:stylesheet>
]]></programlisting>
      </listitem>
  <listitem>The result of executing the fct_example.vsp should be:
     <figure id="fcapiex1" float="1">
       <title>Faceted API Example</title>
       <graphic fileref="ui/fcapiex1.png"/>
     </figure>
</listitem>
</orderedlist>
   </sect4>
</sect3>
<sect3 id="virtuosospongerfacentuirest">
  <title>SOAP interface</title>
<para>The facet web service is also available via SOAP protocol.</para>
<para>The request message contains single element 'query' with syntax explained earlier. Also the
SOAPAction HTTP header should be '#query' . After successful evaluation of the query, the service
will return a SOAP envelope containing in the Body element single 'facets' element described above.</para>
<para>Example:</para>
<para>This example shows execution of same command as in example for REST interface here it using SOAP:</para>
<para>Request message:</para>
<programlisting><![CDATA[
<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP:Body>
    <query xmlns="http://openlinksw.com/services/facets/1.0/" inference="" same-as="">
      <text>Seattle Mariners traveled all the way to Japan to watch</text>
      <view type="text" limit="20" offset="0"/>
    </query>
  </SOAP:Body>
</SOAP:Envelope>
]]></programlisting>
<para>Response message:</para>
<programlisting><![CDATA[
<SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP:Body>
    <fct:facets xmlns:fct="http://openlinksw.com/services/facets/1.0/">
      <fct:sparql>SELECT distinct ?s1 as ?c1, (bif:search_excerpt (bif:vector ('THE', 'MARINERS', 'WAY', 'SEATTLE', 'WATCH', 'ALL', 'TO', 'JAPAN', 'TRAVELED'), ?o1)) as ?c2  where { ?s1 ?s1textp ?o1 . filter (bif:contains (?o1, '(THE AND MARINERS AND WAY AND SEATTLE AND WATCH AND ALL AND TO AND JAPAN AND TRAVELED)')) . } LIMIT 20  OFFSET 0</fct:sparql>
      <fct:time>114</fct:time>
      <fct:complete>yes</fct:complete>
      <fct:db-activity>   134R rnd  9.488KR seq      0P disk  8.966MB /    602 messages</fct:db-activity>
      <fct:result>
        <fct:row>
          <fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html</fct:column>
          <fct:column/>
          <fct:column><span class="srch_xerpt">... While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b> Ichiro... for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without a doubt... leading <b>the</b> Dominican <b>to</b> its...</span></fct:column>
        </fct:row>
        <fct:row>
          <fct:column datatype="url" shortform="http://bobdupuy.mlbl...ld_baseball__6.html">http://bobdupuy.mlblogs.com/bobdupuy/2006/03/world_baseball__6.html</fct:column>
          <fct:column/>
          <fct:column><span class="srch_xerpt">Orlando While Chuck Armstrong president of <b>the</b> <b>Seattle</b> <b>Mariners</b> <b>traveled</b> <b>all</b> <b>the</b> <b>way</b> <b>to</b> <b>Japan</b> <b>to</b> <b>watch</b>... perform for <b>the</b> advancing <b>Japan</b> team last week <b>the</b> star from <b>the</b> <b>Seattle</b> roster so far in Round 1 has without...</span></fct:column>
        </fct:row>
      </fct:result>
    </fct:facets>
  </SOAP:Body>
</SOAP:Envelope>
]]></programlisting>      	
  </sect3>
</sect2>
</sect1>


<sect1 id="rdfiridereferencing"><title>Linked Data</title>
<para>There are many cases when RDF data should be retrieved from remote sources only when really needed.
E.g., a scheduling application may read personal calendars from personal sites of its users.
Calendar data expire quickly, so there's no reason to frequently re-load them in hope that they are queried before expired.
</para>
<para>Virtuoso extends SPARQL so it is possible to download RDF resource from a given IRI, parse them and store the resulting triples in a graph, all three operations will be performed during the SPARQL query execution.
The IRI of graph to store triples is usually equal to the IRI where the resource is download from, so the feature is named &quot;IRI dereferencing&quot;
There are two different use cases for this feature.
In simple case, a SPARQL query contains <emphasis>from</emphasis> clauses that enumerate graphs to process, but there are no triples in <emphasis>DB.DBA.RDF_QUAD</emphasis> that correspond to some of these graphs.
The query execution starts with dereferencing of these graphs and the rest runs as usual.
In more sophisticated case, the query is executed many times in a loop.
Every execution produces a partial result.
SPARQL processor checks for IRIs in the result such that resources with that IRIs may contain relevant data but not yet loaded into the <emphasis>DB.DBA.RDF_QUAD</emphasis>.
After some iteration, the partial result is identical to the result of the previous iteration, because there's no more data to retrieve.
As the last step, SPARQL processor builds the final result set.
</para>
<sect2 id="rdfinputgrab"><title>IRI Dereferencing For FROM Clauses, &quot;define get:...&quot; Pragmas</title>
<para>Virtuoso extends SPARQL syntax of <emphasis>from</emphasis> and <emphasis>from named</emphasis> clauses.
It allows additional list of options at end of clause: <emphasis>option ( param1 value1, param2 value2, ... )</emphasis>
where parameter names are QNames that start with <emphasis>get:</emphasis> prefix and values are &quot;precode&quot; expressions, i.e. expressions that does not contain variables other than external parameters.
Names of allowed parameters are listed below.
</para>
<itemizedlist>
  <listitem><emphasis>get:soft</emphasis> is the retrieval mode, supported values are &quot;soft&quot; and &quot;replacing&quot;.
If the value is &quot;soft&quot; then the SPARQL processor will not even try to retrieve triples if the destination graph is non-empty.
Other <emphasis>get:...</emphasis> parameters are useless without this one.</listitem>
  <listitem><emphasis>get:uri</emphasis> is the IRI to retrieve if it is not equal to the IRI of the <emphasis>from</emphasis> clause.
These can be used if data should be retrieved from a mirror, not from original resource location or in any other case when the destination graph IRI differs from the location of the resource.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
define get:uri "http://example.com/dataspace/person/kidehen"
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
WHERE { graph ?g { ?id a ?o } }
LIMIT 10;

id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1231
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1231
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1243
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1243
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D

10 Rows. -- 10 msec.

]]></programlisting>
  <listitem><emphasis>get:method</emphasis> is the HTTP method that should be used to retrieve the resource, supported methods are &quot;GET&quot; for plain HTTP and &quot;MGET&quot; for URIQA web service endpoint.
By default, &quot;MGET&quot; is used for IRIs that end with &quot;/&quot; and &quot;GET&quot; for everything else.</listitem>
  <listitem><emphasis>get:refresh</emphasis> is the maximum allowed age of the cached resource, no matter what is specified by the server where the resource resides.
The value is an positive integer (number of seconds). Virtuoso reads HTTP headers and uses &quot;Date&quot;, &quot;ETag&quot;, &quot;Expires&quot;, &quot;Last-Modified&quot;, &quot;Cache-Control&quot; and &quot;Pragma: no-cache&quot; fields to calculate when the resource should be reloaded, this value can become smaller due to <emphasis>get:refresh</emphasis> but can not be incremented.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
define get:refresh "3600"
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
WHERE { graph ?g { ?id a ?o } }
LIMIT 10;

id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1231
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1231
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1243
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1243
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D

10 Rows. -- 10 msec.

]]></programlisting>
  <listitem><emphasis>get:proxy</emphasis> address of the proxy server, as &quot;host:port&quot; string, if direct download is impossible; the default is to not use proxy.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
define get:proxy "www.openlinksw.com:80"
define get:method "GET"
define get:soft "soft"
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
WHERE { graph ?g { ?id a ?o } }
LIMIT 10;

id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1231
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1231
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1243
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1243
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1261
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D
http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D

10 Rows. -- 10 msec.
SQL> limit 10;
]]></programlisting>

<!--
  <listitem><emphasis>get:login</emphasis></listitem>
  <listitem><emphasis>get:password</emphasis></listitem>
  <listitem><emphasis>get:query</emphasis></listitem> -->
<para>If a value of some <emphasis>get:...</emphasis> parameter repeats for every <emphasis>from</emphasis> clause then it can be written as a global
pragma like <emphasis>define get:soft "soft"</emphasis>.
The following two queries will work identically:
</para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
  OPTION (get:soft "soft", get:method "GET")
FROM NAMED <http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/sioc.ttl>
  OPTION (get:soft "soft", get:method "GET")
WHERE { graph ?g { ?id a ?o } }
LIMIT 10;

id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/person/oerling#this
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/958
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/958
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/949
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/949

10 Rows. -- 862 msec.
]]></programlisting>
<programlisting><![CDATA[
SQL>SPARQL
define get:method "GET"
define get:soft "soft"
SELECT ?id
FROM NAMED <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
FROM NAMED <http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/sioc.ttl>
WHERE { graph ?g { ?id a ?o } }
LIMIT 10;

id
VARCHAR
_______________________________________________________________________________

http://www.openlinksw.com/dataspace/person/oerling#this
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/958
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/958
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/949
http://www.openlinksw.com/dataspace/oerling/weblog/Orri%20Erling%27s%20Blog/949

10 Rows. -- 10 msec.
]]></programlisting>
<para>
It can make text shorter and it is especially useful when the query text comes from client but the parameter should have a fixed value due to security reasons:
the values set by <emphasis>define get:...</emphasis> can not be redefined inside the query and the application may prevent the text with desired pragmas before the execution.
</para>
<para>
Note that the user should have <emphasis>SPARQL_UPDATE</emphasis> role in order to execute such a query.
By default SPARQL web service endpoint is owned by <emphasis>SPARQL</emphasis> user that have <emphasis>SPARQL_SELECT</emphasis> but not
<emphasis>SPARQL_UPDATE</emphasis>.
It is possible in principle to grant <emphasis>SPARQL_UPDATE</emphasis> to <emphasis>SPARQL</emphasis> but this breaches the whole security of the RDF storage.
</para>
<listitem><emphasis>FROM CLAUSE with options</emphasis>: options in OPTION() list should be delimited with commas.
grab options are not allowed as they are global for the query. Only specific 'get:xxx' options are useful here.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT DISTINCT ?friend
FROM NAMED  <http://example.com/dataspace/person/kidehen>
OPTION (get:soft "soft", get:method "GET")
WHERE
  {
      <http://example.com/dataspace/person/kidehen#this> foaf:knows
?friend .
  };
friend
VARCHAR
_______________________________________________________________________________

http://www.dajobe.org/foaf.rdf#i
http://www.w3.org/People/Berners-Lee/card#i
http://www.w3.org/People/Connolly/#me
http://my.opera.com/chaals/xml/foaf#me
http://www.w3.org/People/Berners-Lee/card#amy
http://www.w3.org/People/EM/contact#me
http://example.com/dataspace/person/ghard#this
http://example.com/dataspace/person/omfaluyi#this
http://example.com/dataspace/person/alanr#this
http://example.com/dataspace/person/bblfish#this
http://example.com/dataspace/person/danja#this
http://example.com/dataspace/person/tthibodeau#this
...
36 Rows. -- 1693 msec.
]]></programlisting>
</itemizedlist>
</sect2>
<sect2 id="rdfinputgrab"><title>IRI Dereferencing For Variables, &quot;define input:grab-...&quot; Pragmas</title>
<para>
Consider a set of personal data such that one resource can list many persons and point to resources where that persons are described in more details.
E.g. resource about <emphasis>user1</emphasis> describes the user and also contain statements that <emphasis>user2</emphasis> and <emphasis>user3</emphasis> are persons and more data can be found in <emphasis>user2.ttl</emphasis> and <emphasis>user3.ttl</emphasis>,
<emphasis>user3.ttl</emphasis> can contain statements that <emphasis>user4</emphasis> is also person and more data can be found in <emphasis>user4.ttl</emphasis> and so on.
The query should find as many users as it is possible and return their names and e-mails.
</para>
<para>
If all data about all users were loaded into the database, the query could be quite simple:
</para>
<programlisting><![CDATA[
SQL>SPARQL
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?id ?firstname ?nick
where
  {
    graph ?g
      {
        ?id rdf:type foaf:Person.
        ?id foaf:firstName ?firstname.
        ?id foaf:knows ?fn .
        ?fn foaf:nick ?nick.
      }
   }
limit 10;

id                                                      firstname  nick
VARCHAR                                                 VARCHAR    VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/person/pmitchell#this   LaRenda    sdmonroe
http://example.com/dataspace/person/pmitchell#this   LaRenda    kidehen{at}openlinksw.com
http://example.com/dataspace/person/pmitchell#this   LaRenda    alexmidd
http://example.com/dataspace/person/abm#this         Alan       kidehen{at}openlinksw.com
http://example.com/dataspace/person/igods#this       Cameron    kidehen{at}openlinksw.com
http://example.com/dataspace/person/goern#this       Christoph  captsolo
http://example.com/dataspace/person/dangrig#this     Dan        rickbruner
http://example.com/dataspace/person/dangrig#this     Dan        sdmonroe
http://example.com/dataspace/person/dangrig#this     Dan        lszczepa
http://example.com/dataspace/person/dangrig#this     Dan        kidehen

10 Rows. -- 80 msec.
]]></programlisting>
<para>
It is possible to enable IRI dereferencing in such a way that all appropriate resources are loaded during the query execution even if names of some of them are not known a priori.
</para>
<programlisting><![CDATA[
SQL>SPARQL
  define input:grab-var "?more"
  define input:grab-depth 10
  define input:grab-limit 100
  define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
  prefix foaf: <http://xmlns.com/foaf/0.1/>
  prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
  prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?id ?firstname ?nick
WHERE {
    graph ?g {
               ?id rdf:type foaf:Person.
               ?id foaf:firstName ?firstname.
               ?id foaf:knows ?fn .
               ?fn foaf:nick ?nick.
               OPTIONAL { ?id rdfs:SeeAlso ?more }
            }
}
LIMIT 10;

id                                                         firstname  nick
VARCHAR                                                    VARCHAR    VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/person/ghard#this          Yrj+?n+?   kidehen
http://inamidst.com/sbp/foaf#Sean                          Sean       d8uv
http://example.com/dataspace/person/dangrig#this        Dan        rickbruner
http://example.com/dataspace/person/dangrig#this        Dan        sdmonroe
http://example.com/dataspace/person/dangrig#this        Dan        lszczepa
http://example.com/dataspace/person/dangrig#this        Dan        kidehen
http://captsolo.net/semweb/foaf-captsolo.rdf#Uldis_Bojars  Uldis      mortenf
http://captsolo.net/semweb/foaf-captsolo.rdf#Uldis_Bojars  Uldis      danja
http://captsolo.net/semweb/foaf-captsolo.rdf#Uldis_Bojars  Uldis      zool
http://example.com/dataspace/person/rickbruner#this     Rick       dangrig

10 Rows. -- 530 msec.

]]></programlisting>
<para>
The IRI dereferencing is controlled by the following pragmas:
</para>
<itemizedlist>
  <listitem><emphasis>input:grab-var</emphasis> specifies a name of variable whose values should be used as IRIs of resources that should be downloaded.
It is not an error if the variable is sometimes unbound or gets values that can not be converted to IRIs (e.g., integers) -- bad values are silently ignored.
It is also not an error if the IRI can not be retrieved, this makes IRI retrieval somewhat similar to &quot;best effort union&quot; in SQL.
This pragma can be used more than once to specify many variable names.
It is not an error if values of different variables result in same IRI or a variable gets same value many times -- no one IRI is retrieved more than once.</listitem>
  <listitem><emphasis>input:grab-iri</emphasis> specifies an IRI that should be retrieved before executing the rest of the query, if it is not in the <emphasis>DB.DBA.RDF_QUAD</emphasis> already.
This pragma can be used more than once to specify many IRIs.
The typical use of this pragma is querying a set of related resources when only one &quot;root&quot; resource IRI is known but even that resource is not loaded.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
  define input:storage ""
  define input:grab-iri <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
  define input:grab-var "id"
  define input:grab-depth 10
  define input:grab-limit 100
  define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
SELECT ?id
WHERE { graph ?g { ?id a ?o } }
LIMIT 10;

id
VARCHAR
_______________________________________________________________________________

http://example.com/virtrdf-data-formats#default-iid
http://example.com/virtrdf-data-formats#default-iid-nullable
http://example.com/virtrdf-data-formats#default-iid-nonblank
http://example.com/virtrdf-data-formats#default-iid-nonblank-nullable
http://example.com/virtrdf-data-formats#default
http://example.com/virtrdf-data-formats#default-nullable
http://example.com/virtrdf-data-formats#sql-varchar
http://example.com/virtrdf-data-formats#sql-varchar-nullable
http://example.com/virtrdf-data-formats#sql-longvarchar
http://example.com/virtrdf-data-formats#sql-longvarchar-nullable

10 Rows. -- 530 msec.

]]></programlisting>
  <listitem><emphasis>input:grab-all</emphasis> is the simplest possible way to enable the feature but the resulting performance can be very bad.
It turns all variables and IRI constants in all graph, subject and object fields of all triple patterns of the query into values for
<emphasis>input:grab-var</emphasis> and <emphasis>input:grab-iri</emphasis>,
so the SPARQL processor will dereference everything what might be related to the text of the query.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
  define input:grab-all "yes"
  define input:grab-depth 10
  define input:grab-limit 100
  define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
  prefix foaf: <http://xmlns.com/foaf/0.1/>
  prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?id ?firstname ?nick
where
  {
    graph ?g
     {
       ?id rdf:type foaf:Person.
       ?id foaf:firstName ?firstname.
       ?id foaf:knows ?fn .
       ?fn foaf:nick ?nick.
  }
  }
limit 10;

id                                                      firstname   nick
VARCHAR                                                 VARCHAR     VARCHAR
____________________________________________________________________

http://example.com/dataspace/person/pmitchell#this   LaRenda     sdmonroe
http://example.com/dataspace/person/pmitchell#this   LaRenda     kidehen{at}openlinksw.com
http://example.com/dataspace/person/pmitchell#this   LaRenda     alexmidd
http://example.com/dataspace/person/abm#this         Alan        kidehen{at}openlinksw.com
http://example.com/dataspace/person/igods#this       Cameron     kidehen{at}openlinksw.com
http://example.com/dataspace/person/goern#this       Christoph   captsolo
http://example.com/dataspace/person/dangrig#this     Dan         rickbruner
http://example.com/dataspace/person/dangrig#this     Dan         sdmonroe
http://example.com/dataspace/person/dangrig#this     Dan         lszczepa
http://example.com/dataspace/person/dangrig#this     Dan         kidehen

10 Rows. -- 660 msec.

]]></programlisting>
  <listitem><emphasis>input:grab-seealso</emphasis> (and synonym <emphasis>input:grab-follow-predicate</emphasis>) specifies an IRI of an predicate similar to foaf:seeAlso.
Predicates of that sort suggest location of resources that contain more data about predicate subject.
The IRI dereferencing routine may use these predicates to find additional IRIs for loading resources.
This is especially useful when the text of the query comes from remote client and may lack triple patterns like
<emphasis><![CDATA[optional { ?id <SeeAlso> ?more }]]></emphasis> from the previous example.
The use of <emphasis>input:grab-seealso</emphasis> makes the SPARQL query nondeterministic, because the order and the number of retrieved documents will
depend on execution plan and they may change from run to run.
This pragma can be used more than once to specify many IRIs, but this feature is costly.
Every additional predicate may result in significant number of lookups in the RDF storage, affecting total execution time.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
  define input:grab-iri <http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/sioc.ttl>
  define input:grab-var "id"
  define input:grab-depth 10
  define input:grab-limit 100
  define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
  define input:grab-seealso <foaf:maker>
    prefix foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?id
where
  {
    graph ?g
      {
        ?id a foaf:Person .
      }
  }
limit 10;

id
VARCHAR
_______________________________________________________________________________

mailto:somebody@example.domain
http://example.com/dataspace/person/dav#this
http://example.com/dataspace/person/dba#this
mailto:2@F.D
http://example.com/dataspace/person/test1#this
http://www.openlinksw.com/blog/~kidehen/gems/rss.xml#Kingsley%20Uyi%20Idehen
http://digitalmusic.weblogsinc.com/rss.xml#
http://partners.userland.com/nytrss/books.xml#
http://partners.userland.com/nytrss/arts.xml#

9 Rows. -- 105 msec.

]]></programlisting>
  <listitem><emphasis>input:grab-limit</emphasis> should be an integer that is a maximum allowed number of resource retrievals.
The default value is pretty big (few millions of documents) so it is strongly recommended to set smaller value.
Set it even if you're absolutely sure that the set of resources is small, because program errors are always possible.
All resource downloads are counted, both successful and failed, both forced by <emphasis>input:grab-iri</emphasis> and forced by <emphasis>input:grab-var</emphasis>.
Nevertheless, all constant IRIs specified by <emphasis>input:grab-iri</emphasis> (or <emphasis>input:grab-all</emphasis>) are downloaded before the first check of the <emphasis>input:grab-limit</emphasis> counter,
so this limit will never prevent from downloading &quot;root&quot; resources.
</listitem>
  <listitem><emphasis>input:grab-depth</emphasis> should be an integer that is a maximum allowed number of query iterations.
Every iteration may find new IRIs to retrieve, because resources loaded on previous iteration may add these IRIs to <emphasis>DB.DBA.RDF_QUAD</emphasis> and make result set longer.
The default value is 1, so the SPARQL processor will retrieve only resources explicitly named in &quot;root&quot; resources or in quad that are in the database before the query execution.
</listitem>
  <listitem><emphasis>input:grab-base</emphasis> specifies a base IRI used to convert relative IRIs into absolute. The default is an empty string.</listitem>
<programlisting><![CDATA[
SQL>SPARQL
  define input:grab-depth 10
  define input:grab-limit 100
  define input:grab-var "more"
  define input:grab-base "http://www.openlinksw.com/dataspace/kidehen@openlinksw.com/weblog/kidehen@openlinksw.com%27s%20BLOG%20%5B127%5D/1300"
  prefix foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?id
where
  {
    graph ?g
     {
       ?id a foaf:Person .
       optional { ?id foaf:maker ?more }
     }
  }
limit 10;

id
VARCHAR
_______________________________________________________________________________

mailto:somebody@example.domain
http://example.com/dataspace/person/dav#this
http://example.com/dataspace/person/dba#this
mailto:2@F.D
http://example.com/dataspace/person/test1#this
http://www.openlinksw.com/blog/~kidehen/gems/rss.xml#Kingsley%20Uyi%20Idehen
http://digitalmusic.weblogsinc.com/rss.xml#
http://partners.userland.com/nytrss/books.xml#
http://partners.userland.com/nytrss/arts.xml#

9 Rows. -- 115 msec.

]]></programlisting>
  <listitem><emphasis>input:grab-resolver</emphasis> is a name of procedure that resolve IRIs and determines the HTTP method of retrieval.
The default is name of <emphasis>DB.DBA.RDF_GRAB_RESOLVER_DEFAULT()</emphasis> procedure that is described below.
If other procedure is specified, the signature should match to the default one.</listitem>
  <listitem><emphasis>input:grab-destination</emphasis> is to override the default behaviour of the IRI dereferencing and store all retrieved triples in a single graph.
This is convenient when there's no logical difference where any given triple comes from, and changes in remote resources will only add triples but not make cached triples obsolete.
A SPARQL query is usually faster when all graph IRIs are fixed and there are no graph group patterns with an unbound graph variable, so storing everything in one single graph is worth considering.
</listitem>
  <listitem><emphasis>input:grab-loader</emphasis> is a name of procedure that retrieve the resource via HTTP, parse it and store it.
The default is name of <emphasis>DB.DBA.RDF_SPONGE_UP()</emphasis> procedure; this procedure also used by IRI dereferencing for FROM clauses.
You will probably never need to write your own procedure of this sort but some Virtuoso plugins will provide ready-to-use functions that will retrieve non-RDF resources and extract their metadata as triples or
will implement protocols other than HTTP.
</listitem>
</itemizedlist>
<para>Default resolver procedure is <emphasis>DB.DBA.RDF_GRAB_RESOLVER_DEFAULT()</emphasis>. Note that the function produce two absolute URIs,
<emphasis>abs_uri</emphasis> and <emphasis>dest_uri</emphasis>. Default procedure returns two equal strings, but other may return different values,
e.g., return primary and permanent location of the resource as <emphasis>dest_uri</emphasis> and the fastest known mirror location as
<emphasis>abs_uri</emphasis> thus saving HTTP retrieval time. It can even signal an error to block the downloading of some unwanted resource.</para>
<programlisting><![CDATA[
DB.DBA.RDF_GRAB_RESOLVER_DEFAULT (
  in base varchar,         -- base IRI as specified by input:grab-base pragma
  in rel_uri varchar,      -- IRI of the resource as it is specified by input:grab-iri or a value of a variable
  out abs_uri varchar,     -- the absolute IRI that should be downloaded
  out dest_uri varchar,    -- the graph IRI where triples should be stored after download
  out get_method varchar ) -- the HTTP method to use, should be "GET" or "MGET".
]]></programlisting>
</sect2>
<sect2 id="urlrewriting"><title>URL rewriting</title>
<para>URL rewriting is the act of modifying a source URL prior to the final processing of that URL by a
Web Server.</para>
<para>The ability to rewrite URLs may be desirable for many reasons that include:</para>
<itemizedlist>
<listitem>Changing Web information resource URLs on the a Web Server without breaking existing bookmarks
held in User Agents (e.g., Web browsers)</listitem>
<listitem>URL compaction where shorter URLs may be constructed on a conditional basis for specific User
Agents (e.g. Email clients)</listitem>
<listitem>Construction of search engine friendly URLs that enable richer indexing since most search
engines cannot process parameterized URLs effectively.</listitem>
</itemizedlist>
  <sect3 id="usingurlrewritesolelinkdpl">
			<title>Using URL Rewriting to Solve Linked Data Deployment Challenges</title>
<para>URI naming schemes don't resolve the challenges associated with referencing data. To reiterate,
this is demonstrated by the fact that the URIs http://demo.openlinksw.com/Northwind/Customer/ALFKI
and http://demo.openlinksw.com/Northwind/Customer/ALFKI#this both appear as
http://demo.openlinksw.com/Northwind/Customer/ALFKI to the Web Server, since data following the
fragment identifier "#" never makes it that far.</para>
<para>The only way to address data referencing is by pre-processing source URIs
(e.g. via regular expression or sprintf substitutions) as part of a URL rewriting
processing pipeline. The pipeline process has to take the form of a set of rules
that cater for elements such as HTTP Accept headers, HTTP response code, HTTP response
headers, and rule processing order.</para>
<para>An example of such a pipeline is depicted in the table below.</para>
<table><title>Pre-processing source URIs</title>
<tgroup cols="5">
<thead><row>
<entry>URI Source(Regular Expression Pattern)</entry>
<entry>HTTP Accept Headers(Regular Expression)</entry>
<entry>HTTPResponse Code</entry>
<entry>HTTP Response Headers</entry>
<entry>Rule Processing Order</entry>
</row></thead>
<tbody>
<row>
<entry>/Northwind/Customer/([^#]*)</entry>
<entry>None (meaning default)</entry>
<entry>200 or 303 redirect to a resource with default representation.</entry>
<entry>None</entry>
<entry>Normal (order irrelevant)</entry>
</row>
<row>
<entry>/Northwind/Customer/([^#]*)</entry>
<entry>(text/rdf.n3)</entry>
<entry>(application/rdf.xml)</entry>
<entry>303 redirect to location of a descriptive and associated resource (e.g.
RESTful Web Service that returns desired representation)</entry>
<entry>None</entry>
</row>
<row>
<entry>/Northwind/Customer/([^#]*)</entry>
<entry>(text/html)</entry>
<entry>(application/xhtml.xml)</entry>
<entry>406 (Not Acceptable)or303 redirect to location of resource in requested representation</entry>
<entry>Vary: negotiate, acceptAlternates: {"ALFKI" 0.9 {type application/rdf+xml}}</entry>
</row>
</tbody>
</tgroup>
</table>
<para>The source URI patterns refer to virtual or physical directories for ex. at http://demo.openlinksw.com/.
Rules can be placed at the head or tail of the pipeline, or applied in the order they are declared,
by specifying a Rule Processing Order of First, Last, or Normal, respectively. The decision as to
which representation to return for URI http://demo.openlinksw.com/Northwind/Customer/ALFKI is based
on the MIME type(s) specified in any Accept header accompanying the request.</para>
<para>In the case of the last rule, the Alternates response header applies only to response code 406.
406 would be returned if there were no (X)HTML representation available for the requested resource.
In the example shown, an alternative representation is available in RDF/XML.</para>
<para>When applied to matching HTTP requests, the last two rules might generate responses similar to those below:</para>
<programlisting><![CDATA[
$ curl -I -H "Accept: application/rdf+xml" http://demo.openlinksw.com/Northwind/Customer/ALFKI

HTTP/1.1 303 See Other
Server: Virtuoso/05.00.3016 (Solaris) x86_64-sun-solaris2.10-64 PHP5
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Date: Mon, 16 Jul 2007 22:40:03 GMT
Accept-Ranges: bytes
Location: /sparql?query=CONSTRUCT+{+%3Chttp%3A//demo.openlinksw.com/Northwind/Customer/ALFKI%23this%3E+%3Fp+%3Fo+}+FROM+%3Chttp%3A//demo.openlinksw.com/Northwind%3E+WHERE+{+%3Chttp%3A//demo.openlinksw.com/Northwind/Customer/ALFKI%23this%3E+%3Fp+%3Fo+}&format=application/rdf%2Bxml
Content-Length: 0
]]></programlisting>
<para>In the cURL exchange depicted above, the target Virtuoso server redirects to a SPARQL endpoint
that retrieves an RDF/XML representation of the requested entity.</para>
<programlisting><![CDATA[
$ curl -I -H "Accept: text/html" http://demo.openlinksw.com/Northwind/Customer/ALFKI

HTTP/1.1 406 Not Acceptable
Server: Virtuoso/05.00.3016 (Solaris) x86_64-sun-solaris2.10-64 PHP5
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Date: Mon, 16 Jul 2007 22:40:23 GMT
Accept-Ranges: bytes
Vary: negotiate,accept
Alternates: {"ALFKI" 0.9 {type application/rdf+xml}}
Content-Length: 0
]]></programlisting>
<para>In this second cURL exchange, the target Virtuoso server indicates that there is no resource
to deliver in the requested representation. It provides hints in the form of an alternate resource
representation and URI that may be appropriate, i.e., an RDF/XML representation of the requested entity.
</para>
</sect3>
  <sect3 id="virtuosorulebasedurlrewriter">
			<title>The Virtuoso Rules-Based URL Rewriter</title>
<para>Virtuoso provides a URL rewriter that can be enabled for URLs matching specified patterns.
Coupled with customizable HTTP response headers and response codes, Data-Web server administrators
can configure highly flexible rules for driving content negotiation and URL rewriting. The key
elements of the URL rewriter are:</para>
<itemizedlist>
<listitem>Rewriting rule</listitem>
<listitem>Each rule describes how to parse a single source URL, and how to compose the URL of
the page ultimately returned in the "Location:" response headers</listitem>
<listitem>Every rewriting rule is uniquely identified internally (using IRIs).</listitem>
<listitem>Two types of rule are supported, based on the syntax used to describe the source
URL pattern matching: sprintf-based and regex-based.</listitem>
<listitem>Rewrite rules list</listitem>
<listitem>A named ordered list of rewrite rules or rule lists where rules of the list are
processed from top to bottom or in line with processing pipeline precedence instructions</listitem>
<listitem>Configuration API</listitem>
<listitem>The rewriter configuration API defines functions for creating, dropping, and
enumerating rules and rule lists.</listitem>
<listitem>Virtual hosts and virtual paths</listitem>
<listitem>URL rewriting is enabled by associating a rewrite rules list with a virtual directory</listitem>
</itemizedlist>
</sect3>
  <sect3 id="urlrewritevirtdomains">
			<title>Virtual Domains (Hosts) &amp; Directories</title>
<para>A Virtuoso virtual directory maps a logical path to a physical directory that is file
system or WebDAV based. This mechanism allows physical locations to be hidden or simply reorganised.
Virtual directory definitions are held in the system table DB.DBA.HTTP_PATH. Virtual directories
can be administered in three basic ways:</para>
<itemizedlist>
<listitem>Using the Visual Administration Interface via a Web browser;</listitem>
<listitem>Using the functions vhost_define() and vhost_remove(); and</listitem>
<listitem>Using SQL statements to directly update the HTTP_PATH system table.</listitem>
</itemizedlist>
</sect3>
<sect3 id="urlrewriteniceurlsvslongurls">
			<title>"Nice" URLs vs. "Long" URLs</title>
<para>Although we are approaching the URL Rewriter from the perspective of deploying linked data,
the Rewriter was developed with additional objectives in mind. These in turn have influenced
the naming of some of the formal argument names in the Configuration API function prototypes.
In the following sections, long URLs are those containing a query string with named parameters;
nice (aka. source) URLs have data encoded in some other format. The primary goal of the Rewriter
is to accept a nice URL from an application and convert this into a long URL, which then
identifies the page that should actually be retrieved.
</para>
</sect3>
<sect3 id="urlrewriterulesprocessmechanic">
			<title>Rule Processing Mechanics</title>
<para>When an HTTP request is accepted by the Virtuoso HTTP server, the received nice URL is
passed to an internal path translation function. This function takes the nice URL and, if
the current virtual directory has a url_rewrite option set to an existing ruleset name, tries
to match the corresponding rulesets and rules; that is, it performs a recursive traversal
of any rulelist associated with it. For every rule in the rulelist, the same logic is
applied (only the logic for regex-based rules is described; that for sprintf-based rules
is very similar):
</para>
<itemizedlist>
<listitem>The input for the rule is the resource URL as received from the HTTP header, i.e.,
the portion of the URL from the first '/' after the host:port fields to the end of the URL.</listitem>
<listitem>The input is normalized.</listitem>
<listitem>The input is matched against the rule's regex. If the match fails, the rule is not
applied and the next rule is tried. If the match succeeds, the result is a vector of values.</listitem>
<listitem>If the URL contains a query string, the names and values of the parameters are decoded by
split_and_decode().</listitem>
<listitem>The names and values of any parameters in the request body are also decoded.</listitem>
<listitem>The destination URL is composed</listitem>
<listitem>The value of each parameter in the destination URL is taken from (in order of priority)</listitem>
<listitem>The value of a parameter in the match result;</listitem>
<listitem>The value of a named parameter in the query string of the input nice URL;</listitem>
<listitem>If the original request was submitted by the POST method, the value of a named parameter
in the body of the POST request; or</listitem>
<listitem>if a parameter value cannot be derived from one of these sources, the rule is not applied
and the next rule is tried.</listitem>
</itemizedlist>
<para>The path translation function described above is internal to the Web server, so its signature
is not appropriate for Virtuoso/PL calls and thus is not published. Virtuoso/PL developers can
harness the same functionality using the DB.DBA.URLREWRITE_APPLY API call.
</para>
</sect3>
<sect3 id="urlrewriteruleconductor">
			<title>Enabling URL Rewriting via the Virtuoso Conductor UI</title>
<para>Virtuoso is a full-blown HTTP server in its own right. The HTTP server functionality co-exists
with the product core (i.e., DBMS Engine, Web Services Platform, WebDAV filesystem, and other
components of the Universal Server). As a result, it has the ability to multi-home Web domains
within a single instance across a variety of domain name and port combinations. In addition,
it also enables the creation of multiple virtual directories per domain.
</para>
<para>In addition to the basic functionality, Virtuoso facilitates the association of URL
Rewriting rules with the virtual directories associated with a hosted Web domain.
</para>
<para>In all cases, Virtuoso enables you to configure virtual domains, virtual directories
and URL rewrite rules for one or more virtual directories, via the (X)HTML-based Conductor
Admin User Interface or a collection of Virtuoso Stored Procedure Language (PL)-based APIs.
</para>
<para>The steps for configuring URL Rewrite rules via the Virtuoso Conductor are as follows:</para>
<itemizedlist>
<listitem>Assuming you are using the local demonstration database, load http://example.com/conductor
into your browser, and then proceed through the Conductor as follows:</listitem>
<listitem>Click the "Web Application Server", and "Virtual Domains &amp; Directories" tabs</listitem>
<listitem>Pick the domain that contains the virtual directories to which the rules are to be applied
(in this case the default was taken)</listitem>
<listitem>Click on the "URL-rewrite" link to create, delete, or edit a rule as shown below:</listitem>
<listitem>Create a Rule for HTML Representation Requests (via SPARQL SELECT Query)</listitem>
<listitem>Create a Rule for RDF Representation Requests (via SPARQL CONSTRUCT Query)</listitem>
<listitem>Then save and exit the Conductor, and test your rules with curl or any other User Agent.</listitem>
</itemizedlist>
<figure id="urlrewriteimg1" float="1">
   <title>URL-rewrite UI using Conductor</title>
   <graphic fileref="ui/urlrw1.png"/>
</figure>
</sect3>
<sect3 id="urlrewriterulevirtusopl">
			<title>Enabling URL Rewriting via Virtuoso PL</title>
<para>The vhost_define()API is used to define virtual hosts and virtual paths hosted by the
Virtuoso HTTP server. URL rewriting is enabled through this function's opts parameter.
opts is of type ANY, e.g., a vector of field-value pairs. Numerous fields are recognized
for controlling different options. The field value url_rewrite controls URL rewriting.
The corresponding field value is the IRI of a rule list to apply.
</para>
<sect4 id="urlrewriterulevirtusoplcontrolapi"><title>Configuration API</title>
<para>Virtuoso includes the following functions for managing URL rewriting rules and rule
lists. The names are self-explanatory.</para>
<programlisting><![CDATA[
-- Deletes a rewriting rule
DB.DBA.URLREWRITE_DROP_RULE

-- Creates a rewriting rule which uses sprintf-based pattern matching
DB.DBA.URLREWRITE_CREATE_SPRINTF_RULE

-- Creates a rewriting rule which uses regular expression (regex) based pattern matching
DB.DBA.URLREWRITE_CREATE_REGEX_RULE

-- Deletes a rewriting rule list
DB.DBA.URLREWRITE_DROP_RULELIST

-- Creates a rewriting rule list
DB.DBA.URLREWRITE_CREATE_RULELIST

-- Lists all the rules whose IRI match the specified 'SQL like' pattern
DB.DBA.URLREWRITE_ENUMERATE_RULES

-- Lists all the rule lists whose IRIs match the specified 'SQL like' pattern
DB.DBA.URLREWRITE_ENUMERATE_RULELISTS
]]></programlisting>
</sect4>
<sect4 id="urlrewriterulecreaterewriterule"><title>Creating Rewriting Rules</title>
<para>Rewriting rules take two forms: sprintf-based or regex-based. When used for nice
URL to long URL conversion, the only difference between them is the syntax of format
strings. The reverse long to nice conversion works only for sprintf-based rules,
whereas regex-based rules are unidirectional.
</para>
<para>For the purposes of describing how to make dereferenceable URIs for linked data,
we will stick with the nice to long conversion using regex-based rules.
</para>
<para>Regex rules are created using the <emphasis>URLREWRITE_CREATE_REGEX_RULE()</emphasis> function.</para>
</sect4>
</sect3>
<sect3 id="urlrewriteruleexamplenorthwind">
   <title>Example - URL Rewriting For the Northwind Linked Data View</title>
<para>The Northwind schema is comprised of commonly understood SQL Tables that include: Customers,
Orders, Employees, Products, Product Categories, Shippers, Countries, Provinces etc.
</para>
<para>An Linked Data View of SQL data is an RDF named graph (RDF data set) comprised of RDF Linked Data
(triples) stored in a Virtuoso Quad Store (the native RDF Data Management realm of Virtuoso).
</para>
<para>In this example we are going interact with Linked Data deployed into the Data-Web from
a live instance of Virtuoso, which uses the URL Rewrite rules from the prior section.
</para>
<para>The components used in the example are as follows:</para>
<itemizedlist>
<listitem>Virtuoso SPARQL Endpoint: http://demo.openlinksw.com/sparql</listitem>
<listitem>Named RDF Graph: http://demo.openlinksw.com/Northwind</listitem>
<listitem>Entity ID - http://demo.openlinksw.com/Northwind/Customer/ALFKI#this</listitem>
<listitem>Information Resource: http://demo.openlinksw.com/Northwind/Customer/ALFKI</listitem>
<listitem>Interactive SPARQL Query Builder (iSPARQL) - http://demo.openlinksw.com/DAV/JS/isparql/index.html</listitem>
</itemizedlist>
<sect4 id="urlrewriterulenorthwindverificationcurl"><title>Northwind URL Rewriting Verification Using curl</title>
<para>The curl utility provides a useful tool for verifying HTTP server responses and rewriting
rules. The curl exchanges below show the URL rewriting rules defined for the Northwind RDF
view being applied.
</para>
<para><emphasis>Example 1:</emphasis></para>
<programlisting><![CDATA[
$ curl -I -H "Accept: text/html" http://demo.openlinksw.com/Northwind/Customer/ALFKI

HTTP/1.1 303 See Other
Server: Virtuoso/05.00.3016 (Solaris) x86_64-sun-solaris2.10-64 PHP5
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Date: Tue, 14 Aug 2007 13:30:02 GMT
Accept-Ranges: bytes
Location: http://demo.openlinksw.com/about/html/http/demo.openlinksw.com/Northwind/Customer/ALFKI
Content-Length: 0
]]></programlisting>
<para><emphasis>Example 2:</emphasis></para>
<programlisting><![CDATA[
$ curl -I -H "Accept: application/rdf+xml" http://demo.openlinksw.com/Northwind/Customer/ALFKI

HTTP/1.1 303 See Other
Server: Virtuoso/05.00.3016 (Solaris) x86_64-sun-solaris2.10-64 PHP5
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Date: Tue, 14 Aug 2007 13:30:22 GMT
Accept-Ranges: bytes
Location: /sparql?query=CONSTRUCT+{+%3Chttp%3A//demo.openlinksw.com/Northwind/Customer/ALFKI%23this%3E+%3Fp+%3Fo+}+FROM+%3Chttp%3A//demo.openlinksw.com/Northwind%3E+WHERE+{+%3Chttp%3A//demo.openlinksw.com/Northwind/Customer/ALFKI%23this%3E+%3Fp+%3Fo+}&format=application/rdf%2Bxml
Content-Length: 0
]]></programlisting>
<para><emphasis>Example 3:</emphasis></para>
<programlisting><![CDATA[
$ curl -I -H "Accept: text/html" http://demo.openlinksw.com/Northwind/Customer/ALFKI#this

HTTP/1.1 404 Not Found
Server: Virtuoso/05.00.3016 (Solaris) x86_64-sun-solaris2.10-64 PHP5
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
Date: Tue, 14 Aug 2007 13:31:01 GMT
Accept-Ranges: bytes
Content-Length: 0
]]></programlisting>
<para>The output above shows how RDF entities from the Data-Web, in this case customer ALFKI,
are exposed in the Document Web. The power of SPARQL coupled with URL rewriting enables us to
produce results in line with the desired representation. A SPARQL SELECT or CONSTRUCT query is
used depending on whether the requested representation is text/html or application/rdf+xml,
respectively.</para>
<para>The 404 response in Example 3 indicates that no HTML representation is available for entity
ALFKI#this. In most cases, a URI of this form (containing a '#' fragment identifier) will
not reach the server. This example supposes that it does: i.e., the RDF client and network
routing allows the suffixed request. The presence of the #this suffix implicitly states
that this is a request for a data resource in the Data-Web realm, not a document resource
from the Document Web.2</para>
<para>Rather than return 404, we could instead choose to construct our rewriting rules to
perform a 303 redirect, so that the response for ALFKI#this in Example 3 becomes the same
as that for ALFKI in Example 1.</para>
</sect4>
</sect3>
<sect3 id="urlrewritetransperantcontent">
   <title>Transparent Content Negotiation</title>
<para>So as not to overload our preceding description of Linked Data deployment with excessive
detail, the description of content negotiation presented thus far was kept deliberately brief.
This section discusses content negotiation in more detail.
</para>
<sect4 id="urlrewritetransperantcontenthttp"><title>HTTP/1.1 Content Negotiation</title>
<para>Recall that a resource (conceptual entity) identified by a URI may be associated
with more than one representation (e.g. multiple languages, data formats, sizes, resolutions).
If multiple representations are available, the resource is referred to as negotiable and
each of its representations is termed a variant. For instance, a Web document resource, named
'ALFKI' may have three variants: alfki.xml, alfki.html and alfki.txt all representing the same data.
Content negotiation provides a mechanism for selecting the best variant.</para>
<para>As outlined in the earlier brief discussion of content negotiation, when a user agent
requests a resource, it can include with the request Accept headers (Accept, Accept-Language,
Accept-Charset, Accept-Encoding etc.) which express the user preferences and user agent
capabilities. The server then chooses and returns the best variant based on the Accept headers.
Because the selection of the best resource representation is made by the server, this
scheme is classed as server-driven negotiation.</para>
</sect4>
<sect4 id="urlrewritetransperantcontenttransperant"><title>Transparent Content Negotiation</title>
<para>An alternative content negotiation mechanism is Transparent Content Negotiation (TCN),
a protocol defined by RFC2295 . TCN offers a number of benefits over standard HTTP/1.1 negotiation,
for suitably enabled user agents.</para>
<para>RFC2295 introduces a number of new HTTP headers including the Negotiate request header,
and the TCN and Alternates response headers. (Krishnamurthy et al. note that although the
HTTP/1.1 specification reserved the Alternates header for use in agent driven negotiation,
it was not fully specified. Consequently under a pure HTTP/1.1 implementation as defined by
RFC2616, server-driven content negotiation is the only option. RFC2295 addresses this issue.)</para>
</sect4>
<sect4 id="urlrewritetransperantcontentdefic"><title>Deficiencies of HTTP/1.1 Server-Driven Negotiation</title>
<para>Weaknesses of server-driven negotiation highlighted by RFCs 2295 and 2616 include:</para>
<itemizedlist>
<listitem>Inefficiency - Sending details of a user agent's capabilities and preferences
with every request is very inefficient, not least because very few Web resources have
multiple variants, and expensive in terms of the number of Accept headers required to
fully describe all but the most simple browser's capabilities.</listitem>
<listitem>Server doesn't always know 'best' - Having the server decide on the 'best' variant
may not always result in the most suitable resource representation being returned to
the client. The user agent might often be better placed to decide what is best for its needs.</listitem>
</itemizedlist>
</sect4>
<sect4 id="urlrewritetransperantcontentvariantagent"><title>Variant Selection By User Agent</title>
<para>Rather than rely on server-driven negotiation and variant selection by the server,
a user agent can take full control over deciding the best variant by explicitly requesting
transparent content negotiation through the Negotiate request header. The negotiation is
'transparent' because it makes all the variants on the server visible to the agent.</para>
<para>Under this scheme, the server sends the user agent a list, represented in an
Alternates header, containing the available variants and their properties. The user
agent can then choose the best variant itself. Consequently, the agent no longer
needs to send large Accept headers describing in detail its capabilities
and preferences. (However, unless caching is used, user-agent driven negotiation does
suffer from the disadvantage of needing a second request to obtain the best representation.
By sending its best guess as the first response, server driven negotiation avoids this
second request if the initial best guess is acceptable.)</para>
</sect4>
<sect4 id="urlrewritetransperantcontentvariantserver"><title>Variant Selection By Server</title>
<para>As well as variant selection by the user agent, TCN allows the server to choose on
behalf of the user agent if the user agent explicitly allows it through the Negotiate
request header. This option allows the user agent to send smaller Accept headers
containing enough information to allow the server to choose the best variant and
return it directly. The server's choice is controlled by a 'remote variant selection
algorithm' as defined in RFC2296.</para>
</sect4>
<sect4 id="urlrewritetransperantcontentvariantuser"><title>Variant Selection By End-User</title>
<para>A further option is to allow the end-user to select a variant, in case the choice made
by negotiation process is not optimal. For instance, the user agent could display an
HTML-based 'pick list' of variants constructed from the variant list returned by the server.
Alternatively the server could generate this pick list itself and include it in the response
to a user agent's request for a variant list. (Virtuoso currently responds this way.)</para>
</sect4>
</sect3>
<sect3 id="urlrewritetransperantcontentserver">
   <title>Transparent Content Negotiation in Virtuoso HTTP Server</title>
<para>The following section describes the Virtuoso HTTP server's TCN implementation
which is based on RFC2295, but without "Feature" negotiation. OpenLink's RDF rich clients,
iSparql and the OpenLink RDF Browser, both support TCN. User agents which do not support
transparent content negotiation continue to be handled using HTTP/1.1 style content
negotiation (whereby server-side selection is the only option - the server selects
the best variant and returns a list of variants in an Alternates response header).</para>
<sect4 id="urlrewritetransperantcontentserverdesc"><title>Describing Resource Variants</title>
<para>In order to negotiate a resource, the server needs to be given information about each
of the variants. Variant descriptions are held in SQL table HTTP_VARIANT_MAP.
The descriptions themselves can be created, updated or deleted using Virtuoso/PL or
through the Conductor UI. The table definition is as follows:</para>
<programlisting><![CDATA[
create table DB.DBA.HTTP_VARIANT_MAP (
  VM_ID integer identity, -- unique ID
  VM_RULELIST varchar, -- HTTP rule list name
  VM_URI varchar, -- name of requested resource e.g. 'page'
  VM_VARIANT_URI varchar, -- name of variant e.g. 'page.xml', 'page.de.html' etc.
  VM_QS float, -- Source quality, a number in the range 0.001-1.000, with 3 digit precision
  VM_TYPE varchar, -- Content type of the variant e.g. text/xml
  VM_LANG varchar, -- Content language e.g. 'en', 'de' etc.
  VM_ENC varchar, -- Content encoding e.g. 'utf-8', 'ISO-8892' etc.
  VM_DESCRIPTION long varchar, -- a human readable description about the variant e.g. 'Profile in RDF format'
  VM_ALGO int default 0, -- reserved for future use
  primary key (VM_RULELIST, VM_URI, VM_VARIANT_URI)
 )
create unique index HTTP_VARIANT_MAP_ID on DB.DBA.HTTP_VARIANT_MAP (VM_ID)
]]></programlisting>
</sect4>
<sect4 id="urlrewritetransperantcontentserveconfgpl"><title>Configuration using Virtuoso/PL</title>
<para>Two functions are provided for adding or updating, or removing variant descriptions using Virtuoso/PL:</para>
<programlisting><![CDATA[
-- Adding or Updating a Resource Variant:
DB.DBA.HTTP_VARIANT_ADD (
  in rulelist_uri varchar, -- HTTP rule list name
  in uri varchar, -- Requested resource name e.g. 'page'
  in variant_uri varchar, -- Variant name e.g. 'page.xml', 'page.de.html' etc.
  in mime varchar, -- Content type of the variant e.g. text/xml
  in qs float := 1.0, -- Source quality, a floating point number with 3 digit precision in 0.001-1.000 range
  in description varchar := null, -- a human readable description of the variant e.g. 'Profile in RDF format'
  in lang varchar := null, -- Content language e.g. 'en', 'bg'. 'de' etc.
  in enc varchar := null -- Content encoding e.g. 'utf-8', 'ISO-8892' etc.
)


--Removing a Resource Variant
DB.DBA.HTTP_VARIANT_REMOVE (
  in rulelist_uri varchar, -- HTTP rule list name
  in uri varchar, -- Name of requested resource e.g. 'page'
  in variant_uri varchar := '%' -- Variant name filter
)
]]></programlisting>
</sect4>
<sect4 id="urlrewritetransperantcontentserveconfgconductor"><title>Configuration using Conductor UI</title>
<para>The Conductor 'Content negotiation' panel for describing resource variants and configuring
content negotiation is depicted below. It can be reached by selecting the 'Virtual Domains &amp; Directories'
tab under the 'Web Application Server' menu item, then selecting the 'URL rewrite' option for a logical path
listed amongst those for the relevant HTTP host, e.g. '{Default Web Site}'</para>
<para>The input fields reflect the supported 'dimensions' of negotiation which include content type,
language and encoding. Quality values corresponding to the options for 'Source Quality' are as follows:</para>
<table><title>Source Quality</title>
<tgroup cols="2">
<thead><row>
<entry>Source Quality</entry>
<entry>Quality Value</entry>
</row></thead>
<tbody>
<row>
<entry>perfect representation</entry>
<entry>1.000</entry>
</row>
<row>
<entry>threshold of noticeable loss of quality</entry>
<entry>0.900</entry>
</row>
<row>
<entry>noticeable, but acceptable quality reduction</entry>
<entry>0.800</entry>
</row>
<row>
<entry>barely acceptable quality</entry>
<entry>0.500</entry>
</row>
<row>
<entry>severely degraded quality</entry>
<entry>0.300</entry>
</row>
<row>
<entry>completely degraded quality</entry>
<entry>0.000</entry>
</row>
</tbody>
</tgroup>
</table>
</sect4>
<sect4 id="urlrewritetransperantcontentserveconfgvarselalgr"><title>Variant Selection Algorithm</title>
<para>When a user agent instructs the server to select the best variant, Virtuoso does so
using the selection algorithm below:</para>
<para>If a virtual directory has URL rewriting enabled (has the 'url_rewrite' option set),
the web server:</para>
<itemizedlist>
<listitem>Looks in DB.DBA.HTTP_VARIANT_MAP for a VM_RULELIST matching the one specified in
the 'url_rewrite' option</listitem>
<listitem>If present, it loops over all variants for which VM_URI is equal to the resource requested</listitem>
<listitem>For every variant it calculates the source quality based on the value of VM_QS and the
source quality given by the user agent</listitem>
<listitem>If the best variant is found, it adds TCN HTTP headers to the response and passes the
VM_VARIANT_URI to the URL rewriter</listitem>
<listitem>If the user agent has asked for a variant list, it composes such a list and returns an
'Alternates' HTTP header with response code 300</listitem>
<listitem>If no URL rewriter rules exist for the target URL, the web server returns the content of
the dereferenced VM_VARIANT_URI.</listitem>
</itemizedlist>
<para>The server may return the best-choice resource representation or a list of available
resource variants. When a user agent requests transparent negotiation, the web server returns
the TCN header "choice". When a user agent asks for a variant list, the server returns the
TCN header "list".</para>
</sect4>
<sect4 id="urlrewritetransperantcontentserveconfgexamples"><title>Examples</title>
<para>In this example we assume the following files have been uploaded to the Virtuoso WebDAV
server, with each containing the same information but in different formats:</para>
<itemizedlist>
<listitem>/DAV/TCN/page.xml - a XML variant</listitem>
<listitem>/DAV/TCN/page.html - a HTML variant</listitem>
<listitem>/DAV/TCN/page.txt - a text variant</listitem>
</itemizedlist>
<para>We add TCN rules and define a virtual directory:</para>
<programlisting><![CDATA[
DB.DBA.HTTP_VARIANT_ADD ('http_rule_list_1', 'page', 'page.html','text/html', 0.900000, 'HTML variant');
DB.DBA.HTTP_VARIANT_ADD ('http_rule_list_1', 'page', 'page.txt', 'text/plain', 0.500000, 'Text document');
DB.DBA.HTTP_VARIANT_ADD ('http_rule_list_1', 'page', 'page.xml', 'text/xml', 1.000000, 'XML variant');
DB.DBA.VHOST_DEFINE (lpath=>'/DAV/TCN/',
                     ppath=>'/DAV/TCN/',
                     is_dav=>1,
                     vsp_user=>'dba',
                     opts=>vector ('url_rewrite', 'http_rule_list_1'));
]]></programlisting>
<para>Having done this we can now test the setup with a suitable HTTP client, in this
case the curl command line utility. In the following examples, the curl client supplies
Negotiate request headers containing content negotiation directives which include:</para>
<itemizedlist>
<listitem>"trans" - The user agent supports transparent content negotiation for the current request.</listitem>
<listitem>"vlist" - The user agent requests that any transparently negotiated response
for the current request includes an Alternates header with the variant list bound to
the negotiable resource. Implies "trans".</listitem>
<listitem>"*" - The user agent allows servers and proxies to run any remote variant selection algorithm.</listitem>
</itemizedlist>
<para>The server returns a TCN response header signalling that the resource is transparently negotiated and either
a choice or a list response as appropriate.</para>
<para>In the first curl exchange, the user agent indicates to the server that, of the formats
it recognizes, HTML is preferred and it instructs the server to perform transparent content
negotiation. In the response, the Vary header field expresses the parameters the server used
to select a representation, i.e. only the Negotiate and Accept header fields are considered.</para>
<programlisting><![CDATA[
$ curl -i -H "Accept: text/xml;q=0.3,text/html;q=1.0,text/plain;q=0.5,*/*;
q=0.3" -H "Negotiate: *" http://example.com/DAV/TCN/page
HTTP/1.1 200 OK Server: Virtuoso/05.00.3021 (Linux) i686-pc-linux-gnu
VDB Connection: Keep-Alive Date: Wed, 31 Oct 2007 15:43:18
GMT Accept-Ranges: bytes TCN: choice Vary: negotiate,accept
Content-Location: page.html Content-Type: text/html
ETag: "14056a25c066a6e0a6e65889754a0602"
Content-Length: 49
<html> <body> some html </body> </html>
]]></programlisting>
<para>Next, the source quality values are adjusted so that the user agent indicates that XML is its preferred format.
</para>
<programlisting><![CDATA[
$ curl -i -H "Accept: text/xml,text/html;q=0.7,text/plain;q=0.5,*/*;q=0.3" -H "Negotiate:
*" http://example.com/DAV/TCN/page HTTP/1.1 200 OK Server: Virtuoso/05.00.3021
(Linux) i686-pc-linux-gnu VDB Connection: Keep-Alive Date: Wed, 31 Oct 2007
15:44:07 GMT Accept-Ranges: bytes TCN: choice Vary: negotiate,accept
Content-Location: page.xml Content-Type: text/xml ETag:
"8b09f4b8e358fcb7fd1f0f8fa918973a" Content-Length: 39

<?xml version="1.0" ?> <a>some xml</a>
]]></programlisting>
<para>In the final example, the user agent wants to decide itself which is the most
suitable representation, so it asks for a list of variants. The server provides the
list, in the form of an Alternates response header, and, in addition, sends an
HTML representation of the list so that the end user can decide on the preferred
variant himself if the user agent is unable to.</para>
<programlisting><![CDATA[
$ curl -i -H "Accept: text/xml,text/html;q=0.7,text/plain;q=0.5,*/*;q=0.3" -H "Negotiate:
vlist" http://example.com/DAV/TCN/page HTTP/1.1 300 Multiple Choices Server:
Virtuoso/05.00.3021 (Linux) i686-pc-linux-gnu VDB Connection: close Content-Type:
text/html; charset=ISO-8859-1 Date: Wed, 31 Oct 2007 15:44:35 GMT Accept-Ranges:
bytes TCN: list Vary: negotiate,accept Alternates: {"page.html" 0.900000 {type text/html}},
{"page.txt" 0.500000 {type text/plain}}, {"page.xml" 1.000000 {type text/xml}} Content-Length: 368

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head>
<title>300 Multiple Choices</title>
</head>
<body>
<h1>Multiple Choices</h1>
Available variants:
<ul>
<li>
<a href="page.html">HTML variant</a>, type text/html</li>
<li><a href="page.txt">Text document</a>, type text/plain</li>
<li><a href="page.xml">XML variant</a>, type text/xml</li>
</ul>
</body>
</html>
]]></programlisting>
</sect4>
</sect3>
</sect2>
<sect2 id="rdfiridereferencingexamples"><title>Examples of other Protocol Resolvers</title>
<para>Example of <emphasis>LSIDs</emphasis>: A scientific name from UBio</para>
<programlisting><![CDATA[
SQL>SPARQL
define get:soft "soft"
SELECT *
FROM <urn:lsid:ubio.org:namebank:11815>
WHERE { ?s ?p ?o }
LIMIT 5;

s                                 p                                           o
VARCHAR                           VARCHAR                                     VARCHAR
_______________________________________________________________________________

urn:lsid:ubio.org:namebank:11815  http://purl.org/dc/elements/1.1/title       Pternistis leucoscepus
urn:lsid:ubio.org:namebank:11815  http://purl.org/dc/elements/1.1/subject     Pternistis leucoscepus (Gray, GR) 1867
urn:lsid:ubio.org:namebank:11815  http://purl.org/dc/elements/1.1/identifier  urn:lsid:ubio.org:namebank:11815
urn:lsid:ubio.org:namebank:11815  http://purl.org/dc/elements/1.1/creator     http://www.ubio.org
urn:lsid:ubio.org:namebank:11815  http://purl.org/dc/elements/1.1/type        Scientific Name

5 Rows. -- 741 msec.
]]></programlisting>
<para>Example of <emphasis>LSIDs</emphasis>: A segment of the human genome from GDB</para>
<programlisting><![CDATA[
SQL>SPARQL
define get:soft "soft"
SELECT *
FROM <urn:lsid:gdb.org:GenomicSegment:GDB132938>
WHERE { ?s ?p ?o }
LIMIT 5;

s  	                                        p  	                                           o
VARCHAR                                    VARCHAR                                               VARCHAR
_______________________________________________________________________________

urn:lsid:gdb.org:GenomicSegment:GDB132938  urn:lsid:gdb.org:DBObject-predicates:accessionID      GDB:132938
urn:lsid:gdb.org:GenomicSegment:GDB132938  http://www.ibm.com/LSID/2004/RDF/#lsidLink            urn:lsid:gdb.org:DBObject:GDB132938
urn:lsid:gdb.org:GenomicSegment:GDB132938  urn:lsid:gdb.org:DBObject-predicates:objectClass      DBObject
urn:lsid:gdb.org:GenomicSegment:GDB132938  urn:lsid:gdb.org:DBObject-predicates:displayName      D20S95
urn:lsid:gdb.org:GenomicSegment:GDB132938  urn:lsid:gdb.org:GenomicSegment-predicates:variantsQ  nodeID://1000027961

5 Rows. -- 822 msec.
]]></programlisting>
<para>Example of <emphasis>OAI</emphasis>: an institutional / departmental repository.</para>
<programlisting><![CDATA[
SQL>SPARQL
define get:soft "soft"
SELECT *
FROM <oai:etheses.bham.ac.uk:23>
WHERE { ?s ?p ?o }
LIMIT 5;

s                           p                                           o
VARCHAR                     VARCHAR                                     VARCHAR
_____________________________________________________________________________

oai:etheses.bham.ac.uk:23   http://purl.org/dc/elements/1.1/title       A study of the role of ATM mutations in the pathogenesis of B-cell chronic lymphocytic leukaemia
oai:etheses.bham.ac.uk:23   http://purl.org/dc/elements/1.1/date        2007-07
oai:etheses.bham.ac.uk:23   http://purl.org/dc/elements/1.1/subject     RC0254 Neoplasms. Tumors. Oncology (including Cancer)
oai:etheses.bham.ac.uk:23   http://purl.org/dc/elements/1.1/identifier  Austen, Belinda (2007) A study of the role of ATM mutations in the pathogenesis of B-cell chronic lymphocytic leukaemia. Ph.D. thesis, University of Birmingham.
oai:etheses.bham.ac.uk:23   http://purl.org/dc/elements/1.1/identifier  http://etheses.bham.ac.uk/23/1/Austen07PhD.pdf

5 Rows. -- 461 msec.
]]></programlisting>
<para>Example of <emphasis>DOI</emphasis></para>
<para>In order to execute correctly queries with doi resolver you need to have:</para>
<itemizedlist>
<listitem>the handle.dll file accessible from your system. For ex. you can put it in the Virtuoso bin folder where the rest of the server components are.</listitem>
<listitem>in your Virtuoso database ini file in section Plugins added the hslookup.dll file, which location should be in the plugins folder under your Virtuoso server installation. For ex:
<programlisting><![CDATA[
[Plugins]
LoadPath = ./plugin
...
Load6    = plain,hslookup
]]></programlisting>
</listitem>
</itemizedlist>
<programlisting><![CDATA[
SQL>SPARQL
define get:soft "soft"
SELECT *
FROM <doi:10.1045/march99-bunker>
WHERE { ?s ?p ?o } ;

s                                                      p                                                 o
VARCHAR                                                VARCHAR                                           VARCHAR
_______________________________________________________________________________

http://www.dlib.org/dlib/march99/bunker/03bunker.html  http://www.w3.org/1999/02/22-rdf-syntax-ns#type   http://example.com/schemas/XHTML#
http://www.dlib.org/dlib/march99/bunker/03bunker.html  http://example.com/schemas/XHTML#title     Collaboration as a Key to Digital Library Development: High Performance Image Management at the University of Washington

2 Rows. -- 12388 msec.
]]></programlisting>
<para>Other examples</para>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX doap: <http://usefulinc.com/ns/doap#>
SELECT DISTINCT ?name ?mbox ?projectName
WHERE {
 <http://dig.csail.mit.edu/2005/ajar/ajaw/data#Tabulator>
doap:developer ?dev .
 ?dev foaf:name ?name .
 OPTIONAL { ?dev foaf:mbox ?mbox }
 OPTIONAL { ?dev doap:project ?proj .
            ?proj foaf:name ?projectName }
};

name          mbox              projectName
VARCHAR       VARCHAR           VARCHAR
____________________ ___________________________________________

Adam Lerer    NULL              NULL
Dan Connolly  NULL              NULL
David Li      NULL              NULL
David Sheets  NULL              NULL
James Hollenbach  NULL          NULL
Joe Presbrey  NULL              NULL
Kenny Lu      NULL              NULL
Lydia Chilton NULL              NULL
Ruth Dhanaraj NULL              NULL
Sonia Nijhawan    NULL          NULL
Tim Berners-Lee   NULL          NULL
Timothy Berners-Lee   NULL      NULL
Yuhsin Joyce Chen         NULL NULL

13 Rows. -- 491 msec.
]]></programlisting>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT DISTINCT ?friendsname ?friendshomepage ?foafsname ?foafshomepage
WHERE
 {
  <http://example.com/dataspace/person/kidehen#this> foaf:knows ?friend .
  ?friend foaf:mbox_sha1sum ?mbox .
  ?friendsURI foaf:mbox_sha1sum ?mbox .
  ?friendsURI foaf:name ?friendsname .
  ?friendsURI foaf:homepage ?friendshomepage .
  OPTIONAL { ?friendsURI foaf:knows ?foaf .
              ?foaf foaf:name ?foafsname .
              ?foaf foaf:homepage ?foafshomepage .
           }
 }
LIMIT 10;




friendsname  	   friendshomepage                         foafsname  	    foafshomepage
ANY                ANY                                     ANY              ANY
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Dan Connolly	    http://www.w3.org/People/Connolly/
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Henry J. Story   http://bblfish.net/
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Henry Story	    http://bblfish.net/
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Henry J. Story   http://bblfish.net/people/henry/card
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Henry Story	    http://bblfish.net/people/henry/card
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Ruth Dhanaraj    http://web.mit.edu/ruthdhan/www
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Dan Brickley	    http://danbri.org/
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Dan Brickley	    http://danbri.org/
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Daniel Krech	    http://eikeon.com/
 Tim Berners Lee   http://www.w3.org/People/Berners-Lee/   Daniel Krech	    http://eikeon.com/

]]></programlisting>
</sect2>

<sect2 id="rdfiridereferencingfacet"><title>Faceted Views over Large-Scale Linked Data</title>
<para>Faceted views over structured and semi structured data have been popular in user interfaces for
some years. Deploying such views of arbitrary linked data at arbitrary scale has been hampered by lack
of suitable back end technology. Many ontologies are also quite large, with hundreds of thousands of
classes.
</para>
<para>Also, the linked data community has been concerned with the processing cost and potential for
denial of service presented by public SPARQL end points.
</para>
<para>This section discusses how we use Virtuoso Cluster Edition for providing interactive browsing over
billions of triples, combining full text search, structured querying and result ranking. We discuss query
planning, run-time inferencing and partial query evaluation. This functionality is exposed through SPARQL,
a specialized web service and a web user interface.</para>
<para>The transition of the web from a distributed document repository into a universal, ubiquitous
database requires a new dimension of scalability for supporting rich user interaction. If the web is the
database, then it also needs a query and report writing tool to match. A faceted user interaction paradigm
has been found useful for aiding discovery and query of variously structured data. Numerous implementations
exist but they are chiefly client side and are limited in the data volumes they can handle.
</para>
<para>At the present time, linked data is well beyond prototypes and proofs of concept. This means that
what was done in limited specialty domains before must now be done at real world scale, in terms of both
data volume and ontology size. On the schema, or T box side, there exist many comprehensive general purpose
ontologies such as Yago[1], OpenCyc[2], Umbel[3] and the DBpedia[4] ontology and many domain specific
ones, such as [5]. For these to enter into the user experience, the platform must be able to support
the user's choice of terminology or terminologies as needed, preferably without blow up of data and
concomitant slowdown.
</para>
<para>Likewise, in the LOD world, many link sets have been created for bridging between data sets.
Whether such linkage is relevant will depend on the use case. Therefore we provide fine grained control
over which owl:sameAs assertions will be followed, if any.
</para>
<para>Against this background, we discuss how we tackle incremental interactive query composition on
arbitrary data with <link linkend="clusteroperation">Virtuoso Cluster</link>.
</para>
<para>Using SPARQL or a web/web service interface, the user can form combinations of text search and
structured criteria, including joins to an arbitrary depth. If queries are precise and select a limited
number of results, the results are complete. If queries would select tens of millions of results, partial
results are shown.
</para>
<para>The system being described is being actively developed as of this writing, early March of 2009
and is online at http://lod.openlinksw.com/. The data set is a combination of DBpedia, MusicBrainz,
Freebase, UniProt, NeuroCommons, Bio2RDF, and web crawls from PingTheSemanticWeb.com.
</para>
<para>The hardware consists of two 8-core servers with 16G RAM and 4 disks each. The system runs on
Virtuoso 6 Cluster Edition. All application code is written in SQL procedures with limited client side
Ajax, the Virtuoso platform itself is in C.
</para>
<para>The facets service allows the user to start with a text search or a fixed URI and to refine the
search by specifying classes, property values etc., on the selected subjects or any subjects referenced
therefrom.
</para>
<para>This process generates queries involving combinations of text and structured criteria, often
dealing with property and class hierarchies and often involving aggregation over millions of subjects,
specially at the initial stages of query composition. To make this work with in interactive time, two
things are needed:
</para>
<orderedlist>
  <listitem>a query optimizer that can almost infallibly produce the right join order based on cardinalities of
the specific constants in the query</listitem>
  <listitem>a query execution engine that can return partial results after a timeout.</listitem>
</orderedlist>
<para>It is often the case, specially at the beginning of query formulation, that the user only needs
to know if there are relatively many or few results that are of a given type or involve a given property.
Thus partially evaluating a query is often useful for producing this information. This must however be
possible with an arbitrary query, simply citing precomputed statistics is not enough.
</para>
<para>It has for a long time been a given that any search-like application ranks results by relevance.
Whenever the facets service shows a list of results, not an aggregation of result types or properties,
it is sorted on a composite of text match score and link density.
</para>
<para>The section is divided into the following parts:
</para>
<itemizedlist mark="bullet">
  <listitem>SPARQL query optimization and execution adapted for run time inference over large subclass
structures.</listitem>
  <listitem>Resolving identity with inverse functional properties</listitem>
  <listitem>Ranking entities based on graph link density</listitem>
  <listitem>SPARQL partial query evaluation for displaying partial results in fixed time</listitem>
  <listitem>a facets web service providing an XML interface for submitting queries, so that the
user interface is not required to parse SPARQL</listitem>
  <listitem>a sample web interface for interacting with this</listitem>
  <listitem>sample queries and their evaluation times against combinations of large LOD data sets</listitem>
</itemizedlist>
  <sect3 id="rdfiridereferencingfacetprlh"><title>Processing Large Hierarchies in SPARQL</title>
<para>Virtuoso has for a long time had built-in superclass and superproperty inference. This is enabled by
specifying the <emphasis>DEFINE input:inference "context"</emphasis> option, where context is previously
declared to be all subclass, subproperty, equivalence, inverse functional property and same as relations
defined in a given graph. The ontology file is loaded into its own graph and this is then used to construct
the context. Multiple ontologies and their equivalences can be loaded into a single graph which then makes
another context which holds the union of the ontology information from the merged source ontologies.
</para>
<para>
Let us consider a sample query combining a full text search and a restriction on the class of the desired
matches:
</para>
<programlisting><![CDATA[
DEFINE  input:inference  "yago"
PREFIX  cy:  <http://dbpedia.org/class/yago/>
SELECT DISTINCT ?s1 AS ?c1
                ( bif:search_excerpt
                  ( bif:vector ( 'Shakespeare' ), ?o1 )
                ) AS ?c2
WHERE
  {
    ?s1  ?s1textp ?o1                         .
    FILTER
      ( bif:contains (?o1, '"Shakespeare"') ) .
    ?s1  a        cy:Performer110415638
  }
LIMIT 20
]]></programlisting>
<para>This selects all Yago performers that have a property that contains "Shakespeare" as a whole word.
</para>
<para>The <emphasis>DEFINE input:inference "yago"</emphasis> clause means that subclass, subproperty and
inverse functions property statements contained in the inference context called yago are considered when
evaluating the query. The built-in function <emphasis>bif:search_excerpt</emphasis> makes a search engine style summary of
the found text, highlighting occurrences of Shakespeare.
</para>
<para>The <emphasis>bif:contains</emphasis> function in the filter specifies the full text search
condition on ?o1.
</para>
<para>This query is a typical example of queries that are executed all the time when a user refines a
search. We will now look at how we can make an efficient execution plan for the query. First, we must
know the cardinalities of the search conditions.
</para>
<para>To see the count of subclasses of Yago performer, we can do:
</para>
<programlisting><![CDATA[
SPARQL
PREFIX  cy:  <http://dbpedia.org/class/yago/>
SELECT COUNT (*)
FROM <http://dbpedia.org/yago.owl>
WHERE
  {
    ?s  rdfs:subClassOf  cy:Performer110415638
    OPTION (TRANSITIVE, T_DISTINCT)
  }
]]></programlisting>
<para>There are 4601 distinct subclasses, including indirect ones. Next we look at how many Shakespeare
mentions there are:
</para>
<programlisting><![CDATA[
SPARQL
SELECT COUNT (*)
WHERE
  {
    ?s  ?p  ?o .
    FILTER
      ( bif:contains (?o, 'Shakespeare') )
  }
]]></programlisting>
<para>There are 10267 subjects with Shakespeare mentioned in some literal.
</para>
<programlisting><![CDATA[
SPARQL
DEFINE input:inference "yago"
PREFIX cy: <http://dbpedia.org/class/yago/>
SELECT COUNT (*)
WHERE
  {
    ?s1  a  cy:Performer110415638
  }
]]></programlisting>
<para>There are 184885 individuals that belong to some subclass of performer.
</para>
<para>This is the data that the SPARQL compiler must know in order to have a valid query plan. Since
these values will wildly vary depending on the specific constants in the query, the actual database
must be consulted as needed while preparing the execution plan. This is regular query processing
technology but is now specially adapted for deep subclass and subproperty structures.
</para>
<para>Conditions in the queries are not evaluated twice, once for the cardinality estimate and once
for the actual run. Instead, the cardinality estimate is a rapid sampling of the index trees that reads
at most one leaf page.
</para>
<para>Consider a B tree index, which we descend from top to the leftmost leaf containing a match of
the condition. At each level, we count how many children would match and always select the leftmost one.
When we reach a leaf, we see how many entries are on the page. From these observations, we extrapolate
the total count of matches.
</para>
<para>With this method, the guess for the count of performers is 114213, which is acceptably close to the
real number. Given these numbers, we see that it makes sense to first find the full text matches and
then retrieve the actual classes of each and see if this class is a subclass of performer. This last
check is done against a memory resident copy of the Yago hierarchy, the same copy that was used for
enumerating the subclasses of performer.
</para>
<para>However, the query
</para>
<programlisting><![CDATA[
SPARQL
DEFINE input:inference "yago"
PREFIX cy: <http://dbpedia.org/class/yago/>
SELECT DISTINCT ?s1 AS ?c1,
                ( bif:search_excerpt
                  ( bif:vector ('Shakespeare'), ?o1 )
                ) AS ?c2
WHERE
  {
    ?s1  ?s1textp  ?o1                         .
    FILTER
      ( bif:contains (?o1, '"Shakespeare"') )  .
    ?s1  a         cy:ShakespeareanActors
  }
]]></programlisting>
<para>will start with Shakespearean actors since this is a leaf class with only 74 instances and then
check if the properties contain Shakespeare and return their search summaries.
</para>
<para>In principle, this is common cost based optimization but is here adapted to deep hierarchies
combined with text patterns. An unmodified SQL optimizer would have no possibility of arriving at
these results.
</para>
<para>The implementation reads the graphs designated as holding ontologies when first needed and
subsequently keeps a memory based copy of the hierarchy on all servers. This is used for quick iteration
over sub/superclasses or properties as well as for checking if a given class or property is a
subclass/property of another. Triples with OWL predicates <emphasis>equivalentClass</emphasis>,
<emphasis>equivalentProperty</emphasis> and <emphasis>sameAs</emphasis> are also cached in the same data
structure if they occur in the ontology graphs.
</para>
<para>Also cardinality estimates for members of classes near the root of the class hierarchy take
some time since a sample of each subclass is needed. These are cached for some minutes in the
inference context, so that repeated queries will not redo the sampling.
</para>
  </sect3>
  <sect3 id="rdfiridereferencingfacetinvfpr"><title>Inverse Functional Properties and Same As</title>
<para>Specially when navigating social data, as in FOAF and SIOC spaces, there are many blank nodes that
are identified by properties only. For this, we offer an option for automatically joining to subjects
which share an IFP value with the subject being processed. For example, the query for the friends of
friends of Kjetil Kjernsmo returns empty:
</para>
<programlisting><![CDATA[
SPARQL
SELECT COUNT (?f2)
WHERE
  {
    ?s   a             foaf:Person          ;
         ?p            ?o                   ;
         foaf:knows    ?f1                  .
    ?o   bif:contains  "'Kjetil Kjernsmo'"  .
    ?f1  foaf:knows    ?f2
  }
]]></programlisting>
<para>But with the option
</para>
<programlisting><![CDATA[
SPARQL
DEFINE input:inference "b3sifp"
SELECT COUNT (?f2)
WHERE
  {
    ?s   a             foaf:Person          ;
         ?p            ?o                   ;
         foaf:knows    ?f1                  .
    ?o   bif:contains  "'Kjetil Kjernsmo'"  .
    ?f1  foaf:knows    ?f2
  }
]]></programlisting>
<para>we get 4022. We note that there are many duplicates since the data is blank nodes only,
with people easily represented 10 times. The context <emphasis>b3sifp</emphasis> simple declares that
<emphasis>foaf:name</emphasis> and <emphasis>foaf:mbox</emphasis> sha1sum should be treated as inverse
functional properties (IFP). The name is not an IFP in the actual sense but treating it as such for
the purposes of this one query makes sense, otherwise nothing would be found.
</para>
<para>This option is controlled by the choice of the inference context, which is selectable in the
interface discussed below.
</para>
<para>The IFP inference can be thought of as a transparent addition of a subquery into the join sequence.
The subquery joins each subject to its synonyms given by sharing IFPs. This subquery has the special
property that it has the initial binding automatically in its result set. It could be expressed as:
</para>
<programlisting><![CDATA[
SPARQL
SELECT ?f
WHERE
  {
    ?k  foaf:name  "Kjetil Kjernsmo"  .
    {
      SELECT ?org ?syn
      WHERE
        {
          ?org  ?p  ?key  .
          ?syn  ?p  ?key  .
          FILTER
            ( bif:rdf_is_sub
                ( "b3sifp", ?p, <b3s:any_ifp>, 3 )
              &&
              ?syn  !=  ?org
            )
        }
    }
    OPTION
      (
        TRANSITIVE     ,
        T_IN (?org),
        T_OUT (?syn),
        T_MIN (0),
        T_MAX (1)
      )
    FILTER ( ?org = ?k ) .
    ?syn foaf:knows ?f .
  }
]]></programlisting>
<para>It is true that each subject shares IFP values with itself but the transitive construct with 0
minimum and 1 maximum depth allows passing the initial binding of <emphasis>?org</emphasis> directly to
<emphasis>?syn</emphasis>, thus getting first results more rapidly. The <emphasis>rdf_is_sub</emphasis>
function is an internal that simply tests whether <emphasis>?p</emphasis> is a subproperty of
<emphasis>b3s:any_ifp</emphasis>.
</para>
<para>Internally, the implementation has a special query operator for this and the internal form is more
compact than would result from the above but the above could be used to the same effect.
</para>
<para>Our general position is that identity criteria are highly application specific and thus we offer
the full spectrum of choice between run time and precomputing. Further, weaker identity statements than
sameness are difficult to use in queries, thus we prefer identity with semantics of
<emphasis>owl:sameAs</emphasis> but make this an option that can be turned on and off query by query.
</para>
  </sect3>
  <sect3 id="rdfiridereferencingfaceter"><title>Entity Ranking</title>
<para>It is a common end user expectation to see text search results sorted by their relevance. The term
entity rank refers to a quantity describing the relevance of a URI in an RDF graph.
</para>
<para>This is a sample query using entity rank:
</para>
<programlisting><![CDATA[
SPARQL
PREFIX  yago:  <http://dbpedia.org/class/yago/>
PREFIX  prop:  <http://dbpedia.org/property/>
SELECT DISTINCT ?s2 AS ?c1
WHERE
  {
    ?s1  ?s1textp      ?o1                   .
    ?o1  bif:contains  'Shakespeare'         .
    ?s1  a             yago:Writer110794014  .
    ?s2  prop:writer   ?s1
  }
ORDER BY DESC ( <LONG::IRI_RANK> (?s2) )
LIMIT 20
OFFSET 0
]]></programlisting>
<para>This selects works where a writer with Shakespeare in some property is the writer.
</para>
<para>Here the query returns subjects, thus no text search summaries, so only the entity rank of the
returned subject is used. We order text results by a composite of text hit score and entity rank of the
RDF subject where the text occurs. The entity rank of the subject is defined by the count of references
to it, weighed by the rank of the referrers and the outbound link count of referrers. Such techniques
are used in text based information retrieval.
</para>
<para><emphasis>Example with Entity Ranking and Score</emphasis></para>
<programlisting><![CDATA[
## Searching over labels, with text match
## scores and additional ranks for each
## iri / resource:

SELECT ?s ?page ?label
  ?textScore AS ?Text_Score_Rank
  ( <LONG::IRI_RANK> (?s) ) AS ?Entity_Rank
WHERE
  {
    ?s foaf:page ?page ;
     rdfs:label ?label .
    FILTER( lang( ?label ) = "en" ) .
    ?label bif:contains 'adobe and flash'
    OPTION (score ?textScore ) .
  }
]]></programlisting>
<para>One interesting application of entity rank and inference on IFPs and <emphasis>owl:sameAs</emphasis> is in locating
URIs for reuse. We can easily list synonym URIs in order of popularity as well as locate URIs based
on associated text. This can serve in application such as the Entity Name Server
</para>
<para>Entity ranking is one of the few operations where we take a precomputing approach. Since a rank is
calculated based on a possibly long chain of references, there is little choice but to precompute. The
precomputation itself is straightforward enough: First all outbound references are counted for all
subjects. Next all ranks of subjects are incremented by 1 over the referrer's outbound link count.
On successive iterations, the increment is based on the rank increment the referrer received in
the previous round.
</para>
<para>The operation is easily partitioned, since each partition increments the ranks of subjects it
holds. The referrers are spread throughout the cluster, though. When rank is calculated, each partition
accesses every other partition. This is done with relatively long messages, referee ranks are accessed
in batches of several thousand at a time, thus absorbing network latency.
</para>
<para>On the test system, this operation performs a single pass over the corpus of 2.2 billion triples
and 356 million distinct subjects in about 30 minutes. The operation has 100% utilization of all 16
cores. Adding hardware would speed it up, as would implementing it in C instead of the SQL procedures
it is written in at present.
</para>
<para>The main query in rank calculation is:
</para>
<programlisting><![CDATA[
SPARQL
SELECT O            ,
       P            ,
       iri_rank (S)
FROM rdf_quad TABLE
OPTION (NO CLUSTER)
WHERE isiri_id(O)
ORDER BY O
]]></programlisting>
<para>This is the SQL cursor iterated over by each partition. The no cluster option means that only rows
in this process's partition are retrieved. The RDF_QUAD table holds the RDF quads in the store, i.e.,
triple plus graph. The S, P, O columns are the subject, predicate, and object respectively. The graph
column is not used here. The textttiri rank is a partitioned SQL function. This works by using the S
argument to determine which cluster node should run the function. The specifics of the partitioning
are declared elsewhere. The calls are then batched for each intended recipient and sent when the
batches are full. The SQL compiler automatically generates the relevant control structures. This
is like an implicit map operation in the map-reduce terminology.
</para>
<para>An SQL procedure loops over this cursor, adds up the rank and when seeing a new O, the added
rank is persisted into a table. Since links in RDF are typed, we can use the semantics of the link
to determine how much rank is transferred by a reference. With extraction of named entities from
text content, we can further place a given entity into a referential context and use this as a
weighting factor. This is to be explored in future work. The experience thus far shows that we
greatly benefit from Virtuoso being a general purpose DBMS, as we can create application specific
data structures and control flows where these are efficient. For example, it would make little
sense to store entity ranks as triples due to space consumption and locality considerations. With
these tools, the whole ranking functionality took under a week to develop.
</para>
<para><emphasis>Note:</emphasis> In order to use the IRI_RANK feature you need to have the
Facet (fct) vad package installed as the procedure is part of this vad.
</para>
  </sect3>
  <sect3 id="rdfiridereferencingfacetqel"><title>Query Evaluation Time Limits</title>
<para>When scaling the Linked Data model, we have to take it as a given that the workload will be
unexpected and that the query writers will often be unskilled in databases. Insofar possible, we
wish to promote the forming of a culture of creative reuse of data. To this effect, even poorly
formulated questions deserve an answer that is better than just timeout.
</para>
<para>If a query produces a steady stream of results, interrupting it after a certain quota is simple.
However, most interesting queries do not work in this way. They contain aggregation, sorting, maybe
transitivity.
</para>
<para>When evaluating a query with a time limit in a cluster setup, all nodes monitor the time left
for the query. When dealing with a potentially partial query to begin with, there is little point in
transactionality. Therefore the facet service uses read committed isolation. A read committed query
will never block since it will see the before-image of any transactionally updated row. There will
be no waiting for locks and timeouts can be managed locally by all servers in the cluster.
</para>
<para>Thus, when having a partitioned count, for example, we expect all the partitions to time out
around the same time and send a ready message with the timeout information to the cluster node
coordinating the query. The condition raised by hitting a partial evaluation time limit differs
from a run time error in that it leaves the query state intact on all participating nodes. This
allows the timeout handling to come fetch any accumulated aggregates.
</para>
<para>Let us consider the query for the top 10 classes of things with "Shakespeare" in some literal.
This is typical of the workload generated by the faceted browsing web service:
</para>
<programlisting><![CDATA[
SPARQL
DEFINE  input:inference  "yago"
SELECT ?c
       COUNT (*)
WHERE
  {
    ?s  a             ?c             ;
        ?p            ?o             .
    ?o  bif:contains  "Shakespeare"
  }
GROUP BY ?c
ORDER BY DESC 2
LIMIT 10
]]></programlisting>
<para>On the first execution with an entirely cold cache, this times out after 2 seconds and returns:
</para>
<programlisting><![CDATA[
?c                                       COUNT (*)
yago:class/yago/Entity100001740          566
yago:class/yago/PhysicalEntity100001930  452
yago:class/yago/Object100002684          452
yago:class/yago/Whole100003553           449
yago:class/yago/Organism100004475        375
yago:class/yago/LivingThing100004258     375
yago:class/yago/CausalAgent100007347     373
yago:class/yago/Person100007846          373
yago:class/yago/Abstraction100002137     150
yago:class/yago/Communicator109610660    125
]]></programlisting>
<para>
The next repeat gets about double the counts, starting with 1291 entities.
</para>
<para>With a warm cache, the query finishes in about 300 ms (4 core Xeon, Virtuoso 6 Cluster) and returns:
</para>
<programlisting><![CDATA[
?c                                       COUNT (*)
yago:class/yago/Entity100001740          13329
yago:class/yago/PhysicalEntity100001930  10423
yago:class/yago/Object100002684          10408
yago:class/yago/Whole100003553           10210
yago:class/yago/LivingThing100004258      8868
yago:class/yago/Organism100004475         8868
yago:class/yago/CausalAgent100007347      8853
yago:class/yago/Person100007846           8853
yago:class/yago/Abstraction100002137      3284
yago:class/yago/Entertainer109616922      2356
]]></programlisting>
<para>It is a well known fact that running from memory is thousands of times faster than from disk.
</para>
<para>The query plan begins with the text search. The subjects with "Shakespeare" in some property get
dispatched to the partition that holds their class. Since all partitions know the class hierarchy,
the superclass inference runs in parallel, as does the aggregation of the group by. When all
partitions have finished, the process coordinating the query fetches the partial aggregates,
adds them up and sorts them by count.
</para>
<para>If a timeout occurs, it will most likely occur where the classes of the text matches are being
retrieved. When this happens, this part of the query is reset, but the aggregate states are left
in place. The process coordinating the query then goes on as if the aggregates had completed. If
there are many levels of nested aggregates, each timeout terminates the innermost aggregation that
is still accumulating results, thus a query is guaranteed to return in no more than n timeouts,
where n is the number of nested aggregations or subqueries.
</para>
  </sect3>
  <sect3 id="rdfiridereferencingfacetws"><title>Faceted Web Service and Linked Data</title>
<para>The Virtuoso Faceted Web Service is a general purpose RDF query facility for Faceted based browsing.
It takes an XML description of the view desired and generates the reply as an XML tree containing the
requested data. The user agent or a local web page can use XSLT for rendering this for the end user.
The selection of facets and values is represented as an XML tree. The rationale for this is the fact
that such a representation is easier to process in an application than the SPARQL source text or a
parse tree of SPARQL and more compactly captures the specific subset of SPARQL needed for faceted
browsing. All such queries internally generate SPARQL and the SPARQL generated is returned with
the results. One can therefore use this is a starting point for hand crafted queries.
</para>
<para>The query has the top level element. The child elements of this represents conditions pertaining
to a single subject. A join is expressed with the property or propertyof element. This has in turn
children which state conditions on a property of the first subject. Property and propertyof elements
can be nested to an arbitrary depth and many can occur inside one containing element. In this way,
tree-shaped structures of joins can be expressed.
</para>
<para>Expressing more complex relationships, such as intermediate grouping, subqueries, arithmetic or
such requires writing the query in SPARQL. The XML format is for easy automatic composition of queries
needed for showing facets, not a replacement for SPARQL.
</para>
<para>Consider composing a map of locations involved with Napoleon. Below we list user actions and
the resulting XML query descriptions.
</para>
<itemizedlist mark="bullet">
  <listitem>Enter in the search form "Napoleon":
<programlisting><![CDATA[
<query inference="" same-as="" view3="" s-term="e" c-term="type">
  <text>napoleon</text>
  <view type="text" limit="20" offset="" />
</query>
]]></programlisting>
</listitem>
  <listitem>Select the "types" view:
<programlisting><![CDATA[
<query inference="" same-as="" view3="" s-term="e" c-term="type">
  <text>napoleon</text>
  <view type="classes" limit="20" offset="0" location-prop="0" />
</query>
]]></programlisting>
</listitem>
  <listitem>Choose "MilitaryConflict" type:
<programlisting><![CDATA[
<query inference="" same-as="" view3="" s-term="e" c-term="type">
  <text>napoleon</text>
  <view type="classes" limit="20" offset="0" location-prop="0" />
  <class iri="yago:ontology/MilitaryConflict" />
</query>
]]></programlisting>
</listitem>
  <listitem>Choose "NapoleonicWars":
<programlisting><![CDATA[
<query inference="" same-as="" view3="" s-term="e" c-term="type">
  <text>napoleon</text>
  <view type="classes" limit="20" offset="0" location-prop="0" />
  <class iri="yago:ontology/MilitaryConflict" />
  <class iri="yago:class/yago/NapoleonicWars" />
</query>
]]></programlisting>
</listitem>
  <listitem>Select "any location" in the select list beside the "map" link; then hit "map" link:
<programlisting><![CDATA[
<query inference="" same-as="" view3="" s-term="e" c-term="type">
  <text>napoleon</text>
  <class iri="yago:ontology/MilitaryConflict" />
  <class iri="yago:class/yago/NapoleonicWars" />
  <view type="geo" limit="20" offset="0" location-prop="any" />
</query>
]]></programlisting>
</listitem>
</itemizedlist>
<para>This last XML fragment corresponds to the below text of SPARQL query:
</para>
<programlisting><![CDATA[
SPARQL
SELECT ?location AS ?c1
       ?lat1     AS ?c2
       ?lng1     AS ?c3
WHERE
  {
    ?s1        ?s1textp  ?o1                              .
    FILTER
      ( bif:contains (?o1, '"Napoleon"') )  .
    ?s1        a         <yago:ontology/MilitaryConflict>  .
    ?s1        a         <yago:class/yago/NapoleonicWars>  .
    ?s1        ?anyloc   ?location                         .
    ?location  geo:lat   ?lat1                             ;
               geo:long  ?lng1
  }
LIMIT 200
OFFSET 0
]]></programlisting>
<para>
The query takes all subjects with some literal property with "Napoleon" in it, then filters for
military conflicts and Napoleonic wars, then takes all objects related to these where the related
object has a location. The map has the objects and their locations.
</para>
  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtuosospongerfacent">Virtuoso Faceted Web Service</link></listitem>
      <listitem><link linkend="virtuosospongerfacentuirestapi">Virtuoso APIs for Faceted REST services</link></listitem>
    </itemizedlist>
  </tip>
  </sect3>
  <sect3 id="rdfiridereferencingfacetvd"><title>voiD Discoverability</title>
<para>A long awaited addition to the LOD cloud is the Vocabulary of Interlinked Data (voiD).
Virtuoso automatically generates voiD descriptions of data sets it hosts. Virtuoso incorporates an
SQL function <emphasis>rdf_void_gen</emphasis> which returns a Turtle representation of a given
graph's voiD statistics.
</para>
  </sect3>
  <sect3 id="rdfiridereferencingfacet"><title>Test System and Data</title>
<para>The test system consists of two 2x4 core Xeon 5345, 2.33 GHz servers with 16G RAM and 4 disks
each. The machines are connected by two 1Gbit Ethernet connections. The software is Virtuoso 6
Cluster. The Virtuoso server is split into 16 partitions, 8 for each machine. Each partition is
managed by a separate server process.
</para>
<para>The test database has the following data sets:
</para>
<itemizedlist mark="bullet">
  <listitem>DBpedia 3.2</listitem>
  <listitem>MusicBrainz</listitem>
  <listitem>Bio2RDF</listitem>
  <listitem>NeuroCommons</listitem>
  <listitem>UniProt</listitem>
  <listitem>Freebase (95M triples)</listitem>
  <listitem>PingTheSemanticWeb (1.6M miscellaneous files from http://www.pingthesemanticweb.com/).</listitem>
</itemizedlist>
<para>Ontologies:
</para>
<itemizedlist mark="bullet">
  <listitem>Yago</listitem>
  <listitem>OpenCyc</listitem>
  <listitem>Umbel</listitem>
  <listitem>DBpedia</listitem>
</itemizedlist>
<para>The database is 2.2 billion triples with 356 million distinct URIs.
</para>
  </sect3>
</sect2>
  <tip><title>See Also:</title>
    <itemizedlist mark="bullet">
      <listitem><link linkend="virtuosospongerfacetinstall">Virtuoso Faceted Browser Installation and configuration</link></listitem>
    </itemizedlist>
  </tip>
</sect1>


<sect1 id="rdfsparqlrule"><title>Inference Rules &amp; Reasoning</title>
<sect2 id="rdfsparqlruleintro"><title>Introduction</title>
<para>Virtuoso SPARQL can use an inference context for inferring triples that are not physically stored.
This functionality applies to physically stored quads and not to virtual triples generated from relational data with Linked Data Views.
Such an inference context can be built from one or more graphs containing RDF Schema triples. The supported
RDF Schema or OWL constraints are imported from these graphs and are grouped together into rule bases.
A rule base is a persistent entity that can be referenced by a SPARQL query or end point. Queries running
with a given rule base work as if the triples asserted by this rule base were included in the graph or graphs accessed by the query.
</para>
<para>As of version 5.0, Virtuoso recognizes <emphasis>rdfs:subClassOf</emphasis> and <emphasis>rdfs:subPropertyOf</emphasis>.
owl:sameAs is considered for arbitrary subjects and objects if specially enabled by a pragma in the query.
As of 5.00.3031, owl:sameAs, owl:equivalentClass and owl:equivalentProperty are also considered when determining subclass or subproperty relations.  If two  classes are equivalent, they share all instances, subclasses and superclasses directly or indirectly stated in the data for either class.
Other RDF Schema or OWL information is not taken into account.
</para>
</sect2>
<sect2 id="rdfsparqlrulemake"><title>Making Rule Sets</title>
  <para>Since RDF Schema and OWL schemas are RDF graphs, these can be loaded into the triple store. 
  	Thus, in order to use such a schema as query context, one first loads the corresponding document 
  	into the triple store using <link linkend="ttlp">ttlp()</link> or 
  	<link linkend="rdf_load_rdfxml">rdf_load_rdfxml()</link> or related functions. After the schema 
  	document is loaded, one can add the assertions there into an inference context with the 
  	<link linkend="rdfs_rule_set">rdfs_rule_set()</link> function. This function specifies a logical 
  	name for the rule set plus a graph URI. It is possible to combine multiple schema graphs into a 
  	single rule set. A single schema graph may also independently participate in multiple rule sets.
  </para>
  <para>The <emphasis>DB.DBA.SYS_RDF_SCHEMA</emphasis> table contains information for all 
  RDF rule sets in a Virtuoso instance. This table may be queried to, for instance, verify 
  <link linkend="rdfs_rule_set">rdfs_rule_set()</link> activity:
  </para>
<programlisting><![CDATA[
CREATE TABLE DB.DBA.SYS_RDF_SCHEMA (
  RS_NAME VARCHAR,  -- The name of the rdf rule set 
  RS_URI  VARCHAR,  -- The name of the graph 
  RS_G    VARCHAR,  -- Column for system usage only 
  PRIMARY KEY (RS_NAME, RS_URI))
)
]]></programlisting>  
</sect2>
<sect2 id="rdfsparqlrulechange"><title>Changing Rule Sets</title>
<para>Changing a rule set affects queries made after the change. Some queries may have been previously
compiled and will not be changed as a result of modifying the rule set. When a rule set is changed, i.e.
when <emphasis>rdfs_rule_set</emphasis> is called with the first argument set to a pre-existing rule set's name, all the graphs
associated with this name are read and the relevant facts are added to a new empty rule set. Thus, if
triples are deleted from or added to the graphs comprising the rule set, calling <emphasis>rdfs_rule_set</emphasis> will refresh
the rule set to correspond to the state of the stored graphs.
</para>
</sect2>
<sect2 id="rdfsparqlrulesubclassandsubprop"><title>Subclasses and Subproperties</title>
  <para>Virtuoso SPARQL supports RDF Schema subclasses and subproperties.</para>
<para>The predicates <emphasis>rdfs:subClassOf</emphasis> and <emphasis>rdfs:subPropertyOf</emphasis> are
recognized when they appear in graphs included in a rule set. When such a rule set is specified as a context
for a SPARQL query, the following extra triples are generated as needed.
</para>
<para>For every <emphasis>?s rdf:type ?class</emphasis>, a triple <emphasis>?s rdf:type ?superclass</emphasis> is considered to exist,
such that <emphasis>?superclass</emphasis> is a direct or indirect superclass of <emphasis>?class</emphasis>. Direct superclasses are
declared with the <emphasis>rdfs:subClassOf</emphasis> predicate in the rule set graph. Transitivity of superclasses
is automatically taken into account, meaning that if a is a superclass of b and b a superclass of c,
then a is a superclass of c also. Cyclic superclass relations are not allowed. If such occur in the rule set data,
the behavior is undefined but will not involve unterminating recursion.
</para>
<para>For every <emphasis>?s ?subpredicate ?o</emphasis>, a triple <emphasis>?s ?superpredicate ?o</emphasis>
is considered to exist if the rule context declares <emphasis>?superpredicate</emphasis> to be a superpredicate
of <emphasis>?predicate</emphasis>. This is done by having the triple <emphasis>?subpredicate rdfs:subPropertyOf ?superpredicate</emphasis>
as part of the graphs making up the rule context. Transitivity is observed, thus if a is a subpredicate of b and b
a subpredicate of c, then a is also a subpredicate of c.
</para>

  <para>Two methods can be used for typical recursions, transitivity on inference and plain transitive patterns 
(or subqueries).</para>
  <para>The advantage of inference is that queries are short and one inference rule set may be maintained for 
numerous queries.</para>
  <para>If queries are about trees of classes or properties, or about equivalences of nodes, consider using 
inference rule sets.</para>
  <para>Transitive patterns are inconvenient and may easily result in queries that runs too long or hard 
to debug, but they're unavoidable in traversing social networks or plain querying of RDF lists.</para>
  <para>So consider a rule set, a handful of nodes with classes from the rule set and a couple of RDF 
Lisp-style lists defined on demo.openlinksw.com:</para>
<programlisting><![CDATA[
SQL> SPARQL CLEAR GRAPH <http://example.com/2/owl>;
callret-0
VARCHAR
_______________________________________________________________________________

Clear <http://example.com/2/owl>  -- done

1 Rows. -- 0 msec.

SQL> TTLP (' @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  . 
  @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>  . 
  @prefix owl: <http://www.w3.org/2002/07/owl#>  . 
  @prefix e: <http://example.com/e/>  .   
  e:c1 rdfs:subClassOf e:c1or2 . 
  e:c2 rdfs:subClassOf e:c1or2 . 
  e:c1-10 rdfs:subClassOf e:c1 . 
  e:c1-20 rdfs:subClassOf e:c1 . 
  e:c2-30 rdfs:subClassOf e:c2 . 
  e:c2-40 rdfs:subClassOf e:c2 . 
  ', 'http://example.com/2/owl', 'http://example.com/2/owl' );

Done. -- 0 msec.
]]></programlisting>
   <para>You can also use the SPARUL equivalent variant:</para>
<programlisting><![CDATA[
SPARQL 
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>  
PREFIX owl: <http://www.w3.org/2002/07/owl#>  
PREFIX e: <http://example.com/e/>  

INSERT IN GRAPH <http://example.com/2/owl>
  {
    e:c1 rdfs:subClassOf e:c1or2 .
    e:c2 rdfs:subClassOf e:c1or2 .
    e:c1-10 rdfs:subClassOf e:c1 .
    e:c1-20 rdfs:subClassOf e:c1 .
    e:c2-30 rdfs:subClassOf e:c2 .
    e:c2-40 rdfs:subClassOf e:c2 .
  } ;
]]></programlisting>
   <para>Define the inference rule:</para>
<programlisting><![CDATA[
SQL> rdfs_rule_set ('http://example.com/2/owl', 'http://example.com/2/owl');

Done. -- 0 msec.

SQL> SPARQL CLEAR GRAPH <http://example.com/2/data> ;
callret-0
VARCHAR
_______________________________________________________________________________

Clear <http://example.com/2/data>  -- done

1 Rows. -- 0 msec.

SQL>  TTLP ('
  @prefix e: <http://example.com/e/>  .
  @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  .
  @prefix owl: <http://www.w3.org/2002/07/owl#>  .
  e:s1 a e:c1 ; e:p1 "Value of p1 for s1" .
  e:s2 a e:c2 ; e:p1 "Value of p1 for s2" .
  e:s1-10 a e:c1-10 ; e:p1 "Value of p1 for s1-10" .
  e:s1-20 a e:c1-20 ; e:p1 "Value of p1 for s1-20" .
  e:s2-30 a e:c2-30 ; e:p1 "Value of p1 for s2-30" .
  e:s2-40 a e:c2-40 ; e:p1 "Value of p1 for s2-40" .
  e:lists
       rdf:_1 ( e:list1-item1 e:list1-item2 e:list1-item3 ) ;
       rdf:_2 (
           [ e:p2 "Value of p2 of item1 of list2" ; e:p3 "Value of p3 of item1 of list2" ]
           [ e:p2 "Value of p2 of item2 of list2" ; e:p3 "Value of p3 of item2 of list2" ]
           [ e:p2 "Value of p2 of item3 of list2" ; e:p3 "Value of p3 of item3 of list2" ] ) .
  ', 'http://example.com/2/data', 'http://example.com/2/data' );

Done. -- 0 msec.
]]></programlisting>
  <para>You can also use the SPARUL equivalent variant:</para>
<programlisting><![CDATA[
SPARQL 
PREFIX e: <http://example.com/e/>  
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  
PREFIX owl: <http://www.w3.org/2002/07/owl#>  

INSERT IN GRAPH <http://example.com/2/data>
  {
    e:s1 a e:c1 ; e:p1 "Value of p1 for s1" .
    e:s2 a e:c2 ; e:p1 "Value of p1 for s2" .
    e:s1-10 a e:c1-10 ; e:p1 "Value of p1 for s1-10" .
    e:s1-20 a e:c1-20 ; e:p1 "Value of p1 for s1-20" .
    e:s2-30 a e:c2-30 ; e:p1 "Value of p1 for s2-30" .
    e:s2-40 a e:c2-40 ; e:p1 "Value of p1 for s2-40" .
    e:lists
      rdf:_1 ( e:list1-item1 e:list1-item2 e:list1-item3 ) ;
      rdf:_2 (
           [ e:p2 "Value of p2 of item1 of list2" ; e:p3 "Value of p3 of item1 of list2" ]
           [ e:p2 "Value of p2 of item2 of list2" ; e:p3 "Value of p3 of item2 of list2" ]
           [ e:p2 "Value of p2 of item3 of list2" ; e:p3 "Value of p3 of item3 of list2" ] )
  };
]]></programlisting>  
  <para>SPARQL DESCRIBE works fine with inference, deriving additional type information:</para>
<programlisting><![CDATA[
DEFINE input:inference <http://example.com/2/owl>
DESCRIBE <http://example.com/e/s1>
FROM <http://example.com/2/data>

fmtaggret-
LONG VARCHAR
_______________________________________________________________________________

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  .
@prefix ns1: <http://example.com/e/>  .
ns1:s1 rdf:type ns1:c1or2 ,
ns1:c1 ;
ns1:p1 "Value of p1 for s1" .

1 Rows. -- 0 msec.
]]></programlisting>

  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E%0D%0ADESCRIBE+%3Chttp%3A%2F%2Fexample.com%2Fe%2Fs1%3E%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E&amp;should-sponge=&amp;format=application%2Frdf%2Bxml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E%0D%0ADESCRIBE+%3Chttp%3A%2F%2Fexample.com%2Fe%2Fs1%3E%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E&amp;should-sponge=&amp;format=application%2Frdf%2Bxml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>
  
<programlisting><![CDATA[
DEFINE input:inference <http://example.com/2/owl>
DESCRIBE <http://example.com/e/s2> 
FROM <http://example.com/2/data>
fmtaggret-
LONG VARCHAR
_______________________________________________________________________________

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>  .
@prefix ns1: <http://example.com/e/>  .
ns1:s2 rdf:type ns1:c1or2 ,
ns1:c2 ;
ns1:p1 "Value of p1 for s2" .

1 Rows. -- 0 msec.
]]></programlisting>

  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E%0D%0ADESCRIBE+%3Chttp%3A%2F%2Fexample.com%2Fe%2Fs2%3E+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E&amp;should-sponge=&amp;format=application%2Frdf%2Bxml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E%0D%0ADESCRIBE+%3Chttp%3A%2F%2Fexample.com%2Fe%2Fs2%3E+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E&amp;should-sponge=&amp;format=application%2Frdf%2Bxml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>

  <para>Querying is simple as well:</para>

<programlisting><![CDATA[
SQL>SPARQL DEFINE input:inference <http://example.com/2/owl>  
PREFIX e:<http://example.com/e/>
SELECT * 
FROM <http://example.com/2/data> 
WHERE 
  { 
    ?s a e:c1or2 ; 
          e:p1 ?o 
  }
  
s                            o
VARCHAR                      VARCHAR
___________________________

http://example.com/e/s1      Value of p1 for s1
http://example.com/e/s1-10   Value of p1 for s1-10
http://example.com/e/s1-20   Value of p1 for s1-20
http://example.com/e/s2-30   Value of p1 for s2-30
http://example.com/e/s2-40   Value of p1 for s2-40
http://example.com/e/s2      Value of p1 for s2

6 Rows. -- 0 msec.
]]></programlisting>

  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E++%0D%0APREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+*+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E+%0D%0AWHERE+%0D%0A++{+%0D%0A++++%3Fs+a+e%3Ac1or2+%3B+%0D%0A++++++++++e%3Ap1+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E++%0D%0APREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+*+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E+%0D%0AWHERE+%0D%0A++{+%0D%0A++++%3Fs+a+e%3Ac1or2+%3B+%0D%0A++++++++++e%3Ap1+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>

<programlisting><![CDATA[
SQL>SPARQL DEFINE input:inference <http://example.com/2/owl>  
PREFIX e:<http://example.com/e/>
SELECT * FROM <http://example.com/2/data>  
WHERE 
  { 
    ?s a e:c1 ; 
       e:p1 ?o 
  }
  
s                            o
VARCHAR                      VARCHAR
___________________________

http://example.com/e/s1      Value of p1 for s1
http://example.com/e/s1-10   Value of p1 for s1-10
http://example.com/e/s1-20   Value of p1 for s1-20

3 Rows. -- 0 msec.
]]></programlisting>

  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E++%0D%0APREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+*+FROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{+%0D%0A++++%3Fs+a+e%3Ac1+%3B+%0D%0A+++++++e%3Ap1+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E++%0D%0APREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+*+FROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{+%0D%0A++++%3Fs+a+e%3Ac1+%3B+%0D%0A+++++++e%3Ap1+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>


  <para>However you should care about duplicates if both types and properties are queried: the join will 
  result in all combinations of types and property values.</para>

<programlisting><![CDATA[
SQL>SPARQL DEFINE input:inference <http://example.com/2/owl>  
PREFIX e:<http://example.com/e/>
SELECT * FROM <http://example.com/2/data>  
WHERE 
  { 
    ?s a ?t ; 
     e:p1 ?o 
  }
  
s                            t                            o
VARCHAR                      VARCHAR                      VARCHAR
___________________________

http://example.com/e/s1      http://example.com/e/c1      Value of p1 for s1
http://example.com/e/s1      http://example.com/e/c1or2   Value of p1 for s1
http://example.com/e/s1-10   http://example.com/e/c1-10   Value of p1 for s1-10
http://example.com/e/s1-10   http://example.com/e/c1      Value of p1 for s1-10
http://example.com/e/s1-10   http://example.com/e/c1or2   Value of p1 for s1-10
http://example.com/e/s1-20   http://example.com/e/c1-20   Value of p1 for s1-20
http://example.com/e/s1-20   http://example.com/e/c1      Value of p1 for s1-20
http://example.com/e/s1-20   http://example.com/e/c1or2   Value of p1 for s1-20
http://example.com/e/s2-30   http://example.com/e/c2-30   Value of p1 for s2-30
http://example.com/e/s2-30   http://example.com/e/c2      Value of p1 for s2-30
http://example.com/e/s2-30   http://example.com/e/c1or2   Value of p1 for s2-30
http://example.com/e/s2-40   http://example.com/e/c2-40   Value of p1 for s2-40
http://example.com/e/s2-40   http://example.com/e/c2      Value of p1 for s2-40
http://example.com/e/s2-40   http://example.com/e/c1or2   Value of p1 for s2-40
http://example.com/e/s2      http://example.com/e/c2      Value of p1 for s2
http://example.com/e/s2      http://example.com/e/c1or2   Value of p1 for s2

16 Rows. -- 0 msec.
]]></programlisting>

  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E++%0D%0APREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+*+FROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{+%0D%0A++++%3Fs+a+%3Ft+%3B+%0D%0A+++++e%3Ap1+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=DEFINE+input%3Ainference+%3Chttp%3A%2F%2Fexample.com%2F2%2Fowl%3E++%0D%0APREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+*+FROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{+%0D%0A++++%3Fs+a+%3Ft+%3B+%0D%0A+++++e%3Ap1+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>


  <para>Transitive queries are convenient as SPARQL 1.1 "predicate+" equivalent. The equivalent of 
  "predicate*" requires the use of a union:</para>

<programlisting><![CDATA[
SQL>SPARQL PREFIX e:<http://example.com/e/>
SELECT ?item 
FROM <http://example.com/2/data>  
WHERE 
  {
    { 
      ?lists rdf:_1 ?node 
    }
    UNION
    { 
      ?lists rdf:_1 ?l .
      ?l rdf:rest ?node option (transitive) . 
    }
    ?node rdf:first ?item 
  }
  
item
VARCHAR
_______________________________________________________________________________

http://example.com/e/list1-item1
http://example.com/e/list1-item2
http://example.com/e/list1-item3

3 Rows. -- 0 msec.
]]></programlisting>

  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=PREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+%3Fitem+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_1+%3Fnode+%0D%0A++++}%0D%0A++++UNION%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_1+%3Fl+.%0D%0A++++++%3Fl+rdf%3Arest+%3Fnode+option+%28transitive%29+.+%0D%0A++++}%0D%0A++++%3Fnode+rdf%3Afirst+%3Fitem+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=PREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+%3Fitem+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_1+%3Fnode+%0D%0A++++}%0D%0A++++UNION%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_1+%3Fl+.%0D%0A++++++%3Fl+rdf%3Arest+%3Fnode+option+%28transitive%29+.+%0D%0A++++}%0D%0A++++%3Fnode+rdf%3Afirst+%3Fitem+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>


<programlisting><![CDATA[
SQL> SPARQL PREFIX e:<http://example.com/e/>
SELECT ?p ?o 
FROM <http://example.com/2/data>  
WHERE 
  {
    { 
      ?lists rdf:_2 ?node 
    }
    UNION
    { 
      ?lists rdf:_2 ?l .
      ?l rdf:rest ?node option (transitive) . 
    }
    ?node rdf:first ?item .
    ?item ?p ?o 
  }
  
p                         o
VARCHAR                   VARCHAR
________________________

http://example.com/e/p2   Value of p2 of item1 of list2
http://example.com/e/p3   Value of p3 of item1 of list2
http://example.com/e/p2   Value of p2 of item2 of list2
http://example.com/e/p3   Value of p3 of item2 of list2
http://example.com/e/p2   Value of p2 of item3 of list2
http://example.com/e/p3   Value of p3 of item3 of list2

6 Rows. -- 0 msec.
]]></programlisting>


  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=PREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+%3Fp+%3Fo+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fnode+%0D%0A++++}%0D%0A++++UNION%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fl+.%0D%0A++++++%3Fl+rdf%3Arest+%3Fnode+option+%28transitive%29+.+%0D%0A++++}%0D%0A++++%3Fnode+rdf%3Afirst+%3Fitem+.%0D%0A++++%3Fitem+%3Fp+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=PREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+%3Fp+%3Fo+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fnode+%0D%0A++++}%0D%0A++++UNION%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fl+.%0D%0A++++++%3Fl+rdf%3Arest+%3Fnode+option+%28transitive%29+.+%0D%0A++++}%0D%0A++++%3Fnode+rdf%3Afirst+%3Fitem+.%0D%0A++++%3Fitem+%3Fp+%3Fo+%0D%0A++}&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>


  <para>Note that the result set can be in order of items in the list, but it don't have to. 
  If the order should be preserved, then fix the direction of transitive scan, get step number as a 
  variable, order by that variable.</para>

<programlisting><![CDATA[
-- Line 82:
SQL> SPARQL PREFIX e:<http://example.com/e/>
SELECT ?p ?o bif:coalesce(?step_no, 0) 
FROM <http://example.com/2/data>  
WHERE 
  {
    { 
      ?lists rdf:_2 ?node 
    }
    UNION
    { 
      ?lists rdf:_2 ?l .
      ?l rdf:rest ?node OPTION (transitive, t_direction 1, t_step("step_no") as ?step_no) . 
    }
    ?node rdf:first ?item .
    ?item ?p ?o 
  } 
ORDER BY ASC (?step_no)

p                         o                               callret-2
VARCHAR                   VARCHAR                         VARCHAR
________________________

http://example.com/e/p2   Value of p2 of item1 of list2   0
http://example.com/e/p3   Value of p3 of item1 of list2   0
http://example.com/e/p2   Value of p2 of item2 of list2   1
http://example.com/e/p3   Value of p3 of item2 of list2   1
http://example.com/e/p2   Value of p2 of item3 of list2   2
http://example.com/e/p3   Value of p3 of item3 of list2   2

6 Rows. -- 7 msec.
]]></programlisting>


  <para>Example links against <ulink url="http://demo.openlinksw.com/sparql">Virtuoso Demo Server SPARQL Endpoint</ulink> with SPARQl Protocol URLs:</para>
  <itemizedlist mark="bullet">
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;query=PREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+%3Fp+%3Fo+bif%3Acoalesce%28%3Fstep_no%2C+0%29+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fnode+%0D%0A++++}%0D%0A++++UNION%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fl+.%0D%0A++++++%3Fl+rdf%3Arest+%3Fnode+OPTION+%28transitive%2C+t_direction+1%2C+t_step%28%22step_no%22%29+as+%3Fstep_no%29+.+%0D%0A++++}%0D%0A++++%3Fnode+rdf%3Afirst+%3Fitem+.%0D%0A++++%3Fitem+%3Fp+%3Fo+%0D%0A++}+%0D%0AORDER+BY+ASC+%28%3Fstep_no%29%0D%0A&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View results page</ulink></listitem>
    <listitem><ulink url="http://demo.openlinksw.com/sparql?default-graph-uri=&amp;qtxt=PREFIX+e%3A%3Chttp%3A%2F%2Fexample.com%2Fe%2F%3E%0D%0ASELECT+%3Fp+%3Fo+bif%3Acoalesce%28%3Fstep_no%2C+0%29+%0D%0AFROM+%3Chttp%3A%2F%2Fexample.com%2F2%2Fdata%3E++%0D%0AWHERE+%0D%0A++{%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fnode+%0D%0A++++}%0D%0A++++UNION%0D%0A++++{+%0D%0A++++++%3Flists+rdf%3A_2+%3Fl+.%0D%0A++++++%3Fl+rdf%3Arest+%3Fnode+OPTION+%28transitive%2C+t_direction+1%2C+t_step%28%22step_no%22%29+as+%3Fstep_no%29+.+%0D%0A++++}%0D%0A++++%3Fnode+rdf%3Afirst+%3Fitem+.%0D%0A++++%3Fitem+%3Fp+%3Fo+%0D%0A++}+%0D%0AORDER+BY+ASC+%28%3Fstep_no%29%0D%0A&amp;should-sponge=&amp;format=text%2Fhtml&amp;CXML_redir_for_subjs=121&amp;CXML_redir_for_hrefs=&amp;timeout=0&amp;debug=on">View editor page</ulink></listitem>
  </itemizedlist>


</sect2>
<sect2 id="rdfsameas"><title>OWL sameAs Support</title>
<para>
Virtuoso has limited support for the OWL sameAs predicate.
</para>
<para>
If sameAs traversal is enabled and a triple pattern with a given
subject or object is being matched, all the synonyms of the S and O
will be tried and results generated for all the tried bindings of S
and O. The set of synonyms is generated at run time by following all
owl:sameAs triples where the IRI in question is either the subject or
the object. These are followed recursively from object to subject and
subject to object until the complete transitive closure is generated.
All sameAs triples from all the graphs applicable to instantiating
the triple pattern at hand are considered.
</para>
<para>
Thus for example:
</para>
<para>The inital SPARQL query:</para>
<programlisting><![CDATA[
SQL>SPARQL
prefix foaf: <http://xmlns.com/foaf/0.1/>
prefix owl: <http://www.w3.org/2002/07/owl#>
prefix sioc: <http://rdfs.org/sioc/ns#>
SELECT *
from <http://example.com/dataspace>
where
 {
    ?person a foaf:Person FILTER REGEX(?person ,"http://example.com/dataspace/person/kidehen#this").
    ?person foaf:name ?name .
    ?person owl:sameAs ?sameas .
  }
limit 8;
person                                               name             sameas
VARCHAR                                              VARCHAR          VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://my.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://kidehen.idehen.net/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://qdos.com/user/e922b748a2eb667bf37b188018002dec
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://knowee.net/kidehen/ids/id3684976382
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://dbpedia.org/resource/Kingsley_Idehen
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://identi.ca/user/14092
http://example.com/dataspace/person/kidehen#this  Kingsley Idehen  http://myopenlink.net/proxy?url=http%3A%2F%2Fwww.facebook.com%2Fpeople%2FKingsley_Idehen%2F605980750&force=rdf&login=kidehen

8 Rows. -- 181 msec.
]]></programlisting>
<para>So if we have:</para>
<programlisting><![CDATA[
<http://example.com/dataspace/person/kidehen#this>  	<http://www.w3.org/2002/07/owl#sameAs> <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this> .
<http://example.com/dataspace/person/kidehen#this>  	<http://xmlns.com/foaf/0.1/name>  	   Kingsley Idehen
]]></programlisting>
<para>
and we instantiate <emphasis>?s &lt;http://xmlns.com/foaf/0.1/name> "Kingsley Idehen"</emphasis>
we get <emphasis>?s</emphasis> bound to <emphasis>&lt;http://example.com/dataspace/person/kidehen#this></emphasis>.
</para>
<para>
If we instantiate <emphasis>&lt;http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
&lt;http://xmlns.com/foaf/0.1/name> ?l</emphasis>
we get <emphasis>?l</emphasis> bound to <emphasis>"Kingsley Idehen"</emphasis> because the subject was given and it was expanded to its synonyms.
</para>




<para>
If binding a variable in a pattern where the variable was free, we do not expand the value to the complete set of its synonyms.
</para>
<para>
Same-as expansion is enabled in a query by <emphasis>define input:same-as "yes"</emphasis> in the beginning of the SPARQL query.
This has a significant run time cost but is in some cases useful when joining data between sets which are mapped to each other with same-as.
</para>
<para>
We note that the number of same-as expansions will depend on the join order used for the SPARQL query.
The compiler does not know the number of synonyms and cannot set the join order accordingly.
Regardless of the join order we will however get at least one IRI of the each synonym set as answer.
Also when interactively navigating a graph with a browser, the same-as expansion will take all synonyms into account.
</para>
<para>
For getting the complete entailment of same-as, a forward
chaining approach should be used, effectively asserting all the
implied triples.
</para>
<sect3 id="rdfsameasexmp"><title>OWL sameAs Example</title>
<programlisting><![CDATA[
SQL>SPARQL
DEFINE input:same-as "yes"
SELECT *
WHERE
 {
   ?s <http://xmlns.com/foaf/0.1/name> "Kingsley Idehen" .
 }
LIMIT 10;

s
VARCHAR
___________________________________________________
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
http://demo.openlinksw.com/dataspace/kingsley#person
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
http://example.com/dataspace/person/kidehen#this
No. of rows in result: 10
]]></programlisting>
</sect3>
</sect2>
<sect2 id="rdfsparqlruleimpl"><title>Implementation</title>
<para>Triples entailed by subclass or subproperty statements in an inference context are not physically stored.
Such triples are added to the result set by the query run time as needed. Also queries involving subclass or subproperty
rules are not rewritten into unions of all the possible triple patterns that might imply the pattern that is requested.
Instead, the SQL compiler adds special nodes that iterate over subclasses or subproperties at run time. The cost model
also takes subclasses and subproperties into account when determining the approximate cardinality of triple patterns.
</para>
<para>In essence, Virtuoso's support for subclasses and subproperties is backward chaining, i.e. it does not materialize
all implied triples but rather looks for the basic facts implying these triples at query evaluation time.
</para>
</sect2>
<sect2 id="rdfsparqlruleenableinfr"><title>Enabling Inferencing</title>
<para>In a SPARQL query, the define input:inference clause is used to instruct the compiler to use the rules in the named rule set. For example:
</para>
<programlisting>
SQL> rdfs_rule_set ('sample', 'rule_graph');

SQL> SPARQL
define input:inference "sample"
SELECT *
FROM &lt;g&gt;
WHERE {?s ?p ?o};
</programlisting>
<para>will include all the implied triples in the result set, using the rules in the sample rule set.
</para>
<para>Inference can be enabled triple pattern by triple pattern. This is done with the option
(inference 'rule_set') clause after the triple pattern concerned. Specifying option (inference none)
will disable inference for the pattern concerned while the default inference context applies to the
rest of the patterns. Note that the keyword is input:inference in the query header and simply inference
in the option clause. See the examples section below for examples.
</para>
<para>In SQL, if RDF_QUAD occurs in a select from clause, inference can be added with the table option <emphasis>WITH</emphasis>, as follows:
</para>
<programlisting>
SPARQL
SELECT *
FROM rdf_quad table OPTION (with 'sample')
WHERE g = iri_to_id ('xx', 0);
</programlisting>
<para>This is about the same as:
</para>
<programlisting>
SPARQL
define input:inference "sample"
SELECT *
FROM &lt;xx&gt;
WHERE {?s ?p ?o}
</programlisting>
</sect2>
<sect2 id="rdfsparqlruleexamples"><title>Examples</title>
<sect3 id="rdfsparqlruleexamples1"><title>Example for loading data space instance data Triples
into a Named Graph for schema/ontology data</title>
<para>The following example shows how to load data space instance data Triples
into a Named Graph: &lt;http://example.com/test&gt;, for schema/ontology data called:
&lt;http://example.com/schema/test&gt; that expresses assertions about subclasses and
subproperties.</para>
<programlisting><![CDATA[
ttlp ('
   <http://example.com/dataspace>  	                        <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  <http://rdfs.org/sioc/ns#Space>.
   <http://example.com/dataspace/test2/weblog/test2tWeblog>  <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  <http://rdfs.org/sioc/types#Weblog> .
   <http://example.com/dataspace/discussion/oWiki-test1Wiki> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  <http://rdfs.org/sioc/types#MessageBoard>.
   <http://example.com/dataspace>                            <http://rdfs.org/sioc/ns#link>                     <http://example.com/ods> .
   <http://example.com/dataspace/test2/weblog/test2tWeblog>  <http://rdfs.org/sioc/ns#link>                     <http://example.com/dataspace/test2/weblog/test2tWeblog>.
   <http://example.com/dataspace/discussion/oWiki-test1Wiki> <http://rdfs.org/sioc/ns#link>                     <http://example.com/dataspace/discussion/oWiki-test1Wiki> .
   ', '', 'http://example.com/test');
]]></programlisting>
<programlisting><![CDATA[
ttlp (' @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
  <http://rdfs.org/sioc/ns#Space> rdfs:subClassOf <http://www.w3.org/2000/01/rdf-schema#Resource> .
  <http://rdfs.org/sioc/ns#Container> rdfs:subClassOf <http://rdfs.org/sioc/ns#Space> .
  <http://rdfs.org/sioc/ns#Forum> rdfs:subClassOf <http://rdfs.org/sioc/ns#Container> .
  <http://rdfs.org/sioc/types#Weblog> rdfs:subClassOf <http://rdfs.org/sioc/ns#Forum> .
  <http://rdfs.org/sioc/types#MessageBoard> rdfs:subClassOf <http://rdfs.org/sioc/ns#Forum> .
  <http://rdfs.org/sioc/ns#link> rdfs:subPropertyOf <http://rdfs.org/sioc/ns> .
  ', '', 'http://example.com/schema/test');
]]></programlisting>
<programlisting>
rdfs_rule_set ('http://example.com/schema/property_rules1', 'http://example.com/schema/test');
</programlisting>
<para>This defines the rule context http://example.com/schema/property_rules1 that is initialized
from the contents of graph http://example.com/schema/test.
</para>
<programlisting><![CDATA[
SQL>SPARQL
define input:inference "http://example.com/schema/property_rules1"
SELECT ?s
FROM <http://example.com/test>
WHERE {?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  <http://rdfs.org/sioc/ns#Space> };
s
VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/test2/weblog/test2tWeblog
http://example.com/dataspace/discussion/oWiki-test1Wiki
http://example.com/dataspace

3 Rows. -- 0 msec.
]]></programlisting>
<para>This returns the instances of http://rdfs.org/sioc/ns#Space. Since http://rdfs.org/sioc/types#Weblog
and http://rdfs.org/sioc/types#MessageBoard are subclasses of http://rdfs.org/sioc/ns#Space,
instances of http://rdfs.org/sioc/ns#Space, http://rdfs.org/sioc/types#Weblog and
http://rdfs.org/sioc/types#MessageBoard are all returned. This results in the subjects
http://example.com/dataspace, http://example.com/dataspace/test2/weblog/test2tWeblog and
http://example.com/dataspace/discussion/oWiki-test1Wiki.
</para>
<programlisting><![CDATA[
SQL>SELECT id_to_iri (s)
FROM rdf_quad table option (with 'http://example.com/schema/property_rules1')
WHERE g = iri_to_id ('http://example.com/test',0)
  AND p = iri_to_id ('http://www.w3.org/1999/02/22-rdf-syntax-ns#type', 0)
  AND o = iri_to_id ('http://rdfs.org/sioc/ns#Space', 0);
callret
VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/test2/weblog/test2tWeblog
http://example.com/dataspace/discussion/oWiki-test1Wiki
http://example.com/dataspace

3 Rows. -- 10 msec.
]]></programlisting>
<para>This is the corresponding SQL query, internally generated by the SPARQL query.</para>
<para>Below we first look for all instances of http://rdfs.org/sioc/ns#Space with some
property set to http://example.com/dataspace/test2/weblog/test2tWeblog.
We get the subject http://example.com/dataspace/test2/weblog/test2tWeblog and the
properties http://rdfs.org/sioc/ns#link and http://rdfs.org/sioc/ns.
The join involves both subclass and subproperty inference. Then we turn off the inference
for the second pattern and only get the property http://rdfs.org/sioc/ns#link. Then we do
the same but now specify that inference should apply only to the first triple pattern.
</para>
<programlisting><![CDATA[
SQL>SPARQL
define input:inference  "http://example.com/schema/property_rules1"
SELECT *
FROM <http://example.com/test>
WHERE
  {
    ?s ?p <http://rdfs.org/sioc/ns#Space> .
    ?s ?p1 <http://example.com/dataspace/test2/weblog/test2tWeblog> .
  };

s             p              p1
VARCHAR       VARCHAR       VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/test2/weblog/test2tWeblog  http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://rdfs.org/sioc/ns#link
http://example.com/dataspace/test2/weblog/test2tWeblog  http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://rdfs.org/sioc/ns

2 Rows. -- 0 msec.


SQL>SPARQL
SELECT *
FROM <http://example.com/test>
WHERE
  {
    ?s ?p <http://rdfs.org/sioc/ns#Space> OPTION (inference 'http://example.com/schema/property_rules1') .
    ?s ?p1 <http://example.com/dataspace/test2/weblog/test2tWeblog> .
  };

s             p              p1
VARCHAR       VARCHAR        VARCHAR
_______________________________________________________________________________

http://example.com/dataspace/test2/weblog/test2tWeblog  http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://rdfs.org/sioc/ns#link

1 Rows. -- 10 msec.

]]></programlisting>
</sect3>
<sect3 id="rdfsparqlruleexamples2"><title>DBpedia example</title>
<programlisting><![CDATA[
ttlp ('
 prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
 <http://dbpedia.org/property/birthcity> rdfs:subPropertyOf <http://dbpedia.org/property/birthPlace> .
 <http://dbpedia.org/property/birthcountry> rdfs:subPropertyOf  <http://dbpedia.org/property/birthPlace> .
 <http://dbpedia.org/property/cityofbirth> rdfs:subPropertyOf <http://dbpedia.org/property/birthPlace> .
 <http://dbpedia.org/property/countryofbirth> rdfs:subPropertyOf <http://dbpedia.org/property/birthPlace> .
 <http://dbpedia.org/property/countyofbirth> rdfs:subPropertyOf <http://dbpedia.org/property/birthPlace> .
 <http://dbpedia.org/property/cityofdeath> rdfs:subPropertyOf <http://dbpedia.org/property/deathPlace> .
 <http://dbpedia.org/property/countryofdeath> rdfs:subPropertyOf <http://dbpedia.org/property/deathPlace> . ', '',
 'http://dbpedia.org/inference/rules#') ;

rdfs_rule_set ('http://dbpedia.org/schema/property_rules1', 'http://dbpedia.org/inference/rules#');
]]></programlisting>
<programlisting><![CDATA[
SQL>SPARQL
define input:inference "http://dbpedia.org/schema/property_rules1"
prefix p: <http://dbpedia.org/property/>
SELECT ?s
FROM <http://dbpedia.org>
WHERE {?s p:birthcity ?o }
LIMIT 50
s
VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Britt_Janyk
http://dbpedia.org/resource/Chiara_Costazza
http://dbpedia.org/resource/Christoph_Gruber
http://dbpedia.org/resource/Daron_Rahlves
http://dbpedia.org/resource/Finlay_Mickel
http://dbpedia.org/resource/Genevi%C3%A8ve_Simard
http://dbpedia.org/resource/Johann_Grugger
http://dbpedia.org/resource/Kalle_Palander
http://dbpedia.org/resource/Marc_Gini
http://dbpedia.org/resource/Mario_Scheiber
http://dbpedia.org/resource/Prince_Hubertus_of_Hohenlohe-Langenburg
http://dbpedia.org/resource/Resi_Stiegler
http://dbpedia.org/resource/Steven_Nyman
http://dbpedia.org/resource/Hannes_Reichelt
http://dbpedia.org/resource/Jeremy_Transue

15 Rows. -- 167 msec.

SQL>SPARQL
define input:inference "http://dbpedia.org/schema/property_rules1"
prefix p: <http://dbpedia.org/property/>
SELECT ?s
FROM <http://dbpedia.org>
WHERE {?s p:countryofbirth ?o }
LIMIT 50
s
VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/A._J._Wood
http://dbpedia.org/resource/A._J._Godbolt
http://dbpedia.org/resource/Ac%C3%A1cio_Casimiro
http://dbpedia.org/resource/Adam_Fry
http://dbpedia.org/resource/Adam_Gilchrist
http://dbpedia.org/resource/Adam_Griffin
http://dbpedia.org/resource/Adam_Gross
...

50 Rows. -- 324 msec.

SQL>SPARQL
define input:inference "http://dbpedia.org/schema/property_rules1"
prefix p: <http://dbpedia.org/property/>
SELECT ?s
FROM <http://dbpedia.org>
WHERE {?s p:countyofbirth ?o }
LIMIT 50

s
VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Eddie_Colman

1 Rows. -- 163 msec.

SQL>SPARQL
define input:inference "http://dbpedia.org/schema/property_rules1"
prefix p: <http://dbpedia.org/property/>
SELECT ?s
FROM <http://dbpedia.org>
WHERE {?s p:birthPlace ?o }

s
VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Eddie_Colman
http://dbpedia.org/resource/Jeremy_Transue
http://dbpedia.org/resource/Finlay_Mickel
http://dbpedia.org/resource/Prince_Hubertus_of_Hohenlohe-Langenburg
http://dbpedia.org/resource/Hannes_Reichelt
http://dbpedia.org/resource/Johann_Grugger
http://dbpedia.org/resource/Chiara_Costazza
...
155287 Rows. -- 342179 msec.

]]></programlisting>
</sect3>
<sect3 id="rdfsparqlruleexamples3"><title>Example for loading script of the Yago Class hierarchy as inference rules</title>
<programlisting><![CDATA[
--- Load Class Hierarchy into a Named Graph
SELECT ttlp_mt (file_to_string_output ('yago-class-hierarchy_en.nt'),
'', 'http://dbpedia.org/resource/classes/yago#');

-- Create an  Inference Rule that references the Yago Class Hierarchy
Named Graph

SQL>rdfs_rule_set ('http://dbpedia.org/resource/inference/rules/yago#',
'http://dbpedia.org/resource/classes/yago#');

-- Query for the "The Lord of the Rings" which is a "Fantasy Novel" as explicitly
-- claimed in the DBpedia data set (instance data)

SQL>SPARQL
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dbpedia: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>
SELECT ?s
FROM <http://dbpedia.org>
WHERE
  {
  ?s a <http://dbpedia.org/class/yago/FantasyNovels> .
  ?s dbpedia:name "The Lord of the Rings"@en .
  };

s
VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/The_Lord_of_the_Rings

1 Rows. -- 241 msec.

-- Query aimed at Novel via query scoped to the "Fiction" class of
-- which it is a subclass in the Yago Hierarchy
SQL>SPARQL
define input:inference "http://dbpedia.org/resource/inference/rules/yago#"
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dbpedia: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>

SELECT ?s
FROM <http://dbpedia.org>
WHERE {
?s a <http://dbpedia.org/class/yago/Fiction106367107> .
?s dbpedia:name "The Lord of the Rings"@en .
};

s
VARCHAR
_______________________________________________________________________________


http://dbpedia.org/resource/The_Lord_of_the_Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings

4 Rows. -- 4767 msec.


-- # Variant of query with Virtuoso's Full Text Index extension: bif:contains
SQL>SPARQL
define input:inference "http://dbpedia.org/resource/inference/rules/yago#"
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dbpedia: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>

SELECT ?s ?n
FROM <http://dbpedia.org>
WHERE {
?s a <http://dbpedia.org/class/yago/Fiction106367107> .
?s dbpedia:name ?n .
?n bif:contains 'Lord and Rings'
};

s                                                    n
VARCHAR                                              VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/The_Lord_of_the_Rings    The Lord of the Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings    The Lord of the Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings    The Lord of the Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings    The Lord of the Rings

4 Rows. -- 5538 msec.

-- Retrieve all individuals instances of the FantasyNovels Class
SQL>SPARQL
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dbpedia: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>

SELECT ?s ?n
FROM <http://dbpedia.org>
WHERE
{
  ?s a <http://dbpedia.org/class/yago/FantasyNovels> .
  ?s dbpedia:name ?n .
}
limit 10;
s                                                                                 n
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/ATLA_-_A_Story_of_the_Lost_Island                     Atla
http://dbpedia.org/resource/A_Crown_of_Swords                                     A Crown of Swords
http://dbpedia.org/resource/A_Game_of_Thrones                                     A Game of Thrones
http://dbpedia.org/resource/A_Secret_Atlas                                        A Secret Atlas
http://dbpedia.org/resource/A_Storm_of_Swords                                     A Storm of Swords
http://dbpedia.org/resource/A_Voyage_to_Arcturus                                  A Voyage to Arcturus
http://dbpedia.org/resource/A_Wizard_Alone                                        A Wizard Alone
http://dbpedia.org/resource/Above_the_Veil                                        Above the Veil
http://dbpedia.org/resource/Black_Easter                                          Black Easter
http://dbpedia.org/resource/Lord_of_Chaos                                         Lord of Chaos

10 Rows. -- 781 msec.

-- Retrieve all individuals instances of Fiction Class which should
-- include all Novels.
SQL>SPARQL
define input:inference "http://dbpedia.org/resource/inference/rules/yago#"
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dbpedia: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>

SELECT ?s ?n
FROM <http://dbpedia.org>
WHERE {
?s a <http://dbpedia.org/class/yago/Fiction106367107> .
?s dbpedia:name ?n .
};
s                                                                                 n
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Last_Son_of_Krypton                                   Last Son of Krypton
http://dbpedia.org/resource/Tuvaluan_language                                     Tuvaluan
http://dbpedia.org/resource/Card_Walker                                           E. Cardon Walker
http://dbpedia.org/resource/Les_Clark                                             Les Clark
http://dbpedia.org/resource/Marc_Davis                                            Marc Davis
http://dbpedia.org/resource/Eric_Larson                                           Eric Larson
http://dbpedia.org/resource/Marty_Sklar                                           Marty Sklar
http://dbpedia.org/resource/Peter_Ellenshaw                                       Peter Ellenshaw
http://dbpedia.org/resource/Adriana_Caselotti                                     Adriana Caselotti
http://dbpedia.org/resource/Jimmie_Dodd                                           Jimmie Dodd
...
15296 Rows.
]]></programlisting>
</sect3>
<sect3 id="rdfsparqlruleexamples4"><title>Pure SPARQL Example</title>
<programlisting><![CDATA[
-- Query aimed at Fantasy Novel via query scoped to the "Fiction" class of
-- which it is a subclass in the Yago Hierarchy

SQL>SPARQL
define input:inference "http://dbpedia.org/resource/inference/rules/yago#"
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dbpedia: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>
SELECT ?s
FROM <http://dbpedia.org>
WHERE {
?s a <http://dbpedia.org/class/yago/Fiction106367107> .
?s dbpedia:name "The Lord of the Rings"@en .
};
s
VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/The_Lord_of_the_Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings
http://dbpedia.org/resource/The_Lord_of_the_Rings

4 Rows. -- 150 msec.
]]></programlisting>
</sect3>
<sect3 id="rdfsparqlruleexamples5"><title>Example for equivalence between classes</title>
<para>This example is based on <ulink url="http://umbel.org/documentation.html">UMBEL</ulink> and
DBpedia integration:</para>
<programlisting><![CDATA[
-- Load UMBEL & DBpedia Instance Level Cross-Links (owl:sameAs) Triples
SELECT ttlp_mt (file_to_string_output ('umbel_dbpedia_linkage_v071.n3'), '', 'http://dbpedia.org');

-- Load UMBEL and DBpedia Type (rdf:type) association Triples
SELECT ttlp_mt (file_to_string_output ('umbel_dbpedia_types_v071.n3'), '', 'http://dbpedia.org');

--- Load UMBEL Subject Concept Class Hierarchy into a Named Graph
SELECT ttlp_mt (file_to_string_output ('umbel_class_hierarchy_v071.n3'), '', 'http://dbpedia.org/resource/classes/umbel#');

--- load UMBEL Subject Concepts Instance Data
SELECT ttlp_mt (file_to_string_output ('umbel_subject_concepts.n3'), '', 'http://dbpedia.org/resource/classes/umbel#');

--- Load UMBEL Abstract Concepts Instance Data
SELECT ttlp_mt (file_to_string_output ('umbel_abstract_concepts.n3'), '', 'http://dbpedia.org/resource/classes/umbel#');


-- Load UMBEL External Ontology Mapping into a Named Graph
SELECT ttlp_mt (file_to_string_output ('umbel_external_ontologies_linkage.n3'), '', 'http://dbpedia.org/resource/classes/umbel#');

-- Create UMBEL Inference Rules

rdfs_rule_set ('http://dbpedia.org/resource/inference/rules/umbel#', 'http://dbpedia.org/resource/classes/umbel#');
]]></programlisting>
<para>Now let's execute the following queries:</para>
<programlisting><![CDATA[
SQL>SPARQL define input:inference "http://dbpedia.org/resource/inference/rules/umbel#"
prefix umbel: <http://umbel.org/umbel/sc/>
PREFIX dbpedia: <http://dbpedia.org/property/>
prefix opencyc: <http://sw.opencyc.org/2008/06/10/concept/en/>
SELECT ?s
where
{
 ?s a opencyc:Motorcycle.
 ?s dbpedia:name ?n.
 ?n bif:contains "BMW".
};

s
____________________________________________
http://dbpedia.org/resource/BMW_K1200GT
http://dbpedia.org/resource/BMW_F650CS
http://dbpedia.org/resource/BMW_C1
http://dbpedia.org/resource/BMW_R75
4 Rows. -- 26 msec.
]]></programlisting>
<programlisting><![CDATA[
SQL>SPARQL define input:inference "http://dbpedia.org/resource/inference/rules/umbel#"
prefix umbel: <http://umbel.org/umbel/sc/>
PREFIX dbpedia: <http://dbpedia.org/property/>
prefix opencyc: <http://sw.opencyc.org/2008/06/10/concept/en/>
SELECT ?s
where
{
 ?s a umbel:Motorcycle.
 ?s dbpedia:name ?n.
 ?n bif:contains "BMW".
};

s
____________________________________________
http://dbpedia.org/resource/BMW_K1200GT
http://dbpedia.org/resource/BMW_F650CS
http://dbpedia.org/resource/BMW_C1
http://dbpedia.org/resource/BMW_R75
4 Rows. -- 26 msec.
]]></programlisting>
</sect3>
<sect3 id="rdfsparqlruleexamples6"><title>Example for finding celebrities which are not fans of their own fans</title>
<programlisting><![CDATA[
SQL>SPARQL
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
INSERT INTO GRAPH <urn:rules.skos> { foaf:knows rdfs:subPropertyOf sioc:follows . };

callret-0
VARCHAR
Insert into <urn:rules.skos>, 1 triples -- done
No. of rows in result: 1

SQL>rdfs_rule_set ('foaf-trans', 'urn:rules.skos');

Done.

SPARQL>SPARQL
DEFINE input:inference "foaf-trans"
PREFIX sioc: <http://rdfs.org/sioc/ns#>
SELECT ?celeb COUNT (*)
WHERE
  {
    ?claimant sioc:follows ?celeb .
    FILTER
      (
        !bif:exists
         (
           ( SELECT (1)
             WHERE
               {
                 ?celeb sioc:follows ?claimant
               }
           )
         )
      )
  }
GROUP BY ?celeb
ORDER BY DESC 2
LIMIT 10

celeb                                                                            callret-1
ANY 	                                                                         ANY
__________________________________________________________________________________________
http://localhost.localdomain/about/id/entity/http/twitter.com/kidehen	         100
http://localhost.localdomain/about/id/entity/http/twitter.com/shawnafennell	 77
http://localhost.localdomain/about/id/entity/http/twitter.com/thines01	         71
http://localhost.localdomain/about/id/entity/http/twitter.com/mhausenblas	 50
http://localhost.localdomain/about/id/entity/http/twitter.com/DirDigEng	         2
http://localhost.localdomain/about/id/entity/http/twitter.com/SarahPalinUSA	 1
http://localhost.localdomain/about/id/entity/http/twitter.com/zbrox	         1
http://localhost.localdomain/about/id/entity/http/twitter.com/LamarLee	         1
http://localhost.localdomain/about/id/entity/http/twitter.com/HackerChick	 1
http://localhost.localdomain/about/id/entity/http/twitter.com/programmingfeed	 1
No. of rows in result: 10
]]></programlisting>
</sect3>
</sect2>
<!-- Uncommented for Virtuoso 6 Release-->
  <sect2 id="rdfsparqlruleinversefunc"><title>Identity With Inverse Functional Properties</title>
  <para>A graph used used with rdfs_rule_set may declare certain properties to
  be inversely functional. If one or more inverse functional properties (IFP's)
  are declared in the inference context used with the query, enabled
  with define input:inference = "context_name", then the following
  semantics apply:</para>
  <orderedlist>
  <listitem>If a literal is compared with an IRI, then the literal is substituted by all the subject IRI's where this literal occurs as a value of an IFP.</listitem>
  <listitem>If two IRI's are compared for equality, they will be considered the same if there is an IFP P such that the same P has the same value on both subjects.</listitem>
  <listitem>If an IRI is processed for distinctness in either distinct or group by, the IRI is first translated to be the IRI with the lowest ID among all IRI's that share an IFP value with this IRI.</listitem>
  </orderedlist>
  <para>Thus, if two IRI's are compared for distinctness, they will count as one if there is an IFP P with the same value with both IRI's. Literal data types are not translated into IRI's even if these literals occurred as IFP values of some subject.</para>
  <para>It is possible to declare that specific values, even if they occur as values of an IFP in more than onme subject do not constitute identity between the subjects.
  For example, if two subjects were inferred to be the same because they had the same foaf:mbox_sha1sum, the SHA1 hash of mailto:// would be excluded. Two individuals have an email address that has a common default value are not the same.
  </para>
  <para>In an ontology graph, a property IRI is declared to be inversely
  functional by making it an instance of the
  owl:InverseFunctionalProperty class. A value of an IFP can be
  declared null, i.e. sharing the value does not imply identity by by
  giving the IFP IRI a
  &lt;http://www.openlinksw.com/schemas/virtrdf#nullIFPValue> property with
  the value to be ignored as the object.
  </para>
  <emphasis>Example</emphasis>
  <programlisting><![CDATA[
  SQL>ttlp ('
  <john1> a <person> .
  <john2> a <person> .
  <mary> a <person> .
  <mike> a <person> .
  <john1> <name> "John" .
  <john2> <name> "John" .
  <john1> <address> "101 A street" .
  <john2> <address> "102 B street" .
  <john2> <knows> <mike> .
  <john1> <http://www.w3.org/2002/07/owl#sameAs> <john2> .
  <mary> <knows> "John" .
  <mike> <knows> <john1> .
  <mike> <knows> <john2> .
  <john1> <name> "Tarzan" .
  <mike> <nam> "Tarzan" .
  ', '', 'ifps');


  SQL>ttlp ('
  <name> a <http://www.w3.org/2002/07/owl#InverseFunctionalProperty> .
  <name> <http://www.openlinksw.com/schemas/virtrdf#nullIFPValue> "Tarzan" .
  ', '', 'ifp_list');

  SQL>rdfs_rule_set ('ifps', 'ifp_list');

  SQL>SPARQL define input:inference "ifps"  SELECT * FROM <ifps> WHERE {<john1> ?p ?o};

  p                                                                                 o
  VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

  address                                                                           101 A street
  name                                                                              John
  http://www.w3.org/2002/07/owl#sameAs                                              john2
  http://www.w3.org/1999/02/22-rdf-syntax-ns#type                                   person
  name                                                                              Tarzan
  name                                                                              John
  knows                                                                             mike
  http://www.w3.org/1999/02/22-rdf-syntax-ns#type                                   person
  address                                                                           102 B street
  ]]></programlisting>
  <para>We see that we get the properties of &lt;john2> also.</para>
  <programlisting><![CDATA[
  SQL>SPARQL define input:inference "ifps" SELECT distinct ?p FROM <ifps> WHERE { ?p a <person>};

  john2
  mike
  mary
]]></programlisting>
  <para>We see that we get only one John. But John is not the same as Mike
  because they share the name Tarzan which is not considered as implying
  identity. Which John we get is a matter of which gets the lowest
  internal ID. This is variable and arbitrary at load time but once
  loaded this is permanent as long as the set of subjects with the name
  John does not change.</para>
  </sect2>
  <sect2 id="rdfsparqlruletransoption"><title>Inference Rules and SPARQL with Transitivity Option</title>
<itemizedlist mark="bullet">
  <listitem>See <link linkend="rdfsparqlimplementatiotransexamples7">example</link> with an
inference rule to cater data being skos:broader based, which is no longer transitive.</listitem>
  <listitem>See <link linkend="rdfsparqlimplementatiotransexamples8">example</link> with an
inference rule to find entities that are subcategories of Protestant Churches, no deeper
than 3 levels within the concept scheme hierarchy, filtered by a specific subcategory.</listitem>
</itemizedlist>
  </sect2>
  <sect2 id="rdfsparqlruleowlrelation"><title>Inference Rules, OWL Support and Relationship Ontology</title>
    <para>This section provides queries usage for inference rules, owl support and Relationship Vocabulary.</para>
    <sect3 id="rdfsparqlruleowlrelationexample1"><title>Example 1</title>
<para>Example based on Relationship Vocab:</para>
<programlisting><![CDATA[
## Verify Ontology Data is in Quad Store
## Ontology: <http://vocab.org/relationship/> (Relationship Ontology)
## Use pragma to put latest in Quad store.

DEFINE get:soft "replace"
SELECT *
FROM <http://vocab.org/relationship/>
WHERE {?s ?p ?o}

## Clean up

CLEAR GRAPH <urn:owl.tests>

## Create Instance Data for Relationship Ontology
PREFIX rel: <http://purl.org/vocab/relationship/>

INSERT into GRAPH <urn:owl.tests>
{
    <http://dbpedia.org/resource/Prince_William_of_Wales> rel:siblingOf <http://dbpedia.org/resource/Prince_Harry_of_Wales>.
    <http://dbpedia.org/resource/Elizabeth_Bowes-Lyon> rel:ancestorOf <http://dbpedia.org/resource/Elizabeth_II_of_the_United_Kingdom>.
    <http://dbpedia.org/resource/Elizabeth_II_of_the_United_Kingdom> rel:ancestorOf
    <http://dbpedia.org/resource/Charles%2C_Prince_of_Wales>.
    <http://dbpedia.org/resource/Charles%2C_Prince_of_Wales> rel:ancestorOf <http://dbpedia.org/resource/Prince_William_of_Wales>.
};

## Verify

SELECT *
FROM  <urn:owl.tests>
WHERE
  {
    ?s ?p ?o
  }

## Create an  Inference Rule that references the Relationship Ontology Named Graph

rdfs_rule_set ('urn:owl.tests', 'http://vocab.org/relationship') ;

## Verify Rule's existence

SELECT * FROM SYS_RDF_SCHEMA ;
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlruleowlrelationexample2"><title>Example 2</title>
<programlisting><![CDATA[
## Test owl:TransitiveProperty Reasoning
## Start with a specific URI
## Goal: See inferred Triples
## In this case, relationship between: <http://dbpedia.org/resource/Elizabeth_Bowes-Lyon>
## and her descendants: Queen Elizabeth, Prince Charles, Prince William, and Prince Harry)


DEFINE input:inference 'urn:owl.tests'
PREFIX rel: <http://purl.org/vocab/relationship/>
SELECT *
FROM <urn:owl.tests>
WHERE
  {
    <http://dbpedia.org/resource/Elizabeth_Bowes-Lyon> rel:ancestorOf ?o
  }
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlruleowlrelationexample3"><title>Example 3</title>
<programlisting><![CDATA[
## Test owl:SymmetricalProperty Reasoning
## Should show same result irrespective of rel:siblingOf URI in Subject or Object slots of Triple

DEFINE input:inference 'urn:owl.tests'
PREFIX rel: <http://purl.org/vocab/relationship/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>

SELECT *
FROM <urn:owl.tests>
WHERE
  {
    <http://dbpedia.org/resource/Prince_William_of_Wales> rel:siblingOf ?o
  }

## OR

DEFINE input:inference 'urn:owl.tests'
PREFIX rel: <http://purl.org/vocab/relationship/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>

SELECT *
FROM <urn:owl.tests>
WHERE
  {
    ?s rel:siblingOf <http://dbpedia.org/resource/Prince_William_of_Wales>
  }
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlruleowlrelationexample4"><title>Example 4</title>
<programlisting><![CDATA[
## Test owl:inverseOf Reasoning
## Should show triples exposing the inverseOf relation.
## In this case rel:ancestorOf instance data triples exist,so the system must infer rel:descendant Of triples

DEFINE input:inference 'urn:owl.tests'
PREFIX rel: <http://purl.org/vocab/relationship/>

SELECT *
FROM <urn:owl.tests>
WHERE
{
    <http://dbpedia.org/resource/Elizabeth_II_of_the_United_Kingdom> rel:descendantOf ?o
}

## OR with Transitivity Option applied

DEFINE input:inference 'urn:owl.tests'
PREFIX rel: <http://purl.org/vocab/relationship/>

SELECT *
FROM <urn:owl.tests>
WHERE
  {
    <http://dbpedia.org/resource/Prince_William_of_Wales> rel:descendantOf ?o
    OPTION (T_DISTINCT)
  }
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlruleowlrelationexample5"><title>Example 5</title>
<programlisting><![CDATA[
## Test owl:inverseOf Reasoning
## Should show triples exposing the inverseOf relation.
## In this case rel:employedBy instance data triples exist,
## the system must infer rel:employerOf triples.

DEFINE input:inference 'urn:owl.tests'
PREFIX rel: <http://purl.org/vocab/relationship/>

SELECT *
FROM <urn:owl.tests>
WHERE
  {
    ?s rel:employerOf ?o
  }
]]></programlisting>
</sect3>
    <sect3 id="rdfsparqlruleowlrelationexample6"><title>Example 6</title>
<para>Example based on Relationship Vocab and SKOS</para>
<para>Assume the following test10.rdf file:</para>
<programlisting><![CDATA[
<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
	xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
	xmlns:daml="http://www.daml.org/2001/03/daml+oil#"
	xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
	xmlns:skos="http://www.w3.org/2004/02/skos/core#"
	xmlns:owl="http://www.w3.org/2002/07/owl#"
	xmlns:turnguard="http://www.turnguard.com#"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:sea="http://www.2sea.org/"
	xmlns:foaf="http://xmlns.com/foaf/0.1/"
	xmlns:dct="http://purl.org/dc/terms/">

<rdf:Description rdf:about="http://www.turnguard.com/Music">
	<rdf:type rdf:resource="http://www.w3.org/2004/02/skos/core#Concept"/>
	<skos:prefLabel xml:lang="en">Music</skos:prefLabel>
	<skos:narrower rdf:resource="http://www.turnguard.com/Pop" />
</rdf:Description>
<rdf:Description rdf:about="http://www.turnguard.com/Pop">
      <rdf:type rdf:resource="http://www.w3.org/2004/02/skos/core#Concept"/>
		<skos:prefLabel xml:lang="en">POP</skos:prefLabel>
      <skos:narrower rdf:resource="http://www.turnguard.com/TechnoPop" />
</rdf:Description>
<rdf:Description rdf:about="http://www.turnguard.com/TechnoPop">
        <rdf:type rdf:resource="http://www.w3.org/2004/02/skos/core#Concept"/>
        <skos:prefLabel xml:lang="en">TECHNOPOP</skos:prefLabel>
        <skos:narrower rdf:resource="http://www.turnguard.com/ElectroPop" />
</rdf:Description>

<rdf:Description rdf:about="http://www.turnguard.com/ElectroPop">
        <rdf:type rdf:resource="http://www.w3.org/2004/02/skos/core#Concept"/>
        <skos:prefLabel xml:lang="en">ELECTROPOP</skos:prefLabel>
</rdf:Description>
<rdf:Description rdf:about="http://www.turnguard.com/KrautRock">
        <rdf:type rdf:resource="http://www.w3.org/2004/02/skos/core#Concept"/>
        <skos:prefLabel xml:lang="en">KrautRock</skos:prefLabel>
	<skos:related rdf:resource="http://www.turnguard.com/ElectroPop" />
</rdf:Description>
</rdf:RDF>

]]></programlisting>
<para>Execute the following steps:</para>
<programlisting><![CDATA[
## Graph Cleanup
CLEAR GRAPH <urn:owl.test2.tbox>
CLEAR GRAPH <http://turnguard.com/virtuoso/test10.rdf>

## Load Instance Data into Quad Store
## PL Procedure

## SQL realm
DB.DBA.RDF_LOAD_RDFXML
  (
    http_get('http://www.w3.org/2009/08/skos-reference/skos-owl1-dl.rdf'),
    'no',
    'urn:owl.test2.tbox'
  );
DB.DBA.RDF_LOAD_RDFXML
  (
    http_get ('http://www.w3.org/2002/07/owl.rdf'),
    'no',
    'urn:owl.test2.tbox'
  );
DB.DBA.RDF_LOAD_RDFXML
  (
    file_to_string ('test10.rdf'),
    'no',
    'http://turnguard.com/virtuoso/test10.rdf'
  );



SELECT *
FROM <http://www.w3.org/2004/02/skos/core>
WHERE
  {
    {
      <http://www.w3.org/2004/02/skos/core#related> ?p ?o
    }
    UNION
    {
      ?s ?p <http://www.w3.org/2004/02/skos/core#related>
    }
  }


## Create Rules
## SQL Realm

rdfs_rule_set ('urn:owl.test2.rules', 'urn:owl.test2.tbox');


## Transitivity Query re. SKOS concept hierarchy

DEFINE input:inference "urn:owl.test2.rules"
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>

SELECT *
FROM <http://turnguard.com/virtuoso/test10.rdf>
WHERE
{
    <http://www.turnguard.com/ElectroPop> skos:broaderTransitive ?o
    OPTION  (T_DISTINCT).
  }
]]></programlisting>
    </sect3>
  </sect2>
</sect1>

<sect1 id="rdfsparqlgeospat"><title>RDF and Geometry</title>
<para>A geometry may occur as an object of an RDF quad. The SQL MM functions can then
	be used for geospatial queries.
</para>
<para>
For geometry functions, see the <link linkend="sqlrefgeospatial">SQL Geometry support section</link>.
</para>
<para>A geometry may occur as an object value in an RDF quad. In such a
case, the bare geometry object is not used but instead a special RDF
typed literal is made with the type virtrdf:Geometry. Such a literal
is automatically indexed in an R tree index containing all distinct
geometries occurring in any quad of any graph under any predicate.
Normally, WGS84, SRID 4326 is the SRID of any such geometry.
</para>
<para>In this section, the geo namespace prefix is used to mean &lt;http://www.w3.org/2003/01/geo/wgs84_pos#&gt;.
</para>
<para>The preferred way of adding geometries to RDF graphs is with the ttlp and related functions
which parse a text string in the Turtle syntax and insert the result in a graph.
</para>
<para>For example:</para>
<programlisting><![CDATA[
ttlp ('@prefix virtrdf: <http://www.openlinksw.com/schemas/virtrdf#>
<point> geo:geometry "point(1.2 22.4"^^virtrdf:Geometry .',
'xxx', 'graph');
]]></programlisting>
<para>A typed literal whose text is a WKT representation of a geometry and whose type is virtrdf:geometry
creates a geometry object and adds it to the R tree index of all RDF geometries.
</para>
<para>
Geometries can be queried with geometry predicates, st_intersects, st_contains and st_within, as follows.
As usual, the bif: namespace is used since these are SQL built-in functions.
</para>
<programlisting><![CDATA[
SQL>
SPARQL
SELECT ?c COUNT (*)
WHERE
{
    ?m geo:geometry ?geo .
    ?m a ?c .
    FILTER (bif:st_intersects (?geo, bif:st_point (0, 52), 100))
  }
GROUP BY ?c
ORDER BY DESC 2;

c  	                                            callret-1
VARCHAR                                             VARCHAR
____________________________________________________________
http://linkedgeodata.org/vocabulary#node 	    2317684
http://linkedgeodata.org/vocabulary#way 	    85315
http://linkedgeodata.org/vocabulary#building 	    14257
http://dbpedia.org/class/yago/Landmark108624891     9093
http://linkedgeodata.org/vocabulary#wood 	    7155
http://linkedgeodata.org/vocabulary#gate 	    7079
http://www.w3.org/2002/07/owl#Thing 	            6788
http://linkedgeodata.org/vocabulary#post_box 	    6144
http://linkedgeodata.org/vocabulary#pub 	    5697
http://dbpedia.org/ontology/Place 	            5670
http://linkedgeodata.org/vocabulary#hedge 	    5391
...
]]></programlisting>
<para>
This would return the classes of things within 100 km of 0, 52, which is near London.
</para>
<programlisting><![CDATA[
SQL>
SPARQL
SELECT ?m (bif:st_distance (?geo, bif:st_point (0, 52)))
             WHERE
               {
    ?m geo:geometry ?geo .
    ?m a <http://dbpedia.org/ontology/City> .
    FILTER (bif:st_intersects (?geo, bif:st_point (0, 52), 30))
  }
ORDER BY DESC 2
LIMIT 20;

m                                                                                 callret-1
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            39.13180985471543
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            39.13180985471543
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            39.13180985471543
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            39.13180985471543
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            37.36907252285992
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            34.49432513061792
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            33.7676326404143
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            33.24238654570499
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            32.60139660515003
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            32.60139660515003
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            32.17414911350438
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            31.45681319171456
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            31.17750625349044
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            31.115377038
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            31.115377038
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            30.56388658524301
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            29.89662974046085
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            29.85090625132639
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            29.82605254366244
http://dbpedia.org/resource/Kingston%2C_Cambridgeshire                            29.60102064794003

20 Rows. -- 13600 msec.

]]></programlisting>
<para>This would be the cities within 20 km of 0, 52, ordered by increasing distance from this point.
</para>
<para>When SPARQL is called from SQL, the geometries can be bound to SQL
variables as anything else returned from SPARQL. The <emphasis>st_</emphasis> functions
can then be used for retrieving properties of these objects.
</para>
  <tip><title>See Also</title>
    <para><link linkend="fn_st_point"><function>st_point</function></link></para>
    <para><link linkend="fn_st_x"><function>st_x</function></link></para>
    <para><link linkend="fn_st_y"><function>st_y</function></link></para>
    <para><link linkend="fn_st_distance"><function>st_distance</function></link></para>
    <para><link linkend="fn_ST_SRID"><function>ST_SRID</function></link></para>
    <para><link linkend="fn_ST_SetSRID"><function>ST_SetSRID</function></link></para>
    <para><link linkend="fn_st_astext"><function>st_astext</function></link></para>
    <para><link linkend="fn_st_geomfromtext"><function>st_geomfromtext</function></link></para>
    <para><link linkend="fn_st_contains"><function>st_contains</function></link></para>
    <para><link linkend="fn_st_intersects"><function>st_intersects</function></link></para>
    <para><link linkend="fn_st_within"><function>st_within</function></link></para>
    <para><link linkend="fn_isgeometry"><function>isgeometry</function></link></para>
    <para><link linkend="fn_geo_insert"><function>geo_insert</function></link></para>
    <para><link linkend="fn_geo_delete"><function>geo_delete</function></link></para>
  </tip>
  <sect2 id="rdfsparqlgeospatprog"><title>Programmatic Manipulation of Geometries in RDF</title>
<para>The <link linkend="fn_ttlp"><function>ttlp</function></link> function is the preferred
way of inserting geometries. The more are inserted at one time, the more efficient the operation is.
This loader function will also deal with cluster message optimization.
</para>
<para>For deleting quads with geometries, normal
<link linkend="rdfsparqlimplementationextent">SPARUL operations</link> apply.
</para>
<para>A geometry occurring in an RDF quad object is a member of the RDF box data type. This data type
stands for a typed RDF literal. The type of all geometries is 256. This is mapped to a URI in
the RDF_DATATYPE system table.
</para>
<para>A geometry does not occur directly in the object position of a quad. It is referenced by an id
that is stored in the RDF typed literal box and references RO_ID of the RDF_OBJ system table.
To translate a geometry into a RDF box that can be stored, do as in the example below:
</para>
<programlisting><![CDATA[
INSERT INTO RDF_QUAD (g, s, p, o)
VALUES (
         "g",
         "s",
         iri_to_id ('http://www.w3.org/2003/01/geo/wgs84_pos#geometry'),
         DB.DBA.rdf_geo_add (rdf_box (st_point (lng, lat), 256, 257, 0, 1)));
]]></programlisting>
<para>The DB.DBA.RDF_GEO_ADD function looks if an identical geometry already exists and if so assigns the
existing id to it. If the geometry is new, it gets a new ID and is stored in the RDF literals
table RDF_OBJ. At this time it is also automatically inserted into the RDF geometry index.
</para>
<para>In a cluster situation one should use the dpipe mechanism for inserting into RDF quad so as to
get large numbers of inserts into a single message. This is essential for performance.
</para>
</sect2>
  <sect2 id="rdfsparqlgeospatcrg"><title>Creating Geometries From RDF Data</title>
<para>Many data sets use the geo:lat and geo:long properties for describing a position.
Virtuoso comes with a function for converting these properties into geometries. This operation
reads through all graphs and for each subject with at least one geo:lat and geo:long, a point
geometry is made for each distinct lat/long pair where lat and long are in the same graph. It
should not happen in practice that a single subject has multiple lats or long within one graph.
If this still happens, a geometry is made for each combination.
The geometry is added to the subject with the lat and long as the value of the geo:geometry property.
This is added to the same graph where the lat and long were.
  </para>
<para>The SQL procedure DB.DBA.RDF_GEO_FILL () performs this operation. This is performed in parallel on
multiple threads and is optimized for cluster execution. This is done without transaction logging
and is not transactional. To make the result persistent, the operator should do an explicit checkpoint.
This is done by executing:</para>
<programlisting><![CDATA[
SQL>cl_exec ('checkpoint');
]]></programlisting>
<para>on any process of a cluster or single server.
Otherwise the result may be lost if the server terminates abnormally before an automatic checkpoint is made.
  </para>
<para>The DB.DBA.RDF_GEO_FILL procedure may in principle be called several times but it will read
every lat and long in the database. This is inefficient if there are large numbers of geometries.
</para>
<para>Application logic must generally be used for constructing geometries and adding these to RDF subjects.
It is easiest for the application to construct a text representation of the geometries in TTL and to
use the <link linkend="fn_ttlp"><function>ttlp</function></link> function for loading this.
</para>
  </sect2>
  <sect2 id="rdfsparqlgeospatusg"><title>Using Geometries With Existing Databases</title>
<para>
The geometry feature is compatible with any Virtuoso 6 databases.
Once geometries are used, the database should not be opened with a
server older than the one used for first inserting geometries, older
servers will consider the storage format a physical corruption.
</para>
  </sect2>
  <sect2 id="rdfsparqlgeospatexmp"><title>GEO Spatial Examples</title>
    <sect3 id="rdfsparqlgeospatexmp1"><title>Example 1</title>
  <programlisting><![CDATA[
## Get All Stuff For Given Coordinates
SQL>SPARQL
SELECT ?c COUNT (*)
WHERE
   {
     ?m geo:geometry ?geo .
     ?m a ?c .
     FILTER (bif:st_intersects (?geo, bif:st_point (0, 52), 100))
   }
GROUP BY ?c
ORDER BY desc 2;

c                                                                                 callret-1
  VARCHAR                                                                           VARCHAR
  _______________________________________________________________________________

http://linkedgeodata.org/vocabulary#node                                          2317684
http://linkedgeodata.org/vocabulary#way                                           85315
http://linkedgeodata.org/vocabulary#building                                      14257
http://dbpedia.org/class/yago/Landmark108624891                                   9093
http://linkedgeodata.org/vocabulary#wood                                          7155
....
  ]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp2"><title>Example 2</title>
  <programlisting><![CDATA[
## Get City Stuff Around Catholic Churches In Paris
SQL>
SPARQL
SELECT ?m (bif:st_distance (?geo, bif:st_point (0, 52)))
WHERE
  {
    ?m geo:geometry ?geo .
    ?m a <http://dbpedia.org/ontology/City> .
    FILTER (bif:st_intersects (?geo, bif:st_point (0, 52), 30))
  }
ORDER BY DESC 2
LIMIT 20;
m                                                                                 callret-1
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Stansted_Mountfitchet                                 39.13180985471543
http://dbpedia.org/resource/Stansted_Mountfitchet                                 39.13180985471543
http://dbpedia.org/resource/Stansted_Mountfitchet                                 39.13180985471543
http://dbpedia.org/resource/Stansted_Mountfitchet                                 39.13180985471543
http://dbpedia.org/resource/Stansted_Mountfitchet                                 37.36907252285992
http://dbpedia.org/resource/Stansted_Mountfitchet                                 34.49432513061792
http://dbpedia.org/resource/Stansted_Mountfitchet                                 33.7676326404143
http://dbpedia.org/resource/Stansted_Mountfitchet                                 33.24238654570499
http://dbpedia.org/resource/Stansted_Mountfitchet                                 32.60139660515003
http://dbpedia.org/resource/Stansted_Mountfitchet                                 32.60139660515003
http://dbpedia.org/resource/Stansted_Mountfitchet                                 31.45681319171456
http://dbpedia.org/resource/Stansted_Mountfitchet                                 31.115377038
http://dbpedia.org/resource/Stansted_Mountfitchet                                 31.115377038
http://dbpedia.org/resource/Stansted_Mountfitchet                                 30.56388658524301
http://dbpedia.org/resource/Stansted_Mountfitchet                                 29.89662974046085
http://dbpedia.org/resource/Stansted_Mountfitchet                                 29.85090625132639
http://dbpedia.org/resource/Stansted_Mountfitchet                                 29.82605254366244
http://dbpedia.org/resource/Stansted_Mountfitchet                                 29.60102064794003
http://dbpedia.org/resource/Stansted_Mountfitchet                                 29.44147385851453
http://dbpedia.org/resource/Stansted_Mountfitchet                                 29.421242437379
  ]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp3"><title>Example 3</title>
<programlisting><![CDATA[
## Get City Stuff Around Catholic Churches In Paris Extended
SQL>
SPARQL
SELECT ?m (bif:st_distance (?geo, bif:st_point (0, 52)))
WHERE
  {
    ?m geo:geometry ?geo .
    ?m a <http://dbpedia.org/ontology/City> .
    FILTER (bif:st_intersects (?geo, bif:st_point (0, 52), 100))
  }
ORDER BY DESC 2
LIMIT 20;
m                                                                                 callret-1
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Weston-on-Trent                                       138.7082197019335
http://dbpedia.org/resource/Weston-on-Trent                                       137.7213767969613
http://dbpedia.org/resource/Weston-on-Trent                                       136.4597167847218
http://dbpedia.org/resource/Weston-on-Trent                                       134.1807668663677
http://dbpedia.org/resource/Weston-on-Trent                                       133.104337839536
http://dbpedia.org/resource/Weston-on-Trent                                       133.104337839536
http://dbpedia.org/resource/Nonington                                             132.7368236183588
http://dbpedia.org/resource/Nonington                                             132.1339163200362
http://dbpedia.org/resource/Nonington                                             132.1339163200362
http://dbpedia.org/resource/Nonington                                             130.5478483560461
http://dbpedia.org/resource/Nonington                                             130.1620410981843
http://dbpedia.org/resource/Nonington                                             129.8549842943355
http://dbpedia.org/resource/Nonington                                             129.6459280567849
http://dbpedia.org/resource/Nonington                                             129.4504858595742
http://dbpedia.org/resource/Nonington                                             129.2790713235814
http://dbpedia.org/resource/Nonington                                             128.9081040147881
http://dbpedia.org/resource/Nonington                                             128.8845164618929
http://dbpedia.org/resource/Nonington                                             128.6676189617872
http://dbpedia.org/resource/Nonington                                             128.2565253458452
http://dbpedia.org/resource/Nonington                                             128.2551696344652

20 Rows. -- 120 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp4"><title>Example 4</title>
<programlisting><![CDATA[
## Text Or Geo
SQL>
SPARQL
SELECT ?c COUNT (*)
WHERE
  {
    ?m geo:geometry ?geo .
    ?m a ?c .
    FILTER (bif:st_intersects (?geo, bif:st_point (0, 52), 100)  &&  REGEX (str (?c), "London") )
  }
GROUP BY ?c
ORDER BY DESC 2
LIMIT 10;

c  	                                                        callret-1
____________________________________________________________________________

http://dbpedia.org/class/yago/DistrictsOfLondon 	            861
http://dbpedia.org/class/yago/GradeIListedBuildingsInLondon 	    199
http://dbpedia.org/class/yago/MuseumsInLondon 	                    107
http://dbpedia.org/class/yago/ArtMuseumsAndGalleriesInLondon 	    92
http://dbpedia.org/class/yago/GradeIIListedBuildingsInLondon 	    89
http://dbpedia.org/class/yago/SportsVenuesInLondon 	            80
http://dbpedia.org/class/yago/RoyalBuildingsInLondon 	            72
http://dbpedia.org/class/yago/LondonOvergroundStations 	            69
http://dbpedia.org/class/yago/NationalGovernmentBuildingsInLondon   69
http://dbpedia.org/class/yago/SkyscrapersInLondon 	            60
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp5"><title>Example 5</title>
<programlisting><![CDATA[
## Example "Places Of Worship, Within 5 km Of Paris":

## Describes places of worship, within 5 km of Paris,
## that have cafes in close proximity(0.2 km).
## The query requires V6 or higher.
SQL>
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
DESCRIBE ?cafe ?church
WHERE
  {
    ?church a lgv:place_of_worship .
    ?church geo:geometry ?churchgeo .
    ?church lgv:name ?churchname .
    ?cafe a lgv:cafe .
    ?cafe lgv:name ?cafename .
    ?cafe geo:geometry ?cafegeo .
    ?cafe geo:lat ?lat .
    ?cafe geo:long ?long .
    FILTER ( bif:st_intersects ( ?churchgeo, bif:st_point ( 2.3498, 48.853 ), 5 ) &&
             bif:st_intersects ( ?cafegeo, ?churchgeo, 0.2 ) )
  }
LIMIT 10;

@prefix rdf:	<http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:	<http://linkedgeodata.org/triplify/node/243360870#> .
@prefix ns2:	<http://linkedgeodata.org/vocabulary#> .
ns1:id	rdf:type	ns2:place_of_worship ,
		ns2:node .
@prefix geo:	<http://www.w3.org/2003/01/geo/wgs84_pos#> .
ns1:id	geo:lat	48.8794 ;
	geo:long	2.3748 ;
	ns2:created_by	"Potlatch 0.6c" ;
	ns2:name	"Saint-Georges de la Villette" ;
	ns2:religion	"christian" ,
		ns2:christian .
@prefix virtrdf:	<http://www.openlinksw.com/schemas/virtrdf#> .
ns1:id	geo:geometry	"POINT(2.3748 48.8794)"^^virtrdf:Geometry .
@prefix ns5:	<http://linkedgeodata.org/triplify/node/266632049#> .
ns5:id	rdf:type	ns2:node ,
		ns2:cafe ;
	geo:lat	48.8518 ;
	geo:long	2.325 ;
	ns2:created_by	"Potlatch 0.9a" ;
	ns2:name	"Le Babylone" ;
	geo:geometry	"POINT(2.325 48.8518)"^^virtrdf:Geometry .
....
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp6"><title>Example 6</title>
<programlisting><![CDATA[
## Count Geo
SQL>
SPARQL
SELECT ?c COUNT (*)
WHERE
  {
     ?s geo:geometry ?geo .
     FILTER (bif:st_intersects (?geo, bif:st_point (2.3498, 48.853), 5)) .
     ?s a ?c
   }
GROUP BY ?c
ORDER BY desc 2
LIMIT 10;

c                                                                                 callret-1
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://linkedgeodata.org/vocabulary#node                                          37792
http://dbpedia.org/class/yago/Landmark108624891                                   4003
http://linkedgeodata.org/vocabulary#way                                           1688
http://linkedgeodata.org/vocabulary#building                                      719
http://linkedgeodata.org/vocabulary#station                                       257
http://linkedgeodata.org/vocabulary#post_box                                      247
http://www.w3.org/2002/07/owl#Thing                                               227
http://linkedgeodata.org/vocabulary#park                                          208
http://linkedgeodata.org/vocabulary#restaurant                                    198
http://dbpedia.org/ontology/Place                                                 192

10 Rows. -- 932 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp7"><title>Example 7</title>
<programlisting><![CDATA[
## Get Stuff Around Notre Dame De Paris
SQL>
SPARQL
SELECT ?c COUNT (*)
WHERE
  {
    ?s a ?c .
    ?s geo:geometry ?geo .
    FILTER (bif:st_intersects (?geo, bif:st_point (2.3498, 48.853), 0.3))
  }
GROUP BY ?c
ORDER BY desc 2
LIMIT 10;

c                                                                                 callret-1
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://linkedgeodata.org/vocabulary#node                                          408
http://dbpedia.org/class/yago/Landmark108624891                                   134
http://linkedgeodata.org/vocabulary#way                                           17
http://dbpedia.org/class/yago/RomanCatholicChurchesInParis                        17
http://dbpedia.org/class/yago/TallBuildingsAndStructuresInParis                   13
http://dbpedia.org/class/yago/CathedralsInFrance                                  13
http://sw.opencyc.org/2008/06/10/concept/Mx4rvVigPpwpEbGdrcN5Y29ycA               13
http://sw.opencyc.org/2008/06/10/concept/Mx4rjm5QanS6EdaAAACgyZzFrg               13
http://sw.opencyc.org/2008/06/10/concept/Mx4rwQwtGpwpEbGdrcN5Y29ycA               13
http://www.w3.org/2002/07/owl#Thing                                               10

10 Rows. -- 241 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp8"><title>Example 8</title>
<programlisting><![CDATA[
## Things within 10 km proximity of place of worship
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
SELECT ?c COUNT (*)
WHERE
  {
    ?s a ?c .
    ?s a lgv:place_of_worship .
    ?s geo:geometry ?geo .
    FILTER (bif:st_intersects (?geo, bif:st_point (2.3498, 48.853), 10))
  }
GROUP BY ?c
ORDER BY desc 2
LIMIT 10;

c                                                                                 callret-1
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://linkedgeodata.org/vocabulary#place_of_worship                              147
http://linkedgeodata.org/vocabulary#node                                          146
http://linkedgeodata.org/vocabulary#way                                           46
http://linkedgeodata.org/vocabulary#building                                      36
http://linkedgeodata.org/vocabulary#attraction                                    3
http://linkedgeodata.org/vocabulary#church                                        1

6 Rows. -- 120 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp9"><title>Example 9</title>
<programlisting><![CDATA[
## Get Stuff Around Notre Dame De Paris with Names
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
SELECT ?cn
WHERE
  {
    ?s lgv:name ?cn  .
    ?s geo:geometry ?geo .
    FILTER (bif:st_intersects (?geo, bif:st_point (2.3498, 48.853), 0.3))
  }
LIMIT 20;
cn
VARCHAR
_______________________________________________________________________________

Parking Lagrange
Maitre Albert B&B
Le Grenier de Notre Dame
Eglise Saint-Julien-le-Pauvre
Eglise Saint Julien le Pauvre
Polly Magoo
Point 0 des Routes de France
Square Jean XXIII
....
20 Rows. -- 140 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp10"><title>Example 10</title>
<programlisting><![CDATA[
## Get Churches With The Most Bars
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
SELECT ?churchname ?cafename (bif:st_distance (?churchgeo, ?cafegeo))
WHERE
  {
    ?church a lgv:place_of_worship .
    ?church geo:geometry ?churchgeo .
    ?church lgv:name ?churchname .
    ?cafe a lgv:cafe .
    ?cafe lgv:name ?cafename .
    ?cafe geo:geometry ?cafegeo .
    FILTER (bif:st_intersects (?churchgeo, bif:st_point (2.3498, 48.853), 5)
      && bif:st_intersects (?cafegeo, ?churchgeo, 0.2))
  }
LIMIT 10;

churchname                            cafename                            callret-2
VARCHAR                               VARCHAR                             VARCHAR
_______________________________________________________________________________

Eglise Saint-Julien-le-Pauvre         Le Saint R+?-?gis                   0.09759308692691648
Eglise Saint-Germain des Pr+?-?s      Caf+?-? de Flore                    0.08774468391412803
Eglise Saint-Germain des Pr+?-?s      Les Deux Magots                     0.05235923473923059
Eglise Saint-Germain des Pr+?-?s      Caf+?-? Mabillon                    0.1712042770289815
Eglise Saint-Germain-des-Pr+?-?s      Caf+?-? de Flore                    0.1466502865197912
Eglise Saint-Germain-des-Pr+?-?s      Les Deux Magots                     0.1096767137079839
Eglise Saint-Germain-des-Pr+?-?s      Bar du march+?-?                    0.1831441251868126
Eglise Saint-Germain-des-Pr+?-?s      Caf+?-? Mabillon                    0.1174051745495528
Synagogue                             La Chaise au Plafond                0.1038387283609551
Synagogue                             Le Loir dans la Th+?-?i+?-?re       0.1632848322062273

10 Rows. -- 511225 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp11"><title>Example 11</title>
<programlisting><![CDATA[
## Things around highly populated places
SQL>
SPARQL
SELECT ?s ( sql:num_or_null (?o) ) COUNT (*)
WHERE
  {
    ?s <http://dbpedia.org/ontology/populationTotal> ?o .
        FILTER ( sql:num_or_null (?o) > 6000000 ) .
    ?s geo:geometry ?geo .
        FILTER ( bif:st_intersects (?pt, ?geo,2) ) .
    ?xx geo:geometry ?pt
  }
GROUP BY ?s ( sql:num_or_null (?o) )
ORDER BY desc 3
LIMIT 20;

s                                                         callret-1        callret-2
VARCHAR                                                   VARCHAR          VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/London                        7556900          312307
http://dbpedia.org/resource/Toronto                       8102163          115859
http://dbpedia.org/resource/New_York_City                 8363710          95629
http://dbpedia.org/resource/The_Hague                     6659300          84410
http://dbpedia.org/resource/Tokyo                         12790000         78618
http://dbpedia.org/resource/Philadelphia                  6385461          67115
http://dbpedia.org/resource/Los_Angeles                   17755322         64394
http://dbpedia.org/resource/Bangkok                       8160522          62519
http://dbpedia.org/resource/Barcelona                     2147483648       57635
http://dbpedia.org/resource/Cairo                         6758581          52738
http://dbpedia.org/resource/Istanbul                      12697164         50745
http://dbpedia.org/resource/Seoul                         10421782         43962
http://dbpedia.org/resource/Beijing                       17430000         35979
http://dbpedia.org/resource/Purmerend                     6659300          33508
http://dbpedia.org/resource/Baghdad                       6554126          33426
http://dbpedia.org/resource/Bogot%C3%A1                   6776009          30429
http://dbpedia.org/resource/Mexico_City                   8836045          30127
http://dbpedia.org/resource/Jakarta                       8500000          28944
http://dbpedia.org/resource/Boston                        7514759          27705
http://dbpedia.org/resource/Baden-W%C3%BCrttemberg        10755000         25112

20 Rows. -- 4296 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp12"><title>Example 12</title>
<programlisting><![CDATA[
## Example "Places Of Worship, Within 5 km Of Paris":

## Constructs a custom Linked Data Mesh (graph) about
## places of worship, within 5 km of Paris, that have
## cafes in close proximity(0.2 km).

## Note: we have distinct pin colors that identify
## for places of worship distinct from cafes.

## The query requires V6 or higher.
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
PREFIX rtb: <http://www.openlinksw.com/schemas/oat/rdftabs#>
CONSTRUCT
  {
    ?cafe geo:geometry ?cafegeo        ;
                   rtb:useMarker '01'  ;
                  lgv:name ?cafename   .
    ?church geo:geometry ?churchgeo    ;
                   rtb:useMarker '02'  ;
                lgv:name ?churchname   .
  }
WHERE
  {
    ?church a lgv:place_of_worship .
    ?church geo:geometry ?churchgeo .
    ?church lgv:name ?churchname .
    ?cafe a lgv:cafe .
    ?cafe lgv:name ?cafename .
    ?cafe geo:geometry ?cafegeo .
    ?cafe geo:lat ?lat .
    ?cafe geo:long ?long .
    FILTER ( bif:st_intersects ( ?churchgeo, bif:st_point ( 2.3498, 48.853 ), 5 ) &&
             bif:st_intersects ( ?cafegeo, ?churchgeo, 0.2 ) )
  }
LIMIT 10;

@prefix ns0:	<http://linkedgeodata.org/vocabulary#> .
@prefix ns1:	<http://linkedgeodata.org/triplify/node/237435716#> .
ns1:id	ns0:name	"Chapelle du Val de Gr\u00C3\u00A2ce" .
@prefix ns2:	<http://www.openlinksw.com/schemas/oat/rdftabs#> .
ns1:id	ns2:useMarker	"02" .
@prefix virtrdf:	<http://www.openlinksw.com/schemas/virtrdf#> .
@prefix geo:	<http://www.w3.org/2003/01/geo/wgs84_pos#> .
ns1:id	geo:geometry	"POINT(2.3418 48.8406)"^^virtrdf:Geometry .
@prefix ns5:	<http://linkedgeodata.org/triplify/node/218147750#> .
ns5:id	ns0:name	"Synagogue" ;
	ns2:useMarker	"02" ;
	geo:geometry	"POINT(2.3593 48.857)"^^virtrdf:Geometry .
@prefix ns6:	<http://linkedgeodata.org/triplify/node/218145208#> .
ns6:id	ns0:name	"Synagogue" ;
	ns2:useMarker	"02" ;
	geo:geometry	"POINT(2.3589 48.8567)"^^virtrdf:Geometry .
...
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp13"><title>Example 13</title>
<programlisting><![CDATA[
## Example "Places Of Worship, Within 5 km Of Paris":

## Asks for places of worship, within 5 km of Paris,
## that have cafes in close proximity(0.2 km).
## The query requires V6 or higher.

SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
ASK
WHERE
  {
    ?church a lgv:place_of_worship .
    ?church geo:geometry ?churchgeo .
    ?church lgv:name ?churchname .
    ?cafe a lgv:cafe .
    ?cafe lgv:name ?cafename .
    ?cafe geo:geometry ?cafegeo .
    ?cafe geo:lat ?lat .
    ?cafe geo:long ?long .
    FILTER ( bif:st_intersects ( ?churchgeo, bif:st_point ( 2.3498, 48.853 ), 5 ) &&
             bif:st_intersects ( ?cafegeo, ?churchgeo, 0.2 ) )
  };

Done.

true
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp14"><title>Example 14</title>
<programlisting><![CDATA[
## Places of worship, within 5 km of Paris,
## that have cafes in close proximity(0.2 km)
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
SELECT DISTINCT ?cafe ?lat ?long ?cafename ?churchname
(bif:round(bif:st_distance (?churchgeo, ?cafegeo)))
WHERE
  {
    ?church a lgv:place_of_worship .
    ?church geo:geometry ?churchgeo .
    ?church lgv:name ?churchname .
    ?cafe a lgv:cafe .
    ?cafe lgv:name ?cafename .
    ?cafe geo:geometry ?cafegeo .
    ?cafe geo:lat ?lat.
    ?cafe geo:long ?long.
    FILTER ( bif:st_intersects (?churchgeo, bif:st_point (2.3498, 48.853), 5) &&
                   bif:st_intersects (?cafegeo, ?churchgeo, 0.2) )
  }
LIMIT 10;

cafe                                                 lat       long     cafename                           churchname                             callret-5
VARCHAR                                              VARCHAR   VARCHAR  VARCHAR                            VARCHAR                                VARCHAR
_______________________________________________________________________________________________________________________________________________________________

http://linkedgeodata.org/triplify/node/321932192#id  48.8522   2.3484   Le Saint R+?-?gis                  Eglise Saint-Julien-le-Pauvre          0
http://linkedgeodata.org/triplify/node/251699776#id  48.8541   2.3326   Caf+?-? de Flore                   Eglise Saint-Germain des Pr+?-?s       0
http://linkedgeodata.org/triplify/node/251699775#id  48.854    2.3331   Les Deux Magots                    Eglise Saint-Germain des Pr+?-?s       0
http://linkedgeodata.org/triplify/node/315769036#id  48.8533   2.3358   Caf+?-? Mabillon                   Eglise Saint-Germain des Pr+?-?s       0
http://linkedgeodata.org/triplify/node/251699776#id  48.8541   2.3326   Caf+?-? de Flore                   Eglise Saint-Germain-des-Pr+?-?s       0
http://linkedgeodata.org/triplify/node/251699775#id  48.854    2.3331   Les Deux Magots                    Eglise Saint-Germain-des-Pr+?-?s       0
http://linkedgeodata.org/triplify/node/315769035#id  48.8539   2.3371   Bar du march+?-?                   Eglise Saint-Germain-des-Pr+?-?s       0
http://linkedgeodata.org/triplify/node/315769036#id  48.8533   2.3358   Caf+?-? Mabillon                   Eglise Saint-Germain-des-Pr+?-?s       0
http://linkedgeodata.org/triplify/node/251126326#id  48.8572   2.3577   La Chaise au Plafond               Synagogue                              0
http://linkedgeodata.org/triplify/node/251043135#id  48.8562   2.361    Le Loir dans la Th+?-?i+?-?re      Synagogue                              0

10 Rows. -- 120 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp15"><title>Example 15</title>
<programlisting><![CDATA[
## Stuff around Notre Dame de Paris
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
SELECT ?s ?cn ?lat ?long
WHERE
  {
    ?s lgv:name ?cn  .
    ?s geo:geometry ?geo .
    ?s geo:lat ?lat.
    ?s geo:long ?long.
    FILTER ( bif:st_intersects (?geo, bif:st_point (2.3498, 48.853), 0.3) )
  }
LIMIT 20;

s                                                      cn                                                        lat        long
VARCHAR                                                VARCHAR                                                   VARCHAR    VARCHAR
______________________________________________________________________________________________________________

http://linkedgeodata.org/triplify/node/237004656#id    Parking Lagrange                                          48.8506    2.3487
http://linkedgeodata.org/triplify/node/237003117#id    Mus+?-?e de l'Assistance Publique H+?-opitaux de Paris    48.8507    2.3519
http://linkedgeodata.org/triplify/way/23071565#id      Jardin de la Rue de Bi+?-?vre                             48.8504    2.3502
http://linkedgeodata.org/triplify/node/251652818#id    Maitre Albert B&B                                         48.8507    2.3496
http://linkedgeodata.org/triplify/node/251373384#id    Le Grenier de Notre Dame                                  48.8513    2.35
http://linkedgeodata.org/triplify/node/205266764#id    Eglise Saint-Julien-le-Pauvre                             48.852     2.3471
http://linkedgeodata.org/triplify/way/19741083#id      Eglise Saint Julien le Pauvre                             48.8521    2.3469
http://linkedgeodata.org/triplify/node/251474112#id    Polly Magoo                                               48.8526    2.3467
http://linkedgeodata.org/triplify/node/251531803#id    H+?-otel Esmerelda                                        48.8523    2.3468
http://linkedgeodata.org/triplify/node/191031796#id    Point 0 des Routes de France                              48.8533    2.3489
http://linkedgeodata.org/triplify/way/20444455#id      Square Jean XXIII                                         48.8529    2.3511
http://linkedgeodata.org/triplify/way/19740745#id      Square Ren+?-? Viviani                                    48.8525    2.3476
http://linkedgeodata.org/triplify/node/321932192#id    Le Saint R+?-?gis                                         48.8522    2.3484
http://linkedgeodata.org/triplify/node/27440965#id     Notre-Dame de Paris                                       48.853     2.3499
http://linkedgeodata.org/triplify/node/243461762#id    Parking Notre-Dame                                        48.8537    2.3475
http://linkedgeodata.org/triplify/way/21816758#id      Notre-Dame de Paris                                       48.8531    2.349
http://linkedgeodata.org/triplify/way/22972062#id      La Seine                                                  48.8538    2.3531
http://linkedgeodata.org/triplify/way/25463927#id      La Seine                                                  48.8548    2.3518
http://linkedgeodata.org/triplify/node/251128395#id    H+?-otel Hospitel                                         48.854     2.3484
http://linkedgeodata.org/triplify/way/14155323#id      H+?-otel Dieu                                             48.8555    2.3485

20 Rows. -- 167 msec.
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp16"><title>Example 16</title>
<programlisting><![CDATA[
## Stuff around Notre Dame de Paris
SQL>
SPARQL
PREFIX lgv: <http://linkedgeodata.org/vocabulary#>
DESCRIBE ?s
WHERE
  {
    ?s lgv:name ?cn  .
    ?s geo:geometry ?geo .
    ?s geo:lat ?lat.
    ?s geo:long ?long.
    FILTER (bif:st_intersects (?geo, bif:st_point (2.3498, 48.853), 0.3))
  }
LIMIT 20;

@prefix rdf:	<http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix ns1:	<http://linkedgeodata.org/triplify/node/27440966#> .
@prefix ns2:	<http://linkedgeodata.org/vocabulary#> .
ns1:id	rdf:type	ns2:node ,
		ns2:police .
@prefix geo:	<http://www.w3.org/2003/01/geo/wgs84_pos#> .
ns1:id	geo:lat	48.8542 ;
	geo:long	2.3473 ;
	ns2:created_by	"Potlatch 0.6a" ;
	ns2:name	"Pr\u00C3\u00A9fecture de Police de Paris" ,
		"Pr\u00E9fecture de Police de Paris" .
@prefix virtrdf:	<http://www.openlinksw.com/schemas/virtrdf#> .
ns1:id	geo:geometry	"POINT(2.3473 48.8542)"^^virtrdf:Geometry .
@prefix ns5:	<http://linkedgeodata.org/triplify/node/27440965#> .
ns5:id	rdf:type	ns2:node ,
		ns2:place_of_worship ;
	geo:lat	48.853 ;
	geo:long	2.3499 ;
	ns2:denomination	"catholic" ;
	ns2:name	"Notre-Dame de Paris" ;
	ns2:religion	"christian" ,
		ns2:christian ;
	geo:geometry	"POINT(2.3499 48.853)"^^virtrdf:Geometry .

......
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp17"><title>Example 17</title>
<programlisting><![CDATA[
## Cities within 30 km proximity of London
SQL>
SPARQL
SELECT ?m (bif:round(bif:st_distance (?geo, ?gm)))
WHERE
  {
    <http://dbpedia.org/resource/London> geo:geometry ?gm .
    ?m geo:geometry ?geo .
    ?m a <http://dbpedia.org/ontology/City> .
    FILTER (bif:st_intersects (?geo, ?gm, 30))
  }
ORDER BY DESC 2
LIMIT 20;

m                                                   callret-1
VARCHAR                                             VARCHAR
____________________________________________________________

http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Ebbsfleet_Valley        30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30
http://dbpedia.org/resource/Bletchingley            30

20 Rows. -- 727666 msec.

]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp18"><title>Example 18</title>
<programlisting><![CDATA[
## Motorways across England & Scotland from DBpedia
SQL>
SPARQL
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dbpprop: <http://dbpedia.org/property/>
PREFIX yago: <http://dbpedia.org/class/yago/>
SELECT ?road ?services ?lat ?long
WHERE
  {
    {
      ?services dbpprop:road ?road .
      ?road a yago:MotorwaysInEngland .
      ?services dbpprop:lat ?lat .
      ?services dbpprop:long ?long .
    }
    UNION
    {
      ?services dbpprop:road ?road .
      ?road a yago:MotorwaysInScotland .
      ?services dbpprop:lat ?lat .
      ?services dbpprop:long ?long .
    }
  }
LIMIT 20;

road                                          services                                                        lat             long
VARCHAR                                       VARCHAR                                                         VARCHAR         VARCHAR
______________________________________________________________________________________________________________________________________

http://dbpedia.org/resource/M90_motorway      http://dbpedia.org/resource/Kinross_services                    56.209628       -3.439257
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Leicester_Forest_East_services      52.6192         -1.206
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Woodall_services                    53.3152         -1.2813
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Tibshelf_services                   53.13708        -1.33179
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/London_Gateway_services             51.631          -0.264
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Donington_Park_services             52.823651       -1.305887
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Watford_Gap_services                52.3069         -1.1226
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Newport_Pagnell_services            52.083066       -0.748508
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Trowell_services                    52.963198       -1.265988
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Woolley_Edge_services               53.62259        -1.549422
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Toddington_services                 51.9478         -0.502075
http://dbpedia.org/resource/M1_motorway       http://dbpedia.org/resource/Northampton_services                52.209201       -0.944799
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Chieveley_services                  51.449          -1.3112
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Magor_services                      51.58786        -2.83713
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Pont_Abraham_services               51.74712        -4.0655
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Swansea_services                    51.678197       -3.994646
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Leigh_Delamere_services             51.511528       -2.159468
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Reading_services                    51.424527       -1.035633
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Cardiff_West_services               51.50626        -3.30535
http://dbpedia.org/resource/M4_motorway       http://dbpedia.org/resource/Heston_services                     51.48807        -0.39106

20 Rows. -- 531 msec.

]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp18"><title>Example 19</title>
<programlisting><![CDATA[
SELECT DISTINCT ?s (bif:round(?lat)) as ?lat (bif:round(?long)) as ?long
WHERE
  {
    {
      SELECT ?g ?s WHERE
        {
          graph ?g {
            ?s geo:geometry ?geo }
        }
      LIMIT 100
    }
    graph ?g {
      ?s geo:lat ?lat .
      ?s geo:long ?long . }
    FILTER (datatype (?lat) in (xsd:integer, xsd:float, xsd:double)) .
    FILTER (datatype (?long) in (xsd:integer, xsd:float, xsd:double))
  }

s                                                                             lat        long
ANY 	                                                                      ANY 	 ANY
________________________________________________________________________________________________
http://dbpedia.org/resource/QUaD	                                      -90	 -139
http://dbpedia.org/resource/Amundsen-Scott_South_Pole_Station	              -90	 -139
http://dbpedia.org/resource/Amundsen-Scott_South_Pole_Station	              -90	 0
http://dbpedia.org/resource/Degree_Angular_Scale_Interferometer	              -90	 -139
http://dbpedia.org/resource/South_Pole_Telescope	                      -90	 -139
http://dbpedia.org/resource/Arcminute_Cosmology_Bolometer_Array_Receiver      -90	 -139
http://dbpedia.org/resource/Viper_telescope	                              -90	 -139
http://dbpedia.org/resource/Mount_Weaver	                              -87	 -154
http://dbpedia.org/resource/Axel_Heiberg_Glacier	                      -85	 -163
http://dbpedia.org/resource/Mount_Ray	                                      -85	 -171
http://linkedgeodata.org/triplify/node/275487234#id	                      -85	 -142
http://linkedgeodata.org/triplify/node/303732928#id	                      -85	 -142
http://linkedgeodata.org/triplify/node/332036611#id	                      -85	 -85
http://linkedgeodata.org/triplify/node/303732935#id	                      -85	 -143
http://linkedgeodata.org/triplify/node/303732951#id	                      -85	 -144
http://linkedgeodata.org/triplify/node/303732953#id	                      -85	 -144
http://linkedgeodata.org/triplify/node/276208684#id	                      -85	 -166
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp19"><title>Example 19</title>
<programlisting><![CDATA[
## "Find things within 20km of New York City":

SELECT DISTINCT ?resource ?label ?location
WHERE
  {
    <http://dbpedia.org/resource/New_York_City>
        geo:geometry ?sourcegeo                   .
    ?resource geo:geometry ?location              ;
                rdfs:label ?label .
    FILTER( bif:st_intersects( ?location, ?sourcegeo, 20 ) ) .
    FILTER( lang(?label) = "en" )
  }
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp20"><title>Example 20</title>
<programlisting><![CDATA[
## "Find Distance between New York City
## and London, England":

SELECT ( bif:st_distance( ?nyl,?ln ) )
      AS ?distanceBetweenNewYorkCityAndLondon
WHERE
  {
    <http://dbpedia.org/resource/New_York_City>
        geo:geometry ?nyl .
    <http://dbpedia.org/resource/London>
        geo:geometry ?ln  .
  }
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp21"><title>Example 21</title>
<programlisting><![CDATA[
## "Find "All Educational Institutions
## within 10km of Oxford, UK; ordered by
## date of establishment":

SELECT DISTINCT ?thing AS ?uri
      ?thingLabel AS ?name
      ?date AS ?established
      ?matchgeo AS ?location
WHERE
  {
    <http://dbpedia.org/resource/Oxford>
             geo:geometry ?sourcegeo .
    ?resource geo:geometry ?matchgeo .
    FILTER( bif:st_intersects( ?matchgeo, ?sourcegeo, 5 ) ) .
       ?thing ?somelink ?resource                           ;
       <http://dbpedia.org/ontology/established> ?date      ;
       rdfs:label ?thingLabel                               .
    FILTER( lang(?thingLabel) = "en" )
  }
ORDER BY ASC( ?date )
]]></programlisting>
    </sect3>
    <sect3 id="rdfsparqlgeospatexmp22"><title>Example 22</title>
<programlisting><![CDATA[
## "Find Historical cross section of events related
## to Edinburgh and the surrounding area (within 30km)
## during the 19th century":

SELECT DISTINCT ?thing ?thingLabel
      ?dateMeaningLabel ?date ?matchgeo
WHERE
  {
    {
      SELECT DISTINCT ?thing ?matchgeo
      WHERE
        {
          <http://dbpedia.org/resource/Edinburgh>
              geo:geometry ?sourcegeo .
            ?resource geo:geometry ?matchgeo .
            FILTER( bif:st_intersects (
                ?matchgeo, ?sourcegeo, 30 ) ) .
            ?thing ?somelink ?resource
  }
    }
    {
      ?property rdf:type owl:DatatypeProperty ;
                          rdfs:range xsd:date
    } .
    ?thing ?dateMeaning ?date .
    FILTER( ?dateMeaning IN ( ?property ) ) .
    FILTER( ?date >= xsd:gYear("1800")
         && ?date <= xsd:gYear("1900") )
    ?dateMeaning rdfs:label ?dateMeaningLabel .
  FILTER( lang(?dateMeaningLabel) = "en" ) .
  ?thing rdfs:label ?thingLabel .
  FILTER( lang(?thingLabel) = "en" )
  }
ORDER BY ASC ( ?date )
]]></programlisting>
    </sect3>
  </sect2>
</sect1>

<sect1 id="rdfreplication"><title>RDF Replication</title>
  <para>Tables of RDF storage, such as DB.DBA.RDF_QUAD and DB.DBA.RDF_OBJ, can not be replicated in a usual way, because it's
  	content is cached in memory in special ways and synchronized with values outside these tables, such as current values of
  	special sequence objects.</para>
  <para>Moreover, same IRI may have different internal IRI_IDs on different boxes, because the assigned IDs vary if new IRIs
  	appear in data in different order. Similarly, there will be different IDs of RDF literal, datatypes and languages,
  	blocking any attempt of one-to-one replication between RDF storages.</para>
  <para>However, a special asynchronous RDF replication makes it possible to configure a "publisher" Virtuoso instance to keep
  	the log of changes in some RDF graphs and subscribe some Virtuoso instances to replay all these changes.</para>
  <para>Configuration functions are quite straightforward.</para>
  <para>RDF graphs to replicate are all members of &lt;http://www.openlinksw.com/schemas/virtrdf#rdf_repl_graph_group&gt; graph
  	group. That group can be filled in with graphs like any other graph group, but it is better to get the advantage of proper
  	security check made by <link linkend="fn_rdf_repl_graph_ins"><function>DB.DBA.RDF_REPL_GRAPH_INS()</function></link> that inserts
  	a graph to the group and <link linkend="fn_rdf_repl_graph_del"><function>DB.DBA.RDF_REPL_GRAPH_DEL()</function></link> that
  	removes a graph from the group.</para>
  <para>Only publicly readable graphs can be replicated, an error is signalled otherwise, and it is better to know about a
  	security issue as early as possible.</para>
  <para>The <link linkend="fn_rdf_repl_start"><function>DB.DBA.RDF_REPL_START()</function></link>	function starts the RDF replication
    at the publishing side. It creates replication "publication" named '__rdf_repl' and makes a log file '__rdf_repl.log' to
    record changes in replicated graphs. If the replication has been started before then an error is signalled; passing value 1
    for parameter "quiet" elimintaes the error so the incorrect call has no effect at all. If the replication is enabled then
    the value of registry variable 'DB.DBA.RDF_REPL' indicates the moment of replication start.</para>
  <para> The <link linkend="fn_rdf_repl_start"><function>DB.DBA.RDF_REPL_START()</function></link> function performs a
  security check before starting the replication to check.</para>
  <para>The <link linkend="fn_rdf_repl_stop"><function>DB.DBA.RDF_REPL_STOP()</function></link> stops the RDF replication at the
  publishing side. It calls <link linkend="fn_repl_unpublish"><function>repl_unpublish()</function></link> but does not make
  empty reates replication "publication" named '__rdf_repl' and makes a log file '__rdf_repl.log' to record changes in
  replicated graphs.</para>
  <para>Replication is asynchronous and the order of insertion and removal operations at the subscriber's side may not match
  	the order at the publisher. As a result, it is not recommended to make few subscriptions that writes changes of few
  	publishers into one common graph. A client-side application can force the synchronuzation by calling
  	<link linkend="fn_rdf_repl_sync"><function>DB.DBA.RDF_REPL_SYNC()</function></link> that acts like
  	<link linkend="fn_repl_sync"><function>repl_sync()</function></link> but for an RDF subscription.
  	<link linkend="fn_rdf_repl_sync"><function>DB.DBA.RDF_REPL_SYNC()</function></link> will not only initial synchronisation
  	but also wait for the end of subscription to guarantee that the total effect of INSERT and DELETE operations is correct
  	even if these operations were made in an order that differs from the original one.</para>
</sect1>

<sect1 id="rdfperformancetuning"><title>RDF Performance Tuning</title>
<para>For RDF query performance, we have the following possible questions:</para>
<itemizedlist mark="bullet" spacing="compact">
<listitem>Is the Virtuoso process properly configured to handle big data sets?</listitem>
<listitem>Is the graph always specified?</listitem>
<listitem>Are public web service endpoints protected against bad queries?</listitem>
<listitem>Are there patterns where only a predicate is given?</listitem>
<listitem>Is there a bad query plan because of cost model error?</listitem>
</itemizedlist>
    <sect2 id="rdfperfgeneral"><title>General</title>
	    <para>When running with large data sets, one should configure the Virtuoso process to use between 2/3 
		to 3/5 of system RAM and to stripe storage on all available disks. 
		See <link linkend="VIRTINI">NumberOfBuffers</link>, 
		<link linkend="VIRTINI">MaxDirtyBuffers</link>, and 
		<link linkend="VIRTINI">Striping</link> INI file parameters. 
		  </para>
<programlisting><![CDATA[
; default installation
NumberOfBuffers          = 2000
MaxDirtyBuffers          = 1200	
]]></programlisting>		
      <para>Typical sizes for the <link linkend="VIRTINI">NumberOfBuffers</link> and
      <link linkend="VIRTINI">MaxDirtyBuffers</link> (3/4 of NumberOfBuffers) parameters in the Virtuoso
      configuration file (virtuoso.ini) for various memory sizes are as follows, with each buffer 
      consisting of 8K bytes:      
      </para>	
      <table><title>recommended NumberOfBUffers and MaxDirtyBuffers</title>
        <tgroup cols="3">
          <thead>
          	<row>
              <entry>System RAM</entry>
              <entry>NumberOfBuffers</entry>
              <entry>MaxDirtyBuffers</entry>
            </row>
          </thead>
          <tbody>
            <row>
            	<entry>2 GB</entry><entry>170000</entry><entry>130000</entry>
            </row>
            <row>
            	<entry>4 GB</entry><entry>340000</entry><entry>250000</entry>
            </row>
            <row>
            	<entry>8 GB</entry><entry>680000</entry><entry>500000</entry>
            </row>
            <row>
            	<entry>16 GB</entry><entry>1360000</entry><entry>1000000</entry>
            </row>
            <row>
            	<entry>32 GB</entry><entry>2720000</entry><entry>2000000</entry>
            </row>
            <row>
            	<entry>48 GB</entry><entry>4000000</entry><entry>3000000</entry>
            </row>
            <row>
            	<entry>64 GB</entry><entry>5450000</entry><entry>4000000</entry>
            </row>
          </tbody>
        </tgroup>
      </table>      
      <para>Also, if running with a large database, setting <link linkend="VIRTINI">MaxCheckpointRemap</link> to
      1/4th of the database size is recommended. This is in pages, 8K per page.</para>
    </sect2>
    <sect2 id="rdfperfrdfscheme"><title>RDF Index Scheme</title>
<para>
        Starting with version 6.00.3126, the default RDF index scheme consists of 2 full indices over RDF quads
        plus 3 partial indices. This index scheme is generally adapted to all kinds of workloads, regardless of
        whether queries generally specify a graph. As indicated the default index scheme in Virtuoso is almost
        always applicable as is, whether one has a RDF database with very large numbers of small graphs or just
        one or a few large graphs. With Virtuoso 7 the indices are column-wise by default, which results in them
        to consuming usually about 1/3 of the space the equivalent row-wise structures would consume.
</para>
<para>
        Alternate indexing schemes are possible but will not be generally needed. For upgrading old databases with
        a different index scheme see the corresponding documentation.
</para>
<para>
The index scheme consists of the following indices:
</para>

<itemizedlist mark="bullet">
  <listitem><emphasis>PSOG</emphasis> - primary key</listitem>
  <listitem><emphasis>POGS</emphasis> - bitmap index for lookups on object value.</listitem>
  <listitem><emphasis>SP</emphasis> - partial index for cases where only S is specified.</listitem>
  <listitem><emphasis>OP</emphasis> - partial index for cases where only O is specified.</listitem>
  <listitem><emphasis>GS</emphasis> - partial index for cases where only G is specified.</listitem>
</itemizedlist>

<para>
This index scheme is created by the following statements:
</para>
<programlisting><![CDATA[
CREATE TABLE DB.DBA.RDF_QUAD (
  G IRI_ID_8,
  S IRI_ID_8,
  P IRI_ID_8,
  O ANY,
  PRIMARY KEY (P, S, O, G)
  )
ALTER INDEX RDF_QUAD ON DB.DBA.RDF_QUAD
  PARTITION (S INT (0hexffff00));

CREATE DISTINCT NO PRIMARY KEY REF BITMAP INDEX RDF_QUAD_SP
  ON RDF_QUAD (S, P)
  PARTITION (S INT (0hexffff00));

CREATE BITMAP INDEX RDF_QUAD_POGS
  ON RDF_QUAD (P, O, G, S)
  PARTITION (O VARCHAR (-1, 0hexffff));

CREATE DISTINCT NO PRIMARY KEY REF BITMAP INDEX RDF_QUAD_GS
  ON RDF_QUAD (G, S)
  PARTITION (S INT (0hexffff00));

CREATE DISTINCT NO PRIMARY KEY REF INDEX RDF_QUAD_OP
  ON RDF_QUAD (O, P)
  PARTITION (O VARCHAR (-1, 0hexffff));
]]></programlisting>
      <para>The idea is to favor queries where the predicate is specified in triple patterns. The entire quad can
      be efficiently accessed when <code>P</code> and at least one of <code>S</code> and <code>O</code> are known.
      This has the advantage of clustering data by the predicate which improves working set. A page read from disk
      will only have entries pertaining to the same predicate; chances of accessing other entries of the page are
      thus higher than if the page held values for arbitrary predicates. For less frequent cases where only
      <code>S</code> is known, as in <code>DESCRIBE</code>, the distinct <code>P</code>s of the <code>S</code> are
      found in the <code>SP</code> index. These <code>SP</code> pairs are then used for accessing the <code>PSOG</code>
      index to get the <code>O</code> and <code>G</code>. For cases where only the <code>G</code> is known, as when
      dropping a graph, the distinct <code>S</code>s of the <code>G</code> are found in the <code>GS</code> index.
      The <code>P</code>s of the <code>S</code> are then found in the <code>SP</code> index. After this, the whole
      quad is found in the <code>PSOG</code> index.
      </para>
      <para>The <code>SP</code>, <code>OP</code>, and <code>GS</code> indices do not store duplicates. If an
      <code>S</code> has many values of the <code>P</code>, there is only one entry. Entries are not deleted from
      <code>SP</code>, <code>OP</code>, or <code>GS</code>. This does not lead to erroneous results since a full
      index (that is, either <code>POSG</code> or <code>PSOG</code>) is always consulted in order to know if a quad
      actually exists. When updating data, most often a graph is entirely dropped and a substantially similar graph
      inserted in its place. The <code>SP</code>, <code>OP</code>, and <code>GS</code> indices get to stay
      relatively unaffected.
      </para>
      <para>Still, over time, especially if there are frequent updates and values do not repeat between consecutive
      states, the <code>SP</code>, <code>OP</code>, and <code>GS</code> indices will get polluted, which may affect
      performance. Dropping and recreating the index will remedy this situation.
      </para>
      <para>In cases where this is not practical, the index scheme should only have full indices; i.e., each key
      holds all columns of the primary key of the quad. This will be the case if the
      <code>DISTINCT NO PRIMARY KEY REF</code> options are not specified in the <code>CREATE INDEX</code> statement.
      In such cases, all indices remain in strict sync across deletes.
      </para>
      <para>Many RDF workloads have bulk-load and read-intensive access patterns with few deletes. The default index
      scheme is optimized for these. With these situations, this scheme offers significant space savings, resulting
      in better working set. Typically, this layout takes 60-70% of the space of a layout with 4 full indices.
      </para>
    </sect2>
    <sect2 id="rdfindexschemesel"><title>Index Scheme Selection</title>
      <para>The indexes in place on the <code>RDF_QUAD table</code> can greatly affect the performance of SPARQL
      queries, as can be determined by running the <code>STATISTICS</code> command on the table as follows:
      </para>
<programlisting><![CDATA[
SQL> STATISTICS DB.DBA.RDF_QUAD;
Showing SQLStatistics of table(s) 'DB.DBA.RDF_QUAD'
TABLE_QUALIFIER  TABLE_OWNER      TABLE_NAME       NON_UNIQUE  INDEX_QUALIFIER  INDEX_NAME       TYPE        SEQ_IN_INDEX  COLUMN_NAME      COLLATION  CARDINALITY  PAGES       FILTER_CONDITION
VARCHAR          VARCHAR          VARCHAR          SMALLINT    VARCHAR          VARCHAR          SMALLINT    SMALLINT    VARCHAR          VARCHAR  INTEGER     INTEGER     VARCHAR
_______________________________________________________________________________

DB               DBA              RDF_QUAD         NULL        NULL             NULL             0           NULL        NULL             NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         0           DB               RDF_QUAD         3           1           P                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         0           DB               RDF_QUAD         3           2           S                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         0           DB               RDF_QUAD         3           3           O                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         0           DB               RDF_QUAD         3           4           G                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_GS      3           1           G                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_GS      3           2           S                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_OP      3           1           O                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_OP      3           2           P                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_POGS    3           1           P                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_POGS    3           2           O                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_POGS    3           3           G                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_POGS    3           4           S                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_SP      3           1           S                NULL     NULL        NULL        NULL
DB               DBA              RDF_QUAD         1           DB               RDF_QUAD_SP      3           2           P                NULL     NULL        NULL        NULL

15 Rows. -- 24 msec.
SQL>
]]></programlisting>
      <para>With only one index (<code>OGPS</code>) created by default, if the graph is always given, as with one
      or more <code>FROM</code> or <code>FROM NAMED</code> clauses, and there are no patterns where only graph and
      predicate are given, then the default indices should be sufficient. If predicate and graph are given but
      subject is not, then it is sometimes useful to add:
      </para>
<programlisting><![CDATA[
CREATE BITMAP INDEX RDF_QUAD_PGOS
  ON DB.DBA.RDF_QUAD (G, P, O, S)
  PARTITION (O VARCHAR (-1, 0hexffff));
]]></programlisting>
      <para>Note: If the server version is pre-5.0.7, leave out the partitioning clause.</para>
      <para>Making the <code>PGOS</code> index can help in some cases even if it is not readily apparent from the
      queries that one is needed. This is so, for example, if the predicate by itself is selective; i.e., there is
      a predicate that occurs in only a few triples.
      </para>
      <para>There is one known application scenario that requires a small alteration to the default index scheme.
      If the application has a large number of small graphs, e.g. millions of graphs of tens or hundreds of triples
      each, and it commonly happens that large numbers of graphs contain exactly the same triple, for example the
      same triple is found in 100000 or one million graphs, then some operations will become inefficient with the
      default index scheme. In specific, dropping a graph may have to scan through large amounts of data in order
      to find the right quad to delete from the set of quads that differ only in the graph.
      </para>
      <para>This will affect a graph replace and a graph drop or generally any deletion that falls on a quad of the
      described sort. If this is the situation in the application, then dropping the <code>RDF_QUAD_GS</code>
      distinct projection and replacing it with a covering index that starts with <code>G</code> is appropriate:
      </para>
<programlisting><![CDATA[
Drop index RDF_QUAD_GS;
Create column index RDF_QUAD_GPSO on RDF_QUAD (G, P, S, O) partition (S int (0hexffff00);
]]></programlisting>
      <para>The partition clause only affects cluster settings and is ignored in the single server case.
      Partitioning on <code>S</code> is usually better than on <code>O</code> since the distribution of <code>S</code>
      is generally less skewed than that of <code>O</code>. That is, there usually are some very common <code>O</code>
      values (e.g. class "thing"). This will increase space consumption by maybe 25% compared to the default scheme.
</para>
    </sect2>
    <sect2 id="rdfperfindexes"><title>Manage Public Web Service Endpoints</title>
      <para>
      Public web service endpoints have proven to be sources of especially bad queries. While local
      application developers can obtain instructions from database administrators and use ISQL access to the
      database to tune execution plans, "external" clients do not know details of configuration and/or lack
      appropriate skills. The most common problem is that public endpoints usually get requests that do not mention
      the required graph, because the queries were initially written for use with triple stores. If the web service
      provides access to a single graph (or to a short list of graphs), then it is strongly recommended to configure
      it by adding a row into <code>DB.DBA.SYS_SPARQL_HOST</code>.
      </para>
<programlisting><![CDATA[
create table DB.DBA.SYS_SPARQL_HOST (
  SH_HOST	varchar not null primary key, -- host mask
  SH_GRAPH_URI varchar,                 -- default graph uri
  SH_USER_URI	varchar,                  -- reserved for any use in applications
  SH_BASE_URI varchar,                  -- for future use (not used currently) to set BASE in sparql queries. Should be NULL for now.
  SH_DEFINES long varchar,              -- additional defines for requests
  PRIMARY KEY (SH_HOST)
)
]]></programlisting>
<para>You can find detailed descriptions of the table columns <link linkend="rdfdefaultgraph">here</link>.</para>
      <para>The idea is that if the client specifies the default graph in the request, or uses named graphs and
      group graph patterns, then he is probably smarter than average and will provide meaningful queries. If no
      graph names are specified, then the query will benefit from preset graph because this will give the compiler
      some more indexes to choose from -- indexes that begin with <code>G</code>.
      </para>
      <para>Sometimes web service endpoints are used to access data of only one application, not all data in the
      system. In that case, one may wish to declare a separate storage that consists of only RDF Views made by that
      application and <code>define input:storage</code> in the appropriate row of <code>DB.DBA.SYS_SPARQL_HOST</code>.
      </para>
</sect2>
<sect2 id="rdfperfcost"><title>Erroneous Cost Estimates and Explicit Join Order</title>
	    <para>The selectivity of triple patterns is determined at query compile time from sampling the data. It is
	    possible that misleading data is produced. To see if the cardinality guesses are generally valid, look at
	    the query plan with the <link linkend="fn_explain"><function>explain</function> ()</link> function.
	    </para>
      <para>Below is a sample from the LUBM qualification data set in the Virtuoso distribution. After running
      <emphasis>make test</emphasis> in <emphasis>binsrc/test/lubm</emphasis>, there is a loaded database with
      the data. Start a server in the same directory to see the data.</para>
<programlisting><![CDATA[
SQL> EXPLAIN 
  ('SPARQL 
  PREFIX  ub:  <http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#>
  SELECT *
  FROM <lubm>
  WHERE { ?x  rdf:type  ub:GraduateStudent }
  ');

REPORT
VARCHAR
_______________________________________________________________________________

{

Precode:
      0: $25 "callret" := Call __BOX_FLAGS_TWEAK (<constant (lubm)>, <constant (1)>)
      5: $26 "lubm" := Call DB.DBA.RDF_MAKE_IID_OF_QNAME_SAFE ($25 "callret")
      12: $27 "callret" := Call __BOX_FLAGS_TWEAK (<constant (http://www.w3.org/1999/02/22-rdf-syntax-ns#type)>, <constant (1)>)
      17: $28 "-ns#type" := Call DB.DBA.RDF_MAKE_IID_OF_QNAME_SAFE ($27 "callret")
      24: $29 "callret" := Call __BOX_FLAGS_TWEAK (<constant (http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#GraduateStudent)>, <constant (1)>)
      29: $30 "owl#GraduateStudent" := Call DB.DBA.RDF_MAKE_IID_OF_QNAME_SAFE ($29 "callret")
      36: BReturn 0
from DB.DBA.RDF_QUAD by RDF_QUAD_OGPS    1.9e+03 rows
Key RDF_QUAD_OGPS  ASC ($32 "s-3-1-t0.S")
<col=415 O = $30 "owl#GraduateStudent"> , <col=412 G = $26 "lubm"> , <col=414 P = $28 "-ns#type">
row specs: <col=415 O LIKE <constant (T)>>

Current of: <$34 "<DB.DBA.RDF_QUAD s-3-1-t0>" spec 5>

After code:
      0: $35 "x" := Call ID_TO_IRI ($32 "s-3-1-t0.S")
      5: BReturn 0
Select ($35 "x", <$34 "<DB.DBA.RDF_QUAD s-3-1-t0>" spec 5>)
}

22 Rows. -- 1 msec.
]]></programlisting>
      <para>This finds the graduate student instances in the LUBM graph. First the query converts the IRI literals
      to IDs. Then, using a match of <code>OG</code> on <code>OGPS</code>, it finds the IRIs of the graduate
      students. Then, it converts the IRI ID to return to the string form.</para>
      <para>The cardinality estimate of 1.9e+03 rows is on the <code>FROM</code> line.</para>
      <para>Doing an <code>EXPLAIN()</code> on the queries will show the cardinality estimates. To drill down
      further, one can split the query into smaller chunks and see the estimates for these, up to doing it at the
      triple pattern level. To indicate a variable that is bound but whose value is not a literal known at compile
      time, one can use the parameter marker <emphasis>??</emphasis>.</para>
<programlisting><![CDATA[
SQL> EXPLAIN 
  ('
      SPARQL 
      DEFINE  sql:table-option "order"  
      PREFIX  ub:  <http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#>
      SELECT *
      FROM <lubm>
      WHERE { ?x  rdf:type  ?? }
  ');
]]></programlisting>
      <para>This will not know the type but will know that a type will be provided. So instead of guessing 1900
      matches, this will guess a smaller number, which is obviously less precise. Thus literals are generally
      better.</para>
<para>In some cases, generally to work around an optimization error, one can specify an explicit 
      <emphasis>JOIN</emphasis> order. This is done with the <emphasis>sql:select-option "order"</emphasis> clause
      in the SPARQL query prefix:</para>
<programlisting><![CDATA[
SQL> SELECT SPARQL_to_sql_text 
  ('  
      DEFINE sql:select-option "order" 
      PREFIX  ub:  <http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#>
      SELECT *
      FROM <lubm>
      WHERE
        {
          ?x  rdf:type        ub:GraduateStudent                                       .
          ?x  ub:takesCourse  <http://www.Department0.University0.edu/GraduateCourse0>
        }
  ');
]]></programlisting>
<para>shows the SQL text with the order option at the end.</para>
<para>If an estimate is radically wrong then this should be reported as a bug.</para>
      <para>If there is a <code>FROM</code> with a <code>KEY</code> on the next line and no column specs then
      this is a full table scan. The more columns are specified the less rows will be passed to the next operation
      in the chain. In the example above, there are three columns whose values are known before reading the table
      and these columns are leading columns of the index in use so column specs are:</para>
<programlisting><![CDATA[
<col=415 O = $30 "owl#GraduateStudent"> , 
<col=412 G = $26 "lubm"> , 
<col=414 P = $28 "-ns#type">
]]></programlisting>
      <note><para>Note: A <code>KEY</code> with only a row spec is a full table scan with the row spec applied as
      a filter. This is usually not good unless this is specifically intended.</para></note>
      <para>If queries are compiled to make full table scans when this is not specifically intended, this should
      be reported as a bug. The <code>explain ()</code> output and the query text should be included in the
      report.</para>
<para>Consider:</para>
<programlisting><![CDATA[
SQL> EXPLAIN 
  ('
      SPARQL 
      DEFINE sql:select-option "order, loop" 
      PREFIX  ub:  <http://www.lehigh.edu/~zhp2/2004/0401/univ-bench.owl#>
      SELECT *
      FROM <lubm>
      WHERE
        {
          ?x  ub:takesCourse  ?c                  .
          ?x  rdf:type        ub:GraduateStudent
        }
  ');
]]></programlisting>
      <para>One will see in the output that the first table access is to retrieve all in the LUBM graph which take
      some course and then later to check if this is a graduate student. This is obviously not the preferred order
      but the <emphasis>sql:select-option "order"</emphasis> forces the optimizer to join from left to right.
      </para>
      <para>It is very easy to end up with completely unworkable query plans in this manner but if the optimizer
      really is in error, then this is the only way of overriding its preferences. The effect of
      <emphasis>sql:select-option</emphasis> is pervasive, extending inside unions, optionals, subqueries etc.
      within the statement.</para>
      <para>We note that if, in the above query, both the course taken by the student and the type of the student
      are given, the query compilation will be, at least for all non-cluster cases, an index intersection. This is
      not overridden by the <code>sql:select-option</code> clause since an index intersection is always a safe
      guess, regardless of the correctness of the cardinality guesses of the patterns involved.</para>

  <sect3 id="rdfperfcosttransanalyze"><title>Translate and Analyze modes for analyzing sparql queries</title>
    <para>Virtuoso Release 6.4 ISQL offers 2 new modes for analyzing sparql queries:</para>
    <orderedlist>
      <listitem>Translate a sparql query into the correspondent sql: 
<programlisting><![CDATA[
SQL> SET SPARQL_TRANSLATE ON;
SQL> SELECT * FROM <graph> WHERE {?S a ?O};
SQL> SET SPARQL_TRANSLATE OFF;	
]]></programlisting>
      </listitem>
      <listitem>Analyze a given SQL query: 
<programlisting><![CDATA[
SQL> SET EXPLAIN ON;
SQL> SELECT * FROM TABLE WHERE field = 'text';
SQL> SET EXPLAIN OFF;	
]]></programlisting>
       <itemizedlist mark="bullet">
         <listitem><link linkend="fn_explain"><function>explain</function> ()</link> is much more 
         difficult to use since you cannot just cut and past a query as all quotes need to be 
         doubled inside the <code>explain (' ... ')</code>:
<programlisting><![CDATA[
SQL> explain('select * from table where field = ''text''');
]]></programlisting>         
         </listitem>
       </itemizedlist>
      </listitem>
    </orderedlist>    
    <para>Here is simple example of how to combine the two options to get a full explain plan for a simple SPARQL query:</para>
    <orderedlist>
      <listitem>Assume the following query: 
<programlisting><![CDATA[
SELECT * 
FROM <http://dbpedia.org> 
WHERE 
  {
    ?s a ?o
  } 
LIMIT 10	
]]></programlisting>
      </listitem>
      <listitem>Connect using the ISQL command line tool to your database and execute: 
<programlisting><![CDATA[
SQL> SET BLOBS ON;			-- in case output is very large
SQL> SET SPARQL_TRANSLATE ON;
SQL> SELECT * FROM <http://dbpedia.org> WHERE {?s a ?o} LIMIT 10;

SPARQL_TO_SQL_TEXT
VARCHAR
_______________________________________________________________________________

SELECT TOP 10 __id2i ( "s_1_0-t0"."S" ) AS "s",
  __ro2sq ( "s_1_0-t0"."O" ) AS "o"
FROM DB.DBA.RDF_QUAD AS "s_1_0-t0"
WHERE "s_1_0-t0"."G" = __i2idn ( __bft( 'http://dbpedia.org' , 1))
  AND  "s_1_0-t0"."P" = __i2idn ( __bft( 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' , 1))
OPTION (QUIETCAST)

1 Rows. -- 1 msec.

SQL> SET SPARQL_TRANSLATE OFF;	
]]></programlisting>
      </listitem>
      <listitem>Use mouse to select the above query output and paste it after the <code>SET EXPLAIN ON;</code> 
      	command. After pasting in the command, followed by the ENTER key: 
<programlisting><![CDATA[
SQL> SET EXPLAIN ON;
SQL> SELECT TOP 10 __id2i ( "s_1_0-t0"."S" ) AS "s", __ro2sq ( "s_1_0-t0"."O" ) AS "o"
 FROM DB.DBA.RDF_QUAD AS "s_1_0-t0"
 WHERE "s_1_0-t0"."G" = __i2idn ( __bft( 'http://dbpedia.org' , 1))
   AND  "s_1_0-t0"."P" = __i2idn ( __bft( 'http://www.w3.org/1999/02/22-rdf-syn tax-ns#type' , 1))
 OPTION (QUIETCAST)
;

REPORT
VARCHAR
_______________________________________________________________________________

{ 
from DB.DBA.RDF_QUAD by RDF_QUAD_POGS    4.5e+05 rows   
Key RDF_QUAD_POGS  ASC ($22 "s_1_0-t0.S", $21 "s_1_0-t0.O")
 inlined <col=556 P =  #type >
row specs: <col=554 G =  #http://dbpedia.org >
 
 
After code:
      0: $25 "s" := Call __id2i ($22 "s_1_0-t0.S")
      5: $26 "o" := Call __ro2sq ($21 "s_1_0-t0.O")
      10: BReturn 0
Select (TOP  10 ) ($25 "s", $26 "o", <$24 "<DB.DBA.RDF_QUAD s_1_0-t0>" spec 5>)
}

13 Rows. -- 1 msec.

SQL> SET EXPLAIN OFF;	
]]></programlisting>
      </listitem>
    </orderedlist>
  </sect3>
</sect2>
<sect2><title>Using "swappiness" parameter ( Linux only )</title>
  <para><emphasis>For Linux users only</emphasis>, there is a kernel tuning parameter called "<code>swappiness</code>"
  that controls how much the kernel favors swap over RAM.
  </para>
  <para>When hosting large data sets, it is recommended that this parameter be changed from its default value of 60
  to something closer to 10, to reduce the amount of swapping that takes place on the server. Useful tidbits
  regarding swappiness include:
  </para>
  <itemizedlist mark="bullet">
    <listitem>The swappiness setting is found in the file <code>/proc/sys/vm/swappiness</code>.</listitem>
    <listitem>The command <code>/sbin/sysctl vm.swappiness</code> can be used to view its setting.</listitem>
    <listitem>The command <code>/sbin/sysctl -w vm.swappiness=10</code> can be used to change its value.</listitem>
    <listitem>Adding <code>vm.swappiness = 10</code> to the file <code>/etc/sysctl.conf</code> will force the
    value to be set at machine boot time.</listitem>
  </itemizedlist>
</sect2>
<sect2 id="rdfperfgetallgraphs"><title>Get All Graphs</title>
<para>In order to get all graphs URIs, one might use the Virtuoso 
<link linkend="fn_sparql_select_known_graphs"><function>DB.DBA.SPARQL_SELECT_KNOWN_GRAPHS()</function></link> 
built-in function.</para>
</sect2>
<sect2 id="rdfrenamegraph"><title>Rename RDF Graph and RDF Graph Groups</title>
  <para>A RDF Graph in the Virtuoso Quad Store can be renamed without copying each assertion from the old 
  	graph to the new graph using a SQL statement, this being what the Conductor "rename" option does, 
  	which is:</para>
<programlisting><![CDATA[
UPDATE DB.DBA.RDF_QUAD TABLE OPTION (index RDF_QUAD_GS) 
   SET g = iri_to_id ('new') 
 WHERE g = iri_to_id ('old', 0);  
]]></programlisting>  	
  <para><emphasis>Note:</emphasis> this operation must be run in row-autocommit mode i.e. 
  log_enable (3), and then restore back to the default logging mode of 1.</para>
  <para>For Virtuoso Graph Groups two tables need to be updated:</para>
<programlisting><![CDATA[
UPDATE DB.DBA.RDF_GRAPH_GROUP_MEMBER 
   SET RGGM_GROUP_IID = iri_to_id ('new') 
 WHERE RGGM_GROUP_IID = iri_to_id (old)	
]]></programlisting>   
  <para>and</para>
<programlisting><![CDATA[
UPDATE DB.DBA.RDF_GRAPH_GROUP 
   SET RGG_IID = iri_to_id ('new') , RGG_IRI = 'new' 
 WHERE RGG_IRI = 'old'	
]]></programlisting>   
</sect2>
<sect2 id="rdfperfdumpandreloadgraphs"><title>Dump and Reload Graphs</title>
  <sect3 id="rdfperfdumpandreloadgraphswhat"><title>What?</title>
    <para>How to export RDF model data from Virtuoso's Quad Store.</para>
  </sect3>
  <sect3 id="rdfperfdumpandreloadgraphswhy"><title>Why?</title>
    <para>Every DBMS needs to offer a mechanism for bulk export and import of data.</para>
    <para>Virtuoso supports dumping and reloading graph model data (e.g., RDF), as well as 
    	relational data (e.g., SQL) (discussed elsewhere).</para>
  </sect3>
  <sect3 id="rdfperfdumpandreloadgraphshow"><title>How?</title>
    <para>We have created stored procedures for the task. The dump procedures leverage SPARQL to 
    	facilitate selective data dump(s) from one or more Named Graphs, each denoted by an IRI.
    </para>	
    <sect4 id="rdfperfdumpandreloadgraphdumponegraph"><title>Dump One Graph</title>
      <sect5 id="rdfperfdumpandreloadgraphdumponegraphparams"><title>Parameters</title>
        <para>The procedure dump_one_graph has the following parameters:</para>
        <itemizedlist mark="bullet">
          <listitem>IN <emphasis>srcgraph</emphasis> VARCHAR -- source graph</listitem>
          <listitem>IN <emphasis>out_file</emphasis> VARCHAR -- output file </listitem>
          <listitem>IN <emphasis>file_length_limit</emphasis> INTEGER -- maximum length of dump files</listitem>          
        </itemizedlist>
      </sect5>
      <sect5 id="rdfperfdumpandreloadgraphdumponegraphsource"><title>Source</title>
        <para>The procedure <emphasis>dump_one_graph</emphasis> has the following source:</para>
<programlisting><![CDATA[
CREATE PROCEDURE dump_one_graph 
  ( IN  srcgraph           VARCHAR  , 
    IN  out_file           VARCHAR  , 
    IN  file_length_limit  INTEGER  := 1000000000
  )
  {
    DECLARE  file_name     VARCHAR;
    DECLARE  env, 
             ses           ANY;
    DECLARE  ses_len, 
             max_ses_len, 
             file_len, 
             file_idx      INTEGER;
    SET ISOLATION = 'uncommitted';
    max_ses_len  := 10000000;
    file_len     := 0;
    file_idx     := 1;
    file_name    := sprintf ('%s%06d.ttl', out_file, file_idx);
    string_to_file ( file_name || '.graph', 
                     srcgraph, 
                     -2
                   );
    string_to_file ( file_name, 
                     sprintf ( '# Dump of graph <%s>, as of %s\n', 
                               srcgraph, 
                               CAST (NOW() AS VARCHAR)
                             ), 
                     -2
                   );
    env := vector (dict_new (16000), 0, '', '', '', 0, 0, 0, 0);
    ses := string_output ();
    FOR (SELECT * FROM ( SPARQL DEFINE input:storage "" 
                         SELECT ?s ?p ?o { GRAPH `iri(?:srcgraph)` { ?s ?p ?o } } 
                       ) AS sub OPTION (LOOP)) DO
      {
        http_ttl_triple (env, "s", "p", "o", ses);
        ses_len := length (ses);
        IF (ses_len > max_ses_len)
          {
            file_len := file_len + ses_len;
            IF (file_len > file_length_limit)
              {
                http (' .\n', ses);
                string_to_file (file_name, ses, -1);
                file_len := 0;
                file_idx := file_idx + 1;
                file_name := sprintf ('%s%06d.ttl', out_file, file_idx);
                string_to_file ( file_name, 
                                 sprintf ( '# Dump of graph <%s>, as of %s (part %d)\n', 
                                           srcgraph, 
                                           CAST (NOW() AS VARCHAR), 
                                           file_idx), 
                                 -2
                               );
                 env := VECTOR (dict_new (16000), 0, '', '', '', 0, 0, 0, 0);
              }
            ELSE
              string_to_file (file_name, ses, -1);
            ses := string_output ();
          }
      }
    IF (LENGTH (ses))
      {
        http (' .\n', ses);
        string_to_file (file_name, ses, -1);
      }
  }
;	
]]></programlisting>        
      </sect5>
      <sect5 id="rdfperfdumpandreloadgraphdumponegraphex"><title>Example</title>
        <orderedlist>
          <listitem>Call the <emphasis>dump_one_graph</emphasis> procedure with appropriate arguments:
<programlisting><![CDATA[

$ pwd 
/Applications/OpenLink Virtuoso/Virtuoso 6.1/database

$ grep DirsAllowed virtuoso.ini
DirsAllowed              = ., ../vad, ./dumps

$ /opt/virtuoso/bin/isql 1111
Connected to OpenLink Virtuoso
Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> dump_one_graph ('http://daas.openlinksw.com/data#', './data_', 1000000000); 
Done. -- 1438 msec.	
]]></programlisting>          
          </listitem>
          <listitem>As a result, a dump of the graph will be found in the files data_XX (located in your Virtuoso db folder): 
<programlisting><![CDATA[
<verbatim>
$ ls dumps
data_000001.ttl
data_000002.ttl
....
data_000001.ttl.graph	
]]></programlisting>          	
          </listitem>
        </orderedlist>
      </sect5>
    </sect4>
    <sect4 id="rdfperfdumpandreloadgraphdumpmlpgraph"><title>Dump Multiple Graphs</title>
      <para>The <emphasis>dump_graphs</emphasis> procedure can be used to dump all the graphs in a Virtuoso server to 
      	a set of turtle (.ttl) data files in the specified dump directory.
      </para>	
      <sect5 id="rdfperfdumpandreloadgraphdumpmlpgraphparams"><title>Parameters</title>
        <para>The procedure dump_graphs has the following parameters:</para>
        <itemizedlist mark="bullet">
          <listitem>IN <emphasis>dir</emphasis> VARCHAR -- dump directory</listitem>
          <listitem>IN <emphasis>file_length_limit</emphasis> INTEGER -- maximum length of dump files</listitem>          
        </itemizedlist>
        <para><i>Note: The dump directory must be included in the DirsAllowed parameter of the Virtuoso 
        configuration file (virtuoso.ini), or the Virtuoso server will not be able to create or access 
        the data files.</i>
        </para>	
      </sect5>
      <sect5 id="rdfperfdumpandreloadgraphdumpmlpgraphsource"><title>Source</title>
        <para>The procedure <emphasis>dump_graphs</emphasis> has the following source:</para>
<programlisting><![CDATA[
CREATE PROCEDURE dump_graphs 
  ( IN  dir               VARCHAR  :=  'dumps'   , 
    IN  file_length_limit INTEGER  :=  1000000000
  )
  {
    DECLARE inx INT;
    inx := 1;
    SET ISOLATION = 'uncommitted';
    FOR ( SELECT * 
            FROM ( SPARQL DEFINE input:storage "" 
                   SELECT DISTINCT ?g { GRAPH ?g { ?s ?p ?o } . 
                                        FILTER ( ?g != virtrdf: ) 
                                      } 
                 ) AS sub OPTION ( LOOP )) DO
      {
        dump_one_graph ( "g", 
                         sprintf ('%s/graph%06d_', dir, inx), 
                         file_length_limit
                       );
        inx := inx + 1;
      }
  }
;
]]></programlisting>        
      </sect5>
      <sect5 id="rdfperfdumpandreloadgraphdumpmlpgraphex"><title>Example</title>
        <orderedlist>
          <listitem>Call the <emphasis>dump_graphs </emphasis> procedure:
<programlisting><![CDATA[
$ pwd
/Applications/OpenLink Virtuoso/Virtuoso 6.1/database

$ grep DirsAllowed virtuoso.ini
DirsAllowed              = ., ../vad, ./dumps

$ /opt/virtuoso/bin/isql 1111
Connected to OpenLink Virtuoso
Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> dump_graphs();

Done. -- 998 msec.
SQL> quit;
]]></programlisting>          
          </listitem>
          <listitem>As a result, a dump of the graph will be found in the files dumps/data_XX (located in your Virtuoso db folder): 
<programlisting><![CDATA[
<verbatim>
$ ls dumps
graph000001_000001.ttl		graph000005_000001.ttl
graph000001_000001.ttl.graph	graph000005_000001.ttl.graph
graph000002_000001.ttl		graph000006_000001.ttl
graph000002_000001.ttl.graph	graph000006_000001.ttl.graph
graph000003_000001.ttl		graph000007_000001.ttl
graph000003_000001.ttl.graph	graph000007_000001.ttl.graph
graph000004_000001.ttl		graph000008_000001.ttl
graph000004_000001.ttl.graph	graph000008_000001.ttl.graph
]]></programlisting>          	
          </listitem>
        </orderedlist>
      </sect5>
    </sect4>
    <sect4 id="rdfperfdumpandreloadgraphloadgraph"><title>Load Graphs</title>
      <para>The stored procedure <emphasis>load_graphs</emphasis> procedure performs a bulk 
      load from a file.
      </para>	
      <sect5 id="rdfperfdumpandreloadgraphloadgraphparams"><title>Parameters</title>
        <para>The procedure load_graphs has the following parameters:</para>
        <itemizedlist mark="bullet">
          <listitem>IN <emphasis>dir</emphasis> VARCHAR -- dump directory</listitem>
        </itemizedlist>
        <para><i>Note: The dump directory must be included in the DirsAllowed parameter of the Virtuoso 
        configuration file (virtuoso.ini), or the Virtuoso server will not be able to create or access 
        the data files.</i>
        </para>	
      </sect5>
      <sect5 id="rdfperfdumpandreloadgraphloadgraphsource"><title>Source</title>
        <para>The procedure <emphasis>load_graphs</emphasis> has the following source:</para>
<programlisting><![CDATA[
CREATE PROCEDURE load_graphs 
  ( IN  dir  VARCHAR := 'dumps/' )
{
  DECLARE arr ANY;
  DECLARE g VARCHAR;

  arr := sys_dirlist (dir, 1);
  log_enable (2, 1);
  FOREACH (VARCHAR f IN arr) DO
    {
      IF (f LIKE '*.ttl')
	{
	  DECLARE CONTINUE HANDLER FOR SQLSTATE '*'
	    {
	      log_message (sprintf ('Error in %s', f));
	    };
  	  g := file_to_string (dir || '/' || f || '.graph');
	  DB.DBA.TTLP_MT (file_open (dir || '/' || f), g, g, 255);
	}
    }
  EXEC ('CHECKPOINT');
}
;
]]></programlisting>        
      </sect5>
      <sect5 id="rdfperfdumpandreloadgraphloadgraphex"><title>Example</title>
<programlisting><![CDATA[
$ /opt/virtuoso/bin/isql 1112
Connected to OpenLink Virtuoso
Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> load_graphs();

Done. -- 2392 msec.
SQL> 
]]></programlisting>          
      </sect5>
    </sect4>
  </sect3>  
</sect2>
<sect2 id="rdfperfdumpintonquads"><title>RDF dumps from Virtuoso Quad store hosted data into NQuad dumps</title>
  <sect3 id="rdfperfdumpintonquadswhat"><title>What?</title>
    <para>How to export RDF model data from Virtuoso's Quad Store in NQuad format.</para>
  </sect3>
  <sect3 id="rdfperfdumpintonquadswhy"><title>Why?</title>
    <para>When exporting RDF model data from Virtuoso's Quad Store, having the ability to 
    retain and reflect Named Graph IRI based data partitioning is provide significant value to 
    a variety of application profiles.
    </para>
  </sect3>
  <sect3 id="rdfperfdumpintonquadshow"><title>How?</title>
    <para>We have created stored procedures for the task. The dump procedure <emphasis>dump_nquads</emphasis> 
    leverage SPARQL to facilitate data dump(s) for all graphs excluding the predefined "virtrdf:" one.
    </para>	
    <sect4 id="rdfperfdumpintonquadsprc"><title>Dump NQuads</title>
      <sect5 id="rdfperfdumpintonquadsprcparams"><title>Parameters</title>
        <para>The procedure <emphasis>dump_nquads</emphasis> has the following parameters:</para>
        <itemizedlist mark="bullet">
          <listitem>IN <emphasis>dir</emphasis> VARCHAR -- folder where the dumps will be stored </listitem>
          <listitem>IN <emphasis>outstart_fromfile</emphasis> INTEGER  -- output start from number n</listitem>
          <listitem>IN <emphasis>file_length_limit</emphasis> INTEGER -- maximum length of dump files</listitem>          
          <listitem>IN <emphasis>comp</emphasis> INTEGER -- when set to 0, then no gzip will be done. By default is set to 1.</listitem>          
        </itemizedlist>
      </sect5>
      <sect5 id="rdfperfdumpintonquadsprcsource"><title>Source</title>
        <para>The procedure <emphasis>dump_nquads</emphasis> has the following source:</para>
<programlisting><![CDATA[
create procedure dump_nquads (in dir varchar := 'dumps', in start_from int := 1, in file_length_limit integer := 100000000, in comp int := 1)
{
  declare inx, ses_len int;
  declare file_name varchar;
  declare env, ses any;

  inx := start_from;
  set isolation = 'uncommitted';
  env := vector (0,0,0);
  ses := string_output (10000000);
  for (select * from (sparql define input:storage "" select ?s ?p ?o ?g { graph ?g { ?s ?p ?o } . filter ( ?g != virtrdf: ) } ) as sub option (loop)) do
    {
      declare exit handler for sqlstate '22023' 
	{
	  goto next;
	};
      http_nquad (env, "s", "p", "o", "g", ses);
      ses_len := length (ses);
      if (ses_len >= file_length_limit)
	{
	  file_name := sprintf ('%s/output%06d.nq', dir, inx);
	  string_to_file (file_name, ses, -2);
	  if (comp)
	    {
	      gz_compress_file (file_name, file_name||'.gz');
	      file_delete (file_name);
	    }
	  inx := inx + 1;
	  env := vector (0,0,0);
	  ses := string_output (10000000);
	}
      next:;
    }
  if (length (ses))
    {
      file_name := sprintf ('%s/output%06d.nq', dir, inx);
      string_to_file (file_name, ses, -2);
      if (comp)
	{
	  gz_compress_file (file_name, file_name||'.gz');
	  file_delete (file_name);
	}
      inx := inx + 1;
      env := vector (0,0,0);
    }
}
;
]]></programlisting>        
      </sect5>
      <sect5 id="rdfperfdumpintonquadsprcex"><title>Example</title>
        <para>This example demonstrates calling the <emphasis>dump_nquads</emphasis> procedure in order to dump all graphs 
        in a compressed nquad dumps, each uncompressed with length 10Mb (./dumps/output000001.nq.gz) :
        </para>
<programlisting><![CDATA[
SQL> dump_nquads ('dumps', 1, 10000000, 1);
]]></programlisting>          
      </sect5>
    </sect4>
  </sect3>  
</sect2>
<sect2 id="rdfperfdumpandreloadgraphsn3"><title>Dump Linked Data View Graph to n3</title>
<para>The RDF_QM_TREE_DUMP procedure and its associated procedures below are used 
for dumping one or more RDFView Graphs in a Virtuoso server to a set of 
turtle ttl dataset files in the specified dump directory. The dump generation 
is made as fast as possible by grouping mappings by underlying tables so many 
properties from neighbor database columns can be extracted in one table scan. 
The size of the generated files is limited to 5MB. The dump process creates 
internal stored procedures; their texts are saved in file .dump_procedures.sql in 
the directory of dump files for debugging purposes.
</para>

<para>Note that the dump directory must be included in the <code>DirsAllowed</code> 
parameter of the Virtuoso configuration file (e.g., <code>virtuoso.ini</code>), or the 
server will not be allowed to create nor access the dataset file(s).
</para>

<para>
The <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtBulkRDFLoader">Virtuoso RDF bulk loader</ulink> 
scripts can then be used to load the dumped datasets for the RDFView graphs directly into 
a Virtuoso RDF QUAD store.
</para>

<sect3 id="rdfperfdumpandreloadgraphsn3params"><title>Parameters</title>
<itemizedlist mark="bullet">
  <listitem><code>in</code> <emphasis>dest_dir</emphasis> <code>VARCHAR</code> - dump directory </listitem>
  <listitem><code>in</code> <emphasis>graph_iri</emphasis> <code>VARCHAR</code> - IRI of the graph to be dumped; triples from other graphs will be excluded. If NULL, then there's no restriction by graph.</listitem>
  <listitem><code>in</code> <emphasis>storage</emphasis> <code>VARCHAR</code> - IRI of the quad map storage to use. NULL means use default storage.</listitem>
  <listitem><code>in</code> <emphasis>root</emphasis> <code>VARCHAR</code> - IRI of the quad map to use, e.g., an IRI of an Linked Data View (or its part). NULL means use all Linked Data Views of the storage (and the default mapping as well).</listitem>    
</itemizedlist>
</sect3>
<sect3 id="rdfperfdumpandreloadgraphsn3code"><title>Procedure Code</title>
<programlisting><![CDATA[
CREATE PROCEDURE DB.DBA.RDF_QM_TREE_DUMP 
  ( in  dest_dir  VARCHAR, 
    in  graph_iri VARCHAR := NULL, 
    in  storage   VARCHAR := NULL, 
    in  root      VARCHAR := NULL
  )
{
 DECLARE all_qms, 
         grouped_qmvs, 
         launcher_text  ANY;
 DECLARE grp_ctr, 
         qm_ctr, 
         qm_count       INTEGER;
 DECLARE sql_file, 
         launcher_name  VARCHAR;
 IF (NOT (dest_dir LIKE '%/'))
   dest_dir := dest_dir || '/';
 sql_file := dest_dir || '.dump_procedures.sql';
 IF (storage IS NULL)
   storage := 'http://www.openlinksw.com/schemas/virtrdf#DefaultQuadStorage';
 string_to_file (
   sql_file, 
   '-- This file contains procedure created by DB.DBA.RDF_QM_TREE_DUMP() for storage ' 
      || COALESCE (storage, 'NULL') 
      || ' and root quad map ' 
      || COALESCE (root, 'NULL') 
      || '\n\n', 
   -2);
 all_qms := dict_list_keys (DB.DBA.RDF_QM_CONTENT_OF_QM_TREE (graph_iri, storage, root), 2);
 grouped_qmvs := DB.DBA.RDF_QM_GROUP_BY_SOURCE_TABLES (all_qms);
 launcher_name := 'RDF_QM_TREE_DUMP_BATCH_' || md5 (serialize (graph_iri) || storage || serialize (root));
 launcher_text := string_output ();
 http ('CREATE PROCEDURE DB.DBA."' || launcher_name || '" (in dest_dir VARCHAR)\n{\n', launcher_text);
 FOR (grp_ctr := length (grouped_qmvs); grp_ctr > 0; grp_ctr := grp_ctr-2)
   {
     DECLARE tables, qms, proc_text ANY;
     DECLARE group_key, proc_name, dump_prefix, cmt VARCHAR;
     tables := grouped_qmvs [grp_ctr-2];
     qms := grouped_qmvs [grp_ctr-1];
     qm_count := length (qms);
     group_key := md5 (serialize (graph_iri) || storage || serialize (root) || serialize (tables));
     proc_name := 'RDF_QM_TREE_DUMP_GRP_' || group_key;
     proc_text := string_output ();
     cmt := sprintf ('%d quad maps on join of', qm_count);
     FOREACH (VARCHAR t IN tables) DO cmt := cmt || ' ' || t;
     http ('  --  ' || cmt || '\n', launcher_text);
     http ('  DB.DBA."' || proc_name || '" (dest_dir);\n', launcher_text);
     http ('CREATE PROCEDURE DB.DBA."' || proc_name || '" (in dest_dir VARCHAR)\n', proc_text);
     http ('{\n', proc_text);
     http ('  -- ' || cmt || '\n', proc_text);
     http ('  DECLARE ses, env ANY;\n', proc_text);
     http ('  DECLARE file_ctr, cmt_len INTEGER;\n', proc_text);
     http ('  file_ctr := 0;\n', proc_text);
     http ('  dbg_obj_princ (' || WS.WS.STR_SQL_APOS (cmt) || ', '', file '', file_ctr);\n', proc_text);
     http ('  ses := string_output ();\n', proc_text);
     http ('  http (' || WS.WS.STR_SQL_APOS ('#' || cmt || '\n') || ', ses);\n', proc_text);
     http ('  env := VECTOR (dict_new (16000), 0, '''', '''', '''', 0, 0, 0, 0);\n', proc_text);
     http ('  cmt_len := LENGTH (ses);\n', proc_text);
     http ('  FOR (SPARQL DEFINE input:storage <' || storage || '>\n', proc_text);
     http ('    SELECT ?s1, ?p1, ?o1\n', proc_text);
     IF (graph_iri IS NOT NULL)
       {
         http ('    WHERE { GRAPH <', proc_text); http_escape (graph_iri, 12, proc_text, 1, 1); http ('> {\n', proc_text);
       }
     ELSE
       http ('    WHERE { GRAPH ?g1 {\n', proc_text);
     FOR (qm_ctr := 0; qm_ctr < qm_count; qm_ctr := qm_ctr + 1)
       {
         IF (qm_ctr > 0) http ('            UNION\n', proc_text);
         http ('            { quad map <' || qms[qm_ctr] || '> { ?s1 ?p1 ?o1 } }\n', proc_text);
       }
     http ('          } } ) DO {\n', proc_text);
     http ('      http_ttl_triple (env, "s1", "p1", "o1", ses);\n', proc_text);
     http ('      IF (LENGTH (ses) > 5000000)\n', proc_text);
     http ('        {\n', proc_text);
     http ('          http ('' .\\n'', ses);\n', proc_text);
     http ('          string_to_file (sprintf (''%s' || group_key || '_%05d.ttl'', dest_dir, file_ctr), ses, -2);\n', proc_text);
     http ('          file_ctr := file_ctr + 1;\n', proc_text);
     http ('          dbg_obj_princ (' || WS.WS.STR_SQL_APOS (cmt) || ', '', file '', file_ctr);\n', proc_text);
     http ('          ses := string_output ();\n', proc_text);
     http ('          http (' || WS.WS.STR_SQL_APOS ('#' || cmt || '\n') || ', ses);\n', proc_text);
     http ('          env := VECTOR (dict_new (16000), 0, '''', '''', '''', 0, 0, 0, 0);\n', proc_text);
     http ('        }\n', proc_text);
     http ('    }\n', proc_text);
     http ('  IF (LENGTH (ses) > cmt_len)\n', proc_text);
     http ('    {\n', proc_text);
     http ('      http ('' .\\n'', ses);\n', proc_text);
     http ('      string_to_file (sprintf (''%s' || group_key || '_%05d.ttl'', dest_dir, file_ctr), ses, -2);\n', proc_text);
     http ('    }\n', proc_text);
     http ('}\n', proc_text);
     proc_text := string_output_string (proc_text);
     string_to_file (sql_file, proc_text || ';\n\n' , -1);
     EXEC (proc_text);
   }
 http ('}\n', launcher_text);
 launcher_text := string_output_string (launcher_text);
 string_to_file (sql_file, launcher_text || ';\n\n' , -1);
 EXEC (launcher_text);
 CALL ('DB.DBA.' || launcher_name)(dest_dir);
}
;

CREATE FUNCTION DB.DBA.RDF_QM_CONTENT_OF_QM_TREE 
  ( in  graph_iri  VARCHAR := NULL,
    in  storage    VARCHAR := NULL, 
    in  root       VARCHAR := NULL, 
    in  dict       ANY := NULL
  ) returns ANY
{
 DECLARE res, subqms any;
 DECLARE graphiri varchar;
 graphiri := DB.DBA.JSO_SYS_GRAPH();
 IF (storage IS NULL)
   storage := 'http://www.openlinksw.com/schemas/virtrdf#DefaultQuadStorage';
 DB.DBA.RDF_QM_ASSERT_STORAGE_FLAG (storage, 0);
 IF (dict IS NULL)
   dict := dict_new ();
 IF (root IS NULL)
   {
     subqms := ((SELECT DB.DBA.VECTOR_AGG (sub."qmiri")
         FROM (
           SPARQL DEFINE input:storage ""
           SELECT DISTINCT (str(?qm)) AS ?qmiri
           WHERE { GRAPH `iri(?:graphiri)` {
                     { `iri(?:storage)` virtrdf:qsUserMaps ?lst .
                       ?lst ?p ?qm .
                       FILTER (0 = bif:strstr (str(?p), str(rdf:_)))
                     } UNION {
                       `iri(?:storage)` virtrdf:qsDefaultMap ?qm .
                     } } } ) AS sub ) );
     FOREACH (varchar qmid IN subqms) DO
       DB.DBA.RDF_QM_CONTENT_OF_QM_TREE (graph_iri, storage, qmid, dict);
     RETURN dict;
   }
 DB.DBA.RDF_QM_ASSERT_JSO_TYPE (root, 'http://www.openlinksw.com/schemas/virtrdf#QuadMap');
 IF (graph_iri IS NOT NULL AND
   EXISTS ((SPARQL DEFINE input:storage ""
       SELECT (1) WHERE {
           GRAPH `iri(?:graphiri)` {
               `iri(?:root)` virtrdf:qmGraphRange-rvrFixedValue ?g .
               FILTER (str (?g) != str(?:graph_iri))
             } } ) ) )
   RETURN dict;
 IF (NOT EXISTS ((SPARQL DEFINE input:storage ""
       SELECT (1) WHERE {
           GRAPH `iri(?:graphiri)` {
               `iri(?:root)` virtrdf:qmMatchingFlags virtrdf:SPART_QM_EMPTY .
             } } ) ) )
   dict_put (dict, root, 1);
 subqms := ((SELECT DB.DBA.VECTOR_AGG (sub."qmiri")
     FROM (
       SPARQL DEFINE input:storage ""
       SELECT DISTINCT (str(?qm)) as ?qmiri
       WHERE { GRAPH `iri(?:graphiri)` {
   		`iri(?:root)` virtrdf:qmUserSubMaps ?lst .
               ?lst ?p ?qm .
               FILTER (0 = bif:strstr (str(?p), str(rdf:_)))
             } } ) AS sub ) );
 FOREACH (VARCHAR qmid IN subqms) DO
   DB.DBA.RDF_QM_CONTENT_OF_QM_TREE (graph_iri, storage, qmid, dict);
 RETURN dict;
}
;

CREATE FUNCTION DB.DBA.RDF_QM_GROUP_BY_SOURCE_TABLES (in qms ANY) returns ANY
{
 DECLARE res ANY;
 DECLARE ctr INTEGER;
 DECLARE graphiri VARCHAR;
 graphiri := DB.DBA.JSO_SYS_GRAPH();
 res := dict_new (LENGTH (qms) / 20);
 FOREACH (VARCHAR qmiri IN qms) DO
   {
     DECLARE tbls, acc ANY;
     tbls := ((SELECT DB.DBA.VECTOR_AGG (sub."tbl")
         FROM (SELECT subsub."tbl"
           FROM (
             SPARQL DEFINE input:storage ""
             SELECT DISTINCT ?tbl
             WHERE { GRAPH `iri(?:graphiri)` {
                       { `iri(?:qmiri)` virtrdf:qmTableName ?tbl .
                       } UNION {
                         `iri(?:qmiri)` virtrdf:qmATables ?atbls .
                         ?atbls ?p ?atbl .
                         ?atbl virtrdf:qmvaTableName ?tbl
                       } UNION {
                         `iri(?:qmiri)` ?fldmap ?qmv .
                         ?qmv virtrdf:qmvATables ?atbls .
                         ?atbls ?p ?atbl .
                         ?atbl virtrdf:qmvaTableName ?tbl .
                       } } } ) subsub
           ORDER BY 1 ) AS sub ) );
     acc := dict_get (res, tbls);
     IF (acc IS NULL)
       vectorbld_init (acc);
     vectorbld_acc (acc, qmiri);
     dict_put (res, tbls, acc);
   }
 res := dict_to_vector (res, 2);
 FOR (ctr := LENGTH (res); ctr > 0; ctr := ctr-2)
   {
     DECLARE acc ANY;
     acc := aref_set_0 (res, ctr-1);
     vectorbld_final (acc);
     aset_zap_arg (res, ctr-1, acc);
   }
 RETURN res;
}
;

--test dbg_obj_princ (DB.DBA.RDF_QM_GROUP_BY_SOURCE_TABLES (dict_list_keys (DB.DBA.RDF_QM_CONTENT_OF_QM_TREE (null), 2)));
--test dbg_obj_princ (dict_list_keys (DB.DBA.RDF_QM_CONTENT_OF_QM_TREE (null), 2));
--test DB.DBA.RDF_QM_TREE_DUMP ('dump/demo', null, null, null);
--test DB.DBA.RDF_QM_TREE_DUMP ('dump/tpch', 'http://localhost:8600/tpch', null, null);	
]]></programlisting>
</sect3>	
</sect2>	
    <sect2 id="rdfperfloading"><title>Loading RDF</title>
	<para>There are many functions for loading RDF text, in RDF/XML and Turtle.</para>
	<para>For loading RDF/XML, the best way is to split the data to be loaded into
multiple streams and load these in parallel using <link linkend="fn_rdf_load_rdfxml"><function>RDF_LOAD_RDFXML ()</function></link>.
To avoid running out of rollback space for large files and in order to have multiple concurrent loads not
interfere with each other, the row autocommit mode should be enabled.</para>
	<para>For example, </para>
<programlisting><![CDATA[
log_enable (2);
-- switch row-by-row autocommit on and logging off for this session
DB.DBA.RDF_LOAD_RDFXML (file_to_string_output ('file.xml'), 'base_uri', 'target_graph');
-- more files here ...
checkpoint;
]]></programlisting>
<para>Loading a file with text like the above with isql will load the data. Since the transaction
logging is off, make a manual checkpoint at the end to ensure that data is persisted upon server
restart since there is no roll forward log.</para>
<para>If large amounts of data are to be loaded, run multiple such streams in parallel. One may have
for example 6 streams for 4 cores. This means that if up to two threads wait for disk, there is still work
for all cores.</para>
<para>Having substantially more threads than processors or disks is not particularly useful.</para>
<para>There exist multithreaded load functions which will load one file on multiple threads: 
<link linkend="rdfapidataimportttlpmt">the DB.DBA.TTLP_MT() function</link> and 
<link linkend="rdfapidataimportxmlttlpmt">the DB.DBA.RDF_LOAD_RDFXML_MT() function</link>. Experience
shows that loading multiple files on one thread per file is better.</para>
<para>For loading Turtle, some platforms may have a non-reentrant Turtle parser. This means that only
one load may run at a time. One can try this by calling
<link linkend="rdfapidataimport"><function>ttlp ()</function></link> from two sessions at the same time.
If these do not execute concurrently, then the best way may be to try
<link linkend="rdfapidataimport"><function>ttlp_mt</function></link> and see if this runs faster than
a single threaded ttlp call.</para>
       <sect3 id="rdfperfloadingutility"><title>RDF Bulk Load Utility</title>
         <para>The RDF loader utility facilitates parallel bulk loading of multiple RDF files. The utility 
         	maintains a database table containing a list of files to load and the status of each file, 
         	whether not loaded, loaded or loaded with error. The table also records load start and 
         	end times.</para>
         <para>One must have a dba group login for using this and the virtuoso.ini file access 
         	control list must be set up so that the Virtuoso server can open  the files to load.</para>
         <para>Files are added to the load list with the function <link linkend="fn_ld_dir"><function>ld_dir</function></link>:</para>
<programlisting><![CDATA[
ld_dir (in dir_path varchar, in file_mask varchar, in target_graph varchar);
]]></programlisting>
         <para>The file mask is a SQL like pattern to match against the files in the directory. 
         	For example:</para>
<programlisting><![CDATA[
ld_dir ('/data8/2848260', '%.gz', 'http://bsbm.org');
]]></programlisting>
         <para>would load the RDF in all files ending in .gz from the directory given as first parameter. 
         	The RDF would be loaded in the http://bsbm.org graph.</para>
         <para>If NULL is given for the graph, each file may go to a different graph specified in a 
         	separate file with the name of the RDF source file plus the extension .graph.</para>
         <para>A .graph file contains the target graph URI without any other content or whitespace.</para>  
         <para>The layout of the load_list table is as follows:</para>
<programlisting><![CDATA[
create table DB.DBA.LOAD_LIST (
  ll_file varchar,
  ll_graph varchar,
  ll_state int default 0, -- 0 not started, 1 going, 2 done
  ll_started datetime,
  ll_done datetime,
  ll_host int,
  ll_work_time integer,
  ll_error varchar,
  primary key (ll_file))
alter index LOAD_LIST on DB.DBA.LOAD_LIST partition (ll_file varchar)
create index LL_STATE on DB.DBA.LOAD_LIST (ll_state, ll_file, ll_graph) partition (ll_state int)
;
]]></programlisting>
         <para>This table may be checked at any time during bulk load for the progress of the load. 
         	ll_state is 1 for files being loaded and 2 for files whose loading has finished. 
         	ll_error is NULL if the load finished without error, else it is the error message.</para>
         <para>In order to load data from the files in load_list, run as dba:</para>
<programlisting><![CDATA[
DB.DBA.rdf_loader_run (); 	
]]></programlisting>   
         <para>One may run several of these commands on parallel sessions for better throughput.</para>
         <para>On a cluster one can do:</para>
<programlisting><![CDATA[
cl_exec ('rdf_ld_srv ()'); 
]]></programlisting>
         <para>This will start one <link linkend="fn_rdf_loader_run">rdf_loader_run()</link> on each node of the cluster.
         Note that in such a
         	setting all the server processes must see the same files at the same path.</para>
         <para>On an isql session one may execute rdf_loader_run () &amp; several times, forking a new 
         	isql for each such command, similarly to what a Unix shell does.</para> 
         <para>Because this load is non-transactional and non-logged, one must do an explicit checkpoint 
         	after the load to guarantee a persistent state.</para>
         <para>On a single server do:</para>
<programlisting><![CDATA[
checkpoint; 
]]></programlisting>  
  <para>On a cluster do:</para>
<programlisting><![CDATA[  
cl_exec ('checkpoint');
]]></programlisting>
         <para>The server(s) are online and can process queries and transactions while a bulk load 
         	is in progress. Periodic checkpoints may occur during the load but the state is guaranteed 
         	to be consistent only after running a checkpoint after all the bulk load threads 
         	have finished.</para>
         <para>A bulk load should not be forcibly stopped. To make a controlled stop, run:</para>
<programlisting><![CDATA[
rdf_load_stop ();
]]></programlisting>   
         <para>This will cause the files being loaded at the time to finish load but no new loads 
         	will start until explicitly started with <link linkend="fn_rdf_loader_run">rdf_loader_run()</link>.</para>
         <para>Specially note that on a cluster the database will be inconsistent if one server 
         	process does a checkpoint and another does not. Thus guaranteeing a checkpoint on all 
         	is necessary. This is easily done with an isql script with the following content:</para>
<programlisting><![CDATA[
ld_dir ('/data8/2848260', '%.gz', 'http://bsbm.org');

-- Record CPU time 
select getrusage ()[0] + getrusage ()[1];

rdf_loader_run () &
rdf_loader_run () &
rdf_loader_run () &
rdf_loader_run () &
rdf_loader_run () &
rdf_loader_run () &
rdf_loader_run () &
rdf_loader_run () &

wait_for_children;
checkpoint;

-- Record CPU time
select getrusage ()[0] + getrusage ()[1];
]]></programlisting>
         <para>For a cluster, the equivalent is:</para>
<programlisting><![CDATA[
ld_dir ('/data8/2848260', '%.gz', 'http://bsbm.org');

cl_exec ('DB.DBA.RDF_LD_SRV (2)');

cl_exec ('checkpoint');
]]></programlisting>
         <para><link linkend="fn_rdf_loader_run">rdf_loader_run()</link> recognizes several file types, including .ttl, .nt, .xml, .rdf,
         	.owl, .nq, .n4, and others. Internally the function uses 
         	<link linkend="fn_ttlp"><function>DB.DBA.ttlp()</function></link> or 
         	<link linkend="fn_rdf_load_rdfxml"><function>DB.DBA.rdf_load_rdfxml</function></link>, 
         	as appropriate.</para>
         	<para>See <link linkend="rdfperfloadinglod">the next section</link> for detailed description 
         	of the <link linkend="fn_rdf_loader_run">rdf_loader_run()</link> function.</para>
       </sect3>
       <sect3 id="rdfperfloadinglod"><title>Loading LOD RDF data</title>
<para>To load the rdf data to LOD instance, perform the following steps:
</para>
<itemizedlist mark="bullet">
  <listitem>Configure &amp; start cluster</listitem>
  <listitem>Execute the file:
<programlisting><![CDATA[
--
--  $Id: rdfandsparql.xml,v 1.400 2012/11/01 10:38:13 rtsekova Exp $
--
--  Alternate RDF index scheme for cases where G unspecified
--
--  This file is part of the OpenLink Software Virtuoso Open-Source (VOS)
--  project.
--
--  Copyright (C) 1998-2018 OpenLink Software
--
--  This project is free software; you can redistribute it and/or modify it
--  under the terms of the GNU General Public License as published by the
--  Free Software Foundation; only version 2 of the License, dated June 1991.
--
--  This program is distributed in the hope that it will be useful, but
--  WITHOUT ANY WARRANTY; without even the implied warranty of
--  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
--  General Public License for more details.
--
--  You should have received a copy of the GNU General Public License along
--  with this program; if not, write to the Free Software Foundation, Inc.,
--  51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
--
--

drop index RDF_QUAD_OGPS;
checkpoint;
create table R2 (G iri_id_8, S iri_id_8, P iri_id_8, O any, primary key (S, P, O, G))
alter index R2 on R2 partition (S int (0hexffff00));

log_enable (2);
insert into R2 (G, S, P, O) SELECT G, S, P, O from rdf_quad;

drop table RDF_QUAD;
alter table r2 rename RDF_QUAD;
checkpoint;
create bitmap index RDF_QUAD_OPGS on RDF_QUAD (O, P, G, S) partition (O varchar (-1, 0hexffff));
create bitmap index RDF_QUAD_POGS on RDF_QUAD (P, O, G, S) partition (O varchar (-1, 0hexffff));
create bitmap index RDF_QUAD_GPOS on RDF_QUAD (G, P, O, S) partition (O varchar (-1, 0hexffff));

checkpoint;
]]></programlisting>
</listitem>
  <listitem>Execute:
<programlisting><![CDATA[
SQL>cl_exec ('checkpoint);
]]></programlisting>
</listitem>
  <listitem>Execute ld_dir ('directory' , 'mask' , 'graph'), for ex:
<programlisting><![CDATA[
SQL>ld_dir ('/dbs/data', '*.gz', 'http://dbpedia.org');
]]></programlisting>
</listitem>
  <listitem>Execute on every node with separate client:
<programlisting><![CDATA[
SQL>rdf_loader_run();
]]></programlisting>
</listitem>
</itemizedlist>
</sect3>


       <sect3 id="rdfperfloadingunitpro"><title>Loading UniProt RDF data</title>
<para>To load the uniprot data, create a function for example such as:</para>
<programlisting><![CDATA[
create function DB.DBA.UNIPROT_LOAD (in log_mode integer := 1)
{
  DB.DBA.RDF_LOAD_RDFXML_MT (file_to_string_output('filename1'),'http://base_uri_1', 'destination_graph_1', log_mode, 3);
  DB.DBA.RDF_LOAD_RDFXML_MT (file_to_string_output('filename2'),'http://base_uri_2', 'destination_graph_2', log_mode, 3);
  ...
  DB.DBA.RDF_LOAD_RDFXML_MT (file_to_string_output('filename9'),'http://base_uri_9', 'destination_graph_9', log_mode, 3);
}
]]></programlisting>
<para>If you are starting from blank database and you can drop it and re-create in case of error signaled, use it this way:</para>
<programlisting><![CDATA[
checkpoint;
checkpoint_interval(6000);
DB.DBA.UNIPROT_LOAD (0),
checkpoint;
checkpoint_interval(60);
]]></programlisting>
<para>If the database contains important data already and there's no way to stop it and backup before the load then use:</para>
<programlisting><![CDATA[
checkpoint;
checkpoint_interval(6000);
DB.DBA.UNIPROT_LOAD (),
checkpoint;
checkpoint_interval(60);
]]></programlisting>
<para>Note that the 'number of threads' parameter of DB.DBA.RDF_LOAD_RDFXML() mentions threads
used to process data from file, an extra thread will read the text and parse it,
so for 4 CPU cores there's no need in parameter value greater than 3. Three processing
threads per one parsing tread is usually good ratio because parsing is usually three
times faster than the rest of loading so CPU loading is well balanced.
If for example you are using 2 x Quad Xeon, then you can choose between 8
single-threaded parsers or 2 parsers with 3 processing threads each. With 4 cores you may simply load
file after file with 3 processing threads. The most important performance tuning is to set the
[Parameters] section of virtuoso configuration file:</para>
<programlisting><![CDATA[
NumberOfBuffers = 1000000
MaxDirtyBuffers = 800000
MaxCheckpointRemap = 1000000
DefaultIsolation = 2
]]></programlisting>
<para>Note: these numbers are reasonable for 16 GB RAM Linux box. Usually when there are no such massive operations as loading huge database, you can set up the values as:</para>
<programlisting><![CDATA[
NumberOfBuffers = 1500000
MaxDirtyBuffers = 1200000
MaxCheckpointRemap = 1500000
DefaultIsolation = 2
]]></programlisting>
<tip>
	<title>See Also:</title>
    <para><link linkend=""></link></para>
    <para><ulink url="http://www.openlinksw.com/dataspace/dav/wiki/Main/VirtConfigScale#Configuration%20Options">Virtuoso Configuration Options</ulink></para>    	
</tip>
<tip>
	<title>Tip:</title>
<para>Thus after loading all data you may wish to shutdown, tweak and start server again.
If you have ext2fs or ext3fs filesystem, then it's better to have enough free space on disk not to
make it more than 80% full. When it's almost full it may allocate database file badly, resulting
in measurable loss of disk access speed. That is not Virtuoso-specific fact, but a common hint
for all database-like applications with random access to big files.</para>
</tip>
<para>Here is an example of using awk file for splitting big file smaller ones:</para>
<programlisting><![CDATA[
BEGIN {
	file_part=1000
	e_line = "</rdf:RDF>"
        cur=0
        cur_o=0
	file=0
	part=file_part
      }
	{
	    res_file_i="res/"FILENAME
	    line=$0
	    s=$1
	    res_file=res_file_i"_"file".rdf"

	    if (index (s, "</rdf:Description>") == 1)
	    {
		cur=cur+1
		part=part-1
	    }

	    if (part > 0)
	    {
	    	print line >> res_file
	    }

	    if (part == 0)
	    {
#		print "===================== " cur
	    	print line >> res_file
		print e_line >> res_file
		close (res_file)
		file=file+1
		part=file_part
	    	res_file=res_file_i"_"file".rdf"
		system ("cp beg.txt " res_file)
	    }
        }
END { }
]]></programlisting>
       </sect3>
       <sect3 id="rdfperfloadingdbpedia"><title>Loading DBPedia RDF data</title>
<para>You can use the following script as an example for loading DBPedia RDF data in Virtuoso:</para>
<programlisting><![CDATA[
#!/bin/sh

PORT=$1
USER=$2
PASS=$3
file=$4
g=$5
LOGF=`basename $0`.log

if [ -z "$PORT" -o -z "$USER" -o -z "$PASS" -o -z "$file" -o -z "$g" ]
then
  echo "Usage: `basename $0` [DSN] [user] [password] [ttl-file] [graph-iri]"
  exit
fi

if [ ! -f "$file" -a ! -d "$file" ]
then
    echo "$file does not exist"
    exit 1
fi

mkdir READY 2>/dev/null
rm -f $LOGF $LOGF.*

echo "Starting..."
echo "Logging into: $LOGF"

DOSQL ()
{
    isql $PORT $USER $PASS verbose=on banner=off prompt=off echo=ON errors=stdout exec="$1" > $LOGF
}

LOAD_FILE ()
{
    f=$1
    g=$2
    echo "Loading $f (`cat $f | wc -l` lines) `date \"+%H:%M:%S\"`" | tee -a $LOG

    DOSQL "ttlp_mt (file_to_string_output ('$f'), '', '$g', 17); checkpoint;" > $LOGF

    if [ $? != 0 ]
    then
	echo "An error occurred, please check $LOGF"
	exit 1
    fi

    line_no=`grep Error $LOGF | awk '{ match ($0, /line [0-9]+/, x) ; match (x[0], /[0-9]+/, y); print y[0] }'`
    newf=$f.part
    inx=1
    while [ ! -z "$line_no" ]
    do
	cat $f |  awk "BEGIN { i = 1 } { if (i==$line_no) { print \$0; exit; } i = i + 1 }"  >> bad.nt
	line_no=`expr $line_no + 1`
	echo "Retrying from line $line_no"
	echo "@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> ." > tmp.nt
	cat $f |  awk "BEGIN { i = 1 } { if (i>=$line_no) print \$0; i = i + 1 }"  >> tmp.nt
	mv tmp.nt $newf
	f=$newf
	mv $LOGF $LOGF.$inx
	DOSQL "ttlp_mt (file_to_string_output ('$f'), '', '$g', 17); checkpoint;" > $LOGF

	if [ $? != 0 ]
	then
	    echo "An error occurred, please check $LOGF"
	    exit 1
	fi
	line_no=`grep Error $LOGF | awk '{ match ($0, /line [0-9]+/, x) ; match (x[0], /[0-9]+/, y); print y[0] }'`
	inx=`expr $inx + 1`
    done
    rm -f $newf 2>/dev/null
    echo "Loaded.  "
}

echo "======================================="
echo "Loading started."
echo "======================================="

if [ -f "$file" ]
then
    LOAD_FILE $file $g
    mv $file READY 2>> /dev/null
elif [ -d "$file" ]
then
    for ff in `find $file -name '*.nt'`
    do
	LOAD_FILE $ff $g
	mv $ff READY 2>> /dev/null
    done
else
   echo "The input is not file or directory"
fi
echo "======================================="
echo "Final checkpoint."
DOSQL "checkpoint;" > temp.res
echo "======================================="
echo "Check bad.nt file for skipped triples."
echo "======================================="

exit 0

]]></programlisting>
       </sect3>
       <sect3 id="rdfperfloadingbio2rdf"><title>Loading Bio2RDF data</title>
<para>The shell script below was used to import files in n3 notation into OpenLink Virtuoso RDF storage.</para>
<para>When an syntax error it will cut content from next line and will retry. This was used on ubuntu linux to import bio2rdf and freebase dumps.</para>
<para>Note it uses gawk, so it must be available on system where is tried. Also for recovery additional disk space is needed at max the size of original file.</para>
<programlisting><![CDATA[
#!/bin/bash

PASS=$1
f=$2
g=$3

# Usage
if [ -z "$PASS" -o -z "$f" -o -z "$g" ]
then
  echo "Usage: $0 [password] [ttl-file] [graph-iri]"
  exit
fi

if [ ! -f "$f" ]
then
    echo "$f does not exist"
    exit
fi

# Your port here
PORT=1111  #`inifile -f dbpedia.ini -s Parameters -k ServerPort`
if test -z "$PORT"
then
    echo "Cannot find INI and inifile command"
    exit
fi

# Initial run
isql $PORT dba $PASS verbose=on banner=off prompt=off echo=ON errors=stdout exec="ttlp_mt (file_to_string_output ('$f'), '', '$g'); checkpoint;" > $0.log

# If disconnect etc.
if [ $? != 0 ]
then
    echo "An error occurred, please check $0.log"
    exit
fi

# Check for error
line_no=`grep Error $0.log | awk '{ match ($0, /line [0-9]+/, x) ; match (x[0], /[0-9]+/, y); print y[0] }'`
newf=$f.part
inx=1

# Error recovery
while [ ! -z "$line_no" ]
do
    cat $f |  awk "BEGIN { i = 0 } { if (i==$line_no) { print \$0; exit; } i = i + 1 }"  >> bad.nt
    line_no=`expr $line_no + 1`
    echo "Retrying from line $line_no"
    cat $f |  awk "BEGIN { i = 0 } { if (i>=$line_no) print \$0; i = i + 1 }"  > tmp.nt
    mv tmp.nt $newf
    f=$newf
    mv $0.log $0.log.$inx
    # Run the recovered part
    isql $PORT dba $PASS verbose=on banner=off prompt=off echo=ON errors=stdout exec="ttlp_mt (file_to_string_output ('$f'), '', '$g'); checkpoint;" > $0.log

    if [ $? != 0 ]
    then
	echo "An error occurred, please check $0.log"
	exit
    fi
   line_no=`grep Error $0.log | awk '{ match ($0, /line [0-9]+/, x) ; match (x[0], /[0-9]+/, y); print y[0] }'`
   inx=`expr $inx + 1`
done
]]></programlisting>

       </sect3>
    </sect2>
    <sect2 id="rdfperfsparul"><title>Using SPARUL</title>
	<para>Since SPARUL updates are generally not meant to be transactional, it is
	    best to run these in <link linkend="fn_log_enable"><function>log_enable (2)</function></link> mode,
	    which commits every operation as it is done. This prevents one from running out of rollback space. Also for bulk updates, transaction logging can be turned off. If so, one should do a manual checkpoint after the operation to ensure persistence across server restart since there is no roll forward log.</para>
<para>To have a roll forward log and row by row autocommit, one may use <link linkend="fn_log_enable"><function>log_enable (3)</function></link>. This will write constantly into the log which takes extra time. Having no logging and doing a checkpoint when the whole work is finished is faster.</para>
<para>Many SPARUL operations can be run in parallel in this way. If they are independent with respect to their input and output, they can run in parallel and row by row autocommit will ensure they do not end up waiting for each others' locks.</para>
    </sect2>
    <sect2 id="rdfperfgeneraldbpedia"><title>DBpedia Benchmark</title>
<para>We ran the DBpedia benchmark queries again with different configurations of Virtuoso.
Comparing numbers given by different parties is a constant problem. In the case reported here,
we loaded the full DBpedia 3, all languages, with about 198M triples, onto Virtuoso v5 and Virtuoso Cluster
v6, all on the same 4 core 2GHz Xeon with 8G RAM. All databases were striped on 6 disks. The Cluster
configuration was with 4 processes in the same box.
We ran the queries in two variants:
</para>
<itemizedlist>
  <listitem>With graph specified in the SPARQL FROM clause, using the default indices.</listitem>
  <listitem>With no graph specified anywhere, using an alternate indexing scheme.</listitem>
</itemizedlist>
<para>The times below are for the sequence of 5 queries.
As there is a query in the set that specifies no condition on S or O and only P,
thus cannot be done with the default indices With Virtuoso v5. With Virtuoso Cluster v6 it can,
because v6 is more space efficient. So we added the index:</para>
<programlisting><![CDATA[
create bitmap index rdf_quad_pogs on rdf_quad (p, o, g, s);
]]></programlisting>
<table>
<tgroup cols="4">
<thead>
<row>
  <entry></entry>
  <entry>Virtuoso v5 with  gspo, ogps, pogs</entry>
  <entry>Virtuoso Cluster v6 with gspo, ogps</entry>
  <entry>Virtuoso Cluster v6 with gspo, ogps, pogs</entry>
</row>
</thead>
<tbody>
<row><entry>cold</entry><entry>210 s</entry><entry>136 s</entry><entry>33.4 s</entry></row>
<row><entry>warm</entry><entry>0.600 s</entry><entry>4.01 s</entry><entry>0.628 s</entry></row>
</tbody>
</tgroup>
</table>
<para>Now let us do it without a graph being specified. Note that alter index is valid for v6 or higher.
For all platforms, we drop any existing indices, and:</para>
<programlisting><![CDATA[
create table r2 (g iri_id_8, s, iri_id_8, p iri_id_8, o any, primary key (s, p, o, g))
alter index R2 on R2 partition (s int (0hexffff00));

log_enable (2);
insert into r2 (g, s, p, o) SELECT g, s, p, o from rdf_quad;

drop table rdf_quad;
alter table r2 rename RDF_QUAD;
create bitmap index rdf_quad_opgs on rdf_quad (o, p, g, s) partition (o varchar (-1, 0hexffff));
create bitmap index rdf_quad_pogs on rdf_quad (p, o, g, s) partition (o varchar (-1, 0hexffff));
create bitmap index rdf_quad_gpos on rdf_quad (g, p, o, s) partition (o varchar (-1, 0hexffff));
]]></programlisting>
<para>The code is identical for v5 and v6, except that with v5 we use iri_id (32 bit) for
the type, not iri_id_8 (64 bit). We note that we run out of IDs with v5 around a few billion
triples, so with v6 we have double the ID length and still manage to be vastly more space efficient.</para>
<para>With the above 4 indices, we can query the data pretty much in any combination without hitting
a full scan of any index. We note that all indices that do not begin with s end with s as a bitmap.
This takes about 60% of the space of a non-bitmap index for data such as DBpedia.</para>
<para>If you intend to do completely arbitrary RDF queries in Virtuoso, then chances are
you are best off with the above index scheme.</para>
<table>
<tgroup cols="3">
<thead>
<row>
  <entry></entry>
  <entry>Virtuoso v5 with  gspo, ogps, pogs</entry>
  <entry>Virtuoso Cluster v6 with gspo, ogps, pogs</entry>
</row>
</thead>
<tbody>
<row><entry>warm</entry><entry>0.595 s</entry><entry>0.617 s</entry></row>
</tbody>
</tgroup>
</table>
<para>The cold times were about the same as above, so not reproduced.</para>
<para>It is in the SPARQL spirit to specify a graph and for pretty much any application,
there are entirely sensible ways of keeping the data in graphs and specifying which ones are
concerned by queries. This is why Virtuoso is set up for this by default.</para>
<para>On the other hand, for the open web scenario, dealing with an unknown large number of graphs,
enumerating graphs is not possible and questions like which graph of which source asserts x become
relevant. We have two distinct use cases which warrant different setups of the database, simple as that.</para>
<para>The latter use case is not really within the SPARQL spec, so implementations may or may not
support this.</para>
<para>Once the indices are right, there is no difference between specifying a graph and not specifying a
graph with the queries considered. With more complex queries, specifying a graph or set of graphs does
allow some optimizations that cannot be done with no graph specified. For example, bitmap intersections
are possible only when all leading key parts are given.</para>
<para>The best warm cache time is with v5; the five queries run under 600 ms after the first go.
This is noted to show that all-in-memory with a single thread of execution is hard to beat.</para>
<para>Cluster v6 performs the same queries in 623 ms. What is gained in parallelism is lost in latency
if all operations complete in microseconds. On the other hand, Cluster v6 leaves v5 in the dust in any
situation that has less than 100% hit rate. This is due to actual benefit from parallelism if operations
take longer than a few microseconds, such as in the case of disk reads. Cluster v6 has substantially
better data layout on disk, as well as fewer pages to load for the same content.</para>
<para>This makes it possible to run the queries without the pogs index on Cluster v6 even when v5 takes prohibitively long.</para>
<para>The purpose is to have a lot of RAM and space-efficient data representation.</para>
<para>For reference, the query texts specifying the graph are below. To run without specifying
the graph, just drop the FROM &lt;http://dbpedia.org&gt; from each query. The returned row counts are
indicated below each query's text.</para>
<programlisting><![CDATA[
SQL>SPARQL
SELECT ?p ?o
FROM <http://dbpedia.org>
WHERE
  {
    <http://dbpedia.org/resource/Metropolitan_Museum_of_Art> ?p ?o .
  };

p                                                                                 o
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://www.w3.org/1999/02/22-rdf-syntax-ns#type                                   http://umbel.org/umbel/ac/Artifact
http://www.w3.org/1999/02/22-rdf-syntax-ns#type                                   http://dbpedia.org/class/yago/MuseumsInNewYorkCity
http://www.w3.org/1999/02/22-rdf-syntax-ns#type                                   http://dbpedia.org/class/yago/ArtMuseumsAndGalleriesInTheUnitedStates
http://www.w3.org/1999/02/22-rdf-syntax-ns#type                                   http://dbpedia.org/class/yago/Museum103800563
..
-- 335 rows

SQL>SPARQL
PREFIX p: <http://dbpedia.org/property/>
SELECT ?film1 ?actor1 ?film2 ?actor2
FROM <http://dbpedia.org>
WHERE
  {
    ?film1 p:starring <http://dbpedia.org/resource/Kevin_Bacon> .
    ?film1 p:starring ?actor1 .
    ?film2 p:starring ?actor1 .
    ?film2 p:starring ?actor2 .
};

film1                                       actor1                                    film2                                        ctor2
VARCHAR                                     VARCHAR                                   VARCHAR                                      ARCHAR
http://dbpedia.org/resource/The_River_Wild  http://dbpedia.org/resource/Kevin_Bacon   http://dbpedia.org/resource/The_River_Wild   http://dbpedia.org/resource/Kevin_Bacon
http://dbpedia.org/resource/The_River_Wild  http://dbpedia.org/resource/Kevin_Bacon   http://dbpedia.org/resource/The_River_Wild   http://dbpedia.org/resource/Meryl_Streep
http://dbpedia.org/resource/The_River_Wild  http://dbpedia.org/resource/Kevin_Bacon   http://dbpedia.org/resource/The_River_Wild   http://dbpedia.org/resource/Joseph_Mazzello
http://dbpedia.org/resource/The_River_Wild  http://dbpedia.org/resource/Kevin_Bacon   http://dbpedia.org/resource/The_River_Wild   http://dbpedia.org/resource/David_Strathairn
http://dbpedia.org/resource/The_River_Wild  http://dbpedia.org/resource/Kevin_Bacon   http://dbpedia.org/resource/The_River_Wild   http://dbpedia.org/resource/John_C._Reilly
...
--  23910 rows

SQL>SPARQL
PREFIX p: <http://dbpedia.org/property/>
SELECT ?artist ?artwork ?museum ?director
FROM <http://dbpedia.org>
WHERE
  {
    ?artwork p:artist ?artist .
    ?artwork p:museum ?museum .
    ?museum p:director ?director
  };

artist                                          artwork                                              museum                                                                            director
VARCHAR                                         VARCHAR                                              VARCHAR                                                                           VARCHAR
_______________________________________________

http://dbpedia.org/resource/Paul_C%C3%A9zanne   http://dbpedia.org/resource/The_Basket_of_Apples     http://dbpedia.org/resource/Art_Institute_of_Chicago                              James Cuno
http://dbpedia.org/resource/Paul_Signac         http://dbpedia.org/resource/Neo-impressionism        http://dbpedia.org/resource/Art_Institute_of_Chicago                              James Cuno
http://dbpedia.org/resource/Georges_Seurat      http://dbpedia.org/resource/Neo-impressionism        http://dbpedia.org/resource/Art_Institute_of_Chicago                              James Cuno
http://dbpedia.org/resource/Edward_Hopper       http://dbpedia.org/resource/Nighthawks               http://dbpedia.org/resource/Art_Institute_of_Chicago                              James Cuno
http://dbpedia.org/resource/Mary_Cassatt        http://dbpedia.org/resource/The_Child%27s_Bath       http://dbpedia.org/resource/Art_Institute_of_Chicago                              James Cuno
..
-- 303 rows

SQL>SPARQL
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?s ?homepage
FROM <http://dbpedia.org>
WHERE
  {
    <http://dbpedia.org/resource/Berlin> geo:lat ?berlinLat .
    <http://dbpedia.org/resource/Berlin> geo:long ?berlinLong .
    ?s geo:lat ?lat .
    ?s geo:long ?long .
    ?s foaf:homepage ?homepage .
    FILTER (
      ?lat        <=     ?berlinLat + 0.03190235436 &&
      ?long       >=     ?berlinLong - 0.08679199218 &&
      ?lat        >=     ?berlinLat - 0.03190235436 &&
      ?long       <=     ?berlinLong + 0.08679199218) };

s                                                                                 homepage
VARCHAR                                                                           VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/Berlin_University_of_the_Arts                         http://www.udk-berlin.de/
http://dbpedia.org/resource/Berlin_University_of_the_Arts                         http://www.udk-berlin.de/
http://dbpedia.org/resource/Berlin_Zoological_Garden                              http://www.zoo-berlin.de/en.html
http://dbpedia.org/resource/Federal_Ministry_of_the_Interior_%28Germany%29        http://www.bmi.bund.de
http://dbpedia.org/resource/Neues_Schauspielhaus                                  http://www.goya-berlin.com/
http://dbpedia.org/resource/Bauhaus_Archive                                       http://www.bauhaus.de/english/index.htm
http://dbpedia.org/resource/Canisius-Kolleg_Berlin                                http://www.canisius-kolleg.de
http://dbpedia.org/resource/Franz%C3%B6sisches_Gymnasium_Berlin                   http://www.fg-berlin.cidsnet.de
..
-- 48 rows

SQL>SPARQL
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX p: <http://dbpedia.org/property/>
SELECT ?s ?a ?homepage
FROM <http://dbpedia.org>
WHERE
  {
    <http://dbpedia.org/resource/New_York_City> geo:lat ?nyLat .
    <http://dbpedia.org/resource/New_York_City> geo:long ?nyLong .
    ?s geo:lat ?lat .
    ?s geo:long ?long .
    ?s p:architect ?a .
    ?a foaf:homepage ?homepage .
    FILTER (
      ?lat        <=     ?nyLat + 0.3190235436 &&
      ?long       >=     ?nyLong - 0.8679199218 &&
      ?lat        >=     ?nyLat - 0.3190235436 &&
      ?long       <=     ?nyLong + 0.8679199218) };
s                                                                                 a               homepage
VARCHAR                                                                           VARCHAR              VARCHAR
_______________________________________________________________________________

http://dbpedia.org/resource/GE_Building                                           http://dbpedia.org/resource/Associated_Architects              http://www.associated-architects.co.uk
http://dbpedia.org/resource/Giants_Stadium                                        http://dbpedia.org/resource/HNTB              http://www.hntb.com/
http://dbpedia.org/resource/Fort_Tryon_Park_and_the_Cloisters                     http://dbpedia.org/resource/Frederick_Law_Olmsted              http://www.asla.org/land/061305/olmsted.html
http://dbpedia.org/resource/Central_Park                                          http://dbpedia.org/resource/Frederick_Law_Olmsted              http://www.asla.org/land/061305/olmsted.html
http://dbpedia.org/resource/Prospect_Park_%28Brooklyn%29                          http://dbpedia.org/resource/Frederick_Law_Olmsted              http://www.asla.org/land/061305/olmsted.html
http://dbpedia.org/resource/Meadowlands_Stadium                                   http://dbpedia.org/resource/360_Architecture              http://oakland.athletics.mlb.com/oak/ballpark/new/faq.jsp
http://dbpedia.org/resource/Citi_Field                                            http://dbpedia.org/resource/HOK_Sport_Venue_Event              http://www.hoksve.com/
http://dbpedia.org/resource/Citigroup_Center                                      http://dbpedia.org/resource/Hugh_Stubbins_Jr.              http://www.klingstubbins.com
http://dbpedia.org/resource/150_Greenwich_Street                                  http://dbpedia.org/resource/Fumihiko_Maki              http://www.pritzkerprize.com/maki2.htm
http://dbpedia.org/resource/Freedom_Tower                                         http://dbpedia.org/resource/David_Childs              http://www.som.com/content.cfm/www_david_m_childs
http://dbpedia.org/resource/7_World_Trade_Center                                  http://dbpedia.org/resource/David_Childs              http://www.som.com/content.cfm/www_david_m_childs
http://dbpedia.org/resource/The_New_York_Times_Building                           http://dbpedia.org/resource/Renzo_Piano              http://www.rpbw.com/
http://dbpedia.org/resource/Trump_World_Tower                                     http://dbpedia.org/resource/Costas_Kondylis              http://www.kondylis.com

13 Rows. -- 2183 msec.
]]></programlisting>
    </sect2>
<sect2 id="rdfstorebenchmarks"><title>RDF Store Benchmarks</title>
    <sect3 id="rdfstorebenchmarksintroduction"><title>Introduction</title>
       <para>In a particular RDF Store Benchmarks there is difference if the queries are
executed with specified graph or with specified multiple graphs. As Virtuoso is quad store,
not triple store with many tables, it runs queries inefficiently if graphs are specified
and there are no additional indexes except pre-set GSPO and OGPS. Proper use of the FROM clause
or adding indexes with graph column will contribute for better results.
       </para>
    </sect3>
    <sect3 id="rdfstorebenchmarksindexusage"><title>Using bitmap indexes</title>
    <para>If is known in advance for the current RDF Store Benchmarks that some
users will not indicate specific graphs then should be done: </para>
    <itemizedlist>
       <listitem>either create indexes with graph in last position</listitem>
       <listitem>or load everything into single graph and specify it somewhere in querying application.</listitem>
    </itemizedlist>
    <para>Both methods do not require any changes in query texts</para>
<itemizedlist mark="bullet">
  <listitem>For users using Virtuoso 5 is strongly recommended is the usage of additional bitmap indexes:
<programlisting><![CDATA[
SQL> create bitmap index RDF_QUAD_POGS on DB.DBA.RDF_QUAD (P,O,G,S);
SQL> create bitmap index RDF_QUAD_PSOG on DB.DBA.RDF_QUAD (P,S,O,G);
]]></programlisting>
</listitem>
  <listitem>For users using Virtuoso 6 or higher, see the new layout <link linkend="rdfperfrdfscheme">here</link>.</listitem>
</itemizedlist>
    <para>You can create other indexes as well. Bitmap indexes are preferable, but
if O is the last column, then the index can not be bitmap, so it could be, for e.g.:</para>
<programlisting><![CDATA[
create index RDF_QUAD_PSGO on DB.DBA.RDF_QUAD (P, S, G, O);
]]></programlisting>
    <para>but cannot be:</para>
<programlisting><![CDATA[
create bitmap index RDF_QUAD_PSGO on DB.DBA.RDF_QUAD (P, S, G, O);
]]></programlisting>
    </sect3>
  </sect2>
  <sect2 id="fastapproxdiffandpatch"><title>Fast Approximate RDF Graph Diff and Patch</title>
<para>Two algorithms described below resemble "unified diff" and "patch by unified diff" 
but they work on RDF graphs, not on plain texts.
</para>
<para>They work reasonably for graphs composed from CBDs (concise bounded descriptions) of 
some subjects, if these subjects are either "named" IRIs or can be identified by values of 
their inverse functional properties.
</para>
<para>Many sorts of commonly used graphs match these restrictions, including all graphs 
without blank nodes, most of FOAF files, graphs that can be "pretty-printed" in JSON, most 
of dumps of relational databases etc.
</para>
<para>The basic idea is as simple as zipper: 
</para>
<itemizedlist mark="bullet">
  <listitem>Place one graph at the left and one to the right,</listitem>
  <listitem>Find a retainer box at the right and a matching pin at the left,</listitem>
  <listitem>Join them</listitem>
  <listitem>Pull the slider as long as possible.</listitem>
  <listitem>Repeat this while there are pins and boxes that can be matched and sliders that can be moved.</listitem>
</itemizedlist>
<para>An IRI in left graph <code>(say, G1)</code> matches to same IRI in right graph <code>(G2)</code> 
as pin to box. The same is true for literals too.
</para>
<para>Functional and inverse functional properties are teeth that form chains, algorithm "moves sliders" along these chains, incrementally connecting more and more nodes.
</para>
<para>If there is a match of this sort <code>(O1 in G1 matches O2 in G2)</code> and the matched nodes 
are values of same inverse functional property <code>P</code> (there are <code>{ S1 P O1 }</code> in 
<code>G1</code> and <code>{ S2 P O2 }</code>  in <code>G2</code>) then we guess that <code>S1</code> 
matches <code>S2</code>.
</para>
<para>If <code>S1</code> in <code>G1</code> matches <code>S2</code> in <code>G2</code> and the matched 
nodes are subjects of same functional property <code>P</code> ( there are <code>{ S1 P N1 }</code> in 
<code>G1</code> and <code>{ S2 P N2 }</code> in <code>G2</code> ) then we guess that <code>N1</code> 
matches <code>N2</code>, now it's possible to try same interaction on triples where <code>N1</code> and 
<code>N2</code> are in subject position, that's how slides move. A typical example of a long zipper 
is closed list with matched heads.
</para>
<sect3 id="fastapproxdiffandpatchhow"><title>Make a Diff And Use It</title>
<itemizedlist mark="bullet">
  <listitem>Using <link linkend="fn_rdf_graph_diff">DB.DBA.RDF_GRAPH_DIFF</link></listitem>
  <listitem>Using <link linkend="fn_rdf_suo_diff_ttl">DB.DBA.RDF_SUO_DIFF_TTL</link></listitem>
  <listitem>Using <link linkend="fn_rdf_suo_apply_patch">DB.DBA.RDF_SUO_APPLY_PATCH</link></listitem>  
</itemizedlist>
</sect3>
<sect3 id="fastapproxdiffandpatchclt"><title>Collect Functional And Inverse Functional Properties</title>
<para>Lists of functional properties can be retrieved from an ontology graph by query like:</para>
<programlisting><![CDATA[
SPARQL define output:valmode "LONG"
SELECT (<LONG::sql:VECTOR_AGG(?s)) 
FROM <my-ontology-graph>
WHERE 
       {
    ?s a owl:functionalProperty 
       }
]]></programlisting>
<para>Inverse functional properties could be retrieved by a similar query, but unfortunately the 
	ontology may mention so called NULL values that can be property values for many subjects. Current 
	implementation of diff and patch does not recognize NULL values so they can cause patch with 
	"false alarm" errors. The workaround is to retrieve only properties that have no NULL values declared:
</para>
<programlisting><![CDATA[
SPARQL define output:valmode "LONG"
SELECT (<LONG::sql:VECTOR_AGG(?s)) 
FROM <my-ontology-graph>
WHERE
  {
    ?s a owl:inverseFunctionalProperty .
    OPTIONAL { ?s owl:nullIFPValue ?v }
    FILTER (!Bound(?v)) 
  }
]]></programlisting>
<para>If no ontology is available then appropriate predicates can be obtained from sample graphs using 
	<link linkend="fn_rdf_graph_collect_fp_list">DB.DBA.RDF_GRAPH_COLLECT_FP_LIST</link>.
</para>
</sect3>
<sect3 id="fastapproxdiffandpatchimpl"><title>Implementation-Specific Extensions of GUO Ontology</title>
<para><emphasis>Note</emphasis>: This section contains implementation details that are needed only 
if you want to write your own patch or diff procedure, you don't have to worry about internals if 
you want to use existing procedures.
</para>
<para>Basic GUO ontology is not expressive enough to work with blank nodes, so some custom extensions $
are needed.
</para>
<para>In the rest of the description:</para>
<programlisting><![CDATA[
@prefix guo: <http://webr3.org/owl/guo#>
]]></programlisting>
<para>is assumed.</para>
<para>The diff contains one node of <code>rdf:type guo:diff</code>.
</para>
<para>For debugging purpose it has properties <code>guo:graph1</code> and <code>guo:graph2</code> that 
corespond to <code>gfrom</code> and <code>gto</code> arguments of <link linkend="fn_rdf_suo_diff_ttl">DB.DBA.RDF_SUO_DIFF_TTL</link>.
</para>
<para>The diff also contains zero or more nodes of <code>rdf:type guo:UpdateInstruction</code>. These 
nodes are as described in basic GUO ontology, but <code>guo:target_graph</code> is now optional, 
<code>guo:target_subject</code> can be a blank node and objects of predicates "inside" values of 
<code>guo:insert</code> and <code>guo:delete</code> can also be blank nodes. These blank nodes are 
"placeholders" for values, calculated according to the most important GUO extension - rule nodes.
</para>
<para>There are eight sorts of rule nodes, four for <code>gfrom</code> side of diff and four similar for 
<code>gto</code> side. Out of four sorts related to one side, two are for functional properties and 
two similar are for inverse functional properties. Thus <code>rdf:type-s</code> of these nodes are:
</para>
<programlisting><![CDATA[
guo:from-rule-FP0, 
guo:from-rule-FP1, 
guo:from-rule-IFP0, 
guo:from-rule-IFP1
]]></programlisting>
<para>and
</para>
<programlisting><![CDATA[
guo:to-rule-FP0, 
guo:to-rule-FP1, 
guo:to-rule-IFP , 
guo:to-rule-IFP1 .
]]></programlisting>
<para>Each rule node has property <code>guo:order</code> that is an non-negative integer.
</para>
<para>These integers enumerate all <code>guo:from-rule-</code>... nodes, starting from zero.
</para>
<para>When patch procedure works, these rules are used in this order, the result of each rule 
	is a blank node that either exists in the graph or just created.
</para>
<para>All results are remembered for use in the rest of the patch procedure.
</para>
<para>Similarly, other sequence of these integers enumerate all <code>guo:to-rule-</code>... nodes, 
also starting from zero.
</para>
<para>Consider a sequence of <code>guo:from-rule-</code>... nodes, because <code>guo:to-rule-</code> 
nodes have identical properties.
</para>
<para>A rule node can have zero or more values of <code>guo:dep</code> property, each value is a 
bnode that is rule node that should be calculated before the current one.
</para>
<para>Every rule has exactly one predicate <code>guo:path</code> that is a blank node. Each property 
of this blank node describes one possible "move of slider": predicate to follow is in predicate 
position and a node to start from is in object position. An IRI or a literal in object position is 
used as is, a blank node in object position should be of type <code>guo:from-rule-</code>... and 
have smaller <code>guo:order</code> so it refers to already calculated result bnode of some 
preceding rule.
</para>
<para>Rule of form:
</para>
<programlisting><![CDATA[
R a guo:from-rule-IFP1 ;
  guo:path [ P1 O1 ; P2 O2 ; ... ; Pn On ] .
]]></programlisting>
<para>searches for a unique blank node <code>_:Rres</code> that is a common subject of triples:
</para>
<programlisting><![CDATA[
 _:Rres P1 O1
 _:Rres P2 O2
  . . .
 _:Rres Pn On
]]></programlisting>
<para>in the gfrom graph.
</para>
<para>If subjects differ in these triples or some triples are not found or the subject is not a 
	blank node then an appropriate error is logged and rule fails, otherwise <code>_:Rres</code> 
	is remembered as the result of the rule.
</para>
<para>Similarly, rule of form:
</para>
<programlisting><![CDATA[
R a guo:from-rule-FP1 ;
  guo:path [ P1 O1 ; P2 O2 ; ... ; Pn On ] .
]]></programlisting>
<para>searches for a unique blank node <code>_:Rres</code> that is a common object of triples:
</para>
<programlisting><![CDATA[
 O1 P1 _:Rres
 O2 P2 _:Rres
  . . .
 On Pn _:Rres
]]></programlisting>
<para>in the gfrom graph.
</para>
<para>Rule of form:
</para>
<programlisting><![CDATA[
R a guo:from-rule-IFP0 ;
  guo:path [ P1 O1 ; P2 O2 ; ... ; Pn On ] .
]]></programlisting>
<para>ensures that the <code>gfrom</code> graph does not contain any triple like:
</para>
<programlisting><![CDATA[
 _:Rres P1 O1
 _:Rres P2 O2
]]></programlisting>
<para>or
</para>
<programlisting><![CDATA[
_:Rres Pn On
]]></programlisting>
<para>It is an error if something exists. If nothing found then the result of the rule is 
	newly created unique blank node. That's how patch procedure creates new blank nodes when 
	it inserts "totally new" data.
</para>
<para>Similarly, rule of form:
</para>
<programlisting><![CDATA[
R a guo:from-rule-IFP0 ;
  guo:path [ P1 O1 ; P2 O2 ; ... ; Pn On ] .
]]></programlisting>
<para>ensures that the <code>gfrom</code> graph does not contain any triple like:
</para>
<programlisting><![CDATA[
O1 P1 _:Rres
O2 P2 _:Rres
]]></programlisting>
<para>or
</para>
<programlisting><![CDATA[
 On Pn _:Rres
]]></programlisting>
<para>Current version of patch procedure does not use rules <code>guo:to-rule-</code>...  , 
however they can be used by custom procedure of few sorts. First, these rules can be used to 
produce a "reversed diff". Next, these rules can be used to validate the result of the patch - 
if the patch can not be reverted then the result is "suspicious". 
</para>
</sect3>
</sect2>
  <sect2 id="rdb2rdftriggers"><title>RDB2RDF Triggers</title>
<para>Linked Data Views have many advantages, if compared to static dumps of the database in RDF triples. 
	However, they does not solve few problems. First, inference is supported only for physically stored 
	triples, so one had to chose between convenience of inference and convenience of Linked Data Views. Next, 
	algorithms that selects triples with non-constant graphs and predicates tend to produce enormous 
	texts of SQL queries if Linked Data Views are complicated enough. Finally, there may be a need in export 
	of big and fresh static RDF dump but preparing this dump would take too much time via both RDF 
	Views and traditional methods.
</para>
<para>The solution is set of triggers on source tables of an Linked Data View that edit parts of physical 
	dump on each change of source data. Unlike Linked Data Views that cost nothing while not queried, these 
	triggers add a significant overhead on any data manipulation on sources, continuously. To 
	compensate this, the dump should be in an intensive use and not replaceable by Linked Data Views. In 
	other cases, do not add these triggers.
</para>
<para>It is next to impossible to write such triggers by hands so a small API is provided to 
	generate SQL texts from metadata of Linked Data Views.
</para>
<para>First of all, views in an RDF storage does not work in full isolation from each other. 
	Some of them may partially disable others due to OPTION(EXCLUSIVE) and some may produce one 
	triple in different ways. As a result, triggers are not made on per-view basis. Instead, a 
	special RDF storage is introduced, namely virtrdf:SyncToQuads , all required triples are 
	added to it and triggers are created for the whole storage. Typically an Linked Data View is created 
	in some other storage, e.g., virtrdf:DefaultQuadStorage and then added to virtrdf:SyncToQuads via:
</para>
<programlisting><![CDATA[
sparql alter quad storage virtrdf:SyncToQuads {
   create <my_rdf_view> using storage virtrdf:DefaultQuadStorage };
]]></programlisting>
<para>The following example procedure copies all user-defined Linked Data Views from default quad storage 
	to virtrdf:SyncToQuads:
</para>
<programlisting><![CDATA[
create procedure DB.DBA.RDB2RDF_COPY_ALL_RDF_VIEWS_TO_SYNC ()
       {
  for (sparql define input:storage ""
    select (bif:aref(bif:sprintf_inverse (str(?idx), bif:concat (str(rdf:_), "%d"), 0), 0)) ?qm
    from virtrdf:
    where { virtrdf:DefaultQuadStorage-UserMaps ?idx ?qm . ?qm a virtrdf:QuadMap }
    order by asc (bif:sprintf_inverse (bif:concat (str(rdf:_), "%d"), str (?idx), 1)) ) do
    exec (sprintf ('sparql alter quad storage virtrdf:SyncToQuads { create <%s> using storage virtrdf:DefaultQuadStorage }', "qm"));
}
;
]]></programlisting>
<para>When the virtrdf:SyncToQuads storage is fully prepared, two API functions can be used:
</para>
<itemizedlist mark="bullet">
  <listitem><link linkend="fn_sparql_rdb2rdf_list_tables">DB.DBA.SPARQL_RDB2RDF_LIST_TABLES</link>: 
  The function returns a vector of names of tables that are used as sources for Linked Data Views. Application 
  developer should decide what to do with each of them - create triggers or do some application-specific 
  workarounds.
  <para>Note that if some SQL views are used as sources for Linked Data Views and these views does not have 
  	INSTEAD triggers then workarounds become mandatory for them, not just a choice, because BEFORE 
  	or AFTER triggers on views are not allowed if there is no appropriate INSTEAD trigger. The mode 
  	argument should be zero in current version.
  </para>
  </listitem>
  <listitem><link linkend="fn_sparql_rdb2rdf_codegen">DB.DBA.SPARQL_RDB2RDF_CODEGEN</link>: The 
  function creates an SQL text for a given table and an operation specified by an opcode.
  </listitem>
</itemizedlist>
<para>In some cases, Linked Data Views are complicated enough so that BEFORE UPDATE and AFTER DELETE 
	triggers are required in additional to the minimal set. In this case, sparql_rdb2rdf_codegen 
	calls will return a vector of two string sessions, not single string session, and both sessions 
	are sql texts to inspect or execute. In this case, the BEFORE trigger will not delete obsolete 
	quads from RDF_QUAD table, instead it will create records in a special table RDF_QUAD_DELETE_QUEUE 
	as guesses what can be deleted. The AFTER trigger will re-check these guesses, delete related quads 
	if needed and shorten the RDF_QUAD_DELETE_QUEUE.
</para>
<para>The extra activity of triggers on RDF_QUAD, RDF_OBJ, RDF_QUAD_DELETE_QUEUE and other tables 
	and indexes of the storage of "physical" triples may cause deadlocks so the application should 
	be carefully checked for proper support of deadlocks if they were very seldom before turning 
	RDB2RDF triggers on. In some cases, the whole processing of RDB2RDF can be moved to a separate 
	server and connected to the main workhorse server via replication.
</para>
<para>The following example functions create texts of all triggers, save them to files in for 
	further studying and try to load them. That's probably quite bad scenario for a production 
	database, because it's better to read procedures before loading them, especially if they're 
	triggers, especially if some of them may contain errors.
</para>
<programlisting><![CDATA[
-- This creates one or two files with one or two triggers or other texts and try to load the 
generated sql texts.
create procedure DB.DBA.RDB2RDF_EXEC_CODEGEN1_FOR_TABLE 
(  in dump_prefix varchar, 
   in tbl varchar, 
   in dump_id any, 
   in txt any )
   {
  declare fname varchar;
  declare stat, msg varchar;
  if (isinteger (dump_id))
    dump_id := cast (dump_id as varchar);
  if (__tag of vector = __tag (txt))
    {
      DB.DBA.RDB2RDF_EXEC_CODEGEN1_FOR_TABLE (dump_prefix, tbl, dump_id, txt[0]);
      DB.DBA.RDB2RDF_EXEC_CODEGEN1_FOR_TABLE (dump_prefix, tbl, dump_id || 'p' , txt[1]);
      return;
    }
  if (__tag of varchar <> __tag (txt))
    txt := string_output_string (txt);
  fname := sprintf ('%s_Rdb2Rdf.%s.%s.sql', dump_prefix, tbl, dump_id);
  string_to_file (fname, txt || '\n;\n', -2);
  if ('0' = dump_id)
    return;
  stat := '00000';
  msg := '';
  exec (txt, stat, msg);
  if ('00000' <> stat)
    {
      string_to_file (fname, '\n\n- - - - - 8< - - - - -\n\nError ' || stat || ' ' || msg, -1);
      if (not (subseq (msg, 0, 5) in ('SQ091')))
        signal (stat, msg);
    }
}
;

-- This creates and loads all triggers, init procedure and debug dump related to one table.
create procedure DB.DBA.RDB2RDF_PREPARE_TABLE (in dump_prefix varchar, in tbl varchar)
   {
  declare ctr integer;
  for (ctr := 0; ctr <= 4; ctr := ctr+1 )
    DB.DBA.RDB2RDF_EXEC_CODEGEN1_FOR_TABLE (dump_prefix, tbl, ctr, sparql_rdb2rdf_codegen (tbl, ctr));
   }
;

-- This creates and loads all triggers, init procedure and debug dump related to all tables used by and Linked Data View.
create procedure DB.DBA.RDB2RDF_PREPARE_ALL_TABLES (in dump_prefix varchar)
   {
  declare tbl_list any;
  tbl_list := sparql_rdb2rdf_list_tables (0);
  foreach (varchar tbl in tbl_list) do
    {
      DB.DBA.RDB2RDF_PREPARE_TABLE (dump_prefix, tbl);
   }
}
;
]]></programlisting>
<para>The following combination of calls prepares all triggers for all Linked Data Views of the default storage:
</para>
<programlisting><![CDATA[
DB.DBA.RDB2RDF_COPY_ALL_RDF_VIEWS_TO_SYNC ();
DB.DBA.RDB2RDF_PREPARE_ALL_TABLES (cast (now() as varchar));
]]></programlisting>
<para>This does not copy the initial state of RDB2RDF graphs to the physical storage, because this can 
	be dangerous for existing RDF data and even if all procedures will work as expected then they may 
	produce huge amounts of RDF data, run out of transaction log limits and thus require 
	application-specific precautions. It is also possible to make initial loading by a SPARUL statements 
	like:
</para>
<programlisting><![CDATA[
SPARQL 
INSERT IN <snapshot-graph> { ?s ?p ?o } 
FROM <snapshot-htaph> 
WHERE 
 { quad map <id-of-rdf-view> 
 	  { ?s ?p ?o } 
 };
]]></programlisting>
</sect2>	
</sect1>



<sect1 id="rdfnativestorageproviders"><title>RDF Data Access Providers (Drivers)</title>
  <sect2 id="rdfnativestorageprovidersjena"><title>Virtuoso Jena Provider</title>
    <sect3 id="rdfnativestorageprovidersjenawhatis"><title>What is Jena</title>
    <para>Jena is an open source Semantic Web framework for Java. It provides an API to
extract data from and write to RDF graphs. The graphs are represented as an abstract "model".
A model can be sourced with data from files, databases, URIs or a combination of these. A Model
can also be queried through SPARQL and updated through SPARUL.
    </para>
    </sect3>
    <sect3 id="rdfnativestorageprovidersjenawhatisv"><title>What is the Virtuoso Jena Provider</title>
    <para>The Virtuoso Jena RDF Data Provider is a fully operational Native Graph Model Storage Provider for the
    Jena Framework, which enables Semantic Web applications written using the Jena RDF Frameworks to directly
    query the Virtuoso RDF Quad Store. Providers are available for the latest
    <ulink url="http://jena.sourceforge.net/">Jena</ulink> 2.6.x and 2.10.x versions.
    </para>
    <figure id="rdfnativestorageprovidersjena1" float="1">
      <title>Virtuoso Jena RDF Data Provider</title>
      <graphic fileref="ui/VirtJenaProvider.png"/>
    </figure>
    </sect3>
    <sect3 id="rdfnativestorageprovidersjenasetup"><title>Setup</title>
    <sect4 id="rdfnativestorageprovidersjenareqfiles"><title>Prerequisites</title>
      <para><ulink url="http://edit-wiki.usnet.private/dataspace/dav/wiki/VOS/VOSDownload#Jena%20Provider">Download the latest</ulink> Virtuoso Jena Provider for your Jena framework version, Virtuoso JDBC 3 Driver, Jena Framework, and associated classes and sample programs.</para>
      <itemizedlist mark="bullet">
        <listitem><emphasis>Note:</emphasis> The Jena Provider is explicitly bound to the Virtuoso JDBC 3 Driver. You cannot use the Virtuoso JDBC 4 Driver for this purpose at this time.</listitem>
        <listitem>The version of the Jena Provider (<code>virt_jena.jar</code>) can be verified with the command:
<programlisting><![CDATA[
$ java -jar virt_jena.jar
OpenLink Virtuoso(TM) Provider for Jena(TM) Version 2.6.2 [Build 1.2]
]]></programlisting>
        </listitem>
        <listitem>Files contained in the zip files are generally older than specifically linked downloads (e.g., the
        Virtuoso JDBC Driver, virtjdbc3.jar), so don't replace if prompted during extraction. Instead, rename the
        file extracted from the zip, and compare their versions to be sure you keep only the most recent.
<programlisting><![CDATA[
$ java -cp virtjdbc3.jar virtuoso.jdbc3.Driver
OpenLink Virtuoso(TM) Driver for JDBC(TM) Version 3.x [Build 3.57]
$ java -cp virtjdbc3.fromzip.jar virtuoso.jdbc3.Driver
OpenLink Virtuoso(TM) Driver for JDBC(TM) Version 3.x [Build 3.11]
]]></programlisting>
        </listitem>
        <listitem>Downloads:
    <itemizedlist mark="bullet">
      <listitem>Virtuoso Jena Provider JAR file, <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload/virt_jena.jar">virt_jena.jar</ulink></listitem>
      <listitem>Virtuoso JDBC Driver 3 JAR file, <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload/virtjdbc3.jar">virtjdbc3.jar</ulink></listitem>
      <listitem>Virtuoso JDBC Driver 4 JAR file, <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload/virtjdbc4.jar">virtjdbc4.jar</ulink></listitem>      
      <listitem>Jena Framework and associated classes, <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload/jenajars.zip">jenajars.zip</ulink></listitem>
      <listitem>Sample programs, <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload/virtjenasamples.zip">virtjenasamples.zip</ulink></listitem>
    </itemizedlist>
        </listitem>
      </itemizedlist>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenacmsmpr"><title>Compiling Jena Sample Programs</title>
    <orderedlist>
      <listitem>Edit the sample programs VirtuosoSPARQLExampleX.java, where X = 1 to 9.
Set the JDBC connection strings within to point to a valid Virtuoso Server instance of the form:
<programlisting><![CDATA[
"jdbc:virtuoso://localhost:1111/charset=UTF-8/log_enable=2"
]]></programlisting>
      <itemizedlist>
         <listitem>charset=UTF-8  will be added by Jena provider, if it isn't in connection string.
So now you don't need add "charset=UTF-8" to the connection string any more, it is done by Jena provider.

         </listitem>
         <listitem>log_enable=2: to use row auto commit
         </listitem>
         <listitem>use these settings to process large rdf data.
         </listitem>
      </itemizedlist>
      </listitem>
      <listitem>Ensure that full paths to <emphasis>jena.jar, arq.jar,</emphasis> and
<emphasis>virtjdbc3.jar</emphasis> are included in the active CLASSPATH setting.
      </listitem>
      <listitem>Compile the Jena Sample applications using the following command:
<programlisting><![CDATA[
javac -cp "jena.jar:arq.jar:virtjdbc3.jar:virt_jena.jar:." VirtuosoSPARQLExample1.java
VirtuosoSPARQLExample2.java VirtuosoSPARQLExample3.java VirtuosoSPARQLExample4.java
VirtuosoSPARQLExample5.java VirtuosoSPARQLExample6.java VirtuosoSPARQLExample7.java
VirtuosoSPARQLExample8.java VirtuosoSPARQLExample9.java
]]></programlisting>
      </listitem>
    </orderedlist>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenatesting"><title>Testing</title>
    <para>Once the Provider classes and sample program have been successfully compiled,
the Provider can be tested using the sample programs included. Ensure your active CLASSPATH
includes full paths to all of the following files, before executing the example commands:
    </para>
    <itemizedlist mark="bullet">
      <listitem>icu4j_3_4.jar</listitem>
      <listitem>iri.jar</listitem>
      <listitem>xercesImpl.jar</listitem>
      <listitem>axis.jar</listitem>
      <listitem>commons-logging-1.1.1.jar</listitem>
      <listitem>jena.jar</listitem>
      <listitem>arq.jar</listitem>
      <listitem>virtjdbc3.jar</listitem>
      <listitem>virt_jena.jar</listitem>
    </itemizedlist>
    <orderedlist>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples1">VirtuosoSPARQLExample1</link> returns the contents of the RDF Quad store of the targeted Virtuoso instance, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample1
]]></programlisting>
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples2">VirtuosoSPARQLExample2</link> reads in the contents of the following FOAF URIs --
<programlisting><![CDATA[
http://kidehen.idehen.net/dataspace/person/kidehen#this
http://www.w3.org/People/Berners-Lee/card#i
http://demo.openlinksw.com/dataspace/person/demo#this
]]></programlisting>
      <para>-- and returns the RDF data stored, with the following command:</para>
<programlisting><![CDATA[
java VirtuosoSPARQLExample2
]]></programlisting>
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples3">VirtuosoSPARQLExample3</link> performs simple addition and deletion operation on
the content of the triple store, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample3
]]></programlisting>
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples4">VirtuosoSPARQLExample4</link> demonstrates the use of the <emphasis>graph.contains</emphasis> method for searching triples, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample4
]]></programlisting>
</listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples5">VirtuosoSPARQLExample5</link> demonstrates the use of the <emphasis>graph.find</emphasis> method for searching triples, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample5
]]></programlisting>
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples6">VirtuosoSPARQLExample6</link> demonstrates the use of the <emphasis>graph.getTransactionHandler</emphasis> method, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample6
]]></programlisting>
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples7">VirtuosoSPARQLExample7</link> demonstrates the use of the graph.getBulkUpdateHandler method, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample7
]]></programlisting>   
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples8">VirtuosoSPARQLExample8</link> demonstrates how to insert triples into a graph, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample8
]]></programlisting>
      </listitem>
      <listitem><link linkend="rdfnativestorageprovidersjenaexamples9">VirtuosoSPARQLExample9</link> demonstrates the use of the <emphasis>CONSTRUCT, DESCRIBE,</emphasis> and <emphasis>ASK</emphasis> SPARQL query forms, with the following command:
<programlisting><![CDATA[
java VirtuosoSPARQLExample9
]]></programlisting>  
     </listitem>
    </orderedlist>
    </sect4>
    </sect3>
    <sect3 id="rdfnativestorageprovidersjenaexamples"><title>Examples</title>
    <sect4 id="rdfnativestorageprovidersjenaexamples1"><title>VirtJenaSPARQLExample1</title>
<programlisting><![CDATA[  
import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.rdf.model.RDFNode;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample1 {

	/**
	 * Executes a SPARQL query against a virtuoso url and prints results.
	 */
	public static void main(String[] args) {

		String url;
		if(args.length == 0)
		    url = "jdbc:virtuoso://localhost:1111";
		else
		    url = args[0];

/*			STEP 1			*/
		VirtGraph set = new VirtGraph (url, "dba", "dba");

/*			STEP 2			*/


/*			STEP 3			*/
/*		Select all data in virtuoso	*/
		Query sparql = QueryFactory.create("SELECT * WHERE { GRAPH ?graph { ?s ?p ?o } } limit 100");

/*			STEP 4			*/
		VirtuosoQueryExecution vqe = VirtuosoQueryExecutionFactory.create (sparql, set);

		ResultSet results = vqe.execSelect();
		while (results.hasNext()) {
			QuerySolution result = results.nextSolution();
		    RDFNode graph = result.get("graph");
		    RDFNode s = result.get("s");
		    RDFNode p = result.get("p");
		    RDFNode o = result.get("o");
		    System.out.println(graph + " { " + s + " " + p + " " + o + " . }");
		}
	}
}

]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples2"><title>VirtJenaSPARQLExample2</title>
<programlisting><![CDATA[
import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.rdf.model.RDFNode;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample2 {

	/**
	 * Executes a SPARQL query against a virtuoso url and prints results.
	 */
	public static void main(String[] args) {

		String url;
		if(args.length == 0)
		    url = "jdbc:virtuoso://localhost:1111";
		else
		    url = args[0];

/*			STEP 1			*/
		VirtGraph graph = new VirtGraph ("Example2", url, "dba", "dba");

/*			STEP 2			*/
/*		Load data to Virtuoso		*/
		graph.clear ();

		System.out.print ("Begin read from 'http://www.w3.org/People/Berners-Lee/card#i'  ");
		graph.read("http://www.w3.org/People/Berners-Lee/card#i", "RDF/XML");
		System.out.println ("\t\t\t Done.");

		System.out.print ("Begin read from 'http://demo.openlinksw.com/dataspace/person/demo#this'  ");
		graph.read("http://demo.openlinksw.com/dataspace/person/demo#this", "RDF/XML");
		System.out.println ("\t Done.");

		System.out.print ("Begin read from 'http://kidehen.idehen.net/dataspace/person/kidehen#this'  ");
		graph.read("http://kidehen.idehen.net/dataspace/person/kidehen#this", "RDF/XML");
		System.out.println ("\t Done.");


/*			STEP 3			*/
/*		Select only from VirtGraph	*/
		Query sparql = QueryFactory.create("SELECT ?s ?p ?o WHERE { ?s ?p ?o }");

/*			STEP 4			*/
		VirtuosoQueryExecution vqe = VirtuosoQueryExecutionFactory.create (sparql, graph);

		ResultSet results = vqe.execSelect();
		while (results.hasNext()) {
			QuerySolution result = results.nextSolution();
		    RDFNode graph_name = result.get("graph");
		    RDFNode s = result.get("s");
		    RDFNode p = result.get("p");
		    RDFNode o = result.get("o");
		    System.out.println(graph_name + " { " + s + " " + p + " " + o + " . }");
		}

		System.out.println("graph.getCount() = " + graph.getCount());
	}
}
]]></programlisting>   
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples3"><title>VirtJenaSPARQLExample3</title>
<programlisting><![CDATA[
import java.util.*;

import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.rdf.model.RDFNode;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample3
{
    public static void main(String[] args)
    {
	String url;

	if(args.length == 0)
	    url = "jdbc:virtuoso://localhost:1111";
	else
	    url = args[0];

	Node foo1 = Node.createURI("http://example.org/#foo1");
	Node bar1 = Node.createURI("http://example.org/#bar1");
	Node baz1 = Node.createURI("http://example.org/#baz1");

	Node foo2 = Node.createURI("http://example.org/#foo2");
	Node bar2 = Node.createURI("http://example.org/#bar2");
	Node baz2 = Node.createURI("http://example.org/#baz2");

	Node foo3 = Node.createURI("http://example.org/#foo3");
	Node bar3 = Node.createURI("http://example.org/#bar3");
	Node baz3 = Node.createURI("http://example.org/#baz3");

	List <Triple> triples = new ArrayList <Triple> ();

	VirtGraph graph = new VirtGraph ("Example3", url, "dba", "dba");

	graph.clear ();

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("Add 3 triples to graph <Example3>.");

	graph.add(new Triple(foo1, bar1, baz1));
	graph.add(new Triple(foo2, bar2, baz2));
	graph.add(new Triple(foo3, bar3, baz3));

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	triples.add(new Triple(foo1, bar1, baz1));
	triples.add(new Triple(foo2, bar2, baz2));

	graph.isEmpty();

	System.out.println("Remove 2 triples from graph <Example3>");
	graph.remove(triples);
	System.out.println("graph.getCount() = " + graph.getCount());
	System.out.println("Please check result with isql tool.");

	/* EXPECTED RESULT:

SQL> SPARQL
SELECT ?s ?p ?o
FROM <Example3>
WHERE {?s ?p ?o};
s                                                    p                                                             o
VARCHAR                                    VARCHAR                                              VARCHAR
_______________________________________________________________________________

http://example.org/#foo3              http://example.org/#bar3                         http://example.org/#baz3

1 Rows. -- 26 msec.
SQL>

*/

	}
}
]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples4"><title>VirtJenaSPARQLExample4</title>
<programlisting><![CDATA[
import java.util.*;

import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.rdf.model.RDFNode;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample4
{

    public static void main(String[] args)
    {
	String url;
	if(args.length == 0)
	    url = "jdbc:virtuoso://localhost:1111";
	else
	    url = args[0];

	Node foo1 = Node.createURI("http://example.org/#foo1");
	Node bar1 = Node.createURI("http://example.org/#bar1");
	Node baz1 = Node.createURI("http://example.org/#baz1");

	Node foo2 = Node.createURI("http://example.org/#foo2");
	Node bar2 = Node.createURI("http://example.org/#bar2");
	Node baz2 = Node.createURI("http://example.org/#baz2");

	Node foo3 = Node.createURI("http://example.org/#foo3");
	Node bar3 = Node.createURI("http://example.org/#bar3");
	Node baz3 = Node.createURI("http://example.org/#baz3");

	VirtGraph graph = new VirtGraph ("Example4", url, "dba", "dba");

	graph.clear ();

	System.out.println("graph.isEmpty() = " + graph.isEmpty());

	System.out.println("Add 3 triples to graph <Example4>.");

	graph.add(new Triple(foo1, bar1, baz1));
	graph.add(new Triple(foo2, bar2, baz2));
	graph.add(new Triple(foo3, bar3, baz3));

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	System.out.println ("graph.contains(new Triple(foo2, bar2, baz2) - " + graph.contains(new Triple(foo2, bar2, baz2)));
	System.out.println ("graph.contains(new Triple(foo2, bar2, baz3) - " + graph.contains(new Triple(foo2, bar2, baz3)));

	graph.clear ();

    }
}
]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples5"><title>VirtJenaSPARQLExample5</title>
<programlisting><![CDATA[
import java.util.*;

import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.util.iterator.ExtendedIterator;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample5
	{

    public static void main(String[] args)
    {
	String url;
	if(args.length == 0)
	    url = "jdbc:virtuoso://localhost:1111";
	else
	    url = args[0];

	Node foo1 = Node.createURI("http://example.org/#foo1");
	Node bar1 = Node.createURI("http://example.org/#bar1");
	Node baz1 = Node.createURI("http://example.org/#baz1");

	Node foo2 = Node.createURI("http://example.org/#foo2");
	Node bar2 = Node.createURI("http://example.org/#bar2");
	Node baz2 = Node.createURI("http://example.org/#baz2");

	Node foo3 = Node.createURI("http://example.org/#foo3");
	Node bar3 = Node.createURI("http://example.org/#bar3");
	Node baz3 = Node.createURI("http://example.org/#baz3");

	VirtGraph graph = new VirtGraph ("Example5", url, "dba", "dba");

	graph.clear ();

	System.out.println("graph.isEmpty() = " + graph.isEmpty());

	System.out.println("Add 3 triples to graph <Example5>.");

	graph.add(new Triple(foo1, bar1, baz1));
	graph.add(new Triple(foo2, bar2, baz2));
	graph.add(new Triple(foo3, bar3, baz3));
	graph.add(new Triple(foo1, bar2, baz2));
	graph.add(new Triple(foo1, bar3, baz3));

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	ExtendedIterator iter = graph.find(foo1, Node.ANY, Node.ANY);
	System.out.println ("\ngraph.find(foo1, Node.ANY, Node.ANY) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());

	iter = graph.find(Node.ANY, Node.ANY, baz3);
	System.out.println ("\ngraph.find(Node.ANY, Node.ANY, baz3) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());

	iter = graph.find(foo1, Node.ANY, baz3);
	System.out.println ("\ngraph.find(foo1, Node.ANY, baz3) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());

	graph.clear ();

	    }
}
]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples6"><title>VirtJenaSPARQLExample6</title>
<programlisting><![CDATA[
import java.util.*;

import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.util.iterator.ExtendedIterator;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample6
	    {

    public static void main(String[] args)
	    {
	String url;
	if(args.length == 0)
	    url = "jdbc:virtuoso://localhost:1111";
	else
	    url = args[0];

	Node foo1 = Node.createURI("http://example.org/#foo1");
	Node bar1 = Node.createURI("http://example.org/#bar1");
	Node baz1 = Node.createURI("http://example.org/#baz1");

	Node foo2 = Node.createURI("http://example.org/#foo2");
	Node bar2 = Node.createURI("http://example.org/#bar2");
	Node baz2 = Node.createURI("http://example.org/#baz2");

	Node foo3 = Node.createURI("http://example.org/#foo3");
	Node bar3 = Node.createURI("http://example.org/#bar3");
	Node baz3 = Node.createURI("http://example.org/#baz3");

	VirtGraph graph = new VirtGraph ("Example6", url, "dba", "dba");

	graph.clear ();

	System.out.println("graph.isEmpty() = " + graph.isEmpty());

	System.out.println("test Transaction Commit.");
	graph.getTransactionHandler().begin();
	System.out.println("begin Transaction.");
	System.out.println("Add 3 triples to graph <Example6>.");

	graph.add(new Triple(foo1, bar1, baz1));
	graph.add(new Triple(foo2, bar2, baz2));
	graph.add(new Triple(foo3, bar3, baz3));

	graph.getTransactionHandler().commit();
	System.out.println("commit Transaction.");
	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	ExtendedIterator iter = graph.find(Node.ANY, Node.ANY, Node.ANY);
	System.out.println ("\ngraph.find(Node.ANY, Node.ANY, Node.ANY) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());

	graph.clear ();
	System.out.println("\nCLEAR graph <Example6>");
	System.out.println("graph.isEmpty() = " + graph.isEmpty());

	System.out.println("Add 1 triples to graph <Example6>.");
	graph.add(new Triple(foo1, bar1, baz1));

	System.out.println("test Transaction Abort.");
	graph.getTransactionHandler().begin();
	System.out.println("begin Transaction.");
	System.out.println("Add 2 triples to graph <Example6>.");

	graph.add(new Triple(foo2, bar2, baz2));
	graph.add(new Triple(foo3, bar3, baz3));

	graph.getTransactionHandler().abort();
	System.out.println("abort Transaction.");
	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	iter = graph.find(Node.ANY, Node.ANY, Node.ANY);
	System.out.println ("\ngraph.find(Node.ANY, Node.ANY, Node.ANY) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());

	graph.clear ();
	System.out.println("\nCLEAR graph <Example6>");

	    }
        }


]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples7"><title>VirtJenaSPARQLExample7</title>
<programlisting><![CDATA[
import java.util.*;

import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.util.iterator.ExtendedIterator;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample7
{

    public static void main(String[] args)
    {
	String url;
	if(args.length == 0)
	    url = "jdbc:virtuoso://localhost:1111";
	else
	    url = args[0];

	Node foo1 = Node.createURI("http://example.org/#foo1");
	Node bar1 = Node.createURI("http://example.org/#bar1");
	Node baz1 = Node.createURI("http://example.org/#baz1");

	Node foo2 = Node.createURI("http://example.org/#foo2");
	Node bar2 = Node.createURI("http://example.org/#bar2");
	Node baz2 = Node.createURI("http://example.org/#baz2");

	Node foo3 = Node.createURI("http://example.org/#foo3");
	Node bar3 = Node.createURI("http://example.org/#bar3");
	Node baz3 = Node.createURI("http://example.org/#baz3");

	List triples1 = new ArrayList();
	triples1.add(new Triple(foo1, bar1, baz1));
	triples1.add(new Triple(foo2, bar2, baz2));
	triples1.add(new Triple(foo3, bar3, baz3));

	List triples2 = new ArrayList();
	triples2.add(new Triple(foo1, bar1, baz1));
	triples2.add(new Triple(foo2, bar2, baz2));

	VirtGraph graph = new VirtGraph ("Example7", url, "dba", "dba");

	graph.clear ();

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("Add List with 3 triples to graph <Example7> via BulkUpdateHandler.");

	graph.getBulkUpdateHandler().add(triples1);

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	ExtendedIterator iter = graph.find(Node.ANY, Node.ANY, Node.ANY);
	System.out.println ("\ngraph.find(Node.ANY, Node.ANY, Node.ANY) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());


	System.out.println("\n\nDelete List of 2 triples from graph <Example7> via BulkUpdateHandler.");

	graph.getBulkUpdateHandler().delete(triples2);

	System.out.println("graph.isEmpty() = " + graph.isEmpty());
	System.out.println("graph.getCount() = " + graph.getCount());

	iter = graph.find(Node.ANY, Node.ANY, Node.ANY);
	System.out.println ("\ngraph.find(Node.ANY, Node.ANY, Node.ANY) \nResult:");
	for ( ; iter.hasNext() ; )
	    System.out.println ((Triple) iter.next());

	graph.clear ();
	System.out.println("\nCLEAR graph <Example7>");

    }
}
]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples8"><title>VirtJenaSPARQLExample8</title>
<programlisting><![CDATA[
import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.rdf.model.RDFNode;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample8 {

	/**
	 * Executes a SPARQL query against a virtuoso url and prints results.
	 */
	public static void main(String[] args) {

		String url;
		if(args.length == 0)
		    url = "jdbc:virtuoso://localhost:1111";
		else
		    url = args[0];

/*			STEP 1			*/
		VirtGraph set = new VirtGraph (url, "dba", "dba");

/*			STEP 2			*/
System.out.println("\nexecute: CLEAR GRAPH <http://test1>");
                String str = "CLEAR GRAPH <http://test1>";
                VirtuosoUpdateRequest vur = VirtuosoUpdateFactory.create(str, set);
                vur.exec();

System.out.println("\nexecute: INSERT INTO GRAPH <http://test1> { <aa> <bb> 'cc' . <aa1> <bb1> 123. }");
                str = "INSERT INTO GRAPH <http://test1> { <aa> <bb> 'cc' . <aa1> <bb1> 123. }";
                vur = VirtuosoUpdateFactory.create(str, set);
                vur.exec();

/*			STEP 3			*/
/*		Select all data in virtuoso	*/
System.out.println("\nexecute: SELECT * FROM <http://test1> WHERE { ?s ?p ?o }");
		Query sparql = QueryFactory.create("SELECT * FROM <http://test1> WHERE { ?s ?p ?o }");

/*			STEP 4			*/
		VirtuosoQueryExecution vqe = VirtuosoQueryExecutionFactory.create (sparql, set);

		ResultSet results = vqe.execSelect();
		while (results.hasNext()) {
			QuerySolution rs = results.nextSolution();
		    RDFNode s = rs.get("s");
		    RDFNode p = rs.get("p");
		    RDFNode o = rs.get("o");
		    System.out.println(" { " + s + " " + p + " " + o + " . }");
		}


System.out.println("\nexecute: DELETE FROM GRAPH <http://test1> { <aa> <bb> 'cc' }");
                str = "DELETE FROM GRAPH <http://test1> { <aa> <bb> 'cc' }";
                vur = VirtuosoUpdateFactory.create(str, set);
                vur.exec();

System.out.println("\nexecute: SELECT * FROM <http://test1> WHERE { ?s ?p ?o }");
		vqe = VirtuosoQueryExecutionFactory.create (sparql, set);
                results = vqe.execSelect();
		while (results.hasNext()) {
			QuerySolution rs = results.nextSolution();
		    RDFNode s = rs.get("s");
		    RDFNode p = rs.get("p");
		    RDFNode o = rs.get("o");
		    System.out.println(" { " + s + " " + p + " " + o + " . }");
		}


	}
}
]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageprovidersjenaexamples9"><title>VirtJenaSPARQLExample9</title>
<programlisting><![CDATA[
import com.hp.hpl.jena.query.*;
import com.hp.hpl.jena.rdf.model.RDFNode;
import com.hp.hpl.jena.graph.Triple;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Graph;
import com.hp.hpl.jena.rdf.model.*;
import java.util.Iterator;

import virtuoso.jena.driver.*;

public class VirtuosoSPARQLExample9 {

	/**
	 * Executes a SPARQL query against a virtuoso url and prints results.
	 */
	public static void main(String[] args) {

		String url;
		if(args.length == 0)
		    url = "jdbc:virtuoso://localhost:1111";
		else
		    url = args[0];

/*			STEP 1			*/
		VirtGraph set = new VirtGraph (url, "dba", "dba");

/*			STEP 2			*/
                String str = "CLEAR GRAPH <http://test1>";
                VirtuosoUpdateRequest vur = VirtuosoUpdateFactory.create(str, set);
                vur.exec();

                str = "INSERT INTO GRAPH <http://test1> { <http://aa> <http://bb> 'cc' . <http://aa1> <http://bb> 123. }";
                vur = VirtuosoUpdateFactory.create(str, set);
                vur.exec();


/*		Select all data in virtuoso	*/
		Query sparql = QueryFactory.create("SELECT * FROM <http://test1> WHERE { ?s ?p ?o }");
		VirtuosoQueryExecution vqe = VirtuosoQueryExecutionFactory.create (sparql, set);
		ResultSet results = vqe.execSelect();
                System.out.println("\nSELECT results:");
		while (results.hasNext()) {
			QuerySolution rs = results.nextSolution();
		    RDFNode s = rs.get("s");
		    RDFNode p = rs.get("p");
		    RDFNode o = rs.get("o");
		    System.out.println(" { " + s + " " + p + " " + o + " . }");
		}

		sparql = QueryFactory.create("DESCRIBE <http://aa> FROM <http://test1>");
		vqe = VirtuosoQueryExecutionFactory.create (sparql, set);

		Model model = vqe.execDescribe();
 	        Graph g = model.getGraph();
                System.out.println("\nDESCRIBE results:");
	        for (Iterator i = g.find(Node.ANY, Node.ANY, Node.ANY); i.hasNext();)
  { 
	              Triple t = (Triple)i.next();
		      System.out.println(" { " + t.getSubject() + " " +
		      				 t.getPredicate() + " " +
		      				 t.getObject() + " . }");
  }



		sparql = QueryFactory.create("CONSTRUCT { ?x <http://test> ?y } FROM <http://test1> WHERE { ?x <http://bb> ?y }");
		vqe = VirtuosoQueryExecutionFactory.create (sparql, set);

		model = vqe.execConstruct();
 	        g = model.getGraph();
                System.out.println("\nCONSTRUCT results:");
	        for (Iterator i = g.find(Node.ANY, Node.ANY, Node.ANY); i.hasNext();)
  {
	              Triple t = (Triple)i.next();
		      System.out.println(" { " + t.getSubject() + " " +
		      				 t.getPredicate() + " " +
		      				 t.getObject() + " . }");
	        }


		sparql = QueryFactory.create("ASK FROM <http://test1> WHERE { <http://aa> <http://bb> ?y }");
		vqe = VirtuosoQueryExecutionFactory.create (sparql, set);

		boolean res = vqe.execAsk();
                System.out.println("\nASK results: "+res);


	}
  }
]]></programlisting>
    </sect4>
</sect3>
    <sect3 id="rdfnativestorageprovidersjenajavadoc"><title>Javadoc API Documentation</title>
    <para>Javadocs covers the complete set of classes, interfaces, and methods implemented by the provider:</para>
    <itemizedlist mark="bullet">
      <listitem><ulink url="http://docs.openlinksw.com/jena/">Javadoc API Documentation for the Jena 2.6 Provider</ulink></listitem>
      <listitem><ulink url="http://docs.openlinksw.com/jena2/">Javadoc API Documentation for the Jena 2.10+ Provider</ulink></listitem>
    </itemizedlist>
    </sect3>
  </sect2>
  <sect2 id="rdfnativestorageproviderssesame"><title>Virtuoso Sesame Provider</title>
    <sect3 id="rdfnativestorageproviderssesamewhatis"><title>What is Sesame</title>
      <para>Sesame is an open source Java framework for storing, querying and reasoning with RDF and RDF Schema.
      It can be used as a database for RDF and RDF Schema, or as a Java library for applications that need to work
      with RDF internally. For example, suppose you need to read a big RDF file, find the relevant information
      for your application, and use that information. Sesame provides you with the necessary tools to parse,
      interpret, query and store all this information, embedded in your own application if you want, or, if
      you prefer, in a separate database or even on a remote server. More generally: Sesame provides an application
      developer a toolbox that contains useful hammers screwdrivers etc. for doing 'Do-It-Yourself' with RDF.
      </para>
    </sect3>
    <sect3 id="rdfnativestorageproviderssesamewhatisvirtuososesameprovider">
      <title>What is the Virtuoso Sesame Provider</title>
      <para>The Virtuoso Sesame Provider is a fully operational Native Graph Model Storage Providers for the Sesame
      Framework, allowing users of Virtuoso to leverage the Sesame framework for modifying, querying, and reasoning
      with the Virtuoso quad store using the Java language. The Sesame Repository API offers a central access point
      for connecting to the Virtuoso quad store. Its purpose is to provides a java-friendly access point to
      Virtuoso. It offers various methods for querying and updating the data, while abstracting the details of the
      underlying machinery. The Provider has been tested against the two latest currently available versions,
      <ulink url="http://www.openrdf.org/download_sesame2.jsp">Sesame</ulink> 2.6.x and 2.7.x.
</para>
    <figure id="rdfnativestorageproviderssesame1" float="1">
      <title>Fig. 1 Sesame Component Stack</title>
      <graphic fileref="ui/VirtSesame2Provider.png"/>
    </figure>
    <para>If you need more information about how to set up your environment for working with the Sesame APIs, take a look at
    	Chapter 4 of the Sesame User Guide, <ulink url="http://www.openrdf.org/doc/sesame2/users/ch04.html">Setting up to use the Sesame libraries</ulink>.
</para>
    </sect3>
    <sect3 id="rdfnativestorageproviderssesamesetup"><title>Setup</title>
    <sect4 id="rdfnativestorageproviderssesamereqfiles"><title>Required Files</title>
      <para>This tutorial assumes you have Virtuoso server installed and that the database is accessible at
      "localhost:1111". In addition, you will need the latest version of the Virtuoso Sesame Provider, and Sesame
      2 or greater installed.</para>
      <para>You should download the Virtuoso Sesame 2 Provider JAR archive for the version of Sesame being used,
      Virtuoso JDBC Driver, Sesame Framework and associated classes and sample programs from our
      <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload">download page</ulink> .
      Note the version of the Sesame Provider (virt_sesame2.jar) can be determined with the command:
</para>
<programlisting><![CDATA[
$ java -jar virt_sesame2.jar
OpenLink Virtuoso(TM) Provider for Sesame2(TM) Version 2.6.5 [Build 1.7]
]]></programlisting>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamecmppr"><title>Compiling Sesame 2 Sample Program</title>
    <orderedlist>
      <listitem>Ensure that full paths to the following files are all included in the active CLASSPATH setting --
    <itemizedlist mark="bullet">
      <listitem>openrdf-sesame-2.1.2-onejar.jar</listitem>
      <listitem>slf4j-api-1.5.0.jar</listitem>
      <listitem>slf4j-jdk14-1.5.0.jar</listitem>
      <listitem>commons-io-2.0.jar</listitem>
      <listitem>virtjdbc3.jar</listitem>
      <listitem>virt_sesame2.jar</listitem>
   </itemizedlist>
      </listitem>
      <listitem>Execute the following command:
<programlisting><![CDATA[
javac VirtuosoTest.java
]]></programlisting>
      <para>Note: we recommend adding the following to the connect string, to use utf-8 and row-auto-commit:
</para>
<programlisting><![CDATA[
"/charset=UTF-8/log_enable=2"
-- i.e. in VirtuosoTest.java the line:
Repository repository = new VirtuosoRepository("jdbc:virtuoso://" + sa[0] + ":" + sa[1], sa[2], sa[3]);
-- should become:
Repository repository = new VirtuosoRepository("jdbc:virtuoso://" + sa[0] + ":" + sa[1]+ "/charset=UTF-8/log_enable=2", sa[2], sa[3]);
]]></programlisting>
      </listitem>
    </orderedlist>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesametesting"><title>Testing</title>
    <orderedlist>
      <listitem>Ensure that full paths to the following files are all included in the active CLASSPATH setting (note the addition of virtuoso_driver, here):
    <itemizedlist mark="bullet">
      <listitem>openrdf-sesame-2.1.2-onejar.jar</listitem>
      <listitem>slf4j-api-1.5.0.jar</listitem>
      <listitem>slf4j-jdk14-1.5.0.jar</listitem>
      <listitem>commons-io-2.0.jar</listitem>
      <listitem>virtjdbc3.jar</listitem>
      <listitem>virt_sesame2.jar</listitem>
      <listitem>virtuoso_driver</listitem>
    </itemizedlist>
      </listitem>
      <listitem>Run the <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSesame2Provider?att=VirtuosoTest.java">VirtuosoTest</ulink> program to test the Sesame 2 Provider with the following command:
<programlisting><![CDATA[
java VirtuosoTest <hostname> <port> <uid> <pwd>
]]></programlisting>
      </listitem>
      <listitem>The test run should look like this:
<programlisting><![CDATA[

$ java VirtuosoTest localhost 1111 dba dba

== TEST 1:  : Start
   Loading data from URL: http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com/foaf.rdf
== TEST 1:  : End
PASSED: TEST 1

== TEST 2:  : Start
   Clearing triple store
== TEST 2:  : End
PASSED: TEST 2

== TEST 3:  : Start
   Loading data from file: virtuoso_driver/data.nt
== TEST 3:  : End
PASSED: TEST 3

== TEST 4:  : Start
   Loading UNICODE single triple
== TEST 4:  : End
PASSED: TEST 4

== TEST 5:  : Start
   Loading single triple
== TEST 5:  : End
PASSED: TEST 5

== TEST 6:  : Start
   Casted value type
== TEST 6:  : End
PASSED: TEST 6

== TEST 7:  : Start
   Selecting property
== TEST 7:  : End
PASSED: TEST 7

== TEST 8:  : Start
   Statement does not exist
== TEST 8:  : End
PASSED: TEST 8

== TEST 9:  : Start
   Statement exists (by resultset size)
== TEST 9:  : End
PASSED: TEST 9

== TEST 10:  : Start
   Statement exists (by hasStatement())
== TEST 10:  : End
PASSED: TEST 10

== TEST 11:  : Start
   Retrieving namespaces
== TEST 11:  : End
PASSED: TEST 11

== TEST 12:  : Start
   Retrieving statement (http://example.com/dataspace/person/kidehen http://myopenlink.net/foaf/name null)
== TEST 12:  : End
PASSED: TEST 12

== TEST 13:  : Start
   Writing the statements to file: (/Users/src/virtuoso-opensource/binsrc/sesame2/results.n3.txt)
== TEST 13:  : End
PASSED: TEST 13

== TEST 14:  : Start
   Retrieving graph ids
== TEST 14:  : End
PASSED: TEST 14

== TEST 15:  : Start
   Retrieving triple store size
== TEST 15:  : End
PASSED: TEST 15

== TEST 16:  : Start
   Sending ask query
== TEST 16:  : End
PASSED: TEST 16

== TEST 17:  : Start
   Sending construct query
== TEST 17:  : End
PASSED: TEST 17

== TEST 18:  : Start
   Sending describe query
== TEST 18:  : End
PASSED: TEST 18

============================
PASSED:18 FAILED:0
]]></programlisting>
      </listitem>
    </orderedlist>
    </sect4>
    </sect3>
    <sect3 id="rdfnativestorageproviderssesamegettingstarted"><title>Getting Started</title>
    <para>This section covers the essentials for connecting to and manipulating data stored in a Virtuoso repository using the Sesame API. More information on the Sesame Framework, including extended examples on how to use the API, can be found in Chapter 8 of the Sesame User's guide, 
    	<ulink url="http://www.openrdf.org/doc/sesame2/2.1.2/users/ch08.html#d0e833">the RepositoryConnection API</ulink>.
</para>
    <para>The interfaces for the Repository API can be found in packages virtuoso.sesame2.driver and org.openrdf.repository. Several implementations for these interface exist in the Virtuoso Provider download package. The 
    	<ulink url="http://www.openrdf.org/doc/sesame2/2.1.2/apidocs/">Javadoc reference for the Sesame API</ulink>
is available online and can also be found in the doc directory of the download.
</para>
    <sect4 id="rdfnativestorageproviderssesamegettingstartedcrrep"><title>Creating a Virtuoso Repository RDF object</title>
      <para>The first step to connecting to Virtuoso through the Sesame API is to create a Repository for it. The Repository object operates on (stacks of) Sail object(s) for storage and retrieval of RDF data.
</para>
      <para>One of the simplest configurations is a repository that just stores RDF data in main memory without applying any inference or whatsoever. This is also by far the fastest type of repository that can be used. The following code creates and initialize a non-inferencing main-memory repository:
</para>
<programlisting><![CDATA[
import virtuoso.sesame2.driver.VirtuosoRepository;

Repository myRepository = VirtuosoRepository("jdbc:virtuoso://localhost:1111","dba","dba");

myRepository.initialize();
]]></programlisting>
      <para>The constructor of the VirtuosoRepository class accepts the JDBC URL of the Virtuoso engine (the default port is 1111), the username and password of an authorized user. Following this example, the repository needs to be initialized to prepare the Sail(s) that it operates on, which includes operations such as restoring previously stored data, setting up connections to a relational database, etc.
</para>
      <para>The repository that is created by the above code is volatile: its contents are lost when the object is garbage collected or when the program is shut down. This is fine for cases where, for example, the repository is used as a means for manipulating an RDF model in memory.
</para>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamegettingstartedrdfvirtcr"><title>Creating a Virtuoso Repository Connection</title>
      <para>Now that we have created a VirtuosoRepository object instance, we want to do something 
      	with it. This is achieved through the use of the VirtuosoRepositoryConnection class, which 
      	can be created by the VirtuosoRepository class.
</para>
      <para>A VirtuosoRepositoryConnection represents - as the name suggests - a connection to the 
      	actual Virtuoso quad store. We can issue operations over this connection, and close it when 
      	we are done to make sure we are not keeping resources unnecessarily occupied.
</para>
      <para>In the following sections, we will show some examples of basic operations using the Northwind dataset.
</para>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamegettingstartedrdfvirt"><title>Adding RDF to Virtuoso</title>
      <para>The Repository implements the Sesame Repository API offers various methods for adding data to a repository. Data can be added pro grammatically by specifying the location of a file that contains RDF data, and statements can be added individually or in collections.
</para>
      <para>We perform operations on the repository by requesting a RepositoryConnection from the 
      	repository, which returns a VirtuosoRepositoryConnection object. On this 
      	VirtuosoRepositoryConnection object we can perform the various operations, such as query 
      	evaluation, getting, adding, or removing statements, etc.
</para>
      <para>The following example code adds two files, one local and one located on the WWW, to a repository:
</para>
<programlisting><![CDATA[
import org.openrdf.repository.RepositoryException;

import org.openrdf.repository.Repository;

import org.openrdf.repository.RepositoryConnection;

import org.openrdf.rio.RDFFormat;

import java.io.File;

import java.net.URL;

File file = new File("/path/to/example.rdf");

String baseURI = "http://example.org/example/localRDF";

?

try {

   RepositoryConnection con = myRepository.getConnection();

   try {

      con.add(file, baseURI, RDFFormat.RDFXML);

      URL url = new URL("http://example.org/example/remoteRDF");

      con.add(url, url.toString(), RDFFormat.RDFXML);

   }

   finally {

      con.close();

   }

}

catch (RepositoryException rex) {

   // handle exception

}

catch (java.io.IOEXception e) {

   // handle io exception

}
]]></programlisting>
      <para>More information on other available methods can be found in the javadoc
reference of the RepositoryConnection interface.
</para>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamegettingstartedqr"><title>Querying Virtuoso</title>
      <para>The Repository API has a number of methods for creating and evaluating queries. Three types of queries are distinguished: tuple queries, graph queries and boolean queries. The query types differ in the type of results that they produce.
</para>
      <para><emphasis>Select Query:</emphasis> The result of a select query is a set of tuples (or 
      variable bindings), where each tuple represents a solution of a query. This type of query is 
      commonly used to get specific values (URIs, blank nodes, literals) from the stored RDF data. 
      The method QueryFactory.executeQuery() returns a Value [ ][ ] for sparql "SELECT" queries.
      The method QueryFactory.executeQuery() also calls the QueryFactory.setResult() which populates 
      a set of tuples for SPARQL "SELECT" queries. The graph can be retrieved using 
      QueryFactory.getBooleanResult().
</para>
      <para><emphasis>Graph Query:</emphasis> The result of graph queries is an RDF graph (or set of statements). This type of query is very useful for extracting sub-graphs from the stored RDF data, which can then be queried further, serialized to an RDF document, etc. The method QueryFactory.executeQuery() calls the QueryFactory.setGraphResult() which populates a graph for SPARQL "DESCRIBE" and "CONSTRUCT" queries. The graph can be retrieved using QueryFactory.getGraphResult().
</para>
      <para><emphasis>Boolean Query:</emphasis> The result of boolean queries is a simple boolean value, i.e. true of false. This type of query can be used to check if a repository contains specific information. The method QueryFactory.executeQuery() calls the QueryFactory.setBooleanResult() which sets a boolean value for sparql "ASK" queries. The value can be retrieved using QueryFactory.getBooleanResult().
</para>
      <para>Note: Although Sesame 2 currently supports two query languages: SeRQL and SPARQL, the Virtuoso provider only supports the W3C SPARQL specification. 
</para>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamegettingstartevq"><title>Evaluating a SELECT Query</title>
      <para>To evaluate a tuple query we simply do the following:
</para>
<programlisting><![CDATA[
import java.util.List;

import org.openrdf.OpenRDFException;

import org.openrdf.repository.RepositoryConnection;

import org.openrdf.query.TupleQuery;

import org.openrdf.query.TupleQueryResult;

import org.openrdf.query.BindingSet;

import org.openrdf.query.QueryLanguage;

?

try {

   RepositoryConnection con = myRepository.getConnection();

   try {

      String queryString = "SELECT x, y FROM  WHERE {x} p {y}";

      TupleQuery tupleQuery = con.prepareTupleQuery(QueryLanguage.SPARQL, queryString);

      TupleQueryResult result = tupleQuery.evaluate();

      try {

         ? // do something with the result

      }

      finally {

         result.close();

      }

   }

   finally {

      con.close();

   }

}

catch (RepositoryException e) {

   // handle exception

}
]]></programlisting>
      <para>This evaluates a SPARQL query and returns a TupleQueryResult, which consists of a sequence 
      	of BindingSet objects. Each BindingSet contains a set of pairs called Binding objects. A 
      	Binding object represents a name/value pair for each variable in the query's projection.
</para>
      <para>We can use the TupleQueryResult to iterate over all results and get each individual result for x and y:
</para>
<programlisting><![CDATA[
while (result.hasNext()) {

   BindingSet bindingSet = result.next();

   Value valueOfX = bindingSet.getValue("x");

   Value valueOfY = bindingSet.getValue("y");

   // do something interesting with the query variable values here?

}
]]></programlisting>
      <para>As you can see, we retrieve values by name rather than by an index. The names used should be the names of variables as specified in your query. The TupleQueryResult.getBindingNames() method returns a list of binding names, in the order in which they were specified in the query. To process the bindings in each binding set in the order specified by the projection, you can do the following:
</para>
<programlisting><![CDATA[
List bindingNames = result.getBindingNames();

while (result.hasNext()) {

   BindingSet bindingSet = result.next();

   Value firstValue = bindingSet.getValue(bindingNames.get(0));

   Value secondValue = bindingSet.getValue(bindingNames.get(1));

   // do something interesting with the values here?

}
]]></programlisting>
      <para>It is important to invoke the close() operation on the TupleQueryResult, 
      	after we are done with it. A TupleQueryResult evaluates lazily and keeps resources 
      	(such as connections to the underlying database) open. Closing the TupleQueryResult 
      	frees up these resources. Do not forget that iterating over a result may cause exceptions! 
      	The best way to make sure no connections are kept open unnecessarily is to invoke close() 
      	in the finally clause.
</para>
      <para>An alternative to producing a TupleQueryResult is to supply an object that implements the 
      	TupleQueryResultHandler interface to the query's evaluate() method. The main difference is 
      	that when using a return object, the caller has control over when the next answer is retrieved, 
      	whereas with the use of a handler, the connection simply pushes answers to the handler object 
      	as soon as it has them available.
</para>
      <para>As an example we will use SPARQLResultsXMLWriter, which is a TupleQueryResultHandler 
      	implementation that writes SPARQL Results XML documents to an output stream or to a writer:
</para>
<programlisting><![CDATA[
import org.openrdf.query.resultio.sparqlxml.SPARQLResultsXMLWriter;

?

FileOutputStream out = new FileOutputStream("/path/to/result.srx");

try {

   SPARQLResultsXMLWriter sparqlWriter = new SPARQLResultsXMLWriter(out);

   RepositoryConnection con = myRepository.getConnection();

   try {

      String queryString = "SELECT * FROM  WHERE {x} p {y}";

      TupleQuery tupleQuery = con.prepareTupleQuery(QueryLanguage.SPARQL, queryString);

      tupleQuery.evaluate(sparqlWriter);

}

   finally {

      con.close();

}

    }

finally {

   out.close();

}
]]></programlisting>
      <para>You can just as easily supply your own application-specific implementation of TupleQueryResultHandler though.
      </para>
      <para>Lastly, an important warning: as soon as you are done with the RepositoryConnection 
      	object, you should close it. Notice that during processing of the TupleQueryResult object 
      	(for example, when iterating over its contents), the RepositoryConnection should still be 
      	open. We can invoke con.close() after we have finished with the result. 
      </para>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamegettingstartevcnq"><title>Evaluating a CONSTRUCT query</title>
      <para>The following code evaluates a graph query on a repository:
</para>
<programlisting><![CDATA[
import org.openrdf.query.GraphQueryResult;

GraphQueryResult graphResult = con.prepareGraphQuery(

      QueryLanguage.SPARQL, "CONSTRUCT * FROM {x} p {y}").evaluate();
]]></programlisting>
      <para> A GraphQueryResult is similar to TupleQueryResult in that is an object that 
      	iterates over the query results. However, for graph queries the query results are RDF 
      	statements, so a GraphQueryResult iterates over Statement objects:
</para>
<programlisting><![CDATA[
while (graphResult.hasNext()) {

   Statement st = graphResult.next();

   // ? do something with the resulting statement here.


}
]]></programlisting>
      <para>The TupleQueryResultHandler equivalent for graph queries is org.openrdf.rio.RDFHandler. 
      	Again, this is a generic interface, each object implementing it can process the reported RDF 
      	statements in any way it wants.
      </para>
      <para>All writers from Rio (such as the RDFXMLWriter, TurtleWriter, TriXWriter, etc.) implement 
      	the RDFHandler interface. This allows them to be used in combination with querying quite easily. 
      	In the following example, we use a TurtleWriter to write the result of a SPARQL graph query to 
      	standard output in Turtle format:
      </para>
<programlisting><![CDATA[
import org.openrdf.rio.turtle.TurtleWriter;

?

RepositoryConnection con = myRepository.getConnection();

try {

   TurtleWriter turtleWriter = new TurtleWriter(System.out);

   con.prepareGraphQuery(QueryLanguage.SPARQL, "CONSTRUCT * FROM  WHERE {x} p {y}").evaluate(turtleWriter);

}

finally {

   con.close();

}
]]></programlisting>
      <para>Again, note that as soon as we are done with the result of the query (either after iterating over the contents of the GraphQueryResult? or after invoking the RDFHandler), we invoke con.close() to close the connection and free resources.
    </para>
    </sect4>
    <sect4 id="rdfnativestorageproviderssesamegettingstartevcnqapi"><title>Javadoc API Documentation</title>
      <para>Javadocs covers the complete set of classes, interfaces, and methods implemented by the provider:</para>
      <itemizedlist mark="bullet">
        <listitem><ulink url="http://docs.openlinksw.com/sesame/">Javadoc API Documentation for the Sesame 2.6 Provider</ulink></listitem>
        <listitem><ulink url="http://docs.openlinksw.com/sesame2/">Javadoc API Documentation for the Sesame 2.7+ Provider</ulink></listitem>
      </itemizedlist>
    </sect4>
    </sect3>
    <sect3 id="rdfnativestorageproviderssesamestpandtesting"><title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
      <sect4 id="rdfnativestorageproviderssesamestpandtestingwhat"><title>What</title>
<para>Sesame is an open source Java framework for storing, querying and reasoning with RDF and RDF Schema. It can be used as a
database for RDF and RDF Schema, or as a Java library for applications that need to work with RDF internally. The Sesame HTTP
repository serves as a proxy for a RDF store hosted on a remote Sesame server, enabling the querying of the RDF store using
the Sesame HTTP protocol.
    </para>
      </sect4>
      <sect4 id="rdfnativestorageproviderssesamestpandtestingwhy"><title>Why</title>
<para>The Sesame HTTP repository endpoint provides users with the greater flexibility for manipulating the RDF store via a common
interface. Sesame provides you with the necessary tools to parse, interpret, query and store all this information, embedded
in your own application if you want, or, if you prefer, in a separate database or even on a remote server.
</para>
      </sect4>
      <sect4 id="rdfnativestorageproviderssesamestpandtestinghow"><title>How</title>
<para>To create a new Sesame HTTP repository, the Console needs to create such an RDF document and submit it to the SYSTEM
repository. The Console uses so called repository configuration templates to accomplish this. Repository configuration templates
are simple Turtle RDF files that describe a repository configuration, where some of the parameters are replaced with variables.
The Console parses these templates and asks the user to supply values for the variables. The variables are then substituted with
the specified values, which produces the required configuration data.
</para>
      </sect4>
      <sect4 id="rdfnativestorageproviderssesamestpandtestingmain"><title>Setup and Testing</title>
<para>This section details the steps required for configuring and testing a Virtuoso Sesame
Repository, both using the HTTP and Console Sesame repositories.</para>
      <sect5 id="rdfnativestorageproviderssesamestpandtestingreq"><title>Requirements</title>
    <itemizedlist mark="bullet">
  <listitem><ulink url="http://www.openrdf.org/download.jsp">Sesame 2.3.1</ulink> or higher</listitem>
  <listitem><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSesame2HttpRepository/virt_sesame2.jar">Virtuoso Sesame 2 Provider </ulink> (virt_sesame2.jar)</listitem>
  <listitem><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSesame2HttpRepository/virtjdbc3.jar">Virtuoso JDBC Driver</ulink> (virtjdbc3.jar)</listitem>
  <listitem><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSesame2HttpRepository/create.xsl">Sesame System Repository config file</ulink> (create.xsl)</listitem>
  <listitem><ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSesame2HttpRepository/create-virtuoso.xsl">Sesame Virtuoso Repository config file </ulink> (create-virtuoso.xsl)</listitem>
  <listitem><ulink url="http://tomcat.apache.org/download-60.cgi">Apache Tomcat</ulink> version 5 or 6</listitem>
    </itemizedlist>
      </sect5>
      <sect5 id="rdfnativestorageproviderssesamestpandtestinghttprep"><title>Setup Sesame HTTP Repository</title>
<para>This section details the steps required for configuring and testing a Virtuoso HTTP Sesame Repository.</para>
    <orderedlist>
  <listitem>Install <ulink url="http://tomcat.apache.org/tomcat-6.0-doc/index.html">Apache Tomcat</ulink> web server</listitem>
  <listitem>From the Sesame 2.3.1 or higher "lib" directory copy the "openrdf-sesame.war" and "openrdf-worbbench.war" files to the
tomcat "webapps" directory where they will automatically be deployed creating two new sub directories "openrdf-sesame" and
"openrdf-workbench".</listitem>
  <listitem>Place the Virtuoso Sesame Provider "virt_sesame2.jar" and JDBC Driver "virtjdbc3.jar" into the Tomcat
<code>~/webapps/openrdf-sesame/WEB-INF/lib/</code> and <code>~/webapps/openrdf-workbench/WEB-INF/lib/</code> directories for
use by the Sesame HTTP Repository for accessing the Virtuoso RDF repository.</listitem>
  <listitem>Place the "create.xsl" and "create-virtuoso.xsl" files in the Tomcat
<code>~/webapps/openrdf-workbench/transformations/</code> directory. Note "create.xsl" replaces the default provided with Sesame and contains the necessary entries required to reference the new "create-virtuoso.xsl" template file for Virtuoso repository configuration.</listitem>
  <listitem>The Sesame HTTP Repository will now be accessible on the URLs
<programlisting><![CDATA[
http://example.com/openrdf-sesame
http://example.com/openrdf-workbench
]]></programlisting>
         </listitem>
   <listitem>The Sesame OpenRDF Workbench is used for accessing the Sesame HTTP Repositories, loading
"<ulink url="http://example.com/openrdf-workbench">http://example.com/openrdf-workbench</ulink>" will enable the
default "SYSTEM" repository to be accessed.
     <figure id="ss1" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss1.png"/>
     </figure>
         </listitem>
   <listitem>Click on the "New Repository" link in the left frame to create a new Sesame Repository.
     <figure id="ss2" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss2.png"/>
     </figure>
         </listitem>
   <listitem>Select the "Virtuoso RDF Store" from the "Type" drop down list box presented.
     <figure id="ss3" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss3.png"/>
     </figure>
      </listitem>
   <listitem>Choose suitable repository "ID" and "Title" for the Virtuoso repository to be created and click "Next".
     <figure id="ss4" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss4.png"/>
     </figure>
      </listitem>
   <listitem>Fill in the connection parameters for the target Virtuoso sever the repository is to be created for and
click the "create" button. The minimum required are the hostname, port number, username and password of the Virtuoso Server.
     <figure id="ss5" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss5.png"/>
     </figure>
      </listitem>
   <listitem>The new Virtuoso respository will be created and its summary page displayed.
     <figure id="ss6" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss6.png"/>
     </figure>
      </listitem>
   <listitem>Click on the "Namespaces" link in the left frame to obtain a list of the available namespaces in the Virtuoso repository.
     <figure id="ss7" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss7.png"/>
     </figure>
      </listitem>
   <listitem>Click on the "Context" link in the left frame to obtain a list of the available contexts in the Virtuoso repository.
     <figure id="ss8" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss8.png"/>
     </figure>
      </listitem>
   <listitem>Click on the "Types" link in the left frame to obtain a list of the available types in the Virtuoso repository.
     <figure id="ss9" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss9.png"/>
     </figure>
</listitem>
   <listitem>Click on the "Query" link in the left frame, enter a suitable SPARQL query to execute against the Virtuoso repository
and click the "execute" button.
     <figure id="ss10" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss10.png"/>
     </figure>
      </listitem>
   <listitem>The results of the SPARQL query are returned.
     <figure id="ss11" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss11.png"/>
     </figure>
  </listitem>
   <listitem>Click on the "Repositories" link in the left frame and the newly created Virtuoso repository entry is displayed along side
the default SYSTEM repository.
     <figure id="ss12" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss12.png"/>
     </figure>
  </listitem>
</orderedlist>
      </sect5>
      <sect5 id="rdfnativestorageproviderssesamestpandtestingcons"><title>Setup Sesame Console Repository</title>
<para>This section details the steps required for configuring and testing a Virtuoso Sesame Console Repository:</para>
<orderedlist>
  <listitem>Extract Sesame 2.3.1 or higher archive to a location of choice and place the virt_sesame2.jar and virtjdbc3.jar
files to the sesame 2.3.1 "lib" directory</listitem>
  <listitem>Start the <ulink url="http://www.openrdf.org/doc/sesame2/users/ch07.html#section-console-repository-creation">sesame console application</ulink> by running the "console.bat" script in the sesame "bin" directory and then "exit." the program
<programlisting><![CDATA[
$ sh console.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/myuser/openrdf-sesame-2.3.1/lib/logback-classic-0.9.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/myuser/openrdf-sesame-2.3.1/lib/slf4j-jdk14-1.5.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
10:32:38.317 [main] DEBUG info.aduna.platform.PlatformFactory - os.name <http://os.name> <http://os.name> = mac os x
10:32:38.351 [main] DEBUG info.aduna.platform.PlatformFactory - Detected Mac OS X platform
Connected to default data directory
Commands end with '.' at the end of a line
Type 'help.' for help
exit.
]]></programlisting>
      </listitem>
  <listitem>This will create the necessary sesame application data directories as detailed in the sesame
<ulink url="http://www.openrdf.org/doc/sesame2/2.3.1/users/userguide.html#chapter-datadir-config">data directory configuration</ulink> documentation.
<programlisting><![CDATA[
Windows - C:\Documents and Settings\LocalService\Application Data\Aduna\
Mac OS X - /Users/myuser/Library/Application Support/Aduna/
Linux - $HOME/.aduna/
]]></programlisting>
      </listitem>
  <listitem>If you do not want to use the default sesame data directory location the Sesame console application can be started by
specifying a custom data directory location with the "-d" option. Note in this case the directory "OpenRDF Sesame console" always
has to be manually appended to the directory as Sesame assumes the data file will reside in a sub directory of this name.
<programlisting><![CDATA[
$ sh console.sh -d /Users/myuser/OpenRDF Sesame console
]]></programlisting>
      </listitem>
  <listitem>Start the sesame console application with the required data directory location and create a Virtuoso repository as
detailed in the steps below, the key parameters to be specified being the target Virtuoso server hostname, port number,
username, password and a unique "Repository ID".
<programlisting><![CDATA[
$ sh console.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/myuser/openrdf-sesame-2.3.1/lib/logback-classic-0.9.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/myuser/openrdf-sesame-2.3.1/lib/slf4j-jdk14-1.5.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
10:32:38.317 [main] DEBUG info.aduna.platform.PlatformFactory - os.name <http://os.name> <http://os.name> = mac os x
10:32:38.351 [main] DEBUG info.aduna.platform.PlatformFactory - Detected Mac OS X platform
Connected to default data directory
Commands end with '.' at the end of a line
Type 'help.' for help
create virtuoso .
Please specify values for the following variables:
Host list [localhost:1111]:
Username [dba]:
Password [dba]:
Default graph name [sesame:nil]:
Enable using batch optimization (false|true) [false]:
Use RoundRobin for connection (false|true) [false]:
Buffer fetch size [200]:
Inference RuleSet name [null]:
Repository ID [virtuoso]: myvirt
Repository title [Virtuoso repository]:
Repository created
show r .
+----------
|SYSTEM
|myvirt ("Virtuoso repository")
+----------
open myvirt .
Opened repository 'myvirt'
myvirt> show n .
+----------
|bif  bif:
|dawgt  http://www.w3.org/2001/sw/DataAccess/tests/test-dawg#
|dbpedia  http://dbpedia.org/resource/
|dbpprop  http://dbpedia.org/property/
|dc  http://purl.org/dc/elements/1.1/
|foaf  http://xmlns.com/foaf/0.1/
|geo  http://www.w3.org/2003/01/geo/wgs84_pos#
|go  http://purl.org/obo/owl/GO#
|math  http://www.w3.org/2000/10/swap/math#
|mesh  http://purl.org/commons/record/mesh/
|mf  http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#
|nci  http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#
|obo  http://www.geneontology.org/formats/oboInOwl#
|owl  http://www.w3.org/2002/07/owl#
|protseq  http://purl.org/science/protein/bysequence/
|rdf  http://www.w3.org/1999/02/22-rdf-syntax-ns#
|rdfdf  http://www.openlinksw.com/virtrdf-data-formats#
|rdfs  http://www.w3.org/2000/01/rdf-schema#
|sc  http://purl.org/science/owl/sciencecommons/
|scovo  http://purl.org/NET/scovo#
|skos  http://www.w3.org/2004/02/skos/core#
|sql  sql:
|vcard  http://www.w3.org/2001/vcard-rdf/3.0#
|virtrdf  http://www.openlinksw.com/schemas/virtrdf#
|void  http://rdfs.org/ns/void#
|xf  http://www.w3.org/2004/07/xpath-functions
|xml  http://www.w3.org/XML/1998/namespace
|xsd  http://www.w3.org/2001/XMLSchema#
|xsl10  http://www.w3.org/XSL/Transform/1.0
|xsl1999  http://www.w3.org/1999/XSL/Transform
|xslwd  http://www.w3.org/TR/WD-xsl
|yago  http://dbpedia.org/class/yago/
+----------
exit.
]]></programlisting>
     </listitem>
    </orderedlist>
      </sect5>
      <sect5 id="rdfnativestorageproviderssesamestpandtestinghttpcons"><title>Connection to Sesame HTTP repository from Console repository</title>
<para>The Sesame Console repository can connect to a Sesame HTTP repository and vice-versa, enabling access to remote Sesame
HTTP repositories from a local server.</para>
<orderedlist>
  <listitem>The Sesame Console repository can connect to a Sesame HTTP repository and query it as if local using the "connect" command.
<programlisting><![CDATA[
$ sh console.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/myuser/openrdf-sesame-2.3.1/lib/logback-classic-0.9.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/myuser/openrdf-sesame-2.3.1/lib/slf4j-jdk14-1.5.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
10:32:38.317 [main] DEBUG info.aduna.platform.PlatformFactory - os.name <http://os.name> <http://os.name> = mac os x
10:32:38.351 [main] DEBUG info.aduna.platform.PlatformFactory - Detected Mac OS X platform
Connected to default data directory
Commands end with '.' at the end of a line
Type 'help.' for help
> connect http://example.com/openrdf-sesame.
Connected to http://example.com/openrdf-sesame
> show r.
+----------
|SYSTEM ("System configuration repository")
|VirtSesRep ("Virtuoso Sesame HTTP Repository")
+----------
> open VirtSesRep.
Opened repository 'VirtSesRep'
VirtSesRep> sparql select * from <http://example.com/Northwind> where {?s ?p ?o} Limit 10.
Evaluating query...
+------------------------+------------------------+------------------------+
| s                      | p                      | o                      |
+------------------------+------------------------+------------------------+
| <http://example.com/Northwind/CustomerContact/ALFKI#this>| rdf:type  | foaf:Person            |
| <http://example.com/Northwind/CustomerContact/ALFKI#this>| rdf:type  | northwind:CustomerContact|
| <http://example.com/Northwind/CustomerContact/ALFKI#this>| opl:isDescribedUsing   | northwind:             |
| <http://example.com/Northwind/CustomerContact/ANATR#this>| rdf:type | foaf:Person            |
| <http://example.com/Northwind/CustomerContact/ANATR#this>| rdf:type | northwind:CustomerContact|
| <http://example.com/Northwind/CustomerContact/ANATR#this>| opl:isDescribedUsing   | northwind:             |
| <http://example.com/Northwind/CustomerContact/ANTON#this>| rdf:type  | foaf:Person            |
| <http://example.com/Northwind/CustomerContact/ANTON#this>| rdf:type  | northwind:CustomerContact|
| <http://example.com/Northwind/CustomerContact/ANTON#this>| opl:isDescribedUsing   | northwind:             |
| <http://example.com/Northwind/CustomerContact/AROUT#this>| rdf:type  | foaf:Person            |
+------------------------+------------------------+------------------------+
10 result(s) (530 ms)
VirtSesRep> show n.
+----------
|SearchResults  http://www.zillow.com/static/xsd/SearchResults.xsd
|UpdatedPropertyDetails  http://www.zillow.com/static/xsd/UpdatedPropertyDetails.xsd
|a  http://www.w3.org/2005/Atom
|aapi  http://rdf.alchemyapi.com/rdf/v1/s/aapi-schema#
|address  http://schemas.talis.com/2005/address/schema#
|admin  http://webns.net/mvcb/
|amz  http://webservices.amazon.com/AWSECommerceService/2005-10-05
|atom  http://atomowl.org/ontologies/atomrdf#
|audio  http://purl.org/media/audio#
|awol  http://bblfish.net/work/atom-owl/2006-06-06/#
|aws  http://soap.amazon.com/
|b3s  http://b3s.openlinksw.com/
]]></programlisting>
</listitem>
  <listitem>Conversely the Sesame HTTP repository can be configured to access the repository created by the Sesame console.
To do this the location of the data directory for both needs to be reconfigured using the Java system property
info.aduna.platform.appdata.basedir (does not include "OpenRDF Sesame console directory) to point to the same location.
When you are using Tomcat as the servlet container then you can set this property using the JAVA_OPTS parameter.
Note, if you are using Apache Tomcat as a Windows Service you should use the Windows Services configuration tool to
set this property. Other users can either edit the Tomcat startup script or set the property some other way.
<programlisting><![CDATA[
 * set JAVA_OPTS=-Dinfo.aduna.platform.appdata.basedir=\path\to\other\dir\ (on Windows)
    * export JAVA_OPTS='-Dinfo.aduna.platform.appdata.basedir=/path/to/other/dir/' (on Linux/UNIX/Mac OS X)
]]></programlisting>
     <figure id="ss13" float="1">
       <title>Virtuoso Sesame HTTP Repository Configuration and Usage</title>
       <graphic fileref="ui/ss13.png"/>
     </figure>

</listitem>
</orderedlist>
       </sect5>
    </sect4>
    </sect3>
    <sect3 id="rdfnativestorageproviderssesamejavadoc"><title>Javadoc API Documentation</title>
    <para><ulink url="http://docs.openlinksw.com/sesame/">Sesame Provider Javadoc API Documentation</ulink>
is available enabling the complete set of classes, interfaces and methods implemented for the provider to be viewed.
    </para>
    </sect3>
  </sect2>
  <sect2 id="rdfnativestorageproviderredland"><title>Virtuoso Redland Provider</title>
     <sect3 id="rdfnativestorageproviderredlandwhatis"><title>What is Redland</title>
     <para><ulink url="http://librdf.org/">Redland</ulink> is a set of free software 'C' libraries that
provide support for the Resource Description Framework (RDF), providing modular, object based libraries
and APIs for manipulating the RDF graph, triples, URIs and Literals. Redland includes several high-level
language APIs providing RDF manipulation and storage and requires the
<ulink url="http://librdf.org/raptor/">Raptor</ulink> RDF parser and <ulink url="http://librdf.org/rasqal/">Rasqal</ulink>
RDF syntax and query library
for its use.
    </para>
    </sect3>
    <sect3 id="rdfnativestorageproviderredlandwhatisv"><title>What is the Virtuoso Redland Provider</title>
    <para>The Virtuoso Redland RDF Provider is an implementation of the Storage API, Model and Query
interfaces of the Redland framework for RDF. This provider enables the execution of queries via the
Redland Rasqal query engine or via Virtuoso query engine directly against the Virtuoso Quad store.
The Virtuoso Redland Provider uses ODBC as the data access mechanism for communicating the Virtuoso
Quad Store and requires the Virtuoso ODBC Driver be installed on the Redland client and a suitable
ODBC DSN be configured for connecting to the target Virtuoso Quad Store instance. The provider has
been tested against the <ulink url="http://download.librdf.org/source/">Redland 1.0.8</ulink> version currently available for download.
    </para>
    <figure id="rdfnativestorageproviderredland1" float="1">
      <title>Redland Component Stack</title>
      <graphic fileref="ui/VirtRedLand.png"/>
    </figure>
    <para>As indicated in the above diagram the Virtuoso Provider can be used to execute RDF queries either
directly against the Virtuoso graph storage module supporting the <ulink url="http://dbpedia.org/resource/SPARQL">SPARQL</ulink>,
<ulink url="http://dbpedia.org/resource/SPARUL">SPARQL</ulink>SPARUL, <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSArticleBISPARQL2">SPARQL-BI</ulink>
query languages or via the Rasqal query engine built into Redland which supports the SPARQL query language.
This is done by simply changing the syntax of the query using the "vsparql" rather then default "sparql"
construct when executing a query as indicated in the sample queries below:
    </para>
<programlisting><![CDATA[
rdfproc -r xml -t "user='dba',password='dba',dsn='Demo'" gr query sparql - "SELECT * WHERE { ?s ?p ?o }"   ;; via Redland Rasqal engine

rdfproc -r xml -t "user='dba',password='dba',dsn='Demo'" gr query vsparql - "SELECT * WHERE { ?s ?p ?o }"  ;; direct to Virtuoso storage module
]]></programlisting>
    <para>The Virtuoso Provider uses the <ulink url="http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSSQL2RDF">SPASQL</ulink> query language for querying the remote Virtuoso QUAD store.
    </para>
    </sect3>
    <sect3 id="rdfnativestorageproviderredlandsetup"><title>Setup</title>
      <sect4 id="rdfnativestorageproviderredlandreqfiles"><title>Required Files</title>
      <para>The Virtuoso Redland Provider has been integrated into the Redland RDF Framework and submitted to
the open source project to become part of the standard distribution available for
<ulink url="http://librdf.org/INSTALL.html">download</ulink>. Until this
submission has been accepted and committed into the available Redland release a tar ball created by
OpenLink Software and a diff for application to a Redland 1.0.8 tree can be obtained from:
      </para>
      <itemizedlist mark="bullet">
        <listitem><ulink url="ftp://download.openlinksw.com/support/vos/redland-vos-1.0.8.tar.gz"></ulink>Redland 1.0.8 tar ball with Virtuoso storage support</listitem>
        <listitem><ulink url="ftp://download.openlinksw.com/support/vos/redland-vos.diff">Redland 1.0.8 Diff file of changes made for Virtuoso storage support</ulink></listitem>
      </itemizedlist>
    </sect4>
      <sect4 id="rdfnativestorageprovidersredlandcmsmpr"><title>Compiling Redland with Virtuoso storage support</title>
        <itemizedlist mark="bullet">
          <listitem><ulink url="http://svn.librdf.org/">Download Redland</ulink>, extract and apply diff
above or download the tar ball above with diff already applied and extract to a location of choice.</listitem>
          <listitem>The following additional configure options are available for enabling the Virtuoso
storage support:
<programlisting><![CDATA[
  --with-virtuoso(=yes|no) Enable Virtuoso RDF store (default=auto)
  --with-iodbc(=DIR)        Select iODBC support
                            DIR is the iODBC base install directory
                            (default=/usr/local)
  --with-unixodbc(=DIR)   Select UnixODBC support
                            DIR is the UnixODBC base install directory
                            (default=/usr/local)
  --with-datadirect(=DIR) Select DataDirect support
                            DIR is the DataDirect base install directory
                            (default=/usr/local)
  --with-odbc-inc=DIR     Specify custom ODBC include directory
                            (default=/usr/local/include)
  --with-odbc-lib=DIR     Specify custom ODBC lib directory
                            (default=/usr/local/lib)
]]></programlisting>
          </listitem>
          <listitem>The "--with-virtuoso" option default to being auto enable if a valid ODBC Driver Manager
(iODBC, UnixODBC? or DataDirect?) or include and lib directories for required ODBC header files and libraries
are located with the suitable setting for one or more of the other ODBC related options above. Assuming
iODBC is installed the following option can be used to enable Virtuoso storage support to be configured
for compilation into your Redland build:
<programlisting><![CDATA[
./configure --with-iodbc=/usr/local/iODBC
]]></programlisting>
          </listitem>
          <listitem>Run "make" to compile the Redland libraries and "sudo make install" to install in
the default "/usr/local" location</listitem>
          <listitem>Test compilation with test utility utils/rdfproc:
<programlisting><![CDATA[

rdfproc test parse http://planetrdf.com/guide/rss.rdf
rdfproc test print
rdfproc test serialize ntriples
]]></programlisting>
          <para>This test will use the default 'hashes' storage.</para>
          </listitem>
          <listitem>Ensure you have the Virtuoso ODBC Driver installed and a valid ODBC DSN called
"Local Virtuoso" configured for your target Virtuoso Server</listitem>
          <listitem>Set the following environment variable:
<programlisting><![CDATA[
export RDFPROC_STORAGE_TYPE=virtuoso                                   ;; Enable Virtuoso Storage
export ODBCINI=<path_to_odbcini_directory>/odbc.ini                      ;; Enable ODBC DSN to be located
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH                     ;; May be required to enable Redland libraries to be located

]]></programlisting>
          </listitem>
          <listitem>Test Virtuoso storage with the provided test program utils/vtest:
<programlisting><![CDATA[
$ utils/vtest
  1: Remove all triples in <http://red> context
**PASSED**: removed context triples from the graph
  2: Add triples to <http://red> context
**PASSED**: add triple to context
  3: Print all triples in <http://red> context
[[
  {[aa], [bb], [cc]} with context [http://red]
  {[aa], [bb1], [cc]} with context [http://red]
  {[aa], [a2], "cc"} with context [http://red]
  {[aa], [a2], (cc)} with context [http://red]
  {[mm], [nn], "Some long literal with language@en"} with context [http://red]
  {[oo], [pp], "12345^^<http://www.w3.org/2001/XMLSchema#int>"} with context [http://red]
]]
**PASSED**:
  4: Count of triples in <http://red> context
**PASSED**: graph has 6 triples
  5: Exec:  ARC  aa bb
Matched node: [cc]
**PASSED**:
  6: Exec:  ARCS  aa cc
Matched node: [bb] with context [http://red]
Matched node: [bb1] with context [http://red]
: matching nodes: 2
**PASSED**:
  7: Exec:  ARCS-IN  cc
Matched arc: [bb] with context [http://red]
Matched arc: [bb1] with context [http://red]
**PASSED**: matching arcs: 2
  8: Exec:  ARCS-OUT  aa
Matched arc: [bb] with context [http://red]
Matched arc: [bb1] with context [http://red]
Matched arc: [a2] with context [http://red]
Matched arc: [a2] with context [http://red]
**PASSED**: matching arcs: 4
  9: Exec:  CONTAINS aa bb1 cc
**PASSED**: the graph contains the triple
 10: Exec:  FIND aa - -
Matched triple: {[aa], [bb], [cc]} with context [http://red]
Matched triple: {[aa], [bb1], [cc]} with context [http://red]
Matched triple: {[aa], [a2], "cc"} with context [http://red]
Matched triple: {[aa], [a2], (cc)} with context [http://red]
**PASSED**: matching triples: 4
 11: Exec:  HAS-ARC-IN cc bb
**PASSED**: the graph contains the arc
 12: Exec:  HAS-ARC-OUT aa bb
**PASSED**: the graph contains the arc
 13: Exec:  SOURCE  aa cc
Matched node: [aa]
**PASSED**:
 14: Exec:  SOURCES  bb cc
Matched node: [aa] with context [http://red]
: matching nodes: 1
**PASSED**:
 15: Exec:  TARGET  aa bb
Matched node: [cc]
**PASSED**:
 16: Exec:  TARGETS  aa bb
Matched node: [cc] with context [http://red]
: matching nodes: 1
**PASSED**:
 17: Exec:  REMOVE aa bb1 cc
**PASSED**: removed triple from the graph
 18: Exec:  QUERY "CONSTRUCT {?s ?p ?o} FROM <http://red> WHERE {?s ?p ?o}"
Matched triple: {[aa], [a2], "cc"}
Matched triple: {[oo], [pp], "12345^^<http://www.w3.org/2001/XMLSchema#int>"}
Matched triple: {[aa], [a2], (cc)}
Matched triple: {[aa], [bb], [cc]}
Matched triple: {[mm], [nn], "Some long literal with language@en"}
**PASSED**: matching triples: 5
 19: Exec1:  QUERY_AS_BINDINGS "SELECT * WHERE {graph <http://red> { ?s ?p ?o }}"
**: Formatting query result as 'xml':
<?xml version="1.0" encoding="utf-8"?>
<sparql xmlns="http://www.w3.org/2005/sparql-results#">
  <head>
    <variable name="s"/>
    <variable name="p"/>
    <variable name="o"/>
  </head>
  <results>
    <result>
      <binding name="s"><uri>aa</uri></binding>
      <binding name="p"><uri>bb</uri></binding>
      <binding name="o"><uri>cc</uri></binding>
    </result>
    <result>
      <binding name="s"><uri>aa</uri></binding>
      <binding name="p"><uri>a2</uri></binding>
      <binding name="o"><literal>cc</literal></binding>
    </result>
    <result>
      <binding name="s"><uri>aa</uri></binding>
      <binding name="p"><uri>a2</uri></binding>
      <binding name="o"><bnode>cc</bnode></binding>
    </result>
    <result>
      <binding name="s"><uri>mm</uri></binding>
      <binding name="p"><uri>nn</uri></binding>
      <binding name="o"><literal>Some long literal with language@en</literal></binding>
    </result>
    <result>
      <binding name="s"><uri>oo</uri></binding>
      <binding name="p"><uri>pp</uri></binding>
      <binding name="o"><literal>12345^^<http://www.w3.org/2001/XMLSchema#int></literal></binding>
    </result>
  </results>
</sparql>
**PASSED**:
 20: Exec2:  QUERY_AS_BINDINGS "SELECT * WHERE {graph <http://red> { ?s ?p ?o }}"
: Query returned bindings results:
result: [s=[aa], p=[bb], o=[cc]]
result: [s=[aa], p=[a2], o=cc]
result: [s=[aa], p=[a2], o=(cc)]
result: [s=[mm], p=[nn], o=Some long literal with language@en]
result: [s=[oo], p=[pp], o=12345^^<http://www.w3.org/2001/XMLSchema#int>]
: Query returned 5 results
**PASSED**:
=============================================
PASSED: 20  FAILED:  0
]]></programlisting>
          </listitem>
        </itemizedlist>
    </sect4>
      <sect4 id="rdfnativestorageprovidersredlandcp"><title>Connection Parameters</title>
        <para>The Virtuoso provider has the following connection parameters available fro use:</para>
        <itemizedlist mark="bullet">
          <listitem><emphasis>dsn</emphasis> - ODBC datasource name</listitem>
          <listitem><emphasis>user</emphasis> - user name of database server</listitem>
          <listitem><emphasis>password</emphasis> - password of database server</listitem>
          <listitem><emphasis>host</emphasis> - hostname:portno of the database server</listitem>
          <listitem><emphasis>charset</emphasis> - database charset to use</listitem>
        </itemizedlist>
        <para>NOTE: Take care exposing the password as for example, program arguments or environment
variables. The rdfproc utility can help this by reading the password from standard input. Inside programs,
one way to prevent storing the password in a string is to construct a Redland hash of the storage options
such as via librdf hash_from_string and use librdf_new_storage_with_options to create a storage. The
rdfproc utility source code demonstrates this.
        </para>
        <para>The storage name parameter given to the storage constructor librdf new_storage is used
inside the virtuoso store to allow multiple stores inside one Virtuoso database instance as parameterized
with the above options.
        </para>
        <para>This store always provides contexts; the boolean storage option contexts is not checked.</para>
        <para>Examples:</para>
<programlisting><![CDATA[
  /* A new Virtuoso store */
  storage=librdf_new_storage(world, "virtuoso", "db1",
      "dsn='Local Virtuoso',user='demo',password='demo'");

  /* A different, existing Virtuoso store in the same database as above */
  storage=librdf_new_storage(world, "virtuoso", "db2",
      "dsn='Local Virtuoso',user='demo',password='demo'");

  /* An existing Virtuoso store on a different database server */
  storage=librdf_new_storage(world, "virtuoso", "http://red3",
      "dsn='Remote Virtuoso',user='demo',password='demo'");

  /* Opening with an options hash */
  options=librdf_new_hash(world, NULL);
  librdf_hash_from_string(options,
      "dsn='Local Virtuoso',user='demo'");
  librdf_hash_put_strings(options, "password", user_password);
  storage=librdf_new_storage_with_options(world, "virtuoso", "http://red3", options);
]]></programlisting>
      </sect4>
    </sect3>
    <sect3 id="rdfnativestorageprovidersredlandref"><title>References</title>
      <itemizedlist mark="bullet">
        <listitem><ulink url="http://librdf.org/docs/api/redland-storage.html">RedLand Triple 
        Store</ulink></listitem>
        <listitem><ulink url="http://librdf.org/docs/api/redland-storage-modules.html">RedLand 
        Storage Modules</ulink></listitem>
      </itemizedlist>
    </sect3>
  </sect2>
</sect1>

<sect1 id="rdfgraphreplication"><title>RDF Graph Replication</title>
<para>The following section demonstrates how to replicate graphs from one Virtuoso 
	instance to (an)other Virtuoso instance(s), using the RDF Replication Feature.</para>
<para>Terms used in this section:</para> 
<itemizedlist mark="bullet">
  <listitem><emphasis>Host Virtuoso Instance</emphasis>, aka the publisher: the instance where we 
  will insert RDF data into a Named Graph; then create a publication of this graph.</listitem>
  <listitem><emphasis>Destination Virtuoso Instance</emphasis>, aka the subscriber: the instance 
  which will subscribe to the publication from the Host Virtuoso Instance.</listitem>
</itemizedlist>
  <tip><title>See Also:</title>
     <para><link linkend="fn_rdf_repl_start"><function>DB.DBA.RDF_REPL_START()</function></link></para>  
     <para><link linkend="fn_rdf_repl_graph_ins"><function>DB.DBA.RDF_REPL_GRAPH_INS()</function></link></para>  
     <para><link linkend="fn_rdf_repl_graph_del"><function>DB.DBA.RDF_RDF_REPL_GRAPH_DEL()</function></link></para>            
  </tip>
<para>The basic outline:</para>
<itemizedlist mark="bullet">
   <listitem>First, use the Virtuoso Conductor on a Host Virtuoso Instance to publish a named 
   	graph.</listitem>
   <listitem>Then, use the Virtuoso Conductor on a Destination Virtuoso Instance to subscribe 
   	to deltas from the published graph.</listitem>
   <listitem>Finally, see how a change in the publisher's graph will appear in the subscriber's 
   	graph.</listitem>
</itemizedlist>
  <sect2 id="rdfgraphreplicationscenr"><title>Replication Scenarios</title>
     <figure id="topo_scenario" float="1">
      <title></title>
      <graphic fileref="ui/topo-scenario.png"/>
    </figure> 
    <sect3 id="rdfgraphreplicationscenrint"><title>Introduction</title>
      <para>In this section we will examine a proposed setup for a back-end server called MASTER which 
      	publishes a number of graphs to a set of front-end machines called FARM-1 .. FARM-n and discuss 
      	a couple of common scenarios like adding an extra machine to the farm, or replacing a broken 
      	instance of MASTER.</para>    
      <para>In this example we will assume each virtuoso instance running on its own machine, so they can 
      	use the same port numbers for both the main server (default 1111) as well as the http port 
      	(default 8890) as each machine has an unique IP addresses. In the example we use MASTER-IP and 
      	FARM-x-IP which should be replaced by either the real IP address or the DNS name of the machine 
      	in question.</para>      
      <para>Since there will be a reverse-proxy service in front of the farm, all virtuoso instances 
      	should have the URIQA Default host set to the outside name for this service. In this example 
      	we will use http://test.example.com as the web service we are trying to setup.</para>            
    </sect3> 
    <sect3 id="rdfgraphreplicationscenrsup"><title>Setup</title>
      <sect4 id="rdfgraphreplicationscenrsupinstv"><title>Installing Virtuoso</title>
        <para>All machines in this setup should be installed with similar installation paths like:</para>
        <itemizedlist mark="bullet">
          <listitem>/opt/virtuoso</listitem>
          <listitem>/dbs/virtuoso</listitem>
          <listitem>/virtuoso</listitem>
          <listitem>...</listitem>          
        </itemizedlist>        
        <para>The partition should be big enough to have room for the Virtuoso binaries and libraries, 
        	the transaction logs, backups and, if you do not want to use the striping feature of Virtuoso, 
        	it will need to have room for the main database files as well.</para>
        <para>Here are the quick installation steps:</para>                
        <orderedlist>
          <listitem>Login as root.</listitem>
          <listitem>Create local user called virtuoso using the chosen installation path as home 
          	direcotory.</listitem>
          <listitem>Login as virtuoso.</listitem>
          <listitem>Extract virtuoso-universal-server-6.1.tar in home directory.</listitem>
          <listitem>Run sh install.sh to install Virtuoso.</listitem>
          <listitem>Remove the file install.sh virtuoso-universal-server-6.1.tar virtuoso-server.taz if 
          	not otherwise needed.</listitem>
          <listitem>Run bin/virtuoso-stop.sh to shutdown this Virtuoso instance.</listitem>
          <listitem>Install virtuoso.lic for this system in $HOME/bin directory.</listitem>
        </orderedlist>        
        <para>As the replication process needs to make an ODBC connection to the MASTER machine, all 
        	machines should have the following information in the $HOME/bin/odbc.ini:</para>
<programlisting><![CDATA[
[ODBC Data Sources]
..
MASTER_DSN = OpenLink Virtuoso

..
[MASTER_DSN]
Driver = OpenLink Virtuoso
Address = MASTER_IP:1111	
]]></programlisting>        	        
      </sect4>    
      <sect4 id="rdfgraphreplicationscenrsupinstm"><title>Setting up MASTER</title>
        <para>The MASTER machine is the back-end server machine. Various applications feed SPARQL data 
        	into this machine it publishes a set of graphs using RDF Replication.</para>      
        <para>The MASTER machine should ideally be equipped with multiple redundant disks in RAID-1 
        	or RAID-6 mode to minimize the risk that a single bad disk takes down the system. From a 
        	Virtuoso point of view we will use a combination of online backups combined with checkpoint 
        	audit trail to backup the content of the database in a safe way. The online backups, the 
        	checkpoint audit trail as well as the replication logs can also be copied to secondary 
        	storage using the rsync command and can be easily scripted as a cron job.</para>   
        <para>Changes to database/virtuoso.ini:</para>   
<programlisting><![CDATA[
...
[Parameters]
SchedulerInterval    = 1     ; run the internal scheduler every minute
CheckpointAuditTrail = 1     ; enable audit trail on transaction logs
CheckpointInterval   = 60    ; perform an automated checkpoint every 60 minutes
...
[URIQA]
DefaultHost = test.example.com
...  
[Replication]
ServerName   = MASTER
ServerEnable = 1 
QueueMax     = 5000000
...
]]></programlisting>        
        <para>Once the MASTER is started using the bin/virtuoso-start.sh script we must enable RDF 
        	replication before we start add data to the graphs we wish to replicate, so every record is 
        	accounted for by the replication process. If there is existing data in the graphs to be 
        	published, then this data would need to be added to a subscriber manually since the 
        	replication process creates a delta set of changes since publishing was enabled.</para>   
        <para>To enable publishing of the graph we use the isql program to connect to the MASTER 
        	instance:</para>            
<programlisting><![CDATA[
$ isql MASTER-IP:1111

-- and run the following commands:

-- enable this instance as a publisher
rdf_repl_start();

-- add graphs to replication list
rdf_repl_graph_ins('http://test.example.com');
]]></programlisting>        	
        <para>Next we create a backup directory inside the database directory and setup the online 
        	backup, again using the isql program:</para>   
<programlisting><![CDATA[
$ cd database
$ mkdir backup
$ isql MASTER_IP:1111

-- and run the following commands:

-- clear any previous context
backup_context_clear();

-- start the backup
backup_online ('bkup-#', 1000000, 0, vector ('backup'));

]]></programlisting>
        <para>The following files can now be backed up using rsync or similar tool to another machine:</para> 
<table><title>Files that can be backed up using rsync or similar tool to another machine</title>
<tgroup cols="2">
<thead><row>
<entry>Files</entry><entry>Description</entry>
</row></thead>
<tbody>
<row><entry>database/backup/*.bp</entry><entry>	the incremental backup files</entry></row>
<row><entry>database/virtuoso.trx</entry><entry>	the main transaction log containing the most recent updates to the database that have not been checkpointed into the database</entry></row>
<row><entry>database/virtuosoTIMESTAMP.trx</entry><entry>all the previous transaction logs which can be used to reconstruct the database</entry></row>
<row><entry>database/__rdf_repl*.log</entry><entry>all the replication logs containing the changes to the published graph</entry></row>
</tbody>
</tgroup>
</table>        
        <para>NOTE: Since the database is constantly modified during operation, it is of NO use to 
        	backup the virtuoso.db using an rsync script unless the virtuoso instance was shutdown 
        	beforehand, or certain extra precautions are taken which we will explain later on.</para>   
      </sect4>     
      <sect4 id="rdfgraphreplicationscenrsupinstp"><title>Setup SPARE master</title>
        <para>The SPARE machine is a replica of the MASTER machine. This machine subscribes to the 
        	publication of the MASTER to keep an exact match of the RDF graphs, but also publishes 
        	this data without any initial subscribers.</para>      
        <para>The SPARE machine should ideally be equipped similar to the MASTER machine, with multiple 
        	redundant disks in RAID-1 or RAID-6 mode to minimize the risk that a single bad disk takes down 
        	the system. From a Virtuoso point of view we will use a combination of online backups combined 
        	with checkpoint audit trail to backup the content of the database in a safe way. The online 
        	backups, the checkpoint audit trail as well as the replication logs can also be copied to 
        	secondary storage using the rsync command and can be easily scripted as a cron job.</para>   
        <para>Changes to database/virtuoso.ini:</para>                                                
<programlisting><![CDATA[
...
[Parameters]
SchedulerInterval    = 1     ; run the internal scheduler every minute
CheckpointAuditTrail = 1     ; enable audit trail on transaction logs
CheckpointInterval   = 60    ; perform an automated checkpoint every 60 minutes
...
[URIQA]
DefaultHost = test.example.com
...  
[Replication]
ServerName   = SPARE
ServerEnable = 1 
QueueMax     = 5000000
...
]]></programlisting>        
        <para>We must enable RDF replication before we start add data to the graphs we wish to 
        	replicate, so every record is accounted for by the replication process. If there is 
        	existing data in the graphs to be published, then this data would need to be added 
        	to a subscriber manually since the replication process creates a delta set of changes 
        	since publishing was enabled.</para>                                                
        <para>To enable publishing of the graph, as well as subscribing to the MASTER, we first start 
        	up this Virtuoso instance with bin/virtuoso-start.sh and then use the isql program to connect 
        	to the SPARE instance:</para>                                                
<programlisting><![CDATA[
$ bin/virtuoso-start.sh
$ isql SPARE-IP:1111

-- and run the following commands:

-- enable this instance as a publisher
rdf_repl_start();

-- add graphs to replication list
rdf_repl_graph_ins('http://test.example.com');

-- connect to master
repl_server ('MASTER', 'MASTER_DSN');

-- start subscribing to __rdf_repl
repl_subscribe ('MASTER', '__rdf_repl', 'dav', 'dav', 'dba', 'dba');

-- start initial replication
repl_sync_all ();

-- add subscription to scheduler
DB.DBA.SUB_SCHEDULE ('MASTER', '__rdf_repl', 1);
]]></programlisting>        	
        <para>Next we create a backup directory inside the database directory and setup the online backup, 
        	again using the isql program:</para>    
<programlisting><![CDATA[
$ cd database
$ mkdir backup
$ isql SPARE_IP:1111

--and run the following commands:

-- clear any previous context
backup_context_clear();

-- start the backup
backup_online ('bkup-#', 1000000, 0, vector ('backup'));
]]></programlisting>        	                                            
        <para>The following files can now be backed up using rsync or similar tool to another machine:</para> 
<table><title>Files that can be backed up using rsync or similar tool to another machine</title>
<tgroup cols="2">
<thead><row>
<entry>Files</entry><entry>Description</entry>
</row></thead>
<tbody>
<row><entry>database/backup/*.bp</entry><entry>	the incremental backup files</entry></row>
<row><entry>database/virtuoso.trx</entry><entry>the main transaction log containing the most recent updates to the database that have not been checkpointed into the database</entry></row>
<row><entry>database/virtuosoTIMESTAMP.trx</entry><entry>all the previous transaction logs which can be used to reconstruct the database</entry></row>
<row><entry>database/__rdf_repl*.log</entry><entry>all the replication logs containing the changes to the published graph</entry></row>
</tbody>
</tgroup>
</table>          
        <para>Note: Since the database is constantly modified during operation, it is of NO use to 
        	backup the virtuoso.db using an rsync script unless the virtuoso instance was shutdown 
        	beforehand, or certain extra precautions are taken which we will explain later on.</para>
      </sect4>      
      <sect4 id="rdfgraphreplicationscenrsupinstpf"><title>Setup FARM-1</title>
        <para>The FARM-1 machine is the first front-end server machine. It subscribes to the publication 
        	of the MASTER instance to keep up-to-date.</para>   
        <para>The FARM-1 machine can be run on simpler hardware than the MASTER instance.It does not 
        	require the same level of redundancy in terms of hard disks etc, as there are a number of 
        	these machines running in parallel each capable of returning results to the proxy. If one 
        	FARM machine dies, it can simply be taken from the reverse-proxy list, repaired or replaced 
        	with a fresh machine before it is added to the list of servers in the reverse proxy. As such 
        	it does not need to be backed up separately, although we could make a backup of this 
        	installation to quickly install the rest of the identical FARM boxes.</para>
        <para>Change the database/virtuoso.ini file:</para>
<programlisting><![CDATA[
...
[Parameters]
SchedulerInterval    = 1     ; run the internal scheduler every minute
CheckpointAuditTrail = 0     ; disable audit trail on transaction logs
CheckpointInterval   = 60    ; perform an automated checkpoint every 60 minutes
...
[URIQA]
DefaultHost = test.example.com
...  
[Replication]
ServerName   = FARM-1        ; each FARM machine needs to have a unique replication name
ServerEnable = 1 
QueueMax     = 5000000
...
]]></programlisting>        
        <para>Next we start up the Virtuoso instance using the bin/virtuoso-start.sh command and 
        	use the isql program to subscribe to the MASTER:</para>            
<programlisting><![CDATA[
$ bin/virtuoso-start.sh
$ isql FARM-1-IP:1111

-- connect to master
repl_server ('MASTER', 'MASTER_DSN');

-- start subscribing to __rdf_repl
repl_subscribe ('MASTER', '__rdf_repl', 'dav', 'dav', 'dba', 'dba');

-- start initial replication
repl_sync_all ();

-- add subscription to scheduler
DB.DBA.SUB_SCHEDULE ('MASTER', '__rdf_repl', 1);
]]></programlisting>        	
        <para>At this point we can shutdown this Virtuoso instance using the bin/virtuoso-stop.sh 
        	command and make a copy of the whole virtuoso installation as a blueprint to copy to 
        	another FARM-x machine.</para>            
      </sect4>    
      <sect4 id="rdfgraphreplicationscenrsupinstpfs"><title>Setup FARM-2 from scratch</title>
        <para>We can repeat the same steps we did for the FARM-1 machine, and just make sure we use 
        	FARM-2 as the replication name in the database/virtuoso.ini file and use FARM-2-IP:1111 as 
        	an argument to the isql program.</para>      
        <para>Change bin/virtuoso.ini:</para>            
<programlisting><![CDATA[
[Replication]
ServerName = FARM-2	
]]></programlisting>        
      </sect4>
      <sect4 id="rdfgraphreplicationscenrsupinstpfsi"><title>Setup FARM-3 using blueprint from FARM-1 installation</title>
        <para>Extract the tarred/zipped copy of the installation made at the end of the setup of FARM-1.</para>  
        <para>Before starting up the instance, we only need to give this instance a unique name for 
        	replication:</para>            
        <para>Change bin/virtuoso.ini:</para>            
<programlisting><![CDATA[
[Replication]
ServerName = FARM-3	
]]></programlisting>        
        <para>Next we start up the Virtuoso instance using the bin/virtuoso-start.sh command and since 
        	the subscription records and schedule are already performed in the previous step, we just 
        	use the isql program to perform a sync against the MASTER:</para>                                                            
<programlisting><![CDATA[
$ bin/virtuoso-start.sh
$ isql FARM-3-IP:1111

-- change replication name
DB.DBA.REPL_SERVER_RENAME ('FARM-1', 'FARM-3')

-- sync against master
repl_sync_all();
]]></programlisting>        	
      </sect4>     
      <sect4 id="rdfgraphreplicationscenrsupinstpfsc"><title>Setup FARM-4 using clone of FARM-1</title>
        <para>If the system has been running for some time, it may not be practical to do a replication 
        	from start, so there is an alternative way to setup a new FARM-4 machine.</para>      
        <para>We can either restore the blue-print backup we make at the end of FARM-1 installation, 
        	or we do a fresh installation of virtuoso on the FARM-4 machine.</para>              
        <para>In both cases we shutdown the virtuoso instance and remove the database, as we are going 
        	to replace this.</para>      
<programlisting><![CDATA[
$ bin/virtuoso-stop.sh
$ cd database
$ rm virtuoso.db virtuoso.trx virtuoso.log virtuoso.pxa	
]]></programlisting>        	
        <para>Change the database/virtuoso.ini file:</para>      
<programlisting><![CDATA[
...
[Parameters]
SchedulerInterval    = 1     ; run the internal scheduler every minute
CheckpointAuditTrail = 0     ; disable audit trail on transaction logs
CheckpointInterval   = 60    ; perform an automated checkpoint every 60 minutes
...
[URIQA]
DefaultHost = test.example.com
...  
[Replication]
ServerName   = FARM-4        ; each FARM machine needs to have a unique replication name
ServerEnable = 1 
QueueMax     = 5000000
...
]]></programlisting>        
        <para>Next we are going to temporarily disable checkpointing on FARM-1 machine so 
        	we can copy its database without risking corruption: </para>            
<programlisting><![CDATA[
$ isql FARM-1-IP:1111

-- disable automatic checkpointing
checkpoint_interval (-1);

-- and do an explicit checkpoint
checkpoint;	
]]></programlisting>        	
        <para>It is now safe to copy the database across using the rsync command:</para> 
<programlisting><![CDATA[
$ rsync -avz virtuoso@FARM-1-IP:/path/to/virtuoso/database/virtuoso.db database/virtuoso.db
]]></programlisting>        
        <para>Next we re-enable checkpoint interval on FARM-1:</para> 
<programlisting><![CDATA[
$ isql FARM-1-IP:1111

-- re-enable checkpointing
checkpoint_interval(60);	
]]></programlisting>        
        <para>The last step is to start the database:</para>                                           
<programlisting><![CDATA[
$ bin/virtuoso-start.sh
$ isql FARM-4-IP:1111

-- change replication name
DB.DBA.REPL_SERVER_RENAME ('FARM-1', 'FARM-4')

-- sync against master
repl_sync_all();	
]]></programlisting>
    </sect4>
    </sect3>
</sect2>  
  <sect2 id="rdfgraphreplicationtopl"><title>Replication Topologies</title>
<para>Typical replication topologies are Chains, Stars and Bi-directional. They can be achieved with 
	Virtuoso, by repeating the "Publish" and/or "Subscribe" steps on each relevant node.</para>
	  <sect3 id="rdfgraphreplicationtoplstar"><title>Star Replication Topology</title> 
	     <para>In a Star, there is one Publisher, and many Subscribers.</para>
	     <figure id="star1" float="1">
         <title>Star Replication Topology</title>
         <graphic fileref="ui/topo-star.png"/>
    </figure>
       <para>To set up a Star, follow the scenario:</para>
    <orderedlist>
         <listitem>Configure Instance #1 to Publish.</listitem>
         <listitem>Configure Instance #2 to Subscribe to #1.</listitem>
         <listitem>Repeat as necessary.</listitem>
       </orderedlist>
       <sect4 id="rdfgraphreplicationtoplstarex"><title>Star Replication Topology Example</title> 
         <para>The following How-To walks you through setting up Virtuoso RDF Graph Replication in a Star Topology.</para>
         <sect5 id="rdfgraphreplicationtoplstarexpr"><title>Prerequisites</title> 
           <sect6 id="rdfgraphreplicationtoplstarexprini"><title>Database INI Parameters</title> 
             <para>Suppose there are 3 Virtuoso instances respectively with the following ini parameters values:</para>
             <orderedlist>
               <listitem>virtuoso1.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso1.db
TransactionFile = virtuoso1.trx
ErrorLogFile     = virtuoso1.log
...
[Parameters]
ServerPort               = 1111
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8891
...
[URIQA]
DefaultHost = example.com:8891
...
[Replication]
ServerName   = db1
...
	
]]></programlisting>
      </listitem>
               <listitem>virtuoso2.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso2.db
TransactionFile = virtuoso2.trx
ErrorLogFile     = virtuoso2.log
...
[Parameters]
ServerPort               = 1112
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8892
...
[URIQA]
DefaultHost = localhost:8892
...
[Replication]
ServerName   = db2
...	
]]></programlisting>
               </listitem>             	
               <listitem>virtuoso3.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso3.db
TransactionFile = virtuoso3.trx
ErrorLogFile     = virtuoso3.log
...
[Parameters]
ServerPort               = 1113
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8893
...
[URIQA]
DefaultHost = example.com:8893
...
[Replication]
ServerName   = db3
...
]]></programlisting>
      </listitem>
    </orderedlist>
           </sect6> 
           <sect6 id="rdfgraphreplicationtoplstarexprdsn"><title>Database DSNs</title> 
             <para>Use the ODBC Administrator on your Virtuoso host (e.g., on Windows, Start menu -> Control Panel -> Administrative Tools -> Data Sources (ODBC); on Mac OS X, /Applications/Utilities/OpenLink ODBC Administrator.app) to create a System DSN for each of db1, db2, db3, with names db1, db2 and db3, respectively.</para>
           </sect6>   
           <sect6 id="rdfgraphreplicationtoplstarexprcnd"><title>Install Conductor package</title> 
             <para>On each of the 3 Virtuoso instances install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> package.</para>
           </sect6>                            
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplstarexprph"><title>Create a Publication on the Host Virtuoso Instance db1</title> 
    <orderedlist>
   <listitem>Go to Conductor -> Replication -> Transactional -> Publications 
     <figure id="star2" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r6.png"/>
     </figure>
  </listitem>
   <listitem>Click Enable RDF Publishing</listitem>
    <listitem>A publication with the name RDF Publication should be created: 
     <figure id="star3" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r7.png"/>
     </figure>
  </listitem>
    <listitem>Click the link which is the publication name.</listitem>
    <listitem>You will be shown the publication items page: 
     <figure id="star4" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r8.png"/>
     </figure>
  </listitem>
    <listitem>Enter for Graph IRI:
<programlisting><![CDATA[
http://example.org	
]]></programlisting>    	 
     <figure id="star5" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r9.png"/>
     </figure>
  </listitem>
    <listitem>Click Add New</listitem>
    <listitem>The item will be created and shown in the list of items for the currently viewed publication. 
     <figure id="star6" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r10.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplstarexprih"><title>Insert Data into a Named Graph on the Host 
         Virtuoso Instance</title> 
           <para>There are several ways to insert data into a Virtuoso Named Graph. In this example, we 
           	will use the Virtuoso Conductor's Import RDF feature:</para>
<orderedlist>
  <listitem>In the Virtuoso Conductor, go to Linked Data -&gt; Quad Store Upload:
     <figure id="star7" float="1">
       <title>Replication Topology</title>
       <graphic fileref="ui/uc1.png"/>
     </figure>
  </listitem> 
  <listitem>In the form: 
    <itemizedlist mark="bullet">
  <listitem>Tick the box for Resource URL and enter your resource URL, for e.g.: 
<programlisting><![CDATA[
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this	
]]></programlisting>  	
  </listitem>
  <listitem>Enter for Named Graph IRI:
<programlisting><![CDATA[
http://example.org	
]]></programlisting>  	
</listitem>
    </itemizedlist>
     <figure id="star8" float="1">
       <title>Replication Topology</title>
       <graphic fileref="ui/r2.png"/>
     </figure>
      </listitem>
  <listitem>Click Upload</listitem> 
  <listitem>A successful upload will result in this message: 
     <figure id="star9" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r3.png"/>
     </figure>
  </listitem> 
  <listitem>Check the inserted triples by executing a query like the following against the SPARQL endpoint, http://cname:port/sparql:
<programlisting><![CDATA[
SELECT * 
  FROM <http://example.org>
 WHERE { ?s ?p ?o }
]]></programlisting>
     <figure id="star10" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r4.png"/>
     </figure>
      </listitem>
  <listitem>See how many triples have been inserted in your graph: 
<programlisting><![CDATA[
SELECT COUNT(*) 
  FROM <http://example.org>
 WHERE { ?s ?p ?o }	
]]></programlisting>  	
     <figure id="star11" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r5.png"/>
     </figure>
  </listitem> 
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplstarexprsp"><title>Subscribe to the Publication on the a 
         Destination Virtuoso Instance db2, db3, etc.</title> 
<orderedlist>
  <listitem>Go to Conductor -> Replication -> Transactional -> Subscriptions 
     <figure id="star12" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r11.png"/>
     </figure>
  </listitem> 
  <listitem>Click New Subscription 
     <figure id="star13" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r12.png"/>
     </figure>
  </listitem> 
  <listitem>Specify a new Data Source Enter or selected target data source from the available connected Data Sources: 
     <figure id="star14" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r13.png"/>
     </figure>
     <figure id="star15" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r13a.png"/>
     </figure>
  </listitem> 
  <listitem>Click Publications list
     <figure id="star16" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r15.png"/>
     </figure>
  </listitem> 
  <listitem>Select the RDF Publication and click List Items 
     <figure id="star17" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r16.png"/>
     </figure>
  </listitem> 
  <listitem>Click Subscribe</listitem> 
  <listitem>The subscription will be created 
     <figure id="star18" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r18.png"/>
     </figure>
  </listitem> 
  <listitem>Click Sync</listitem>       
  <listitem>Check the retrieved triples by executing the following query 
<programlisting><![CDATA[
SELECT * 
  FROM <http://example.org>
 WHERE {?s ?p ?o}	
]]></programlisting>  	
     <figure id="star19" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r19.png"/>
     </figure>
  </listitem>
  <listitem>See how many triples have been inserted into your graph by executing the following query: 
<programlisting><![CDATA[
SELECT COUNT(*) 
  FROM <http://example.org>
 WHERE {?s ?p ?o}	
]]></programlisting>  	
     <figure id="star20" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r5.png"/>
     </figure>
  </listitem>  
</orderedlist>   
<para>These steps may be repeated for any number of Subscriber.</para>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplstarexprch"><title>Insert Triples into the Host Virtuoso 
         Instance Graph and check availability at Destination Virtuoso Instance Graph</title> 
<orderedlist>
  <listitem>To check the starting count, on the Destination Virtuoso Instance SPARQL Endpoint, execute:
<programlisting><![CDATA[
SELECT COUNT(*) 
  FROM <http://example.org>
 WHERE { ?s ?p ?o }	
]]></programlisting>  	
  </listitem>
  <listitem>On the Host Virtuoso Instance go to Conductor -> Database -> Interactive SQL and execute the following statement:
<programlisting><![CDATA[
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
     <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
     <http://xmlns.com/foaf/0.1/interest>
     <http://dbpedia.org/resource/Web_Services> 
  } ;
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
    <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>  	
    <http://xmlns.com/foaf/0.1/interest>  	
    <http://dbpedia.org/resource/Web_Clients> 
  } ;
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
    <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>  	
    <http://xmlns.com/foaf/0.1/interest>  	
    <http://dbpedia.org/resource/SPARQL> 
  } ;	
]]></programlisting> 
     <figure id="star21" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r22.png"/>
     </figure> 	
     <figure id="star22" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r23.png"/>
     </figure>     
  </listitem>
  <listitem>To confirm that the triple count has increased by the number of inserted triples, execute the following on the Destination Virtuoso Instance SPARQL Endpoint: 
<programlisting><![CDATA[
SELECT COUNT(*) 
  FROM <http://example.org>
 WHERE { ?s ?p ?o }	
]]></programlisting>
     <figure id="star23" float="1">
       <title>Star Replication Topology</title>
       <graphic fileref="ui/r24.png"/>
     </figure> 	
      </listitem>
    </orderedlist>
         </sect5>                                     
    </sect4>
    </sect3>
	  <sect3 id="rdfgraphreplicationtoplchain"><title>Chain Replication Topology</title> 
      <para>In a Chain, there is one original Publisher, to which there is only one Subscriber. That 
      	Subscriber may also serve as a Publisher, again with only one Subscriber. The chain ends with 
      	a Subscriber which does not Publish.</para>	  
     <figure id="chain1" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/topo-chain.png"/>
     </figure>   
     <para>To set up a Chain, follow the scenario:</para>   	
<orderedlist>
   <listitem>Configure Instance #1 to Publish.</listitem>
   <listitem>Configure Instance #2 to Subscribe to #1.</listitem>
   <listitem>Configure Instance #2 to Publish.</listitem>
   <listitem>Configure Instance #3 to Subscribe to #2.</listitem>
   <listitem>Repeat as necessary.</listitem>
</orderedlist>
       <sect4 id="rdfgraphreplicationtoplchainex"><title>Chain Replication Topology Example</title> 
         <para>The following How-To walks you through setting up Virtuoso RDF Graph Replication in a 
         	Chain Topology.</para>
         <sect5 id="rdfgraphreplicationtoplchainexpr"><title>Prerequisites</title> 
           <sect6 id="rdfgraphreplicationtoplchainexprini"><title>Database INI Parameters</title> 
<para>Suppose there are 3 Virtuoso instances respectively with the following ini parameters values:</para>
             <orderedlist>
               <listitem>virtuoso1.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso1.db
TransactionFile = virtuoso1.trx
ErrorLogFile     = virtuoso1.log
...
[Parameters]
ServerPort               = 1111
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8891
...
[URIQA]
DefaultHost = example.com:8891
...
[Replication]
ServerName   = db1
...

]]></programlisting>
               </listitem>
               <listitem>virtuoso2.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso2.db
TransactionFile = virtuoso2.trx
ErrorLogFile     = virtuoso2.log
...
[Parameters]
ServerPort               = 1112
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8892
...
[URIQA]
DefaultHost = localhost:8892
...
[Replication]
ServerName   = db2
...	
]]></programlisting>
               </listitem>             	
               <listitem>virtuoso3.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso3.db
TransactionFile = virtuoso3.trx
ErrorLogFile     = virtuoso3.log
...
[Parameters]
ServerPort               = 1113
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8893
...
[URIQA]
DefaultHost = example.com:8893
...
[Replication]
ServerName   = db3
...
]]></programlisting>
               </listitem>               	
             </orderedlist>           
           </sect6> 
           <sect6 id="rdfgraphreplicationtoplchainexprdsn"><title>Database DSNs</title> 
             <para>Use the ODBC Administrator on your Virtuoso host (e.g., on Windows, Start menu -> Control Panel -> Administrative Tools -> Data Sources (ODBC); on Mac OS X, /Applications/Utilities/OpenLink ODBC Administrator.app) to create a System DSN for each of db1, db2, db3, with names db1, db2 and db3, respectively.</para>           
           </sect6>   
           <sect6 id="rdfgraphreplicationtoplchainexprcnd"><title>Install Conductor package</title> 
             <para>On each of the 3 Virtuoso instances install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> package.</para>           
           </sect6>                            
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplchainexpr"><title>Create Publication on db1</title> 
<orderedlist>
  <listitem>Go to http://example.com:8891/conductor and log in as dba</listitem>
  <listitem>Go to Conductor - &gt; Replication - &gt; Transactional - &gt; Publications
     <figure id="chain2" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m1.png"/>
     </figure>
  </listitem>    
  <listitem>Click <emphasis>Enable RDF Publishing</emphasis></listitem> 
  <listitem>As result publication with the name <emphasis>RDF Publication</emphasis> should be created
     <figure id="chain3" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m2.png"/>
     </figure>
   </listitem>   
  <listitem>Click the link which is the publication name. </listitem> 
  <listitem>You will be shown the publication items page
     <figure id="chain4" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m3.png"/>
     </figure>
  </listitem>    
  <listitem>Enter for Graph IRI:
<programlisting><![CDATA[
http://example.org
]]></programlisting>
     <figure id="chain5" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m4.png"/>
     </figure>
  </listitem>    
  <listitem>Click Add New</listitem> 
  <listitem>The item will be created and shown in the list of items for the currently viewed publication. 
     <figure id="chain6" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m5.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplchainexpr"><title>Create subscription from db2 to db1's Publication</title> 
<orderedlist>
  <listitem>Log in at http://example.com:8892/conductor</listitem>
  <listitem>Go to Replication - &gt; Transactional - &gt; Subscriptions 
     <figure id="chain7" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m6.png"/>
     </figure>
  </listitem>   
  <listitem>Click <emphasis>New Subscription</emphasis>
     <figure id="chain8" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m7.png"/>
     </figure>
  </listitem>      
  <listitem>From the list of "Specify new data source" select Data Source db1
     <figure id="chain9" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m8.png"/>
     </figure>
  </listitem>    
  <listitem>Enter for db1 dba user credentials
     <figure id="chain10" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m9.png"/>
     </figure>
  </listitem>    
  <listitem>Click "Add Data Source"</listitem> 
  <listitem>As result <emphasis>db1</emphasis> will be shown in the "Connected Data Sources" list. 
     <figure id="chain11" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m10.png"/>
     </figure>
  </listitem>    
  <listitem>Select <emphasis>db1</emphasis> the "Connected Data Sources" list and click "Publications list"
     <figure id="chain12" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m11.png"/>
     </figure>
  </listitem>    
  <listitem>As result will be shown the list of available publications for the selected data source. Select the one with name "RDF Publication" and click "List Items".
     <figure id="chain13" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m12.png"/>
     </figure>
  </listitem>    
  <listitem>As result will be shown the "Confirm subscription" page. 
     <figure id="chain14" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m13.png"/>
     </figure>
  </listitem>    
  <listitem>The sync interval by default is 10 minutes. For the testing purposes, we will change it to 1 minute.
     <figure id="chain15" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m14.png"/>
     </figure>
  </listitem>    
  <listitem>Click "Subscribe"</listitem> 
  <listitem>The subscription will be created. 
     <figure id="chain16" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m15.png"/>
     </figure>
  </listitem>   
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplchainexpr"><title>Create Publication on db2</title> 
         <orderedlist>
  <listitem>Go to http://example.com:8892/conductor and log in as dba</listitem>
  <listitem>Go to Conductor - &gt; Replication - &gt; Transactional - &gt; Publications
     <figure id="chain17" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m16.png"/>
     </figure>
  </listitem>   
  <listitem>Click <emphasis>Enable RDF Publishing</emphasis></listitem> 
  <listitem>As result publication with the name <emphasis>RDF Publication</emphasis> should be created
     <figure id="chain18" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m17.png"/>
     </figure>
  </listitem>    
  <listitem>Click the link which is the publication name.</listitem>   
  <listitem>You will be shown the publication items page
     <figure id="chain19" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m18.png"/>
     </figure>
  </listitem>    
  <listitem>Enter for Graph IRI:
<programlisting><![CDATA[
http://example.org
]]></programlisting>
     <figure id="chain20" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m19.png"/>
     </figure>
  </listitem>   
  <listitem>Click Add New</listitem> 
  <listitem>The item will be created and shown in the list of items for the currently viewed publication. 
     <figure id="chain21" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m20.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplchainexpr"><title>Create subscription from db3 to db2's Publication</title> 
         <orderedlist>
  <listitem>Log in at http://example.com:8893/conductor</listitem>
  <listitem>Go to Replication - &gt; Transactional - &gt; Subscriptions 
     <figure id="chain22" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m21.png"/>
     </figure>
  </listitem>    
  <listitem>Click <emphasis>New Subscription</emphasis>
     <figure id="chain23" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m22.png"/>
     </figure>
  </listitem>    
  <listitem>From the list of "Specify new data source" select Data Source db2
     <figure id="chain24" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m23.png"/>
     </figure>
  </listitem>    
  <listitem>Enter for db2 dba user credentials
     <figure id="chain25" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m24.png"/>
     </figure>
  </listitem>    
  <listitem>Click "Add Data Source"
     <figure id="chain26" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m25.png"/>
     </figure>
  </listitem>    
  <listitem>As result <emphasis>db2</emphasis> will be shown in the "Connected Data Sources" list. Select it and click "Publications list"
     <figure id="chain27" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m26.png"/>
     </figure>
  </listitem>    
  <listitem>As result will be shown the list of available publications for the selected data source. Select the one with name "RDF Publication" and click "List Items".
     <figure id="chain28" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m27.png"/>
     </figure>
  </listitem>    
  <listitem>As result will be shown the "Confirm subscription" page.  
     <figure id="chain29" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m28.png"/>
     </figure>
  </listitem>    
  <listitem>The sync interval by default is 10 minutes. For the testing purposes, we will change it to 1 minute.
     <figure id="chain30" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m29.png"/>
     </figure>
  </listitem>    
  <listitem>Click "Subscribe"</listitem>
  <listitem>The subscription will be created. 
     <figure id="chain31" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m30.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5>  
         <sect5 id="rdfgraphreplicationtoplchainind"><title>Insert Data into a Named Graph on the db1 Virtuoso Instance</title>                                    
<orderedlist>
  <listitem>Log in at http://example.com:8891/conductor</listitem>
  <listitem>Go to Linked Data -&gt; Quad Store Upload:
     <figure id="chain32" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/uc1.png"/>
     </figure>
  </listitem>    
  <listitem>In the shown form:
      <orderedlist>
     <listitem>Tick the box for <emphasis>Resource URL</emphasis> and enter your resource URL, e.g.:
<programlisting><![CDATA[
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
]]></programlisting>
     </listitem> 
     <listitem>Enter for Named Graph IRI:
<programlisting><![CDATA[
http://example.org
]]></programlisting>
     <figure id="chain33" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m32.png"/>
     </figure>
     </listitem> 
      </orderedlist>
  </listitem>     
  <listitem>Click Upload</listitem> 
  <listitem>A successful upload will result in a shown message.
     <figure id="chain34" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m33.png"/>
     </figure>
  </listitem>    
  <listitem>Check the count of the inserted triples by executing a query like the following against the SPARQL endpoint, 
http://example.com:8891/sparql:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
     <figure id="chain35" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m34.png"/>
     </figure>
  </listitem>    
  <listitem>Should return <emphasis>57</emphasis> as total.
     <figure id="chain36" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m35.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5>           
         <sect5 id="rdfgraphreplicationtoplchainexcdd"><title>Check data on the Destination instances db2 and db3</title>                                    
<orderedlist>
  <listitem>To check the starting count, on each of the Destination Virtuoso Instances db2 and db3 from  SPARQL Endpoint execute:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
  </listitem> 
  <listitem>Should return <emphasis>57</emphasis> as total.
     <figure id="chain37" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m35.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5>  
         <sect5 id="rdfgraphreplicationtoplchainexadd"><title>Add new data on db1</title>                                    
<orderedlist>
  <listitem>Disconnect db2 and db3.</listitem> 
  <listitem>On the Host Virtuoso Instance db1 go to Conductor - &gt; Database - &gt; Interactive SQL enter the following statement:
<programlisting><![CDATA[
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
     <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
     <http://xmlns.com/foaf/0.1/interest>
     <http://dbpedia.org/resource/Web_Services> 
  } ;
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
    <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>  	
    <http://xmlns.com/foaf/0.1/interest>  	
    <http://dbpedia.org/resource/Web_Clients> 
  } ;
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
    <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>  	
    <http://xmlns.com/foaf/0.1/interest>  	
    <http://dbpedia.org/resource/SPARQL> 
  } ;
]]></programlisting>
     <figure id="chain38" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m36.png"/>
     </figure>
  </listitem>    
  <listitem>Click "Execute"</listitem> 
  <listitem>As result the triples will be inserted
     <figure id="chain39" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m36a.png"/>
     </figure>
  </listitem>    
  <listitem>Check the count of the destination instance graph's triples by executing the following query like against the SPARQL endpoint, 
http://example.com:8891/sparql:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
  </listitem>  
  <listitem>Should return <emphasis>60</emphasis> as total.
     <figure id="chain40" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m38.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5>  
         <sect5 id="rdfgraphreplicationtoplchainexchki"><title>Check data on the Destination instances db2 and db3</title>                                    
         <orderedlist> 
  <listitem>Start instances db2 and db3</listitem> 
  <listitem>To confirm that the triple count has increased by the number of inserted triples, execute the following on the Destination Virtuoso Instance db2 and db3 SPARQL Endpoint:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
  </listitem>  
  <listitem>Should return <emphasis>60</emphasis> as total.
     <figure id="chain41" float="1">
       <title>Chain Replication Topology</title>
       <graphic fileref="ui/m38.png"/>
     </figure>
  </listitem>    
</orderedlist>
         </sect5>                             
    </sect4>
    </sect3>
	  <sect3 id="rdfgraphreplicationtoplbid"><title>Bi-directional Replication Topology</title> 
       <sect4 id="rdfgraphreplicationtoplbidex"><title>Bi-directional Replication Topology Example</title> 
         <para>The following How-To walks you through setting up Virtuoso RDF Graph Replication in a 
         Bi-directional Topology.</para>
<programlisting><![CDATA[
db1 <---- db2
db1 ----> db2
]]></programlisting>
         <sect5 id="rdfgraphreplicationtoplbidexpr"><title>Prerequisites</title> 
           <sect6 id="rdfgraphreplicationtoplbidexprini"><title>Database INI Parameters</title> 
<para>Suppose there are 2 Virtuoso instances respectively with the following ini parameters values:</para>
             <orderedlist>
               <listitem>virtuoso1.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso1.db
TransactionFile = virtuoso1.trx
ErrorLogFile     = virtuoso1.log
...
[Parameters]
ServerPort               = 1111
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8891
...
[URIQA]
DefaultHost = example.com:8891
...
[Replication]
ServerName   = db1
...
	
]]></programlisting>
               </listitem>
               <listitem>virtuoso2.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso2.db
TransactionFile = virtuoso2.trx
ErrorLogFile     = virtuoso2.log
...
[Parameters]
ServerPort               = 1112
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8892
...
[URIQA]
DefaultHost = localhost:8892
...
[Replication]
ServerName   = db2
...	
]]></programlisting>
               </listitem>             	
             </orderedlist>           
           </sect6> 
           <sect6 id="rdfgraphreplicationtoplbidexprdsn"><title>Database DSNs</title> 
           <para>Use the ODBC Administrator on your Virtuoso host (e.g., on Windows, Start menu -> Control Panel -> Administrative Tools -> Data Sources (ODBC); on Mac OS X, /Applications/Utilities/OpenLink ODBC Administrator.app) to create a System DSN for db1 and db2 with names db1 and db2 respectively.</para>
           </sect6>   
           <sect6 id="rdfgraphreplicationtoplbidexprcnd"><title>Install Conductor package</title> 
             <para>On each of the 2 Virtuoso instances install the <ulink url="http://s3.amazonaws.com/opldownload/uda/vad-packages/6.1/virtuoso/conductor_dav.vad">conductor_dav.vad</ulink> package.</para>
           </sect6>                            
      </sect5>
         
         <sect5 id="rdfgraphreplicationtoplbidexprcph"><title>Create Publication on db2</title> 
<orderedlist>
  <listitem>Go to http://example.com:8892/conductor and log in as dba</listitem>
  <listitem>Go to Conductor -&gt; Replication -&gt; Transactional -&gt; Publications
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd1.png"/>
     </figure>
  </listitem>   
  <listitem>Click <emphasis>Enable RDF Publishing</emphasis></listitem>
  <listitem>As result publication with the name <emphasis>RDF Publication</emphasis> should be created
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd2.png"/>
     </figure>
  </listitem>   
  <listitem>Click the link which is the publication name.</listitem> 
  <listitem>You will be shown the publication items page
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd3.png"/>
     </figure>
  </listitem>
  <listitem>Enter for Graph IRI:
<programlisting><![CDATA[
http://example.org
]]></programlisting>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd4.png"/>
     </figure>
</listitem>
  <listitem>Click Add New
  </listitem>
  <listitem>The item will be created and shown in the list of items for the currently viewed publication. 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd5.png"/>
     </figure>
  </listitem>
</orderedlist>         
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplbidexprcs"><title>Create subscription from db1 to db2's Publication</title> 
<orderedlist>
  <listitem>Log in at http://example.com:8891/conductor
  </listitem>
  <listitem>Go to Replication -&gt; Transactional -&gt; Subscriptions 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd6.png"/>
     </figure>
  </listitem>
  <listitem>Click <emphasis>New Subscription</emphasis>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd7.png"/>
     </figure>
  </listitem>
  <listitem>From the list of "Specify new data source" select Data Source db2
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd8.png"/>
     </figure>
  </listitem>
  <listitem>Enter for db2 dba user credentials
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd9.png"/>
     </figure>
  </listitem>
  <listitem>Click "Add Data Source"
  </listitem>
  <listitem>As result <emphasis>db2</emphasis> will be shown in the "Connected Data Sources" list. 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd10.png"/>
     </figure>
  </listitem>
  <listitem>Select <emphasis>db2</emphasis> the "Connected Data Sources" list and click "Publications list"
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd11.png"/>
     </figure>
  </listitem>
  <listitem>As result will be shown the list of available publications for the selected data source. Select the one with name "RDF Publication" and click "List Items".
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd12.png"/>
     </figure>
  </listitem>
  <listitem>As result will be shown the "Confirm subscription" page. 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd13.png"/>
     </figure>
  </listitem>
  <listitem>The sync interval by default is 10 minutes. For the testing purposes, we will change it to 1 minute.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd14.png"/>
     </figure>
  </listitem>
  <listitem>Click "Subscribe"
  </listitem>
  <listitem>The subscription will be created. 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd15.png"/>
     </figure>
  </listitem>
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplbidexprcpd"><title>Create Publication on db1</title> 
<orderedlist>
  <listitem>Go to http://example.com:8891/conductor and log in as dba
  </listitem>
  <listitem>Go to Conductor -&gt; Replication -&gt; Transactional -&gt; Publications
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd16.png"/>
     </figure>
  </listitem>
  <listitem>Click <emphasis>Enable RDF Publishing</emphasis>
  </listitem>
  <listitem>As result publication with the name <emphasis>RDF Publication</emphasis> should be created
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd17.png"/>
     </figure>
  </listitem>
  <listitem>Click the link which is the publication name. 
  </listitem>
  <listitem>You will be shown the publication items page
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd18.png"/>
     </figure>
  </listitem>
  <listitem>Enter for Graph IRI:
<programlisting><![CDATA[
http://example.org
]]></programlisting>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd19.png"/>
     </figure>
  </listitem>
  <listitem>Click Add New
  </listitem>
  <listitem>The item will be created and shown in the list of items for the currently viewed publication. 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd20.png"/>
     </figure>
  </listitem>
</orderedlist>
      </sect5>
         <sect5 id="rdfgraphreplicationtoplbidexprcsh"><title>Create subscription from db2 to db1's Publication</title> 
<orderedlist>
  <listitem>Log in at http://example.com:8892/conductor
  </listitem>
  <listitem>Go to Replication -&gt; Transactional -&gt; Subscriptions 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd21.png"/>
     </figure>
  </listitem>
  <listitem>Click <emphasis>New Subscription</emphasis>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd22.png"/>
     </figure>
  </listitem>
  <listitem>From the list of "Specify new data source" select Data Source db1
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd23.png"/>
     </figure>
  </listitem>
  <listitem>Enter for db1 dba user credentials
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd24.png"/>
     </figure>
  </listitem>
  <listitem>Click "Add Data Source"
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd25.png"/>
     </figure>
  </listitem>
  <listitem>As result <emphasis>db1</emphasis> will be shown in the "Connected Data Sources" list. Select it and click "Publications list"
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd26.png"/>
     </figure>
  </listitem>
  <listitem>As result will be shown the list of available publications for the selected data source. Select the one with name "RDF Publication" and click "List Items".
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd27.png"/>
     </figure>
  </listitem>
  <listitem>As result will be shown the "Confirm subscription" page.  
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd28.png"/>
     </figure>
  </listitem>
  <listitem>The sync interval by default is 10 minutes. For the testing purposes, we will change it to 1 minute.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd29.png"/>
     </figure>
  </listitem>
  <listitem>Click "Subscribe"
  </listitem>
  <listitem>The subscription will be created. 
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd30.png"/>
     </figure>
  </listitem>
</orderedlist>  
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplbidexprinsh"><title>Insert Data into a Named Graph on the db2 Virtuoso Instance</title> 
<orderedlist>
  <listitem>Log in at http://example.com:8892/conductor
  </listitem>
  <listitem>Go to Linked Data -&gt; Quad Store Upload:
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/uc1.png"/>
     </figure>
  </listitem>
  <listitem>In the shown form:
     </listitem>
  <listitem>Tick the box for <emphasis>Resource URL</emphasis> and enter your resource URL, e.g.:
<programlisting><![CDATA[
http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this
]]></programlisting>
</listitem>
  <listitem>Enter for Named Graph IRI:
<programlisting><![CDATA[
http://example.org
]]></programlisting>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd32.png"/>
     </figure>
</listitem>
  <listitem>Click Upload
  </listitem>
  <listitem>A successful upload will result in a shown message.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/m33.png"/>
     </figure>
  </listitem>
  <listitem>Check the count of the inserted triples by executing a query like the following against the SPARQL endpoint, 
http://example.com:8892/sparql:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd34.png"/>
     </figure>
  </listitem>
  <listitem>Should return <emphasis>57</emphasis> as total.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd35.png"/>
     </figure>
  </listitem>
</orderedlist>
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplbidexprcdin"><title>Check data on the Destination instance db1</title> 
<orderedlist>
  <listitem>To check the starting count, execute from db1's SPARQL Endpoint:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
  </listitem>
  <listitem>Should return <emphasis>57</emphasis> as total.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd35.png"/>
     </figure>
  </listitem>
</orderedlist>       
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplbidexprinsd"><title>Add new data on db2</title> 
<orderedlist>
  <listitem>Disconnect db1.
  </listitem>
  <listitem>On the Host Virtuoso Instance db2 go to Conductor -&gt; Database -&gt; Interactive SQL enter the following statement:
<programlisting><![CDATA[
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
     <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>
     <http://xmlns.com/foaf/0.1/interest>
     <http://dbpedia.org/resource/Web_Services> 
  } ;
]]></programlisting>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd36.png"/>
     </figure>
  </listitem>
  <listitem>Click "Execute"
  </listitem>
  <listitem>As result the triples will be inserted
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd37.png"/>
     </figure>
  </listitem>
  <listitem>Check the count of the destination instance graph's triples by executing the following query like against the SPARQL endpoint, 
http://example.com:8892/sparql:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
</listitem>
  <listitem>Should return <emphasis>58</emphasis> as total.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd38.png"/>
     </figure>
  </listitem>
</orderedlist>  
         </sect5> 
         <sect5 id="rdfgraphreplicationtoplbidexprcddi"><title>Check data on the Destination instance db1</title> 
<orderedlist> 
  <listitem>Start instance db1
  </listitem>
  <listitem>To confirm that the triple count has increased by the number of inserted triples, execute the following statement on db1's SPARQL Endpoint:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
</listitem>
  <listitem>Should return <emphasis>58</emphasis> as total.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd38.png"/>
     </figure>
  </listitem>
</orderedlist>
      </sect5>
         <sect5 id="rdfgraphreplicationtoplbidexprinsah"><title>Add new data on db1</title> 
<orderedlist>
  <listitem>Disconnect db2.
  </listitem>
  <listitem>On the Host Virtuoso Instance db1 go to Conductor -&gt; Database -&gt; Interactive SQL enter the following statement:
<programlisting><![CDATA[
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
    <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>  	
    <http://xmlns.com/foaf/0.1/interest>  	
    <http://dbpedia.org/resource/Web_Clients> 
  } ;
SPARQL INSERT INTO GRAPH <http://example.org> 
  { 
    <http://www.openlinksw.com/dataspace/person/kidehen@openlinksw.com#this>  	
    <http://xmlns.com/foaf/0.1/interest>  	
    <http://dbpedia.org/resource/SPARQL> 
  } ;
]]></programlisting>
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd39.png"/>
     </figure>
</listitem>
  <listitem>Click "Execute"
  </listitem>
  <listitem>As result the triples will be inserted
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd40.png"/>
     </figure>
  </listitem>
  <listitem>Check the count of the destination instance graph's triples by executing the following query like against the SPARQL endpoint, 
http://example.com:8891/sparql:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
  </listitem>
  <listitem>Should return <emphasis>60</emphasis> as total.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd41.png"/>
     </figure>
</listitem>
</orderedlist>
       </sect5>
         <sect5 id="rdfgraphreplicationtoplbidexprcdadi"><title>Check data on the Destination instance db2</title> 
<orderedlist> 
  <listitem>Start instance db2
  </listitem>
  <listitem>To confirm that the triple count has increased by the number of inserted triples, execute the following statement on db2's SPARQL Endpoint:
<programlisting><![CDATA[
SELECT COUNT(*) 
   FROM <http://example.org>
WHERE { ?s ?p ?o }
]]></programlisting>
          </listitem>
  <listitem>Should return <emphasis>60</emphasis> as total.
     <figure id="bid1" float="1">
        <title>Bi-directional Replication Topology</title>
        <graphic fileref="ui/bd41.png"/>
     </figure>
  </listitem>
</orderedlist>                
         </sect5> 
       </sect4> 
	  </sect3> 
  </sect2> 
  <sect2 id="rdfgraphreplicationsql"><title>Set up RDF Replication via procedure calls</title>
   <sect3 id="rdfgraphreplicationsqlex"><title>Example</title> 
         <para>The following example shows how to use SQL procedures to set up Virtuoso RDF Graph Replication in a Chain Topology.</para>
         <figure id="chain1" float="1">
           <title>Chain Replication Topology</title>
           <graphic fileref="ui/topo-chain.png"/>
         </figure> 
         <para>This can also be done <link linkend="rdfgraphreplicationtoplchainex">through the HTTP-based Virtuoso Conductor</link>.</para>
         <sect4 id="rdfgraphreplicationsqlexprx"><title>Prerequisites</title> 
           <sect5 id="rdfgraphreplicationsqlexprxini"><title>Database INI Parameters</title> 
             <para>Suppose there are 3 Virtuoso instances on the same machine.</para>
             <para>The first instance holds the master copy of the data and publishes its changes to all other instances that subscribe to this master.</para>             
             <para>The second instance subscribes to the publication of the master copy, but also publishes all of these changes to any instance that subscribes to it.</para>
             <para>The third instance only subscribes to the publication of the second instance.</para>
             <para>Each of these 3 servers need unique ports and ServerName, DefaultHost for this replication scheme to work properly. Although not needed, this example also sets separate names for the database and related files. This results in the following ini parameters values (only changes are shown, the rest can remain default):</para>
             <orderedlist>
               <listitem>repl1/virtuoso.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso1.db
TransactionFile = virtuoso1.trx
ErrorLogFile     = virtuoso1.log
...
[Parameters]
ServerPort               = 1111
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8891
...
[URIQA]
DefaultHost = example.com:8891
...
[Replication]
ServerName   = db1-r
...
]]></programlisting>
          </listitem>
               <listitem>repl2/virtuoso.ini:
<programlisting><![CDATA[
...
[Database]
DatabaseFile    = virtuoso2.db
TransactionFile = virtuoso2.trx
ErrorLogFile     = virtuoso2.log
...
[Parameters]
ServerPort               = 1112
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8892
...
[URIQA]
DefaultHost = localhost:8892
...
[Replication]
ServerName   = db2-r
...
]]></programlisting>
          </listitem>
               <listitem>repl3/virtuoso.ini:
<programlisting><![CDATA[

...
[Database]
DatabaseFile    = virtuoso3.db
TransactionFile = virtuoso3.trx
ErrorLogFile     = virtuoso3.log
...
[Parameters]
ServerPort               = 1113
SchedulerInterval        = 1
...
[HTTPServer]
ServerPort                  = 8893
...
[URIQA]
DefaultHost = example.com:8893
...
[Replication]
ServerName   = db3-r
...
]]></programlisting>
          </listitem>
             </orderedlist>
           </sect5> 
           
           <sect5 id="rdfgraphreplicationsqlexprxdsn"><title>Database DSNs</title> 
             <para>Use the ODBC Administrator on your Virtuoso host (e.g., on Windows, Start menu -> Control Panel -> Administrative Tools -> Data Sources (ODBC); on Mac OS X, /Applications/Utilities/OpenLink ODBC Administrator.app) to create a System DSN for each of db1, db2, db3, with names db1, db2 and db3, respectively.</para>
           </sect5>   
         </sect4>
         
         <sect4 id="rdfgraphreplicationsqlcrb"><title>Configure Publishers and Subscribers</title> 
<orderedlist>
   <listitem>Run the databases by starting start.sh, which has the following content:
<programlisting><![CDATA[
cd repl1
virtuoso -f &
cd ../repl2
virtuoso -f &
cd ../repl3
virtuoso -f &
cd ..	
]]></programlisting>
          </listitem>
   <listitem>Use the <emphasis>isql</emphasis> command to execute the following rep.sql file:
<programlisting><![CDATA[
--
--  connect to the first database which is only a publisher
--
set DSN=localhost:1111;
reconnect;

--
-- start publishing the graph http://test.org
---
DB.DBA.RDF_REPL_START();
DB.DBA.RDF_REPL_GRAPH_INS ('http://test.org');



--
--  connect to the second database in the chain, which is both a publisher and a subscriber
--
set DSN=localhost:1112;
reconnect;

--
--  start publishing the graph http://test.org
--
DB.DBA.RDF_REPL_START();
DB.DBA.RDF_REPL_GRAPH_INS ('http://test.org');

--
--  contact the first database 
--
repl_server ('db1-r', 'db1', 'localhost:1111');

--
--  subscribe to its RDF publication(s)
--
repl_subscribe ('db1-r', '__rdf_repl', 'dav', 'dav', 'dba', 'dba');

--
--  bring the replication service online
--
repl_sync_all();

--
--  and set scheduler to check every minute
--
DB.DBA.SUB_SCHEDULE ('db1-r', '__rdf_repl', 1);



--
--  connect to the third database in the chain, which is only a subscriber
--
set DSN=localhost:1113;
reconnect;

--
-- uncomment next 2 commands if this database should also be a publisher
--
--DB.DBA.RDF_REPL_START();
--DB.DBA.RDF_REPL_GRAPH_INS ('http://test.org');

--
--  contact second database
--
repl_server ('db2-r', 'db2', 'localhost:1112');

--
--  subscribe to its RDF publication(s)
--
repl_subscribe ('db2-r', '__rdf_repl', 'dav', 'dav', 'dba', 'dba');

--
--  bring the replication service online
--
repl_sync_all();

--
--  and set schedule to check every minute
--
DB.DBA.SUB_SCHEDULE ('db2-r', '__rdf_repl', 1);
]]></programlisting>
</listitem>
</orderedlist>
    </sect4>
    </sect3>
  </sect2>
</sect1>
</chapter>