1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>7.22. Sharding — Groonga v10.1.1-31-g1e46ba6 documentation</title>
<link rel="stylesheet" href="../_static/groonga.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/language_data.js"></script>
<link rel="shortcut icon" href="../_static/favicon.ico"/>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="7.23. Log" href="log.html" />
<link rel="prev" title="7.21. Indexing" href="indexing.html" />
</head><body>
<div class="header">
<h1 class="title">
<a id="top-link" href="../index.html">
<span class="project">groonga</span>
<span class="separator">-</span>
<span class="description">An open-source fulltext search engine and column store.</span>
</a>
</h1>
<div class="other-language-links">
<ul>
<li><a href="../../../ja/html/reference/sharding.html">日本語</a></li>
</ul>
</div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="log.html" title="7.23. Log"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="indexing.html" title="7.21. Indexing"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Groonga v10.1.1-31-g1e46ba6 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../reference.html" accesskey="U"><span class="section-number">7. </span>Reference manual</a> »</li>
<li class="nav-item nav-item-this"><a href=""><span class="section-number">7.22. </span>Sharding</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="sharding">
<h1><span class="section-number">7.22. </span>Sharding<a class="headerlink" href="#sharding" title="Permalink to this headline">¶</a></h1>
<div class="versionadded">
<p><span class="versionmodified added">New in version 5.0.0.</span></p>
</div>
<p>Groonga has <a class="reference internal" href="../limitations.html"><span class="doc">Limitations</span></a> against table size. You can’t add
268,435,455 more records in one table.</p>
<p>Groonga supports time based sharding to resolve the limitation.</p>
<p>It works in the same database. It doesn’t work with multiple
databases. It means that this sharding feature isn’t for distributing
large data to multiple hosts.</p>
<p>If you want distributed sharding feature, use <a class="reference external" href="http://mroonga.org/">Mroonga</a> or <a class="reference external" href="http://pgroonga.github.io/">PGroonga</a>. You can use sharding feature by MySQL
or PostgreSQL. You’ll be able to use <a class="reference external" href="http://droonga.org/">Droonga</a>
for distributed sharding feature soon.</p>
<div class="section" id="summary">
<h2><span class="section-number">7.22.1. </span>Summary<a class="headerlink" href="#summary" title="Permalink to this headline">¶</a></h2>
<p>Sharding is implemented in <code class="docutils literal notranslate"><span class="pre">sharding</span></code> plugin. The plugin is written
in mruby. You need to enable mruby when you build Groonga.</p>
<p>You can confirm whether your Groonga supports mruby or not by
<code class="docutils literal notranslate"><span class="pre">--version</span></code> command line argument of
<a class="reference internal" href="executables/groonga.html"><span class="doc">groonga executable file</span></a>:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>% groonga --version
groonga 5.0.5 [...,mruby,...]
configure options: <...>
</pre></div>
</div>
<p>If you find <code class="docutils literal notranslate"><span class="pre">mruby</span></code>, your Groonga supports mruby.</p>
<p><code class="docutils literal notranslate"><span class="pre">sharding</span></code> plugin provides only search commands. They have
<code class="docutils literal notranslate"><span class="pre">logical_</span></code> prefix in their command names such as
<a class="reference internal" href="commands/logical_select.html"><span class="doc">logical_select</span></a> and
<a class="reference internal" href="commands/logical_range_filter.html"><span class="doc">logical_range_filter</span></a>.</p>
<p><code class="docutils literal notranslate"><span class="pre">sharding</span></code> plugin doesn’t provide schema define commands and data
load commands yet. You need to use existing commands such as
<a class="reference internal" href="commands/table_create.html"><span class="doc">table_create</span></a>,
<a class="reference internal" href="commands/column_create.html"><span class="doc">column_create</span></a> and
<a class="reference internal" href="commands/load.html"><span class="doc">load</span></a>.</p>
<p><code class="docutils literal notranslate"><span class="pre">sharding</span></code> plugin requires some rules against table and column. You
need to follow these rules. They are described later.</p>
</div>
<div class="section" id="glossary">
<span id="sharding-glossary"></span><h2><span class="section-number">7.22.2. </span>Glossary<a class="headerlink" href="#glossary" title="Permalink to this headline">¶</a></h2>
<table class="docutils align-default">
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr class="row-odd"><th class="head"><p>Name</p></th>
<th class="head"><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p>Logical table</p></td>
<td><p>It’s a table that consists of shards. It doesn’t exist in
Groonga database. It just exists in our minds.</p></td>
</tr>
<tr class="row-odd"><td><p>Logical table name</p></td>
<td><p>The name of logical table. It’s prefix of shard names. For
example, <code class="docutils literal notranslate"><span class="pre">Logs</span></code> is a logical table name and <code class="docutils literal notranslate"><span class="pre">Logs_20150814</span></code>
and <code class="docutils literal notranslate"><span class="pre">Logs_20150815</span></code> are shard names.</p></td>
</tr>
<tr class="row-even"><td><p>Shard</p></td>
<td><p>It’s a table that has records in a day or month. One shard has
only partial records.</p>
<p>Shard name (= table name) must follow
<code class="docutils literal notranslate"><span class="pre">${LOGICAL_TABLE_NAME}_${YYYYMMDD}</span></code> format or
<code class="docutils literal notranslate"><span class="pre">${LOGICAL_TABLE_NAME}_${YYYYMM}</span></code>
format. <code class="docutils literal notranslate"><span class="pre">${LOGICAL_TABLE_NAME}</span></code> is expanded to logical table
name. <code class="docutils literal notranslate"><span class="pre">${YYYYMMDD}</span></code> is expanded to day. <code class="docutils literal notranslate"><span class="pre">${YYYYMM}</span></code> is
expanded to month.</p>
<p>For example, <code class="docutils literal notranslate"><span class="pre">Logs_20150814</span></code> is consists of <code class="docutils literal notranslate"><span class="pre">Logs</span></code> logical
name and <code class="docutils literal notranslate"><span class="pre">20150814</span></code> day.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="rules">
<span id="sharding-rules"></span><h2><span class="section-number">7.22.3. </span>Rules<a class="headerlink" href="#rules" title="Permalink to this headline">¶</a></h2>
<p>TODO</p>
</div>
<div class="section" id="commands">
<span id="sharding-commands"></span><h2><span class="section-number">7.22.4. </span>Commands<a class="headerlink" href="#commands" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><p><a class="reference internal" href="commands/logical_count.html"><span class="doc">logical_count</span></a></p></li>
<li><p><a class="reference internal" href="commands/logical_parameters.html"><span class="doc">logical_parameters</span></a></p></li>
<li><p><a class="reference internal" href="commands/logical_range_filter.html"><span class="doc">logical_range_filter</span></a></p></li>
<li><p><a class="reference internal" href="commands/logical_select.html"><span class="doc">logical_select</span></a></p></li>
<li><p><a class="reference internal" href="commands/logical_shard_list.html"><span class="doc">logical_shard_list</span></a></p></li>
<li><p><a class="reference internal" href="commands/logical_table_remove.html"><span class="doc">logical_table_remove</span></a></p></li>
</ul>
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">7.22. Sharding</a><ul>
<li><a class="reference internal" href="#summary">7.22.1. Summary</a></li>
<li><a class="reference internal" href="#glossary">7.22.2. Glossary</a></li>
<li><a class="reference internal" href="#rules">7.22.3. Rules</a></li>
<li><a class="reference internal" href="#commands">7.22.4. Commands</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="indexing.html"
title="previous chapter"><span class="section-number">7.21. </span>Indexing</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="log.html"
title="next chapter"><span class="section-number">7.23. </span>Log</a></p>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="log.html" title="7.23. Log"
>next</a> |</li>
<li class="right" >
<a href="indexing.html" title="7.21. Indexing"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Groonga v10.1.1-31-g1e46ba6 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../reference.html" ><span class="section-number">7. </span>Reference manual</a> »</li>
<li class="nav-item nav-item-this"><a href=""><span class="section-number">7.22. </span>Sharding</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2009-2021, Brazil, Inc.
</div>
</body>
</html>
|
