1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
|
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>System Architecture — Cyrus IMAP 3.10.2 documentation</title>
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
<link rel="stylesheet" href="../../_static/cyrus.css" type="text/css" />
<script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
<script src="../../_static/jquery.js"></script>
<script src="../../_static/underscore.js"></script>
<script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="../../_static/doctools.js"></script>
<script src="../../_static/sphinx_highlight.js"></script>
<script src="../../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="next" title="File & Directory Locations" href="admin/locations.html" />
<link rel="prev" title="Administrator Guide" href="admin.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../../index.html" class="icon icon-home">
Cyrus IMAP
</a>
<div class="version">
3.10.2
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Cyrus IMAP</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../download.html">Download</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../quickstart.html">Quickstart Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../overview.html">Overview</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../setup.html">Setup</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../operations.html">Operations</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="manpages/index.html">Man pages</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="admin.html">Administrator Guide</a><ul class="current">
<li class="toctree-l3 current"><a class="reference internal" href="admin.html#architecture">Architecture</a><ul class="current">
<li class="toctree-l4 current"><a class="current reference internal" href="#">System Architecture</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="admin.html#management">Management</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="faq.html">Frequently Asked Questions</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../developers.html">Developers</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../support.html">Support/Community</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Cyrus SASL</span></p>
<ul>
<li class="toctree-l1"><a class="reference external" href="http://www.cyrusimap.org/sasl">Cyrus SASL</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../../index.html">Cyrus IMAP</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="../../operations.html">Operations</a></li>
<li class="breadcrumb-item"><a href="admin.html">Administrator Guide</a></li>
<li class="breadcrumb-item active">System Architecture</li>
<li class="wy-breadcrumbs-aside">
<a href="https://github.com/cyrusimap/cyrus-imapd/blob/master/docsrc/imap/reference/architecture.rst" class="fa fa-github"> Edit on GitHub</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="system-architecture">
<span id="architecture"></span><h1>System Architecture<a class="headerlink" href="#system-architecture" title="Permalink to this heading"></a></h1>
<section id="high-level-architecture">
<h2>High Level Architecture<a class="headerlink" href="#high-level-architecture" title="Permalink to this heading"></a></h2>
<p>A birds-eye view of Cyrus.</p>
<a class="reference internal image-reference" href="../../_images/architecture.jpg"><img alt="High level architecture diagram." class="align-center" src="../../_images/architecture.jpg" style="width: 636px; height: 400px;" /></a>
<p>Mail is delivered over smtp to the <a class="reference external" href="https://en.wikipedia.org/wiki/Message_transfer_agent">MTA</a> (Message/Mail Transfer Agent).
This then is delivered to <strong>Cyrus imapd</strong> over <a class="reference external" href="https://en.wikipedia.org/wiki/Local_Mail_Transfer_Protocol">lmtp</a> (Local Mail
Transfer Protocol).</p>
<p>Cyrus processes the inbound message and makes it available to the user
over POP3, IMAP or even NNTP. Cyrus does not provide outbound SMTP
services: these are hooked back into the MTA.</p>
<p>Cyrus usually uses <strong>saslauthd</strong> (Cyrus SASL) to provide authentication
services. It is not the only way to provide authentication, but it is
the usual option.</p>
<p>Cyrus stores the mailspool, indexes and search data on disk. While
these are inherently files, their structure and function is
database-centric and should be treated as such. <em>(Do not attempt to
manually edit these files. That way lies madness.)</em> Cyrus itself
provides no inherent backup capacity: this must be configured
externally using tools best suited for your environment.</p>
<p>For administrative actions on your server - such as creating users,
editing mailbox details, etc - use <a class="reference internal" href="manpages/systemcommands/cyradm.html#std-cyrusman-cyradm-8">cyradm(8)</a>. This is a
command, not a daemon, and it communicates with cyrus imapd via the
IMAP protocol, and authenticating as an admin user.</p>
<p>For security between the user and cyrus, usually SSL is applied.</p>
</section>
<section id="cyrus-murder">
<span id="architecture-murder"></span><h2>Cyrus Murder<a class="headerlink" href="#cyrus-murder" title="Permalink to this heading"></a></h2>
<p>Cyrus Murder, or IMAP Aggregation, provides the ability to split a
common IMAP mailbox namespace(s) across multiple backends. Cyrus
murder is not <a class="reference internal" href="#architecture-replication"><span class="std std-ref">replication</span></a>; it is load
and resource sharing for performance.</p>
<a class="reference internal image-reference" href="../../_images/image2-murder.jpg"><img alt="Cyrus Murder architecture." class="align-center" src="../../_images/image2-murder.jpg" style="width: 496px; height: 416px;" /></a>
<p>Consider a set of frontends (F1..Fn) which contain no user data. They
are essentially stateless. Users access mailboxes from any frontend.
Frontends are IMAP/POP3/Sieve/LMTP proxies.</p>
<p>There is also a set of backends (B1..Bn) which manage user data
stores. Unlike the frontends, each backends is unique: each holds a
subset of data. Backends are IMAP/POP3/Sieve/LMTP servers.</p>
<p>So when a user logs in to a frontend (say F1), how does F1 know from
which backend to fetch the user's data? This is where <strong>mupdate</strong>
comes in: the <strong>mupdate</strong> master holds the mapping of mailboxes to back
ends. Any time a backend has a change to their user data (i.e. mailbox
add/delete/move/subscribe), it sends the change to the <strong>mupdate</strong>
master which then notifies all the frontends of the latest mapping.</p>
<p>A user's data is not necessarily all stored on a single backend,
either: it can be split across disk partitions or backends.</p>
<p>The mapping on mupdate goes something like this:</p>
<table class="docutils align-default">
<thead>
<tr class="row-odd"><th class="head"><p><user>.<folder></p></th>
<th class="head"><p><backend>!<disk partition></p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p>u1</p></td>
<td><p>b2!default</p></td>
</tr>
<tr class="row-odd"><td><p>u2</p></td>
<td><p>b2!default</p></td>
</tr>
<tr class="row-even"><td><p>u2.Trash</p></td>
<td><p>b1!data</p></td>
</tr>
<tr class="row-odd"><td><p>u3</p></td>
<td><p>b1!bigdisk</p></td>
</tr>
</tbody>
</table>
<p>Mupdate is multithreaded.</p>
<p>Should each datastore be configured to contain the complete set of all
data, and any frontend can also behave as a backend (and vice versa),
this is known as <strong>Unified Murder</strong>: where everything has everything.</p>
<p>See the <a class="reference internal" href="admin/murder/murder.html#murder"><span class="std std-ref">Cyrus Murder installation and admin guide</span></a>
for more information.</p>
</section>
<section id="replication">
<span id="architecture-replication"></span><h2>Replication<a class="headerlink" href="#replication" title="Permalink to this heading"></a></h2>
<p>Replication is not Aggregation or <a class="reference internal" href="#architecture-murder"><span class="std std-ref">Cyrus Murder</span></a>.
Replication provides high availability and hot backups. It is designed to
replicate the mailstore on a standalone Cyrus install, or multiple backend
servers in a <a class="reference internal" href="#architecture-murder"><span class="std std-ref">murder</span></a> configuration. It does
not replicate mupdate master servers: frontends have no state to replicate.</p>
<a class="reference internal image-reference" href="../../_images/image3-replication.jpg"><img alt="Cyrus replication architecture" class="align-center" src="../../_images/image3-replication.jpg" style="width: 507px; height: 385px;" /></a>
<p>A replication master is configured with a number of <strong>channels</strong>: each
channel defines the set of replicas the master is replicating to, and
the configuration (credentials on the replica host, port and hostname)
for how to communicate to that replica.</p>
<p>Each channel (ie: each replica) has its own set of log files on the
master, and its own <strong>sync_client</strong> instance which processes those log
files and sends them to the replica's <strong>sync_server</strong>.</p>
<p>When a replication master generates a change, it calls <strong>sync_log</strong>.
This writes the change to all the log files (one per channel). The log
files are rotated every few seconds. The $PID on the log file set is
that of the particular sync_client who processed the change.</p>
<p>Replication is idempotent: you can safely run the same log multiple times.</p>
<p>See the <a class="reference internal" href="admin/sop/replication.html#replication"><span class="std std-ref">Replication installation and admin guide</span></a>
for more information.</p>
<section id="channels">
<h3>Channels<a class="headerlink" href="#channels" title="Permalink to this heading"></a></h3>
<p>A channel is a way of describing the linkage between a replication master and one
of its replicas. It encompasses the configuration on the master to know
which sync logs to write to, and the configuration on the master to
know the imap credentials on the replica to allow it to send details to
the replica. It is the port configuration on the replica to know where
to listen for change updates.</p>
<p>There's two standard channel configurations:</p>
<ol class="arabic simple">
<li><p>Single master keeping all replicas up to date.</p></li>
<li><p>Single master updates the replicas via chaining. (master updates
replica 1, which updates replica 2)</p></li>
</ol>
<p>The only real benefit to chaining is bandwidth use reduction - if
you have two replicas in a different data centre, you can chain them and
avoid sending all the data over the link twice. You can always re-
establish replication to the second replica by creating a direct channel
and running <code class="docutils literal notranslate"><span class="pre">sync_client</span> <span class="pre">-A</span></code> to make sure everything is up-to-date.</p>
</section>
<section id="log-file">
<h3>Log file<a class="headerlink" href="#log-file" title="Permalink to this heading"></a></h3>
<p>The log file is a list of users and/or mailboxes which have been
altered. When sync_log is enabled, all of the daemons which might
alter a mailbox or user will write a line to this log each time they do
so. That means the obvious suspects -- imapd, pop3d, timsieved, lmtpd,
etc. -- but also cyr_expire and friends.</p>
<p>So when sync_client processes a sync_log, it needs to look at an actual
copy of the user/mailbox in order to determine its current state, and
needs to look at both copies to work out what to replicate between
them. It uses sync_send_lookup to ask for the current state of an item,
and sync_response_parse to process the response to then make suitable
changes to perform the synchronisation.</p>
<p>Sync client supports doing a single user with '-u', a single mailbox
with '-m', etc. All the entries in the sync_log file are triggers to
replicate with that same value, so a line <code class="docutils literal notranslate"><span class="pre">USER</span> <span class="pre">vader@darth.net</span></code> is
the same as running <code class="docutils literal notranslate"><span class="pre">sync_client</span> <span class="pre">-u</span> <span class="pre">vader@darth.net</span></code>.</p>
</section>
</section>
<section id="invoking-cyrus">
<span id="architecture-invoking"></span><h2>Invoking Cyrus<a class="headerlink" href="#invoking-cyrus" title="Permalink to this heading"></a></h2>
<p>Cyrus has two primary configuration files which control its operation:</p>
<blockquote>
<div><ol class="arabic simple">
<li><p>/etc/imapd.conf</p></li>
<li><p>/etc/cyrus.conf</p></li>
</ol>
</div></blockquote>
<section id="imapd-conf">
<h3>imapd.conf<a class="headerlink" href="#imapd-conf" title="Permalink to this heading"></a></h3>
<p>Cyrus is invoked via <code class="docutils literal notranslate"><span class="pre">imapd</span> <span class="pre">-C</span> <span class="pre">path/to/imapd.conf</span></code>.</p>
<p>The format of <a class="reference internal" href="manpages/configs/imapd.conf.html#std-cyrusman-imapd.conf-5">imapd.conf(5)</a> is in <code class="docutils literal notranslate"><span class="pre">key:</span> <span class="pre">value</span></code> pairs.</p>
</section>
<section id="cyrus-conf">
<h3>cyrus.conf<a class="headerlink" href="#cyrus-conf" title="Permalink to this heading"></a></h3>
<p>This is the config file for the Cyrus master process. The format of
<a class="reference internal" href="manpages/configs/cyrus.conf.html#std-cyrusman-cyrus.conf-5">cyrus.conf(5)</a> defines</p>
<blockquote>
<div><ol class="arabic simple">
<li><p><strong>START</strong>: What processes to begin once at startup,</p></li>
<li><p><strong>SERVICES</strong>: Which daemons to spawn</p></li>
<li><p><strong>EVENTS</strong>: What processes to periodically invoke, similar to cron.</p></li>
</ol>
</div></blockquote>
</section>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="admin.html" class="btn btn-neutral float-left" title="Administrator Guide" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="admin/locations.html" class="btn btn-neutral float-right" title="File & Directory Locations" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 1993–2025, The Cyrus Team.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
|
