1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
|
<pre>Network Working Group A. Gulbrandsen
Request for Comments: 4978 Oryx Mail Systems GmbH
Category: Standards Track August 2007
<span class="h1">The IMAP COMPRESS Extension</span>
Status of this Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Abstract
The COMPRESS extension allows an IMAP connection to be effectively
and efficiently compressed.
Table of Contents
<a href="#section-1">1</a>. Introduction and Overview .......................................<a href="#page-2">2</a>
<a href="#section-2">2</a>. Conventions Used in This Document ...............................<a href="#page-2">2</a>
<a href="#section-3">3</a>. The COMPRESS Command ............................................<a href="#page-3">3</a>
<a href="#section-4">4</a>. Compression Efficiency ..........................................<a href="#page-4">4</a>
<a href="#section-5">5</a>. Formal Syntax ...................................................<a href="#page-6">6</a>
<a href="#section-6">6</a>. Security Considerations .........................................<a href="#page-6">6</a>
<a href="#section-7">7</a>. IANA Considerations .............................................<a href="#page-6">6</a>
<a href="#section-8">8</a>. Acknowledgements ................................................<a href="#page-7">7</a>
<a href="#section-9">9</a>. References ......................................................<a href="#page-7">7</a>
<a href="#section-9.1">9.1</a>. Normative References .......................................<a href="#page-7">7</a>
<a href="#section-9.2">9.2</a>. Informative References .....................................<a href="#page-7">7</a>
<span class="grey">Gulbrandsen Standards Track [Page 1]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction and Overview</span>
A server which supports the COMPRESS extension indicates this with
one or more capability names consisting of "COMPRESS=" followed by a
supported compression algorithm name as described in this document.
The goal of COMPRESS is to reduce the bandwidth usage of IMAP.
Compared to PPP compression (see [<a href="./rfc1962" title=""The PPP Compression Control Protocol (CCP)"">RFC1962</a>]) and modem-based
compression (see [<a href="#ref-MNP" title=""The Complete Modem Reference"">MNP</a>] and [<a href="#ref-V42BIS" title=""V.42bis: Data compression procedures for data circuit-terminating equipment (DCE) using error correction procedures"">V42BIS</a>]), COMPRESS offers much better
compression efficiency. COMPRESS can be used together with Transport
Security Layer (TLS) [<a href="./rfc4346" title=""The Transport Layer Security (TLS) Protocol Version 1.1"">RFC4346</a>], Simple Authentication and Security
layer (SASL) encryption, Virtual Private Networks (VPNs), etc.
Compared to TLS compression [<a href="./rfc3749" title=""Transport Layer Security Protocol Compression Methods"">RFC3749</a>], COMPRESS has the following
(dis)advantages:
- COMPRESS can be implemented easily both by IMAP servers and
clients.
- IMAP COMPRESS benefits from an intimate knowledge of the IMAP
protocol's state machine, allowing for dynamic and aggressive
optimization of the underlying compression algorithm's parameters.
- When the TLS layer implements compression, any protocol using that
layer can transparently benefit from that compression (e.g., SMTP
and IMAP). COMPRESS is specific to IMAP.
In order to increase interoperation, it is desirable to have as few
different compression algorithms as possible, so this document
specifies only one. The DEFLATE algorithm (defined in [<a href="./rfc1951" title=""DEFLATE Compressed Data Format Specification version 1.3"">RFC1951</a>]) is
standard, widely available and fairly efficient, so it is the only
algorithm defined by this document.
In order to increase interoperation, IMAP servers that advertise this
extension SHOULD also advertise the TLS DEFLATE compression mechanism
as defined in [<a href="./rfc3749" title=""Transport Layer Security Protocol Compression Methods"">RFC3749</a>]. IMAP clients MAY use either COMPRESS or TLS
compression, however, if the client and server support both, it is
RECOMMENDED that the client choose TLS compression.
The extension adds one new command (COMPRESS) and no new responses.
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Conventions Used in This Document</span>
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [<a href="./rfc2119" title=""Key words for use in RFCs to Indicate Requirement Levels"">RFC2119</a>].
Formal syntax is defined by [<a href="./rfc4234" title=""Augmented BNF for Syntax Specifications: ABNF"">RFC4234</a>] as modified by [<a href="./rfc3501" title=""INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1"">RFC3501</a>].
<span class="grey">Gulbrandsen Standards Track [Page 2]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
In the examples, "C:" and "S:" indicate lines sent by the client and
server respectively. "[...]" denotes elision.
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. The COMPRESS Command</span>
Arguments: Name of compression mechanism: "DEFLATE".
Responses: None
Result: OK The server will compress its responses and expects the
client to compress its commands.
NO Compression is already active via another layer.
BAD Command unknown, invalid or unknown argument, or COMPRESS
already active.
The COMPRESS command instructs the server to use the named
compression mechanism ("DEFLATE" is the only one defined) for all
commands and/or responses after COMPRESS.
The client MUST NOT send any further commands until it has seen the
result of COMPRESS. If the response was OK, the client MUST compress
starting with the first command after COMPRESS. If the server
response was BAD or NO, the client MUST NOT turn on compression.
If the server responds NO because it knows that the same mechanism is
active already (e.g., because TLS has negotiated the same mechanism),
it MUST send COMPRESSIONACTIVE as resp-text-code (see <a href="./rfc3501#section-7.1">[RFC3501],
Section 7.1</a>), and the resp-text SHOULD say which layer compresses.
If the server issues an OK response, the server MUST compress
starting immediately after the CRLF which ends the tagged OK
response. (Responses issued by the server before the OK response
will, of course, still be uncompressed.) If the server issues a BAD
or NO response, the server MUST NOT turn on compression.
For DEFLATE (as for many other compression mechanisms), the
compressor can trade speed against quality. When decompressing there
isn't much of a tradeoff. Consequently, the client and server are
both free to pick the best reasonable rate of compression for the
data they send.
When COMPRESS is combined with TLS (see [<a href="./rfc4346" title=""The Transport Layer Security (TLS) Protocol Version 1.1"">RFC4346</a>]) or SASL (see
[<a href="./rfc4422" title=""Simple Authentication and Security Layer (SASL)"">RFC4422</a>]) security layers, the sending order of the three extensions
MUST be first COMPRESS, then SASL, and finally TLS. That is, before
data is transmitted it is first compressed. Second, if a SASL
security layer has been negotiated, the compressed data is then
signed and/or encrypted accordingly. Third, if a TLS security layer
has been negotiated, the data from the previous step is signed and/or
<span class="grey">Gulbrandsen Standards Track [Page 3]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
encrypted accordingly. When receiving data, the processing order
MUST be reversed. This ensures that before sending, data is
compressed before it is encrypted, independent of the order in which
the client issues COMPRESS, AUTHENTICATE, and STARTTLS.
The following example illustrates how commands and responses are
compressed during a simple login sequence:
S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
C: a starttls
S: a OK TLS active
From this point on, everything is encrypted.
C: b login arnt tnra
S: b OK Logged in as arnt
C: c compress deflate
S: d OK DEFLATE active
From this point on, everything is compressed before being
encrypted.
The following example demonstrates how a server may refuse to
compress twice:
S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE]
[...]
C: c compress deflate
S: c NO [COMPRESSIONACTIVE] DEFLATE active via TLS
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Compression Efficiency</span>
This section is informative, not normative.
IMAP poses some unusual problems for a compression layer.
Upstream is fairly simple. Most IMAP clients send the same few
commands again and again, so any compression algorithm that can
exploit repetition works efficiently. The APPEND command is an
exception; clients that send many APPEND commands may want to
surround large literals with flushes in the same way as is
recommended for servers later in this section.
Downstream has the unusual property that several kinds of data are
sent, confusing all dictionary-based compression algorithms.
<span class="grey">Gulbrandsen Standards Track [Page 4]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
One type is IMAP responses. These are highly compressible; zlib
using its least CPU-intensive setting compresses typical responses to
25-40% of their original size.
Another type is email headers. These are equally compressible, and
benefit from using the same dictionary as the IMAP responses.
A third type is email body text. Text is usually fairly short and
includes much ASCII, so the same compression dictionary will do a
good job here, too. When multiple messages in the same thread are
read at the same time, quoted lines etc. can often be compressed
almost to zero.
Finally, attachments (non-text email bodies) are transmitted, either
in binary form or encoded with base-64.
When attachments are retrieved in binary form, DEFLATE may be able to
compress them, but the format of the attachment is usually not IMAP-
like, so the dictionary built while compressing IMAP does not help.
The compressor has to adapt its dictionary from IMAP to the
attachment's format, and then back. A few file formats aren't
compressible at all using deflate, e.g., .gz, .zip, and .jpg files.
When attachments are retrieved in base-64 form, the same problems
apply, but the base-64 encoding adds another problem. 8-bit
compression algorithms such as deflate work well on 8-bit file
formats, however base-64 turns a file into something resembling 6-bit
bytes, hiding most of the 8-bit file format from the compressor.
When using the zlib library (see [<a href="./rfc1951" title=""DEFLATE Compressed Data Format Specification version 1.3"">RFC1951</a>]), the functions
deflateInit2(), deflate(), inflateInit2(), and inflate() suffice to
implement this extension. The windowBits value must be in the range
-8 to -15, or else deflateInit2() uses the wrong format.
deflateParams() can be used to improve compression rate and resource
use. The Z_FULL_FLUSH argument to deflate() can be used to clear the
dictionary (the receiving peer does not need to do anything).
A client can improve downstream compression by implementing BINARY
(defined in [<a href="./rfc3516" title=""IMAP4 Binary Content Extension"">RFC3516</a>]) and using FETCH BINARY instead of FETCH BODY.
In the author's experience, the improvement ranges from 5% to 40%
depending on the attachment being downloaded.
A server can improve downstream compression if it hints to the
compressor that the data type is about to change strongly, e.g., by
sending a Z_FULL_FLUSH at the start and end of large non-text
literals (before and after '*CHAR8' in the definition of literal in
<a href="./rfc3501">RFC 3501</a>, page 86). Small literals are best left alone. A possible
boundary is 5k.
<span class="grey">Gulbrandsen Standards Track [Page 5]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
A server can improve the CPU efficiency both of the server and the
client if it adjusts the compression level (e.g., using the
deflateParams() function in zlib) at these points, to avoid trying to
compress incompressible attachments. A very simple strategy is to
change the level to 0 at the start of a literal provided the first
two bytes are either 0x1F 0x8B (as in deflate-compressed files) or
0xFF 0xD8 (JPEG), and to keep it at 1-5 the rest of the time. More
complex strategies are possible.
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Formal Syntax</span>
The following syntax specification uses the Augmented Backus-Naur
Form (ABNF) notation as specified in [<a href="./rfc4234" title=""Augmented BNF for Syntax Specifications: ABNF"">RFC4234</a>]. This syntax augments
the grammar specified in [<a href="./rfc3501" title=""INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1"">RFC3501</a>]. [<a href="./rfc4234" title=""Augmented BNF for Syntax Specifications: ABNF"">RFC4234</a>] defines SP and
[<a href="./rfc3501" title=""INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1"">RFC3501</a>] defines command-auth, capability, and resp-text-code.
Except as noted otherwise, all alphabetic characters are case-
insensitive. The use of upper or lower case characters to define
token strings is for editorial clarity only. Implementations MUST
accept these strings in a case-insensitive fashion.
command-auth =/ compress
compress = "COMPRESS" SP algorithm
capability =/ "COMPRESS=" algorithm
;; multiple COMPRESS capabilities allowed
algorithm = "DEFLATE"
resp-text-code =/ "COMPRESSIONACTIVE"
Note that due the syntax of capability names, future algorithm names
must be atoms.
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Security Considerations</span>
As for TLS compression [<a href="./rfc3749" title=""Transport Layer Security Protocol Compression Methods"">RFC3749</a>].
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. IANA Considerations</span>
The IANA has added COMPRESS=DEFLATE to the list of IMAP capabilities.
<span class="grey">Gulbrandsen Standards Track [Page 6]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Acknowledgements</span>
Eric Burger, Dave Cridland, Tony Finch, Ned Freed, Philip Guenther,
Randall Gellens, Tony Hansen, Cullen Jennings, Stephane Maes, Alexey
Melnikov, Lyndon Nerenberg, and Zoltan Ordogh have all helped with
this document.
The author would also like to thank various people in the rooms at
meetings, whose help is real, but not reflected in the author's
mailbox.
<span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. References</span>
<span class="h3"><a class="selflink" id="section-9.1" href="#section-9.1">9.1</a>. Normative References</span>
[<a id="ref-RFC1951">RFC1951</a>] Deutsch, P., "DEFLATE Compressed Data Format Specification
version 1.3", <a href="./rfc1951">RFC 1951</a>, May 1996.
[<a id="ref-RFC2119">RFC2119</a>] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>, March 1997.
[<a id="ref-RFC3501">RFC3501</a>] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
4rev1", <a href="./rfc3501">RFC 3501</a>, March 2003.
[<a id="ref-RFC4234">RFC4234</a>] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", <a href="./rfc4234">RFC 4234</a>, October 2005.
<span class="h3"><a class="selflink" id="section-9.2" href="#section-9.2">9.2</a>. Informative References</span>
[<a id="ref-RFC1962">RFC1962</a>] Rand, D., "The PPP Compression Control Protocol (CCP)",
<a href="./rfc1962">RFC 1962</a>, June 1996.
[<a id="ref-RFC3516">RFC3516</a>] Nerenberg, L., "IMAP4 Binary Content Extension", <a href="./rfc3516">RFC 3516</a>,
April 2003.
[<a id="ref-RFC3749">RFC3749</a>] Hollenbeck, S., "Transport Layer Security Protocol
Compression Methods", <a href="./rfc3749">RFC 3749</a>, May 2004.
[<a id="ref-RFC4346">RFC4346</a>] Dierks, T. and E. Rescorla, "The Transport Layer Security
(TLS) Protocol Version 1.1", <a href="./rfc4346">RFC 4346</a>, April 2006.
[<a id="ref-RFC4422">RFC4422</a>] Melnikov, A. and K. Zeilenga, "Simple Authentication and
Security Layer (SASL)", <a href="./rfc4422">RFC 4422</a>, June 2006.
[<a id="ref-V42BIS">V42BIS</a>] ITU, "V.42bis: Data compression procedures for data
circuit-terminating equipment (DCE) using error correction
procedures", <a href="http://www.itu.int/rec/T-REC-V.42bis">http://www.itu.int/rec/T-REC-V.42bis</a>, January
1990.
<span class="grey">Gulbrandsen Standards Track [Page 7]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
[<a id="ref-MNP">MNP</a>] Gilbert Held, "The Complete Modem Reference", Second
Edition, Wiley Professional Computing, ISBN 0-471-00852-4,
May 1994.
Author's Address
Arnt Gulbrandsen
Oryx Mail Systems GmbH
Schweppermannstr. 8
D-81671 Muenchen
Germany
Fax: +49 89 4502 9758
EMail: arnt@oryx.com
<span class="grey">Gulbrandsen Standards Track [Page 8]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
<span class="grey"><a href="./rfc4978">RFC 4978</a> The IMAP COMPRESS Extension August 2007</span>
Full Copyright Statement
Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions
contained in <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a>, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and <a href="https://www.rfc-editor.org/bcp/bcp79">BCP 79</a>.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
<a href="http://www.ietf.org/ipr">http://www.ietf.org/ipr</a>.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Gulbrandsen Standards Track [Page 9]
</pre>
|
