1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
|
=head1 NAME
nbdkit-protocol - which parts of the NBD protocol nbdkit supports
=head1 SYNOPSIS
nbdkit [-n|--newstyle] [--mask-handshake MASK] [--no-mc|--no-meta-contexts]
[--no-sr|--no-structured-replies] [-o|--oldstyle] [...]
=head1 DESCRIPTION
This page documents the level of support in nbdkit for various parts
of the NBD protocol.
=head1 NEW STYLE VS OLD STYLE PROTOCOL
The NBD protocol comes in two incompatible forms that we call
"oldstyle" and "newstyle". Unfortunately which protocol you should
use depends on the client and cannot be known in advance, nor can it
be negotiated from the server side.
nbdkit defaults to the newstyle protocol since nbdkit E<ge> 1.3 (the
command line flag I<-n> or I<--newstyle> is ignored for backwards
compatibility with older versions). The newstyle protocol is better
in every respect than the oldstyle protocol and you should prefer it
if possible. The newstyle protocol also includes extensions where a
client may request structured replies for even more capabilities, such
as sparse reads or meta contexts for obtaining block status. By
default, nbdkit advertises as many features as it can support (in some
cases, this can be limited by what callbacks the plugin handles), even
if the client does not negotiate to use all advertised features.
Nbdkit also includes some options that are useful mainly when
performing integration tests, for proving whether clients have sane
fallback behavior when dealing various older servers permitted by the
NBD protocol. The following options intentionally disable optional
parts of the NBD protocol, with successively larger impacts:
=over 4
=item B<--no-mc>
=item B<--no-meta-contexts>
Force the newstyle protocol to treat all requests for meta context
negotiation from the client as unsupported; the client will be unable
to query block status. By default, nbdkit gracefully handles all meta
context requests, even though it currently supports only the
C<base:allocation> context (possibly by synthesizing a context that
represents an all-data disk when the plugin lacks support for
extents); but treating meta contexts as unsupported is useful for
emulating nbd-server 3.25.
=item B<--no-sr>
=item B<--no-structured-replies>
Force the newstyle protocol to decline any client request for
structured replies; this is stronger than I<--no-meta-contexts> in
that it also disables the opportunity for sparse reads. This is
useful for emulating nbd-server 3.24.
=item B<--mask-handshake=>MASK
Alter the mask of which particular global features are advertised
during new-style handshake (defaulting to all supported bits set).
Clearing bit 0 (the low order bit) limits a client to using just
C<NBD_OPT_EXPORT_NAME> (incompatible with TLS, and includes all
effects of I<--no-sr>); clearing bit 1 causes the handshake to send
more padding bytes in response to C<NBD_OPT_EXPORT_NAME>. Other bits
in the mask will only have an effect if the NBD protocol is extended
in the future to define other global bits.
=item B<-o>
=item B<--oldstyle>
Force the oldstyle protocol. In this mode, I<--no-meta-contexts>,
I<--no-sr> and I<--mask-handshake> have no effect.
=back
=head2 Common clients and the protocol they require
Client Protocol
------------------------------------------------------------
qemu <= 2.5 without exportname oldstyle
qemu <= 2.5 with exportname newstyle
qemu >= 2.6 client can talk either protocol
qemu >= 2.11 client tries structured replies
nbd-client < 3.10 client can talk either protocol
nbd-client >= 3.10 newstyle, no structured replies
any TLS (encrypted) client newstyle
nbdkit nbd plugin client can talk either protocol
nbdkit >= 1.13.3 nbd plugin tries structured replies
libnbd either protocol, tries structured replies
=head2 Errors seen if using the wrong protocol
If you use qemu E<le> 2.5 without the exportname field against a
newstyle server, it will give the error:
Server requires an export name
If you use qemu E<le> 2.5 with the exportname field against an
oldstyle server, it will give the error:
Server does not support export names
If you use the oldstyle protocol with nbd-client E<ge> 3.10, it will
give the error:
Error: It looks like you're trying to connect to an oldstyle server.
=head2 NBD protocol and port number
Port 10809/tcp is reserved by IANA for the NBD protocol, but you can
use nbdkit on any port or on Unix domain sockets.
The NBD protocol specification claims that you should always use
newstyle when using port 10809, and use oldstyle on all other ports,
but this claim is not based on the reality of what NBD servers do, and
nbdkit does not require or encourage this.
=head1 NBD PROTOCOL FEATURES SUPPORTED BY NBDKIT
=over 4
=item newstyle protocol
Supported in nbdkit E<ge> 1.1.12, and the default in nbdkit E<ge> 1.3.
=item export names
Partially supported in nbdkit E<ge> 1.1.12. Support for plugins to
read the client export name added in nbdkit E<ge> 1.15.2.
Versions of nbdkit before 1.16 could advertise a single export name to
clients, via a now deprecated side effect of the I<-e> option. In
nbdkit 1.15.2, plugins could read the client requested export name
using L<nbdkit_export_name(3)> and serve different content. In nbdkit
1.21.22, plugins could implement C<.list_exports> to answer
C<NBD_OPT_LIST> queries.
=item C<NBD_FLAG_NO_ZEROES>
Supported in nbdkit E<ge> 1.1.13.
This protocol optimization avoids sending a useless block of zero
bytes during protocol negotiation.
=item C<NBD_CMD_WRITE_ZEROES>
Supported in nbdkit E<ge> 1.1.13.
=item TLS with X.509 certificates
Supported in nbdkit E<ge> 1.1.15.
=item TLS with Pre-Shared Keys (PSK)
Supported in nbdkit E<ge> 1.4.0.
=item C<NBD_OPT_GO>
Supported in nbdkit E<ge> 1.5.3.
This protocol enhancement allows the server to return errors when
negotiating the export name.
=item C<NBD_OPT_INFO>
Supported in nbdkit E<ge> 1.9.3.
This protocol enhancement allows a client to inspect details about
the export without actually connecting.
=item C<NBD_FLAG_CAN_MULTI_CONN>
Supported in nbdkit E<ge> 1.9.9.
=item Structured Replies
Supported in nbdkit E<ge> 1.11.8.
However we don’t expose the capability to send structured replies to
plugins yet, nor do we send human-readable error messages using this
facility.
In nbdkit E<ge> 1.13.9, the command-line option I<--no-sr> can be used
to disable server support for structured replies, for testing client
fallbacks; disabling structured replies also disables features like
block status queries that depend on structured replies.
=item Metadata Querying
Supported in nbdkit E<ge> 1.11.8.
In nbdkit E<ge> 1.37.9, the command-line option I<--no-meta-contexts>
can be used to disable server support for meta contexts, for testing
client fallbacks.
=item Block Status
Supported in nbdkit E<ge> 1.11.10.
Only C<base:allocation> (ie. querying which parts of an image are
sparse) is supported.
Sparse reads (using C<NBD_REPLY_TYPE_OFFSET_HOLE> are not directly
supported, but a client can use block status to infer which portions
of the export do not need to be read.
=item C<NBD_FLAG_DF>
Supported in nbdkit E<ge> 1.11.11.
This protocol extension allows a client to force an all-or-none read
when structured replies are in effect. However, the flag is a no-op
until we extend the plugin API to allow a fragmented read in the first
place.
=item C<NBD_CMD_CACHE>
Supported in nbdkit E<ge> 1.13.4.
This protocol extension allows a client to inform the server about
intent to access a portion of the export, to allow the server an
opportunity to cache things appropriately.
=item C<NBD_CMD_FLAG_FAST_ZERO>
Supported in nbdkit E<ge> 1.15.0.
This protocol extension allows a server to advertise that it can rank
all zero requests as fast or slow, at which point the client can make
fast zero requests which fail immediately with C<ENOTSUP> if the
request is no faster than a counterpart write would be, while normal
zero requests still benefit from compressed network traffic regardless
of the time taken.
=item C<NBD_INFO_NAME>, C<NBD_INFO_DESCRIPTION>
Supported in nbdkit E<ge> 1.23.6.
These protocol extensions allow a client to learn more information
about an export during C<NBD_OPT_GO>. The C<.default_export> callback
can inform a client of a canonical non-empty name in place of the
default export C<"">, and the C<.export_description> callback can give
a client more details about the export.
=item C<NBD_INFO_BLOCK_SIZE>
Supported in nbdkit E<ge> 1.30.
=item Resize Extension
I<Not supported>.
=item Extended Headers Extension
I<Not supported>.
=back
=head1 SEE ALSO
L<nbdkit(1)>,
L<https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md>,
L<https://nbd.sourceforge.io/>.
=head1 AUTHORS
Eric Blake
Richard W.M. Jones
Pino Toscano
=head1 COPYRIGHT
Copyright Red Hat
|
