1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<!--
This file is autogenerated from pci-hotplug.html.in
Do not edit this file. Changes will be lost.
-->
<!--
This page was generated at Thu Jan 10 20:48:17 UTC 2019.
-->
<head>
<meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<link rel="stylesheet" type="text/css" href="main.css"/>
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png"/>
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png"/>
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png"/>
<link rel="manifest" href="/manifest.json"/>
<meta name="theme-color" content="#ffffff"/>
<title>libvirt: PCI topology and hotplug</title>
<meta name="description" content="libvirt, virtualization, virtualization API"/>
<script type="text/javascript">
<!--
function init() {
window.addEventListener('scroll', function(e){
var distanceY = window.pageYOffset || document.documentElement.scrollTop,
shrinkOn = 94
home = document.getElementById("home");
links = document.getElementById("jumplinks");
search = document.getElementById("search");
body = document.getElementById("body");
if (distanceY > shrinkOn) {
if (home.className != "navhide") {
body.className = "navhide"
home.className = "navhide"
links.className = "navhide"
search.className = "navhide"
}
} else {
if (home.className == "navhide") {
body.className = ""
home.className = ""
links.className = ""
search.className = ""
}
}
});
}
window.onload = init();
-->
</script>
</head>
<body>
<div id="body">
<div id="content">
<h1>PCI topology and hotplug</h1>
<ul>
<li>
<a href="#x86_64">x86_64 architecture</a>
<ul>
<li>
<a href="#x86_64-q35">q35 machine type</a>
</li>
<li>
<a href="#x86_64-i440fx">i440fx (pc) machine type</a>
</li>
</ul>
</li>
<li>
<a href="#ppc64">ppc64 architecture</a>
<ul>
<li>
<a href="#ppc64-pseries">pseries machine type</a>
</li>
</ul>
</li>
<li>
<a href="#aarch64">aarch64 architecture</a>
<ul>
<li>
<a href="#aarch64-virt">mach-virt (virt) machine type</a>
</li>
</ul>
</li>
</ul>
<p>
Perhaps surprisingly, most libvirt guests support only limited PCI
device hotplug out of the box, or even none at all.
</p>
<p>
The reason for this apparent limitation is the fact that each
hotplugged PCI device might require additional PCI controllers to
be added to the guest. Since most PCI controllers can't be
hotplugged, they need to be added before the guest is started;
however, libvirt has no way of knowing in advance how many devices
will be hotplugged during the guest's lifetime, thus making it
impossible to automatically provide the right amount of PCI
controllers: any arbitrary number would end up being too big
for some users, and too small for others.
</p>
<p>
Ultimately, the user is the only one who knows how much the guest
will need to grow dynamically, so the responsibility of planning
a suitable PCI topology in advance falls on them.
</p>
<p>
This document aims at providing all the information needed to
successfully plan the PCI topology of a guest. Note that the
details can vary a lot between architectures and even machine
types, hence the way it's organized.
</p>
<h2>
<a id="x86_64">x86_64 architecture</a>
<a class="headerlink" href="#x86_64" title="Permalink to this headline">¶</a>
</h2>
<h3>
<a id="x86_64-q35">q35 machine type</a>
<a class="headerlink" href="#x86_64-q35" title="Permalink to this headline">¶</a>
</h3>
<p>
This is a PCI Express native machine type. The default PCI topology
looks like
</p>
<pre>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
</controller></pre>
<p>
and supports hotplugging a single PCI Express device, either
emulated or assigned from the host.
</p>
<p>
If you have a very specific use case, such as the appliances
used by <a href="http://libguestfs.org/">libguestfs</a> behind
the scenes to access disk images, and this automatically-added
<code>pcie-root-port</code> controller ends up being a nuisance,
you can prevent libvirt from adding it by manually managing PCI
controllers and addresses according to your needs.
</p>
<p>
Slots on the <code>pcie-root</code> controller do not support
hotplug, so the device will be hotplugged into the
<code>pcie-root-port</code> controller. If you plan to hotplug
more than a single PCI Express device, you should add a suitable
number of <code>pcie-root-port</code> controllers when defining
the guest: for example, add
</p>
<pre>
<controller type='pci' model='pcie-root'/>
<controller type='pci' model='pcie-root-port'/>
<controller type='pci' model='pcie-root-port'/>
<controller type='pci' model='pcie-root-port'/></pre>
<p>
if you expect to hotplug up to three PCI Express devices,
either emulated or assigned from the host. That's all the
information you need to provide: libvirt will fill in the
remaining details automatically. Note that you need to add
the <code>pcie-root</code> controller along with the
<code>pcie-root-port</code> controllers or you will get an
error.
</p>
<p>
Note that if you're adding PCI controllers to a guest and at
the same time you're also adding PCI devices, some of the
controllers will be used for the newly-added devices and won't
be available for hotplug once the guest has been started.
</p>
<p>
If you expect to hotplug legacy PCI devices, then you will need
specialized controllers, since all those mentioned above are
intended for PCI Express devices only: add
</p>
<pre>
<controller type='pci' model='dmi-to-pci-bridge'/>
<controller type='pci' model='pci-bridge'/></pre>
<p>
and you'll be able to hotplug up to 31 legacy PCI devices,
either emulated or assigned from the host, in the slots
from 0x01 to 0x1f of the <code>pci-bridge</code> controller.
</p>
<h3>
<a id="x86_64-i440fx">i440fx (pc) machine type</a>
<a class="headerlink" href="#x86_64-i440fx" title="Permalink to this headline">¶</a>
</h3>
<p>
This is a legacy PCI native machine type. The default PCI
topology looks like
</p>
<pre>
<controller type='pci' index='0' model='pci-root'/></pre>
<p>
where each of the 31 slots (from 0x01 to 0x1f) on the
<code>pci-root</code> controller is hotplug capable and
can accept a legacy PCI device, either emulated or
assigned from the guest.
</p>
<h2>
<a id="ppc64">ppc64 architecture</a>
<a class="headerlink" href="#ppc64" title="Permalink to this headline">¶</a>
</h2>
<h3>
<a id="ppc64-pseries">pseries machine type</a>
<a class="headerlink" href="#ppc64-pseries" title="Permalink to this headline">¶</a>
</h3>
<p>
The default PCI topology for the <code>pseries</code> machine
type looks like
</p>
<pre>
<controller type='pci' index='0' model='pci-root'>
<model name='spapr-pci-host-bridge'/>
<target index='0'/>
</controller></pre>
<p>
The 31 slots, from 0x01 to 0x1f, on a <code>pci-root</code>
controller are all hotplug capable and, despite the name
suggesting otherwise, starting with QEMU 2.9 all of them
can accept PCI Express devices in addition to legacy PCI
devices; however, libvirt will only place emulated devices
on the default <code>pci-root</code> controller.
</p>
<p>
In order to take advantage of improved error reporting and
recovering capabilities, PCI devices assigned from the
host need to be isolated by placing each on a separate
<code>pci-root</code> controller, which has to be prepared
in advance for hotplug to work: for example, add
</p>
<pre>
<controller type='pci' model='pci-root'/>
<controller type='pci' model='pci-root'/>
<controller type='pci' model='pci-root'/></pre>
<p>
if you expect to hotplug up to three PCI devices assigned
from the host.
</p>
<h2>
<a id="aarch64">aarch64 architecture</a>
<a class="headerlink" href="#aarch64" title="Permalink to this headline">¶</a>
</h2>
<h3>
<a id="aarch64-virt">mach-virt (virt) machine type</a>
<a class="headerlink" href="#aarch64-virt" title="Permalink to this headline">¶</a>
</h3>
<p>
This machine type mostly behaves the same as the
<a href="#x86_64-q35">q35 machine type</a>, so you can just
refer to that section for information.
</p>
<p>
The only difference worth mentioning is that using legacy
PCI for <code>mach-virt</code> guests is extremely uncommon,
so you'll probably never need to add controllers other than
<code>pcie-root-port</code>.
</p>
</div>
</div>
<div id="nav">
<div id="home">
<a href="index.html">Home</a>
</div>
<div id="jumplinks">
<ul>
<li>
<a href="downloads.html">Download</a>
</li>
<li>
<a href="contribute.html">Contribute</a>
</li>
<li>
<a href="docs.html">Docs</a>
</li>
</ul>
</div>
<div id="search">
<form action="search.php" enctype="application/x-www-form-urlencoded" method="get">
<div>
<input name="query" type="text" size="12" value=""/>
<input name="submit" type="submit" value="Go"/>
</div>
</form>
</div>
</div>
<div id="footer">
<div id="contact">
<h3>Contact</h3>
<ul>
<li>
<a href="contact.html#email">email</a>
</li>
<li>
<a href="contact.html#irc">irc</a>
</li>
</ul>
</div>
<div id="community">
<h3>Community</h3>
<ul>
<li>
<a href="https://twitter.com/hashtag/libvirt">twitter</a>
</li>
<li>
<a href="https://plus.google.com/communities/109522598353007505282">google+</a>
</li>
<li>
<a href="http://stackoverflow.com/questions/tagged/libvirt">stackoverflow</a>
</li>
<li>
<a href="http://serverfault.com/questions/tagged/libvirt">serverfault</a>
</li>
</ul>
</div>
<div id="conduct">
Participants in the libvirt project agree to abide by <a href="governance.html#codeofconduct">the project code of conduct</a></div>
<br class="clear"/>
</div>
</body>
</html>
|
