1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<!--
This file is autogenerated from internals/oomtesting.html.in
Do not edit this file. Changes will be lost.
-->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="stylesheet" type="text/css" href="../main.css" />
<link rel="SHORTCUT ICON" href="../32favicon.png" />
<title>libvirt: Out of memory testing</title>
<meta name="description" content="libvirt, virtualization, virtualization API" />
</head>
<body>
<div id="header">
<div id="headerLogo"></div>
<div id="headerSearch">
<form action="../search.php" enctype="application/x-www-form-urlencoded" method="get"><div>
<input id="query" name="query" type="text" size="12" value="" />
<input id="submit" name="submit" type="submit" value="Search" />
</div></form>
</div>
</div>
<div id="body">
<div id="menu">
<ul class="l0"><li>
<div>
<a title="Front page of the libvirt website" class="inactive" href="../index.html">Home</a>
</div>
</li><li>
<div>
<a title="Details of new features and bugs fixed in each release" class="inactive" href="../news.html">News</a>
</div>
</li><li>
<div>
<a title="Applications known to use libvirt" class="inactive" href="../apps.html">Applications</a>
</div>
</li><li>
<div>
<a title="Get the latest source releases, binary builds and get access to the source repository" class="inactive" href="../downloads.html">Downloads</a>
</div>
</li><li>
<div>
<a title="Information for users, administrators and developers" class="active" href="../docs.html">Documentation</a>
<ul class="l1"><li>
<div>
<a title="How to compile libvirt" class="inactive" href="../compiling.html">Compiling</a>
</div>
</li><li>
<div>
<a title="Information about deploying and using libvirt" class="inactive" href="../deployment.html">Deployment</a>
</div>
</li><li>
<div>
<a title="Overview of the logical subsystems in the libvirt API" class="inactive" href="../intro.html">Architecture</a>
</div>
</li><li>
<div>
<a title="Description of the XML formats used in libvirt" class="inactive" href="../format.html">XML format</a>
</div>
</li><li>
<div>
<a title="Hypervisor specific driver information" class="inactive" href="../drivers.html">Drivers</a>
</div>
</li><li>
<div>
<a title="Reference manual for the C public API" class="inactive" href="../html/index.html">API reference</a>
</div>
</li><li>
<div>
<a title="Bindings of the libvirt API for other languages" class="inactive" href="../bindings.html">Language bindings</a>
</div>
</li><li>
<div>
<a title="Working on the internals of libvirt API, driver and daemon code" class="active" href="../internals.html">Internals</a>
<ul class="l2"><li>
<div>
<a title="General hacking guidelines for contributors" class="inactive" href="../hacking.html">Contributor guidelines</a>
</div>
</li><li>
<div>
<a title="Adding new public libvirt APIs" class="inactive" href="../api_extension.html">API extensions</a>
</div>
</li><li>
<div>
<a title="Spawning commands from libvirt driver code" class="inactive" href="../internals/command.html">Spawning commands</a>
</div>
</li><li>
<div>
<a title="RPC protocol information and API / dispatch guide" class="inactive" href="../internals/rpc.html">RPC protocol & APIs</a>
</div>
</li><li>
<div>
<a title="Use lock managers to protect disk content" class="inactive" href="../internals/locking.html">Lock managers</a>
</div>
</li><li>
<div>
<span class="active">Out of memory testing</span>
</div>
</li></ul>
</div>
</li><li>
<div>
<a title="A guide and reference for developing with libvirt" class="inactive" href="../devguide.html">Development Guide</a>
</div>
</li><li>
<div>
<a title="Command reference for virsh" class="inactive" href="../virshcmdref.html">Virsh Commands</a>
</div>
</li><li>
<div>
<a title="Project governance and code of conduct" class="inactive" href="../governance.html">Governance</a>
</div>
</li></ul>
</div>
</li><li>
<div>
<a title="User contributed content" class="inactive" href="http://wiki.libvirt.org">Wiki</a>
</div>
</li><li>
<div>
<a title="Frequently asked questions" class="inactive" href="http://wiki.libvirt.org/page/FAQ">FAQ</a>
</div>
</li><li>
<div>
<a title="How and where to report bugs and request features" class="inactive" href="../bugs.html">Bug reports</a>
</div>
</li><li>
<div>
<a title="How to contact the developers via email and IRC" class="inactive" href="../contact.html">Contact</a>
</div>
</li><li>
<div>
<a title="Available test suites for libvirt" class="inactive" href="../testsuites.html">Test suites</a>
</div>
</li><li>
<div>
<a title="Miscellaneous links of interest related to libvirt" class="inactive" href="../relatedlinks.html">Related Links</a>
</div>
</li><li>
<div>
<a title="Overview of all content on the website" class="inactive" href="../sitemap.html">Sitemap</a>
</div>
</li></ul>
</div>
<div id="content">
<h1>Out of memory testing</h1>
<ul><li>
<a href="#basicoom">Basic OOM testing support</a>
<ul><li>
<a href="#valgrind">Tracking failures with valgrind</a>
</li><li>
<a href="#stacktraces">Tracking failures with stack traces</a>
</li><li>
<a href="#noncrash">Non-crash related problems</a>
</li></ul>
</li></ul>
<p>
This page describes how to use the test suite todo out of memory
testing.
</p>
<h2>Building with OOM testing</h2>
<p>
Since OOM testing requires hooking into the malloc APIs, it is
not enabled by default. The flag <code>--enable-test-oom</code>
must be given to <code>configure</code>. When this is done the
libvirt allocation APIs will have some hooks enabled.
</p>
<pre xml:space="preserve">
$ ./configure --enable-test-oom
</pre>
<h2>
<a name="basicoom" shape="rect" id="basicoom">Basic OOM testing support</a>
<a class="headerlink" href="#basicoom" title="Permalink to this headline">¶</a>
</h2>
<p>
The first step in validating OOM usage is to run a test suite
with full OOM testing enabled. This is done by setting the
<code>VIR_TEST_OOM=1</code> environment variable. The way this
works is that it runs the test once normally to "prime" any
static memory allocations. Then it runs it once more counting
the total number of memory allocations. Then it runs it in a
loop failing a different memory allocation each time. For every
memory allocation failure triggered, it expects the test case
to return an error. OOM testing is quite slow requiring each
test case to be executed O(n) times, where 'n' is the total
number of memory allocations. This results in a total number
of memory allocations of '(n * (n + 1) ) / 2'
</p>
<pre xml:space="preserve">
$ VIR_TEST_OOM=1 ./qemuxml2argvtest
1) QEMU XML-2-ARGV minimal ... OK
Test OOM for nalloc=42 .......................................... OK
2) QEMU XML-2-ARGV minimal-s390 ... OK
Test OOM for nalloc=28 ............................ OK
3) QEMU XML-2-ARGV machine-aliases1 ... OK
Test OOM for nalloc=38 ...................................... OK
4) QEMU XML-2-ARGV machine-aliases2 ... OK
Test OOM for nalloc=38 ...................................... OK
5) QEMU XML-2-ARGV machine-core-on ... OK
Test OOM for nalloc=37 ..................................... OK
...snip...
</pre>
<p>
In this output, the first line shows the normal execution and
the test number, and the second line shows the total number
of memory allocations from that test case.
</p>
<h3>
<a name="valgrind" shape="rect" id="valgrind">Tracking failures with valgrind</a>
<a class="headerlink" href="#valgrind" title="Permalink to this headline">¶</a>
</h3>
<p>
The test suite should obviously *not* crash during OOM testing.
If it does crash, then to assist in tracking down the problem
it is worth using valgrind and only running a single test case.
For example, supposing test case 5 crashed. Then re-run the
test with
</p>
<pre xml:space="preserve">
$ VIR_TEST_OOM=1 VIR_TEST_RANGE=5 ../run valgrind ./qemuxml2argvtest
...snip...
5) QEMU XML-2-ARGV machine-core-on ... OK
Test OOM for nalloc=37 ..................................... OK
...snip...
</pre>
<p>
Valgrind should report the cause of the crash - for example a
double free or use of uninitialized memory or NULL pointer
access.
</p>
<h3>
<a name="stacktraces" shape="rect" id="stacktraces">Tracking failures with stack traces</a>
<a class="headerlink" href="#stacktraces" title="Permalink to this headline">¶</a>
</h3>
<p>
With some really difficult bugs valgrind is not sufficient to
identify the cause. In this case, it is useful to identify the
precise allocation which was failed, to allow the code path
to the error to be traced. The <code>VIR_TEST_OOM</code>
env variable can be given a range of memory allocations to
test. So if a test case has 150 allocations, it can be told
to only test allocation numbers 7-10. The <code>VIR_TEST_OOM_TRACE</code>
variable can be used to print out stack traces.
</p>
<pre xml:space="preserve">
$ VIR_TEST_OOM_TRACE=2 VIR_TEST_OOM=1:7-10 VIR_TEST_RANGE=5 \
../run valgrind ./qemuxml2argvtest
5) QEMU XML-2-ARGV machine-core-on ... OK
Test OOM for nalloc=37 !virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11786 (discriminator 1)
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
!virAlloc
/home/berrange/src/virt/libvirt/src/util/viralloc.c:133
virDomainDiskDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:4790
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11797
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
!virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virXPathNodeSet
/home/berrange/src/virt/libvirt/src/util/virxml.c:609
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11805
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
!virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11808 (discriminator 1)
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
</pre>
<h3>
<a name="noncrash" shape="rect" id="noncrash">Non-crash related problems</a>
<a class="headerlink" href="#noncrash" title="Permalink to this headline">¶</a>
</h3>
<p>
Not all memory allocation bugs result in code crashing. Sometimes
the code will be silently ignoring the allocation failure, resulting
in incorrect data being produced. For example the XML parser may
mistakenly treat an allocation failure as indicating that an XML
attribute was not set in the input document. It is hard to identify
these problems from the test suite automatically. For this, the
test suites should be run with <code>VIR_TEST_DEBUG=1</code> set
and then stderr analysed for any unexpected data. For example,
the XML conversion may show an embedded "(null)" literal, or the
test suite might complain about missing elements / attributes
in the actual vs expected data. These are all signs of bugs in
OOM handling. In the future the OOM tests will be enhanced to
validate that an error VIR_ERR_NO_MEMORY is returned for each
allocation failed, rather than some other error.
</p>
</div>
</div>
<div id="footer">
<p id="sponsor">
Sponsored by:<br /><a href="http://et.redhat.com/"><img src="../et.png" alt="Project sponsored by Red Hat Emerging Technology" /></a></p>
</div>
</body>
</html>
|
