1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<!--
This file is autogenerated from internals/oomtesting.html.in
Do not edit this file. Changes will be lost.
-->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="stylesheet" type="text/css" href="../main.css" />
<link rel="SHORTCUT ICON" href="../32favicon.png" />
<title>libvirt: Out of memory testing</title>
<meta name="description" content="libvirt, virtualization, virtualization API" />
</head>
<body>
<div id="body">
<div id="content">
<h1>Out of memory testing</h1>
<ul><li>
<a href="#basicoom">Basic OOM testing support</a>
<ul><li>
<a href="#valgrind">Tracking failures with valgrind</a>
</li><li>
<a href="#stacktraces">Tracking failures with stack traces</a>
</li><li>
<a href="#noncrash">Non-crash related problems</a>
</li></ul>
</li></ul>
<p>
This page describes how to use the test suite todo out of memory
testing.
</p>
<h2>Building with OOM testing</h2>
<p>
Since OOM testing requires hooking into the malloc APIs, it is
not enabled by default. The flag <code>--enable-test-oom</code>
must be given to <code>configure</code>. When this is done the
libvirt allocation APIs will have some hooks enabled.
</p>
<pre xml:space="preserve">
$ ./configure --enable-test-oom
</pre>
<h2>
<a name="basicoom" shape="rect" id="basicoom">Basic OOM testing support</a>
<a class="headerlink" href="#basicoom" title="Permalink to this headline">¶</a>
</h2>
<p>
The first step in validating OOM usage is to run a test suite
with full OOM testing enabled. This is done by setting the
<code>VIR_TEST_OOM=1</code> environment variable. The way this
works is that it runs the test once normally to "prime" any
static memory allocations. Then it runs it once more counting
the total number of memory allocations. Then it runs it in a
loop failing a different memory allocation each time. For every
memory allocation failure triggered, it expects the test case
to return an error. OOM testing is quite slow requiring each
test case to be executed O(n) times, where 'n' is the total
number of memory allocations. This results in a total number
of memory allocations of '(n * (n + 1) ) / 2'
</p>
<pre xml:space="preserve">
$ VIR_TEST_OOM=1 ./qemuxml2argvtest
1) QEMU XML-2-ARGV minimal ... OK
Test OOM for nalloc=42 .......................................... OK
2) QEMU XML-2-ARGV minimal-s390 ... OK
Test OOM for nalloc=28 ............................ OK
3) QEMU XML-2-ARGV machine-aliases1 ... OK
Test OOM for nalloc=38 ...................................... OK
4) QEMU XML-2-ARGV machine-aliases2 ... OK
Test OOM for nalloc=38 ...................................... OK
5) QEMU XML-2-ARGV machine-core-on ... OK
Test OOM for nalloc=37 ..................................... OK
...snip...
</pre>
<p>
In this output, the first line shows the normal execution and
the test number, and the second line shows the total number
of memory allocations from that test case.
</p>
<h3>
<a name="valgrind" shape="rect" id="valgrind">Tracking failures with valgrind</a>
<a class="headerlink" href="#valgrind" title="Permalink to this headline">¶</a>
</h3>
<p>
The test suite should obviously *not* crash during OOM testing.
If it does crash, then to assist in tracking down the problem
it is worth using valgrind and only running a single test case.
For example, supposing test case 5 crashed. Then re-run the
test with
</p>
<pre xml:space="preserve">
$ VIR_TEST_OOM=1 VIR_TEST_RANGE=5 ../run valgrind ./qemuxml2argvtest
...snip...
5) QEMU XML-2-ARGV machine-core-on ... OK
Test OOM for nalloc=37 ..................................... OK
...snip...
</pre>
<p>
Valgrind should report the cause of the crash - for example a
double free or use of uninitialized memory or NULL pointer
access.
</p>
<h3>
<a name="stacktraces" shape="rect" id="stacktraces">Tracking failures with stack traces</a>
<a class="headerlink" href="#stacktraces" title="Permalink to this headline">¶</a>
</h3>
<p>
With some really difficult bugs valgrind is not sufficient to
identify the cause. In this case, it is useful to identify the
precise allocation which was failed, to allow the code path
to the error to be traced. The <code>VIR_TEST_OOM</code>
env variable can be given a range of memory allocations to
test. So if a test case has 150 allocations, it can be told
to only test allocation numbers 7-10. The <code>VIR_TEST_OOM_TRACE</code>
variable can be used to print out stack traces.
</p>
<pre xml:space="preserve">
$ VIR_TEST_OOM_TRACE=2 VIR_TEST_OOM=1:7-10 VIR_TEST_RANGE=5 \
../run valgrind ./qemuxml2argvtest
5) QEMU XML-2-ARGV machine-core-on ... OK
Test OOM for nalloc=37 !virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11786 (discriminator 1)
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
!virAlloc
/home/berrange/src/virt/libvirt/src/util/viralloc.c:133
virDomainDiskDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:4790
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11797
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
!virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virXPathNodeSet
/home/berrange/src/virt/libvirt/src/util/virxml.c:609
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11805
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
!virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11808 (discriminator 1)
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12677
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12621
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:107
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:266
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:388 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:791
__libc_start_main
??:?
_start
??:?
</pre>
<h3>
<a name="noncrash" shape="rect" id="noncrash">Non-crash related problems</a>
<a class="headerlink" href="#noncrash" title="Permalink to this headline">¶</a>
</h3>
<p>
Not all memory allocation bugs result in code crashing. Sometimes
the code will be silently ignoring the allocation failure, resulting
in incorrect data being produced. For example the XML parser may
mistakenly treat an allocation failure as indicating that an XML
attribute was not set in the input document. It is hard to identify
these problems from the test suite automatically. For this, the
test suites should be run with <code>VIR_TEST_DEBUG=1</code> set
and then stderr analysed for any unexpected data. For example,
the XML conversion may show an embedded "(null)" literal, or the
test suite might complain about missing elements / attributes
in the actual vs expected data. These are all signs of bugs in
OOM handling. In the future the OOM tests will be enhanced to
validate that an error VIR_ERR_NO_MEMORY is returned for each
allocation failed, rather than some other error.
</p>
</div>
</div>
<div id="nav">
<div id="home">
<a href="../index.html">Home</a>
</div>
<div id="jumplinks">
<ul><li>
<a href="../downloads.html">Download</a>
</li><li>
<a href="../contribute.html">Contribute</a>
</li><li>
<a href="../docs.html">Learn</a>
</li></ul>
</div>
<div id="search">
<form action="../search.php" enctype="application/x-www-form-urlencoded" method="get"><div>
<input name="query" type="text" size="12" value="" />
<input name="submit" type="submit" value="Go" />
</div></form>
</div>
</div>
<div id="footer">
<div id="contact">
<h3>Contact</h3>
<ul><li>
<a href="../contact.html#email">email</a>
</li><li>
<a href="../contact.html#irc">irc</a>
</li></ul>
</div>
<div id="community">
<h3>Community</h3>
<ul><li>
<a href="https://twitter.com/hashtag/libvirt">twitter</a>
</li><li>
<a href="https://plus.google.com/communities/109522598353007505282">google+</a>
</li><li>
<a href="http://stackoverflow.com/questions/tagged/libvirt">stackoverflow</a>
</li><li>
<a href="http://serverfault.com/questions/tagged/libvirt">serverfault</a>
</li></ul>
</div>
<div id="conduct">
Participants in the libvirt project agree to abide by <a href="../governance.html#codeofconduct">the project code of conduct</a></div>
<br class="clear" />
</div>
</body>
</html>
|
