1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<!--
This file is autogenerated from logging.html.in
Do not edit this file. Changes will be lost.
-->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="stylesheet" type="text/css" href="main.css" />
<link rel="SHORTCUT ICON" href="32favicon.png" />
<title>libvirt: Logging in the library and the daemon</title>
<meta name="description" content="libvirt, virtualization, virtualization API" />
</head>
<body>
<div id="header">
<div id="headerLogo"></div>
<div id="headerSearch">
<form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><div>
<input id="query" name="query" type="text" size="12" value="" />
<input id="submit" name="submit" type="submit" value="Search" />
</div></form>
</div>
</div>
<div id="body">
<div id="menu">
<ul class="l0"><li>
<div>
<a title="Front page of the libvirt website" class="inactive" href="index.html">Home</a>
</div>
</li><li>
<div>
<a title="Details of new features and bugs fixed in each release" class="inactive" href="news.html">News</a>
</div>
</li><li>
<div>
<a title="Applications known to use libvirt" class="inactive" href="apps.html">Applications</a>
</div>
</li><li>
<div>
<a title="Get the latest source releases, binary builds and get access to the source repository" class="inactive" href="downloads.html">Downloads</a>
</div>
</li><li>
<div>
<a title="Information for users, administrators and developers" class="active" href="docs.html">Documentation</a>
<ul class="l1"><li>
<div>
<a title="How to compile libvirt" class="inactive" href="compiling.html">Compiling</a>
</div>
</li><li>
<div>
<a title="Information about deploying and using libvirt" class="active" href="deployment.html">Deployment</a>
<ul class="l2"><li>
<div>
<a title="The URI formats used for connecting to libvirt" class="inactive" href="uri.html">URI format</a>
</div>
</li><li>
<div>
<a title="Enable remote access over TCP" class="inactive" href="remote.html">Remote access</a>
</div>
</li><li>
<div>
<a title="Configure authentication for the libvirt daemon" class="inactive" href="auth.html">Authentication</a>
</div>
</li><li>
<div>
<a title="Configure access control libvirt APIs" class="inactive" href="acl.html">Access control</a>
</div>
</li><li>
<div>
<a title="Migrating guests between machines" class="inactive" href="migration.html">Migration</a>
</div>
</li><li>
<div>
<a title="Access the libvirt daemon from a native Windows client" class="inactive" href="windows.html">Windows port</a>
</div>
</li><li>
<div>
<span class="active">Logging</span>
</div>
</li><li>
<div>
<a title="Audit trail logs for host operations" class="inactive" href="auditlog.html">Audit log</a>
</div>
</li><li>
<div>
<a title="Firewall and network filter configuration" class="inactive" href="firewall.html">Firewall</a>
</div>
</li><li>
<div>
<a title="Ensuring exclusive guest access to disks" class="inactive" href="locking.html">Disk locking</a>
</div>
</li><li>
<div>
<a title="Control groups integration" class="inactive" href="cgroups.html">CGroups</a>
</div>
</li><li>
<div>
<a title="Hooks for system specific management" class="inactive" href="hooks.html">Hooks</a>
</div>
</li></ul>
</div>
</li><li>
<div>
<a title="Overview of the logical subsystems in the libvirt API" class="inactive" href="intro.html">Architecture</a>
</div>
</li><li>
<div>
<a title="Description of the XML formats used in libvirt" class="inactive" href="format.html">XML format</a>
</div>
</li><li>
<div>
<a title="Hypervisor specific driver information" class="inactive" href="drivers.html">Drivers</a>
</div>
</li><li>
<div>
<a title="Reference manual for the C public API" class="inactive" href="html/index.html">API reference</a>
</div>
</li><li>
<div>
<a title="Bindings of the libvirt API for other languages" class="inactive" href="bindings.html">Language bindings</a>
</div>
</li><li>
<div>
<a title="Working on the internals of libvirt API, driver and daemon code" class="inactive" href="internals.html">Internals</a>
</div>
</li><li>
<div>
<a title="A guide and reference for developing with libvirt" class="inactive" href="devguide.html">Development Guide</a>
</div>
</li><li>
<div>
<a title="Command reference for virsh" class="inactive" href="virshcmdref.html">Virsh Commands</a>
</div>
</li><li>
<div>
<a title="Project governance and code of conduct" class="inactive" href="governance.html">Governance</a>
</div>
</li></ul>
</div>
</li><li>
<div>
<a title="User contributed content" class="inactive" href="http://wiki.libvirt.org">Wiki</a>
</div>
</li><li>
<div>
<a title="Frequently asked questions" class="inactive" href="http://wiki.libvirt.org/page/FAQ">FAQ</a>
</div>
</li><li>
<div>
<a title="How and where to report bugs and request features" class="inactive" href="bugs.html">Bug reports</a>
</div>
</li><li>
<div>
<a title="How to contact the developers via email and IRC" class="inactive" href="contact.html">Contact</a>
</div>
</li><li>
<div>
<a title="Available test suites for libvirt" class="inactive" href="testsuites.html">Test suites</a>
</div>
</li><li>
<div>
<a title="Miscellaneous links of interest related to libvirt" class="inactive" href="relatedlinks.html">Related Links</a>
</div>
</li><li>
<div>
<a title="Overview of all content on the website" class="inactive" href="sitemap.html">Sitemap</a>
</div>
</li></ul>
</div>
<div id="content">
<h1>Logging in the library and the daemon</h1>
<p>Libvirt includes logging facilities starting from version 0.6.0,
this complements the <a href="errors.html" shape="rect">error handling</a>
mechanism and APIs to allow tracing through the execution of the
library as well as in the libvirtd daemon.</p>
<ul><li>
<a href="#log_library">Logging in the library</a>
</li><li>
<a href="#log_config">Configuring logging in the library</a>
</li><li>
<a href="#log_daemon">Logging in the daemon</a>
</li><li>
<a href="#log_syntax">Syntax for filters and output values</a>
</li><li>
<a href="#journald">Systemd journal fields</a>
<ul><li>
<a href="#journaldids">Well known message ID values</a>
</li></ul>
</li><li>
<a href="#log_examples">Examples</a>
</li></ul>
<h2>
<a name="log_library" shape="rect" id="log_library">Logging in the library</a>
<a class="headerlink" href="#log_library" title="Permalink to this headline">¶</a></h2>
<p>The logging functionalities in libvirt are based on 3 key concepts,
similar to the one present in other generic logging facilities like
log4j:</p>
<ul><li><b>log messages</b>: they are information generated at runtime by
the libvirt code. Each message includes a priority level (DEBUG = 1,
INFO = 2, WARNING = 3, ERROR = 4), a category, function name and
line number, indicating where it originated from, and finally
a formatted message. In addition the library adds a timestamp
at the beginning of the message</li><li><b>log filters</b>: a set of patterns and priorities to accept
or reject a log message. If the message category matches a filter,
the message priority is compared to the filter priority, if lower
the message is discarded, if higher the message is output. If
no filter matches, then a general priority level is applied to
all remaining messages. This allows, for example, capturing all
debug messages for the QEmu driver, but otherwise only allowing
errors to show up from other parts.</li><li><b>log outputs</b>: once a message has gone through filtering a set of
output defines where to send the message, they can also filter
based on the priority, for example it may be useful to output
all messages to a debugging file but only allow errors to be
logged through syslog.</li></ul>
<h2>
<a name="log_config" shape="rect" id="log_config">Configuring logging in the library</a>
<a class="headerlink" href="#log_config" title="Permalink to this headline">¶</a></h2>
<p>The library configuration of logging is through 3 environment variables
allowing to control the logging behaviour:</p>
<ul><li>LIBVIRT_DEBUG: it can take the four following values:
<ul><li>1 or "debug": asking the library to log every message emitted,
though the filters can be used to avoid filling up the output</li><li>2 or "info": log all non-debugging information</li><li>3 or "warn": log warnings and errors, that's the default value</li><li>4 or "error": log only error messages</li></ul></li><li>LIBVIRT_LOG_FILTERS: defines logging filters</li><li>LIBVIRT_LOG_OUTPUTS: defines logging outputs</li></ul>
<p>Note that, for example, setting LIBVIRT_DEBUG= is the same as unset. If
you specify an invalid value, it will be ignored with a warning. If you
have an error in a filter or output string, some of the settings may be
applied up to the point at which libvirt encountered the error.</p>
<h2>
<a name="log_daemon" shape="rect" id="log_daemon">Logging in the daemon</a>
<a class="headerlink" href="#log_daemon" title="Permalink to this headline">¶</a></h2>
<p>Similarly the daemon logging behaviour can be tuned using 3 config
variables, stored in the configuration file:</p>
<ul><li>log_level: accepts the following values:
<ul><li>4: only errors</li><li>3: warnings and errors</li><li>2: information, warnings and errors</li><li>1: debug and everything</li></ul></li><li>log_filters: defines logging filters</li><li>log_outputs: defines logging outputs</li></ul>
<p>When starting the libvirt daemon, any logging environment variable
settings will override settings in the config file. Command line options
take precedence over all. If no outputs are defined for libvirtd, it
will try to use</p>
<ul><li>0.10.0 or later: systemd journal, if <code>/run/systemd/journal/socket</code> exists</li><li>0.9.0 or later: file <code>/var/log/libvirt/libvirtd.log</code> if running as a daemon</li><li>before 0.9.0: syslog if running as a daemon</li><li>all versions: to stderr stream if running in the foreground</li></ul>
<p>Libvirtd does not reload its logging configuration when issued a SIGHUP.
If you want to reload the configuration, you must do a <code>service
libvirtd restart</code> or manually stop and restart the daemon
yourself.</p>
<p>Starting from 0.9.0, the daemon can save all the content of the debug
buffer to the defined error channels (or /var/log/libvirt/libvirtd.log
by default) in case of crash, this can also be activated explicitly
for debugging purposes by sending the daemon a USR2 signal:</p>
<pre xml:space="preserve">killall -USR2 libvirtd</pre>
<h2>
<a name="log_syntax" shape="rect" id="log_syntax">Syntax for filters and output values</a>
<a class="headerlink" href="#log_syntax" title="Permalink to this headline">¶</a></h2>
<p>The syntax for filters and outputs is the same for both types of
variables.</p>
<p>The format for a filter is one of:</p>
<pre xml:space="preserve">
x:name (log message only)
x:+name (log message + stack trace)</pre>
<p>where <code>name</code> is a string which is matched against source
file name, e.g., <code>remote</code>, <code>qemu</code>, or
<code>util/json</code>, the optional <code>+</code> prefix tells libvirt
to log stack trace for each message matching <code>name</code>, and
<code>x</code> is the minimal level where matching messages should
be logged:</p>
<ul><li>1: DEBUG</li><li>2: INFO</li><li>3: WARNING</li><li>4: ERROR</li></ul>
<p>Multiple filters can be defined in a single string, they just need to be
separated by spaces, e.g: <code>"3:remote 4:event"</code> to only get
warning or errors from the remote layer and only errors from the event
layer.</p>
<p>If you specify a log priority in a filter that is below the default log
priority level, messages that match that filter will still be logged,
while others will not. In order to see those messages, you must also have
an output defined that includes the priority level of your filter.</p>
<p>The format for an output can be one of those 3 forms:</p>
<ul><li><code>x:stderr</code> output goes to stderr</li><li><code>x:syslog:name</code> use syslog for the output and use the
given <code>name</code> as the ident</li><li><code>x:file:file_path</code> output to a file, with the given
filepath</li><li><code>x:journald</code> output goes to systemd journal</li></ul>
<p>In all cases the x prefix is the minimal level, acting as a filter:</p>
<ul><li>1: DEBUG</li><li>2: INFO</li><li>3: WARNING</li><li>4: ERROR</li></ul>
<p>Multiple output can be defined, they just need to be separated by
spaces, e.g.: <code>"3:syslog:libvirtd 1:file:/tmp/libvirt.log"</code>
will log all warnings and errors to syslog under the libvirtd ident
but also log all debug and information included in the
file <code>/tmp/libvirt.log</code></p>
<h2>
<a name="journald" shape="rect" id="journald">Systemd journal fields</a>
<a class="headerlink" href="#journald" title="Permalink to this headline">¶</a>
</h2>
<p>
When logging to the systemd journal, the following fields
are defined, in addition to any automatically recorded
<a href="http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html" shape="rect">standard fields</a>:
</p>
<dl><dt><code>MESSAGE</code></dt><dd>The log message string</dd><dt><code>PRIORITY</code></dt><dd>The log priority value</dd><dt><code>LIBVIRT_SOURCE</code></dt><dd>The source type, one of "file", "error", "audit", "trace", "library"</dd><dt><code>CODE_FILE</code></dt><dd>The name of the file emitting the log record</dd><dt><code>CODE_LINE</code></dt><dd>The line number of the file emitting the log record</dd><dt><code>CODE_FUNC</code></dt><dd>The name of the function emitting the log record</dd><dt><code>LIBVIRT_DOMAIN</code></dt><dd>The libvirt error domain (values from virErrorDomain enum), if LIBVIRT_SOURCE="error"</dd><dt><code>LIBVIRT_CODE</code></dt><dd>The libvirt error code (values from virErrorCode enum), if LIBVIRT_SOURCE="error"</dd></dl>
<h3>
<a name="journaldids" shape="rect" id="journaldids">Well known message ID values</a>
<a class="headerlink" href="#journaldids" title="Permalink to this headline">¶</a>
</h3>
<p>
Certain areas of the code will emit log records tagged with well known
unique id values, which are guaranteed never to change in the future.
This allows applications to identify critical log events without doing
string matching on the <code>MESSAGE</code> field.
</p>
<dl><dt><code>MESSAGE_ID=8ae2f3fb-2dbe-498e-8fbd-012d40afa361</code></dt><dd>Generated by the QEMU driver when it identifies a QEMU system
emulator binary, but is unable to extract information about its
capabilities. This is usually an indicator of a broken QEMU
build or installation. When this is emitted, the <code>LIBVIRT_QEMU_BINARY</code>
message field will provide the full path of the QEMU binary that failed.
</dd></dl>
<p>
The <code>journalctl</code> command can be used to search the journal
matching on specific message ID values
</p>
<pre xml:space="preserve">
$ journalctl MESSAGE_ID=8ae2f3fb-2dbe-498e-8fbd-012d40afa361 --output=json
{ ...snip...
"LIBVIRT_SOURCE" : "file",
"PRIORITY" : "3",
"CODE_FILE" : "qemu/qemu_capabilities.c",
"CODE_LINE" : "2770",
"CODE_FUNC" : "virQEMUCapsLogProbeFailure",
"MESSAGE_ID" : "8ae2f3fb-2dbe-498e-8fbd-012d40afa361",
"LIBVIRT_QEMU_BINARY" : "/bin/qemu-system-xtensa",
"MESSAGE" : "Failed to probe capabilities for /bin/qemu-system-xtensa:" \
"internal error: Child process (LC_ALL=C LD_LIBRARY_PATH=/home/berrange" \
"/src/virt/libvirt/src/.libs PATH=/usr/lib64/ccache:/usr/local/sbin:" \
"/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin HOME=/root " \
"USER=root LOGNAME=root /bin/qemu-system-xtensa -help) unexpected " \
"exit status 127: /bin/qemu-system-xtensa: error while loading shared " \
"libraries: libglapi.so.0: cannot open shared object file: No such " \
"file or directory\n" }
</pre>
<h2>
<a name="log_examples" shape="rect" id="log_examples">Examples</a>
<a class="headerlink" href="#log_examples" title="Permalink to this headline">¶</a></h2>
<p>For example setting up the following:</p>
<pre xml:space="preserve">export LIBVIRT_DEBUG=1
export LIBVIRT_LOG_OUTPUTS="1:file:virsh.log"</pre>
<p>and then running virsh will accumulate the logs in the
<code>virsh.log</code> file in a way similar to:</p>
<pre xml:space="preserve">14:29:04.771: debug : virInitialize:278 : register drivers
14:29:04.771: debug : virRegisterDriver:618 : registering Test as driver 0</pre>
<p>the messages are timestamped, there is also the level recorded,
if debug the name of the function is also printed and then the formatted
message. This should be sufficient to at least get a precise idea of
what is happening and where things are going wrong, allowing to then
put the correct breakpoints when running under a debugger.</p>
<p>To activate full debug of the libvirt entry points, utility
functions and the QEmu/KVM driver, set:</p>
<pre xml:space="preserve">log_filters="1:libvirt 1:util 1:qemu"
log_outputs="1:file:/var/log/libvirt/libvirtd.log"</pre>
<p>in libvirtd.conf and restart the daemon will allow to
gather a copious amount of debugging traces for the operations done
in those areas.</p>
</div>
</div>
<div id="footer">
<p id="sponsor">
Sponsored by:<br /><a href="http://et.redhat.com/"><img src="et.png" alt="Project sponsored by Red Hat Emerging Technology" /></a></p>
</div>
</body>
</html>
|
