1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<!--
This file is autogenerated from locking-lockd.html.in
Do not edit this file. Changes will be lost.
-->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="stylesheet" type="text/css" href="main.css" />
<link rel="SHORTCUT ICON" href="32favicon.png" />
<title>libvirt: Virtual machine lock manager, virtlockd plugin</title>
<meta name="description" content="libvirt, virtualization, virtualization API" />
</head>
<body>
<div id="body">
<div id="content">
<h1>Virtual machine lock manager, virtlockd plugin</h1>
<ul><li>
<a href="#background">virtlockd background</a>
</li><li>
<a href="#sanlock">virtlockd daemon setup</a>
</li><li>
<a href="#lockdplugin">libvirt lockd plugin configuration</a>
</li><li>
<a href="#qemuconfig">QEMU/KVM driver configuration</a>
</li></ul>
<p>
This page describes use of the <code>virtlockd</code>
service as a <a href="locking.html" shape="rect">lock driver</a>
plugin for virtual machine disk mutual exclusion.
</p>
<h2>
<a name="background" shape="rect" id="background">virtlockd background</a>
<a class="headerlink" href="#background" title="Permalink to this headline">¶</a>
</h2>
<p>
The virtlockd daemon is a single purpose binary which
focuses exclusively on the task of acquiring and holding
locks on behalf of running virtual machines. It is
designed to offer a low overhead, portable locking
scheme can be used out of the box on virtualization
hosts with minimal configuration overheads. It makes
use of the POSIX fcntl advisory locking capability
to hold locks, which is supported by the majority of
commonly used filesystems.
</p>
<h2>
<a name="sanlock" shape="rect" id="sanlock">virtlockd daemon setup</a>
<a class="headerlink" href="#sanlock" title="Permalink to this headline">¶</a>
</h2>
<p>
In most OS, the virtlockd daemon itself will not require
any upfront configuration work. It is installed by default
when libvirtd is present, and a systemd socket unit is
registered such that the daemon will be automatically
started when first required. With OS that predate systemd
though, it will be necessary to start it at boot time,
prior to libvirtd being started. On RHEL/Fedora distros,
this can be achieved as follows
</p>
<pre xml:space="preserve">
# chkconfig virtlockd on
# service virtlockd start
</pre>
<p>
The above instructions apply to the instance of virtlockd
that runs privileged, and is used by the libvirtd daemon
that runs privileged. If running libvirtd as an unprivileged
user, it will always automatically spawn an instance of
the virtlockd daemon unprivileged too. This requires no
setup at all.
</p>
<h2>
<a name="lockdplugin" shape="rect" id="lockdplugin">libvirt lockd plugin configuration</a>
<a class="headerlink" href="#lockdplugin" title="Permalink to this headline">¶</a>
</h2>
<p>
Once the virtlockd daemon is running, or setup to autostart,
the next step is to configure the libvirt lockd plugin.
There is a separate configuration file for each libvirt
driver that is using virtlockd. For QEMU, we will edit
<code>/etc/libvirt/qemu-lockd.conf</code>
</p>
<p>
The default behaviour of the lockd plugin is to acquire locks
directly on the virtual disk images associated with the guest
<disk> elements. This ensures it can run out of the box
with no configuration, providing locking for disk images on
shared filesystems such as NFS. It does not provide any cross
host protection for storage that is backed by block devices,
since locks acquired on device nodes in /dev only apply within
the host. It may also be the case that the filesystem holding
the disk images is not capable of supporting fcntl locks.
</p>
<p>
To address these problems it is possible to tell lockd to
acquire locks on an indirect file. Essentially lockd will
calculate the SHA256 checksum of the fully qualified path,
and create a zero length file in a given directory whose
filename is the checksum. It will then acquire a lock on
that file. Assuming the block devices assigned to the guest
are using stable paths (eg /dev/disk/by-path/XXXXXXX) then
this will allow for locks to apply across hosts. This
feature can be enabled by setting a configuration setting
that specifies the directory in which to create the lock
files. The directory referred to should of course be
placed on a shared filesystem (eg NFS) that is accessible
to all hosts which can see the shared block devices.
</p>
<pre xml:space="preserve">
$ su - root
# augtool -s set \
/files/etc/libvirt/qemu-lockd.conf/file_lockspace_dir \
"/var/lib/libvirt/lockd/files"
</pre>
<p>
If the guests are using either LVM and SCSI block devices
for their virtual disks, there is a unique identifier
associated with each device. It is possible to tell lockd
to use this UUID as the basis for acquiring locks, rather
than the SHA256 sum of the filename. The benefit of this
is that the locking protection will work even if the file
paths to the given block device are different on each
host.
</p>
<pre xml:space="preserve">
$ su - root
# augtool -s set \
/files/etc/libvirt/qemu-lockd.conf/scsi_lockspace_dir \
"/var/lib/libvirt/lockd/scsi"
# augtool -s set \
/files/etc/libvirt/qemu-lockd.conf/lvm_lockspace_dir \
"/var/lib/libvirt/lockd/lvm"
</pre>
<p>
It is important to remember that the changes made to the
<code>/etc/libvirt/qemu-lockd.conf</code> file must be
propagated to all hosts before any virtual machines are
launched on them. This ensures that all hosts are using
the same locking mechanism
</p>
<h2>
<a name="qemuconfig" shape="rect" id="qemuconfig">QEMU/KVM driver configuration</a>
<a class="headerlink" href="#qemuconfig" title="Permalink to this headline">¶</a>
</h2>
<p>
The QEMU driver is capable of using the virtlockd plugin
since the release <span>1.0.2</span>.
The out of the box configuration, however, currently
uses the <strong>nop</strong> lock manager plugin.
To get protection for disks, it is thus necessary
to reconfigure QEMU to activate the <strong>lockd</strong>
driver. This is achieved by editing the QEMU driver
configuration file (<code>/etc/libvirt/qemu.conf</code>)
and changing the <code>lock_manager</code> configuration
tunable.
</p>
<pre xml:space="preserve">
$ su - root
# augtool -s set /files/etc/libvirt/qemu.conf/lock_manager lockd
# service libvirtd restart
</pre>
<p>
Every time you start a guest, the virtlockd daemon will acquire
locks on the disk files directly, or in one of the configured
lookaside directories based on SHA256 sum. To check that locks
are being acquired as expected, the <code>lslocks</code> tool
can be run.
</p>
</div>
</div>
<div id="nav">
<div id="home">
<a href="index.html">Home</a>
</div>
<div id="jumplinks">
<ul><li>
<a href="downloads.html">Download</a>
</li><li>
<a href="contribute.html">Contribute</a>
</li><li>
<a href="docs.html">Learn</a>
</li></ul>
</div>
<div id="search">
<form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><div>
<input name="query" type="text" size="12" value="" />
<input name="submit" type="submit" value="Go" />
</div></form>
</div>
</div>
<div id="footer">
<div id="contact">
<h3>Contact</h3>
<ul><li>
<a href="contact.html#email">email</a>
</li><li>
<a href="contact.html#irc">irc</a>
</li></ul>
</div>
<div id="community">
<h3>Community</h3>
<ul><li>
<a href="https://twitter.com/hashtag/libvirt">twitter</a>
</li><li>
<a href="https://plus.google.com/communities/109522598353007505282">google+</a>
</li><li>
<a href="http://stackoverflow.com/questions/tagged/libvirt">stackoverflow</a>
</li><li>
<a href="http://serverfault.com/questions/tagged/libvirt">serverfault</a>
</li></ul>
</div>
<div id="conduct">
Participants in the libvirt project agree to abide by <a href="governance.html#codeofconduct">the project code of conduct</a></div>
<br class="clear" />
</div>
</body>
</html>
|
