=head1 NAME

virt-sparsify - Make a virtual machine disk sparse

=head1 SYNOPSIS

 virt-sparsify [--options] indisk outdisk

 virt-sparsify [--options] --in-place disk

=head1 DESCRIPTION

Virt-sparsify is a tool which can make a virtual machine disk (or any
disk image) sparse a.k.a. thin-provisioned.  This means that free
space within the disk image can be converted back to free space on the
host.

Virt-sparsify can locate and sparsify free space in most filesystems
(eg. ext2/3/4, btrfs, NTFS, etc.), and also in LVM physical volumes.

Virt-sparsify can also convert between some disk formats, for example
converting a raw disk image to a thin-provisioned qcow2 image.

Virt-sparsify can operate on any disk image, not just ones from
virtual machines.  However if a virtual machine has multiple disks
and uses volume management, then virt-sparsify will work but not be
very effective (L<http://bugzilla.redhat.com/887826>).

=head2 IMPORTANT NOTE ABOUT SPARSE OUTPUT IMAGES

If the input is raw, then the default output is raw sparse.  B<You
must check the output size using a tool that understands sparseness>
such as C<du -sh>.  It can make a huge difference:

 $ ls -lh test1.img
 -rw-rw-r--. 1 rjones rjones 100M Aug  8 08:08 test1.img
 $ du -sh test1.img
 3.6M	test1.img

(Compare the apparent size B<100M> vs the actual size B<3.6M>)

=head2 IMPORTANT LIMITATIONS

=over 4

=item *

The virtual machine I<must be shut down> before using this tool.

=item *

Virt-sparsify may require up to 2x the virtual size of the source disk
image (1 temporary copy + 1 destination image).  This is in the worst
case and usually much less space is required.

If you are using the I<--in-place> option, then large amounts of
temporary space are B<not> required.

=item *

Virt-sparsify cannot resize disk images.  To do that, use
L<virt-resize(1)>.

=item *

Virt-sparsify cannot handle encrypted disks.  Libguestfs supports
encrypted disks, but encrypted disks themselves cannot be sparsified.

=item *

Virt-sparsify cannot yet sparsify the space between partitions.  Note
that this space is often used for critical items like bootloaders so
it's not really unused.

=item *

In copy mode, qcow2 internal snapshots are not copied over to the
destination image.

=back

You may also want to read the manual pages for the associated tools
L<virt-filesystems(1)> and L<virt-df(1)> before starting.

=head1 EXAMPLES

Typical usage is:

 virt-sparsify indisk outdisk

which copies C<indisk> to C<outdisk>, making the output sparse.
C<outdisk> is created, or overwritten if it already exists.  The
format of the input disk is detected (eg. qcow2) and the same format
is used for the output disk.

To convert between formats, use the I<--convert> option:

 virt-sparsify disk.raw --convert qcow2 disk.qcow2

Virt-sparsify tries to zero and sparsify free space on every
filesystem it can find within the source disk image.  You can get it
to ignore (don't zero free space on) certain filesystems by doing:

 virt-sparsify --ignore /dev/sda1 indisk outdisk

See L<virt-filesystems(1)> to get a list of filesystems within a disk
image.

Since virt-sparsify E<ge> 1.26, you can now sparsify a disk image
in place by doing:

 virt-sparsify --in-place disk.img

=head1 OPTIONS

=over 4

=item B<--help>

Display help.

=item B<--check-tmpdir> B<ignore>

=item B<--check-tmpdir> B<continue>

=item B<--check-tmpdir> B<warn>

=item B<--check-tmpdir> B<fail>

Check if L</TMPDIR> or I<--tmp> directory has enough space to complete
the operation.  This is just an estimate.

If the check indicates a problem, then you can either:

=over 4

=item *

B<ignore> it,

=item *

print a warning and B<continue>,

=item *

B<warn> and wait for the user to press the Return key
(this is the default), or:

=item *

B<fail> and exit.

=back

You cannot use this option and I<--in-place> together.

=item B<--colors>

=item B<--colours>

Use ANSI colour sequences to colourize messages.  This is the default
when the output is a tty.  If the output of the program is redirected
to a file, ANSI colour sequences are disabled unless you use this
option.

=item B<--compress>

Compress the output file.  This I<only> works if the output format is
C<qcow2>.

You cannot use this option and I<--in-place> together.

=item B<--convert> raw

=item B<--convert> qcow2

=item B<--convert> [other formats]

Use C<output-format> as the format for the destination image.  If this
is not specified, then the input format is used.

Supported and known-working output formats are: C<raw>, C<qcow2>, C<vdi>.

You can also use any format supported by the L<qemu-img(1)> program,
eg. C<vmdk>, but support for other formats is reliant on qemu.

Specifying the I<--convert> option is usually a good idea, because
then virt-sparsify doesn't need to try to guess the input format.

For fine-tuning the output format, see: I<--compress>, I<-o>.

You cannot use this option and I<--in-place> together.

=item B<--echo-keys>

When prompting for keys and passphrases, virt-sparsify normally turns
echoing off so you cannot see what you are typing.  If you are not
worried about Tempest attacks and there is no one else in the room
you can specify this flag to see what you are typing.

=item B<--format> raw

=item B<--format> qcow2

Specify the format of the input disk image.  If this flag is not
given then it is auto-detected from the image itself.

If working with untrusted raw-format guest disk images, you should
ensure the format is always specified.

=item B<--ignore> filesystem

=item B<--ignore> volgroup

Ignore the named filesystem.

When not using I<--in-place>: Free space on the filesystem will not be
zeroed, but existing blocks of zeroes will still be sparsified.

When using I<--in-place>, the filesystem is ignored completely.

In the second form, this ignores the named volume group.  Use the
volume group name without the F</dev/> prefix, eg. I<--ignore vg_foo>

You can give this option multiple times.

=item B<--in-place>

Do in-place sparsification instead of copying sparsification.
See L</IN-PLACE SPARSIFICATION> below.

__INCLUDE:key-option.pod__

__INCLUDE:keys-from-stdin-option.pod__

=item B<--machine-readable>

=item B<--machine-readable>=format

This option is used to make the output more machine friendly
when being parsed by other programs.  See
L</MACHINE READABLE OUTPUT> below.

=item B<-o> option[,option,...]

Pass I<-o> option(s) to the L<qemu-img(1)> command to fine-tune the
output format.  Options available depend on the output format (see
I<--convert>) and the installed version of the qemu-img program.

You should use I<-o> at most once.  To pass multiple options, separate
them with commas, eg:

 virt-sparsify --convert qcow2 \
   -o cluster_size=512,preallocation=metadata ...

You cannot use this option and I<--in-place> together.

=item B<-q>

=item B<--quiet>

This disables progress bars and other unnecessary output.

=item B<--tmp> block_device

=item B<--tmp> dir

In copying mode only, use the named device or directory as the
location of the temporary overlay (see also L</TMPDIR> below).

If the parameter given is a block device, then the block device is
written to directly.
B<Note this erases the existing contents of the block device>.

If the parameter is a directory, then this is the same as setting the
L</TMPDIR> environment variable.

You cannot use this option and I<--in-place> together.

=item B<--tmp> prebuilt:file

In copying mode only, the specialized option I<--tmp prebuilt:file>
(where C<prebuilt:> is a literal string) causes virt-sparsify to use
the qcow2 C<file> as temporary space.

=over 4

=item *

The file B<must> be freshly formatted as qcow2, with indisk as the
backing file.

=item *

If you rerun virt-sparsify, you B<must> recreate the file before
each run.

=item *

Virt-sparsify does not delete the file.

=back

This option is used by oVirt which requires a specially formatted
temporary file.

=item B<-v>

=item B<--verbose>

Enable verbose messages for debugging.

=item B<-V>

=item B<--version>

Display version number and exit.

=item B<-x>

Enable tracing of libguestfs API calls.

=item B<--zero> partition

=item B<--zero> logvol

Zero the contents of the named partition or logical volume in the
guest.  All data on the device is lost, but sparsification is
excellent!  You can give this option multiple times.

=back

=head1 IN-PLACE SPARSIFICATION

Since virt-sparsify E<ge> 1.26, the tool is able to do in-place
sparsification (instead of copying from an input disk to an output
disk).  This is more efficient.  It is not able to recover quite as
much space as copying sparsification.

To use this mode, specify a disk image which will be modified in
place:

 virt-sparsify --in-place disk.img

Some options are not compatible with this mode: I<--convert>,
I<--compress> and I<-o> because they require wholesale disk format
changes; I<--check-tmpdir> because large amounts of temporary space
are not required.

In-place sparsification works using discard (a.k.a trim or unmap)
support.

=head1 MACHINE READABLE OUTPUT

The I<--machine-readable> option can be used to make the output more
machine friendly, which is useful when calling virt-sparsify from
other programs, GUIs etc.

There are two ways to use this option.

Firstly use the option on its own to query the capabilities of the
virt-sparsify binary.  Typical output looks like this:

 $ virt-sparsify --machine-readable
 virt-sparsify
 ntfs
 btrfs

A list of features is printed, one per line, and the program exits
with status 0.

Secondly use the option in conjunction with other options to make the
regular program output more machine friendly.

At the moment this means:

=over 4

=item 1.

Progress bar messages can be parsed from stdout by looking for this
regular expression:

 ^[0-9]+/[0-9]+$

=item 2.

The calling program should treat messages sent to stdout (except for
progress bar messages) as status messages.  They can be logged and/or
displayed to the user.

=item 3.

The calling program should treat messages sent to stderr as error
messages.  In addition, virt-sparsify exits with a non-zero status
code if there was a fatal error.

=back

All versions of virt-sparsify have supported the I<--machine-readable>
option.

It is possible to specify a format string for controlling the output;
see L<guestfs(3)/ADVANCED MACHINE READABLE OUTPUT>.

=head1 WINDOWS 8

Windows 8 "fast startup" can prevent virt-sparsify from working.
See L<guestfs(3)/WINDOWS HIBERNATION AND WINDOWS 8 FAST STARTUP>.

=head1 ENVIRONMENT VARIABLES

=over 4

=item TMPDIR

Location of the temporary directory used for the potentially large
temporary overlay file.

In virt-sparsify E<ge> 1.28, you can override this environment
variable using the I<--tmp> option.

You should ensure there is enough free space in the worst case for a
full copy of the source disk (I<virtual> size), or else set C<$TMPDIR>
to point to another directory that has enough space.

This defaults to F</tmp>.

Note that if C<$TMPDIR> is a tmpfs (eg. if F</tmp> is on tmpfs, or if
you use C<TMPDIR=/dev/shm>), tmpfs defaults to a maximum size of
I<half> of physical RAM.  If virt-sparsify exceeds this, it will hang.
The solution is either to use a real disk, or to increase the maximum
size of the tmpfs mountpoint, eg:

 mount -o remount,size=10G /tmp

If you are using the I<--in-place> option, then large amounts of
temporary space are B<not> required.

=back

For other environment variables, see L<guestfs(3)/ENVIRONMENT VARIABLES>.

=head1 EXIT STATUS

This program returns 0 if the operation completed without errors.
(This doesn't necessarily mean that space could be freed up.)

A non-zero exit code indicates an error.

If the exit code is C<3> and the I<--in-place> option was used, that
indicates that discard support is not available in libguestfs, so
copying mode must be used instead.

=head1 SEE ALSO

L<virt-df(1)>,
L<virt-filesystems(1)>,
L<virt-resize(1)>,
L<virt-rescue(1)>,
L<guestfs(3)>,
L<guestfish(1)>,
L<truncate(1)>,
L<fallocate(1)>,
L<qemu-img(1)>,
L<http://libguestfs.org/>.

=head1 AUTHOR

Richard W.M. Jones L<http://people.redhat.com/~rjones/>

=head1 COPYRIGHT

Copyright (C) 2011-2020 Red Hat Inc.