File: simplesnap.html

package info (click to toggle)
simplesnap 1.0.4%2Bnmu1
links: PTS, VCS
area: main
in suites: buster
size: 256 kB
sloc: sh: 418; makefile: 55
file content (1415 lines) | stat: -rw-r--r-- 34,021 bytes
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN""http://www.w3.org/TR/html4/loose.dtd">
<HTML
><HEAD
><TITLE
>simplesnap</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.79"></HEAD
><BODY
CLASS="REFENTRY"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><H1
><A
NAME="AEN1"
></A
>simplesnap</H1
><DIV
CLASS="REFNAMEDIV"
><A
NAME="AEN14"
></A
><H2
>Name</H2
>simplesnap&nbsp;--&nbsp;Simple and powerful way to send ZFS snapshots across a
    network</DIV
><DIV
CLASS="REFSYNOPSISDIV"
><A
NAME="AEN17"
></A
><H2
>Synopsis</H2
><P
><B
CLASS="COMMAND"
>simplesnap</B
>  [<CODE
CLASS="OPTION"
>--sshcmd</CODE
>
        <TT
CLASS="REPLACEABLE"
><I
>COMMAND</I
></TT
>] [<CODE
CLASS="OPTION"
>--wrapcmd</CODE
> <TT
CLASS="REPLACEABLE"
><I
>COMMAND</I
></TT
>] [<CODE
CLASS="OPTION"
>--local</CODE
>] [<CODE
CLASS="OPTION"
>--backupdataset</CODE
>
        <TT
CLASS="REPLACEABLE"
><I
>DATASET</I
></TT
>
         [<CODE
CLASS="OPTION"
>--datasetdest</CODE
> <TT
CLASS="REPLACEABLE"
><I
>DEST</I
></TT
>]]  <CODE
CLASS="OPTION"
>--store</CODE
> <TT
CLASS="REPLACEABLE"
><I
>STORE</I
></TT
>   <CODE
CLASS="OPTION"
>--setname</CODE
>
        <TT
CLASS="REPLACEABLE"
><I
>NAME</I
></TT
>   <CODE
CLASS="OPTION"
>--host</CODE
>
      <TT
CLASS="REPLACEABLE"
><I
>HOST</I
></TT
> </P
><P
><B
CLASS="COMMAND"
>simplesnap</B
>   <CODE
CLASS="OPTION"
>--check</CODE
> <TT
CLASS="REPLACEABLE"
><I
>TIMEFRAME</I
></TT
>   <CODE
CLASS="OPTION"
>--store</CODE
> <TT
CLASS="REPLACEABLE"
><I
>STORE</I
></TT
>   <CODE
CLASS="OPTION"
>--setname</CODE
>
        <TT
CLASS="REPLACEABLE"
><I
>NAME</I
></TT
>  [<CODE
CLASS="OPTION"
>--host</CODE
>
      <TT
CLASS="REPLACEABLE"
><I
>HOST</I
></TT
>]</P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN57"
></A
><H2
>Description</H2
><P
>      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> is a simple way to send ZFS snapshots across a
      network.  Although it can serve many purposes, its primary goal
      is to manage backups from one ZFS filesystem to a backup
      filesystem also running ZFS, using incremental backups to
      minimize network traffic and disk usage.
    </P
><P
>      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> is <SPAN
CLASS="emphasis"
><I
CLASS="EMPHASIS"
>FLEXIBLE</I
></SPAN
>; it is designed to
      perfectly compliment snapshotting tools, permitting rotating
      backups with arbitrary retention periods.  It lets multiple
      machines back up a single target, lets one machine back up
      multiple targets, and keeps it all straight.
    </P
><P
>      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> is <SPAN
CLASS="emphasis"
><I
CLASS="EMPHASIS"
>EASY</I
></SPAN
>; there is no
      configuration file needed.  One ZFS property is available to
      exclude datasets/filesystems.  ZFS datasets are automatically
      discovered on machines being backed up.
    </P
><P
>      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> is <SPAN
CLASS="emphasis"
><I
CLASS="EMPHASIS"
>SAFE</I
></SPAN
>; it is robust in the
      face of interrupted transfers, and needs little help to keep
      running.
    </P
><P
>      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> is <SPAN
CLASS="emphasis"
><I
CLASS="EMPHASIS"
>SECURE</I
></SPAN
>; unlike many similar
      tools, it does not require full root access to the machines
      being backed up.  It runs only a small wrapper as root, and the
      wrapper has only three commands it implements.
    </P
><DIV
CLASS="REFSECT2"
><A
NAME="AEN73"
></A
><H3
>Feature List</H3
><P
>        Besides the above, <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>:
      </P
><P
></P
><UL
><LI
><P
>            Does one thing and does it well.  It is designed to be used with
            a snapshot auto-rotator on both ends (such as zfSnap).  simplesnap
            will transfer snapshots made by other tools, but will not destroy
            them on either end.
          </P
></LI
><LI
><P
>            Requires ssh public key authorization to the host being backed up,
            but does not require permission to run arbitrary commands.  It has
            a wrapper to run on the backup host, written in bash, which accepts
            only three operations and performs them simply.  It is suitable for
            a locked-down authorized_keys file.
          </P
></LI
><LI
><P
>            Creates minimal snapshots for its own internal purposes, generally
            leaving no more than 1 or 2 per dataset, and reaps them
            automatically without touching others.
          </P
></LI
><LI
><P
>            Is a small program, easily audited.  In fact, most of the code is devoted to sanity-checking, security, and error
            checking.
          </P
></LI
><LI
><P
>            Automatically discovers what datasets to back up from the remote.
            Uses a user-defined zfs property to exclude filesystems that should
            not be backed up.
          </P
></LI
><LI
><P
>            Logs copiously to syslog on all hosts involved in backups.
          </P
></LI
><LI
><P
>            Intelligently supports a single machine being backed up by multiple
            backup hosts, or onto multiple sets of backup media (when, for
            instance, backup media is cycled into offsite storage)
          </P
></LI
></UL
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN92"
></A
><H3
>Method of Operation</H3
><P
>        <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>'s operation is very simple.
      </P
><P
>        The <B
CLASS="COMMAND"
>simplesnap</B
> program runs on the machine
        that stores the backups -- we'll call it the backuphost.
        There is a restricted remote command wrapper called
        <B
CLASS="COMMAND"
>simplesnapwrap</B
> that runs on the machine
        being backed up -- we'll call it the activehost.
        <B
CLASS="COMMAND"
>simplesnapwrap</B
> is never invoked directly by
        the end-user; it is always called remotely by
        <B
CLASS="COMMAND"
>simplesnap</B
>.
      </P
><P
>        With <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>, the backuphost always connects to the
        activehost -- never the other way round.
      </P
><P
>        <B
CLASS="COMMAND"
>simplesnap</B
> runs in the backuphost, and
        first connects to the <B
CLASS="COMMAND"
>simplesnapwrap</B
> on the
        activehost and asks it for a
        list of the ZFS datasets ("listfs" operation).  <B
CLASS="COMMAND"
>simplesnapwrap</B
>
        responds with a list of all ZFS datasets that were not flagged for
        exclusion.
      </P
><P
>        Next, <B
CLASS="COMMAND"
>simplesnap</B
> connects back to <B
CLASS="COMMAND"
>simplesnapwrap</B
> once for each dataset
        to be backed up -- the "sendback" operation.  <B
CLASS="COMMAND"
>simplesnap</B
> passes along
        to it only two things: the setname and the dataset
        (filesystem) name.
      </P
><P
>        <B
CLASS="COMMAND"
>simplesnapwrap</B
> looks to see if there is an existing simplesnap
        snapshot corresponding to that <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
>.  If not, it creates one and
        sends it as a full, non-incremental backup.  That completes the job
        for that dataset.
      </P
><P
>        If there is an existing snapshot for that <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
>, simplesnapwrap
        creates a new one, constructing the snapshot name containing a
        timestamp and the <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
>, then sends an incremental, using the oldest
        snapshot from that setname as the basis for zfs send -I.
      </P
><P
>        After the backuphost has observed <B
CLASS="COMMAND"
>zfs receive</B
> exiting without error,
        it contacts <B
CLASS="COMMAND"
>simplesnapwrap</B
> once more and requests the "reap"
        operation.  This cleans up the old snapshots for the given <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
>,
        leaving only the most recent.  This is a separate operation in
        <B
CLASS="COMMAND"
>simplesnapwrap</B
> ensuring that even if the transmission is interrupted,
        still it will be OK in the end because <B
CLASS="COMMAND"
>zfs receive -F</B
> is used, and the
        data will come across next time.
      </P
><P
>        The idea is that some system like <B
CLASS="COMMAND"
>zfSnap</B
> will be used on both ends to
        make periodic snapshots and clean them up.  One can use careful prefix
        names with zfSnap to use different prefixes on each activehost, and
        then implement custom cleanup rules with -F on the holderhost.
      </P
></DIV
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN125"
></A
><H2
>Quick Start</H2
><P
>      This section will describe how a first-time <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> user
      can get up and running quickly.  It assumes you already have
      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> installed and working on your system.  If not,
      please follow the instructions in the
      <TT
CLASS="FILENAME"
>INSTALL.txt</TT
> file in the source
      distribution.
    </P
><P
>      As above, I will refer to the machine storing the backups as the
      "backuphost" and the machine being backed up as the
      "activehost".
    </P
><P
>      First, on the backuphost, as root, generate an ssh keypair that
      will be used exclusively for <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>.
    </P
><P
>      <B
CLASS="COMMAND"
>ssh-keygen -t rsa -f ~/.ssh/id_rsa_simplesnap</B
>
    </P
><P
>      When prompted for a passphrase, leave it empty.
    </P
><P
>      Now, on the activehost, edit or create a file called
      <TT
CLASS="FILENAME"
>~/.ssh/authorized_keys</TT
>.  Initialize it with the content of
      <TT
CLASS="FILENAME"
>~/.ssh/id_rsa_simplesnap.pub</TT
> from the backuphost.  (Or, add to the
      end, if you already have lines in the file.)  Then, at the
      beginning of that one very long line, add text like this:
    </P
><PRE
CLASS="PROGRAMLISTING"
>command="/usr/sbin/simplesnapwrap",from="1.2.3.4",
no-port-forwarding,no-X11-forwarding,no-pty </PRE
><P
>      (I broke that line into two for readability, but this must all
      be on a single line in your file.)
    </P
><P
>      The <TT
CLASS="REPLACEABLE"
><I
>1.2.3.4</I
></TT
> is the IP address that
      connections from the backuphost
      will appear to come from.  It may be omitted if the IP is not static,
      but it affords a little extra security.  The line will wind up looking
      like:
    </P
><PRE
CLASS="PROGRAMLISTING"
>command="/usr/sbin/simplesnapwrap",from="1.2.3.4",
no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAA....</PRE
><P
>      (Again, this should all be on one huge line.)
    </P
><P
>      If there are any ZFS datasets you do not want to be backed up, set
      <CODE
CLASS="PARAMETER"
>org.complete.simplesnap:exclude</CODE
> property
      on the activehost
      to <CODE
CLASS="PARAMETER"
>on</CODE
>.  For instance:
    </P
><P
>      <B
CLASS="COMMAND"
>zfs set org.complete.simplesnap:exclude=on
      tank/junkdata</B
>
    </P
><P
>      Now, back on the backuphost, you should be able to run:
    </P
><P
>      <B
CLASS="COMMAND"
>ssh -i ~/.ssh/id_rsa_simplesnap activehost</B
>
    </P
><P
>      say yes when asked if you want to add the key to the known_hosts
      file.  At this point, you should see output containing:
    </P
><P
>      "simplesnapwrap: This program is to be run from ssh."
    </P
><P
>      If you see that, then simplesnapwrap was properly invoked
      remotely.
    </P
><P
>      Now, create a ZFS filesystem to hold your backups.  For
      instance:
    </P
><P
>      <B
CLASS="COMMAND"
>zfs create tank/simplesnap</B
>
    </P
><P
>      I often recommend compression for <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> datasets, so:
    </P
><P
>      <B
CLASS="COMMAND"
>zfs set compression=lz4 tank/simplesnap</B
>
    </P
><P
>      (If that gives an error, use compression=on instead.)
    </P
><P
>      Now, you can run the backup:
    </P
><P
>      <B
CLASS="COMMAND"
>simplesnap --host activehost --setname mainset
        --store tank/simplesnap
        --sshcmd "ssh -i /root/.ssh/id_rsa_simplesnap"
      </B
>
    </P
><P
>      You can monitor progress in <TT
CLASS="FILENAME"
>/var/log/syslog</TT
>.  If all goes well, you
      will see filesystems start to be populated under
      <TT
CLASS="FILENAME"
>tank/simplesnap/host</TT
>.
    </P
><P
>      Simple!
    </P
><P
>      Now, go test that you have the data you expected to: look at
      your <TT
CLASS="REPLACEABLE"
><I
>STORE</I
></TT
> filesystems and make sure
      they have everything expected.  Test repeatedly over time that
      you can restore as you expect from your backups.
    </P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN174"
></A
><H2
>Advanced: SETNAME usage</H2
><P
>      Most people will always use the same <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
>.  The <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
> is used to
      track and name the snapshots on the remote end.  simplesnap tries to always
      leave one snapshot on the remote, to serve as the base for a future
      incremental.
    </P
><P
>      In some situations, you may have multiple bases for incrementals.  The
      two primary examples are two different backup servers backing up the
      same machine, or having two sets of backup media and rotating them to
      offsite storage.  In these situations, you will have to keep different
      snapshots on the activehost for the different backups, since they will
      be current to different points in time.
    </P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN180"
></A
><H2
>Options</H2
><P
>      All <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> options begin with two dashes (`--').  Most take
      a parameter, which is to be separated from the option by a
      space.  The equals sign is not a valid separator for
      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>.
    </P
><P
>      The normal <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> mode is backing up.  An alternative
      check mode is available, which requires fewer parameters.  This
      mode is described below.
    </P
><P
></P
><DIV
CLASS="VARIABLELIST"
><DL
><DT
><CODE
CLASS="OPTION"
>--backupdataset <TT
CLASS="REPLACEABLE"
><I
>DATASET</I
></TT
></CODE
></DT
><DD
><P
>            Normally, <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> automatically obtains a list of
            datasets to back up from the remote, and backs up all of
            them except those that define the
            <CODE
CLASS="PARAMETER"
>org.complete.simplesnap:exclude=on</CODE
>
            property.  With this option, <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> does not bother
            to ask the remote for a list of datasets, and instead
            backs up only the one precise
            <TT
CLASS="REPLACEABLE"
><I
>DATASET</I
></TT
> given.  For now, ignored when
            <CODE
CLASS="OPTION"
>--check</CODE
> is given, but that may change in
            the future.  It would be best to not specify this option
            with --check for now.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--check <TT
CLASS="REPLACEABLE"
><I
>TIMEFRAME</I
></TT
></CODE
></DT
><DD
><P
>            Do not back up, but check existing backups.  If any
            datasets' newest backup is older than
          <TT
CLASS="REPLACEABLE"
><I
>TIMEFRAME</I
></TT
>, print an error and
            exit with a nonzero code.  Scans all hosts unless a
            specific host is given with <CODE
CLASS="OPTION"
>--host</CODE
>.  The
            parameter is in the format given to GNU <SPAN
CLASS="APPLICATION"
>date</SPAN
>(1); for
            instance,
            --check "30 days ago".  Remember to enclose it in quotes
            if it contains spaces.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--datasetdest
        <TT
CLASS="REPLACEABLE"
><I
>DEST</I
></TT
></CODE
></DT
><DD
><P
>            Valid only with <CODE
CLASS="OPTION"
>--backupdataset</CODE
>, gives a
            specific destination for the backup, whith may be outside
            the <TT
CLASS="REPLACEABLE"
><I
>STORE</I
></TT
>.  The <TT
CLASS="REPLACEABLE"
><I
>STORE</I
></TT
>
            must still exist, as it is used for storing lockfiles and
            such.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--host</CODE
>
        <TT
CLASS="REPLACEABLE"
><I
>HOST</I
></TT
></DT
><DD
><P
>            Gives the name of the host to back up.  This is both
            passed to ssh and used to name the backup sets.
          </P
><P
>            In a few situations, one may not wish to use the same name
            for both.  It is recommend to use the Host and HostName
            options in <TT
CLASS="FILENAME"
>~/.ssh/config</TT
> to configure aliases in this
            situation.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--local</CODE
></DT
><DD
><P
>            Specifies that the host being backed up is local to the
            machine.  Do not use ssh to contact it, and invoke the
            wrapper directly.  You would not need to
            give <CODE
CLASS="OPTION"
>--sshcmd</CODE
> in this case.  For
            instance: <B
CLASS="COMMAND"
>simplesnap --local --store
                /bakfs/simplesnap --host server1 --setname bak1</B
>
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--sshcmd
            <TT
CLASS="REPLACEABLE"
><I
>COMMAND</I
></TT
></CODE
></DT
><DD
><P
>            Gives the command to use to connect to the remote host.
            Defaults to "ssh".  It may be used to select an
            alternative configuration file or keypair.  Remember to
            quote it per your shell if it contains spaces.  For example: 
            --sshcmd "ssh -i /root/.id_rsa_simplesnap".  This command
            is ignored when <CODE
CLASS="OPTION"
>--local</CODE
> or
            <CODE
CLASS="OPTION"
>--check</CODE
> is given.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--setname <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
></CODE
></DT
><DD
><P
>            Gives the backup set name.  Can just be a made-up word if
            multiple sets are not needed; for instance, the hostname of
            the backup server.  This is used as part of the snapshot
            name.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--store
        <TT
CLASS="REPLACEABLE"
><I
>STORE</I
></TT
></CODE
></DT
><DD
><P
>Gives the ZFS dataset name where the data
            will be stored.  Should not begin with a slash.  The
            mountpoint will be obtained from the ZFS subsystem.
            Always required.
          </P
></DD
><DT
><CODE
CLASS="OPTION"
>--wrapcmd
            <TT
CLASS="REPLACEABLE"
><I
>COMMAND</I
></TT
></CODE
></DT
><DD
><P
>            Gives the path to simplesnapwrap (which must be on the
            remote machine unless <CODE
CLASS="OPTION"
>--local</CODE
> is given).
            Not usually relevant, since the
            <CODE
CLASS="PARAMETER"
>command</CODE
> parameter in
            <TT
CLASS="FILENAME"
>~root/.ssh/authorized_keys</TT
> gives the
            path.  Default: "simplesnapwrap"
          </P
></DD
></DL
></DIV
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN261"
></A
><H2
>Backup Interrogation</H2
><P
>      Since <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> stores backups in standard ZFS datasets, you
      can use standard ZFS tools to obtain information about backups.
      Here are some examples.
    </P
><DIV
CLASS="REFSECT2"
><A
NAME="AEN265"
></A
><H3
>Space used per host</H3
><P
>        Try something like this:
      </P
><PRE
CLASS="PROGRAMLISTING"
># zfs list -r -d 1 tank/store
NAME               USED  AVAIL  REFER  MOUNTPOINT
tank/store         540G   867G    34K  /tank/store
tank/store/host1   473G   867G    32K  /tank/store/host1
tank/store/host2  54.9G   867G    32K  /tank/store/host2
tank/store/host3  12.2G   867G    31K  /tank/store/host3</PRE
><P
>        Here, you can see that the total size of the <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> data
        is 540G - the USED value from the top level.  In this example,
        host1 was using the most space -- 473G -- and host3 the least --
        12.2G.  There is 867G available on this zpool for backups.
      </P
><P
>        The <CODE
CLASS="PARAMETER"
>-r</CODE
> parameter to <B
CLASS="COMMAND"
>zfs
          list</B
> requests a recursive report, but the
        <CODE
CLASS="PARAMETER"
>-d 1</CODE
> parameter sets a maximum depth of 1
        -- so you can see just the top-level hosts without all their
        component datasets.
      </P
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN275"
></A
><H3
>Space used by a host</H3
><P
>        Let's say that you had the above example, and want to drill down
        into more detail.  Perhaps, for instance, we continue the above
        example and drill down into host2:
      </P
><PRE
CLASS="PROGRAMLISTING"
># zfs list -r tank/store/host2
NAME                                 USED  AVAIL  REFER  MOUNTPOINT
tank/store/host2                    54.9G   867G    32K  /tank/...
tank/store/host2/tank               49.8G   867G    32K  /tank/...
tank/store/host2/tank/home          7.39G   867G  6.93G  /tank/...
tank/store/host2/tank/vm            42.4G   867G    30K  /tank/...
tank/store/host2/tank/vm/vm1        32.0G   867G  29.7G  -
tank/store/host2/tank/vm/vm2        10.4G   867G  10.4G  -
tank/store/host2/rpool              5.12G   867G    32K  /tank/...
tank/store/host2/rpool/misc          521M   867G   521M  /tank/...
tank/store/host2/rpool/host2-1      4.61G   867G    33K  /tank/...
tank/store/host2/rpool/host2-1/ROOT  317M   867G   312M  /tank/...
tank/store/host2/rpool/host2-1/usr  3.76G   867G  3.76G  /tank/...
tank/store/host2/rpool/host2-1/var   554M   867G   401M  /tank/...</PRE
><P
>        I've trimmed the "mountpoint" column here so it doesn't get
        too wide for the screen.
      </P
><P
>        You see here the same 54.9G used as in the previous example,
        but now you can trace it down.  There were two zpools on
        host2: tank and rpool.  Most of the backup space -- 49.8G of
        the 54.9G -- is used by tank, and only 5.12G by rpool.  And in
        tank, 42.4G is used by vm.  Tracing it down, of that 42.4G
        used by vm, 32G is in vm1 and 10.4G in vm2.  Notice how the
        values at each level of the tree include their descendents.
      </P
><P
>        So in this example, vm1 and vm2 are zvols corresponding to
        virtual machines, and clearly take up a lot of space.  Notice
        how vm1 says it uses 32.0G but in the refer column, it only
        refers to 29.7G?  That means that the latest backup for vm2
        used 29.7G, but when you add in the snapshots for that
        dataset, the total space consumed is 32.0G.
      </P
><P
>        Let's look at an alternative view that will make the size
        consumed by snapshots more clear:
      </P
><PRE
CLASS="PROGRAMLISTING"
># zfs list -o space -r tank/store/host2
NAME                         AVAIL   USED  USEDSNAP  USEDDS  USEDCHILD
.../host2                     867G  54.9G         0     32K      54.9G
.../host2/tank                867G  49.8G         0     32K      49.8G
.../host2/tank/home           867G  7.39G      474M   6.93G          0
.../host2/tank/vm             867G  42.4G       50K     30K      42.4G
.../host2/tank/vm/vm1         867G  32.0G     2.35G   29.7G          0
.../host2/tank/vm/vm1         867G  10.4G       49K   10.4G          0
.../host2/rpool               867G  5.12G         0     32K      5.12G
.../host2/rpool/misc          867G   521M       51K    521M          0
.../host2/rpool/host2-1       867G  4.61G       51K     33K      4.61G
.../host2/rpool/host2-1/ROOT  867G   317M     5.44M    312M          0
.../host2/rpool/host2-1/usr   867G  3.76G      208K   3.76G          0
.../host2/rpool/host2-1/var   867G   554M      153M    401M          0</PRE
><P
>        (Again, I've trimmed some irrelevant columns from this
        output.)
      </P
><P
>        The AVAIL and USED columns are the same as before, but now you
        have a breakdown of what makes up the USED column.  USEDSNAP
        is the space used by the snapshots of that particular
        dataset.  USEDDS is the space used by that dataset directly --
        the same value as was in REFER before.  And USEDCHILD is the
        space used by descendents of that dataset.  
      </P
><P
>        The USEDSNAP column is the
        easiest way to see the impact your retention policies have on
        your backup space consumption.
      </P
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN287"
></A
><H3
>Viewing snapshots of a dataset</H3
><P
>        Let's take one example from
        before -- the 153M of snapshots in host2-1/var, and see what we
        can find.
      </P
><PRE
CLASS="PROGRAMLISTING"
># zfs list -t snap -r tank/store/host2/rpool/host2-1/var 
NAME                                              USED  AVAIL  REFER
...
.../var@host2-hourly-2014-02-11_05.17.02--2d       76K      -   402M
.../var@host2-hourly-2014-02-11_06.17.01--2d       77K      -   402M
.../var@host2-hourly-2014-02-11_07.17.01--2d     18.8M      -   402M
.../var@host2-daily-2014-02-11_07.17.25--1w        79K      -   402M
.../var@host2-hourly-2014-02-11_08.17.01--2d      156K      -   402M
.../var@host2-monthly-2014-02-11_09.01.36--1m     114K      -   402M
...</PRE
><P
>        In this output, the REFER column is the amount of data pointed
        to by that snapshot -- that is, the size of /var at the moment
        the snapshot is made.  And the USED column is the amount of
        space that would be freed if just that snapshot were deleted.
      </P
><P
>        Note this important point: it is normal for the sum of the
        values in the USED column to be less than the space consumed
        by the snapshots of the datasets as reported by USEDSNAP in
        the previous example.  The reason is that the USED column is
        the data unique to that one snapshot.  If, for instance, 100MB
        of data existed on the system being backed up for
        three hours yesterday, each snapshot could very well show less
        than 100KB used, because that 100MB isn't unique to a
        particular snapshot.  Until, that is, two of the three
        snapshots referncing the 100MB data are destroyed; then the
        USED value of the last one referencing it will suddenly jump
        to 100MB higher because now it references unique data.
      </P
><P
>        One other point -- an indication that the last backup was
        successfully transmitted is the presence of a
        __simplesnap_...__ snapshot at the end of the list.  Do not
        delete it.
      </P
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN294"
></A
><H3
>Finding what changed over time</H3
><P
>        The <B
CLASS="COMMAND"
>zfs diff</B
> command can let you see what
        changed over time -- either across a single snapshot, or
        across many.  Let's take a look.
      </P
><PRE
CLASS="PROGRAMLISTING"
># zfs diff .../var@host2-hourly-2014-02-11_05.17.02--2d \
  .../var@host2-hourly-2014-02-11_06.17.01--2d \
  | sort -k2 | less
M	/tank/store/host2/rpool/host2-1/var/log/Xorg.0.log
M	/tank/store/host2/rpool/host2-1/var/log/auth.log
M	/tank/store/host2/rpool/host2-1/var/log/daemon.log
...
M	/tank/store/host2/rpool/host2-1/var/spool/anacron/cron.daily
M	/tank/store/host2/rpool/host2-1/var/spool/anacron/cron.monthly
M	/tank/store/host2/rpool/host2-1/var/spool/anacron/cron.weekly
M	/tank/store/host2/rpool/host2-1/var/tmp</PRE
><P
>        Here you can see why there was just a few KB of changes in
        that snapshot: mostly just a little bit of logging was
        happening on the system.  Now let's inspect the larger
        snapshot:
      </P
><PRE
CLASS="PROGRAMLISTING"
># zfs diff .../var@host2-hourly-2014-02-11_07.17.01--2d \
   .../var@host2-daily-2014-02-11_07.17.25--1w \
   | sort -k2 | less
M	/tank/store/host2/rpool/host2-1/var/backups
+	/tank/store/host2/rpool/host2-1/var/backups/dpkg.status.0
-	/tank/store/host2/rpool/host2-1/var/backups/dpkg.status.0
+	/tank/store/host2/rpool/host2-1/var/backups/dpkg.status.1.gz
R	/tank/store/host2/rpool/host2-1/var/backups/dpkg.status.1.gz -&#62; /tank/store/host2/rpool/host2-1/var/backups/dpkg.status.2.gz
R	/tank/store/host2/rpool/host2-1/var/backups/dpkg.status.2.gz -&#62; /tank/store/host2/rpool/host2-1/var/backups/dpkg.status.3.gz
...
M	/tank/store/host2/rpool/host2-1/var/cache/apt
R	/tank/store/host2/rpool/host2-1/var/cache/apt/pkgcache.bin.KdsMLu -&#62; /tank/store/host2/rpool/host2-1/var/cache/apt/pkgcache.bin</PRE
><P
>        Here you can see some file rotation going on, and a temporary
        file being renamed to permanent.  Normal daily activity on a
        system, but now you know what was taking up space.
      </P
></DIV
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN302"
></A
><H2
>Warnings, Cautions, and Good Practices</H2
><DIV
CLASS="REFSECT2"
><A
NAME="AEN304"
></A
><H3
>Importance of Testing</H3
><P
>        Any backup scheme should be tested carefully before being
        relied upon to serve its intended purpose.  This item is not
        <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>-specific, but pertains to every backup solution:
        test that you are backing up the data you expect to before you
        need it.
      </P
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN308"
></A
><H3
>Use of zfs receive -F</H3
><P
>        In order to account for various situations that could lead to
        divergence of filesystems, including the simple act of mounting
        them, <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> always uses <B
CLASS="COMMAND"
>zfs receive
          -F</B
>.  Any local changes you make to the <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>
        store datasets will be lost at any time.  If you need to make
        local changes there, it is best to copy them elsewhere.
      </P
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN314"
></A
><H3
>Extraneous Snapshot Buildup</H3
><P
>        Since <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> sends all snapshots, it is possible that
        locally-created snapshots made outside of your rotation scheme
        will also be sent to your backuphost.  These may not be
        automatically reaped there, and may stick around.  An example
        at the end of the
        <TT
CLASS="FILENAME"
>cron.daily.simplesnap.backuphost</TT
> file
        included with <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> is one way to check for these.
        They could automatically be reaped with <B
CLASS="COMMAND"
>zfs
        destroy</B
> as well, but this must be carefully tuned to
        local requirements, so an example of doign that is
        intentionally not supplied with the distribution.
      </P
></DIV
><DIV
CLASS="REFSECT2"
><A
NAME="AEN321"
></A
><H3
>Internal simplesnap snapshots</H3
><P
>        <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> creates snapshots beginning with __simplesnap_
        followed by your <TT
CLASS="REPLACEABLE"
><I
>SETNAME</I
></TT
>.  Do not
        create, remove, or alter these snapshots in any way, either on
        the activehost or the backuphost.   Doing so may lead to
        unpredictable side-effects.
      </P
></DIV
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN326"
></A
><H2
>Bugs</H2
><P
>      Ordinarily, an interrupted transfer is no problem for
      <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
>.  However, the very first transfer of a dataset
      poses a bit of a problem, since the <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> wrapper can't
      detect failure in this one special case.  If your first transfer
      gets interrupted, simply zfs destroy the __simplesnap_...__
      snapshot on the activehost and rerun.  NEVER DESTROY
      __simplesnap SNAPSHOTS IN ANY OTHER SITUATION!
    </P
><P
>      If, by way of the 
      <CODE
CLASS="PARAMETER"
>org.complete.simplesnap:exclude</CODE
>
      property or the <CODE
CLASS="OPTION"
>--backupdataset</CODE
> or 
      <CODE
CLASS="OPTION"
>--datasetdest</CODE
> parameters, you do not request a
      parent dataset to be backed up, but do request a descendent
      dataset to be backed up, you may get an error on the first
      backup
      because the
      dataset tree leading to the destination location for that
      dataset has not yet been created.  <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> performs only
      the narrow actions you request.  Running an appropriate
      <B
CLASS="COMMAND"
>zfs create</B
> command will rectify the
      situation.
    </P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN337"
></A
><H2
>See Also</H2
><P
>zfSnap (1), zfs (8).</P
><P
>      The <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> homepage: <A
HREF="https://github.com/jgoerzen/simplesnap"
TARGET="_top"
>https://github.com/jgoerzen/simplesnap</A
>
    </P
><P
>      The examples included with the <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> distribution, or on
      its homepage.
    </P
><P
>      The zfSnap package compliments <SPAN
CLASS="APPLICATION"
>simplesnap</SPAN
> perfectly.  Find it
      at
      <A
HREF="https://github.com/graudeejs/zfSnap"
TARGET="_top"
>https://github.com/graudeejs/zfSnap</A
>.
    </P
></DIV
><DIV
CLASS="REFSECT1"
><A
NAME="AEN348"
></A
><H2
>AUTHOR</H2
><P
>This software and manual page was written by John Goerzen <CODE
CLASS="EMAIL"
>&#60;<A
HREF="mailto:jgoerzen@complete.org"
>jgoerzen@complete.org</A
>&#62;</CODE
>.
      Permission is
      granted to copy, distribute and/or modify this document under
      the terms of the <ACRONYM
CLASS="ACRONYM"
>GNU</ACRONYM
> General Public License, Version 3 any
      later version published by the Free Software Foundation.  The
      complete text of the GNU General Public License is included in
      the file COPYING in the source distribution.
    </P
></DIV
></BODY
></HTML
>