File: benchmark_logs.html

package info (click to toggle)
dar 2.8.1-3
links: PTS, VCS
area: main
in suites: forky, sid
size: 12,024 kB
sloc: cpp: 86,219; sh: 6,978; ansic: 895; makefile: 489; python: 242; csh: 115; perl: 43; sed: 16
file content (5333 lines) | stat: -rw-r--r-- 221,639 bytes
parent folder | download | duplicates (4)
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <link href="style.css" rel="stylesheet">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta content="text/html; charset=ISO-8859-1" http-equiv="content-type">
    <title>Benchmarking backup tools</title>
  </head>
  <body>

    <div class=top>
      <img alt="DAR's Documentation" src="dar_s_doc.jpg" style="float:left;">
      <div class=h1>
	<h1>Benchmarking backup tools</h1>
	<h1>Test logs</h1>
      </div>
    </div>

    <h2>Introduction</h2>
    <p>
      This document keep trace of the tests performed that lead to the summarized results presented in the
      <a href="benchmark.html">benchmark document</a>. Several <a href="#scripts">helper scripts</a>
      detailed at the end of this document have been necessary to obtain these results.
    </p>


    <h2>Testing Context</h2>
    <h3>Software under test</h3>
    <ul>
      <li><b>dar</b> version 2.7.0</li>
      <li><b>rsync</b> version 3.1.3</li>
      <li><b>tar</b> GNU tar 1.30 under Linux and GNU tar (gdar) 1.32 under FreeBSD</li>
    </ul>

    <h3>Hardware used for testing</h3>

    <p>
      Performance tests has been performed on an <a href="https://www.hpe.com">HPE</a> Proliant server
      (a <i>ProLiant XL230a Gen9</i> running two <i>Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz</i> processors) running
      a Devuan beowulf Linux system. Other tests have been run from Virtual Machines (<a href="https://www.freebsd.org">FreeBSD</a> 12.1,
      <a href="https://www.devuan.org">Devuan</a> 3.0.0)
      of a <a href="https://www.proxmox.com/en/">Proxmox</a> hypervisor running one on an
      <i>Intel Core i5-7400 (3 GHz)</i> based computer.
    </p>

    <h2>Test logs</h2>

    <h3><a name"completness">Completness</a> of backup and restoration</h3>
    <h4>Dar</h4>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# ./build_test_tree.bash  SRC</b>
	1024+0 records in
	1024+0 records out
	1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00395381 s, 265 MB/s
	1024+0 records in
	1024+0 records out
	1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621889 s, 169 MB/s
	1+0 records in
	1+0 records out
	1 byte copied, 0.000386102 s, 2.6 kB/s
	<b>root@terre:/mnt/localdisk/Benchmark_tools# dar -c backup -R SRC</b>


	--------------------------------------------
	14 inode(s) saved
	including 3 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 14
	--------------------------------------------
	EA saved for 1 inode(s)
	FSA saved for 5 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/localdisk/Benchmark_tools# mkdir DST</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# dar -x backup -R DST</b>


	--------------------------------------------
	14 inode(s) restored
	including 3 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 14
	--------------------------------------------
	EA restored for 1 inode(s)
	FSA restored for 1 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      We simply performed backup of <code>SRC</code> directory with dar's default options, then
      restore this backup into the <code>DST</code> directory, let's now compare <code>SRC</code> and <code>DST</code> contents:
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk# du -s SRC DST</b>
	2068    SRC
	<e>1048</e>    DST
	<b>root@terre:/mnt/localdisk#</b>
    </code>
    <p>
      The space used by <code>DST</code> is less than the space used by <code>SRC</code>! At first
      we could beleive that not all data could be restored, let's looking for the explanation:
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk#ls -iRl SRC DST</b>
	DST:
	total 1044
	414844 drwxr-xr-x  2 root   root     4096 Oct 28 11:09 SUB
	414850 drwxr-xr-x  2 root   root     4096 Oct 22 11:09 dev
	414848 brw-r--r--  1 root   root     2, 1 Oct 28 11:09 fd1
	414842 crw-r--r--  1 root   root     3, 1 Oct 28 11:09 null
	<e class="red">414841</e> prw-r--r--  2 root   root        0 Oct 28 11:09 pipe
	414840 -rw-rwxr--+ 1 root   root  1048576 Oct 28 11:09 plain_zeroed
	414849 -rw-r--r--  1 nobody root  1048582 Oct 28 11:09 random
	<e>414843</e> -rw-r--r--  2 root   root 10240000 Oct 28 11:09 sparse_file

	DST/SUB:
	total 4
	<e class="red">414841</e> prw-r--r-- 2 root root          0 Oct 28 11:09 hard_linked_pipe
	<e class="blue">414846</e> srw-rw-rw- 2 root root          0 Oct 12 23:00 hard_linked_socket
	<e>414843</e> -rw-r--r-- 2 root root   10240000 Oct 28 11:09 hard_linked_sparse_file
	414845 lrwxrwxrwx 1 root root          6 Oct 28 11:09 symlink-broken -> random
	414847 lrwxrwxrwx 1 bin  daemon        9 Oct 28 11:09 symlink-valid -> ../random

	DST/dev:
	total 0
	<e class="blue">414846</e> srw-rw-rw- 2 root root 0 Oct 12 23:00 log

	SRC:
	total 2064
	411386 drwxr-xr-x  2 root   root     4096 Oct 28 11:09 SUB
	414836 drwxr-xr-x  2 root   root     4096 Oct 22 11:09 dev
	414835 brw-r--r--  1 root   root     2, 1 Oct 28 11:09 fd1
	414834 crw-r--r--  1 root   root     3, 1 Oct 28 11:09 null
	414832 prw-r--r--  2 root   root        0 Oct 28 11:09 pipe
	414826 -rw-rwxr--+ 1 root   root  1048576 Oct 28 11:09 plain_zeroed
	414827 -rw-r--r--  1 nobody root  1048582 Oct 28 11:09 random
	414828 -rw-r--r--  2 root   root 10240000 Oct 28 11:09 sparse_file

	SRC/SUB:
	total 4
	414832 prw-r--r-- 2 root root          0 Oct 28 11:09 hard_linked_pipe
	414837 srw-rw-rw- 2 root root          0 Oct 12 23:00 hard_linked_socket
	414828 -rw-r--r-- 2 root root   10240000 Oct 28 11:09 hard_linked_sparse_file
	414830 lrwxrwxrwx 1 root root          6 Oct 28 11:09 symlink-broken -> random
	414831 lrwxrwxrwx 1 bin  daemon        9 Oct 28 11:09 symlink-valid -> ../random

	SRC/dev:
	total 0
	414837 srw-rw-rw- 2 root root 0 Oct 12 23:00 log
	<b>root@terre:/mnt/localdisk#</b>
    </code>
    <p>
      All files are present in <code>DST</code> and use the expected space usage, as reported by the <code>ls</code> command.
      We can also see that the hard linked inode were properly restored for plain file, named pipe and unix socket: the
      inode number in first column is the same (see colorized output above).
    </p>
    <p>
      Maybe something is missing elsewhere?
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk# getfacl SRC/plain_zeroed DST/plain_zeroed</b>
	# file: SRC/plain_zeroed
	# owner: root
	# group: root
	user::rw-
	<e>user:nobody:rwx</e>
	group::r--
	mask::rwx
	other::r--

	# file: DST/plain_zeroed
	# owner: root
	# group: root
	user::rw-
	<e>user:nobody:rwx</e>
	group::r--
	mask::rwx
	other::r--

	<b>root@terre:/mnt/localdisk# getfattr -d SRC/plain_zeroed DST/plain_zeroed</b>
	# file: SRC/plain_zeroed
	<e>user.hello="hello world!!!"</e>

	# file: DST/plain_zeroed
	<e>user.hello="hello world!!!"</e>

	<b>root@terre:/mnt/localdisk/Benchmark_tools# lsattr SRC/plain_zeroed DST/plain_zeroed</b>
	<e>s---i-d-------e----</e> SRC/plain_zeroed
	<e>s---i-d-------e----</e> DST/plain_zeroed
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      To summarize:
    </p>
    <ul>
      <li>user and group ownership are restored</li>
      <li>permission are set correctly</li>
      <li>ACL are properly restored</li>
      <li>Extended Attributes also</li>
      <li>file system specific attributes are too</li>
      <li>hard links are restored</li>
    </ul>
    <p>
      So what? Let's rerun <code>du</code> file by file:
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# du -B1 SRC/* DST/*</b>
	8192    SRC/SUB
	4096    SRC/dev
	0       SRC/fd1
	0       SRC/null
	<e>1048576 SRC/plain_zeroed</e>
	1052672 SRC/random
	8192    DST/SUB
	4096    DST/dev
	0       DST/fd1
	0       DST/null
	<e>4096    DST/plain_zeroed</e>
	1052672 DST/random
	<b>root@terre:/mnt/localdisk/Benchmark_tools# ls -l SRC/plain_zeroed DST/plain_zeroed</b>
	-rw-rwxr--+ 1 root root <e>1048576</e> Oct 21 18:40 DST/plain_zeroed
	-rw-rwxr--+ 1 root root <e>1048576</e> Oct 21 18:40 SRC/plain_zeroed
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      OK here is the explanation: <code>plain_zeroed</code> file was using 1048576 bytes of disk space in SRC
      and consumes only 4096 bytes in DST, but it has its file size is still officially 1048576, it has thus become
      now a sparse file (not all zeroed bytes are stored).
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# diff -s SRC/plain_zeroed DST/plain_zeroed</b>
	Files SRC/plain_zeroed and DST/plain_zeroed are <e>identical</e>
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>

    <p>
      But nothing changes from the user point of view, the restoration process with dar just optimized
      the space usage.
    </p>
    <p>
      Let's continue checking the inode dates. As you know, Unix inode have several dates:
      </p>
    <ul>
      <li><b>atime</b>, the access time, gives the last time the file's data has been accessed (read)</li>
      <li><b>mtime</b>, the modification time, gives the last time the file's data has been modified (wrote)</li>
      <li><b>ctime</b>, the change time, gives the last time the file's metadata in other word the inode properties (ownership, ACL, permissions, dates, ...) have been modified</li>
      <li><b>btime</b>, the birth time or yet creation time, gives the time the file has been created on the current filesystem, this date is not present on all Unix system</li>
    </ul>

    <p>
      The <code>ls -iRl</code> command we used so far does only show the <i>mtime</i> date moreover with
      a time accuracy of only one minute, while modern systems provide nanosecond precision. For that
      reason we will use the <code>stat</code> command instead to have all dates at the system time accuracy:
    </p>

    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# stat SRC/random DST/random</b>
	File: SRC/random
	Size: 1048576         Blocks: 2048       IO Block: 4096   regular file
	Device: 802h/2050d      Inode: 414840      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (65534/  nobody)   Gid: (    0/    root)
	<e>Access: 2020-10-22 12:13:01.813319506 +0200</e>
	<e>Modify: 2020-10-22 12:12:57.765328555 +0200</e>
	<e class=red>Change: 2020-10-22 12:12:59.805323991 +0200</e>
	Birth: -
	File: DST/random
	Size: 1048576         Blocks: 2048       IO Block: 4096   regular file
	Device: 802h/2050d      Inode: 414889      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (65534/  nobody)   Gid: (    0/    root)
	<e>Access: 2020-10-22 12:13:01.813319506 +0200</e>
	<e>Modify: 2020-10-22 12:12:57.765328555 +0200</e>
	<e class=red>Change: 2020-10-22 12:14:34.877131738 +0200</e>
	Birth: -
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>

    <p>
      From the above output we see that:
    </p>
    <ul>
      <li>atime is restored</li>
      <li>mtime is restored</li>
      <li>ctime is not restored</li>
    </ul>

    <p>
      As we targeted this benchmark mainly for Linux which has not yet the <code>btime</code> available
      (Well some Linux <a href="https://en.wikipedia.org/wiki/Comparison_of_file_systems#Metadata">file systems</a> support
      <i>btime</i> but its access is not yet fully available to applications), we will thus momentarily change to a BSD system
      to play with <code>btime</code>. BSD systems include MACOS X, FreeBSD, NetBSD, butterflyBSD,... we will use FreeBSD here.
      Under FreeBSD, the <code>stat</code> command is not as easy to read as under Linux, however it is
      very flexible which we will leverage to mimic the Linux output:
    </p>

    <code class=block>
	<b>root@FreeBSD:~denis # which mystat</b>
	mystat:          aliased to <e>stat -f "%N%nAccess: %Sa%nModify: %Sm%nChange: %Sc%nBirth: %SB%n" !*</e>
	<b>root@FreeBSD:~denis # mystat SRC/random</b>
	SRC/random
	Access: Oct 27 13:28:41 2020
	Modify: Oct 22 15:34:07 2020
	Change: Oct 22 15:34:09 2020
	<e>Birth: Oct 22 15:34:07 2020</e>
	<b>root@FreeBSD:~denis # dar -c backup -R SRC -q</b>
	<b>root@FreeBSD:~denis # mkdir DST</b>
	<b>root@FreeBSD:~denis # dar -x backup -R DST -q</b>
	<b>root@FreeBSD:~denis # mystat DST/random</b>
	DST/random
	Access: Oct 27 13:28:41 2020
	Modify: Oct 22 15:34:07 2020
	Change: Oct 27 13:31:50 2020
	<e>Birth: Oct 22 15:34:07 2020</e>
	<b>root@FreeBSD:~denis #</b>
    </code>

    <p>
      In conclusion <i>dar</i> also saves and restores <code>btime</code> properly.
    </p>


    <h4>Rsync</h4>
    <p>
      Let's do the same we did previously using <b>rsync</b>. We start by copying <code>SRC</code> directory to <code>DST</code>:
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# chattr -i DST/plain_zeroed</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# rm -rf DST</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# mkdir DST</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# rsync -arvHAXS SRC/* DST</b>
	sending incremental file list
	created directory DST
	fd1
	null
	pipe
	plain_zeroed
	random
	SUB/
	SUB/hard_linked_pipe => pipe
	SUB/hard_linked_socket
	SUB/hard_linked_sparse_file
	SUB/symlink-broken -> random
	SUB/symlink-valid -> ../random
	dev/
	dev/log => SUB/hard_linked_socket
	sparse_file => SUB/hard_linked_sparse_file

	sent 12,340,852 bytes  received 198 bytes  24,682,100.00 bytes/sec
	total size is 22,577,173  speedup is 1.83
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      First note, the backup and restoration is done in one step, where <i>dar</i> was decorelating the backup operation
      from the restoration operation. The resulting backup needs not software to be restored (<code>DST</code>
      is a copy of <code>SRC</code>). For dar to reach the same result (without using storage for the backup) this
      implies two dar commands: <code> dar -c - -R SRC | dar -x - --sequential-read -R DST</code>. The situation is
      similar with <code>tar</code>, you need two commands to perform the same task: <code>tar -cf - | tar -xf -</code>
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# du -s SRC DST</b>
	2056    SRC
	<e>1028</e>    DST
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      Here too, the restored data uses less space than the original data, sparse file have been taken into
      account (need specifying -S option) and space optimization of non sparse file is performed.
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# ls -iRl SRC DST</b>
	DST:
	total 12060
	414843 drwxr-xr-x  2 root   root     4096 Oct 28 11:09 SUB
	414844 drwxr-xr-x  2 root   root     4096 Oct 22 11:09 dev
	414840 brw-r--r--  1 root   root     2, 1 Oct 28 11:09 fd1
	414841 crw-r--r--  1 root   root     3, 1 Oct 28 11:09 null
	<e class=red>414842</e> prw-r--r--  2 root   root        0 Oct 28 11:09 pipe
	414848 -rw-rwxr--+ 1 root   root  1048576 Oct 28 11:09 plain_zeroed
	414849 -rw-r--r--  1 nobody root  1048582 Oct 28 11:09 random
	<e>414850</e> -rw-r--r--  2 root   root 10240000 Oct 28 11:09 sparse_file

	DST/SUB:
	total 10000
	<e class=red>414842</e> prw-r--r-- 2 root root          0 Oct 28 11:09 hard_linked_pipe
	<e class=blue>414845</e> srw-rw-rw- 2 root root          0 Oct 12 23:00 hard_linked_socket
	<e>414850</e> -rw-r--r-- 2 root root   10240000 Oct 28 11:09 hard_linked_sparse_file
	414846 lrwxrwxrwx 1 root root          6 Oct 28 11:09 symlink-broken -> random
	414847 lrwxrwxrwx 1 bin  daemon        9 Oct 28 11:09 symlink-valid -> ../random

	DST/dev:
	total 0
	<e class=blue>414845</e> srw-rw-rw- 2 root root 0 Oct 12 23:00 log

	SRC:
	total 2064
	411386 drwxr-xr-x  2 root   root     4096 Oct 28 11:09 SUB
	414836 drwxr-xr-x  2 root   root     4096 Oct 22 11:09 dev
	414835 brw-r--r--  1 root   root     2, 1 Oct 28 11:09 fd1
	414834 crw-r--r--  1 root   root     3, 1 Oct 28 11:09 null
	414832 prw-r--r--  2 root   root        0 Oct 28 11:09 pipe
	414826 -rw-rwxr--+ 1 root   root  1048576 Oct 28 11:09 plain_zeroed
	414827 -rw-r--r--  1 nobody root  1048582 Oct 28 11:09 random
	414828 -rw-r--r--  2 root   root 10240000 Oct 28 11:09 sparse_file

	SRC/SUB:
	total 4
	414832 prw-r--r-- 2 root root          0 Oct 28 11:09 hard_linked_pipe
	414837 srw-rw-rw- 2 root root          0 Oct 12 23:00 hard_linked_socket
	414828 -rw-r--r-- 2 root root   10240000 Oct 28 11:09 hard_linked_sparse_file
	414830 lrwxrwxrwx 1 root root          6 Oct 28 11:09 symlink-broken -> random
	414831 lrwxrwxrwx 1 bin  daemon        9 Oct 28 11:09 symlink-valid -> ../random

	SRC/dev:
	total 0
	414837 srw-rw-rw- 2 root root 0 Oct 12 23:00 log
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      All files are present in <code>DST</code> and use the expected space usage, as reported by the <code>ls</code>.
      We can also see that all three hard linked inode (plain file, socket and named pipe) are restored properly.
      So we can suspect the cause of the size difference to be linked with sparse files:
    </p>
    <p>
      Let's now check file's metadata:
    </p>
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# getfacl SRC/plain_zeroed DST/plain_zeroed</b>
	# file: SRC/plain_zeroed
	# owner: root
	# group: root
	user::rw-
	<e>user:nobody:rwx</e>
	group::r--
	mask::rwx
	other::r--

	# file: DST/plain_zeroed
	# owner: root
	# group: root
	user::rw-
	<e>user:nobody:rwx</e>
	group::r--
	group::rwx
	other::r--

	<b>root@terre:/mnt/localdisk/Benchmark_tools# getfattr -d SRC/plain_zeroed DST/plain_zeroed</b>
	# file: SRC/plain_zeroed
	<e>user.hello="hello world!!!"</e>

	# file: DST/plain_zeroed
	<e>user.hello="hello world!!!"</e>
	<br/>

	<b>root@terre:/mnt/localdisk/Benchmark_tools# lsattr SRC/plain_zeroed DST/plain_zeroed</b>
	<e class=red>s---i-d-------e----</e> SRC/plain_zeroed
	<e class=red>--------------e----</e> DST/plain_zeroed
	<b>root@terre:/mnt/localdisk/Benchmark_tools#stat SRC/random DST/random</b>
	File: SRC/random
	Size: 1048582         Blocks: 2056       IO Block: 4096   regular file
	Device: 802h/2050d      Inode: 414827      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (65534/  nobody)   Gid: (    0/    root)
	<e class=red>Access: 2020-10-28 11:09:59.977926733 +0100</e>
	<e>Modify: 2020-10-28 11:09:57.973931318 +0100</e>
	<e class=red>Change: 2020-10-28 11:09:57.973931318 +0100</e>
	Birth: -
	File: DST/random
	Size: 1048582         Blocks: 2056       IO Block: 4096   regular file
	Device: 802h/2050d      Inode: 414849      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (65534/  nobody)   Gid: (    0/    root)
	<e class=red>Access: 2020-10-28 12:07:53.622841733 +0100</e>
	<e>Modify: 2020-10-28 11:09:57.973931318 +0100</e>
	<e class=red>Change: 2020-10-28 12:07:53.622841733 +0100</e>
	Birth: -
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      So in summary:
    </p>
    <ul>
      <li>Permission are restored,</li>
      <li>user and group ownership are restored too,</li>
      <li>mtime is restored,</li>
      <li>File ACL are restored,</li>
      <li>Extended Attributes are restored</li>
    </ul>
    <p>
      But
    </p>
    <ul>
      <li>filesystem specific attributes are not restored,</li>
      <li>atime is not restored,</li>
      <li>ctime is not restored</li>
    </ul>

    <p>
      For <b>btime</b> as we did before, let's test under a FreeBSD system:
    </p>

    <code class=block>
	<b>root@FreeBSD:~denis # rm -rf DST</b>
	<b>root@FreeBSD:/home/denis # which mystat</b>
	mystat:          aliased to stat -f "%N%nAccess: %Sa%nModify: %Sm%nChange: %Sc%nBirth: %SB%n" !*
	<b>root@FreeBSD:/home/denis # mystat SRC/random</b>
	SRC/random
	Access: Oct 27 14:27:59 2020
	Modify: Oct 22 15:34:07 2020
	Change: Oct 22 15:34:09 2020
	<e>Birth: Oct 22 15:34:07 2020</e>
	root@FreeBSD:/home/denis # mkdir DST
	root@FreeBSD:/home/denis # rsync -arv SRC/* DST
	sending incremental file list
	fd1
	null
	pipe
	plain_zeroed
	random
	sparse_file
	SUB/
	SUB/hard_linked_socket
	SUB/hard_linked_sparse_file
	SUB/symlink-broken -> random
	SUB/symlink-valid -> ../random
	dev/
	dev/log -> /var/run/log

	sent 22,583,283 bytes  received 129 bytes  45,166,824.00 bytes/sec
	total size is 22,577,179  speedup is 1.00
	root@FreeBSD:/home/denis # mystat DST/random
	DST/random
	Access: Oct 27 14:28:53 2020
	Modify: Oct 22 15:34:07 2020
	Change: Oct 27 14:28:53 2020
	<e>Birth: Oct 22 15:34:07 2020</e>
	<b>root@FreeBSD:/home/denis #</b>
    </code>
    <p>
      So, birthtime is properly restored.
    </p>

    <h4>Tar</h4>

    <p>
      As done with previously, let's save and restore the <code>SRC</code> directory to <code>DST</code>... Note that by default
      no sparse file is taken into account (this is why we added the <code>-S option</code>), same with acl (so we added
      the <code>--acl</code> option) and Extended Attributes (unless <code>--xattrs</code> is added). The
      <i>tar</i> command-line becomes thus a bit longer:
    </p>

    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# rm -rf DST</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# cd SRC</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools/SRC# tar --acl --xattrs -cSf ../backup.tar *</b>
	tar: SUB/hard_linked_socket: socket ignored
	tar: dev/log: socket ignored
	<b>root@terre:/mnt/localdisk/Benchmark_tools/SRC# cd ../</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# mkdir DST</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# cd DST</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools/DST# tar --acl --xattrs -xSf ../backup.tar</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools/DST# cd ..</b>
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>

    <p>
     Now let's compare the restored data with the original:
    </p>

    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# du -s SRC DST</b>
	2068    SRC
	<e>2068</e>    DST
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>

    <p>
      The sparse file has been properly restored (thanks to the <code>-S option</code> for that) but not space optimization
      has been performed.
    </p>

    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# ls -iRl SRC DST</b>
	DST:
	total 12060
	414841 drwxr-xr-x 2 root   root     4096 Oct 28 11:09 SUB
	414846 drwxr-xr-x 2 root   root     4096 Oct 22 11:09 dev
	414847 brw-r--r-- 1 root   root     2, 1 Oct 28 11:09 fd1
	414848 crw-r--r-- 1 root   root     3, 1 Oct 28 11:09 null
	414849 prw-r--r-- <e class=red>1</e> root   root        0 Oct 28 11:09 pipe
	414850 -rw-rwxr-- 1 root   root  1048576 Oct 28 11:09 plain_zeroed
	414852 -rw-r--r-- 1 nobody root  1048582 Oct 28 11:09 random
	<e>414843</e> -rw-r--r-- 2 root   root 10240000 Oct 28 11:09 sparse_file

	DST/SUB:
	total 10000
	414845 prw-r--r-- 1 root root          0 Oct 28 11:09 hard_linked_pipe
	<e>414843</e> -rw-r--r-- 2 root root   10240000 Oct 28 11:09 hard_linked_sparse_file
	414842 lrwxrwxrwx 1 root root          6 Oct 28 11:09 symlink-broken -> random
	414844 lrwxrwxrwx 1 bin  daemon        9 Oct 28 11:09 symlink-valid -> ../random

	DST/dev:
	total 0

	SRC:
	total 2064
	411386 drwxr-xr-x  2 root   root     4096 Oct 28 11:09 SUB
	414836 drwxr-xr-x  2 root   root     4096 Oct 22 11:09 dev
	414835 brw-r--r--  1 root   root     2, 1 Oct 28 11:09 fd1
	414834 crw-r--r--  1 root   root     3, 1 Oct 28 11:09 null
	414832 prw-r--r--  <e class=red>2</e> root   root        0 Oct 28 11:09 pipe
	414826 -rw-rwxr--+ 1 root   root  1048576 Oct 28 11:09 plain_zeroed
	414827 -rw-r--r--  1 nobody root  1048582 Oct 28 11:09 random
	414828 -rw-r--r--  2 root   root 10240000 Oct 28 11:09 sparse_file

	SRC/SUB:
	total 4
	414832 prw-r--r-- 2 root root          0 Oct 28 11:09 hard_linked_pipe
	<e class=red>414837 srw-rw-rw- 2 root root          0 Oct 12 23:00 hard_linked_socket</e>
	414828 -rw-r--r-- 2 root root   10240000 Oct 28 11:09 hard_linked_sparse_file
	414830 lrwxrwxrwx 1 root root          6 Oct 28 11:09 symlink-broken -> random
	414831 lrwxrwxrwx 1 bin  daemon        9 Oct 28 11:09 symlink-valid -> ../random

	SRC/dev:
	total 0
	<e class=red>414837 srw-rw-rw- 2 root root 0 Oct 12 23:00 log</e>
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>

    <p>
      The warning was not vain, <code>SUB/hard_linked_socket</code> and <code>log</code> are missing in <code>DST</code>.
      This is however a minor problem as usually unix sockets get recreated by the process using them. However
      we might have some permission and ownership to set back, by hand. A possible use case is <code>syslog</code> daemon,
      when let available for a chrooted process or container (MTA, or other network service).
    </p>
    <p>
      The second problem is a bit more annoying: the hard linked fifo (aka named pipe)
      is silently restored as two independent named pipes (the inode number are different in the first column
      for <code>pipe</code> and <code>SUB/hard_linked_pipe</code> and their respective link count was <code>2</code>
      in <code>SRC</code> but is now <code>1</code> in <code>DST</code>. If two processes in different namespaces or
      chrooted environment, exchange data by mean of such hardlinked pipe, after restoration, if you are not
      aware of this failure, it will thus be difficult to identify why the two processes are just locked out, one
      waiting for data that will never come from the pipe, the other stuck for the pipe to be read.
    </p>

    <p>
      Let's continue by checking the file's metadata:
    </p>

    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# getfacl SRC/plain_zeroed DST/plain_zeroed</b>
	# file: SRC/plain_zeroed
	# owner: root
	# group: root
	user::rw-
	<e>user:nobody:rwx</e>
	group::r--
	mask::rwx
	other::r--

	# file: DST/plain_zeroed
	# owner: root
	# group: root
	user::rw-
	<e>user:nobody:rwx</e>
	group::r--
	mask::rwx
	other::r--

	root@terre:/mnt/localdisk/Benchmark_tools# getfattr -d SRC/plain_zeroed DST/plain_zeroed
	# file: SRC/plain_zeroed
	<e>user.hello="hello world!!!"</e>

	# file: DST/plain_zeroed
	<e>user.hello="hello world!!!"</e>

	<b>root@terre:/mnt/localdisk/Benchmark_tools# lsattr SRC/plain_zeroed DST/plain_zeroed</b>
	<e class=red>s---i-d-------e----</e> SRC/plain_zeroed
	<e class=red>--------------e----</e> DST/plain_zeroed
	<b>root@terre:/mnt/localdisk/Benchmark_tools#</b>
    </code>
    <p>
      Note that without <code>--xattrs</code> at creation time the timestamp accuracy of <i>tar</i>
      is 1 second:
    <code class=block>
	<b>root@terre:/mnt/localdisk/Benchmark_tools# stat SRC/random DST/random</b>
	File: SRC/random
	Size: 1048576         Blocks: 2048       IO Block: 4096   regular file
	Device: 802h/2050d      Inode: 414841      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (65534/  nobody)   Gid: (    0/    root)
	<e class=red>Access: 2020-10-27 14:03:46.064046436 +0100</e>
	<e>Modify: 2020-10-27 14:03:42.016050420 +0100</e>
	<e class=red>Change: 2020-10-27 14:03:44.048048418 +0100</e>
	Birth: -
	File: DST/random
	Size: 1048576         Blocks: 2048       IO Block: 4096   regular file
	Device: 802h/2050d      Inode: 414890      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (65534/  nobody)   Gid: (    0/    root)
	<e class=red>Access: 2020-10-27 19:08:14.932424226 +0100</e>
	<e>Modify: 2020-10-27 14:03:42</e>.<e class=red>000000000 +0100</e>
	<e class=red>Change: 2020-10-27 19:08:14.932424226 +0100</e>
	Birth: -
	root@terre:/mnt/localdisk/Benchmark_tools#
    </code>

    <p>
      From the above output we see that:
    </p>
    <ul>
      <li>permission are restored,</li>
      <li>user and group ownership are restored too,</li>
      <li>mtime is restored but it needs <code>--xattrs</code> to take into account today's system common time accuracy of one nanosecond</li>
      <li>ACL are restored,</li>
      <li>Extended Attributes are restored</li>
    </ul>

    <p>
      But
    </p>

    <ul>
      <li>filesystem attributes are not restored,</li>
      <li>atime is not restored,</li>
      <li>ctime is not restored</li>
    </ul>

    <p>
      For the last date, <b>birthtime</b> again we will perform the test under FreeBSD:
    </p>

    <code class=block>
	<b>root@FreeBSD:~denis # which mystat</b>
	mystat:          aliased to stat -f "%N%nAccess: %Sa%nModify: %Sm%nChange: %Sc%nBirth: %SB%n" !*
	<b>root@FreeBSD:~denis # mystat SRC/random</b>
	SRC/random
	Access: Oct 27 19:40:13 2020
	Modify: Oct 22 15:34:07 2020
	Change: Oct 22 15:34:09 2020
	<e>Birth: Oct 22 15:34:07 2020</e>
	<b>root@FreeBSD:~denis # cd SRC</b>
	<b>root@FreeBSD:~denis/SRC # gtar -cf ../backup.tar random</b>
	<b>root@FreeBSD:~denis/SRC # cd ..</b>
	<b>root@FreeBSD:~denis # mkdir DST</b>
	<b>root@FreeBSD:~denis # cd DST</b>
	<b>root@FreeBSD:~denis/DST # tar -xf ../backup.tar</b>
	<b>root@FreeBSD:~denis/DST # cd ..</b>
	<b>root@FreeBSD:~denis # mystat DST/random</b>
	DST/random
	Access: Oct 28 15:43:30 2020
	Modify: Oct 22 15:34:07 2020
	Change: Oct 28 15:43:30 2020
	<e>Birth: Oct 22 15:34:07 2020</e>
	<b>root@FreeBSD:~denis #</b>
    </code>

    <p>
      <i>gtar</i> saved and restored the birthtime
    </p>





    <h3><a name="features">Feature</a> set</h3>

    <h4>Historization</h4>
    <p>
      To evaluate this feature, in a first time we will create two files <i>A.txt</i> and <i>B.txt</i>
      and make a first backup. Then we remove <i>A.txt</i> and add <i>C.txt</i> then make
      a second backup. We should be able to restore the data in both states (A+B and B+C).
      To simplify the operation we use the <a href="#historization_feature"><i>historization_feature</i></a>
      script described at the end of this document.
    </p>


    <h5>Dar</h5>

    <code class=block>
	<b>root@terre:/mnt/memdisk# rm -rf SRC</b>
	<b>root@terre:/mnt/memdisk# ./historization_feature SRC phase1</b>
	<b>root@terre:/mnt/memdisk# dar -c full -g SRC -q</b>
	<b>root@terre:/mnt/memdisk# ./historization_feature SRC phase2</b>
	<b>root@terre:/mnt/memdisk# dar -c diff -A full -g SRC -q</b>
	<b>root@terre:/mnt/memdisk# mkdir DST</b>
	<b>root@terre:/mnt/memdisk# dar -x full -R DST -q</b>
	<b>root@terre:/mnt/memdisk# ls -lR DST</b>
	DST:
	total 0
	drwxr-xr-x 2 root root 80 Nov  6 18:37 SRC

	DST/SRC:
	total 8
	-rw-r--r-- 1 root root 13 Nov  6 18:37 <e>A.txt</e>
	-rw-r--r-- 1 root root 24 Nov  6 18:37 <e>B.txt</e>
	<b>root@terre:/mnt/memdisk# dar -x diff -R DST -w -q</b>
	<b>root@terre:/mnt/memdisk# ls -lR DST</b>
	DST:
	total 0
	drwxr-xr-x 2 root root 80 Nov  6 18:38 SRC

	DST/SRC:
	total 8
	-rw-r--r-- 1 root root 24 Nov  6 18:37 <e>B.txt</e>
	-rw-r--r-- 1 root root 21 Nov  6 18:38 <e>C.txt</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      Historization is present, we can get back from backup both saved states
    </p>
    <p>
      In complement <i>dar</i> proposes a manager <i>dar_manager</i> to easily locate file's status between the archives
      the database has been feeded with, as well as the file's data present in each archive:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar_manager -C base.dmd</b>
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -A full</b>
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -A diff</b>
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -f SRC/A.txt</b>
        1       Fri Nov  6 18:37:51 2020  saved                                 absent
        2       Fri Nov  6 18:38:04 2020  removed                               absent
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -f SRC/B.txt</b>
        1       Fri Nov  6 18:37:51 2020  saved                                 absent
        2       Fri Nov  6 18:37:51 2020  present                               absent
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -f SRC/C.txt</b>
        2       Fri Nov  6 18:38:04 2020  saved                                 absent
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -l</b>

	dar path        :
	dar options     :
	database version: 5
	compression used: gzip

	archive #   |    path      |    basename
	------------+--------------+---------------
        1       .       full
        2       .       diff
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -u 1</b>
	[ Saved ][       ]  SRC/B.txt
	[ Saved ][       ]  SRC/A.txt
	<b>root@terre:/mnt/memdisk# dar_manager -B base.dmd -u 2</b>
	[ Saved ][       ]  SRC
	[ Saved ][       ]  SRC/C.txt
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      <i>dar_manager</i> can even take for you the actions to invoke <i>dar</i> as many time as necessary
      get the file's status of a given date for a given set of subset of the saved files:
    </p>

    <code class=block>
	<b>root@terre:/mnt/memdisk# dar_manager -v -B base.dmd -e "-R DST -w" -r SRC</b>
	Decompressing and loading database to memory...
	Looking in archives for requested files, classifying files archive by archive...
	Checking chronological ordering of files between the archives...
	File recorded as removed at this date in database: SRC/A.txt
	CALLING DAR: restoring 1 files from archive ./full using anonymous pipe to transmit configuration to the dar process
	Arguments sent through anonymous pipe are:
	dar -x ./full -R DST -w -g SRC/B.txt


	--------------------------------------------
	2 inode(s) restored
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	1 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 3
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	CALLING DAR: restoring 2 files from archive ./diff using anonymous pipe to transmit configuration to the dar process
	Arguments sent through anonymous pipe are:
	dar -x ./diff -R DST -w -g SRC -g SRC/C.txt
	Error while restoring /mnt/memdisk/DST/SRC/A.txt : Cannot remove non-existent file from filesystem: /mnt/memdisk/DST/SRC/A.txt


	--------------------------------------------
	2 inode(s) restored
	including 0 hard link(s)
	1 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	1 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 4
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	Final memory cleanup...
	All files asked could not be restored
	DAR sub-process has terminated with exit code 5 Continue anyway ? [return = YES | Esc = NO]
	Continuing...
	<b>root@terre:/mnt/memdisk# ls -lR DST</b>
	DST:
	total 0
	drwxr-xr-x 2 root root 80 Nov  6 18:38 SRC

	DST/SRC:
	total 8
	-rw-r--r-- 1 root root 24 Nov  6 18:37 <e>B.txt</e>
	-rw-r--r-- 1 root root 21 Nov  6 18:38 <e>C.txt</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# ./historization_feature SRC phase1</b>
	<b>root@terre:/mnt/memdisk# rsync -arvHAX SRC DST</b>
	sending incremental file list
	created directory DST
	SRC/
	SRC/A.txt
	SRC/B.txt

	sent 229 bytes  received 84 bytes  626.00 bytes/sec
	total size is 37  speedup is 0.12
	<b>root@terre:/mnt/memdisk# ./historization_feature SRC phase2</b>
	<b>root@terre:/mnt/memdisk# rsync -arvHAX SRC DST</b>
	sending incremental file list
	SRC/
	SRC/C.txt

	sent 172 bytes  received 39 bytes  422.00 bytes/sec
	total size is 45  speedup is 0.21
	<b>root@terre:/mnt/memdisk# ls -l</b>
	total 4
	drwxr-xr-x 3 root root  60 Nov  6 17:06 DST
	drwxr-xr-x 2 root root  80 Nov  6 17:06 SRC
	-rwxr--r-- 1 root root 589 Nov  6 16:32 historization_feature
	<b>root@terre:/mnt/memdisk# ls -l DST</b>
	total 0
	drwxr-xr-x 2 root root 100 Nov  6 17:06 SRC
	<b>root@terre:/mnt/memdisk# ls -l DST/SRC</b>
	total 12
	<e>-rw-r--r-- 1 root root 13 Nov  6 17:05 A.txt</e>
	<e>-rw-r--r-- 1 root root 24 Nov  6 17:05 B.txt</e>
	<e>-rw-r--r-- 1 root root 21 Nov  6 17:06 C.txt</e>
	<b>root@terre:/mnt/memdisk# rsync -arvHAX --delete SRC DST</b>
	sending incremental file list
	deleting SRC/A.txt

	sent 101 bytes  received 26 bytes  254.00 bytes/sec
	total size is 45  speedup is 0.35
	<b>root@terre:/mnt/memdisk# ls -l DST/SRC</b>
	total 8
	<e>-rw-r--r-- 1 root root 24 Nov  6 17:05 B.txt</e>
	<e>-rw-r--r-- 1 root root 21 Nov  6 17:06 C.txt</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <p>
      the "backup" contains all three files, <i>A.txt</i>, <i>B.txt</i>
      and <i>C.txt</i> while the first and the later never existed at the
      same time. Such backup does not allow to have neither the state of
      the <i>phase1</i> nor the state of the <i>phase2</i>.
    </p>
    <p>
      We added the <code>--delete</code> option and as result we got
      to be the <i>phase2</i> state only. But then we cannot restore to the <i>phase1</i> state as the
      file <i>A.txt</i> has been deleted from the backup.
    </p>
    <p>
      To have both states with <i>rsync</i>, we should call rsync to a different destination directory
      at each new backup time, which would consume a lot of space and would also defeat one of the main
      features of <i>rsync</i> which is its ability to synchronize two directories exchanging only
      the minimal information that was modified.
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rmdir SRC</b>
	<b>root@terre:/mnt/memdisk# ./historization_feature SRC phase1</b>
	<b>root@terre:/mnt/memdisk# tar --listed-incremental=snapshot.file -cf full.tar SRC</b>
	<b>root@terre:/mnt/memdisk# ./historization_feature SRC phase2</b>
	<b>root@terre:/mnt/memdisk# tar --listed-incremental=snapshot.file -cf diff.tar SRC</b>
	<b>root@terre:/mnt/memdisk# mkdir DST</b>
	<b>root@terre:/mnt/memdisk# cd DST</b>
	<b>root@terre:/mnt/memdisk/DST# tar --listed-incremental=snapshot.file -xf ../full.tar</b>
	<b>root@terre:/mnt/memdisk/DST# ls -l SRC</b>
	total 8
	<e>-rw-r--r-- 1 root root 13 Nov  6 18:20 A.txt</e>
	<e>-rw-r--r-- 1 root root 24 Nov  6 18:20 B.txt</e>
	<b>root@terre:/mnt/memdisk/DST# tar --listed-incremental=snapshot.file -xf ../diff.tar</b>
	<b>root@terre:/mnt/memdisk/DST# ls -l SRC</b>
	total 8
	<e>-rw-r--r-- 1 root root 24 Nov  6 18:20 B.txt</e>
	<e>-rw-r--r-- 1 root root 21 Nov  6 18:21 C.txt</e>
	<b>root@terre:/mnt/memdisk/DST#</b>
    </code>

    <p>
      We could restore from backup both the phase1 and phase2 status, historization is available with tar.
    </p>


    <h4>Data filtering by directory</h4>

    <h5>Dar</h5>
    <p>
      We want to save <code>/lib</code> except the content of <code>/lib/modules</code>:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R /lib -P modules -vs -q</b>
	<e>Skipping file: /lib/modules</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      What if we want to exclude all of to exclude <code>/lib/module</code> except
      <code>/lib/module/4.19.0-12-amd64</code>?
    </p>

    <code class=block>
	<b>root@terre:/mnt/memdisk# rm backup.1.dar</b>
	rm: remove regular file 'backup.1.dar'? y
	<b>root@terre:/mnt/memdisk# dar -c backup -R /lib -am -P modules -g modules/4.19.0-12-amd64 -vs -q</b>
	<e>Skipping file: /lib/modules/4.19.0-11-amd64</e>
	<e>Skipping file: /lib/modules/4.19.0-10-amd64</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      OK, we can mix included directories and excluded directories
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# mkdir DST</b>
	<b>root@terre:/mnt/memdisk# rsync -arHAXS --exclude /lib/modules /lib DST</b>
	<b>root@terre:/mnt/memdisk# ls -ld DST/lib/m*</b>
	drwxr-xr-x 2 root root 80 Jun 11 23:33 DST/lib/modprobe.d
	<b>root@terre:/mnt/memdisk#</b>
	<b>root@terre:/mnt/memdisk# ls -l DST/lib/modules</b>
	ls: cannot access <e>'DST/lib/modules': No such file or directory</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      We could exclude <code>/lib/modules</code> as expected. As previously, let's exclude it
      except <code>/lib/modules/4.19.0-12-amd64</code>:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# rsync -arHAXS -f "+ /lib/modules" -f "- /lib/modules/4.19.0-12-amd64"  /lib DST</b>
	<b>root@terre:/mnt/memdisk# la DST/lib/modules/</b>
	total 0
	drwxr-xr-x  4 root root  80 Oct 22 10:33 .
	drwxr-xr-x 19 root root 420 Oct 22 11:20 ..
	<e>drwxr-xr-x  3 root root 280 Aug  8 12:58 4.19.0-10-amd64</e>
	<e>drwxr-xr-x  3 root root 280 Oct 12 11:25 4.19.0-11-amd64</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      OK, we can mix included directories and excluded directories
    </p>

    <h5>Tar</h5>
    <p>
      Let's save /lib and excluding /lib/module again:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar --exclude /lib/modules -cf backup.tar /lib</b>
	tar: Removing leading `/' from member names
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | grep modules</b>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      Now let's exclude <code>/lib/modules</code> except <code>/lib/modules/4.19.0-12-amd64</code>:
    <p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar -cf backup.tar /lib/modules/4.19.0-12-amd64/ --exclude /lib/modules /lib</b>
	tar: Removing leading `/' from member names
	tar: Removing leading `/' from hard link targets
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | wc -l</b>
	6017
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | grep -v "lib/modules/4.19.0-12-amd64" | wc -l</b>
	1626
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | grep "lib/modules/4.19.0-12-amd64" | wc -l</b>
	4391
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | grep "lib/modules" | wc -l</b>
	4391
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      The backup contains a total of 6017 entries, 1626 are out of the <code>lib/modules/4.19.0-12-amd64</code>
      directory, the rest is all in that previous directory, nothing else is found in <code>lib/modules</code>
      while there was <code>lib/modules/4.19.0-11-amd64</code> and <code>lib/modules/4.19.0-10-amd64</code>
      subdirectory. We can thus mix included and included directories.
    </p>

    <h4>Data filtering by filename</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R /lib -X "*.ko"</b>


	--------------------------------------------
	4122 inode(s) saved
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	<e>10677 inode(s) ignored (excluded by filters)</e>
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 14799
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 3945 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/memdisk# mkdir DST</b>
	<b>root@terre:/mnt/memdisk# dar -x backup -R DST --fsa-scope none</b>


	--------------------------------------------
	4122 inode(s) restored
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 4122
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/memdisk# find DST -name "*.ko" -ls</b>
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <p>
      we would exclude all file having the <code>ko</code> extension, what if we do not
      want to exclude those that start with <code>ext</code>?
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R /lib -am -X "*.ko" -I "ext*"</b>


	--------------------------------------------
	4128 inode(s) saved
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	10671 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 14799
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 3951 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# mkdir DST</b>
	<b>root@terre:/mnt/memdisk# dar -x backup -R DST --fsa-scope none -q</b>
	<b>root@terre:/mnt/memdisk# find DST -name "*.ko" -print</b>
	DST/modules/4.19.0-10-amd64/kernel/fs/ext4/ext4.ko
	DST/modules/4.19.0-10-amd64/kernel/drivers/extcon/extcon-core.ko
	DST/modules/4.19.0-11-amd64/kernel/fs/ext4/ext4.ko
	DST/modules/4.19.0-11-amd64/kernel/drivers/extcon/extcon-core.ko
	DST/modules/4.19.0-12-amd64/kernel/fs/ext4/ext4.ko
	DST/modules/4.19.0-12-amd64/kernel/drivers/extcon/extcon-core.ko
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      OK, we got what we wanted
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# rsync -arHAXS -f "- *.ko"  /lib DST</b>
	<b>root@terre:/mnt/memdisk# find DST -name "*.ko" -print</b>
	<b>root@terre:/mnt/memdisk# ls DST</b>
	lib
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      Same as previously, we don't want to exclude <code>ko</code> files starting by <code>ext</code>:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# rsync -arHAXS -f "+ ext*" -f "- *.ko" /lib DST</b>
	<b>root@terre:/mnt/memdisk# find DST -name "*.ko" -print</b>
	DST/lib/modules/4.19.0-12-amd64/kernel/fs/ext4/ext4.ko
	DST/lib/modules/4.19.0-12-amd64/kernel/drivers/extcon/extcon-core.ko
	DST/lib/modules/4.19.0-11-amd64/kernel/fs/ext4/ext4.ko
	DST/lib/modules/4.19.0-11-amd64/kernel/drivers/extcon/extcon-core.ko
	DST/lib/modules/4.19.0-10-amd64/kernel/fs/ext4/ext4.ko
	DST/lib/modules/4.19.0-10-amd64/kernel/drivers/extcon/extcon-core.ko
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      OK, we got what we wanted
    </p>

    <h5>Tar</h5>
    <p>Same as previously, let's filter out kernel object files</p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar -cf backup.tar --exclude "*.ko" /lib</b>
	tar: Removing leading `/' from member names
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# mkdir DST</b>
	<b>root@terre:/mnt/memdisk# cd DST</b>
	<b>root@terre:/mnt/memdisk/DST# tar -xf ../backup.tar</b>
	<b>root@terre:/mnt/memdisk/DST# find . -name "*.ko" -print</b>
	<b>root@terre:/mnt/memdisk/DST#</b>
    </code>
    <p>Now, we want to keep only those kernel object files starting with <code>ext</code></p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar -cf backup.tar "ext*" --exclude "*.ko" /lib</b>
	tar: ext*: Cannot stat: No such file or directory
	tar: Removing leading `/' from member names
	tar: Removing leading `/' from hard link targets
	tar: Exiting with failure status due to previous errors
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      Well, argument passed out of option do not seem expanded by tar thus using mask is not
      possible to include some pattern. It seems the only option is to use file listing, thing
      we will evaluate below.
    </p>

    <h4>Data filtering by filesystem</h4>

    <p>
      We will use a tmpfs filesystem mounted twice thanks to mount's --bind option. The objective
      is first to save every thing except a few given filesystems, or only one or save inside a few
      given filesystems. Here is the preparation phase:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# mkdir SRC</b>
	<b>root@terre:/mnt/memdisk# mkdir SRC/D1 SRC/D2 SRC/D3</b>
	<b>root@terre:/mnt/memdisk# mount -t tmpfs tmpfs SRC/D1</b>
	<b>root@terre:/mnt/memdisk# mount --bind SRC/D1 SRC/D2</b>
	<b>root@terre:/mnt/memdisk# mount -t tmpfs tmpfs SRC/D3</b>
	<b>root@terre:/mnt/memdisk# ls SRC/D1 SRC/D2</b>
	SRC/D1:

	SRC/D2:
	<b>root@terre:/mnt/memdisk# echo "Hello World" > SRC/D1/file.txt</b>
	<b>root@terre:/mnt/memdisk# ls SRC/D1 SRC/D2</b>
	SRC/D1:
	file.txt

	SRC/D2:
	file.txt
	<b>root@terre:/mnt/memdisk# echo "give me your data, I'll tell your needs and what to buy" > SRC/gafam.com</b>
	<b>root@terre:/mnt/memdisk# echo "sight" > SRC/D3/democracy.org</b>
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R SRC -MX:/mnt/memdisk/SRC/D1 -vs -vt -q</b>
	Adding folder to archive: /mnt/memdisk/SRC/D3
	Adding file to archive: /mnt/memdisk/SRC/D3/democracy.org
	Adding file to archive: /mnt/memdisk/SRC/gafam.com
	Skipping file: /mnt/memdisk/SRC/D2
	Skipping file: /mnt/memdisk/SRC/D1
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      We could exclude a filesystem, and its second appearance in D2 was also excluded,
      whithout having to mention it. Let's include only <code>D1</code> now:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rm -f backup.1.dar</b>
	<b>root@terre:/mnt/memdisk# dar -c backup -R SRC -MI:/mnt/memdisk/SRC/D1 -vs -vt -q</b>
	Skipping file: /mnt/memdisk/SRC/D3
	Adding file to archive: /mnt/memdisk/SRC/gafam.com
	Adding folder to archive: /mnt/memdisk/SRC/D2
	Adding file to archive: /mnt/memdisk/SRC/D2/file.txt
	Adding folder to archive: /mnt/memdisk/SRC/D1
	Adding file to archive: /mnt/memdisk/SRC/D1/file.txt
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      OK, we got what we wanted
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rsync -arvHAXS --one-file-system SRC DST</b>
	sending incremental file list
	created directory DST
	SRC/
	SRC/gafam.com
	SRC/D1/
	SRC/D2/
	SRC/D3/

	sent 283 bytes  received 77 bytes  720.00 bytes/sec
	total size is 56  speedup is 0.16
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <p>
      <i>rsync</i> has ony one option about filesystems, it sticks recursion to the filesystem
      of the source directory, we cannot exclude specifically some filesystems, they are all excluded,
      and we cannot include specifically some filesystems, none is excluded (default behavior without this option)
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar -cvf backup.tar --one-file-system SRC</b>
	SRC/
	SRC/D3/
	tar: SRC/D3/: file is on a different filesystem; not dumped
	SRC/gafam.com
	SRC/D2/
	tar: SRC/D2/: file is on a different filesystem; not dumped
	SRC/D1/
	tar: SRC/D1/: file is on a different filesystem; not dumped
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      <i>tar</i> does not behaves better than rsync on that topic
    </p>

    <h4>Data filtering by tag</h4>
    <p>
      by tag we mean any mark the user can add to a file that will drive its fate when backup will be done.
      The most common is the <i>dump</i> flag, but it is not always available, using some other mechanisms
      (Extended Attributes,...) can be an interesting alternative.
    </p>

    <h5>Dar</h5>
    <code class=block>
	<b>root@terre:/var/tmp# mkdir SRC</b>
	<b>root@terre:/var/tmp# echo "Hello" > file1.txt</b>
	<b>root@terre:/var/tmp# echo "World" > file2.txt</b>
	<b>root@terre:/var/tmp# chattr <e class=blue>+d</e> file1.txt</b>
	<b>root@terre:/var/tmp# setfattr -n <e>user.no_dump</e> file2.txt</b>
	<b>root@terre:/var/tmp# mv file1.txt file2.txt SRC</b>
	<b>root@terre:/var/tmp# dar -c backup -w -R SRC  <e class=blue>--nodump</e> -vt -q</b>
	<e>Adding file to archive: /var/tmp/SRC/file2.txt</e>
	Saving Extended Attributes for /var/tmp/SRC/file2.txt
	Saving Filesystem Specific Attributes for /var/tmp/SRC/file2.txt
	<b>root@terre:/var/tmp# dar -c backup -w -R SRC  <e>--exclude-by-ea=user.no_dump</e> -vt -q</b>
	<e class=blue>Adding file to archive: /var/tmp/SRC/file1.txt</e>
	Saving Filesystem Specific Attributes for /var/tmp/SRC/file1.txt
	<b>root@terre:/var/tmp#</b>
    </code>
    <p>
      We have two mechanisms, one based on the <i>dump</i> flag and an arbitrary extended attribute.
      However <i>dar</i> only supports exclusion of file, not inclusion for backup based on a tag.
    </p>

    <h5>Rsync</h5>
    <p>
      <i>rsync</i> does not seem to be able to filter based on an arbitrary mark
    </p>

    <h5>Tar</h5>
    <p>
      <i>rsync</i> does not seem to be able to filter based on an arbitrary mark
    </p>

    <h4>Data filtering by files listing</h4>

    <p>
      We build a file listing and expect to either have only those file saved
      or excluded from the performed backup. Here is the listing preparation:
    </p>

    <code class=block>
	<b>root@terre:/mnt/memdisk# find /lib -name "*.ko" -o -print  > include.txt</b>
	<b>root@terre:/mnt/memdisk# wc -l include.txt</b>
	4123 include.txt
	<b>root@terre:/mnt/memdisk# find /lib -name "*.ko" -print  > exclude.txt</b>
	<b>root@terre:/mnt/memdisk# wc -l exclude.txt</b>
	10677 exclude.txt
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R /lib -[ include.txt</b>

	--------------------------------------------
	<e>4122 inode(s) saved</e>
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	<e>10677 inode(s) ignored (excluded by filters)</e>
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 14799
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 3945 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      file inclusion is available, let's see file exclusion:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R /lib -] exclude.txt</b>


	--------------------------------------------
	<e>4122 inode(s) saved</e>
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	<e>10677 inode(s) ignored (excluded by filters)</e>
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 14799
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 3945 inode(s)
	--------------------------------------------
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# rsync -aHAXS --files-from=include.txt / DST</b>
	<b>root@terre:/mnt/memdisk# find DST -print | wc -l</b>
	4124
	<b>root@terre:/mnt/memdisk# find DST -name "*.ko" -print</b>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      file inclusion is available. However if we can exclude a list of pattern
      defined in a file, we cannot exclude a list of files. We should prepend each
      entry by "- " seen the filtering syntax of rsync:
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# sed -r 's/^/- /' exclude.txt  > rsync-exclude.txt</b>
	<b>root@terre:/mnt/memdisk# rm -rf DST</b>
	<b>root@terre:/mnt/memdisk# rsync -aHAXS --exclude-from=rsync-exclude.txt /lib DST</b>
	<b>root@terre:/mnt/memdisk# find DST -print | wc -l</b>
	4124
	<b>root@terre:/mnt/memdisk# find DST -name "*.ko" -print | wc -l</b>
	0
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      So we are good with this <i>sed</i> additional listing adaptation.
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar -cvf backup.tar --files-from=include.txt | wc -l</b>
	tar: Removing leading `/' from member names
	98017
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | grep .ko | wc -l</b>
	73392
	<b>root@terre:/mnt/memdisk# grep .ko include.txt | wc -l</b>
	3
	<b>root@terre:/mnt/memdisk# grep .ko include.txt</b>
	/lib/modules/4.19.0-12-amd64/kernel/sound/pci/korg1212
	/lib/modules/4.19.0-11-amd64/kernel/sound/pci/korg1212
	/lib/modules/4.19.0-10-amd64/kernel/sound/pci/korg1212
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      the include.txt file does not contain any file with the ko extension, however <i>tar</i>
      saved all of them. Reading back the man page concerning this --files-from option
      <i>The names read are handled the same way as command line arguments</i> explains that
      in the listing where all <code>"*.ko"</code> files have been removed, remain their parent
      directory, which implies saving all its content. In consequence we must not list
      directories only their content (which will restrict us saving empty directories as such).
      Let's modify the include.txt file that way:
    </p>
    <code class=block>
	find /lib -type d -o -name "*.ko" -o -print  > tar-include.txt
	<b>root@terre:/mnt/memdisk# tar -cvf backup.tar --files-from=tar-include.txt | wc -l</b>
	tar: Removing leading `/' from member names
	1532
	<b>root@terre:/mnt/memdisk# tar -tvf backup.tar | grep "*.ko"</b>
	<b>root@terre:/mnt/memdisk# wc -l tar-include.txt</b>
	1532 tar-include.txt
	<b>root@terre:/mnt/memdisk# wc -l include.txt</b>
	4123 include.txt
    </code>
    <p>
      the difference between the 1532 entries saved by <i>tar</i> and the
      4123 saved by <i>rsync</i> or <i>dar</i> comes from the many empty directories
      that cannot be saved as such by <i>tar</i> using this method.
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk# tar -cvf backup.tar --exclude-from=exclude.txt /lib | wc -l</b>
	tar: Removing leading `/' from member names
	4123
	<b>root@terre:/mnt/memdisk# wc -l exclude.txt</b>
	10677 exclude.txt
	<b>root@terre:/mnt/memdisk# tar -tf backup.tar | egrep "\.ko$"</b>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      The file listing exclusion works as expected
    </p>

    <h4>Slicing</h4>

    <p>
      For this test we will backup the content of /usr/bin of the running system.
      We select a slice size smaller than the biggest file under backup. The use case for
      slicing implies compression (remote storage, cloud storage, limited removable
      media storage...).
    </p>

    <code class=block>
	<b>root@terre:/mnt/memdisk# ls -lh --sort=size /usr/bin | tac | tail</b>
	-rwxr-xr-x 1 root   root       8.0M Dec 18  2018 luajittex
	-rwxr-xr-x 1 root   root       8.1M Dec 18  2018 luatex53
	-rwxr-xr-x 1 root   root       8.1M Dec 18  2018 luatex
	-rwxr-xr-x 1 root   root       8.2M May 27  2019 wireshark
	-rwxr-xr-x 1 root   root        12M Dec 21  2018 kstars
	-rwxr-xr-x 1 root   root        15M Mar 12  2018 doxygen
	-rwxr-xr-x 1 root   root        16M Jan  4  2019 stellarium
	-rwxr-xr-x 1 root   root        19M Oct 12 19:46 mysql_embedded
	-rwxr-xr-x 1 root   root        <e>39M</e> Sep  5  2019 emacs-gtk
	total 430M
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>

    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R /usr/bin -z6 <e>-s 20M</e> -q</b>
	<b>terre:/mnt/memdisk# ls -lh backup.*</b>
	-rw-r--r-- 1 root root  <e>20M</e> Nov 13 11:30 backup.1.dar
	-rw-r--r-- 1 root root  20M Nov 13 11:30 backup.2.dar
	-rw-r--r-- 1 root root  20M Nov 13 11:30 backup.3.dar
	-rw-r--r-- 1 root root  20M Nov 13 11:30 backup.4.dar
	-rw-r--r-- 1 root root  20M Nov 13 11:30 backup.5.dar
	-rw-r--r-- 1 root root  20M Nov 13 11:30 backup.6.dar
	-rw-r--r-- 1 root root  20M Nov 13 11:30 backup.7.dar
	-rw-r--r-- 1 root root <e>7.7M</e> Nov 13 11:30 backup.8.dar
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# dar -x backup -R DST -g emacs-gtk <e>-E "echo openning slice %p/%b.%N.%e"</e></b>
	<e>openning slice /mnt/memdisk/backup.8.dar</e>
	<e>openning slice /mnt/memdisk/backup.4.dar</e>
	<e>openning slice /mnt/memdisk/backup.5.dar</e>
	Restoration of FSA for /mnt/memdisk/DST/emacs-gtk aborted: Failed reading existing extX family FSA: Inappropriate ioctl for device
	Restoration of linux immutable FSA for /mnt/memdisk/DST/emacs-gtk aborted: Failed reading existing extX family FSA: Inappropriate ioctl for device


	--------------------------------------------
	1 inode(s) restored
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	2591 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 2592
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# diff DST/emacs-gtk /usr/bin/emacs-gtk</b>
	memdiskerre:/mnt/memdisk# echo $?
	0
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      We can also specify a different size for the first slice when using <i>dar</i>, this was used in the past
      to fulfill a disk partially filled by a previous incremental backup when saving onto CD-RW
      and DVD-RW, but that may still make sense when using USB keys or any other removable media.
    </p>

    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R /usr/bin <e class=blue>-s 20M</e> <e>-S 1M</e> -q --min-digit 3</b>
	<b>root@terre:/mnt/memdisk# ls -lh</b>
	total 361M
	-rw-r--r-- 1 root root <e>1.0M</e> Nov  6 18:57 backup.001.dar
	-rw-r--r-- 1 root root  <e class=blue>20M</e> Nov  6 18:57 backup.002.dar
	-rw-r--r-- 1 root root  <e class=blue>20M</e> Nov  6 18:57 backup.003.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.004.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.005.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.006.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.007.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.008.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.009.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.010.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.011.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.012.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.013.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.014.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.015.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.016.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.017.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.018.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:57 backup.019.dar
	<b>root@terre:/mnt/memdisk# dar -c backup -R /usr/bin -s 20M <e>-S 200M</e> -q --min-digit 3</b>
	<b>root@terre:/mnt/memdisk# ls -lh</b>
	total 361M
	-rw-r--r-- 1 root root <e>200M</e> Nov  6 18:58 backup.001.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.002.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.003.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.004.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.005.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.006.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.007.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.008.dar
	-rw-r--r-- 1 root root  20M Nov  6 18:59 backup.009.dar
	-rw-r--r-- 1 root root 913K Nov  6 18:59 backup.010.dar
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <p>
      <i>rsync</i> cannot split any file in slices, and it does not generate any backup, but it copies files.
      You cannot thus split data into slices to fit a particular restricted storage space.
    </p>

    <h5>Tar</h5>

    <code class=block>
    	<b>root@terre:/mnt/memdisk# tar -czf backup.tar -M -L 20480 /usr/bin</b>
	<e class=red>tar: Cannot use multi-volume compressed archives</e>
	Try 'tar --help' or 'tar --usage' for more information.
	<b>root@terre:/mnt/memdisk#</b>
    </code>

    <p>
      As reported by <i>tar</i> above, if a multi-volume support exists, it is quite restrictive as
      one cannot use compression at the same time.
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# tar -cf backup -M -L 20480 /usr/bin</b>
	tar: Removing leading `/' from member names
	<e class=red>Prepare volume #2 for 'backup' and hit return:</e>
	tar: Removing leading `/' from hard link targets
	<e class=red>Prepare volume #3 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #4 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #5 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #6 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #7 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #8 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #9 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #10 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #11 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #12 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #13 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #14 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #15 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #16 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #17 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #18 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #19 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #20 for 'backup' and hit return:</e>
	<e class=red>Prepare volume #21 for 'backup' and hit return:</e>
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# ls -l backup*</b>
	-rw-r--r-- 1 root root 19527680 Nov 13 11:40 backup
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      But even without compression, <i>tar</i> is still restrictive: it does not produce
      different files, you have each new volume around and <code>hit return</code> at each time.
      <br/>
      Note also that without compression, the space required passes
      from 8 volumes with <i>dar</i> to 21 volumes with <i>tar</i>.
    </p>
    <p>
      The multi-volume support for <i>tar</i> seems well defined for
      local tape removable devices, but will cost more than twice more
      tape than what you can do with <i>dar</i> even if tape media
      is your only target. Here is an example with <i>dar</i> on how
      to write to mutli-volume and compressed backup to tape and pause
      between each volume as <i>tar</i> does:
    </p>
    <code class=block>
      <b>terre:/mnt/memdisk# <e>dar</e> -c backup -R /usr/bin -z6 -s 20M <e class=green>-E "echo writing volume %N to tape"</e> <e>-E "cat &lt; %p/%b.%N.%e &gt; /dev/mt"</e> <e class=blue>-p</e></b>
      class=green>writing volume 1 to tape
      Finished writing to file 1, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 2 to tape
      Finished writing to file 2, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 3 to tape
      Finished writing to file 3, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 4 to tape
      Finished writing to file 4, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 5 to tape
      Finished writing to file 5, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 6 to tape
      Finished writing to file 6, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 7 to tape
      Finished writing to file 7, ready to continue ?  [return = YES | Esc = NO]
      Continuing...
      writing volume 8 to tape


      --------------------------------------------
      2592 inode(s) saved
      including 5 hard link(s) treated
      0 inode(s) changed at the moment of the backup and could not be saved properly
      0 byte(s) have been wasted in the archive to resave changing files
      0 inode(s) with only metadata changed
      0 inode(s) not saved (no inode/file change)
      0 inode(s) failed to be saved (filesystem error)
      0 inode(s) ignored (excluded by filters)
      0 inode(s) recorded as deleted from reference backup
      --------------------------------------------
      Total number of inode(s) considered: 2592
      --------------------------------------------
      EA saved for 3 inode(s)
      FSA saved for 2152 inode(s)
      --------------------------------------------
      <b>terre:/mnt/memdisk#</b>
    </code>

    <h4>Symmetric encryption</h4>

    <p>
      Encryption has for target relatively long term lifetime, having compression at the same
      time to increase security as it increases data "randomness" of the data to cipher. So we will use both in our tests (gzip with a compresion level of 6).
    </p>
    <p>
      A point to pay attention to concerns the way the password/passphrase can be provided. Putting this
      to the command-line could let other users on this same system read it. Having interactive
      prompt is better as well as having the password set in a read access restricted file, which
      in addition allows automation.
    </p>

    <h5>Dar</h5>
    <code class=block>
	<b>root@terre:/mnt/memdisk# dar -c backup -R / -g usr/bin -K aes256: -q -z6</b>
	Archive backup <e>requires a password:</e>
	Please <e>confirm your password:</e>
	<b>root@terre:/mnt/memdisk# dar -l backup -q</b>
	Archive backup <e>requires a password:</e>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	Archive version format               : 11
	Compression algorithm used           : <e>gzip</e>
	Compression block size used          : 0
	Symmetric key encryption used        : <e>AES 256</e>
	Asymmetric key encryption used       : none
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : N/A
	KDF iteration count                  : <e>10000</e>
	KDF hash algorithm                   : <e>argon2</e>
	<e>Salt</e> size                            : 32 bytes
	Catalogue size in archive            : 101907 bytes

	Archive is composed of 1 file(s)
	File size: 155070897 bytes
	The global data compression ratio is:   64%

	CATALOGUE CONTENTS :

	total number of inode : 2589
	fully saved           : 2589
	binay delta patch     : 0
	inode metadata only   : 0
	distribution of inode(s)
	- directories        : 2
	- plain files        : 2152
	- symbolic links     : 435
	- named pipes        : 0
	- unix sockets       : 0
	- character devices  : 0
	- block devices      : 0
	- Door entries       : 0
	hard links information
	- number of inode with hard link           : 5
	- number of reference to hard linked inodes: 10
	destroyed entries information
	0 file(s) have been record as destroyed since backup of reference

	<b>root@terre:/mnt/memdisk# touch pass.dcf</b>
	<b>root@terre:/mnt/memdisk# chmod go-rwx pass.dcf</b>
	<b>root@terre:/mnt/memdisk# cat &gt;&gt; pass.dcf</b>
	-K "aes256:hello world!"
	<b>root@terre:/mnt/memdisk# ls -l pass.dcf</b>
	-rw------- 1 root root 25 Nov  9 11:37 pass.dcf
	<b>root@terre:/mnt/memdisk# rm backup.1.dar</b>
	rm: remove regular file 'backup.1.dar'? y
	<b>root@terre:/mnt/memdisk# dar -c backup -R / -g usr/bin -B pass.dcf -q -z6</b>
	<b>root@terre:/mnt/memdisk# dar -l backup -q -B pass.dcf</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	Archive version format               : 11
	Compression algorithm used           : gzip
	Compression block size used          : 0
	Symmetric key encryption used        : AES 256
	Asymmetric key encryption used       : none
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : N/A
	KDF iteration count                  : 10000
	KDF hash algorithm                   : argon2
	Salt size                            : 32 bytes
	Catalogue size in archive            : 102310 bytes

	Archive is composed of 1 file(s)
	File size: 155132433 bytes
	The global data compression ratio is:   64%

	CATALOGUE CONTENTS :

	total number of inode : 2589
	fully saved           : 2589
	binay delta patch     : 0
	inode metadata only   : 0
	distribution of inode(s)
	- directories        : 2
	- plain files        : 2152
	- symbolic links     : 435
	- named pipes        : 0
	- unix sockets       : 0
	- character devices  : 0
	- block devices      : 0
	- Door entries       : 0
	hard links information
	- number of inode with hard link           : 5
	- number of reference to hard linked inodes: 10
	destroyed entries information
	0 file(s) have been record as destroyed since backup of reference

	<b>root@terre:/mnt/memdisk#</b>

    </code>
    <p>
      We can provide password either on command-line (not recommended), prompted by dar once launched
      or from a protected configuration file. In the following we add slicing to encryption to see
      whether or not <i>dar</i> deciphers the whole backup to recover a single file:
    </p>

    <code class=block>
	<b>root@terre:/mnt/localdisk# rm -rf backup.*</b>
	<b>root@terre:/mnt/localdisk# dar -c backup -R / -g usr/bin -K aes256: -s 1M -q -z6</b>
	Archive backup requires a password:
	Please confirm your password:
	<b>root@terre:/mnt/localdisk# ls -l backup.* | wc -l</b>
	<e>148</e>
	<b>root@terre:/mnt/localdisk# dar -x backup -g usr/bin/emacs-gtk -E "echo openning slice %b.%N.%e" -q</b>
	openning slice backup.148.dar
	Archive backup requires a password:
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	<e>openning slice backup.1.dar</e>
	<e>openning slice backup.80.dar</e>
	<e>openning slice backup.81.dar</e>
	<e>openning slice backup.82.dar</e>
	<e>openning slice backup.83.dar</e>
	<e>openning slice backup.84.dar</e>
	<b>root@terre:/mnt/localdisk#</b>
    </code>
    <p>
      As seen above, dar does not need to uncipher nor uncompress the whole backup to recover a single file, the use
      of slicing let us see which slice it accessed to, but the behavior is the same without
      slicing and can be measure by the execution time (see the performance tests logs).
    </p>

    <h5>Rsync</h5>
    <p>
      <i>rsync</i> cannot cipher data, it can rely on ssh to cipher the data over the network
      but data is finally always stored in clear text.
    </p>

    <h5>Tar</h5>

    <p>
      There is no native support for ciphering with <i>tar</i>. You can however pipe <i>tar</i>'s output
      to openssl to cipher the generated backup on fly as a whole.
    </p>
    <code class=block>
	<b>root@terre:/mnt/memdisk#  tar -czf - /usr/bin | openssl enc -e -aes256 -out backup.tar.gz.crypted</b>
	tar: Removing leading `/' from member names
	enter aes-256-cbc encryption password:
	Verifying - enter aes-256-cbc encryption password:
	<e>*** WARNING : deprecated key derivation used.</e>
	<e>Using -iter or -pbkdf2 would be better.</e>
	tar: Removing leading `/' from hard link targets
	<b>root@terre:/mnt/memdisk# openssl enc -d -aes256 -in backup.tar.gz.crypted | tar -xz</b>
	enter aes-256-cbc decryption password:
	<e>*** WARNING : deprecated key derivation used.</e>
	<e>Using -iter or -pbkdf2 would be better.</e>
	<b>root@terre:/mnt/memdisk# tar -czf - /usr/bin | openssl enc -e -aes256 -out backup.tar.gz.crypted -pass file:pass.txt</b>
	tar: Removing leading `/' from member names
	<e>*** WARNING : deprecated key derivation used.</e>
	<e>Using -iter or -pbkdf2 would be better.</e>
	tar: Removing leading `/' from hard link targets
	<b>root@terre:/mnt/memdisk# openssl enc -d -aes256 -in backup.tar.gz.crypted -pass file:pass.txt | tar -xz</b>
	<e>*** WARNING : deprecated key derivation used.</e>
	<e>Using -iter or -pbkdf2 would be better.</e>
	<b>root@terre:/mnt/memdisk#</b>
    </code>
    <p>
      with openssl, <i>tar</i> has both the ability to provide the password/passphrase from an interactive prompt and from a protected file.
      However you will have to remember which algorithm you used in adition to the passphrase. The ciphering being done
      as a whole, you will have to decipher the whole backup even to just restore a single file.
      If the backup is large, this may take a long time and may require to download a lot of stuff from a remote storage.
    </p>
    <p>
      We see that ciphering with <i>tar</i> is possible at the cost of some complex command-line. But this is
      error-prone as we see the shown warning that the key derivation function is deprecated and we should switch to
      another one. Moreover you will have to remember which key derivation function and its parameters in addition to the
      passphrase you provided and in addition to the ciphering algorithm used.
    </p>
    <p>
      <u>Note:</u> you can also use <i>openssl</i> with <i>dar</i> as we did for <i>tar</i> but it brings all the drawbacks we saw with <i>tar</i>
    </p>

    <h4>Asymmetric encryption</h4>

    <p>
      The objective is to create a backup ciphered using GnuPG public/private key pair, restore the whole backup and
      restore a single file from it. We will also use compression (gzip level 6) as it may make sense for the
      corresponding use cases (data exchange over Internet for example).
    </p>
    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -K gnupg::root@terre.systeme-solaire.espace -R SRC -z6 -q</b>
	<b>terre:/mnt/memdisk# dar -l backup -q</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	Archive version format               : 11
	Compression algorithm used           : <e>gzip</e>
	Compression block size used          : 0
	Symmetric key encryption used        : <e>AES 256</e>
	Asymmetric key encryption used       : <e>gnupg</e>
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : N/A
	Catalogue size in archive            : 68669 bytes

	Archive is composed of 1 file(s)
	File size: 158261425 bytes
	The global data compression ratio is:   64%

	CATALOGUE CONTENTS :

	total number of inode : 2593
	fully saved           : 2593
	binay delta patch     : 0
	inode metadata only   : 0
	distribution of inode(s)
	- directories        : 1
	- plain files        : 2157
	- symbolic links     : 435
	- named pipes        : 0
	- unix sockets       : 0
	- character devices  : 0
	- block devices      : 0
	- Door entries       : 0
	hard links information
	- number of inode with hard link           : 0
	- number of reference to hard linked inodes: 0
	destroyed entries information
	0 file(s) have been record as destroyed since backup of reference

	<b>terre:/mnt/memdisk# ls -l backup.1.dar</b>
	-rw-r--r-- 1 root root 158261425 Nov  9 16:04 backup.1.dar
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      As displayed in the backup header output above the underlying encryption is a symmetric encryption
      (AES 256 by default), but the AES key is stored ciphered using the private key of the backup recipient
      which email address is provided (or email adresses, if more than one
      recipient is expected). This key is randomly chosen by dar and stored ciphered in the archive header.
      Thus the overall behavior, performance and security of GnuPG withing dar is equivalent
      to the one of the symmetrical algorithm chosen, with the ability to quickly restore some or all files
      from an archive, and not waiting/downloading first the whole backup to unciphered it.
    </p>
    <p>
      Seen above no password or passphrase is asked as the recipient email is ourselves
      (root@terre.systeme-solaire.espace). Let's cipher for another recipient:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -K <e class=blue>gnupg::dar.linux@free.fr</e> -R SRC -z6 -q -w</b>
	<b>terre:/mnt/memdisk# ls -l backup.1.dar</b>
	-rw-r--r-- 1 root root <e>158230913</e> Nov  9 16:22 backup.1.dar
	<b>terre:/mnt/memdisk# dar -l backup -q</b>
	FATAL error, aborting operation: Unexpected error reported by GPGME: <e>No secret key</e>
	<b>terre:/mnt/memdisk# dar -c backup -K <e class=blue>gnupg::dar.linux@free.fr,root@terre.systeme-solaire.espace</e> -R SRC -z6 -q -w</b>
	<b>terre:/mnt/memdisk# dar -l backup -q</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	Archive version format               : 11
	Compression algorithm used           : gzip
	Compression block size used          : 0
	Symmetric key encryption used        : AES 256
	Asymmetric key encryption used       : gnupg
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : N/A
	Catalogue size in archive            : 68624 bytes

	Archive is composed of 1 file(s)
	File size: 158252223 bytes
	The global data compression ratio is:   64%

	CATALOGUE CONTENTS :

	total number of inode : 2593
	fully saved           : 2593
	binay delta patch     : 0
	inode metadata only   : 0
	distribution of inode(s)
	- directories        : 1
	- plain files        : 2157
	- symbolic links     : 435
	- named pipes        : 0
	- unix sockets       : 0
	- character devices  : 0
	- block devices      : 0
	- Door entries       : 0
	hard links information
	- number of inode with hard link           : 0
	- number of reference to hard linked inodes: 0
	destroyed entries information
	0 file(s) have been record as destroyed since backup of reference

	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      Here we saw that ciphering for a recipient different than ourself does not allow us to read the resulting backup,
      however we can define several recipients and if we add ourself, we can read the backup as well as our primary
      recipients.
    </p>

    <h5>Rsync</h5>

    <i>rsync</i> is not able to perform asymmetric encryption of backed up files.

    <h5>Tar</h5>
    <p>
      <i>Tar</i> cannot hold asymmetrical encryption alone, as for symmetrical encryption we must use an external tool
      that performes the ciphering operation outside the backup.
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# tar -czf - SRC | gpg --encrypt --recipient <e class=blue>root@terre.systeme-solaire.espace</e> --output backup.tar.gz.gpg</b>
	<b>terre:/mnt/memdisk# ls -l backup.tar.gz.gpg</b>
	-rw-r--r-- 1 root root <e>155337814</e> Nov  9 16:45 backup.tar.gz.gpg
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# gpg --decrypt backup.tar.gz.gpg | tar -xzf -</b>
	gpg: encrypted with 3072-bit RSA key, ID 97E13D38B007DF30, created 2020-08-08
	"root@terre &lt;root@terre.systeme-solaire.espace&gt;"
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# tar -czf - SRC | gpg --encrypt --recipient <e class=blue>dar.linux@free.fr</e> --output backup.tar.gz.gpg</b>
	<b>terre:/mnt/memdisk#  gpg --decrypt backup.tar.gz.gpg | tar -xzf -</b>
	gpg: encrypted with 4096-bit RSA key, ID DB0A2141A4D96ECA, created 2012-09-13
	"Denis Corbin (http://dar.linux.free.fr/) &lt;dar.linux@free.fr&gt;"
	gpg: decryption failed: <e>No secret key</e>

	gzip: stdin: unexpected end of file
	tar: Child returned status 1
	tar: Error is not recoverable: exiting now
	<b>terre:/mnt/memdisk#terre:/mnt/memdisk# tar -czf - SRC | gpg --encrypt --recipient <e class=blue>dar.linux@free.fr</e> \</b>
	&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--recipient <e class=blue>root@terre.systeme-solaire.espace</e> --output backup.tar.gz.gpg
	<b>terre:/mnt/memdisk#  gpg --decrypt backup.tar.gz.gpg | tar -xzf -</b>
	gpg: encrypted with 4096-bit RSA key, ID DB0A2141A4D96ECA, created 2012-09-13
	"Denis Corbin (http://dar.linux.free.fr/) &lt;dar.linux@free.fr&gt;"
	gpg: encrypted with 3072-bit RSA key, ID 97E13D38B007DF30, created 2020-08-08
	"root@terre &lt;root@terre.systeme-solaire.espace&gt;"
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      Same as for symmetric encryption, the fact that the whole backup is ciphered at once implies to download
      back the whole backup even to recover just one file.
    </p>



    <h4>Protection against plain-text attack</h4>

    <h5>Dar</h5>

    <code class=block>
      <b>devuan:/mnt/memdisk# time dar -c backup -K "aes256:hello world!" -at -1 0 -R SRC -q -w</b>
      9.782u 3.413s <e>0:06.28</e> 210.0%    0+0k 0+0io 0pf+0w
      <b>devuan:/mnt/memdisk# la backup.1.dar</b>
      -rw-r--r-- 1 root root 1572<e>706497</e> Nov  9 14:50 backup.1.dar
      <b>devuan:/mnt/memdisk# time dar -c backup -K "aes256:hello world!" -at -1 0 -R SRC -q -w</b>
      9.173u 2.845s <e>0:05.50</e> 218.3%    0+0k 0+0io 0pf+0w
      <b>devuan:/mnt/memdisk# la backup.1.dar</b>
      -rw-r--r-- 1 root root 1572<e>655217</e> Nov  9 14:50 backup.1.dar
      <b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      When ciphering the same data several times (with symmetric or asymmetric encryption),
      the resulting backup size changes each time. This is due to the garbage (the elastic
      buffer) dar adds at the beginnning and at the end of the data to cipher. This way,
      even if a dar backup has well known structure it is not easy to know precisely where
      they are positionned in the backup file, which makes plain-text attack much more difficult
      to succeed if even possible in a reasonable time.
    </p>


    <h5>Rsync</h5>
    <p>
      <i>rsync</i> does not provide any way to cipher the backup, it is thus not concerned by
      protecting against plain-text attack.
    </p>

    <h5>Tar</h5>

    <code class=block>
      <b>devuan:/mnt/memdisk/SRC# time ../tar.backup ../backup.tar.crypted usr</b>
      4.112u 2.343s 0:04.72 136.6%    0+0k 0+0io 0pf+0w
      <b>devuan:/mnt/memdisk/SRC# ls -l ../bac</b>
      backup.1.dar        backup.tar.crypted
      <b>devuan:/mnt/memdisk/SRC# ls -l ../backup.tar.crypted</b>
      -rw-r--r-- 1 root root <e>1603594272</e> Nov  9 14:56 ../backup.tar.crypted
      <b>devuan:/mnt/memdisk/SRC# time ../tar.backup ../backup.tar.crypted usr</b>
      3.952u 2.564s 0:04.79 135.9%    0+0k 0+0io 0pf+0w
      <b>devuan:/mnt/memdisk/SRC# ls -l ../backup.tar.crypted</b>
      -rw-r--r-- 1 root root <e>1603594272</e> Nov  9 14:56 ../backup.tar.crypted
      <b>devuan:/mnt/memdisk/SRC#</b>
    </code>

    <p>
      <i>tar</i> by itself does not provide any ciphering mechanism, however you can
      cipher the <i>tar</i> generated backups with external tool (for example <i>openssl</i>
      for symmetric encryption or <i>gpg</i> for asymmetric encryption). However none
      of these mechanism protect against plain-text attack: tar backup have somehow
      predictable header contents.
    </p>

    <h4>Key Derivation Function</h4>

    <h5>Dar</h5>

    <p>
      <i>dar</i> uses <code>argon2</code> by default, with 10,000 iterations. It can
      also use pkcs5 v2 (pbkdf2) with md5, sha1 or sha512 algorithm. The user
      is able to set the KDF function and iteration count, so we are able to measure
      the execution time variation added by the iteration count (taking into account that the data to cipher also
      changes depending on the amount of random garbage <i>dar</i> wraps it with):
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>100k</e>:<e class=blue>sha1</e> -w -q</b>
	4.904u 0.572s <e>0:05.49</e> 99.6%     0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>500k</e>:sha1 -w -q</b>
	5.805u 0.272s <e>0:06.08</e> 99.8%     0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>1M</e>:sha1 -w -q</b>
	6.852u 0.308s <e>0:07.18</e> 99.5%     0+0k 0+0io 0pf+0w
	time dar -c backup -R SRC -K aes:hello --kdf-param 10k:argon2 -w -q
	5.092u 0.870s 0:03.50 170.2%    0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>10k</e>:<e class=blue>argon2</e> -w -q</b>
	5.232u 0.760s <e>0:03.54</e> 169.2%    0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>20k</e>:argon2 -w -q</b>
	5.778u 0.822s <e>0:04.14</e> 159.1%    0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>100k</e>:argon2 -w -q</b>
	10.613u 0.831s <e>0:09.00</e> 127.1%   0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# time dar -c backup -R SRC -K aes:hello --kdf-param <e>1M:</e>argon2 -w -q</b>
	66.862u 0.666s <e>1:05.14</e> 103.6%   0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>

    <p>
      <i>rsync</i> does not provide any way to cipher the backup, it is not concerned by KDF.
    </p>

    <h5>Tar</h5>

    <p>
      As of today (year 2020) <i>openssl</i> only supports PBKDF2: no support for argon2 is available.
      <a href="https://en.wikipedia.org/wiki/Argon2">Argon2</a> was the winner of the Password Hashing
      Competition in July 2015. <a href="https://en.wikipedia.org/wiki/PBKDF2">PBKDF2</a> has been
      published by the IETF in September 2000 with the
      <a href="https://tools.ietf.org/html/rfc2898">RCF 2898</a>

    </p>

    <h4>File change detection</h4>

    <p>
      In order stress each backup software on that aspect, we will use an ugly
      script <a href="#always_change">always_change</a> that loops forever permanently
      invoking <code>touch</code> on a given file. For the test, we create a source
      tree to backup, containing a file of 1 MiB on which we will
      apply this script:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# mkdir SRC</b>
	<b>terre:/mnt/memdisk# dd if=/dev/zero of=SRC/hello_world bs=10240 count=1024</b>
	1024+0 records in
	1024+0 records out
	10485760 bytes (10 MB, 10 MiB) copied, 0.0107294 s, 977 MB/s
	<b>terre:/mnt/memdisk# ./always_change SRC/hello_world &</b>
	[1] 7433
	<b>terre:/mnt/memdisk# stat SRC/hello_world</b>
	File: SRC/hello_world
	Size: 10485760        Blocks: 20480      IO Block: 4096   regular file
	Device: 1bh/27d Inode: 375588      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
	Access: 2020-11-10 16:34:13.806106695 +0100
	<e>Modify: 2020-11-10 16:34:13.806106695 +0100</e>
	Change: 2020-11-10 16:34:13.806106695 +0100
	Birth: -
	<b>terre:/mnt/memdisk# stat SRC/hello_world</b>
	File: SRC/hello_world
	Size: 10485760        Blocks: 20480      IO Block: 4096   regular file
	Device: 1bh/27d Inode: 375588      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
	Access: 2020-11-10 16:34:14.838104981 +0100
	<e>Modify: 2020-11-10 16:34:14.838104981 +0100</e>
	Change: 2020-11-10 16:34:14.838104981 +0100
	Birth: -
	<b>terre:/mnt/memdisk# jobs</b>
	[1]  + Running                       ./always_change SRC/hello_world
	<b>terre:/mnt/memdisk# ls -l SRC</b>
	total 10240
	-rw-r--r-- 1 root root 10485760 Nov 10 16:34 hello_world
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>

    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R SRC -q</b>
	<e>WARNING! File modified while reading it for backup, but no more retry allowed: /mnt/memdisk/SRC/hello_world</e>
	<b>terre:/mnt/memdisk# dar -l backup</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[<e>DIRTY</e>][ ]       [---][  99%][X]  -rw-r--r--   0        0       10 Mio  Tue Nov 10 16:34:55 2020        hello_world
	<b>terre:/mnt/memdisk# dar -x backup -R DST</b>
	<e>File /mnt/memdisk/DST/hello_world has changed during backup and is probably not saved in a valid state ("dirty file"),</e>
	<e>do you want to consider it for restoration anyway? [return = YES | Esc = NO]</e>
	Continuing...


	--------------------------------------------
	1 inode(s) restored
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 1
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      <ul>
	<li>
	  <i>dar</i> detects properly the file change and issues a warning during the backup.
	</li>
	<li>
	  It even retries to save the file several times (3 times by default).
	</li>
	<li>
	  the resulting backup keeps trace of this context by flagging the file as <code>DIRTY</code>
	</li>
	<li>
	  When restoring the data, a warning shows (default behavior) and the user is requested for confirmation.
	</li>
      </ul>
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rsync -arvHAXqz --delete SRC DST</b>
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
     <i>rsync</i> does not shows anything nor behaves differently (no retry, no change notification).
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# tar -cf backup.tar SRC</b>
	<e>tar: SRC/hello_world: file changed as we read it</e>
	<b>terre:/mnt/memdisk# tar -tvf backup.tar</b>
	drwxr-xr-x root/root         0 2020-11-10 16:33 SRC/
	-rw-r--r-- root/root  10485760 2020-11-10 16:41 SRC/hello_world
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xf ../backup.tar</b>
	<b>terre:/mnt/memdisk/DST# ls -l SRC</b>
	total 10240
	-rw-r--r-- 1 root root 10485760 Nov 10 16:41 hello_world
	<b>terre:/mnt/memdisk/DST#</b>
    </code>

    <p>
      <ul>
	<li><i>tar</i> detects properly the file change and issues a warning during the backup.
	</li>
	<li>
	  it does not tries to save the file
	</li>
	<li>
	  the resulting backup keeps no visible trace of this possible data corruption
	</li>
	<li>
	  When restoring the data, no warning is issued and the restoration proceed as if the file was saved properly
	</li>
      </ul>

    <h4>Multi-level backup</h4>

    <p>
      For this test we make a full backup of a Linux source tree, then
      rename the <i>Documentation</i> directory as <i>doc</i> and make a
      differential backup of the whole. Renaming files is expected to do at worse
      the same as removing some and adding new ones, whe should not see all data
      saved again:
    </p>

    <h5>Dar</h5>

    <code class=block>
	<b>devuan:/mnt/memdisk# du -B1 -s SRC</b>
	<e>1121144832</e>      SRC
	<b>devuan:/mnt/memdisk# dar -c full -R SRC -z6 -q</b>
	<b>devuan:/mnt/memdisk# cd SRC/linux-5.9.2/</b>
	<b>devuan:/mnt/memdisk/SRC/linux-5.9.2# mv Documentation/ doc</b>
	<b>devuan:/mnt/memdisk/SRC/linux-5.9.2# cd ../..</b>
	<b>devuan:/mnt/memdisk# dar -c diff -A full -R SRC -z6 -q</b>
	<b>devuan:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root  <e>17858927</e> Nov  1 18:18 diff.1.dar
	-rw-r--r-- 1 root root <e>219047658</e> Nov  1 18:14 full.1.dar
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# dar -x full -R DST -q</b>
	<b>devuan:/mnt/memdisk# dar -x diff -R DST -q -w</b>
	<b>devuan:/mnt/memdisk# diff -r SRC DST && echo "same data" || echo "different data"</b>
	same data
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      We can see that the restoration of the full and differential backup over it
      lead to the exact same directory tree as the source saved files.
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# rsync -arHAXz --delete --info=stats SRC/* BACKUP</b>

	<e>sent 214,380,105 bytes  received 1,359,591 bytes  9,180,412.60 bytes/sec</e>
	total size is 954,869,250  speedup is 4.43
	<b>devuan:/mnt/memdisk# cd SRC/linux-5.9.2/</b>
	<b>devuan:/mnt/memdisk/SRC/linux-5.9.2# mv Documentation/ doc</b>
	<b>devuan:/mnt/memdisk/SRC/linux-5.9.2# cd ../..</b>
	<b>devuan:/mnt/memdisk# rsync -arHAXz --delete --info=stats SRC/* BACKUP</b>

	<e>sent 12,923,292 bytes  received 680,190 bytes  3,886,709.14 bytes/sec</e>
	total size is 954,869,250  speedup is 70.19
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# rsync -arHAXz --delete --info=stats BACKUP/* DST</b>

	<e class=blue>sent 214,371,610 bytes  received 1,359,603 bytes  9,180,051.62 bytes/sec</e>
	total size is 954,869,250  speedup is 4.43
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      We see that after the modification the amount of data pushed to the backup by <i>rsync</i>
      passes from 214 MiB to only 12 MiB we can consider this as a differential backup, thus this
      part of the multi-level backup aspect is addressed, but we have lost the access to the first
      backup: it has been overwritten by the new one, so we lose history but that's a different feature.
    </p>

    <h5>Tar</h5>

    <code class=block>
	<b>devuan:/mnt/memdisk# tar --listed-incremental=snapshot.file -czf full.tar.gz SRC</b>
	<b>devuan:/mnt/memdisk# cd SRC/linux-5.9.2/</b>
	<b>devuan:/mnt/memdisk/SRC/linux-5.9.2# mv Documentation/ doc</b>
	<b>devuan:/mnt/memdisk/SRC/linux-5.9.2# cd ../..</b>
	<b>devuan:/mnt/memdisk# tar --listed-incremental=snapshot.file -czf diff.tar.gz SRC</b>
	<b>devuan:/mnt/memdisk# ls -l</b>
	total 190488
	drwxr-xr-x 3 root root        60 Oct 31 19:37 SRC
	-rw-r--r-- 1 root root   <e>9654445</e> Oct 31 19:49 diff.tar.gz
	-rw-r--r-- 1 root root <e>184036391</e> Oct 31 19:49 full.tar.gz
	-rw-r--r-- 1 root root   <e>1361962</e> Oct 31 19:49 snapshot.file
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# tar --listed-incremental=/dev/null -xf ../full.tar.gz</b>
	<b>devuan:/mnt/memdisk/DST# tar --listed-incremental=/dev/null -xf ../diff.tar.gz</b>
	<b>devuan:/mnt/memdisk/DST# cd ..</b>
	<b>devuan:/mnt/memdisk# diff -r SRC DST/SRC && echo "same data" || echo "different data"</b>
	same data
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      Here too, we got the exact same directory as original and modified data
    </p>


    <h4>Binary Delta</h4>

    <p>
      To evaluate the ability the support for binary delta, we will make a first
      backup of a Debian ISO image, of which we will modify one bit using the
      <a href="#bitflip">bitflip</a> script, then
      make a differential backup of it. We expect to see the differential backup not
      resaving the whole file, and though the restoration of the full and differential
      backup matching the modified file.
    </p>

    <h5>Dar</h5>

    <code class=block>
	<b>devuan:/mnt/memdisk# dar -c full -z6 -R SRC --delta sig -q</b>
	<b>devuan:/mnt/memdisk# ./bitflip 100000 SRC/debian-10.6.0-amd64-DVD-2.iso</b>
	<b>devuan:/mnt/memdisk# dar -c diff -A full -z6 -R SRC -q</b>
	<b>devuan:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root        <e>643</e> Nov  1 19:45 diff.1.dar
	-rw-r--r-- 1 root root <e>4704429776</e> Nov  1 19:05 full.1.dar
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# dar -x full -R DST -q</b>
	<b>devuan:/mnt/memdisk# dar -x diff -R DST -q</b>
	<b>devuan:/mnt/memdisk# diff -s SRC/debian-10.6.0-amd64-DVD-2.iso DST/debian-10.6.0-amd64-DVD-2.iso</b>
	Files SRC/debian-10.6.0-amd64-DVD-2.iso and DST/debian-10.6.0-amd64-DVD-2.iso are identical
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      For <i>dar</i> the backup we used:
    </p>
    <ul>
      <li>4.3 GiB for the full backup (compression ratio of 0,21 %)</li>
      <li>614 bytes for the differential backup (compression ratio of 99,99998%)</li>
    </ul>

    <h5>Rsync</h5>

    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# rsync -arHAX --info=stats SRC/* DST</b>

	<e>sent 4,688,066,109 bytes  received 35 bytes  284,125,220.85 bytes/sec</e>
	total size is 4,686,921,728  speedup is 1.00
	<b>devuan:/mnt/memdisk# ./bitflip  100000 SRC/debian-10.6.0-amd64-DVD-2.iso</b>
	<b>devuan:/mnt/memdisk# rsync -arHAX --info=stats SRC/* DST</b>

	<e>sent 4,688,066,109 bytes  received 35 bytes  302,455,880.26 bytes/sec</e>
	<e class=red>total size is 4,686,921,728  speedup is 1.00</e>
	<b>devuan:/mnt/memdisk# rsync -arHAXt --info=stats <e class=blue>--no-whole-file</e> SRC/* DST</b>

	<e>sent 342,469 bytes  received 547,803 bytes  30,178.71 bytes/sec</e>
	total size is 4,686,921,728  speedup is 5,264.60
	<b>devuan:/mnt/memdisk# diff -s SRC/debian-10.6.0-amd64-DVD-2.iso DST/debian-10.6.0-amd64-DVD-2.iso</b>
	Files SRC/debian-10.6.0-amd64-DVD-2.iso and DST/debian-10.6.0-amd64-DVD-2.iso are identical
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      We had to use <code>--no-whole-file</code> to see the binary delta in action with
      <i>rsync</i>. This feature is not activated when copying on local disk as it
      does not makes sense (for <i>rsync</i>) because the computation time needed for the
      binary delta takes more time the the byte to byte copy and because <i>rsync</i> does
      not store just the delta (no backup history) but modifies the existing backup.
      Anyway, binary delta is supported (of course!) by <i>rsync</i>.
    </p>

    <h5>Tar</h5>

    <code class=block>
	<b>devuan:/mnt/memdisk# tar --listed-incremental=snapshot.file -czf full.tar.gz SRC</b>
	<b>devuan:/mnt/memdisk#  ./bitflip 100000 SRC/debian-10.6.0-amd64-DVD-2.iso</b>
	<b>devuan:/mnt/memdisk# tar --listed-incremental=snapshot.file -czf diff.tar.gz SRC</b>
	<b>devuan:/mnt/memdisk# ls -l</b>
	total 9133304
	drwxr-xr-x 2 root root         40 Oct 31 17:31 SRC
	-rwxr--r-- 1 root root        460 Oct 31 16:34 bitflip
	-rw-r--r-- 1 root root <e class=red>4676243904</e> Oct 31 17:28 diff.tar.gz
	-rw-r--r-- 1 root root <e>4676244172</e> Oct 31 17:24 full.tar.gz
	-rw-r--r-- 1 root root        107 Oct 31 17:28 snapshot.file
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# tar --listed-incremental=/dev/null -xf ../full.tar.gz</b>
	<b>devuan:/mnt/memdisk/DST# tar --listed-incremental=/dev/null -xf ../diff.tar.gz</b>
	<b>devuan:/mnt/memdisk/DST# diff -s ../SRC/debian-10.6.0-amd64-DVD-2.iso SRC/debian-10.6.0-amd64-DVD-2.iso</b>
	Files ../SRC/debian-10.6.0-amd64-DVD-2.iso and SRC/debian-10.6.0-amd64-DVD-2.iso are identical
	<b>devuan:/mnt/memdisk/DST#</b>
    </code>

    <p>
      For <i>tar</i> the backup used:
    </p>

    <ul>
      <li>4.3 GiB for full backup (compression ratio of 0,22 %)</li>
      <li>4.3 GiB for the differential backup (compression ratio of 0,22 %)</li>
    </ul>
    <p>
      Binary delta is not supported by <i>tar</i>
    </p>

    <h4>Detection suspicious modifications</h4>

    <p>For this test we will use the <a href="#hide_change">hide_change</a> script that rely
      on the <a href="#bitflip">bitflip</a> script
      seen above and try to hide the modifications performed, as a virus, keylogger or rootkit would
      tend to do. We will make a full backup before
      the modification and a differential backup after, then observe the behavior.

     <p>
      Here follows the script in action, we see no change using <code>ls -l</code>
      while <code>stat</code> shows the exact same information:
      <ul>
	<li>file size</li>
	<li>block used</li>
	<li>Inode number</li>
	<li>permissions</li>
	<li>user and group ownership</li>
	<li>last access time</li>
	<li>last modification time</li>
      </ul>
      The only change concerns the inode change time (ctime) that cannot be set manually
      and signals that some inode properties (but no file content) has changed. This
      condition occurs, when changing the file permission, ownership, extended
      attributes and so on, but should not occur when only file's data has changed.
    </p>

    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# mkdir SRC</b>
	<b>terre:/mnt/memdisk# echo "Hello World!" > SRC/file.txt</b>
	<b>terre:/mnt/memdisk# cat SRC/file.txt</b>
	Hello World!
	<b>terre:/mnt/memdisk# ls -l SRC/file.txt</b>
	-rw-r--r-- 1 root root 13 Nov 12 13:13 SRC/file.txt
	<b>terre:/mnt/memdisk# stat SRC/file.txt</b>
	File: SRC/file.txt
	Size: 13              Blocks: 8          IO Block: 4096   regular file
	Device: 1bh/27d Inode: 424690      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
	Access: 2020-11-12 13:13:19.021978762 +0100
	Modify: 2020-11-12 13:13:09.213998852 +0100
	Change: 2020-11-12 13:13:<e>09.213998852</e> +0100
	Birth: -
	<b>terre:/mnt/memdisk# ./hide_change SRC/file.txt</b>
	<b>terre:/mnt/memdisk# ls -l SRC/file.txt</b>
	-rw-r--r-- 1 root root 13 Nov 12 13:13 SRC/file.txt
	<b>terre:/mnt/memdisk# stat SRC/file.txt</b>
	File: SRC/file.txt
	Size: 13              Blocks: 8          IO Block: 4096   regular file
	Device: 1bh/27d Inode: 424690      Links: 1
	Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
	Access: 2020-11-12 13:13:19.021978762 +0100
	Modify: 2020-11-12 13:13:09.213998852 +0100
	Change: 2020-11-12 13:13:<e>39.549936636</e> +0100
	Birth: -
	<b>terre:/mnt/memdisk# cat SRC/file.txt</b>
	<e class=red>L</e>ello World!
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# mkdir SRC</b>
	<b>terre:/mnt/memdisk# echo "Hello World!" > SRC/file.txt</b>
	<b>terre:/mnt/memdisk# dar -c full -R SRC -N</b>


	--------------------------------------------
	1 inode(s) saved
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 1
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# ./hide_change SRC/file.txt</b>
	<b>terre:/mnt/memdisk# dar -c diff -A full -R SRC -N -q</b>
	<e>SECURITY WARNING! SUSPICIOUS FILE /mnt/memdisk/SRC/file.txt: ctime changed since archive of reference was done, while no other inode information changed</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      <i>dar</i> issues a warning because of this suspicious condition. Note that we still have the sane file in the full backup, in case of doubt,
      we can compare it with this modified version:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# dar -d full -R SRC -q</b>
	DIFF /mnt/memdisk/SRC/file.txt: <e>different file data, offset of first difference is: 0</e>
	Some file comparisons failed
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      The previous test reports that the first byte to have changed is at offset 0, thus this is not just a metadata change
      that lead to this warning. We can if necessary restore the sane data from the full backup.
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rm -rf SRC</b>
	<b>terre:/mnt/memdisk# mkdir SRC</b>
	<b>terre:/mnt/memdisk# echo "Hello World!" > SRC/file.txt</b>
	<b>terre:/mnt/memdisk# rsync -arvHAX SRC DST</b>
	sending incremental file list
	created directory DST
	SRC/
	SRC/file.txt

	sent 146 bytes  received 65 bytes  422.00 bytes/sec
	total size is 13  speedup is 0.06
	<b>terre:/mnt/memdisk# ./hide_change SRC/file.txt</b>
	<b>terre:/mnt/memdisk# rsync -arvHAX SRC DST</b>
	sending incremental file list

	sent 83 bytes  received 13 bytes  192.00 bytes/sec
	total size is 13  speedup is 0.14
	<b>terre:/mnt/memdisk# cat SRC/file.txt</b>
	<e>Lello World!</e>
	<b>terre:/mnt/memdisk# cat DST/SRC/file.txt</b>
	<e class=red>Hello World!</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      rsync has not reported the problem, but hopefully it has not synchronized the backup,
      thus we end in a sane version in the DST backup directory though, as user is not aware
      of this potential risk, the virus/ransomware can spread silently.
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rm -rf SRC</b>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir SRC</b>
	<b>terre:/mnt/memdisk# echo "Hello World!" > SRC/file.txt</b>
	<b>terre:/mnt/memdisk# tar --listed-incremental=snapshot.file -cf full.tar SRC</b>
	<b>terre:/mnt/memdisk# ./hide_change SRC/file.txt</b>
	<b>terre:/mnt/memdisk# tar --listed-incremental=snapshot.file -cvf diff.tar SRC</b>
	SRC/
	SRC/file.txt
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xf ../full.tar</b>
	<b>terre:/mnt/memdisk/DST# cat SRC/file.txt</b>
	<e>Hello World!</e>
	<b>terre:/mnt/memdisk/DST# tar -xf ../diff.tar</b>
	<b>terre:/mnt/memdisk/DST# cat SRC/file.txt</b>
	<e class=red>Lello World!</e>
	<b>terre:/mnt/memdisk/DST#</b>
    </code>

    <p>
      As seen above <i>tar</i> does not see any problem, but the file has been resaved
      as a whole (while its last modification time was unchanged) which lead to corrupt the
      new backup with potential harmful data. The good point is that you have still the
      full backup with the sane data. But at a next backup cycle, as you were not notified of the
      risk, you will lose it and keep only the corrupted version of this file.
    </p>

    <h4>Snapshot</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c full -z6 -R /usr <e>--on-fly-isolate snapshot</e></b>


	--------------------------------------------
	267245 inode(s) saved
	including 23 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 267245
	--------------------------------------------
	EA saved for 5 inode(s)
	FSA saved for 237962 inode(s)
	--------------------------------------------
	Now performing on-fly isolation...
	<b>terre:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root 4006060941 Nov 12 15:34 full.1.dar
	-rw-r--r-- 1 root root    6662595 Nov 12 15:34 <e class=blue>snapshot.1.dar</e>
	<b>terre:/mnt/memdisk# dar <e>-C recreated_snapshot</e> -A full -z6 -q</b>
	<b>terre:/mnt/memdisk# ls -al *.dar</b>
	-rw-r--r-- 1 root root 4006060941 Nov 12 15:34 full.1.dar
	-rw-r--r-- 1 root root    7907094 Nov 12 16:33 <e class=blue>recreated_snapshot.1.dar</e>
	-rw-r--r-- 1 root root    6662595 Nov 12 15:34 snapshot.1.dar
	<b>terre:/mnt/memdisk# dar -c diff -A snapshot -R /usr -z</b>


	--------------------------------------------
	23 inode(s) saved
	including 23 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	<e>267222 inode(s) not saved (no inode/file change)</e>
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 267245
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# ls -lh *.dar</b>
	-rw-r--r-- 1 root root  25M Nov 12 16:37 diff.1.dar
	-rw-r--r-- 1 root root 3.8G Nov 12 15:34 full.1.dar
	-rw-r--r-- 1 root root 7.6M Nov 12 16:33 recreated_snapshot.1.dar
	-rw-r--r-- 1 root root 6.4M Nov 12 15:34 snapshot.1.dar
	<b>terre:/mnt/memdisk# dar -c diff2 -A recreated_snapshot -R /usr -z</b>


	--------------------------------------------
	23 inode(s) saved
	including 23 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	<e>267222 inode(s) not saved (no inode/file change)</e>
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 267245
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# dar -c snapshot_alone <e>-A +</e> -R /usr -z</b>


	--------------------------------------------
	23 inode(s) saved
	including 23 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	267222 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 267245
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# <e>touch /usr/local/src</e></b>
	<b>terre:/mnt/memdisk# dar -c faked_diff -A snapshot -R /usr <e class=blue>--dry-run</e> -q -vt</b>
	<e>Adding folder to archive: /usr/local/src</e>
	Saving Filesystem Specific Attributes for /usr/local/src
	<b>terre:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root   25537139 Nov 12 16:37 diff.1.dar
	-rw-r--r-- 1 root root   25537139 Nov 12 16:39 diff2.1.dar
	-rw-r--r-- 1 root root 4006060941 Nov 12 15:34 full.1.dar
	-rw-r--r-- 1 root root    7907094 Nov 12 16:33 recreated_snapshot.1.dar
	-rw-r--r-- 1 root root    6662595 Nov 12 15:34 snapshot.1.dar
	-rw-r--r-- 1 root root   25537142 Nov 12 16:44 <e class=blue>snapshot_alone.1.dar</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>As seen above,a snapshot can be created:
      <ul>
	<li>as part of a backup process (full, differential, incremental or even decremental backup) (<code>--on-fly-isolate</code>)</li>
	<li>from a existing backup (<code>-C</code>)</li>
	<li>alone by a dedicated operation (<code>-A +</code>)</li>
      </ul>
    </p>

    <code class=block>
	<b>root@terre:/mnt/memdisk# ls -l full.1.dar</b>
	-rw-r--r-- 1 root root <e>3895581703</e> Nov 29 21:53 full.1.dar
	<b>root@terre:/mnt/memdisk# dar -l full -q</b>
	FATAL error, aborting operation: Cannot open catalogue: unknown compression
	<b>root@terre:/mnt/memdisk# !bitflip</b>
	bitflip 31124653000 full.1.dar
	<b>root@terre:/mnt/memdisk# dar -l full -q</b>
	Archive version format               : 11
	Compression algorithm used           : gzip
	Compression block size used          : 0
	Symmetric key encryption used        : none
	Asymmetric key encryption used       : none
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : N/A
	<e>Catalogue size in archive            : 7799028 bytes</e>

	Archive is composed of 1 file(s)
	File size: 3895581703 bytes
	The global data compression ratio is:   51%

	CATALOGUE CONTENTS :

	total number of inode : 263480
	fully saved           : 263480
	binay delta patch     : 0
	inode metadata only   : 0
	distribution of inode(s)
	- directories        : 18080
	- plain files        : 216142
	- symbolic links     : 29258
	- named pipes        : 0
	- unix sockets       : 0
	- character devices  : 0
	- block devices      : 0
	- Door entries       : 0
	hard links information
	- number of inode with hard link           : 11
	- number of reference to hard linked inodes: 34
	destroyed entries information
	0 file(s) have been record as destroyed since backup of reference

	<b>root@terre:/mnt/memdisk# <e>bitflip 31124653000 full.1.dar</e></b>
	<b>root@terre:/mnt/memdisk# dar -t full</b>
	Final memory cleanup...
	<e class=red>FATAL error, aborting operation: Cannot open catalogue: unknown compression</e>
	<b>root@terre:/mnt/memdisk# dar -t full <e>-A snapshot</e></b>


	--------------------------------------------
	263503 item(s) treated
	0 item(s) with error
	0 item(s) ignored (excluded by filters)
	--------------------------------------------
	Total number of items considered: 263503
	--------------------------------------------
	<b>root@terre:/mnt/memdisk#</b>
	<b>root@terre:/mnt/memdisk# dar -t full <e>--sequential-read</e></b>
	A problem occurred while reading this archive contents: <e>Cannot open catalogue: unknown compression</e>


	--------------------------------------------
	263503 item(s) treated
	0 item(s) with error
	0 item(s) ignored (excluded by filters)
	--------------------------------------------
	Total number of items considered: 263503
	--------------------------------------------
	<b>root@terre:/mnt/memdisk#</b>

    </code>

    <p>
      Once created a snapshot can be used:
      <ul>
	<li>to create a differential or incremental backup</li>
	<li>to list the files that would be saved (thus which have changed,
	  were added or have been removed) thus all changes since the time the snapshot was made (using the <code>--dry-run</code> option)</li>
	<li>
	  rescue a corrupted backup when the corruption falled into the
	  backup table of content located at the end of the <i>dar</i>
	  backup (only if it has been created based on the backup to rescue
	  either using <code>-C option</code> maybe long after the backup was made
	  or using <code>--on-fly-isolate</code> at the same time the backup was created.
	</li>
      </ul>
      Note, as shown above, a table of content (aka "catalogue") corruption, can also partially be recovered
      using the <code>--sequential-read</code> mode, it will just not let <i>dar</i> remove files that
      were removed since the reference backup was made (this does thus not concern full backups, as here).
    </p>

    <h5>Rsync</h5>
    <p>
      This feature is not supported by <i>rsync</i>.
    </p>

    <h5>Tar</h5>
    <p>
      <i>tar</i> can generate snapshot:
      <ul>
	<li>alone redirecting the backup output to /dev/null</li>
	<li>as part of a backup process (in fact <i>tar</i> cannot do else)</li>
      </ul>
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# tar --listed-incremental=snapshot.file -czf full.tar.gz /usr</b>
	tar: Removing leading `/' from member names
	tar: Removing leading `/' from hard link targets
	<b>terre:/mnt/memdisk# ls -l snapshot.file</b>
	-rw-r--r-- 1 root root 6288644 Nov 12 15:12 snapshot.file
	<b>terre:/mnt/memdisk# cp snapshot.file snapshot.file.ref</b>
	tar --listed-incremental=snapshot.file -cvf /dev/null /usr
	/usr/
	/usr/bin/
	/usr/games/
	/usr/include/
	/usr/include/X11/
	/usr/include/X11/bitmaps/
	/usr/include/arpa/
	/usr/include/asm-generic/
	/usr/include/attr/
	/usr/include/c++/
	/usr/include/c++/
	<e>[...]</e>
	/usr/share/zoneinfo/right/Canada/
	/usr/share/zoneinfo/right/Chile/
	/usr/share/zoneinfo/right/Etc/
	/usr/share/zoneinfo/right/Europe/
	/usr/share/zoneinfo/right/Indian/
	/usr/share/zoneinfo/right/Mexico/
	/usr/share/zoneinfo/right/Pacific/
	/usr/share/zoneinfo/right/SystemV/
	/usr/share/zoneinfo/right/US/
	/usr/share/zsh/
	/usr/share/zsh/site-functions/
	/usr/share/zsh/vendor-completions/
	/usr/src/
	<b>terre:/mnt/memdisk# ls -l sna</b>
	snapshot.file      snapshot.file.ref
	<b>terre:/mnt/memdisk# ls -l snapshot.file*</b>
	-rw-r--r-- 1 root root 6288644 Nov 12 15:20 snapshot.file
	-rw-r--r-- 1 root root 6288644 Nov 12 15:18 snapshot.file.ref
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      If a snapshot can be used (and is in fact required) to make a differential backup, it
      cannot really be used to see the difference a current living filesystem has with a given
      snapshot. Worse, doing so modifies the snapshot, so you have first to make a copy
      to not screw up your backup process. Worse, if incremental backup fails and you have
      not created a copy of the backup, your snapshot being modified you will mostly have to
      remake the whole backup process from the full backup to be sure to not miss backing up
      some modified files. Same thing if you lose by mistake the snapshot file.
    </p>


    <h4>On-fly hashing</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R /usr -g usr/bin -z6 <e>--hash sha1</e></b>


	--------------------------------------------
	0 inode(s) saved
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	8 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 8
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# ls -l *.dar*</b>
	-rw-r--r-- 1 root root 171 Nov 12 17:22 backup.1.dar
	-rw-r--r-- 1 root root  55 Nov 12 17:22 <e>backup.1.dar.sha1</e>
	<b>terre:/mnt/memdisk# sha1sum -c backup.1.dar.sha1</b>
	<e>backup.1.dar: OK</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <p>
      not supported by <i>rsync</i>
    </p>

    <h5>Tar</h5>

    <p>
      not supported by <i>tar</i>
    </p>


    <h4>Custom command during operation</h4>

    <p>
      As an example (but there is much more thing that can be done), we take the case
      of a automounted directory. Such type of volume is mounted only when used, if not
      used no mount point directory shows and unless you know it exists, no backup of
      its content is performed. The idea, is when entering the parent directory at backup
      process to trigger the mount point for the backup to include them.
    <p>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# cat /etc/auto.mnt</b>
	Espace  -defaults,relatime,acl,bg,rsize=8192,wsize=8192       nfs.systeme-solaire.espace:/mnt/Externe/Espace
	Commun  -defaults,relatime,acl,bg,rsize=8192,wsize=8192,ro    nfs.systeme-solaire.espace:/mnt/Externe/Commun
	Backup  -defaults,relatime,acl,bg,rsize=8192,wsize=8192,ro    nfs.systeme-solaire.espace:/mnt/Backup
	<b>terre:/mnt/memdisk# ls -l /mnt/Externe/</b>
	total 4
	drwxr-xr-x 7 root root 4096 Jul 14 17:58 Espace
	<b>terre:/mnt/memdisk# dar -c backup -R / -g /mnt -q</b>
	<b>terre:/mnt/memdisk# dar -l backup</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[Saved][-]       [-L-][     ][ ]  drwxr-xr-x   0        0       0       Wed Oct 21 18:17:07 2020        mnt
	[Saved][-]       [-L-][     ][ ]  drwxr-xr-x   1000     1002    0       Mon Nov  9 11:56:54 2020        mnt/localdisk
	[Saved][-]       [---][-----][ ]  lrwxrwxrwx   0        0       0       Thu Aug 15 23:29:46 2019        mnt/Backup
	[Saved][-]       [---][     ][ ]  drwxr-xr-x   0        0       0       Thu Nov 12 17:42:11 2020        mnt/Externe
	[Saved][-][Saved][---][     ][ ]  drwxr-xr-x   0        0       0       Tue Jul 14 17:58:57 2020        <e class=blue>mnt/Externe/Espace</e>
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# rm backup.1.dar</b>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g mnt -q '-&lt;' mnt '-=' 'file %p/Externe/Backup %p/Externe/Commun'</b>
	/mnt/Externe/Backup: directory
	/mnt/Externe/Commun: directory
	<b>terre:/mnt/memdisk# dar -l backup</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[Saved][-]       [-L-][     ][ ]  drwxr-xr-x   0        0       0       Wed Oct 21 18:17:07 2020        mnt
	[Saved][-]       [-L-][     ][ ]  drwxr-xr-x   1000     1002    0       Mon Nov  9 11:56:54 2020        mnt/localdisk
	[Saved][-]       [---][-----][ ]  lrwxrwxrwx   0        0       0       Thu Aug 15 23:29:46 2019        mnt/Backup
	[Saved][-]       [---][     ][ ]  drwxr-xr-x   0        0       0       Thu Nov 12 18:01:41 2020        mnt/Externe
	[Saved][-][Saved][---][     ][ ]  drwxr-x---   993      1002    0       Wed Nov 11 10:21:55 2015        <e>mnt/Externe/Commun</e>
	[Saved][-][Saved][---][     ][ ]  drwxr-xr-x   0        0       0       Sun Sep 13 12:22:24 2020        <e>mnt/Externe/Backup</e>
	[Saved][-][Saved][---][     ][ ]  drwxr-xr-x   0        0       0       Tue Jul 14 17:58:57 2020        <e class=blue>mnt/Externe/Espace</e>
	<b>terre:/mnt/memdisk# ls -l /mnt/Externe/</b>
	total 12
	drwxr-xr-x 9 root   root   4096 Sep 13 12:22 Backup
	drwxr-x--- 4 commun maison 4096 Nov 11  2015 Commun
	drwxr-xr-x 7 root   root   4096 Jul 14 17:58 Espace
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      In the previous example we see that the /mnt/Externe directory is a mount point containing three auto-mounted
      volumes: <code>Espace</code>, <code>Commun</code> and <code>Backup</code>. At first only <code>Espace</code>
      was mounted. Performing a backup without care will skip the two other directories.
    </p>
    <p>
      In a second time, thanks to the <code>-&lt;</code> and <code>-=</code> options, we instructed <i>dar</i> to
      run the <code>file</code> command on the two missing directories when entering <code>/mnt</code>.
      As a result, we now see both of them in the backup. We could do that before executing the backup, but as
      the backup may include many other
      directories the time between such operation done before starting the backup and the time the backup finally
      saves the automount point at <code>/mnt/Externe</code> may exceed the automount timeout leading them to be
      unmounted and disappear before the backup process reaches them.
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin --hash sha512 -s 100M -q</b>
	<b>terre:/mnt/memdisk# ls -l backup.*</b>
	-rw-r--r-- 1 root root 104857600 Nov 12 18:30 backup.1.dar
	-rw-r--r-- 1 root root       143 Nov 12 18:30 backup.1.dar.sha512
	-rw-r--r-- 1 root root 104857600 Nov 12 18:30 backup.2.dar
	-rw-r--r-- 1 root root       143 Nov 12 18:30 backup.2.dar.sha512
	-rw-r--r-- 1 root root 104857600 Nov 12 18:30 backup.3.dar
	-rw-r--r-- 1 root root       143 Nov 12 18:30 backup.3.dar.sha512
	-rw-r--r-- 1 root root  63577207 Nov 12 18:30 backup.4.dar
	-rw-r--r-- 1 root root       143 Nov 12 18:30 backup.4.dar.sha512
	<b>terre:/mnt/memdisk# dar -t backup <e>-E 'sha512sum -c %p/%b.%N.%e.sha512'</e></b>
	backup.4.dar: OK
	backup.1.dar: OK
	backup.2.dar: OK
	backup.3.dar: OK
	backup.4.dar: OK


	--------------------------------------------
	2594 item(s) treated
	0 item(s) with error
	0 item(s) ignored (excluded by filters)
	--------------------------------------------
	Total number of items considered: 2594
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      In this example, this we used slicing with on-fly hashing which generated for each slice
      the corresponding sha512 hash file. Then we tested the archive content and at the same time the hash
      files thanks to the <code>-E</code> option. Of course any user command or shell or python script,
      can be used instead, and for backup, restoration, testing, snashotting,...
    </p>

    <h5>Rsync</h5>
    <p>
      not supported by <i>rsync</i>
    </p>

    <h5>Tar</h5>
    <p>
      <i>tar</i> has the <code>-F option</code> to launch a command after each tape,
      but it is only available with multi-volume tar archive, which in turn
      cannot be used with compression. Thus we won't test it, as it is quite
      restrictive and does not match any common use cases.
    </p>

    <h4>Dry-run execution</h4>
    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk/A# ls -l</b>
	<e>total 0</e>
	<b>terre:/mnt/memdisk/A# dar -c backup -R / -g usr/bin --dry-run</b>


	--------------------------------------------
	2594 inode(s) saved
	including 5 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	34 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2628
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 2154 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk/A# ls -l</b>
	<e>total 0</e>
	<b>terre:/mnt/memdisk/A#</b>
    </code>

    <h5>Rsync</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rsync -arHAX --dry-run /usr/bin DST</b>
	<b>terre:/mnt/memdisk# ls -l DST</b>
	<e>ls: cannot access 'DST': No such file or directory</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Tar</h5>
    <p>
      does not seem supported by <i>tar</i>
    </p>


    <h4>User message within backup</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup <e>--user-comment "passphrase is the usual one. Archive was made on %d on host %h"</e> -R / -g usr/bin -K camellia: -zxz -s 100M</b>
	Archive backup requires a password:
	Please confirm your password:


	--------------------------------------------
	2594 inode(s) saved
	including 5 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	34 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2628
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 2154 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# dar -l backup <e>-aheader</e></b>
	Archive version format               : 11
	Compression algorithm used           : xz
	Compression block size used          : 0
	Symmetric key encryption used        : camellia 256
	Asymmetric key encryption used       : none
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : <e>passphrase is the usual one. Archive was made on Thu Nov 12 18:57:35 2020 on host terre</e>
	KDF iteration count                  : 10000
	KDF hash algorithm                   : argon2
	Salt size                            : 32 bytes
	Final memory cleanup...
	FATAL error, aborting operation: header only mode asked
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      The use of the <code>-aheader</code> let one see the archive header that is always in clear-text. The usual listing
      operation provides some additional informations from the ciphered table of content and thus in that context requires the passphrase:
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# dar <e>-l</e> backup <e>-q</e></b>
	<e>Archive backup requires a password:</e>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	Archive version format               : 11
	Compression algorithm used           : xz
	Compression block size used          : 0
	Symmetric key encryption used        : camellia 256
	Asymmetric key encryption used       : none
	Archive is signed                    : no
	Sequential reading marks             : present
	User comment                         : <e>passphrase is the usual one. Archive was made on Thu Nov 12 18:57:35 2020 on host terre</e>
	KDF iteration count                  : 10000
	KDF hash algorithm                   : argon2
	Salt size                            : 32 bytes
	Catalogue size in archive            : 78268 bytes

	Archive is composed of 2 file(s)
	File size             : 104857600 bytes
	Last file size        : 17168696 bytes
	Archive total size is : 122026296 bytes
	<e>The global data compression ratio is:   72%</e>

	<e>CATALOGUE CONTENTS :</e>

	total number of inode : 2589
	fully saved           : 2589
	binay delta patch     : 0
	inode metadata only   : 0
	distribution of inode(s)
	- directories        : 2
	- plain files        : 2152
	- symbolic links     : 435
	- named pipes        : 0
	- unix sockets       : 0
	- character devices  : 0
	- block devices      : 0
	- Door entries       : 0
	hard links information
	- number of inode with hard link           : 5
	- number of reference to hard linked inodes: 10
	destroyed entries information
	0 file(s) have been record as destroyed since backup of reference

	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <p>
      not supported by <i>rsync</i>
    </p>

    <h5>Tar</h5>
    <p>
      not supported by <i>tar</i>
    </p>

    <h4>backup sanity test</h4>


    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin -zlz4</b>


	--------------------------------------------
	2594 inode(s) saved
	including 5 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	34 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2628
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 2154 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# dar -t backup</b>


	--------------------------------------------
	2594 item(s) treated
	0 item(s) with error
	0 item(s) ignored (excluded by filters)
	--------------------------------------------
	Total number of items considered: 2594
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <p>
      It does not seems possible to let <i>rsync</i> check that the target or destination
      directory is sane and usuable. All operation modify the destination file or save
      modified files in either the destination directory (the backup) or an alternate directory
      (<code>--compare-dest</code> option).
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rm -rf backup.tar.gz</b>
	<b>terre:/mnt/memdisk# tar -czf backup.tar.gz /usr/bin</b>
	tar: Removing leading `/' from member names
	tar: Removing leading `/' from hard link targets
	<b>terre:/mnt/memdisk# tar -tzf backup.tar.gz</b>
	usr/bin/
	usr/bin/bitmap
	usr/bin/dot
	usr/bin/indi_usbdewpoint
	usr/bin/ruby2.5
	usr/bin/pod2man
	usr/bin/iptables-xml
	usr/bin/knotify4
	usr/bin/fakeroot
	usr/bin/xclock
	<e>[...]</e>
	/bin/traceproto
	usr/bin/ofm2opl
	usr/bin/akonadi_archivemail_agent
	usr/bin/resizecons
	usr/bin/rletopnm
	usr/bin/dh_install
	usr/bin/updvitomp
	usr/bin/h2xs
	usr/bin/xmessage
	<b>terre:/mnt/memdisk# echo $?</b>
	0
	<b>terre:/mnt/memdisk#</b>
    </code>


    <h4>Comparing with original data</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R SRC -q</b>
	<b>terre:/mnt/memdisk# dar -d backup -R SRC</b>


	--------------------------------------------
	2594 item(s) treated
	0 item(s) do not match those on filesystem
	0 item(s) ignored (excluded by filters)
	--------------------------------------------
	Total number of items considered: 2594
	--------------------------------------------
	<b>terre:/mnt/memdisk# echo $?</b>
	0
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Rsync</h5>
    <p>
      Does not seems supported by <i>rsync</i>
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk/SRC# tar -czf ../backup.tar.gz .</b>
	<b>terre:/mnt/memdisk/SRC# tar -dzf ../backup.tar.gz</b>
	<b>terre:/mnt/memdisk/SRC# echo $?</b>
	0
	<b>terre:/mnt/memdisk/SRC#</b>
    </code>

    <h4>Tunable verbosity</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin <e>-q</e></b>
	<b>terre:/mnt/memdisk# rm backup.1.dar</b>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin</b>


	--------------------------------------------
	2594 inode(s) saved
	including 5 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	34 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2628
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 2154 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin <e>-vm</e></b>
	Arguments read from /usr/local/etc/darrc :

	Creating low layer: Writing archive into a plain file object...
	Adding a new layer on top: Caching layer for better performances...
	Writing down the archive header...
	Adding a new layer on top: Escape layer to allow sequential reading...
	All layers have been created successfully
	Building the catalog object...
	Processing files for backup...
	Writing down archive contents...
	Closing the escape layer...
	Writing down the first archive terminator...
	Writing down archive trailer...
	Writing down the second archive terminator...
	Closing archive low layer...
	Archive is closed.


	--------------------------------------------
	2594 inode(s) saved
	including 5 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	34 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2628
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 2154 inode(s)
	--------------------------------------------
	Making room in memory (releasing memory used by archive of reference)...
	Final memory cleanup...
	<b>terre:/mnt/memdisk# rm -f backup*</b>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin <e>-vt</e> -q</b>
	Adding folder to archive: /usr
	Saving Filesystem Specific Attributes for /usr
	Adding folder to archive: /usr/bin
	Saving Filesystem Specific Attributes for /usr/bin
	Adding file to archive: /usr/bin/bitmap
	Saving Filesystem Specific Attributes for /usr/bin/bitmap
	<e>[...]</e>
	Saving Filesystem Specific Attributes for /usr/bin/dh_install
	Adding symlink to archive: /usr/bin/updvitomp
	Adding file to archive: /usr/bin/h2xs
	Saving Filesystem Specific Attributes for /usr/bin/h2xs
	Adding file to archive: /usr/bin/xmessage
	Saving Filesystem Specific Attributes for /usr/bin/xmessage
	<b>terre:/mnt/memdisk# rm -f backup*</b>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin -vd</b>
	Inspecting directory /root
	Inspecting directory /bin
	Inspecting directory /sbin
	Inspecting directory /tmp
	Inspecting directory /sys
	Inspecting directory /lib
	<e>[...]</e>
	Inspecting directory /var
	Inspecting directory /proc
	Inspecting directory /dev
	Inspecting directory /etc
	Inspecting directory /media
	Inspecting directory /run
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# rm -f backup.*</b>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin <e>-vf</e> -q</b>
	Finished Inspecting directory /usr/bin , saved 408 Mio, compression ratio   13%
	Finished Inspecting directory /usr , saved 408 Mio, compression ratio   13%
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin <e>-vmasks</e> -q</b>
	directory tree filter:
	AND
	| OR
	|   | Is subdir of: /usr/bin [case sensitive]
	|   +--
	+--

	filename filter:
	AND
	| TRUE
	+--

	EA filter:
	AND
	| TRUE
	+--

	Compression filter:
	TRUE

	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      <i>dar</i> has several options to define which type of message to show or not to show:
      <code> -v, -vs, -vt, -vd, -vf, -vm, -vmasks, -q</code>. They can be combined.
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rsync -arHAX /usr/bin DST</b>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# rsync -arHAX <e>-v</e> /usr/bin DST</b>
	sending incremental file list
	created directory DST
	bin/
	bin/2to3-2.7
	bin/411toppm
	bin/7z
	bin/7za
	bin/7zr
	bin/FvwmCommand
	<e>[...]</e>
	bin/zstdmt -> zstd
	bin/perl => bin/perl5.28.1
	bin/perlbug => bin/perlthanks
	bin/python3.7 => bin/python3.7m
	bin/pkg-config => bin/x86_64-pc-linux-gnu-pkg-config
	bin/unzip => bin/zipinfo

	sent 437,298,617 bytes  received 42,381 bytes  174,936,399.20 bytes/sec
	total size is 445,394,557  speedup is 1.02
	<b>root@terre:/mnt/memdisk# rsync -arHAX <e>--info=progress2</e> /usr/bin DST</b>
	437,083,500  98%  128.42MB/s    0:00:03 (xfr#2152, to-chk=0/2593)
	<b>root@terre:/mnt/memdisk#</b>

    </code>
    <p>
      <code>-v option</code> leads to a more verbose output, while <code>-q</code>
      remove the non error messages. Using both at the same time seems not to
      be different than using <code>-q</code> alone. However <i>rsync</i> has a very rich
      set of additional options like <code>--info</code>, <code>--debug</code>
      that can be added on top.
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# tar -czf backup.tar.gz /usr/bin</b>
	tar: Removing leading `/' from member names
	tar: Removing leading `/' from hard link targets
	<b>terre:/mnt/memdisk# rm backup.tar.gz</b>
	<b>terre:/mnt/memdisk# tar <e>-v</e> -czf backup.tar.gz /usr/bin</b>
	tar: Removing leading `/' from member names
	/usr/bin/
	/usr/bin/bitmap
	/usr/bin/dot
	/usr/bin/indi_usbdewpoint
	/usr/bin/ruby2.5
	/usr/bin/pod2man
	/usr/bin/iptables-xml
	/usr/bin/knotify4
	<e>[...]</e>
	/usr/bin/traceproto
	/usr/bin/ofm2opl
	/usr/bin/akonadi_archivemail_agent
	/usr/bin/resizecons
	/usr/bin/rletopnm
	/usr/bin/dh_install
	/usr/bin/updvitomp
	/usr/bin/h2xs
	/usr/bin/xmessage
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      <i>tar</i> only provides the <code>-v</code> option to increase
      verbosity.
    </p>


    <h4>Modify Backup content</h4>
    <p>
      We will perform two types of tests:
      <ul>
	<li>remove one or several files from an existing backup without having to make a new backup process</li>
	<li>add some forgotten files to an existing backup without performing a new full backup process</li>
      </ul>
    </p>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R / -g usr/bin -z6 -q</b>
	<b>terre:/mnt/memdisk# dar -l backup -g usr/bin/emacs-gtk</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[Saved][-]       [-L-][  64%][ ]  drwxr-xr-x   root     root    408 Mio Sun Jun  2 23:25:09 2019        usr
	[Saved][-]       [-L-][  64%][ ]  drwxr-xr-x   root     root    408 Mio Sun Nov  8 13:43:58 2020        usr/bin
	[Saved][ ]       [-L-][  90%][X]  -rwxr-xr-x   root     root    38 Mio  Thu Sep  5 04:35:24 2019        <e>usr/bin/emacs-gtk</e>
	<b>terre:/mnt/memdisk# dar -A backup <e>-+ without-emacs</e> <e class=blue>-ak</e> <e>-P usr/bin/emacs-gtk</e> -vs -q</b>
	Skipping file: &lt;ROOT&gt;/usr/bin/emacs-gtk
	<b>terre:/mnt/memdisk# dar -l without-emacs -g usr/bin/emacs-gtk</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[Saved][-]       [-L-][  62%][ ]  drwxr-xr-x   root     root    370 Mio Sun Jun  2 23:25:09 2019        usr
	[Saved][-]       [-L-][  62%][ ]  drwxr-xr-x   root     root    370 Mio Sun Nov  8 13:43:58 2020        usr/bin
	<b>terre:/mnt/memdisk#rm backup.*</b>
	<b>terre:/mnt/memdisk#mv without-emacs.1.dar backup.1.dar</b>
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      <i>dar</i> does not modify a existing backup but creates a copy of it with the requested files or
      directory removed. The process can be quick even with compression thanks to the <code>-ak</code>
      option that avoid uncompressing and recompressing
      file that are kept. Before removing the old backup you can test the sanity of the new generated one.
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c emacs -R / -g usr/bin/emacs-gtk -z6 -q</b>
	<b>terre:/mnt/memdisk# dar -l emacs</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[Saved][-]       [-L-][  90%][ ]  drwxr-xr-x   root     root    38 Mio  Sun Jun  2 23:25:09 2019        usr
	[Saved][-]       [-L-][  90%][ ]  drwxr-xr-x   root     root    38 Mio  Sun Nov  8 13:43:58 2020        usr/bin
	[Saved][ ]       [-L-][  90%][X]  -rwxr-xr-x   root     root    38 Mio  Thu Sep  5 04:35:24 2019        usr/bin/emacs-gtk
	<b>terre:/mnt/memdisk# <e>dar -A backup -@ emacs -+ with-emacs</e> <e class=blue>-ak</e></b>


	--------------------------------------------
	2594 inode(s) added to archive
	with 10 hard link(s) recorded
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 2159 inode(s)
	--------------------------------------------
	Total number of inode(s) considered: 2594
	--------------------------------------------
	<b>terre:/mnt/memdisk# dar -l with-emacs -g usr/bin/emacs-gtk</b>
	[Data ][D][ EA  ][FSA][Compr][S]| Permission | User  | Group | Size    |          Date                 |    filename
	--------------------------------+------------+-------+-------+---------+-------------------------------+------------
	[Saved][-]       [-L-][  64%][ ]  drwxr-xr-x   root     root    408 Mio Sun Jun  2 23:25:09 2019        usr
	[Saved][-]       [-L-][  64%][ ]  drwxr-xr-x   root     root    408 Mio Sun Nov  8 13:43:58 2020        usr/bin
	[Saved][ ]       [-L-][  90%][X]  -rwxr-xr-x   root     root    38 Mio  Thu Sep  5 04:35:24 2019        usr/bin/emacs-gtk
	<b>terre:/mnt/memdisk# rm emacs.* backup.*</b>
	<b>terre:/mnt/memdisk# mv with-emacs.1.dar backup.1.dar</b>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      Here to add files to a existing backup we must make a small backup of these files only, then merge
      this backup with the backup we want to modify. Nothing of the source data is touched in this operation,
      is something goes wrong or if you made an error, you can fix and restart without taking the risk
      to lose data.
    </p>

    <h5>Rsync</h5>
    <p>
      The backup made by <i>rsync</i> is just a copy of the save files, removing a file from the backup
      is as simple as calling <code>rm</code> on that file in the repository that is considered the backup.
    </p>
    <p>
      While adding a new file in the backup can be done by using <i>rsync</i> as usual including the directory
      tree where this file resides.
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# tar -czf backup.tar.gz /usr/bin</b>
	tar: Removing leading `/' from member names
	tar: Removing leading `/' from hard link targets
	<b>terre:/mnt/memdisk# tar -tvf backup.tar.gz | grep emacs-gtk</b>
	-rwxr-xr-x root/root    39926024 2019-09-05 04:35 usr/bin/emacs-gtk
	<b>terre:/mnt/memdisk# tar -tvf backup.tar.gz | grep emacs-gtk</b>
	-rwxr-xr-x root/root    39926024 2019-09-05 04:35 usr/bin/emacs-gtk
	<b>terre:/mnt/memdisk# tar --delete usr/bin/emacs-gtk -f backup.tar.gz</b>
	<e class=red>tar: Cannot update compressed archives</e>
	tar: Error is not recoverable: exiting now
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      Well, <i>tar</i> cannot manipulate compressed archives. What the point then to remove a
      file from a backup if storage space is not an issue, else, would compression be used?
    </p>

    <h4>stdin/stdout backup read/write</h4>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c - -z6 -R SRC &gt; backup.file</b>


	--------------------------------------------
	2594 inode(s) saved
	including 5 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2594
	--------------------------------------------
	EA saved for 3 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# dar -x - --sequential-read -R DST &lt; backup.file</b>


	--------------------------------------------
	2594 inode(s) restored
	including 5 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 2594
	--------------------------------------------
	EA restored for 3 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      <i>dar</i> can read a backup from stdin and write a backup to stdout.
    </p>

    <h5>Rsync</h5>
    <p>
      Using stdin/stdout to send to or read from backed up data does not seems possible with <i>rsync</i>
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# tar -czf - SRC &gt; backup.file</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xzf - &lt; ../backup.file</b>
	<b>terre:/mnt/memdisk/DST#</b>
    </code>

    <p>
      <i>tar</i> can read a backup from stdin and write a backup to stdout.
    </p>

    <h4>Remote network storage</h4>

    <p>
      For Remote Network storage, if you use a personal NAS you may avoid generating
      ciphered backup, though you still should transfer
      it using a secured protocol if the underlaying network is not your own from end to end
      (for example a part of the path goes over Internet without IPSec or equivalent).
    </p>
    <p>
      Why ciphering backup if using secure transfer protocol?
      <ul>
	<li>
	  Secure transfer avoids one to
	  read your data in transit in particular your credentials to the remote site, but at the
	  end the transferred data is stored in clear at the other end.
	</li>
	<li>
	  ciphered data avoids the owner of the remote storage to access to your data
	  without your consent, though it does not protect one to intercept your transfer and
	  read the credentials you used to connected to the remote storage. This one could
	  then connect later and delete all your backup... which is surely not what you want.
	</li>
      </ul>
    </p>
    <p>
      In the following we will use both: <b>secure protocol</b> and <b>ciphered backup</b>, without using
      local storage. We will also need <b>compression</b> to save precious space (usually you pay for the
      cloud storage you use) and <b>maybe slicing</b> depending on the constraints
      imposed by the remote storage (some provider ask you to pay an extra amount to store larger files,
      having slicing avoids you paying extra cost in such context). Another use case of slicing is when
      the file transfer protocol is not able to continue an interrupted transfer,
      you will then only need to restart it for the last slice, not the whole backup.
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# sftp denis@dar</b>
	The authenticity of host 'dar (192.168.6.32)' can't be established.
	RSA key fingerprint is SHA256:KN3o/psWC512grcZ5/J5dTSg9PzIXbZAHiig/hqfkc8.
	Are you sure you want to continue connecting (yes/no)? yes
	Warning: Permanently added 'dar,192.168.6.32' (RSA) to the list of known hosts.
	denis@dar's password:
	Connected to denis@dar.
	sftp> bye
	<b>terre:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>

    <code class=block>
	<b>terre:/mnt/memdisk# dar -c sftp://denis@dar/home/denis/backup -R / -g etc -K aes: -zlz4 -s 1M</b>
	<e class=blue>Please provide the password for login denis at host dar:</e>
	<e>Archive backup requires a password:</e>
	<e>Please confirm your password:</e>


	--------------------------------------------
	2360 inode(s) saved
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	27 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 2387
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 1523 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# sftp denis@dar</b>
	denis@dar's password:
	Connected to denis@dar.
	sftp> ls -l
	-rw-r--r--    1 denis    denis     1048576 Nov 26 17:15 backup.1.dar
	-rw-r--r--    1 denis    denis     1048576 Nov 26 17:15 backup.2.dar
	-rw-r--r--    1 denis    denis     1048576 Nov 26 17:15 backup.3.dar
	-rw-r--r--    1 denis    denis      474982 Nov 26 17:15 backup.4.dar
	sftp> bye
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      The backup results in four ciphered slices located on the remote sftp server. Let's add
      a <code>-E option</code> to see which slice are being read while testing the archive.
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# dar -t sftp://denis@dar/home/denis/backup <e class=blue>-E "echo 'openning slice %p/%b.%N.%e'"</e></b>
	<e class=blue>Please provide the password for login denis at host dar:</e>
	openning slice /home/denis/backup.4.dar
	<e>Archive backup requires a password:</e>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	openning slice /home/denis/backup.1.dar
	openning slice /home/denis/backup.2.dar
	openning slice /home/denis/backup.3.dar
	openning slice /home/denis/backup.4.dar


	--------------------------------------------
	2360 item(s) treated
	0 item(s) with error
	0 item(s) ignored (excluded by filters)
	--------------------------------------------
	Total number of items considered: 2360
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      We see that all slices have been read as expected, now let's restore /etc/fstab in the current directory
      and compare the restored files with the real /etc/fstab
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -x sftp://denis@dar/home/denis/backup -E "echo 'openning slice %p/%b.%N.%e'" -g etc/fstab --flat</b>
	Please provide the password for login denis at host dar:
	<e>openning slice /home/denis/backup.4.dar</e>
	Archive backup requires a password:
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	<e>1 inode(s) restored</e>
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	269 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 270
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# <e>diff fstab /etc/fstab</e></b>
	<b>terre:/mnt/memdisk# echo $?</b>
	<e>0</e>
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      As seen above only one slice (slice #4) has been necessary to restore /etc/fstab. But let's save two files,
      a huge one and a small one into a single sliced backup and measure the transfer time of backup
      and restoration of this ciphered an compressed backup through sftp. We have added public key authentification
      for precise time measurement:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# sftp denis@dar</b>
	Connected to denis@dar.
	sftp> bye
	<b>terre:/mnt/memdisk# ls -l SRC</b>
	total 315396
	-rw------- 1 root root 322961408 Nov 26 17:28 devuan_beowulf_3.0.0_amd64-netinstall.iso
	-rw-r--r-- 1 root root       994 Nov 26 17:29 fstab
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# time dar -c sftp://denis@dar/home/denis/backup -R SRC -z6 -K aes:hello -afile-auth -q</b>
	20.769u 2.445s <e>0:22.77</e> 101.8%   0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# time dar -x sftp://denis@dar/home/denis/backup -R DST -K hello -afile-auth -q</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	11.826u 4.211s <e>0:15.88</e> 100.9%   0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk# diff -rs SRC DST</b>
	<e>Files SRC/devuan_beowulf_3.0.0_amd64-netinstall.iso and DST/devuan_beowulf_3.0.0_amd64-netinstall.iso are identical</e>
	<e>Files SRC/fstab and DST/fstab are identical</e>
	<b>terre:/mnt/memdisk# <e class=blue>rm DST/fstab</e></b>
	<b>terre:/mnt/memdisk# time dar -x sftp://denis@dar/home/denis/backup -R DST -K hello -afile-auth -q -g fstab</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	0.680u 0.012s <e>0:00.87</e> 79.3%     0+0k 0+0io 0pf+0w
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      While restoring the whole backup needs 15 seconds of transfer time, restoring <code>fstab</code> alone requires only 0.87 second,
      as there is only one slice, this shows that <i>dar</i> is reading only the necessary part of the archive even within a slice to
      perform the operation.
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rsync -arHAXSzq /etc denis@dar:/home/denis</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# rsync -arHAXSzq denis@dar:/home/denis/etc/fstab .</b>
	<b>terre:/mnt/memdisk# diff fstab /etc/fstab</b>
	<b>terre:/mnt/memdisk# echo $?</b>
	0
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      We can backup to a remote sftp server, we can compress the data on-fly but it is not stored compressed nor ciphered.
      Restoration operation is possible per file or for the whole backup
    </p>

    <h5>Tar</h5>
    <p>
      tar has no way to perform sftp or other secured transfer protocol, nor encryption by itself. When time comes to
      restore a particular file, the whole backup has to be retrieved, unciphered and uncompressed to restore even
      just a sigle file</p>

    <h3>Backup <a name="robustness">Robustness</a></h3>

    <p>
      The objective of this test is to measure the way the backup tools under test behave when the backup has been corrupted.
      We will here just flip one byte of the backup, at the beginning, in the middle or at the end of the backup and
      observe the consequences in term of ability to restore the backup.
    </p>

    <h5>Dar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# dar -c backup -R SRC -z6</b>


	--------------------------------------------
	74725 inode(s) saved
	including 0 hard link(s) treated
	0 inode(s) changed at the moment of the backup and could not be saved properly
	0 byte(s) have been wasted in the archive to resave changing files
	0 inode(s) with only metadata changed
	0 inode(s) not saved (no inode/file change)
	0 inode(s) failed to be saved (filesystem error)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) recorded as deleted from reference backup
	--------------------------------------------
	Total number of inode(s) considered: 74725
	--------------------------------------------
	EA saved for 0 inode(s)
	FSA saved for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# ls -al backup*</b>
	-rw-r--r-- 1 root root 219088536 Nov 17 17:45 backup.1.dar
	<b>terre:/mnt/memdisk# ./hide_change backup.1.dar 1</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# dar -x backup -R DST</b>
	<e>backup.1.dar is not a valid file (wrong magic number), please provide the good file. [return = YES | Esc = NO]</e>
	Escaping...
	Final memory cleanup...
	Aborting program. User refused to continue while asking: backup.1.dar is not a valid file (wrong magic number), please provide the good file.
	<b>terre:/mnt/memdisk# dar -x backup -R DST <e>-alax</e></b>
	LAX MODE: In spite of its name, backup.1.dar does not appear to be a dar slice, assuming a data corruption took place and continuing
	LAX MODE: Archive is flagged as having escape sequence (which is normal in recent archive versions). However if this is not expected, shall I assume a data corruption occurred in this field and that this flag should be ignored? (If unsure, refuse) [return = YES | Esc = NO]
	Escaping...


	--------------------------------------------
	<e>74725 inode(s) restored</e>
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	<e>Total number of inode(s) considered: 74725</e>
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk# diff -r SRC DST</b>
	<b>terre:/mnt/memdisk# echo $?</b>
	0
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      Modifying the first bit <i>dar</i> has seen the corruption. We can use the lax mode (<code>-alax</code> option) to bypass this
      corruption and then the restoration proceeds normally. We can try a bit further for example somewhere in the middle of the archive,
      thus at offset 876354144 (half of the size expressed in bit, not byte):
    </p>
    <code class=block>
	<b>terre:/mnt/memdisk# ls -l backup*</b>
	-rw-r--r-- 1 root  root   219088536 Nov 17 17:45 backup.1.dar
	<b>terre:/mnt/memdisk# hide_change backup.1.dar 876354144</b>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# dar -x backup -R DST</b>
	<e>Error while restoring /mnt/memdisk/DST/linux-5.9.8/drivers/mtd/spi-nor/core.c : compressed data CRC error</e>


	--------------------------------------------
	74724 inode(s) restored
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	<e class=red>1 inode(s) failed to restore (filesystem error)</e>
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 74725
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	Final memory cleanup...
	All files asked could not be restored
	<b>terre:/mnt/memdisk# diff -rq SRC DST</b>
	<e>Files SRC/linux-5.9.8/drivers/mtd/spi-nor/core.c and DST/linux-5.9.8/drivers/mtd/spi-nor/core.c differ</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      One file could not be restored properly as reported by <i>dar</i>,
      but all other files could be and are identical to their respective
      originals. Let's modifying the last bit for completness:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# ls -al *.dar</b>
	-rw-r--r-- 1 root root 219088537 Nov 17 17:45 backup.1.dar
	<b>terre:/mnt/memdisk# cp backup.1.dar backop.1.dar</b>
	<b>terre:/mnt/memdisk# hide_change backup.1.dar 1752708287</b>
	<b>terre:/mnt/memdisk# ls -al *.dar</b>
	-rw-r--r-- 1 root root 219088536 Nov 17 17:58 backop.1.dar
	-rw-r--r-- 1 root root 219088536 Nov 17 17:58 backup.1.dar
	<b>terre:/mnt/memdisk# diff backup.1.dar backop.1.dar</b>
	Binary files backup.1.dar and backop.1.dar differ
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# dar -x backup -R DST</b>


	--------------------------------------------
	74725 inode(s) restored
	including 0 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 74725
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>terre:/mnt/memdisk#</b>
	<b>terre:/mnt/memdisk# diff -rq SRC DST</b>
	<b>terre:/mnt/memdisk# echo $?</b>
	0
	<b>terre:/mnt/memdisk#</b>
    </code>
    <p>
      By chance it did not affected the ability to restore the backup. However if it ever had,
      we have several fallbacks: the <code>--sequential-read</code> mode, the use a already created
      snapshot (aka isolated catalogue) as seen about the snapshot feature to backup the internal
      table of content, or as last resort
      the <code>-alax option</code> eventually combined with the <code>--sequential-read</code>
      mode and a backup snapshot.
    </p>

    <h5>Rsync</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# rsync -arHAXS SRC/* DST</b>
	<b>terre:/mnt/memdisk# ls -al DST</b>
	total 0
	drwxr-xr-x  3 root root  60 Nov 17 18:07 .
	drwxrwxrwt  4 root root 140 Nov 17 18:07 ..
	drwxrwxr-x 24 root root 740 Nov 10 21:16 linux-5.9.8
	<b>terre:/mnt/memdisk# diff -rq SRC DST</b>
	<b>terre:/mnt/memdisk# ./hide_change DST/linux-5.9.8/README 10</b>
	<b>terre:/mnt/memdisk# diff -rq SRC DST</b>
	Files SRC/linux-5.9.8/README and DST/linux-5.9.8/README differ
	<b>terre:/mnt/memdisk# rsync -arvHAXS SRC/* DST</b>
	sending incremental file list

	sent 1,254,054 bytes  received 5,215 bytes  839,512.67 bytes/sec
	total size is 954,980,692  speedup is 758.36
	<b>terre:/mnt/memdisk# diff -rq SRC DST</b>
	<e>Files SRC/linux-5.9.8/README and DST/linux-5.9.8/README differ</e>
	<b>terre:/mnt/memdisk#</b>
    </code>

    <p>
      modifying the backup (the directory we sync with), <i>rsync</i> does not
      report any difference and the backup stay corrupted.
    </p>

    <h5>Tar</h5>
    <code class=block>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# tar -czf backup.tar.gz SRC</b>
	<b>terre:/mnt/memdisk# ls -l backup*</b>
	-rw-r--r-- 1 root root 183659664 Nov 17 18:11 backup.tar.gz
	<b>terre:/mnt/memdisk# ./hide_change backup.tar.gz 1</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xzf ../backup.tar.gz</b>

	gzip: stdin: not in gzip format
	tar: Child returned status 1
	<e class=red>tar: Error is not recoverable: exiting now</e>
	<b>terre:/mnt/memdisk/DST#</b>
	<b>terre:/mnt/memdisk/DST# find . -ls</b>
	1720964      0 drwxr-xr-x   2 root     root           40 Nov 17 18:56 .
	<b>terre:/mnt/memdisk/DST#</b>
    </code>

    <p>
      Modifying the first byte leads to a completely unusable backup. Nothing got restored at all.
      Let's see what going on when modifying a single bit in the middle of the backup:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# ls -l backup*</b>
	-rw-r--r-- 1 root root 183659664 Nov 17 18:11 backup.tar.gz
	<b>terre:/mnt/memdisk# ./hide_change backup.tar.gz 734638656</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xf ../backup.tar.gz</b>
	tar: Skipping to next header

	<e class=red>gzip: stdin: invalid compressed data--crc error</e>

	gzip: stdin: invalid compressed data--length error
	tar: Child returned status 1
	<e>tar: Error is not recoverable: exiting now</e>
	<b>terre:/mnt/memdisk/DST#</b>
	<b>terre:/mnt/memdisk/DST# diff -rq ../SRC SRC | wc -l</b>
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/arc: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/arm: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/arm64: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/c6x: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/h8300: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/microblaze: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/mips: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/nios2: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/openrisc: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/powerpc: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/sh: No such file or directory
	diff: SRC/linux-5.9.8/scripts/dtc/include-prefixes/xtensa: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/copyloops/copy_mc_64.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/copyloops/copyuser_64.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/copyloops/copyuser_power7.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/copyloops/memcpy_64.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/copyloops/memcpy_power7.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/nx-gzip/include/vas-api.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/primitives/asm/asm-compat.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/primitives/asm/asm-const.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/primitives/asm/feature-fixups.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/primitives/asm/ppc_asm.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/primitives/word-at-a-time.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/stringloops/memcmp_32.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/stringloops/memcmp_64.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/stringloops/strlen_32.S: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/vphn/asm/lppaca.h: No such file or directory
	diff: SRC/linux-5.9.8/tools/testing/selftests/powerpc/vphn/vphn.c: No such file or directory
	<e>150</e>
	<b>terre:/mnt/memdisk/DST# find ../SRC | wc -l</b>
	<e class=blue>74726</e>
	<b>terre:/mnt/memdisk/DST# find SRC | wc -l</b>
	<e class=red>32615</e>
	<b>terre:/mnt/memdisk/DST#</b>
    </code>
    <p>
      Only 32615 files on the 74726 that were saved could be restored. Assuming the problem is due to
      the fact the backup is compressed, let's see tar without compression:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# tar -cf backup.tar SRC</b>
	<b>terre:/mnt/memdisk# ls -l backup*</b>
	-rw-r--r-- 1 root root 1011312640 Nov 17 19:28 backup.tar
	<b>terre:/mnt/memdisk# ./hide_change backup.tar 1</b>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xf ../backup.tar</b>
	tar: This does not look like a tar archive
	<e>tar: Skipping to next header</e>
	tar: Exiting with failure status due to previous errors
	<b>terre:/mnt/memdisk/DST# diff -rq ../SRC SRC</b>
	<b>terre:/mnt/memdisk/DST# echo $?</b>
	0
	<b>terre:/mnt/memdisk/DST#</b>
    </code>
    <p>
      Without compression, a <i>tar</i> backup is much more reliable, however we now need more than 5 times
      storage space to hold the backup. Let's see what happens when we modify a single bit in the midle of the backup:
    </p>

    <code class=block>
	<b>terre:/mnt/memdisk# ./hide_change backup.tar 4045250560</b>
	<b>terre:/mnt/memdisk# rm -rf DST</b>
	<b>terre:/mnt/memdisk# mkdir DST</b>
	<b>terre:/mnt/memdisk# cd DST</b>
	<b>terre:/mnt/memdisk/DST# tar -xf ../backup.tar</b>
	<b>terre:/mnt/memdisk/DST#</b>
	<b>terre:/mnt/memdisk/DST# diff -rq ../SRC SRC</b>
	<e class=red>Files ../SRC/linux-5.9.8/drivers/media/pci/bt8xx/bttv-cards.c and SRC/linux-5.9.8/drivers/media/pci/bt8xx/bttv-cards.c differ</e>
	<b>terre:/mnt/memdisk/DST#</b>
    </code>

    <p>
      the backup restoration suceeded according to <i>tar</i> but the corruption has been completely ignored!!!
      The result is both a corrupted backup and a corrupted restored data, with no notification at all...
    </p>

    <h5>Parchive</h5>

    <p>
      We can increase the robustness of any file or set of files by mean of <i>Parchive</i> software. If its use
      is adapted to <i>tar</i> and <i>dar</i> it is not adapted to <i>rsync</i> due to the directory tree structure it uses for its backup.
      We will thus here measure the par2create (<i>Parchive</i>) execution time compared to backup time of <i>tar</i> and <i>dar</i>.
    </p>

    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir SRC</b>
	<b>devuan:/mnt/memdisk# cp --preserve -r /usr SRC</b>
	<b>devuan:/mnt/memdisk# time <e class=blue>tar</e> -czf backup.tar.gz SRC</b>
	62.550u 3.148s <e>1:01.75</e> 106.3%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time <e class=blue>dar</e> -c backup -z6 -1 0 -at -R SRC -q</b>
	60.287u 1.152s <e>1:01.45</e> 99.9%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# ls -l backup.*</b>
	-rw-r--r-- 1 root root <e>601976990</e> Dec  1 10:48 backup.1.dar
	-rw-r--r-- 1 root root <e>588260243</e> Dec  1 10:47 backup.tar.gz
	<b>devuan:/mnt/memdisk# time <e class=blue>par2create</e> -r5 -n1 -q backup.tar.gz</b>

	Opening: backup.tar.gz
	Done
	94.465u 0.535s <e>0:05.74</e> 1654.8%  0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time <e class=blue>par2create</e> -r5 -n1 -q backup.1.dar</b>

	Opening: backup.1.dar
	Done
	110.048u 0.364s <e>0:06.19</e> 1783.5% 0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      We see that the redundancy process is not negligible: for 5% of data redundancy,
      we need around 10% of extra CPU time. Things get worse when the size of the
      data to process by <i>Parchive</i> increases and when the disk I/O comes to play
      (here the /mnt/memdisk was an in-memory tmpfs filesystem). This is the
      case when the amount of data to backup is larger than the available RAM space, which
      is a quite frequent situation:
    </p>

    <code class=block>
	<b>devuan:~/tmp# du -sh SRC</b>
	<e>27G</e>     SRC
	<b>devuan:~/tmp# free -m</b>
	.             total        used        free      shared  buff/cache   available
	Mem:          <e>15776</e>         560         433           1       14782       14823
	Swap:             0           0           0
	<b>devuan:~/tmp#</b>
    </code>
    <p>
      We are now placed in this context, having a 27 GiB data to backup on a machine having only around 16 GiB of
      RAM.
    </p>

    <code class=block>
	<b>devuan:~/tmp# time dar -c backup -z6 -at -1 0 -R SRC -q <e class=blue>-E 'par2create -r5 -n1 -q %b.%N.%e'</e></b>

	Opening: backup.1.dar
	Done
	15237.777u 105.379s <e>36:48.53</e> 694.7%     0+0k 158854544+57572120io 158pf+0w
	<b>devuan:~/tmp#</b>
    </code>
    <p>
      This execution time of around 36 mn above can be improved by using multiple slices. Choosing a slice size smaller than the available RAM
      let <i>Parchive</i> compute parity data right after each slice has been generated, while it is still in the disk cache (RAM),
      bypassing the corresponding disks I/O we had previously in a second time:
    </p>
    <code class=block>
	<b>devuan:~/tmp# time dar -c backup_splitted -z6 -at -1 0 -R SRC -q <e class=blue>-E 'par2create -r5 -n1 -q %b.%N.%e'</e> <e>-s 1G</e></b>

	Opening: backup_splitted.1.dar
	Done

	Opening: backup_splitted.2.dar
	Done
	<i>[...]</i>
	Opening: backup_splitted.26.dar
	Done

	Opening: backup_splitted.27.dar
	Done
	6040.106u 49.201s <e>21:58.73</e> 461.7%       0+0k 61862640+57567104io 123pf+0w
	<b>devuan:~/tmp#</b>
    </code>
    <p>
      The total execution time dropped to around 22 mn! Which makes a 40% time reduction.
      By the way, having here 27 files of 1 GiB is also easier to manipulate (file transfer,
      copy to removable media,...) than a huge equivalent file of 27 GiB.
    </p>
    <code class=block>
	<b>devuan:~/tmp# time tar -czf backup.tar.gz SRC</b>
	<e class=blue>837.662u</e> 80.776s <e>18:26.08</e> 83.0% 0+0k 54926128+54799696io 6pf+0w
	<b>devuan:~/tmp# time par2create -r5 -n1 -q backup.tar.gz</b>
	Opening: backup.tar.gz
	Done
	<e class=blue>13352.393u</e> 71.390s <e>18:24.34</e> 1215.5%     0+0k 95000064+2772144io 9pf+0w
	<b>devuan:~/tmp#</b>

    </code>
    <p>
      We get the same execution time (18 mn) for both <i>tar</i> and <i>par2</i> for thus a total of 36 mn.
      This same time for both software while the real CPU usage is much more important for <i>par2</i>,
      clearly shows that the slowest operation was the disk I/O. Else the overall time of the operation
      is similar to what we say with <i>dar</i> above, except that we cannot use multi-volume
      to speed up the operation as we did with <i>dar</i>: <i>tar</i> is not able to compress
      *and* produce multi-volume backup: What we would gain on one side would be lost on the
      other side...
    </p>


    <h3>Execution <a name="performance">Performance</a></h3>

    </p>
    <p>
      In order to compare performance in a fair manner, we have to take into
      consideration that some CPU intensive features are not implement by all softwares or have
      different default values:
    </p>
    <ul>
      <li>
	The default compression level differs: <i>dar</i> uses level 9 by default while <i>rsync</i>
	and <i>tar</i> use level 6 which is faster. Only <i>dar</i> and <i>rsync</i> seems able
	let the user set this value, so we will have to manually set <i>dar</i> to use level 6 too:
	<code>-z6 option</code>
      </li>
      <li>
	the disk usage optimization is not supported by <i>tar</i> so we will not activate
	sparse file detection and optimization (for <i>rsync</i> no <code>-S option</code>)
	and disable it for <i>dar</i>
	(activated by default): <code>-1 0 option</code>
      </li>
      <li>
	<i>dar</i> spends non negligible CPU cycle to duplicate metadata along the backup, this
	is one of the root of exclusive robusness brought by <i>dar</i> backups, but this is an
	optional feature. We will disable it using the
	<code>-at option</code>
      </li>
      <li>
	<i>rsync</i> and <i>dar</i> calculate checksum on the saved data, while tar completely skips
	this data protection. Unfortunately this is not possible to disable for both,
	thus <i>tar</i> will have a speed advantage thanks to this difference.
      </li>
    </ul>
    </p>
    For the data reduction, we will first compare with the same feature set, then activate all specific
    features each software can leverage to improve data reduction, and will measure the execution
    time impact.
    </p>

    <p>
      All performance aspects are not interesting in all <b>use cases</b>. We can distinguish two main
      types of use cases:
    </p>
    <ul>
      <li>
	A first set of tests will cover the <b>copy operations</b>, they will not use any compression, focusing
	only on execution time.
      </li>
      <li>
	A second set of tests will cover the <b>backup operations</b>, use compression and focus on the data
	reduction mainly but also on the execution time.
      </li>
    </ul>

    <p>
      When using <i>rsync</i> as a backup tool (at the opposite of a copy operation), we assume the remote
      (or local) copy <i>is</i> the backup, and thus restoring implies syncing back this remote (or local)
      copy to the place the original data was located and has been lost.
    </p>

    <p>
      To prepare the data under test we used:
    </p>
    <ul>
      <li>a big ISO file</li>
      <li>a full linux system</li>
    </ul>

    <p>
      All data to backup or to copy has been stored in a tmpfs, which is also the destination of the created backups and
      restored data. The swap has been disabled to avoid any disk I/O penalty, in the intention to provide a fair
      comparison environment (avoiding disk cache variable performance).
    </p>

    <code class=block>
	<b>devuan:/mnt/memdisk# free</b>
	total        used        free      shared  buff/cache   available
	Mem:       16155172      691764      151608     8780152    15311800     6299316
	<e>Swap:             0           0           0</e>
	<b>devuan:/mnt/memdisk# df -h .</b>
	Filesystem      Size  Used Avail Use% Mounted on
	<e>tmpfs</e>            14G  4.0K   14G   1% /mnt/memdisk
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      To prepare the Linux system under backup we installed the <i>Devuan</i> system a few days before in a VM. On day D,
      a full backup has been executed, we updated/upgraded the system using the disto package manager and we made a differential
      backup based on the first one, both backup being wrote to the testing machine (bare-metal server) that was used for the performance tests:
    </p>

    <code class=block>
	<b>root@Georges:~# dar -c sftp://denis@10.13.30.163/home/denis/tmp/full -zlz4 -R / -M -D --hash sha1 -afile-auth -C cat_full</b>
	<b>root@Georges:~# apt-get update</b>
	<i>[...]</i>
	<b>root@Georges:~# apt-get upgrade</b>
	<i>[...]</i>
	<b>root@Georges:~# dar -c sftp://denis@10.13.30.163/home/denis/tmp/diff -A cat_full -zlz4 -R / -M -D --hash sha1 -afile-auth</b>
	<b>root@Georges:~#</b>
    </code>

    <p>
      Back on the testing host (the bare-metal server at 10.13.30.163) we restored the data
      for the performance test, the following way: excluding FSA and EA to avoid a tone of warning as those are not
      supported on tmpfs filesystem:
    </p>

    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir state-1</b>
	<b>devuan:/mnt/memdisk# mkdir state-2</b>
	<b>devuan:/mnt/memdisk# dar -x ~denis/tmp/full -R state-1 --fsa-scope none -u "*"</b>


	--------------------------------------------
	136836 inode(s) restored
	including 27 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 136836
	--------------------------------------------
	EA restored for 3 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>devuan:/mnt/memdisk# dar -x ~denis/tmp/full -R state-2 --fsa-scope none -u "*"</b>


	--------------------------------------------
	136836 inode(s) restored
	including 27 hard link(s)
	0 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 136836
	--------------------------------------------
	EA restored for 3 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>devuan:/mnt/memdisk# dar -x ~denis/tmp/diff -R state-2 --fsa-scope none -u "*" -w</b>


	--------------------------------------------
	568 inode(s) restored
	including 0 hard link(s)
	136670 inode(s) not restored (not saved in archive)
	0 inode(s) not restored (overwriting policy decision)
	0 inode(s) ignored (excluded by filters)
	0 inode(s) failed to restore (filesystem error)
	0 inode(s) deleted
	--------------------------------------------
	Total number of inode(s) considered: 137238
	--------------------------------------------
	EA restored for 0 inode(s)
	FSA restored for 0 inode(s)
	--------------------------------------------
	<b>devuan:/mnt/memdisk#</b>

    </code>
    <p>
      This leads us to have two directories <code>state-1</code> and <code>state-2</code> corresponding to the state of the <i>Devuan</i> machine has two days ago and today respectively.
    </p>


    <h4>Performance of <a name="copy_perf">copy</a> operations</h4>

    <p>
      To perform the copy operation, we have decomposed the operations to precisely measure the execution time. We could have decided to pipe the backup to a second instance of the
      backup tool restoring the data (<i>tar</i> and <i>dar</i> only would benefit from this). But the time measurement was less easy to obtain and doing that way does not seem to
      provide any noticable speed improvement. The data used here in <code>SRC1</code> is a single big ISO file (a <i>Devuan</i> installation DVD image).
    </p>

    <h5>Dar</h5>
    <code class=block>
	<b>devuan:/mnt/memdisk# time dar -c copy -1 0 -at -R SRC1 -q</b>
	1.834u 2.909s <e>0:04.87</e> 97.1%     0+0k 744+0io 12pf+0w
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time dar -x copy -R DST -q</b>
	1.563u 2.836s <e>0:04.41</e> 99.5%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# du -sh SRC1 DST</b>
	4.4G    SRC1
	4.4G    DST
	<b>devuan:/mnt/memdisk# diff -r SRC1 DST</b>
	<b>devuan:/mnt/memdisk# echo $?</b>
	0
	<b>devuan:/mnt/memdisk# rm -rf DST copy.1.dar</b>
	<b>devuan:/mnt/memdisk# time dar -c copy -1 0 -at -R SRC2 -q</b>
	4.739u 3.683s <e>0:08.44</e> 99.6%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time dar -x copy -R DST -q</b>
	4.449u 3.872s <e>0:08.34</e> 99.6%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# du -sh SRC2 DST</b>
	4.1G    SRC2
	4.1G    DST
	<b>devuan:/mnt/memdisk# find DST | wc -l</b>
	136837
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      The overall copy time for <i>dar</i> is:
    </p>
    <ul>
      <li>9.18 seconds for the big file in <code>SRC1</code></li>
      <li>16.78 seconds for the full linux system in <code>SRC2</code></li>
    </ul>

    <h5>Rsync</h5>
    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time rsync -arH SRC1 DST</b>
	23.088u 5.420s <e>0:15.28</e> 186.5%   0+0k 168+0io 4pf+0w
	<b>devuan:/mnt/memdisk# diff -r SRC1/ DST/SRC1/</b>
	<b>devuan:/mnt/memdisk# echo $?</b>
	0
	<b>devuan:/mnt/memdisk# rm -rf DST</b>
	<b>devuan:/mnt/memdisk# time rsync -arH SRC2 DST</b>
	22.408u 8.560s <e>0:16.59</e> 186.6%   0+0k 1224+0io 6pf+0w
	<b>devuan:/mnt/memdisk# du -sh SRC2 DST</b>
	4.1G    SRC2
	4.1G    DST
	<b>devuan:/mnt/memdisk# find DST | wc -l</b>
	136838
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      The overall copy time for <i>rsync</i> is:
    </p>
    <ul>
      <li>15.28 seconds for the big file in <code>SRC1</code></li>
      <li>16.59 seconds for the full linux system in <code>SRC2</code></li>
    </ul>


    <h5>Tar</h5>
    <code class=block>
	<b>devuan:/mnt/memdisk# cd SRC1</b>
	<b>devuan:/mnt/memdisk/SRC1# time tar -cf ../copy.tar *</b>
	0.343u 2.756s <e>0:03.10</e> 99.6%     0+0k 104+0io 1pf+0w
	<b>devuan:/mnt/memdisk/SRC1# cd ../</b>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time tar -xf ../copy.tar</b>
	0.339u 3.071s <e>0:03.41</e> 99.7%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# cd ..</b>
	<b>devuan:/mnt/memdisk# diff -r SRC1/ DST/</b>
	<b>devuan:/mnt/memdisk# echo $?</b>
	0
	<b>devuan:/mnt/memdisk# rm -rf DST copy.tar</b>
	<b>devuan:/mnt/memdisk# cd SRC2</b>
	<b>devuan:/mnt/memdisk/SRC2# time tar -cf ../copy.tar *</b>
	<e class=red>tar: tmp/.ICE-unix/19789: socket ignored</e>
	<e class=red>tar: tmp/.X11-unix/X0: socket ignored</e>
	0.760u 2.887s <e>0:03.66</e> 99.4%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/SRC2# cd ..</b>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time tar -xf ../copy.tar</b>
	0.814u 3.556s <e>0:04.38</e> 99.5%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# cd ..</b>
	<b>devuan:/mnt/memdisk# du -sh SRC2 DST</b>
	4.1G    SRC2
	4.1G    DST
	<b>devuan:/mnt/memdisk# find DST | wc -l</b>
	136834
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      the overall copy time for <i>tar</i> is:
    </p>
    <ul>
      <li>6.51 seconds for the big file in <code>SRC1</code></li>
      <li>8.04 seconds for the full Linux system in <code>SRC2</code></li>
    </ul>

    <h5>Cp</h5>
    <code class=block>
	<b>devuan:/mnt/memdisk# time cp --preserve -r SRC1 DST</b>
	0.051u 2.514s <e>0:02.58</e> 99.2%     0+0k 8+0io 1pf+0w
	<b>devuan:/mnt/memdisk# diff -r SRC1 DST</b>
	<b>devuan:/mnt/memdisk# echo $?</b>
	0
	<b>devuan:/mnt/memdisk# rm -rf DST</b>
	<b>devuan:/mnt/memdisk# time cp --preserve -r SRC2 DST</b>
	0.910u 4.194s <e>0:05.15</e> 99.0%     0+0k 288+0io 1pf+0w
	<b>devuan:/mnt/memdisk# du -sh SRC2 DST</b>
	<e>4.1G</e>    SRC2
	<e class=red>4.2G</e>    DST
	<b>devuan:/mnt/memdisk# find DST | wc -l</b>
	136838
	<b>devuan:/mnt/memdisk# find DST/SRC2/tmp/ -ls</b>
	2315983      0 drwxrwxrwt   4 root     root          100 Dec  3 10:32 DST/SRC2/tmp/
	2315987      0 drwxrwxrwt   2 root     root           60 Dec  3 10:27 DST/SRC2/tmp/.ICE-unix
	2315988      0 srwxrwxrwx   1 denis    denis           0 Dec  3 10:27 DST/SRC2/tmp/.ICE-unix/19789
	2315985      0 drwxrwxrwt   2 root     root           60 Dec  3 10:32 DST/SRC2/tmp/.X11-unix
	2315986      0 srwxrwxrwx   1 root     root            0 Dec  3 10:32 DST/SRC2/tmp/.X11-unix/X0
	2315984      4 -r--r--r--   1 root     root           11 Dec  3 10:32 DST/SRC2/tmp/.X0-lock
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      The overall copy time for <i>cp</i> is:
    </p>
    <ul>
      <li>2.58 seconds for the big file in <code>SRC1</code></li>
      <li>5.15 seconds for the full Linux system in <code>SRC2</code></li>
    </ul>
    <p>
      <i>cp</i> is always the fastest and does reject the unix sockets as <i>tar</i> does.
      However it requires slightly more storage than all other softwares tested here. And if
      metadata (ACL, Extended Attributes, filesystem specific attributes, ...) need to be copied
      with data, it does not match the need.
    </p>

    <h4>Performance of <a name="backup_perf">Backup</a> operations</h4>

    <p>
      For reference:
    </p>

    <code class=block>
	<b>devuan:/mnt/memdisk# du -sb state-*</b>
	<e>4095931349</e>      state-1
	<e>4136318367</e>      state-2
	<b>devuan:/mnt/memdisk# find state-1 | wc -l</b>
	136837
	<b>devuan:/mnt/memdisk# find state-2 | wc -l</b>
	137239
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <h5>Dar</h5>
    <h6>Minimal features</h6>
    <code class=block>
	<b>devuan:/mnt/memdisk# time dar -c dar-full -R state-1 -at -1 0 -z6 -q</b>
	145.970u 3.263s <e>2:29.73</e> 99.6%   0+0k 12344+0io 71pf+0w
	<b>devuan:/mnt/memdisk# time dar -c dar-diff -R state-2 -A dar-full -at -1 0 -z6 -q -asecu</b>
	8.957u 0.959s <e>0:09.93</e> 99.6%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root   <e>49498524</e> Dec  3 16:17 dar-diff.1.dar
	-rw-r--r-- 1 root root <e>1580562224</e> Dec  3 16:16 dar-full.1.dar
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time dar -x dar-full -R DST -q</b>
	18.677u 4.244s <e>0:22.94</e> 99.8%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -x dar-diff -R DST -q -w</b>
	2.585u 0.780s <e>0:03.48</e> 96.5%     0+0k 1856+0io 20pf+0w
	<b>devuan:/mnt/memdisk#</b>
	<b>devuan:/mnt/memdisk# time dar -x dar-full -R DST -q -w -g etc/fstab</b>
	0.934u 0.036s <e>0:00.98</e> 97.9%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      for <i>dar</i> with minimal features (metadata no redundancy <code>-at</code>, no sparse file consideration <code>-1 0</code>):
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 61.41%</li>
      <li>data reduction on storage for the diff backup is: 98.80%</li>
      <li>data reduction over the network is the same as on storage</li>
      <li>execution time to restore a single file is: 0.98 s</li>
      <li>execution time to restore the full backup: 22.94 s</li>
      <li>execution time ot restore the diff backup: 3.48 s </li>
      <li>full backup time: 149.73 s</li>
      <li>diff backup time: 9.93 s</li>
    </ul>

    <h6>sparse file</h6>

    <code class=block>
	<b>devuan:/mnt/memdisk# rm -rf DST *.dar</b>
	<b>devuan:/mnt/memdisk# time dar -c dar-full -R state-1 -at -z6 -q</b>
	154.971u 3.000s <e>2:37.99</e> 99.9%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -c dar-diff -R state-2 -A dar-full -at -z6 -q -asecu</b>
	9.488u 0.871s <e>0:10.37</e> 99.8%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root   <e>49505251</e> Dec  3 16:27 dar-diff.1.dar
	-rw-r--r-- 1 root root <e>1578428790</e> Dec  3 16:25 dar-full.1.dar
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time dar -x dar-full -R DST -q</b>
	24.231u 6.110s <e>0:30.36</e> 99.9%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -x dar-diff -R DST -q -w</b>
	2.677u 0.793s <e>0:03.48</e> 99.4%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -x dar-full -R DST -q -w -g etc/fstab</b>
	1.067u 0.053s <e>0:01.13</e> 98.2%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      for <i>dar</i> with default features (metadata no redundancy <code>-at</code>, with sparse file consideration (activated by default))
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 61.46%</li>
      <li>data reduction on storage for the diff backup is: 98.80%</li>
      <li>data reduction over the network is the same as on storage</li>
      <li>execution time to restore a single file is: 1.13 s</li>
      <li>execution time to restore the full backup: 30.36 s</li>
      <li>execution time ot restore the diff backup: 3.48 s </li>
      <li>full backup time: 157.99 s</li>
      <li>diff backup time: 10.37 s</li>
    </ul>


    <h6>Sparse file and binary delta</h6>
    <code class=block>
	<b>devuan:/mnt/memdisk# rm -rf DST *.dar</b>
	<b>devuan:/mnt/memdisk# time dar -c dar-full -R state-1 -at -z6 --delta sig -q</b>
	159.262u 3.332s <e>2:42.62</e> 99.9%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -c dar-diff -R state-2 -A dar-full -at -z6 -q -asecu</b>
	6.149u 0.950s <e>0:07.11</e> 99.7%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# ls -l *.dar</b>
	-rw-r--r-- 1 root root   <e>23883368</e> Dec  3 16:39 dar-diff.1.dar
	-rw-r--r-- 1 root root <e>1602481058</e> Dec  3 16:38 dar-full.1.dar
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time dar -x dar-full -R DST -q</b>
	24.169u 6.163s <e>0:30.35</e> 99.9%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -x dar-diff -R DST -q -w</b>
	2.481u 0.942s <e>0:03.44</e> 99.4%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time dar -x dar-full -R DST -q -w -g etc/fstab</b>
	1.205u 0.059s <e>0:01.27</e> 98.4%     0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      for <i>dar</i> with advanced features (metadata no redundancy <code>-at</code>, with binary delta computation <code>--delta sig</code>):
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 60,87%</li>
      <li>data reduction on storage for the diff backup is: 99.42%</li>
      <li>data reduction over the network is the same as on storage</li>
      <li>execution time to restore a single file is: 1.27 s</li>
      <li>execution time to restore the full backup: 30.35 s</li>
      <li>execution time ot restore the diff backup: 1.27 s </li>
      <li>full backup time: 162.62 s</li>
      <li>diff backup time: 7.11 s</li>
    </ul>

    <h5>Rsync</h5>
    <h6>minimal features</h6>

    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir rsync-backup</b>
	<b>devuan:/mnt/memdisk# time rsync -arHz --info=stats state-1/* rsync-backup</b>

	sent <e>1,585,540,014 bytes  received 2,174,472 bytes</e>  10,080,726.90 bytes/sec
	total size is 4,260,538,564  speedup is 2.68
	202.640u 8.503s <e>2:36.98</e> 134.5%  0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
	<b>devuan:/mnt/memdisk# time rsync -arHz --info=stats <e class=blue>--no-whole-file</e> state-2/* rsync-backup</b>

	<e>sent 29,077,377 bytes  received 216,581 bytes</e>  3,446,348.00 bytes/sec
	total size is 4,300,916,222  speedup is 146.82
	7.555u 1.115s <e>0:07.33</e> 118.1%    0+0k 1784+0io 7pf+0w
	<b>devuan:/mnt/memdisk# du -sb rsync-backup</b>
	<e>4136318307</e>      rsync-backup
	<b>devuan:/mnt/memdisk#</b>
	<b>devuan:/mnt/memdisk# rm -rf state-1</b>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time rsync -arHz --info=stats <e class=blue>--no-whole-file</e> rsync-backup/* DST</b>

	<e>sent 1,599,585,756 bytes  received 2,181,147 bytes</e>  10,105,784.88 bytes/sec
	total size is 4,300,916,222  speedup is 2.69
	204.192u 8.306s <e>2:37.81</e> 134.6%  0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time rsync -arHz --info=stats rsync-backup/etc/fstab DST/etc</b>

	sent 44 bytes  received 12 bytes  112.00 bytes/sec
	total size is 664  speedup is 11.86
	<e>0.001u 0.002s</e> 0:00.00 0.0%      0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      for <i>rsync</i> with minimal features (no sparse file consideration):
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 0%</li>
      <li>data reduction on storage for the diff backup is: 0%</li>
      <li>data reduction over the network for the full backup is: 61.23%</li>
      <li>data reduction over the network for the diff backup is: 99.29%</li>
      <li>execution time to restore a single file is: 0.003 s</li>
      <li>execution time ot restore the full+diff backup: 157.81 s (this is not due to the <code>--no-whole-file</code> see next text)</li>
      <li>full backup time: 156.98 s</li>
      <li>diff backup time: 7.33 s</li>
    </ul>

    <h6>sparse file + binary delta</h6>

    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir rsync-backup</b>
	<b>devuan:/mnt/memdisk# time rsync -arHSz --info=stats state-1/* rsync-backup</b>

	<e>sent 1,585,540,014 bytes  received 2,174,460 bytes</e>  8,605,498.50 bytes/sec
	total size is 4,260,538,564  speedup is 2.68
	232.038u 13.137s <e>3:03.44</e> 133.6% 0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# time rsync -arHSz --info=stats <e class=blue>--no-whole-file</e> state-2/* rsync-backup</b>

	<e>sent 29,077,381 bytes  received 216,577 bytes</e>  3,446,348.00 bytes/sec
	total size is 4,300,916,222  speedup is 146.82
	7.305u 1.275s <e>0:07.04</e> 121.7%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# du -sb rsync-backup</b>
	<e>4136318307</e>      rsync-backup
	<b>devuan:/mnt/memdisk# rm -rf state-1</b>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time rsync -arHSz --info=stats rsync-backup/* DST</b>

	sent 1,599,585,756 bytes  received 2,181,219 bytes  10,042,426.18 bytes/sec
	total size is 4,300,916,222  speedup is 2.69
	205.089u 12.354s <e>2:38.39</e> 137.2% 0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
	<b>devuan:/mnt/memdisk# time rsync -arHSz --info=stats rsync-backup/etc/fstab DST/etc</b>

	sent 44 bytes  received 12 bytes  112.00 bytes/sec
	total size is 664  speedup is 11.86
	0.001u 0.002s 0:00.00 0.0%      0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      for <i>rsync</i> with advanced features (sparse file consideration <code>-S option</code>, binary delta <code>--no-whole-file</code>)
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 0%</li>
      <li>data reduction on storage for the diff backup is: 0%</li>
      <li>data reduction over the network for the full backup is: 60.64%</li>
      <li>data reduction over the network for the diff backup is: 99.28%</li>
      <li>execution time to restore a single file is: 0.003 s</li>
      <li>execution time to restore the full+diff backup: 158.39 s</li>
      <li>full backup time: 183.44 s</li>
      <li>diff backup time: 7.04 s</li>

    </ul>


    <h5>Tar</h5>
    <h6>minimal features</h6>
    <code class=block>
	<b>devuan:/mnt/memdisk# cd state-1</b>
	<b>devuan:/mnt/memdisk/state-1# time tar --listed-incremental=../snapshot.file -czf ../tar-full.tar.gz *</b>
	<e class=red>tar: tmp/.ICE-unix/19789: socket ignored</e>
	<e class=red>tar: tmp/.X11-unix/X0: socket ignored</e>
	153.624u 8.676s 2:31.71 106.9%  0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/state-1# time tar --listed-incremental=../snapshot.file -czf ../tar-diff.tar.gz *</b>
	0.809u 0.369s 0:00.98 118.3%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/state-1# cd ..</b>
	<b>devuan:/mnt/memdisk# ls -l tar*</b>
	-rw-r--r-- 1 root root     765425 Dec  3 16:49 tar-diff.tar.gz
	-rw-r--r-- 1 root root 1546464033 Dec  3 16:48 tar-full.tar.gz
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time tar -xzf ../tar-full.tar.gz</b>
	27.106u 6.756s 0:26.72 126.6%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# cd ..</b>
	<b>devuan:/mnt/memdisk# diff --no-dereference -r state-1 DST</b>
	Only in state-1: .cache
	Only in state-1/tmp/.ICE-unix: 19789
	Only in state-1/tmp/.X11-unix: X0
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time tar -xzf ../tar-diff.tar.gz</b>
	0.183u 0.085s <e class=red>0:00.18</e> 144.4%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST#</b>
    </code>
    <p>
      Doing that way, the <i>tar</i> differential backup is empty: it only contains empty directories, no file data.
      We will apply the changes over <code>state-1</code> rather than already setup changes at <code>state-2</code>.
      This seems to mean that if the system clock is wrong or was wrong at the time a file was modified
      (like daylight saving? or before NTP synchronization at system startup), which is
      the same as here, were the changes have been brought before full backup was done, <b><u>those changes will not be
      backed up by <i>tar</i></u></b>, while the file's attributes (file size, last modification date,...) changed.
    </p>
    <code class=block>
	<b>devuan:/mnt/memdisk# cd state-1</b>
	<b>devuan:/mnt/memdisk/state-1# time tar --listed-incremental=../snapshot.file -czf ../tar-full.tar.gz *</b>
	<e class=red>tar: tmp/.ICE-unix/19789: socket ignored</e>
	<e class=red>tar: tmp/.X11-unix/X0: socket ignored</e>
	150.751u 8.299s <e>2:28.59</e> 107.0%  0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/state-1# cd ..</b>
	<b>devuan:/mnt/memdisk# dar -x ~denis/tmp/diff -R state-1 --fsa-scope none -u "*" -w -q</b>
	<b>devuan:/mnt/memdisk# cd state-1</b>
	<b>devuan:/mnt/memdisk/state-1# time tar --listed-incremental=../snapshot.file -czf ../tar-diff.tar.gz *</b>
	6.147u 0.559s <e>0:06.40</e> 104.5%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/state-1# cd ..</b>
	<b>devuan:/mnt/memdisk# ls -l tar* snapshot.file</b>
	-rw-r--r-- 1 root root    <e>3350869</e> Dec  3 17:08 snapshot.file
	-rw-r--r-- 1 root root   <e>44607904</e> Dec  3 17:08 tar-diff.tar.gz
	-rw-r--r-- 1 root root <e>1546448179</e> Dec  3 17:04 tar-full.tar.gz
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd dst</b>
	dst: No such file or directory.
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time tar -xzf ../tar-full.tar.gz</b>
	26.807u 7.020s <e>0:26.72</e> 126.5%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# time tar -xzf ../tar-diff.tar.gz</b>
	1.492u 0.381s <e>0:01.48</e> 126.3%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# time tar -xzf ../tar-full.tar.gz etc/fstab</b>
	25.219u 2.581s <e>0:25.15</e> 110.4%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST#</b>
    </code>

    <p>
      for <i>tar</i> with minimal features (no sparse file consideration):
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 62.16%</li>
      <li>data reduction on storage for the diff backup is: 98.92%</li>
      <li>data reduction over the network is the same as on storage</li>
      <li>execution time to restore a single file is: 25.15 s</li>
      <li>execution time to restore the full backup: 26.72 s</li>
      <li>execution time ot restore the diff backup: 1.48 s</li>
      <li>full backup time: 148.59 s</li>
      <li>diff backup time: 6.40 s</li>
    </ul>


    <h6>sparse file</h6>

    <p>
      Whe had to recreate <code>state-1</code> as we needed modifying it at previous test
    </p>

    <code class=block>
	<b>devuan:/mnt/memdisk# rm tar* snapshot.file</b>
	<b>devuan:/mnt/memdisk# cd state-1</b>
	<b>devuan:/mnt/memdisk/state-1# time tar --listed-incremental=../snapshot.file -czSf ../tar-full.tar.gz *</b>
	<e class=red>tar: tmp/.ICE-unix/19789: socket ignored</e>
	<e class=red>tar: tmp/.X11-unix/X0: socket ignored</e>
	152.878u 10.155s <e>2:29.38</e> 109.1% 0+0k 1520+0io 18pf+0w
	<b>devuan:/mnt/memdisk/state-1# dar -x ~denis/tmp/diff --fsa-scope none -u "*" -w -q</b>
	<b>devuan:/mnt/memdisk/state-1# time tar --listed-incremental=../snapshot.file -czSf ../tar-diff.tar.gz *</b>
	6.369u 0.752s <e>0:06.55</e> 108.5%    0+0k 3992+0io 16pf+0w
	<b>devuan:/mnt/memdisk/state-1# cd ..</b>
	<b>devuan:/mnt/memdisk# ls -l tar* snap*</b>
	-rw-r--r-- 1 root root    <e>3350870</e> Dec  3 17:29 snapshot.file
	-rw-r--r-- 1 root root   <e>44604194</e> Dec  3 17:29 tar-diff.tar.gz
	-rw-r--r-- 1 root root <e>1546226992</e> Dec  3 17:27 tar-full.tar.gz
	<b>devuan:/mnt/memdisk# rm -rf DST</b>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time tar -xzSf ../tar-full.tar.gz</b>
	27.331u 7.774s <e>0:26.27</e> 133.6%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# time tar -xzSf ../tar-diff.tar.gz</b>
	1.547u 0.487s <e>0:01.50</e> 134.6%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# time tar -xzSf ../tar-full.tar.gz etc/fstab</b>
	25.068u 2.565s <e>0:25.00</e> 110.4%   0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST#</b>
    </code>

    <p>
      for <i>tar</i> with advanced features (sparse file consideration <code>-S option</code>):
    </p>
    <ul>
      <li>data reduction on storage for the full backup is: 62.16%</li>
      <li>data reduction on storage for the diff backup is: 98.92%</li>
      <li>data reduction over the network is the same as on storage</li>
      <li>execution time to restore a single file is: 25.0 s</li>
      <li>execution time to restore the full backup: 26.27 s</li>
      <li>execution time ot restore the diff backup: 1.50 s</li>
      <li>full backup time: 149.38 s</li>
      <li>diff backup time: 6.55 s</li>
    </ul>


    <h4>Ciphering performance</h4>

    <p>
      We evaluate here ciphering and deciphering performance. To compare on the same base we use the following
      parameters:
      <ul>
	<li>AES-256 algorithm with CBC mode</li>
	<li>pkcs5 v2 (pbkdf2) key derivation function (KDF) algorithm</li>
	<li>KDF with 100,000 iterations</li>
	<li>salt</li>
	<li>password provided on command-line (insecure) to not depend on user or disk access</li>
	<li>local system to backup and backup repository on a tmpfs filesystem</li>
	<li>swap has been disabled to avoid tmpfs latency in case it would have been swapped out</li>
      </ul>

      To measure the ciphering time only, we will not use compression, though most of the time compression
      should be used due to the use case encryption matches: relatively long time storage and/or costing
      cloud space and network transfer time or limited removable media storage.
    </p>
    <p>
      The content that will be backed up is a copy of <code> /usr</code> directory tree. We will measure:
      <ul>
	<li>the time to backup</li>
	<li>the time to restore the whole backup</li>
	<li>and the time to restore just the "diff" binary</li>
      </ul>
    </p>

    <code class=block>
	<b>devuan:/mnt/memdisk# mkdir SRC</b>
	<b>devuan:/mnt/memdisk# cp --preserve -r /usr SRC</b>
	<b>devuan:/mnt/memdisk# time dar -c backup -K "aes256:hello world!" --kdf-param 100000:sha1 -R SRC -q -at -1 0</b>
	9.213u 3.245s <e>0:05.38</e> 231.4%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# time dar -x backup -K "hello world!" -R DST -q</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	4.481u 2.628s <e>0:03.75</e> 189.3%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# rm -rf DST/usr</b>
	<b>devuan:/mnt/memdisk# time dar -x backup -K "hello world!" -R DST -q -g usr/bin/diff</b>
	Warning, the archive backup has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted
	0.419u 0.025s <e>0:00.42</e> 102.3%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk#</b>
    </code>
    <p>
      For <i>dar</i> the operation took:
    </p>
    <ul>
      <li>5.38 seconds to backup with encryption</li>
      <li>3.75 seconds to restore the whole ciphered backup</li>
      <li>0.42 seconds to restore a single file from the ciphered backup</li>
    </ul>
    <p>
      <i>dar</i> is twice quicker to uncipher than to cipher the whole archive, but restoring a particular file is quite immediate.
      By default, <i>dar</i> uses argon2 for KDF, which is the most secure algorithm as of year 2020 to derive a key, but we had to
      adapt to openssl used with <i>tar</i> that does not (yet) support this algorithm.
    </p>
    <p>
      To avoid plain-text attack a variable length elastic buffer containing random data is encrypted with the rest of the backed up files
      at the beginning and at the end of the backup, this has some performance penalties (time to generate and time to cipher/decipher).
      This explains why two identical invocations of <i>dar</i> produce backups of different sizes and execution times:
    </p>
    <code class=block>
	<b>devuan:/mnt/memdisk# time dar -c backup -K "aes256:hello world!" -at -1 0 -R SRC -q -w</b>
	9.782u 3.413s <e>0:06.28</e> 210.0%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# la backup.1.dar</b>
	-rw-r--r-- 1 root root 1572<e>706497</e> Nov  9 14:50 backup.1.dar
	<b>devuan:/mnt/memdisk# time dar -c backup -K "aes256:hello world!" -at -1 0 -R SRC -q -w</b>
	9.173u 2.845s <e>0:05.50</e> 218.3%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk# la backup.1.dar</b>
	-rw-r--r-- 1 root root 1572<e>655217</e> Nov  9 14:50 backup.1.dar
	<b>devuan:/mnt/memdisk#</b>
    </code>

    <p>
      <i>rsync</i> has no way to store the backup ciphered. Testing directly <i>tar</i> now:
    </p>
    <p>
      <i>tar</i> has not support for ciphering. Though it seems the some use openssl workaround
      this restriction. To measure the execution time we have to create as script that pipes <i>tar</i>
      and <i>openssl</i> so we can measure the execution time of this script as a whole. There is thus one
      script for backup and one for the restoration of <i>tar</i>+<i>openssl</i>.
    </p>
    <code class=block>
	<b>devuan:/mnt/memdisk# cat &gt; tar.backup</b>
	#!/bin/bash

	if [ -z "$1" ] ; then
	&nbsp;&nbsp;echo "usage: $0 &lt;backup name&gt; [ &ltfile or dir&gt; ]"
	&nbsp;&nbsp;exit 1
	fi

	tar -cf - "$2" | openssl enc -e -aes256 -out "$1" -pbkdf2 -iter 100000 -salt -pass pass:"hello world!"
	<b>devuan:/mnt/memdisk#</b>
	<b>devuan:/mnt/memdisk# chmod u+x tar.backup</b>
	<b>devuan:/mnt/memdisk# cd SRC</b>
	<b>devuan:/mnt/memdisk/SRC# time ../tar.backup ../backup.tar.crypted usr</b>
	3.954u 2.498s <e>0:04.69</e> 137.3%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/SRC# cd ..</b>
	<b>devuan:/mnt/memdisk# ls -l backup.tar.crypted</b>
	-rw-r--r-- 1 root root 1603594272 Nov  9 13:33 backup.tar.crypted
	<b>devuan:/mnt/memdisk#</b>
	<b>devuan:/mnt/memdisk# cat &gt; tar.restore</b>
	#!/bin/bash

	if [ -z "$1" ] ; then
	&nbsp;&nbsp;echo "usage: $0 &lt;tar.crypted file&gt; [&lt;file or dir&gt;]"
	&nbsp;&nbsp;exit 1
	fi

	openssl enc -d -aes256 -in "$1" -pbkdf2 -iter 100000 -salt -pass pass:"hello world!" | tar -x "$2"
	<b>devuan:/mnt/memdisk# chmod u+x tar.restore</b>
	<b>devuan:/mnt/memdisk# rm -rf DST</b>
	<b>devuan:/mnt/memdisk# mkdir DST</b>
	<b>devuan:/mnt/memdisk# cd DST</b>
	<b>devuan:/mnt/memdisk/DST# time ../tar.restore ../backup.tar.crypted</b>
	1.807u 2.821s <e>0:02.70</e> 171.1%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST#</b>
	<b>devuan:/mnt/memdisk/DST# rm -rf usr</b>
	<b>devuan:/mnt/memdisk/DST# time ../tar.restore ../backup.tar.crypted usr/bin/diff</b>
	1.336u 1.428s <e>0:01.79</e> 153.6%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/DST# find</b>
	.
	./usr
	./usr/bin
	./usr/bin/diff
	<b>devuan:/mnt/memdisk/DST#</b>

    </code>

    <p>
      For <i>tar</i> the operation took:
    </p>
    <ul>
      <li>4.69 seconds for backup</li>
      <li>2.70 seconds to restore the whole backup</li>
      <li>1.79 seconds to restore a single file</li>
    </ul>
    <p>
      <i>tar</i> as <i>dar</i> is also twice longer to cipher than to decipher, this seems to be related to the algorithm itself.
      Though <i>tar</i> is a bit faster than <i>dar</i> but lacks protection against clear-text: the generated encrypted backup
      have the exact same sizes at one byte precision, this means the blocks boundaries and tar file internal structure always
      lay at the same file offset for a given content:
    </p>
    <code class=block>
	<b>devuan:/mnt/memdisk/SRC# time ../tar.backup ../backup.tar.crypted usr</b>
	4.112u 2.343s 0:04.72 136.6%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/SRC# ls -l ../bac</b>
	backup.1.dar        backup.tar.crypted
	<b>devuan:/mnt/memdisk/SRC# ls -l ../backup.tar.crypted</b>
	-rw-r--r-- 1 root root <e>1603594272</e> Nov  9 14:56 ../backup.tar.crypted
	<b>devuan:/mnt/memdisk/SRC# time ../tar.backup ../backup.tar.crypted usr</b>
	3.952u 2.564s 0:04.79 135.9%    0+0k 0+0io 0pf+0w
	<b>devuan:/mnt/memdisk/SRC# ls -l ../backup.tar.crypted</b>
	-rw-r--r-- 1 root root <e>1603594272</e> Nov  9 14:56 ../backup.tar.crypted
	<b>devuan:/mnt/memdisk/SRC#</b>
    </code>


    <h2><a name="scripts">Scripts</a> used in this benchmark</h2>

    <p>
      The following scripts are also available for download
      from the <a href="Benchmark_tools"> directory</a>
    </p>

    <h3><a name="historization_feature"><i>historization_feature</i></a> script</h3>
    <code class=block>
	#!/bin/bash

	if [ -z "$1" -o -z "$2" ] ; then
	echo "usage: $0 &lt;dir&gt; {phase1 | phase2}"
	  exit 1
	fi

	dir="$1"
	phase="$2"

	case "$phase" in
	phase1)
	  if [ -e "$dir" ] ; then
	    echo "$dir exists, remove it first"
	    exit 2
	  fi
	  mkdir "$dir"
	  echo "Hello World!" > "$dir/A.txt"
	  echo "Bonjour tout le monde !" > "$dir/B.txt"
	  ;;
	phase2)
	  if [ ! -d "$dir" ] ; then
	    echo "$dir does not exist or is not a directory, run phase1 first"
	    exit 2
	  fi

	  rm -f "$dir/A.txt"
	  echo "Buongiorno a tutti !" > "$dir/C.txt"
	  ;;
	*)
	  echo "unknown phase"
	  exit 2
	  ;;
	esac

    </code>

    <h3><a name="always_change"><i>always_change</i></a> script</h3>
    <code class=block>
	#!/bin/bash

	if [ -z "$1" ] ; then
	echo "usage: $0 &lt;filename&gt;"
	  exit 1
	fi

	while /bin/true ; do  touch "$1" ; done
    </code>

    <h3><a name="bitflip"><i>bitflip</i></a> script</h3>
    <code class=block>
	#!/bin/bash

	if [ -z "$1" -o -z "$2" ] ; then
  	  echo "usage: $0 &lt;offset in bit&gt; &lt;file&gt;"
	  echo "flip the bit of the file located at the provided offset"
  	  exit 1
	fi

	offbit=$1
	file="$2"

	offbyte=$(( $offbit / 8 ))
	bitinbyte=$(( $offbit - ($offbyte * 8) ))
	readbyte=`xxd -s $offbyte -p -l 1 "$file"`
	mask=$(( 1 &lt;&lt; $bitinbyte ))
	newbyte=$(( 0x$readbyte ^ $mask ))
	hexanewbyte=`printf "%.2x" $newbyte`

	echo $hexanewbyte | xxd -p -l 1 -s $offbyte -r - "$file"
    </code>

    <h3><a name="build_test_tree.bash"><i>build_test_tree.bash</i></a> script</h3>
    <code class=block>
	#!/bin/bash

	if [ -z "$1" ] ; then
	echo "usage: $0 &lt;directory&gt;"
  	  exit 1
	fi

	if [ -e "$1" ] ; then
	  echo "$1 already exists, remove it or use another directory name"
	  exit 1
	fi

	if ! dar -V > /dev/null ; then
	  echo "need dar to copy unix socket to the test tree"
	  exit 1
	fi

	mkdir "$1"
	cd "$1"

	# creating
	mkdir "SUB"
	dd if=/dev/zero of=plain_zeroed bs=1024 count=1024
	dd if=/dev/urandom of=random bs=1024 count=1024
	dd if=/dev/zero of=sparse_file bs=1 count=1 seek=10239999
	ln -s random SUB/symlink-broken
	ln -s ../random SUB/symlink-valid
	mkfifo pipe
	mknod null c 3 1
	mknod fd1 b 2 1
	dar -c - -R / -g dev/log -N -Q -q | dar -x - --sequential-read -N -q -Q
	ln sparse_file SUB/hard_linked_sparse_file
	ln dev/log SUB/hard_linked_socket
	ln pipe SUB/hard_linked_pipe

	# modifying dates and permissions
	sleep 2
	chown nobody random
	chown -h bin SUB/symlink-valid
	chgrp -h daemon SUB/symlink-valid
	sleep 2
	echo hello &gt;&gt; random
	sleep 2
	cat &lt; random &gt; /dev/null

	# adding Extend Attributes, assuming the filesystem as user_xattr and acl option set
	setfacl -m u:nobody:rwx plain_zeroed && setfattr -n "user.hello" -v "hello world!!!" plain_zeroed || (echo "FAILED TO CREATE EXTENDED ATTRIBUTES" && exit 1)

	# adding filesystem specific attributes
	chattr +dis plain_zeroed
    </code>

    <h3><a name="hide_change"><i>hide_change</i></a> script</h3>
    <code class=block>
	#!/bin/bash

	if [ -z "$1" ] ; then
	  echo "usage: $0 &lt;filename&gt; [&lt;bit offset&gt;]"
	  echo "modify one bit and hide the change"
	  exit 1
	fi

	atime=`stat "$1" | sed -rn -s 's/^Access:\s+(.*)\+.*/\1/p'`
	mtime=`stat "$1" | sed -rn -s 's/^Modify:\s+(.*)\+.*/\1/p'`

	bitoffset="$2"

	if [ -z "$bitoffset" ] ; then
	bitoffset=2
	fi

	./bitflip "$bitoffset" "$1"

	touch -d "$mtime" "$1"
	touch -a -d "$atime" "$1"
    </code>
  </body>
</html>