File: FAQ.adoc

package info (click to toggle)
rdiff-backup 2.2.6-3
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid, trixie
  • size: 4,640 kB
  • sloc: python: 24,129; javascript: 9,512; sh: 1,230; ansic: 580; makefile: 36
file content (521 lines) | stat: -rw-r--r-- 32,728 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
= rdiff-backup: {page-name}
:page-name: F.A.Q.
:sectnums:
:toc: macro

link:.[Home,role="button round"] // *{page-name}*

'''''

toc::[]


== What do the different verbosity levels mean?

There is no formal specification, but here is a rough description (settings are always cumulative, so 5 displays everything 4 does):

 0 No information given
 1 Fatal Errors displayed
 2 Warnings
 3 Important notes, and maybe later some global statistics (default)
 4 Some global settings, miscellaneous messages (obsolete)
 5 Mentions which files were changed, and other info
 6 More information on each file processed (obsolete)
 7 More information on various things (obsolete)
 8 Debug, without timestamp
 9 Also debug, but with timestamp

== Is rdiff-backup backwards compatible?

In general, rdiff-backup does not strive to make newer clients compatible with older servers (or vice versa).
However, there is no intention to purposefully make different versions incompatible across the network -- changes are introduced primarily to fix bugs or introduce new features that cannot be implemented without breaking the network protocol.
Furthermore, rdiff-backup does try to make it possible to read older archives.

When running as a client, rdiff-backup checks the version of rdiff-backup running on the server, and prints a warning message if the two versions are different.
If you have any problems with your backup, it is strongly recommended that you upgrade the older version before reporting any issues.

== Does rdiff-backup run under Windows?

Yes, although it is not a heavily tested configuration.
Rdiff-backup can be run as a native Windows application or under Cygwin.
To run as a native Windows application, simply download the provided .exe binary.
To setup remote operation, you will also need an SSH client, such as https://www.chiark.greenend.org.uk/~sgtatham/putty/[Putty] or https://www.ssh.com[SSH Secure Shell].

If you wish to run rdiff-backup under Cygwin, use at least version 1.1.12.
The setup under Cygwin is the same as under other Unix-like operating systems.
From the Cygwin installer you will need Python 3.6 or higher (under Interpreters), autoconf, automake, binutils, gcc, make, and patchutils (all under Devel).
Then you will need to compile and install librsync, which can be downloaded https://sourceforge.net/project/showfiles.php?group_id=56125[from Sourceforge].
Finally, you can compile and install rdiff-backup using the usual instructions.

Although some Windows filesystems lack features like FIFOs, case sensitive filenames, or files with colons (":") in them, all of these situations should be autodetected and compensated for by rdiff-backup.

If you would like more detailed instructions for compiling and installing rdiff-backup on Cygwin, you can read this blog entry: https://katastrophos.net/andre/blog/2005/11/02/rdiff-backup-on-windows/.
Note: The patch that the blog suggests that you download is no longer necessary starting with version 1.1.8 of rdiff-backup.

== Does rdiff-backup run under Mac OS X?

Yes, quite a few people seem to be using rdiff-backup under Mac OS X.
rdiff-backup can also backup resource forks and other Mac OS X metadata to a traditional unix filesystem, which is can be a handy feature for Mac users.
When rdiff-backup is used to do the restore, all of the metadata is recovered from rdiff-backup's storage.

The easiest option is probably to use Fink http://fink.sourceforge.net/, which can install rdiff-backup automatically for you.
With Fink, you should install the `librsync`, `librsync-shlibs`, `python25`, `python25-shlibs`, `xattr-py25`, and `rdiff-backup` packages.
Another option is DarwinPorts https://www.macports.org/, for which you should install the `py-xattr` and `rdiff-backup` packages.

If you want to build rdiff-backup yourself, you should be able to build librsync and rdiff-backup using the standard Unix instructions.
You can also see this message from Gerd Knops:

....
From: Gerd Knops <gerti@bitart.com>
Date: Thu, 3 Oct 2002 03:56:47 -0500 (01:56 PDT)

[parts of original message deleted]
these instructions build it fine with all tests running OK
(librsync-0.9.5.1 on OS X 10.2.1):

    aclocal
    autoconf
    automake --foreign --add-missing
    env CFLAGS=-no-cpp-precomp ./configure
    make
    make install
....

An important note if you use the Apple-provided version of Python: Apple's version of Python will install rdiff-backup in something like `/System/Library/Frameworks/Python.framework/Versions/Current/bin/rdiff-backup` and `rdiff-backup` will not be in your `$PATH`.
You can copy rdiff-backup out of this folder and into someplace reasonable like `/usr/bin` or another directory in your `$PATH` to use it.
For a full explanation of why this happens see this post to the mailing list: https://lists.nongnu.org/archive/html/rdiff-backup-users/2007-06/msg00107.html.

== Does rdiff-backup have a GUI or WebUI?

No, and it doesn't plan to.
It leaves this exercise to external utilities like (we'll list only open source tools here):

* https://rdiffweb.org/[Rdiffweb] is a Web interface to browse and restore data.
* https://minarca.org/[Minarca] is a more complete solution for non-technical users.
It uses Rdiffweb with extra plugins for the server and provides a GUI to be installed on the desktop.
* https://github.com/schaeferservices/simple-rdiff-backup-webui[simple-rdiff-backup-webui] is a very simple "read-only" PHP based web UI

== Can I backup files to a CIFS or smbfs mount?

You can certainly try!
Using a CIFS or smbfs mount as the mirror directory has been troublesome for some users because of the wide variety of Samba configurations.
If possible, the best solution is always to use rdiff-backup over SSH in the default configuration.
Using rdiff-backup in the default configuration is also guaranteed to be faster because there is lower network utilization.
Rdiff-backup uses the rsync algorithm to minimize the amount of bandwidth consumed.
By using smbfs or CIFS, the complete file is transferred over the network.

Under both Linux and Mac OS X, smbfs seems to be working quite well.
However, it has a 2 GB file limit and is deprecated on Linux.
CIFS users sometimes experience one of these common errors:

 ** rdiff-backup fails to run, printing an exception about "`assert not upper_a.lstat()`" failing.
This can be resolved by unmounting the share, running the following command as root:\ `$ echo 0 > /proc/fs/cifs/LookupCacheEnabled`\ and then remounting the CIFS share.\ \
 ** If filenames in the mirror directory have some characters transformed to a '?' instead of remaining the expected Unicode character, you will need to adjust the `iocharset=` mount option.
This happens because the server is using a codepage with only partial Unicode support and is not translating characters correctly.
See the mount.cifs man page for more information.
Using smbfs can also improve this situation since it has both an `iocharset=` and a `codepage=` option.
 ** If you have trouble with filenames containing a colon ':', or another reserved Windows character, try using the `mapchars` option to the CIFS mount.
At least one user has reported success when using this option while mounting a NAS system via CIFS.
See the mount.cifs man page for more information.\ \
 ** Other CIFS mount options which may be helpful include `nocase`, `directio`, and `sfu`.
Also, try changing the value of `/proc/fs/cifs/LinuxExtensionsEnabled` (requires remount).
A user with a DroboShare reported that `-o mapchars,nocase,directio` worked for that NAS appliance.

If you're still having trouble backing up to a CIFS or smbfs mount, try searching the https://lists.gnu.org/archive/html/rdiff-backup-users/[mailing-list archives] and then sending further questions to the list.

== Help! Why do all my filenames now look like ;077y;070ile ?!

When backing up from a case-sensitive filesystem to a case-insensitive filesystem (such as Mac's HFS+ or Windows's FAT32 or NTFS), rdiff-backup escapes uppercase characters in filenames to make sure that no files are accidentally overwritten.
When a filesystem is case-preserving but case-insensitive, it means that it remembers that a file is named "Foo" but doesn't distinguish between "Foo", "foo", "foO", "fOo", etc.
However, filesystems such as Linux's ext3 do treat these names as separate files.

Imagine you have a Linux directory with two files, "bar" and "BAR", and you copy them to a Mac system.
You will wind up with only one file (!) since HFS+ doesn't distinguish between the names, and the second file copied will overwrite the first.
Therefore, when rdiff-backup copies files from case-sensitive to case-insensitive filesystems, it escapes the uppercase characters (eg, "M" is replaced with ";077", and "F" with ";070") so that no filename conflicts occur.
Upon restore (from the Mac backup server to the Linux system), the filenames are unquoted and you will get "MyFile" back.

== My backup set contains some files that I just realized I don't want/need backed up.  How do I remove them from the backup volume to save space?

The only official way to remove files from an rdiff-backup repository is by letting them expire using the --remove-older-than option.
Deleting increments from the rdiff-backup-data directory will prevent you from recovering those files, but shouldn't prevent the rest of the repository from being restored.

== Does rdiff-backup work under Solaris?

There may be a problem with rdiff-backup and Solaris' libthread.
Adding "ulimit -n unlimited" may fix the problem though.
Here is a post by Kevin Spicer on the subject:

....
Subject: RE: Crash report....still not^H^H^H working
From: "Spicer, Kevin" <kevin.spicer@bmrb.co.uk>
Date: Sat, 11 May 2002 23:36:42 +0100
To: rdiff-backup@keywest.Stanford.EDU

Quick mail to follow up on this..
My rdiff backup (on Solaris 2.6 if you remember) has now worked
reliably for nearly two weeks after I added...

    ulimit -n unlimited

to the start of my cron job and created a wrapper script on the remote
machine which looked like this...

    ulimit -n unlimited
    rdiff-backup --server
    exit

And changed the remote schema on the command line of rdiff-backup to
call the wrapper script rather than rdiff-backup itself on the remote
machine.  As for the /dev/zero thing I've done a bit of Googleing and
it seems that /dev/zero is used internally by libthread on Solaris
(which doesn't really explain why its opening more than 64 files - but
at least I think I've now got round it).
....

== How fast is rdiff-backup? Can it be run on large data sets?

rdiff-backup can be limited by the CPU, disk IO, or available bandwidth, and the length of a session can be affected by the amount of data, how much the data changed, and how many files are present.
That said, in the typical case the number/size of changed files is relatively small compared to that of unchanged files, and rdiff-backup is often either CPU or bandwidth bound, and takes time proportional to the total number of files.
Initial mirrorings will usually be bandwidth or disk bound, and will take much longer than subsequent updates.

To give one arbitrary data point, when I back up my personal HD locally (about 36GB, 530000 files, maybe 500 MB turnover, Athlon 2000, 7200 IDE disks, version 0.12.2) rdiff-backup takes about 15 minutes and is usually CPU bound.

== What do the various fields mean in the session statistics and directory statistics files?

Let's examine an example session statistics file:

 StartTime 1028200920.44 (Thu Aug  1 04:22:00 2002)
 EndTime 1028203082.77 (Thu Aug  1 04:58:02 2002)
 ElapsedTime 2162.33 (36 minutes 2.33 seconds)
 SourceFiles 494619
 SourceFileSize 8535991560 (7.95 GB)
 MirrorFiles 493797
 MirrorFileSize 8521756994 (7.94 GB)
 NewFiles 1053
 NewFileSize 23601632 (22.5 MB)
 DeletedFiles 231
 DeletedFileSize 10346238 (9.87 MB)
 ChangedFiles 572
 ChangedSourceSize 86207321 (82.2 MB)
 ChangedMirrorSize 85228149 (81.3 MB)
 IncrementFiles 1857
 IncrementFileSize 13799799 (13.2 MB)
 TotalDestinationSizeChange 28034365 (26.7 MB)
 Errors 0

StartTime and EndTime are measured in seconds since the epoch.
ElapsedTime is just EndTime - StartTime, the length of the rdiff-backup session.

SourceFiles are the number of files found in the source directory, and SourceFileSize is the total size of those files.
MirrorFiles are the number of files found in the mirror directory (not including the rdiff-backup-data directory) and MirrorFileSize is the total size of those files.
All sizes are in bytes.
If the source directory hasn't changed since the last backup, MirrorFiles == SourceFiles and SourceFileSize == MirrorFileSize.

NewFiles and NewFileSize are the total number and size of the files found in the source directory but not in the mirror directory.
They are new as of the last backup.

DeletedFiles and DeletedFileSize are the total number and size of the files found in the mirror directory but not the source directory.
They have been deleted since the last backup.

ChangedFiles are the number of files that exist both on the mirror and on the source directories and have changed since the previous backup.
ChangedSourceSize is their total size on the source directory, and ChangedMirrorSize is their total size on the mirror directory.

IncrementFiles is the number of increment files written to the rdiff-backup-data directory, and IncrementFileSize is their total size.
Generally one increment file will be written for every new, deleted, and changed file.

TotalDestinationSizeChange is the number of bytes the destination directory as a whole (mirror portion and rdiff-backup-data directory) has grown during the given rdiff-backup session.
This is usually close to IncrementFileSize + NewFileSize - DeletedFileSize + ChangedSourceSize - ChangedMirrorSize, but it also includes the space taken up by the hardlink_data file to record hard links.

== Is there some way to limit rdiff-backup's bandwidth usage, as in rsync's --bwlimit option?

There is no internal rdiff-backup option to do this.
However, external utilities such as https://www.cons.org/cracauer/cstream.html[cstream] can be used to monitor bandwidth explicitly.
trevor\@tecnopolis.ca writes:

....
rdiff-backup --remote-schema
  'cstream -v 1 -t 10000 | ssh %s '\''rdiff-backup --server'\'' | cstream -t 20000'
  'netbak@foo.bar.com::/mnt/backup' localbakdir

(must run from a bsh-type shell, not a csh type)

That would apply a limit in both directions [10000 bytes/sec outgoing,
20000 bytes/sec incoming].  I don't think you'd ever really want to do
this though as really you just want to limit it in one direction.
Also, note how I only -v 1 in one direction.  You probably don't want
to output stats for both directions as it will confuse whatever script
you have parsing the output.  I guess it wouldn't hurt for manual runs
however.
....

To only limit bandwidth in one directory, simply remove one of the cstream commands.
Two cstream caveats may be worth mentioning:

 .. Because cstream is limiting the uncompressed data heading into or out of ssh, if ssh compression is turned on, cstream may be overly restrictive.
 .. cstream may be "bursty", limiting average bandwidth but allowing rdiff-backup to exceed it for significant periods.


Another option is to limit bandwidth at a lower (and perhaps more appropriate) level.
Adam Lazur mentions https://lartc.org/wondershaper/[The Wonder Shaper].

== How much memory should rdiff-backup use? Is there a memory leak?

The amount of memory rdiff-backup uses should not depend much on the size of directories being processed.
Keeping track of hard links may use up memory, so if you have, say, hundreds of thousands of files hard linked together, rdiff-backup may need tens of MB.

If rdiff-backup seems to be leaking memory, it is probably because it is using an early version of librsync.
*librsync 0.9.5 leaks lots of memory.* Later versions should not leak and are available from the https://sourceforge.net/projects/librsync/[librsync homepage].

== I use NFS and keep getting some error that includes "OSError: [Errno 39] Directory not empty"

Several users have reported seeing errors that contain lines like this:

 File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py",
     line 661, in rmdir
 OSError: [Errno 39] Directory not empty:
     '/nfs/backup/redfish/win/Program Files/Common Files/GMT/Banners/11132'
 Exception exceptions.TypeError: "'NoneType' object is not callable"
      in <bound method GzipFile.__del__ of

All of these users were backing up onto NFS (Network File System).
I think this is probably a bug in NFS, although tell me if you know how to make rdiff-backup more NFS-friendly.
To avoid this problem, run rdiff-backup locally on both ends instead of over NFS.
This should be faster anyway.

== For some reason rdiff-backup failed while backing up. Now every time it runs it says "regressing destination" and then fails again.  What should I do?

Firstly, this shouldn't happen.
If it does, it indicates a corrupted destination directory, a bug in rdiff-backup, or some other serious recurring problem.

However, here is a workaround that you might want to use, even though it probably won't solve the underlying problem: In the destination's rdiff-backup-data directory, there should be two "current_mirror" files, for instance:

 current_mirror.2003-09-07T16:43:00-07:00.data
 current_mirror.2003-09-08T04:22:01-07:00.data

Delete the one with the earlier date.
Also move the mirror_metadata file with the later date out of the way, because it probably didn't get written correctly because that session was aborted:

 mv mirror_metadata.2003-09-08T04:22:01-07:00.snapshot.gz aborted-metadata.2003-09-08T04:22:01-07:00.snapshot.gz

The next time rdiff-backup runs it won't try regressing the destination.
Metadata will be read from the file system, which may result in some extra files being backed up, but there shouldn't be any data loss.

== Where does rdiff-backup need free space and how much is required? What is the problem when rdiff-backup says "`ValueError: Incorrect length of data produced`"?

When backing up, rdiff-backup needs free space in the mirror directory.
The amount of free space required is usually a bit more than the size of the file getting backed up, but can be as much as twice the size of the current file.
For instance, suppose you ran `rdiff-backup foo bar` and the largest file, `foo/largefile`, was 1GB.
Then rdiff-backup would need 1+GB of free space in the `bar` directory.

When restoring or regressing, rdiff-backup needs free space in the default temp directory.
Under unix systems this is usually the `/tmp` directory.
The temp directory that rdiff-backup uses can be set using the `--tempdir` and `--remote-tempdir` options available in versions 1.1.13 and newer.
See the entry for `tempfile.tempdir` in the https://docs.python.org/3/library/tempfile.html[Python tempfile docs] for more information on the default temp directory.
The amount of free space required can vary, but it usually about the size of the largest file being restored.

Usually free space errors are intelligible, like `IOError: [Errno 28] No space left on device` or similar.
However, due to a gzip quirk they may look like `ValueError: Incorrect length of data produced`.

== What does "internal error: job made no progress" mean?

This error happens due to a bug in `librsync` that prevents it from handling files greater than 4 GB in some situations, such as when transferring between a 32-bit host and a 64-bit host.
https://sourceforge.net/tracker/index.php?func=detail&aid=1439412&group_id=56125&atid=479441[A patch is available] from the librsync project page on Sourceforge.
The https://sourceforge.net/cvs/?group_id=56125[CVS version] of librsync also contains the patch.
More information is also available in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=355178[Debian bug report #355178].

== Why does rdiff-backup say it's not in my $PATH? It is when I login!

If you get an error like `sh: line1: rdiff-backup: command not found`, but rdiff-backup _is_ in your `$PATH` when you login to the remote host, it is happening because the value of bash's `$PATH` is set differently when you login to an interactive shell than when you run a command remotely via SSH.
For more information, read the https://linux.die.net/man/1/bash[bash manpage] and look at your `.bashrc` and `.bash_profile` files.

In particular, this can happen if rdiff-backup was installed via Fink on a remote Mac OS X system.
`/sw/bin` is magically added to your `$PATH` by the script `/sw/bin/init.sh` when you login with an interactive shell.
Fink did this behind the scenes when you set it up.
Simply add `/sw/bin` to your path manually, or copy rdiff-backup to a directory that is in your `$PATH`.

== What does "`tuple index out of range`" mean?

If you see the error "`tuple index out of range`" after running a command like:\ \ `$ rdiff-backup -l /path/to/backup/rdiff-backup-data/`\ \ then the solution is to simply remove the extra "rdiff-backup-data" from the end of the path.
The list increments option, and others like it, take the path to the repository, not the path to the rdiff-backup-data directory.
In the above example, you should run again with:\ \ `$ rdiff-backup -l /path/to/backup`\ \ If you get this error message for an unrelated reason, try contacting the mailing list.

== What does "`IO Error: CRC check failed`" mean?

This error message means that a https://en.wikipedia.org/wiki/Cyclic_redundancy_check[Cyclic Redundancy Check] failed during some operation, most likely while gzip'ing or un-gzip'ing a file.
Possible causes of this error include an incomplete gzip operation, and hardware failure.
A brute-force way to recover from this error is to remove the rdiff-backup-data directory.
However, this will remove all of your past increments.
A better approach may be to delete the particular file that is causing the problem.
A command like:\ \ `$ find rdiff-backup-data -type f -name \*.gz -print0 | xargs -0r gzip --test`\ \ will find the failing file.
For more information on this approach, see this mailing list post: https://lists.nongnu.org/archive/html/rdiff-backup-users/2007-11/msg00008.html.

== What does "`AssertionError: Bad index order`" mean?

If rdiff-backup fails with the message "`AssertionError: Bad index order`," it could be because the files in a directory have changed while rdiff-backup is running.
Possible ways of dealing with this situation include implementing filesystem snapshots using the volume manager, excluding the offending directory, or suspending the process that is changing the directory.
After the text "Bad index order", the error message will indicate which files have caused the problem.

If you get this message for an unrelated reason, try contacting the mailing list.

== How can rdiff-backup use a specific timezone, like UTC?

Like other Unix and Python programs, rdiff-backup respects the `TZ` environment variable, which can be used to temporarily change the timezone.
On Unix, for UTC, simply set `TZ=UTC` in your shell, or prepend ``TZ=UTC `` to the command line used to run rdiff-backup.
On Windows, set the `TZ` environment variable with the `set TZ=UTC` command in the `Cmd.exe` command interpreter (or in a batch script), or with `$env:TZ='UTC'` in PowerShell.
If you want to use a different timezone than UTC, you can refer to the https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/tzset#remarks[`_tzset` CRT documentation] which describes in detail the format Windows expects for the value of the `TZ` variable.

== I've done a blunder in my last backup, how to roll-back?

If you've done something wrong in your last back-up, you have potentially two solutions to get rid of it.
If you've backed-up a file or directory you shouldn't have backed-up, you can remove it again using `rdiff-backup-delete <repo>/<file-or-dir>`;
beware that _all_ files, including all earlier versions, will be removed without any question back!
If the situation is more complicated, you might want to have a look at https://www.timedicer.co.uk/programs/help/rdiff-backup-regress.sh.php[rdiff-backup-regress] which completely removes the last made backup, but beware that regression takes a long time.
Note that we made a copy of this nice utility, placed under `tools/misc` in our Git repo, just to be sure it doesn't get lost.

== Backup on my file-system XYZ fails, can you support it?

rdiff-backup expects certain qualities from a file system and checks for them.
Experience shows that some file systems do fail these tests (or even at runtime) and can't be supported: how could you rely on your backup if rdiff-backup can't rely on the file system?

The list might grow but issues are currently known with especially slow combinations like:

* Fuse-exFAT (the newer kernel version seems fine though)
* "complex" file systems on a NAS (e.g. ZFS on SMB or NFS)
* SSHFS

NOTE: using a Samba/SMB/CIFS share in version `vers=1.0` also causes issues, because features, like ACL support, are not properly detected.
If you get a lot of exceptions '[Errno 13] Permission denied' with info verbosity, you're probably impacted and might want to change the protocol version.

== How to backup safely from case aware to case aware file system?

File systems like VFAT or NTFS are what I call case aware file systems:
they are "officially" case insensitive but they store file names in a case sensitive manner.

This becomes a https://github.com/rdiff-backup/rdiff-backup/issues/11[problem for rdiff-backup if a file is renamed in a case sensitive manner] (e.g. from `MyFile` to `mYfIlE`) because it doesn't recognize them as the same file, but the target file system doesn't accept both files next to each other.

You can either:

. make sure that files are not renamed in such a manner (this doesn't happen that often)
. or you backup _from the beginning_ using the option `--override-chars-to-quote 'A-Z-"*/:<>?\\|;'`, which makes sure that capital letters (and other "strange" characters) are quoted so that there is no name collision on the file system.
+
CAUTION: remember that you can't change the quoting once a backup repository has been setup, so that you might need to create a new one.

== How to work around the limitation to mix file selection and sub-path restore?

Since rdiff-backup 2.1+, a command like `rdiff-backup restore --include myrestore/subdir/somefile --exclude '**' myrepo/subdir myrestore/subdir` isn't possible anymore, because it could lead to https://github.com/rdiff-backup/rdiff-backup/issues/463[data loss].
Such calls are anyway equivalent to something like `rdiff-backup restore --include myrestore/subdir/somefile --exclude '**' myrepo myrestore`.
This means that the new limitation doesn't imply a loss in feature, it only enforces a new approach without risk of losing data.

== How can I integrate the rdiff-backup functionality in my own program?

It can be as easy as importing the `rdiffbackup.run` module and starting the function `rdiffbackup.run.main_run` with the usual command line parameters as list parameter.

The `rdiff-backup` script itself doesn't do it much differently and looks in a simplified manner like this:

[source,python]
----
import rdiffbackup.run
if __name__ == "__main__":
    sys.exit(rdiffbackup.run.main_run(sys.argv[:1]))
----

CAUTION: there is no guarantee as this stage that the `main_run` function can be called more than once in the same program.
The risk is mainly because it makes extensive use of global variables.
Pull requests are more than welcome to fix the issue and make integration easier.

== What user interfaces are available for rdiff-backup ?

Two open source projects exist providing a user interface for
rdiff-backup.

https://rdiffweb.org[Rdiffweb] is a web interface for rdiff-backup. It
can be used to browse and restore your data from the convenience of your
web browser.

https://minarca.org[Minarca] is a centralized backup solution. It
provides an agent to automate the backup process and a server to browse
and restore the data to be installed on your centralized backup server.
The agent is cross-platform allowing you to seamlessly use a single
solution.

== How to use a custom port for SSH access?

If your SSH server is running on a custom port, say 2222, and you are using the OpenSSh client, you may use the URI form `ssh://[user@]hostname[:port]` of the hostname to connect to it, e.g.

----
rdiff-backup backup mylocaldir ssh://myuser@myhost:2222::/backup_repo
----

== Why does rdiff-backup complain about command line interface being deprecated even though I'm using the new syntax?

Calling for example `rdiff-backup backup loc1 server2::loc2` leads to a message `WARNING: this command line interface is deprecated and will disappear, start using the new one as described with '--new --help'`.

You must be using a remote location in your call, as in our example.
In order to make sure that the other side understands the call, rdiff-backup uses the CLI form fitting the default API version.
For example, at time of writing, rdiff-backup 2.2 uses API 200 by default and hence calls `rdiff-backup --server` so that the CLI can be understood by rdiff-backup 2.0.
If the server side is v2.2+, then the warning message will appear.

Call rdiff-backup with the higher API and the message will disappear.

Calling for example `rdiff-backup --api-version 201 backup loc1 server2::loc2` will make sure that `rdiff-backup server` is being called, getting rid of the warning message.

== How to handle a "No space left on device" message while doing a backup?

If you're doing a backup and get an exception `[Errno 28] No space left on device: ...` (or something similar under Windows), it means exactly what it states, there is not enough space to create your backup.
You need to first fix the issue, then recover, then make sure that it doesn't happen again.

In the following lines, we'll assume you've called something like `rdiff-backup backup /from /mnt/bak`, which failed with a "No space left" error.

=== Fixing

Check the actual size left with `df -h /mnt/bak` under Linux or `dir /mnt/bak` under Windows.
Under Linux, you might need to check as well the number of inodes with `df -i /mnt/bak`.

TIP: if the file system doesn't seem to be full, it might be that the default temporary directory instead is full.
In this case, obviously fix this one issue, check for example the `--tempdir` option in the man pages.

If possible, the simplest approach is to increase the size of the underlying filesystem.
There are too many ways to do it to describe here, so check the usual sources for your operating system.

If this is not possible, you can create a new bigger partition using a _similar_ file system type (e.g. POSIX to POSIX, NTFS to NTFS, etc), and move the repository from one to the next file system.

If this is also not an option, and you have nothing else outside of the repository that you can move away (or remove), then you can try to remove the following files:

- `/mnt/bak/rdiff-backup-data/*.log`
- `/mnt/bak/rdiff-backup-data/error_log*`
- `/mnt/bak/rdiff-backup-data/*_statistics.*`

=== Recovering

First you need to regress the repository with something like:

----
rdiff-backup --tempdir /somewherelse/tmp regress /mnt/bak
----

The `--tempdir` option might spare some space on the file system where the repository lies.
Similarly, you can use `--remote-tempdir` if the repository is remote.

// FIXME remove the note once fixed in rpath
NOTE: due to a current limitation, the tempdir options might not help as much as they could.

If this fails due to lack of disk space, you're out of luck and need to increase further the file system size as described above.

Then you can remove old increments, first listing them:

----
rdiff-backup list increments [--size] /mnt/bak
rdiff-backup remove increments [--force] \
	--older-than 'Tue Feb  7 06:43:45 2023' /mnt/bak
----

The date/time can be copied & pasted from the output of the list increments command.
The `--size` parameter can be used to assess the size of the individual increments, and the `--force` option is necessary if you want to remove more than one increment at once.

=== Avoid

There is no magic method, even databases get corrupt if their disk space dwindles.
You need to monitor the disk space and regularly remove old increments, or increase the file system size.
One simple measure can be to call `df -h /mnt/bak` at the end of each backup and keep an eye on space left.

To avoid issues with inodes, use a file system format which doesn't have the issue like XFS, and avoid ones having the limitation like ext4.