File: HOWTO.FAQ.DO-DONT

package info (click to toggle)
afbackup 3.1beta1-1
  • links: PTS
  • area: main
  • in suites: slink
  • size: 1,500 kB
  • ctags: 1,685
  • sloc: ansic: 22,406; csh: 3,597; tcl: 964; sh: 403; makefile: 200
file content (918 lines) | stat: -rw-r--r-- 39,076 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918

                AF's Backup HOWTO
                =================


Index
-----

1. How to optimize the performance to obtain a short backup time ?

2. How to start the backup on several hosts from a central machine ?

3. How to store the backup in a filesystem instead of a tape ?

4. How to use several streamer devices on one machine ?

5. How to recover from a server crash during backup ?

6. How to port to other operating systems ?

7. How to provide recovery from hard crashes (disk crash, ...) ?

8. How to make differential backups ?

9. How to use several servers for one client ?

D. Do-s and Dont-s

F. The FAQ


--------------------------------------------------------------------------

1. How to optimize the performance to obtain a short backup time ?

Basically since version 2.7 the client side tries to optimally adapt
to the actually maximum achievable throughput, so the administrator
doesn't have to do much here.
The crucial point is the location of the bottleneck for the throughput
of the backup data stream. This can be one of:

- The streamer device
- The network connection between backup client and server
- The CPU on the backup client (in case of compression selected)

What usually is not a problem:

- The CPU load of the server

The main influence the administrator has on a good backup performance
is the compression rate on the client side. In most cases the bottleneck
for the data stream will be the network. If it is based on standard
ethernet, the maximum throughput without any other network load will be
around 1 MB/sec. With 100 MBit ethernet or a similar technology about
10 MB/sec might be achieved, so the streamer device is probably the
slowest part (with maybe 5 MB/sec for a Exabyte tape). To use this
capacity it is not clever to plug up the client side CPU with heavy
data compression load. This might be unefficient and thus lead to a
lousy backup performance. The influence of the compression rate on the
backup performance can be made clear with the following table. The
times in seconds have been measured with the (unrepresentative)
configuration given below the table. The raw backup duration gives the
pure data transmission time without tape reeling or cartridge loading
or unloading.

 compression program   |  raw backup duration
-----------------------+----------------------
  gzip -1              |    293 seconds         |
  gzip -5              |    334 seconds         |
  compress             |    440 seconds         | increasing duration
  <no compression>     |    560 seconds         |
  gzip -9              |    790 seconds         V


Configuration:
Server/Client machine:
  586, 133/120MHz (server/client), 32/16 MB (server/client)
Network:
  Standard ethernet (10 MB, 10BASE2 (BNC/Coax), no further load)
Streamer:
  HP-<something>, 190 kByte/sec

Obviously the bottleneck in this configuration is the streamer.
Anyway it shows the big advantage compression can have on the
overall performance. The best performance is here achieved with
the lowest compression rate and thus the fastest compression
program execution. I would expect, that the performance optimum
shifts towards a somewhat better compression with a faster client
CPU (e.g. the latest Alpha-rocket).

So to find an individual performance optimum i suggest to run some
backups with a typical directory containing files and subdirectories
of various sizes. Run these backups manually on the client-side machine
with different compression ratios using the "client"-command as
follows:

/the/path/to/bin/afclient -cvnR -h your_backuphost -z "gzip -1" gunzip \
                             /your/example/directory

Replace "gzip -1" and "gunzip" appropriately for the several runs.


--------------------------------------------------------------------------

2. How to start the backup on several hosts from a central machine ?

For this purpose serves the remote startup utility. To implement this
as fast as possible, a part of the serverside installation must be
made on the client side, where it is requested to start the backup from
a remote site. Choose the appropriate option when running the Install-
script.

To start a backup on another machine, use the -X option of the
client-program. A typical invocation is

/the/path/to/client/bin/afclient -h <hostname> -X incr_backup

This starts an incremental backup on the supplied host. Each
program on the remote host lying in the directory configured
as Program-Directory in the configuration file of the serverside
installation part of the remote host (default: $BASEDIR/server/rexec)
can be started, but no other. The entries may be symlinks, but
they must have the same filename like the programs, they point to.

The machine, where this script is started may be any machine in
the network having the client side of the backup system installed.


--------------------------------------------------------------------------

3. How to store the backup in a filesystem instead of a tape ?

* Option 1 (using symbolic links)

Assumed the directory, where you'd like to store the backup, is
/var/backup/server/vol, change to that directory and create a
symbolic link like this:

 ln -s data.0 data

and create the file data.0 with a

 touch data.0

Then edit your serverside configuration file and make the following
entries (assuming /usr/backup/server/bin is the directory, where the
programs of the server side reside):

Backup-Device:          /var/backup/server/vol/data
Tape-Blocksize:         1024
Cartridge-Handler:      0
Max Bytes Per File:     10485760
Cart-Insert-Gracetime:  0
SetFile-Command:        /bin/rm -f %d;touch %d.%m; ln -s %d.%m %d; exit 0
SkipFiles-Command:      /usr/backup/server/bin/__inc_link -s %d %n
Set-Cart-Command:
Change-Cart-Command:    exit 0
Erase-Tape-Command:     /bin/rm -f %d.[0-9]* %d ; touch %d.0 ; ln -s %d.0 %d ; exit 0

If the directory /var/backup/server/vol/data is on a removable media,
you can supply the number of media you would like to use and an
eject-command as follows:

Number Of Cartridges:   10
# or whatever

Change-Cart-Command:    your_eject_command

If a suitable eject-command does not exist, try to write one yourself.
See below for hints.

Furthermore you can add the appropriate umount command before the eject-
command like this:

Change-Cart-Command:    umount /var/backup/server/vol/data; your_eject_command

To get this working the backup serverside must run as root. Install the
backup stuff supplying the root-user when prompted for the backup user.
Or edit /etc/inetd.conf and replace backup (or whatever user you configured)
(5th column) with root, sending a kill -1 to the inetd afterwards.
Actually you must mount the media manually after having it inserted into
the drive. Afterwards run the command /path/to/server/bin/cartready to
indicate, that the drive is ready to proceed. This is the same procedure
like having a tape drive.

Each media you will use must be prepared creating the file "data.0" and
setting the symbolic link "data" pointing to data.0 like described above.


* Option 2 (supply a directory name as device)

Using one of the serverside configuration programs or editing the
configuration file, supply a directory name as the backup device.
The directory must be writable for the user, under whose ID the
server process is started (whatever you configured during
installation, see /etc/inetd.conf). The backup system then writes
files with automatically generated names into this directory.
The rest of the configuration should e. g. be set as follows:

Backup-Device:          /var/backup/server/vol
Tape-Blocksize:         1024
Cartridge-Handler:      0
Max Bytes Per File:     10485760
Cart-Insert-Gracetime:  0
SetFile-Command:        exit 0
SkipFiles-Command:      exit 0
Set-Cart-Command:
Change-Cart-Command:    exit 0
Erase-Tape-Command:     /bin/rm -f %d/* ; exit 0

A SetFile-Command is mandatory, so this exit 0 is a dummy.
For the further options (using mount or eject commands) refer
to the explanations under * Option 1.


(
   How to write an eject command for my removable media device ?

If the information in the man-pages is not sufficient or you don't
know, where to search, try the following:
Do a grep ignoring case for the words "eject", "offline" and
"unload" over all system header-files like this:

egrep -i '(eject|offl|unload)' /usr/include/sys/*.h

On Linux also try /usr/include/linux/*.h and /usr/include/asm/*.h.
You should find macros defined in headers with names giving hints
to several kinds of devices. Look into the header, whether the
macros could be used with the ioctl system call. The comments
should tell details. Then you can eject the media with the
following code fragment:

#include <sys/ioctl.h>
#include <your_device_related_header>

{
  int   res, fd;
  char  *devicefile = "/dev/whatever";

  fd = open(devicefile, O_RDONLY);

  if(fd < 0){
    /* catch error */
    ...
  }

  res = ioctl(fd, YOUR_EJECT_MACRO);

  if(res < 0){
    /* catch error */ ...
  }

  close(fd);
}

You might want to extend the utility obtainable via ftp from:
ftp://ftp.zn.ruhr-uni-bochum.de/pub/Linux/eject.c and related
files. Please send me any success news. Thanks !


--------------------------------------------------------------------------

4. How to use several streamer devices on one machine ?

Run an installation of the server side for each streamer device,
install everything into a separate directory and give a different
port number to each installed server. This can be done giving each
server an own service name. For the default installation, the
service is named "afbackup" and has port number 2988. Thus, entries
are provided in files in /etc:

/etc/services:
afbackup  2988/tcp

/etc/inetd:
afbackup stream tcp nowait ...

For a second server, you may add appropriate lines, e.g.:

/etc/services:
afbackup2 2989/tcp

/etc/inetd.conf:
afbackup2 stream tcp nowait ...

Note, that the paths to the configuration files later in the inetd.conf-
lines must be adapted to each installation, respectively. To get the
services active, send a Hangup-Signal to the inetd.
(ps ..., kill -HUP <PID>)

The relations between backup clients and streamer devices on the
server must be unique. Thus the /etc/services on the clients must
contain the appropriate port number for the backup entry, e.g.:

afbackup  2990/tcp

Note, that on the clients the service name must always be "afbackup"
and not "afbackup2" or whatever.

As an alternative, you can supply the individual port number in
the clientside configuration. If you do so, no changes must be
made in any clientside system file, here /etc/services.

Do not use NIS (YP) for maintaining the afbackup-services-entry, i.e.
do not add the entry with "afbackup" above to your NIS-master-services-file.
It is anyway better not to use the files /etc/passwd ... as sources
for your NIS-master-server, but to use a copy of them in a separate
directory (as usually configured on Solaris and other Unixes).


--------------------------------------------------------------------------

5. How to recover from a server crash during backup ?

With some devices there will be the problem, that the end-of-tape mark
is not written on power-down during writing to the tape. Even worse,
when power is up again, the position, where the head is actually placed,
gets corrupt, even if no write access has been applied at power-down.
Some streamers are furthermore unable to start to write at a tape
position, where still records follow, e.g. if there are 5 files on tape,
it is e.g. impossible to go to file 2 and start to write there. An
I/O-error will be reported.

The only way to solve this is to tell the backup system to start to
write at the beginning of the next cartridge. If the next cartridge
has e.g. the label-number 5, log on to the backup server, become root
and type:

  /your/path/to/server/bin/cartis -i 5 1


--------------------------------------------------------------------------

6. How to port to other operating systems ?


* Unix-like systems *

This is not that difficult. The GNU-make is mandatory, but this is
usually no problem. A good way to start is to grep for AIX or sun
over all .c- and .h-files, edit them as needed and run the make.
You might want to run the prosname.sh to find out a specifier for
your operating system. This specifier will be defined as a macro
during compilation (exactly: prepocessing).

An important point is the x_types.h-file. Here the types should be
adapted as described in the comments in this file, lines 28-43.
Insert ifdef-s as needed like for the OSF 1 operating system on alpha
(macros __osf__ and __alpha). Note, that depending on the macro
USE_DEFINE_FOR_X_TYPES the types will be #define-d instead of
typedef-d. This gives you more flexibility, if one of those
possibilities is making problems.

The next point is the behaviour of the C-library concerning the
errno-variable in case the tape comes to it's physical end. In most
cases errno is set to ENOSPC, but not always (e.g. AIX is special).
This can be adapted modifying the definition of the macro
END_OF_TAPE (in budefs.h). This macro is only used in if-s as shown:
  if(END_OF_TAPE) ...
Consult your man-pages for the behaviour of the system calls on
your machine. It might be found under rmt, write or ioctl.

The next is the default name of the tape device. Define the macro
DEFAULT_TAPE_DEVICE (in budefs.h) appropriately for your OS.

A little pathological is the statfs(2) system call. It has a different
number of arguments depending on the system. Consult your man-pages,
how it should be used. statfs is only used in write.c

There may be further patches to be done, but if your system is close
to POSIX this should be easy. The output of the compiler and/or the
linker should give the necessary hints.

Please report porting successes to af@muc.de. Thanks.

Good luck !



* Win-whatever *

This is my point of view:

Porting to Microsoft's Features-and-bugs-accumulations is systematically
made complicated by the Gates-empire. They spend a lot of time on taking
care, that it is as difficult as possible to port to/from Win-whatever.
This is one of their monopolization strategies. Developpers starting to
write programs shall have to make the basic decision: "Am i gonna hack
for Micro$oft's "operating systems", or for the others ?" Watching the
so-called market this decision is quite easy: Of couse they will program
for the "market leader". And as few as possible of what they produce
should be usable on other ("dated") platforms. Companies like Cygnus
are providing cool tools (e. g. a port of the GNU-compiler) to make
things easier but due to the fact, that M$ are not providing so many
internals to the public, in my opinion porting is nonetheless an
annoying job. Thank Bill Gates for his genious strageties.

In short, at the moment i'm not gonna provide information how to port
to Micro$oft-platforms. If a port will be available at all, i'll do it
myself and as normal in the M$-world, i wanna get a lot of money for it.
This is my actual point of view. Maybe i'll be forced to change due to
the increasing censorship upcoming in the computer business, but i pray,
this won't ever be the case.


--------------------------------------------------------------------------

7. How to provide recovery from hard crashes (disk crash, ...) ?

A key to this is the clientside StartupInfoProgram parameter. This
command should read the standard input and write it to some place
outside of the local machine - to be more precise - not to a disk
undergoing backups or containing the clientside backup log files.
The information written to the standard input of this program is
the minimum information required to restore everything after a
complete loss of the saved filesystems and the client side of the
backup system. Recovery can be achieved using the restore-utility
with the -e flag (See: PROGRAMS) and supplying the minimum recovery
information to the standard input of restore. Several options exist:

- Write this information to a mail-program (assumed the mail folders
  are outside of the filesystems undergoing backup) and sending this
  information to a backup-user. Later the mailfile can be piped into
  the restore-utility (mail-related protocol lines and other unneeded
  stuff will be ignored). For each machine, that is a backup client,
  an individual mail user should be configured, cause the minimum
  restore information does NOT contain the hostname (to be able to
  restore to a different machine, what might make perfect sense in
  some situations)

- Write the information into a file (of course: always append),
  that resides on an NFS-mounted filesystem, eventually for security
  reasons exported especially to this machine only. To be even more
  secure, the exported directory might be owned by a non-root-user,
  who is the only one, who may write to this directory. This way it
  can be avoided to export a directory with root-access. Then the
  StartupInfoProgram should be something like:
   su myuser -c "touch /path/to/mininfo; cat >> /path/to/mininfo"
  The mininfo-file should have a name, that allows to deduce the
  name of the backup-client, that wrote it. E.g. simply use the
  hostname for this file.

- Write the information to a file on floppy disk. Then the floppy
  disk must always be in the drive, whenever a backup runs. The
  floppy could be mounted using the amd automounter as explained in
  ftp://ftp.zn.ruhr-uni-bochum.de/pub/linux/README.amd.floppy.cdrom
  or using the mtools usually installed for convenience. In the
  former case the command should contain a final sync. In the
  latter case the file must be first copied from floppy, then
  appended the information, finally copied back to floppy e.g. like
  this:
   mcopy -n a:mininfo /tmp/mininfo.$$; touch /tmp/mininfo.$$; \
       cat >> /tmp/mininfo.$$; mcopy -o /tmp/mininfo.$$ a:mininfo; \
       /bin/rm -f /tmp/mininfo.$$; exit 0
  Note, that the whole command must be entered in one line using
  the (x)afclientconfig command. In the configuration file lineend
  escaping is allowed, but not recognized by (x)afclientconfig. An
  alternative is to put everything into one script, that is started
  as StartupInfoProgram (Don't forget to provide a good exit code
  on successful completion)

My personal favourite is the second option, but individual preferences
or requirements might lead to different solutions. There are more
options here. If someone thinks, i have forgotten an important one,
feel free to email me about it.

It might be a good idea to compile afbackup linked statically with
all required libraries (building afbackup e.g. using the command
make EXTRA_LD_FLAGS=-static when using gcc), install it, run the
configuration program(s), if not yet done, tar everything and
put it to a floppy disk (if enough space is available).

To recover from a heavy system crash perform the following steps:
- Replace bad disk(s) as required
- Boot from floppy or cdrom (the booted kernel must be network-able)
- Add the backup server to /etc/hosts and the following line to
  /etc/services: afbackup 2988/tcp
- Mount your new disk filesystem(s) e.g. in /tmp/a and in a way, that
  this directory reflects your original directory hierarchy below
  / (like usually most system setup tools do)
- Untar your packed and statically linked afbackup-distribution, but
  NOT to the place, where it originally lived (e.g. /tmp/a/usr/backup),
  cause it will be overwritten, if you also saved the clientside
  afbackup-installation, what i strongly recommend.
- Run the restore-command with -e providing the minimum restore
  information saved outside of the machine to stdin:
  /path/to/staticlinked/restore -R /tmp/a -e < /path/to/mininfo-file

Bootsector stuff is NOT restored in this procedure. For Linux
you will have to reinstall lilo, but this is usually no problem.


--------------------------------------------------------------------------

8. How to make differential backups ?

A differential backup means for me: Save all filesystem entries modified
since the previous full backup, not only those modified since the last
incremental backup.

This task can be accomplished using the -a option of the incr_backup
command. It tells incr_backup to keep the timestamp. If -a is omitted
one time, another differential backup is no longer possible since the
timestamp is modified without -a. So if differential backups are required,
you have to do without incremental backups.


--------------------------------------------------------------------------

9. How to use several servers for one client ?

Several storage units can be configured for one client. A storage unit
is a combination of a hostname, a port number and a cartridge set number.
Several servers can be configured on one machine, each operating an own
streamer device or directory for storing the data.

The storage units are configured by the first three parameters of the
client side. These are hostnames, port numbers and cartridge set numbers,
respectively. Several entries can be made for each of these parameters.
The port numbers and/or cartridge set numbers can be omitted or fewer
than hostnames can be supplied, then the defaults will apply. If more
port or cartridge set numbers than hostnames are given, the superfluous
ones are ignored. The lists of hostnames and numbers can be seperated
by whitespace and/or commas.

When a full or incremental backup starts on a client, it tests the
servers, one after the other, whether they are ready to service them.
If none is ready, it waits for a minute and tries again.

With each stored filesystem entry, not only the cartridge number and
file number on tape is stored, but now also the name of the host,
where the entry is stored to, and the appropriate port number. Thus
they can be restored without the necessarity, that the user or adminis-
trator knows, where they are now. This all happens transparently and
without additional configuration efforts. For older backups, the first
entry of each list (hostname and port) is used. Therefore, in case of
an upgrade, the first entries MUST be those, that applied for this
before the upgrade.

If there are several clients, the same order of server entries should
not be configured for all of them. This would probably cause most of
the backups to go to the first server, while the other(s) are not
explioted. The entries should be made in a way, that a good balancing
of storage load is achieved. Other considerations are:

- Can the backup be made to a server in the same subnet, where the
  client is
- Has this software been upgraded ? Then the first entry should be
  the same server as configured before (see above)
- The data volume on the clients to be saved (should be balanced)
- The tape capacity of the servers
- other considerations ...


--------------------------------------------------------------------------


D.        Do-s and Dont-s

ALWAYS
------

        - configure a Startup-Info-Command on the clientside
          to save the crucial information for the emergency
          restore after a hard crash losing the filename logfiles
          i.e. the filelists
        - write the numbers of the cartridges onto their cases
          or use adhesive labels
        - be happy and forgive me the bugs
	- try to keep things simple. Unnecessarily making things
	  complicated will strike back mercilessly sooner or later,
	  according to Murphy sooner, if not ASAP


        ... to be continued


NEVER
-----

        - kill any afbackup-related programs with -9 (== -KILL)
          It may take a while until the program has cleaned up
          but it does clean up and it will terminate. If it
          takes longer than 10 minutes, then you might THINK of
          kill -9. But first watch the processor time consumed
          by the process !
        - use fewer cartridges than with a capacity of at least
          two times the size required for one full backup including
          subsequent incremental backups. Otherwise stored files
          could be lost due to an unsuccessful backup overwriting
          previously stored data.

        ... to be continued


--------------------------------------------------------------------------


F.              AF's Backup FAQ
                ===============

Index
-----

Q1: How do I tell how much space is left on the tape?

Q2: Why is the mtime used for deciding what files to save during incremental
    backup and not the ctime or both ?

Q3: Do my current configurations get overwritten during an upgrade ?

Q4: Why should I and how do I use sets of cartridges ?

Q5: How many cartridges should I use ?

Q6: I have a robot with n cartridges. Can i use more than n tapes ?

Q7: Can ordinary users restore their own files and directories ?

Q8: Why does afbackup not have a GUI ?

Q9: What does the warning mean: "Filelist without user-ID information ..." ?

Q10: The whole backup systems hangs in the middle of a backup, what's up ?

Q11: Tape reels back and forth, mail sent "tape not ready ...", what's up ?

Q12: The server seems to have a wrong tape file count, what's wrong ?

Q13: When using crond, the client seems not to start correctly ... ?

Q14: What does AF mean ?

Q15: Though client backup works, remote start does not. Why ?

Q16: My server does not work, tape operates, but nothing seems to be written ?

Q17: I have a ADIC 1200G autoloading DAT and no docs. Can i use it with HPUX ?

Q18: What is a storage unit and how and why should i use it ?


Questions and Answers
=====================

Q1: How do I tell how much space is left on the tape?

A1: This is hard to tell due to the problem to determine exactly, how many
    bytes can be written to a certain physical tape. To get an estimate, how
    many bytes are already on tape, ask the server for the actual writing
    position from any client site:

    /path/to/client/bin/afclient -h <your_backup_server> -Q

    This will tell you also the file to that will be written next time.
    This number times your configured Max Bytes Per File (in the serverside
    configuration file) gives you an upper limit of the written bytes. It
    is an upper limit, cause not all files on tape have the full length.
    To get one more *estimate*, how many bytes can be written to a tape,
    you might search the logfiles for the highest occuring file number.
    Go to your (clientside) /path/to/client/var and look at your
    backup_log.XXX or backup_log.XXX.z -file. Type more (or zmore,
    respectively) and scan the entries. They are of the form
     CC.FF: /files/and/dirs
    where CC is the cartridge, where this file is stored and FF is the
    file number on tape. Look for the highest FF (Assumed of course, that
    you have already several tapes written and at least one full tape).
    This number times your configured Max Bytes Per File results in the
    *estimate* for the number of bytes, that fit on tape.


Q2: Why is the mtime used for deciding what files to save during incremental
    backup and not the ctime ?

A2: First: The ctime changes any time a chmod, a chown, or other operations
      modifying the inode are performed. A change like this is not worth
      selecting this file for backup, cause the file itself did not change.
    Second: After backup of a file, afbackup restores the atime, because
      i found the atime a quite worth information. A restore of the atime
      changes the ctime, no way around this. If the ctime was evaluated
      for choosing the files for incremental backup, a file stored once
      would be saved again all the following backups, cause at any
      backup time the ctime changes. Incremental backup would be senseless,
      cause all files would be saved all the time a backup runs.


Q3: Do my current configurations get overwritten during an upgrade ?

A3: No. Nothing gets overwritten or lost. Newly introduced parameters have
    the old non-configurable behaviour as default. The defaults are applied,
    if the appropriate parameters are not given explicitely in the
    configuration files.


Q4: Why should I and how do I use sets of cartridges ?

A4: The question, why, is not that easy to answer. Maybe you have groups
      of hosts you would like to save to distinguished cartridges, maybe
      you would like to make the full backups to other cartridges than the
      incremental backup. Maybe you have the requirement, that you want to
      use an infinite number of cartridges for the full backup and reuse
      the ones for the incremental backup each time another full backup
      has finished. Maybe you have more exotic requirements ...
    The answer to the how is easy: Set the serverside parameter
      LastCartridges supplying the last cartridge of each set. E.g. if
      your cartridge sets should be 1-3, 4-8 and 9-10, set the parameter
      to 3 8 10. A set may be a single cartridge, but i do not recommend
      this, cause writing to the beginning of that cartridge destroys the
      rest stored on it and making these data unaccessible. The last
      number is usually the number of cartridges you have, but not
      necessarily. Cartridges at the upper end of the numbers might be
      omitted. If the last number is not equal to the number of cartridges,
      this number is NOT automatically added. The many numbers are given
      with this parameter, the many cartridge sets you have. The default,
      if this parameter is not present, is one cartridge set with all
      available cartridges.


Q5: How many cartridges should I use ?

A5: The cartridges should be have enough capacity for at least two times
    a full backup including subsequent incremental backups. Otherwise
    files could get lost due to an unsuccessful backup overwriting
    previously stored data.


Q6: I have a robot with n cartridges. Can i use more than n tapes ?

A6: Yes, you can use any number of tapes, if your robot is in the
    sequential mode. Simply fake a higher number to the backup system
    in the serverside configuration file. The only point is, that you
    have to change the cartridges in the robot manually in time. If
    you have e.g. a robot with 10 cartridges and would like to use 20,
    then you have to watch, when it is time to insert other cartridges
    to the appropriate positions. E.g. when cartridge number 8 is in
    the drive, take out cartridges 1-7 and insert number 10-16 into
    the appropriate slot. Later, when they are in use, you can replace
    8-10 by 17-19 and so on.
     When you want to do a restore, the restore-program tells you,
    where it wants to read from like this:

    Going to restore from cartridge X, file Y ...

    Insert manually the right cartridges into slots, the robot will
    access next time. The system will automatically recognize by the
    label on the tape, that it has found the right cartridge now. A
    warning is written to the serverside log telling, that another
    cartridge was found than expected, but this is just a warning and
    we know, how this happened ...


Q7: Can ordinary users restore their own files and directories ?

A7: Yes, they can, but this feature must be enabled. The restore-
    utility must be installed executable for all users and setuid
    root. This can be achieved entering as administrator root:

    rm -f $BASEDIR/client/bin/afrestore
    cp $BASEDIR/client/bin/full_backup $BASEDIR/client/bin/afrestore
    chmod 4755 $BASEDIR/client/bin/afrestore

    Thus ordinary users can run this program. Built-in safety checks
    provide, that they can only restore or list their own files and
    directories. Changing the restore-directory using the option -R
    allows them only to restore to directories owned by themselves.


Q8: Why does afbackup not have a GUI ?

A8: My ideal imagination of a backup system is, that i do not have
    to care about it at all, once it is installed and configured
    properly. It should do it's job in the background, only noticing
    me, if something goes wrong. Thus i would not want any icons,
    clocks or meters pop up on my workspace plugging me up with
    unnecessary and unimportant stuff. The installation procedure
    is simple enough and would not get better having a graphical
    frontend. My opinion.


Q9: What does the warning mean: "Filelist without user-ID information ..." ?

A9: You are running restore with the list-option as non-root and
    the filelists are of the pre-version-2.9-format. Thus they do
    not contain the user-ID of the owners of the files. The program
    does not know, whether it is permittable to show you the names
    of the files. For security reasons it is hardcoded not to show
    them. With the new format containing the user-IDs, you will see
    the matching names of the files owned by you.


Q10: The whole backup systems hangs in the middle of a backup, what's up ?

A10: This phenomenon has been reported only on Linux, Kernel Version
     2.0.30 and seems to be the result of a bug in this kernel. I
     never experienced this problem on my 2.0.29 Kernel or on other
     platforms.
      Rumors told me, that the 2.0.30 kicks out about every 10000th
     to 20000th Process, i.e. the process is started, appears in the
     process list, but does not do anything and never terminates.
     Thus parent processes wait forever, when this happens. Afbackup
     compresses each saved file separately i.e. starts the
     configured compression program for each file. When the problem
     described above arises, this compression program hangs and
     thus the whole chain up to the server process, that waits for
     requests until eternity.

     Solutions: (i'm aware, these are no real solutions)
      - switch off compression for the saved files or
      - change your Linux Kernel


Q11: Tape reels back and forth, mail sent "tape not ready ...", what's up ?

A11: The current state of investigations is, that this is probably
     a problem of a dirty read-write head. This may sound weird,
     but i'll try to explain.
      I experienced this problem without any warning. One day when
     starting a new backup i watched the tape reeling back and forth,
     later sending an email to the person in charge telling, that
     the device is not ready for use, and requesting to correct this
     problem. Compiling everything with debugging turned on i caught
     the server process during the initiliazation phase and found
     the Set-File-Command (mt -f ... fsf ...) failing. Then i found
     out, that there were fewer files on tape than the backup system
     expected (1 too few) and thus the mt failed. I had no idea, how
     this could happen. I corrected everything manually by decrementing
     the writing position on tape. The next backup, that i started,
     worked fine and another one immediately following, too. A verify
     also succeeded. So i took out the tape and decided to ignore the
     fault for the moment. Before i ran the next backup, i started a
     verify to see, what has changed within the meantime, but now
     again: tape reels back and forth endlessly. Looking onto the
     tape manually using mt and dd once again: too few files on tape.
     Seemed like some file (not the last one on tape !) was lost.
     Strange. The only thing i could imagine causing all the trouble
     was an error of the tape drive, e.g. a dirty read-write-head.
     So i put in the next tape and started all over at the beginning
     of the new tape. Everything worked perfectly from now on. The
     phenomenon has been reported to me only on Linux with HP-DAT-
     streamers. This could mean a correlation and/or a problem of a
     Linux driver, but the reported number of 2 is in my opinion too
     small for a conclusion like this. Furthermore i guess, the com-
     bination Linux + HP-DAT-drive is very common, so the probability,
     that problems might arise in such an environment is quite high
     simply due to the number of installations of this kind.
     Admittedly i had been too lazy for a notable while to use any
     cleaning cartridge, so i guess this had been the problem.

     Solution (to get out of the temporary inconvenience):
      - Use a new cartridge and tell the backup system to use it with
        the serverside command
                    /path/to/cartis -i <nexttape> 1
        where <nexttape> is the number of the next cartridge

     Conclusion:
      - Feel urged to use cleaning cartridges regularly


Q12: The server seems to have a wrong tape file count, what's wrong ?

A12: Probably you experienced the following: The last filename in
     the filename log is preceded by a different pair of cartridge
     number / tape file number than the pair named in the report
     email, written to the tape-position file on the server or
     queried with the client-program option -Q.
      This is perfectly possible. The last saved file can make the
     tape file exceed the configured maximum length. Then one or
     more further tape files are opened appropriately.


Q13: When using crond, the client seems not to start correctly ... ?

A13: You probably get the message "Connection to client lost ..."
     in the clientside logfile. This is a weird problem i only
     experienced on IRIX. The program gets a SIGPIPE and i have no
     clue, why. You might start full_backup or incr_backup with the
     option -d, which causes the program to detach from the terminal
     and to wait for 30 Seconds before continuing. Maybe this solves
     your problem.


Q14: What does AF mean ?

A14: Another F......


Q15: Though client backup works, remote start does not. Why ?

A15: The problem is in most cases, that during the remote start
     the configured (un)compression programs, usually gzip and the
     corresponding gunzip, are not found in the search path. Cause
     the remotely started backup is some child of the inetd, it
     gets of course the inetd's command search path. If this does
     not contain the path to gzip, the start fails.


Q16: My server does not work, tape operates, but nothing seems to be written ?

A16: There seems to be a problem on some platforms. Try to start the
     server with the -b option: Edit /etc/inetd.conf and add -b before
     the last argument of afserver in the line starting with afbackup.
     Then send a hangup signal to the inetd (ps ... |grep inetd -> PID,
     kill -HUP <PID>). Then try again. If it works, be happy, but be
     aware, that the performance is reduced in this mode. This problem
     is worked on.


Q17: I have a ADIC 1200G autoloading DAT and no docs. Can I use it with HPUX ?

A17: Thanks to Gian-Piero Puccioni (gip@ino.it) you can. You will find
     the mtx.c program he wrote helpful. Check out this file how to
     build and use his mtx command. It enables you to load/unload/handle
     certain cartridges.


Q18: What is a storage unit and how and why should i use it ?

A18: See in the HOWTO, Q9