LIBDAR
|
PresentationThe Libdar library provides a complete abstraction layer for handling Dar archives. The general operations provided are:
Note that Disk ARchive and
libdar
have been released under the Gnu
General Public License (GPL). All code linked to libdar
(statically or dynamically), must
also be covered by the GPL. Commercial use is prohibited unless
a contract has been agreed with libdar's author.
This tutorial will show you how
to
use the libdar API. Since release 2.0.0 the dar command-line executable also
relies on this API, looking at it's code may provide a good
illustration on the way to use libdar, the file src/dar_suite/dar.cpp
is the primary consumer of the libdar API.
The sample codes provided here is solely illustrative and is not guaranteed to compile. More detailed API documentation is contained in the source code and can be compiled to the doc/html directory using Doxygen, which is also provided online and is referred below as the API reference documentation. |
Let's StartConventionsLanguageDar and libdar are written in
C++, and so is the libdar API, for other languages check for the
existing bindings, like for example the python bindings.
Header files Only one include file is required in your program to have access to libdar: #include <dar/libdar.hpp>
Libdar namespaceAll libdar symbols are defined under the libdar namespace. You can either add the using namespace libdar; line at the beginning of your source files: |
using
namespace libdar; get_version(); |
or, as shown below, you can explicitly use the namespace in front of libdar symbols, we will use this notation in the following: |
libdar::get_version(); |
Exceptions
|
try { // calls to libdar ... // } catch(libdar::Ebug & e) { std::string msg = e.dump_str();
// do something with msg like for example: std::cerr << msg } catch(libdar::Egeneric & e) { std::string msg = e.get_message();
// do something with msg like for example std::cerr << msg } |
1 - First we must initialize libdar
|
libdar::get_version();
|
2 - We should prepare the end right now
|
libdar::close_and_clean()
|
Note: closes_and_clean()
makes the necessary for memory to be released in the proper order. Not
calling close_and_clean() at the end of your program may result in
uncaught exception message from libdar at the end of
the execution. This depends on the compiler, libc and option
activated in libdar at compilation time.
All in one, at the highest level, you code should look like the following |
try { // calls to libdar ... ... } catch(libdar::Ebug & e) { std::string msg = e.dump_str(); // do something with msg like for example: std::cerr << msg } catch(libdar::Egeneric
& e) { std::string msg = e.get_message(); // do something with msg like for example std::cerr << msg } libdar::close_and_clean();
|
3 - Intercepting signalslibdar by itself does not make
use of any signal (see signal(2) and
kill(2)). However, gpgme library with which libdar may be linked with
in order to
support asymmetrical strong encryption (i.e. encryption using
public/private keys) may trigger the PIPE signal. Your application
shall thus
either ignore it (signal(SIGPIPE, SIG_IGN)) or provide an adhoc handle.
By default the PIPE signal leads the receiving process to terminate.
|
4 - Libdar classes
The main components of libdar are four classes:
|
libdar::archive
my_first_backup(nullptr,
"/home/me",
"/tmp",
"first_backup",
"dar",
archive_options_create(),
nullptr); |
Thus, the
previous command will create a single sliced archive "first_backup.1.dar" located under /tmp. It will contain the content of
the directory /home/me
and its sub-directories, without compression and without ciphering. You
have guessed compression, slicing, ciphering can be set by playing with
passing an adhoc archive_option_create
object to this archive constructor, we will see that later. Once the object has been created there is only little thing we can do with it, like archive listing or archive isolation. But archive extraction, testing or diffing needs creating a object with a "read" constructor first. We could also have allocated the archive on the heap, in that case we would have just added the delete operation after the construction has ended: |
libdar::archive*
my_first_backup = new
libdar::archive(nullptr,
"/home/me",
"/tmp",
"first_backup",
"dar",
archive_options_create(),
nullptr); |
6.2 - Progressive reportDuring the operation we get
nothing shown unless an error occurs. To have more visibility on the
process we will use an libdar::statistics
object passed as last argument of this constructor. The useful method
of class libdar::statistics are:
|
libdar::statistics stats;
"/home/me",
"/tmp",
"first_backup",
"dar",
archive_options_create(),
& stats); |
6.3 - Archive creation options
|
libdar::archive_options_create opt; // providing an empty secu_string leads dar
interactively ask the passphrase in a secure manner opt.set_crypto_pass(secu_string());
"/home/me",
"/tmp",
"first_backup",
"dar",
opt,
nullptr); |
Of course, you can use both libdar::statistics and libdar::archive_options_create at
the same time.7 - Creating a differential or incremental backupMaybe you have guessed?
Compared to the previous operation (full backup) doing an differential
or incremental backup will only ask to open in read-mode an existing
archive and pass this object as argument of class
archive_options_create::set_reference() seen just above.
The read-only constructor for class archive is the following:
same as before:
|
// first we open the
previously created archive in read mode: libdar::archive_options_create opt;
"/home/me",
"/tmp",
"diff_backup",
"dar",
opt,
nullptr); |
creating a
incremental backup is exactly the same, the difference is the nature of
the archive of reference. We used to describe a differential backup
one that has a full backup as reference, while an incremental backup
has another incremental or differential backup as reference (not a full
backup).8 - Archive listingArchive listing operation consist of the creation of an archive object in read-mode as we just did above and invoking a method on that newly object to see all or a sub-directory content of the archive. Before looking at the listing method let's zoom on the class libdar::archive_create_options_read we just skip over previously.8.1 - Archive reading optionsThe same as the class archive_options_create detailed above, the class archive_options_read has a constructor without argument that sets the different options to their default value. You can change them one by one by mean of specific methods. The most usual ones are:
8.2 - Listing methodsThere is several way to read an given archive contents:
archive::get_children_of() use the same callback but only for the different entries of a given directory, that has to exist in the archive of course. It returns false when the end of the directory has been reached. archive::get_children_in_table() is like the previous listing a given directory but returns a vector of objects libdar::list_entry that provide detailed information about each entry, no callback is used here. For the two first methods you have to define a callback function of the following form void (*)(const std::string & the_path, This callback will receive as argument the full path of the object, a libdar::list_entry object providing much details on it and the "context" value passed as argument of archive::op_listing() or archive::get_children_of() Last point to see before going forward with an example is this libdar::list_entry class, we will use here only a few of the rich set of fields/methods this class provides in the following examples: |
// we first create a
read-mode archive object that will be used in the three following
examples "/home/me",
|
8.3 - archive listing using archive::op_listing() |
// first
possibility: we can pass nullptr as callback function to
archive::op_listing, all will be displayed in stdout |
// second
possibility: we use the callback defined above |
// in complement of
both previous variant we can of course set non default listing options |
8.4 - archive listing using archive::get_children_of() |
// With this method
we only list one directory my_backup.get_children_of(my_listing_callback, .gnupg/private-keys-v1.d");
|
8.5 - archive listing using archive::get_children_in_table() |
// still
listing a single directory but this time without callback function: " .gnupg/private-keys-v1.d"); if(it->is_dir())
|
9 - Testing an archive
|
// for the
exercise, we will change some default options: archive_options_test opt; opt.set_info_details(true);
// to have a verbose output
libdar::statistics stats; stats = my_backup.op_test(nullptr,
// still the user_interaction we will see further
opt;
// the non default options set above
nullptr); // we will just use the returned value
std::cout << stats.get_treated_str() << "
file(s) tested" << std::endl; std::cout << stats.get_errored_str()
<< " file(s) with errors" << std::endl; std::cout << stats.get_ea_treated_str()
<< " Extended Attributes tested" << std::endl;
|
10 - Comparing an archive
|
// for the
exercise, we will change some default options: archive_options_diff opt; opt.set_info_details(true);
// to have a verbose output
(void) my_backup.op_diff("/home/me",
opt;
// the non default options set above
nullptr); // not using it for this example
|
11 - Isolating an archive
|
// for the
exercise, we will change some default options: archive_options_isolate opt; opt.set_warn_over(false);
my_backup.op_isolate("/tmp",
opt); // the non default
options set above
// have you noted? There is no libdar statistics field returned nor as argument.
|
12 - Restoring files from an archive
|
// as we still do not have seen masks, we will restore all files contained in the backup
|
13 - Merging archives
|
// assuming you have two backups:
libdar::archive_options_read opt; "/tmp",
|
14 - Decremental backup
|
//
[...] |
15 - Archive repairing
|
// assuming the archive /tmp/home_backup.*.dar is damaged |
16 - Looking at some details
we have covered the different operations the class libdar::archive can be used for, still remains some concepts to details: |
U_I major, medium, minor; if(maj !=
libdar::LIBDAR_COMPILE_TIME_MAJOR ||
|
17 - checking compile-time features activation
once we have called one of the get_version*
function it is possible to access the list of features activated at
compilation
time thanks to a set of function located in the compile_time nested namespace inside libdar: |
bool ea =
libdar::compile_time::ea(); ();/ // for details see the compile_time namespace in the API reference documentation } |
18 - User Interactionwe have seen std::shared_pointer on class libdar::user_interaction previously but did not used this feature.18.1 - Defining your own user_interaction class
class libdar::user_interaction
defines the way libdar interact with the user during an operation, like
an archive creation, restoration, testing and so on. Only four types of
interaction are used by libdar:
void message(const std::string & message);
By default an inherited class of libdar::user_interaction called
libdar::shell_interaction is used and implements these four type of exchange by
mean of text terminal:
For a GUI you will probably not want stdin and stdout to be used. For
that you have the possibility to implement your own inherited class
from user_interaction. It should look like the following:
|
class my_user_interaction: public libdar::user_interaction |
18.2 - Relying on the pre-defined user_interaction_callback class
|
void my_message_cb(const std::string & x, void *context) |
You will also find predefined classes like libdar::user_interaction_blind which always says no in name of the user displays nothing and provide empty strings, as well as libdar::shell_interaction_emulator
which given a user_interaction object send to it formatted information
as if it was a shell_interaction object, leading one to emulate libdar
default behavior under any time of "terminal". IMPORTANT: all
libdar::user_interaction inherited classes provided by libdar are not
designed to be manipulated by more than one thread at a time. The use
of std::shared_ptr is only here to let the caller not have to manage
such object and let libdar release it when no more needed or
to let the caller to reuse the same user_interaction object for a
subsequent call to libdar which would not be possible if a
std::unique_ptr was used.
Now if you design your own user_interaction inherited class and provide them mecanism (mutex, ...) that allow them to be used simultaneously by several thread there is no issue to give pass such one object as argument to different libdar object used by different threads. 19 - Masks
|
class libdar::bool_mask |
boolean mask, either always true
or false, it matches either all files or no files at all |
class libdar::simple_mask |
matches as done by the shell on
the command
lines (see "man 7 glob") |
class libdar::regular_mask |
matches regular expressions (see
"man 7 regex") |
class libdar::not_mask |
negation of another mask |
class libdar::et_mask |
makes an *AND* operator between
two or more masks |
class libdar::ou_mask |
makes the *OR* operator
between two or more masks |
class lbdar::simple_path_mask |
matches if it is subdirectory of mask or is a directory
that contains the specified path itself |
class libdar::same_path_mask |
matches if the string is exactly
the
given mask (no wild card expression) |
class
libdar::exclude_dir_mask |
matches if string is the given
string or a sub directory of it |
class libdar::mask_list |
matches a list of files defined
in a given file |
Let's play with some masks : |
// all files will be
elected by this mask libdar::bool_mask m1(true); // m5 will now match
only strings that are selected by both m2 AND m4 libdar::et_mask m5; // Frankly, it's
possible to have masks referring each other! libdar::not_mask m7(m6); |
The idea here is not to create
object manually, but to link their
creation to the action and choices the user makes from the user
interface (Graphical User Interface of your application, for example) Now that you've seen the power of these masks, you should know that in libdar masks are used at several places:
An exception is the archive testing
operation, which has no fs_root
argument (because the
operation is not relative to an existing filesystem), however the subtree argument exist to receive a
mask for comparing the path of file to include or exclude from the
testing
operation. In this case the situation is as if the fs_root was set to the value "<ROOT>". For example, masks
will be compared to "<ROOT>/some/file"
when performing an archive test operation.
instead of using explicit string "<ROOT>" you can use libdar::PSEUDO_ROOT const std::string variable
|
20 - Aborting an OperationIf the POSIX thread support is
available, libdar
will be
built in a
thread-safe manner, thus you may have several thread using libdar calls
at the same time (but on different objects except concerning the
libdar::statistics which can be shared between threads). You may then
wish to interrupt a given thread. But
aborting a thread form the outside (like sending it a KILL signal) will
most of the time let some memory allocated or even worse can lead to
dead-lock
situation, when the killed thread was inside a critical section and had
not
got the opportunity to release a mutex.
For that reason, libdar proposes a set
of calls to abort any processing libdar call which is ran by a given
thread.
|
// next is the
thread ID in which we want to have
lidbar call canceled// here for simplicity we don't describe the way the ID has been obtained // but it could be for example the result of a call to pthread_self() as // defined in <pthread.h> system header file pthread_t thread_id = 161720; // the most simple call is: libdar::cancel_thread(thread_id); // this will make any libdar call in this thread be canceled immediately // but you can use something a bit more interesting: libdar::cancel_thread(thread_id, false); // this second argument is true for immediate cancellation, // of false for a delayed cancellation, in which case libdar aborts the operation // but produces something usable. For example, if you were backing up something // you get a real usable archive which only contains files saved so far, in place // of having a broken archive which misses a catalogue at the end. Note that this // delayed cancellation needs a bit more time to complete, depending on the // size of the archive under process. |
As seen above, cancellation can
be
very simple. What now succeeds when
you ask for a cancellation this way? Well, an exception of type Ethread_cancel
is thrown. All along his path, memory is released and mutex are freed.
Last, the exception appears to the libdar caller. So, you can catch it
to define a specific comportment. And if you don't want to use
exceptions a
special returned code is used.
|
try libdar::archive my_arch(...); |
Some helper routines are
available to
know the cancellation status for a particular thread or to abort a
cancellation process if it has not yet been engaged.
|
pthread_t
tid; // how to know if the thread tid is under cancellation process? if(libdar::cancel_status(tid)) cout << "thread cancellation is under progress for thread : " << tid << endl; else cout << "no thread cancellation is under progress for thread : " << endl; // how to cancel a pending thread cancellation ? if(libdar::cancel_clear(tid)) cout << "pending thread cancellation has been reset, thread " << tid << " has not been canceled" << endl; else cout << "too late, could not avoid thread cancellation for thread "<< tid << endl; |
Last point, back to the Ethread_cancel exception, this class has two methods you may find useful, when you catch it: |
try{ ... some libdar calls } catch(libdar::Ethread_cancel & e) { if(e.immediate_cancel()) cout << "cancel_thread() has been called with "true" as second argument" << endl; else cout << "cancel_thread() has been called with "false" as second argument" << endl; U64 flag = e.get_flag(); ... do something with the flag variable... } // what is this flag stored in this exception? // You must consider that the complete definition of cancel_thread() is the following: // void cancel_thread(pthread_t tid, bool immediate = true, U_64 flag = 0); // thus, any argument given in third is passed to the thrown Ethread_cancel exception, // value which can be retrieved thanks to its get_flag() method. The value given to this // flag is not used by libdar itself, it is a facility for user program to have the possibility // to include additional information about the thread cancellation. // supposing the thread cancellation has been invoked by: libdar::cancel_thread(thread_id, true, 19); // then the flag variable in the catch() statement above would have received // the value 19.
|
21 - Dar_manager APIFor more about dar_manager, please read the man page where are described in
detail the available features. Note that for dar_manager there is not
a "without exception" flavor,
your
program
must be able to handle exceptions, which by the way are the same as the
ones described above.
To get dar_manager features you
need to use the class
database
which is defined in the libdar/database.hpp
header file so you first
need to include that file. Most of the methods of the database class do use options. For
the same reason as previously seen for archive manipulation, these options
are passed thanks to a container class. These container classes for
options used by the database
class are defined in the
libdar/database_options.hpp file. Let's see the
different method of the class database
:
Database object constructionTwo constructor are available. The first creates a brand-new but empty database in memorydatabase(const std::shared_ptr<user_interaction> & dialog); As seen for libdar::archive dialog can be set to a null pointer if the default interaction mode (stdin/stdout/stderr) suits your need. The second constructor opens an existing database from filesystem and stores its contents into memory ready for further use. database(const std::shared_ptr<user_interaction> & dialog,
|
database base; |
Database's methods First we will see methods that can work with a partial and read-only database |
22 - dar_slave APIdar_slave
role is to read an archive while interacting with a dar process through
a pair of pipes. Dar asks portion of the archive or information about
the archive in the first pipe from dar to dar_slave. And dar_slave
sends the requested information into the other pipe toward dar. |
libdar::libdar_slave slave(nullptr,
|
23 - dar_xform APIdar_xform
creates a copy of a given archive modifying its slicing. it does not
require decompressing nor deciphering the archive to do so. There is
different constructor depending whether the archive is read from
filesystem, from a named pipe of from a provided file descriptor |
libdar::libdar_xform transform(nullptr,
|
24 - Compilation & Linking
|
> cat
my_prog.cpp#include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc `pkg-config --cflags libdar` -c my_prog.cpp |
|
> gcc pkg-confg --libs libdar` my_prog.o -o my_prog |
Libdar's different flavorsWell, all the compilation and
linking
steps described above assume you
have a "full" libdar library.
Beside the full (alias infinint) libdar flavor, libdar
also comes in 32 and 64 bits
versions. In these last ones, in place of internally relying on a
special type (which
is a C++ class called infinint)
to
handle arbitrary large integers, libdar32 relies on 32 bits integers
and
libdar64 relies on 64 bits integers (there are limitations which are
described in doc/LIMITATIONS). But all these libdar version (infinint,
32bits, 64bits) have the same
interface and must be used the same way, except for compilation and
linking.
These different libdar
versions can
coexist on the same system, they
share the same include files. But the LIBDAR_MODE macro must be set to
32
or 64 when compiling or linking with libdar32 or libdar64
respectively. The LIBDAR_MODE macro defines the way the "class infinint" type
is
implemented in libdar, and thus changes the way the libdar headers
files are interpreted by the compiler. pkg-config
--cflags will set the
correct LIBDAR_MODE, so you should only bother calling it with either
libdar, libdar32 or libdar64 depending on your need : "pkg-confg
--cflags libdar64" for example.
|
>
cat my_prog.cpp#include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc -c `pkg-config --cflags libdar32` my_prog.cpp
> gcc `pkg-config --libs libdar32` my_prog.o -o my_prog |
and replace 32 by 64 to link
with libdar64. |