Subject: Star-1.5 source has been released
Please have a look at the German open Source Center BerliOS at www.berlios.de
BerliOS will continue to support free hosting of cryptography projects even
when US laws change and don't allow to host cryptography projects in the USA.
Also look at sourcewell.berlios.de, the first Open Source announcement service
that itself is implemented as Open Source project.
Star is the fastest tar archiver for UNIX
Star has many improvements compared to other tar
implementations (including gnu tar). See below for a short description
of the highlight of star.
Star is located on:
Changes since star-1.5.a14:
- Removed option -I in preparation for using it in a different way
in future releases.
- Trying to make the source get accepted by 'cstyle'.
- changed conf/cc-config.sh to check for a bad PATH set up
(e.g. /usr/ucb before /opt/SUNWspro/bin) which would cause
probems to use cc on Solaris.
- Second fix from Andreas Gr�nbacher for #default comments
in ACLs now also works correctly with multiple #default comments.
- Warning: Environment STAR_FIFO_SIZE changed to STAR_FIFOSIZE
This is to make star and cdrecord behave similar.
- Star now reads a file /etc/default/star for default parameters.
Currently implemented are:
STAR_FIFOSIZE= sets the FIFO size if no fs= option has been specified
STAR_FIFOSIZE_MAX= limits the FIFO size even if a fs= option has been specified
- Try to avoid that the verbose or diagnostic messages are
sometimes lost if called on Linux via "ssh". Unfortunately,
this does not always help. If you like to make sure that
nothing gets lost, call: ssh host "star .... ; sleep 10"
NOTE: The basic problem is caused ba a Linux bug
- Star now opens a file that is going to be extracted with O_DSYNC.
Thanks to a hint from Frank Batschulat from the Solaris filesystem
On operating systems with a decent buffer cache implementation
(like Solaris), this speeds up extraction of many small or medium
sizes files (e.g. a Linux kernel TAR archive) by approx. 20%
On Linux, this change unfortunately is ineffective.
- New option -no-fsync to avoid calling fsync(2) for each file
if in extract mode.
This helps to increase extraction speed on operating systems
with badly implemented buffer cache (e.g. Linux).
Note that using this option may be dangerous as star will not
even be able to detect if a file could not be extracted correctly
if it has been called with -no-fsync
Calling fsync(2) in star's extract mode has been introduced in 1999.
As star is developed on Solaris and on Solaris the performance degradation
that is caused by calling fsync(2) is close to imperceptible, no
performance tests have been made for this feature until recently.
A few weeks ago, some people reported that GNU tar is much faster
than star when extracting files.
Doing performance tests by extracting linux kernel TAR archives
leads to the following results:
- Extration on Solaris without calling fsync(2) did speed up
by ~ 40% compared to the standard case with calling fsync(2).
- After O_DSYNC has been introduced, the speed difference
(compared to extraction without fsync) reduced to ~ 20%
Extraction of a recent 2.4 Linux developer kernel archive
took 50 seconds with star's default that calls fsync and
42 seconds with star -no-fsync (56 seconds with star-1.5a14)
- On Linux (depending on filesystem type and disk type)
extraction without calling fsync(2) is 3x to 8x faster
than with the standard case that includes calling fsync.
Star is develped for best possible data integrity while GNU tar
does not seem to care about data integrity. The fact that GNU tar
on Linux is much faster (for extraction) than star is no reason
to blame star but a reason to blame the quality of the Linux
buffer cache. Star will not change its default to a method that
gives less data integrity just to speed up compared to GNU tar.
The Solaris kernel is developed for best speed with guaranteed
data integrity while the Linux kernel is developed for best speed
without caring for data integrity. If you like that star will
speed up in the star default (secure) mode, send a message to the
Linux kernel developers team and request a change that gives a
better behaviour of the Linux buffer cache.
- New option level= # to set the current incremental dump level
- New option tardumps=name to set file name for tar dump dates
file. This file is similar to the file /etc/dumpdates that
is manages by ufsdump.
- New option -wtardumps to tell star update file for tar dump
dates if in dump mode.
If you like to experiment with the new incremental dump features,
you should start this way:
Create a level 0 dump:
star -cM -dump -sparse level=0 -wtardumps -C /usr .
Create a level 1 dump:
star -cM -dump -sparse level=1 -wtardumps -C /usr .
If you do not have files that changed the type, no files that
have been removed and no files that have been renamed,
star would already be able to do incremental restores.
- Better handling for the exitcode for star (needed for dump
- Allow 'smt' to understand that EACCES for read-only open may
not be caused by a write protected tape but rather by access
problems for a remote tape drive.
- Find a workaround for permission problems when extracting
extended file flags (e.g. on BSD).
Revision history (short)
1982 First version on UNOS (extract only)
1985 Port to UNIX (fully functional version)
1985 Added pre Posix method of handling special files/devices
1986 First experiments with fifo as external process.
1993 Remote tape access
1993 diff option
1994 Fifo with shared memory integrated into star
1994 Very long filenames and sparse files
1994 Gnutar and Ustar(Posix.1-1988) handling added
1994 Xstar format (extended Posix.1-1988) defined and introduced
1995 Ported to many platforms
1999 Support for Win32 (Cygwin)
1999 base 256 support to overcome limitation with octal fields
2001 Large file support
2001 Support for POSIX.1-2001 extended headers
2001 Support for ACLs in POSIX.1-2001 extended headers
2002 Support for extended file flags in POSIX.1-2001 extended headers
2002 Support for extended inode meta data and meta files
2003 Error control added to allow to selectively ignore error contitions
Virtually any! Known to work:
SunOS 4.x, Solaris (SunOS 5.x), Linux,
HP-UX, DG/UX, IRIX, AIX, FreeBSD, NetBSD, OpenBSD,
OSF-1, True-64, QNX, BeOS, Apollo Domain/OS, Next-STep,
OS/2, SCO-openserver, SCO-unixware, Caldera OpenUnix,
Cygwin (Win32), Sony NewsOS, BSDi (BSD/OS), QNX,
Apple Rhapsody, MacOS X, AmigaOS, GNU Hurd
Star is the fastest known implementation of a tar archiver.
Star is able to make backups with far more than 10MB/s if the
disk and tape drive support such a speed. This is more than
double the speed that ufsdump will get.
In 1996, Ampex got 13.5 MB/s with their DLT tape drive which is a
modified professional digital video tape drive and not related
to Quantum's DLT type drives.
Ufsdump got a maximum speed of about 6MB/s with the same hardware.
Star development started 1982, the first complete implementation has
been done in 1985. I never did my backups with other tools than star.
Its main advantages over other tar implementations are:
fifo - keeps the tape streaming.
This gives you faster backups than
you can achieve with ufsdump, if the
size of the filesystem is > 1 GByte.
remote tape support - a fast RMT implementation that has no
probems to saturate a 100 Mb/s network.
accurate sparse files - star is able to reproduce holes in sparse
files accurately if the OS includes
the needed support functions. This is
currently true for Solaris-2.3 to
pattern matcher - for a convenient user interface
(see manual page for more details).
To archive/extract a subset of files.
sophisticated diff - user tailorable interface for comparing
tar archives against file trees
This is one of the most interesting parts
of the star implementation.
no namelen limitation - Pathnames up to 1024 Bytes may be archived.
(The same limitation applies to linknames)
This limit may be expanded in future
without changing the method to record
deals with all 3 times - stores/restores all 3 times of a file
(even creation time)
With POSIX.1-2001 the times are in nanosecond
Star may reset access time after doing
backup. On Solaris this can be done without
changing the ctime.
does not clobber files - more recent copies on disk will not be
clobbered from tape
This may be the main advantage over other
tar implementations. This allows
automatically repairing of corruptions
after a crash & fsck (Check for differences
after doing this with the diff option).
automatic byte swap - star automatically detects swapped archives
and transparently reads them the right way
automatic format detect - star automatically detects several common
archive formats and adopts to them.
Supported archive types are:
Old tar, gnu tar, ansi tar, star,
POSIX.1-2001 PAX, Sun's Solaris tar.
automatic compression detect - star automatically detects whether the
archive is compressed. If it has been
compressed with a compression program that
is compatible to decompression with "gzip"
or "bzip2", star automatically activates
fully ansi compatible - Star is fully ANSI/Posix 1003.1 compatible.
See README.otherbugs for a complete
description of bugs found in other tar
implementations. Star is the first tar
implementation that supports POSIX.1-2001.
support for ACLs and file flags - star supports Access Control Lists
and extended file flags (as found on FreeBSD
and Linux). Support to archive and restore
other file properties may be easily added.
support for all inode metadata - star supports to put all inode
metadata on the archive. This allows future
versions of star to perform true
Have a look at the manual page, it is included in the distribution.
Email: email@example.com, firstname.lastname@example.org
Please mail bugs and suggestions to me.