1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914
|
SQUASHFS CHANGE LOG
4.6.1 25 MAR 2023 Bug fix release to fix race condition and XATTRs
code
1. Race condition which can cause corruption of the "fragment table"
fixed. This is a regression introduced in August 2022, and it has
been seen when tailend packing is used (-tailends option).
2. Fix build failure when the tools are being built without extended
attribute (XATTRs) support.
3. Fix XATTR error message when an unrecognised prefix is found
(Christian Hesse).
4. Fix incorrect free of pointer when an unrecognised XATTR prefix is
found.
4.6 17 MAR 2023 Major improvements in extended attribute handling,
pseudo file handling, and miscellaneous new options and
improvements
1. Extended attribute handling improved in Mksquashfs and Sqfstar
1.1.New -xattrs-exclude option to exclude extended attributes
from files using a regular expression.
1.2 New -xattrs-include option to include extended attributes
from files using a regular expression.
1.3 New -xattrs-add option to add extended attributes to files.
1.4 New Pseudo file xattr definition to add extended attributes
to files.
1.5 New xattrs-add Action to add extended attributes to files
(Mksquashfs only).
2. Extended attribute handling improved in Unsquashfs
2.1 New -xattrs-exclude option to exclude extended attributes
from files using a regular expression.
2.2 New -xattrs-include option to include extended attributes
from files using a regular expression.
2.3 Extended attributes are now supported in Pseudo file output.
3. Other major improvements
3.1 Unsquashfs can now output Pseudo files to standard out.
3.2 Mksquashfs can now input Pseudo files from standard in.
3.3 Squashfs filesystems can now be converted (different block
size compression etc) without unpacking to an intermediate
filesystem or mounting, by piping the output of Unsquashfs
to Mksquashfs.
3.4 Pseudo files are now supported by Sqfstar.
3.5 "Non-anchored" excludes are now supported by Unsquashfs.
4. Mksquashfs minor improvements
4.1 A new -max-depth option has been added, which limits
the depth Mksquashfs descends when creating the filesystem.
4.2 A new -mem-percent option which allows memory for caches to
be specified as a percentage of physical RAM, rather than
requiring an absolute value.
4.3 A new -percentage option added which rather than generating
the full progress-bar instead outputs a percentage. This
can be used with dialog --gauge etc.
4.4 -mkfs-time, -all-time and -root-time options now take
a human date string, in addition to the seconds since
the epoch of 1970 00:00 UTC. For example "now",
"last week", "Wed Mar 8 05:55:01 GMT 2023" are supported.
4.5 -root-uid, -root-gid, -force-uid and -force-gid options now
take a user/group name in addition to the integer uid/gid.
4.6 A new -mem-default option which displays default memory
usage for caches in Mbytes.
4.7 A new -no-compression option which produces no compression,
and it is a short-cut for -noI, -noD, -noF and -noX.
4.8 A new -pseudo-override option which makes pseudo file uids
and gids override -all-root, -force-uid and -force-gid
options. Normally these options take precedence.
5. Unsquashfs minor improvements
5.1 New -all-time option which sets all file timestamps to
<time>, rather than the time stored in the filesystem
inode. <time> can be an integer indicating seconds since
the epoch (1970-01-01) or a human string value such as
"now", "last week", or "Wed Feb 15 21:02:39 GMT 2023".
5.2 New -full-precision option which uses full precision when
displaying times including seconds. Use with -linfo, -lls,
-lln and -llc options.
5.3 New -match option where Unsquashfs will abort if any
extract file does not match on anything, and can not be
resolved.
5.4 New -percentage option added which rather than generating
the full progress-bar instead outputs a percentage. This
can be used with dialog --gauge etc.
6. Sqfstar minor improvements
6.1 A new -ignore-zeros option added which allows tar files to
be concatenated together and fed to Sqfstar. Normally a
tarfile has two consecutive 512 byte blocks filled with
zeros which means EOF and Sqfstar will stop reading after
the first tar file on encountering them. This option makes
Sqfstar ignore the zero filled blocks.
6.2 A new -mem-percent option which allows memory for caches to
be specified as a percentage of physical RAM, rather than
requiring an absolute value.
6.3 A new -percentage option added which rather than generating
the full progress-bar instead outputs a percentage. This
can be used with dialog --gauge etc.
6.4 -mkfs-time, -all-time and -root-time options now take
a human date string, in addition to the seconds since
the epoch of 1970 00:00 UTC. For example "now",
"last week", "Wed Mar 8 05:55:01 GMT 2023" are supported.
6.5 -root-uid, -root-gid, -force-uid and -force-gid options now
take a user/group name in addition to the integer uid/gid.
6.6 A new -mem-default option which displays default memory
usage for caches in Mbytes.
6.7 A new -no-compression option which produces no compression,
and it is a short-cut for -noI, -noD, -noF and -noX.
6.8 A new -pseudo-override option which makes pseudo file uids
and gids override -all-root, -force-uid and -force-gid
options. Normally these options take precedence.
6.9 Do not abort if ZERO filled blocks indicating end of the
TAR archive are missing.
7. Other minor improvements
7.1 If Mksquashfs/Unsquashfs fails to execute generating the
manpages because they have been cross-compiled, fall back
to using the pre-built manpages.
7.2 Add new Makefile configure option USE_PREBUILT_MANPAGES
to always use pre-built manpages rather than generating
them when "make install" is run.
8. Major bug fixes
8.1 Following a symlink in Sqfscat or where -follow-symlinks
option is given with Unsquashfs, incorrectly triggered the
corrupted filesystem loop detection code.
8.2 In Unsquashfs if a file was not writable it could not add
extended attributes to it.
8.3 Sqfstar would incorrectly reject compressor specific
options that have an argument.
8.4 Sqfstar would incorrectly strip pathname components in PAX
header linkpath if symbolic.
8.5 Sqfstar -root-uid, -root-gid and -root-time options were
documented but not implemented.
8.6 Mksquashfs -one-file-system option would not create empty
mount point directory when filesystem boundary crossed.
8.7 Mksquashfs did not check the close() return result.
4.5.1 17 MAR 2022 New Manpages, Fix CVE-2021-41072 and miscellaneous
improvements and bug fixes
1. Major improvements
1.1 This release adds Manpages for Mksquashfs(1), Unsquashfs(1),
Sqfstar(1) and Sqfscat(1).
1.2 The -help text output from the utilities has been improved
and extended as well (but the Manpages are now more
comprehensive).
1.3 CVE-2021-41072 which is a writing outside of destination
exploit, has been fixed.
2. Minor improvements
2.1 The number of hard-links in the filesystem is now also
displayed by Mksquashfs in the output summary.
2.2 The number of hard-links written by Unsquashfs is now
also displayed in the output summary.
2.3 Unsquashfs will now write to a pre-existing destination
directory, rather than aborting.
2.4 Unsquashfs now allows "." to used as the destination, to
extract to the current directory.
2.5 The Unsquashfs progress bar now tracks empty files and
hardlinks, in addition to data blocks.
2.6 -no-hardlinks option has been implemented for Sqfstar.
2.7 More sanity checking for "corrupted" filesystems, including
checks for multiply linked directories and directory loops.
2.8 Options that may cause filesystems to be unmountable have
been moved into a new "experts" category in the Mksquashfs
help text (and Manpage).
3. Bug fixes
3.1 Maximum cpiostyle filename limited to PATH_MAX. This
prevents attempts to overflow the stack, or cause system
calls to fail with a too long pathname.
3.2 Don't always use "max open file limit" when calculating
length of queues, as a very large file limit can cause
Unsquashfs to abort. Instead use the smaller of max open
file limit and cache size.
3.3 Fix Mksquashfs silently ignoring Pseudo file definitions
when appending.
3.4 Don't abort if no XATTR support has been built in, and
there's XATTRs in the filesystem. This is a regression
introduced in 2019 in Version 4.4.
3.5 Fix duplicate check when the last file block is sparse.
4.5 22 JUL 2021 Major improvements including: Actions, Sqfstar,
tar and cpio style reading of input sources, Sqfscat
1. Mksquashfs improvements
1.1 Mksquashfs now supports "Actions". These are modelled on
"find" and allow compression, fragment packing, file
exclusion and file attributes to be changed.
1.2 New sqfstar command which will create a Squashfs image from
a tar archive.
1.3 Tar style handling of source pathnames in Mksquashfs.
1.4 Cpio style handling of source pathnames in Mksquashfs.
1.5 New option to throttle the amount of CPU and I/O.
1.6 New Pseudo file definitions which support timestamps.
1.7 New Pseudo file definitions to create File references.
1.8 New Pseudo file definitions to create Sockets/Fifos.
1.9 Mksquashfs now allows no source directory to be specified.
1.10 New Pseudo file "R" definition which allows a Regular file
to be created with data stored within the Pseudo file.
2. Major improvements in Unsquashfs
2.1 Sqfscat command which outputs files to stdout.
2.2 Symbolic links are now followed in extract files (using
-follow-symlinks or -missing-symlinks).
2.3 Unsquashfs now supports "exclude" files.
2.4 Max depth traversal option added.
2.5 Unsquashfs can now output a "Pseudo file" representing the
input Squashfs filesystem.
3. Minor improvements and bug fixes
3.1 The progress bar is now displayed and updated whilst the
input is being scanned.
3.2 New -one-file-system option in Mksquashfs.
3.3 New -no-hardlinks option in Mksquashfs.
3.4 New -help options in Mksquashfs and Unsquashfs which output
to standard out.
3.5 New -root-uid option in Mksquashfs.
3.6 New -root-gid option in Mksquashfs.
3.7 New -root-time option in Mksquashfs.
3.8 -no-exit-code option added to Unsquashfs which makes it
not output an error exit code.
3.9 Exit code in Unsquashfs changed to distinguish between
non-fatal errors (exit 2), and fatal errors (exit 1).
3.10 Mksquashfs when appending, now writes the recovery file to
the home directory, rather than the current directory.
3.11 New -recovery-path <name> option.
3.12 Xattr id count added in Unsquashfs "-stat" output.
3.13 Unsquashfs "write outside directory" exploit fixed.
3.14 Error handling in Unsquashfs writer thread fixed.
3.15 Fix failure to truncate destination if appending aborted.
3.16 Prevent Mksquashfs reading the destination file.
4.4 29 AUG 2019 Reproducible builds, new compressors,
CVE fixes, security hardening and new options
for Mksquashfs/Unsquashfs.
1. Overall improvements:
1.1 Mksquashfs now generates reproducible images by default.
1.2 Mkfs time and file timestamps can also be specified.
1.3 Support for the Zstandard (ZSTD) compression algorithm.
1.4 CVE-2015-4645 and CVE-2015-4646 have been fixed.
2. Mksquashfs improvements and major bug fixes:
2.1 Pseudo files now support symbolic links.
2.2 New -mkfs-time option.
2.3 New -all-time option.
2.4 New -root-mode option.
2.5 New -quiet option.
2.6 New -noId option.
2.7 New -offset option.
2.8 Update lz4 wrapper to use new functions introduced
in 1.7.0.
2.9 Bug fix, don't allow "/" pseudo filenames.
2.10 Bug fix, allow quoting of pseudo files, to
better handle filenames with spaces.
2.11 Fix compilation with glibc 2.25+.
3. Unsquashfs improvements and major bug fixes:
3.1 CVE-2015-4645 and CVE-2015-4646 have been fixed.
3.2 Unsquashfs has been further hardened against corrupted
filestems.
3.3 Unsquashfs is now more strict about error handling.
3.4 New -ignore-errors option.
3.5 New -strict-errors option.
3.6 New -lln[umeric] option.
3.7 New -lc option.
3.8 New -llc option.
3.9 New -mkfs-time option.
3.10 New -UTC option.
3.11 New -offset option.
3.12 New -quiet option.
3.13 Update lz4 wrapper to use new functions introduced
in 1.7.0.
3.14 Bug fix, fatal and non-fatal errors now set the exit
code to 1.
3.15 Bug fix, fix time setting for symlinks.
3.16 Bug fix, try to set sticky-bit when running as a
user process.
3.17 Fix compilation with glibc 2.25+.
4.3 12 MAY 2014 New compressor options, new Mksquashfs/Unsquashfs
functionality, duplicate checking optimisations,
stability improvements (option/file parsing,
buffer/memory overflow checks, filesystem hardening
on corrupted filesystems), CVE fixes.
Too many changes to do the traditional custom changelog. But, this
is now unnecessary, so instead list most significant 15% of commits
from git changelog in chronological order.
- unsquashfs: add checks for corrupted data in opendir functions
- unsquashfs: completely empty filesystems incorrectly generate an error
- unsquashfs: fix open file limit
- mksquashfs: Use linked list to store directory entries rather
- mksquashfs: Remove qsort and add a bottom up linked list merge sort
- mksquashfs: optimise lookup_inode2() for dirs
- pseudo: fix handling of modify pseudo files
- pseudo: fix handling of directory pseudo files
- xattr: Fix ERROR() so that it is synchronised with the progress bar
- mksquashfs/sort: Fix INFO() so that it is synced with the progress bar
- mksquashfs: Add -progress to force progress bar when using -info
- error.h: consolidate the various error macros into one header file
- mksquashfs: fix stack overflow in write_fragment_table()
- mksquashfs: move list allocation from off the stack
- unsquashfs: fix oversight in directory permission setting
- mksquashfs: dynamically allocate recovery_file
- mksquashfs: dynamically allocate buffer in subpathname()
- mksquashfs: dynamically allocate buffer in pathname()
- unsquashfs: fix CVE-2012-4024
- unsquashfs: fix CVE-2012-4025
- mksquashfs: fix potential stack overflow in get_component()
- mksquashfs: add parse_number() helper for numeric command line options
- mksquasfs: check return value of fstat() in reader_read_file()
- mksquashfs: dynamically allocate filename in old_add_exclude()
- unsquashfs: dynamically allocate pathname in dir_scan()
- unsquashfs: dynamically allocate pathname in pre_scan()
- sort: dynamically allocate filename in add_sort_list()
- mksquashfs: fix dir_scan() exit if lstat of source directory fails
- pseudo: fix memory leak in read_pseudo_def() if exec_file() fails
- pseudo: dynamically allocate path in dump_pseudo()
- mksquashfs: dynamically allocate path in display_path2()
- mksquashfs: dynamically allocate b_buffer in getbase()
- pseudo: fix potential stack overflow in get_component()
- pseudo: avoid buffer overflow in read_pseudo_def() using sscanf()
- pseudo: dynamically allocate filename in exec_file()
- pseudo: avoid buffer overflow in read_sort_file() using fscanf()
- sort: tighten up sort file parsing
- unsquashfs: fix name under-allocation in process_extract_files()
- unsquashfs: avoid buffer overflow in print_filename() using sprintf()
- Fix some limits in the file parsing routines
- pseudo: Rewrite pseudo file processing
- read_fs: fix small memory leaks in read_filesystem()
- mksquashfs: fix fclose leak in reader_read_file() on I/O error
- mksquashfs: fix frag struct leak in write_file_{process|blocks|frag}
- unsquashfs_xattr: fix memory leak in write_xattr()
- read_xattrs: fix xattr free in get_xattr() in error path
- unsquashfs: add -user-xattrs option to only extract user.xxx xattrs
- unsquashfs: add code to only print "not superuser" error message once
- unsquashfs: check for integer overflow in user input
- mksquashfs: check for integer overflow in user input
- mksquashfs: fix "new" variable leak in dir_scan1()
- read_fs: prevent buffer {over|under}flow in read_block() with
corrupted filesystems
- read_fs: check metadata blocks are expected size in scan_inode_table()
- read_fs: check the root inode block is found in scan_inode_table()
- read_fs: Further harden scan_inode_table() against corrupted
filesystems
- unsquashfs: prevent buffer {over|under}flow in read_block() with
corrupted filesystems
- read_xattrs: harden xattr data reading against corrupted filesystems
- unsquash-[23]: harden frag table reading against corrupted filesystems
- unsquash-4.c: harden uid/gid & frag table reading against corruption
- unsquashfs: harden inode/directory table reading against corruption
- mksquashfs: improve out of space in output filesystem handling
- mksquashfs: flag lseek error in writer as probable out of space
- mksquashfs: flag lseek error in write_destination as probable out of
space
- mksquashfs: print file being squashed when ^\ (SIGQUIT) typed
- mksquashfs: make EXIT_MKSQUASHFS() etc restore via new restore thread
- mksquashfs: fix recursive restore failure check
- info: dump queue and cache status if ^\ hit twice within one second
- mksquashfs: fix rare race condition in "locked fragment" queueing
- lz4: add experimental support for lz4 compression
- lz4: add support for lz4 "high compression"
- lzo_wrapper: new implementation with compression options
- gzip_wrapper: add compression options
- mksquashfs: redo -comp <compressor> parsing
- mksquashfs: display compressor options when -X option isn't recognised
- mksquashfs: add -Xhelp option
- mksquashfs/unsquashfs: fix mtime signedness
- Mksquashfs: optimise duplicate checking when appending
- Mksquashfs: introduce additional per CPU fragment process threads
- Mksquashfs: significantly optimise fragment duplicate checking
- read_fs: scan_inode_table(), fix memory leak on filesystem corruption
- pseudo: add_pseudo(), fix use of freed variable
- mksquashfs/unsquashfs: exclude/extract/pseudo files, fix handling of
leaf name
- mksquashfs: rewrite default queue size so it's based on physical mem
- mksquashfs: add a new -mem <mbytes> option
- mksquashfs: fix limit on the number of dynamic pseudo files
- mksquashfs: make -mem take a normal byte value, optionally with a
K, M or G
4.2 28 FEB 2011 XZ compression, and compression options support
1. Filesystem improvements:
1.1 Added XZ compression
1.2 Added compression options support
2. Miscellaneous improvements/bug fixes
1.1 Add missing NO_XATTR filesystem flag to indicate no-xattrs
option was specified and no xattrs should be stored when
appending.
1.2 Add suppport in Unquashfs -stat option for displaying
NO_XATTR flag.
1.3 Remove checkdata entry from Unsquashfs -stat option if a 4.0
filesystem - checkdata is no longer supported.
1.4 Fix appending bug when appending to an empty filesystem - this
would be incorrectly treated as an error.
1.5 Use glibc sys/xattr.h include rather than using attr/xattr.h
which isn't present by default on some distributions.
1.6 Unsquashfs, fix block calculation error with regular files when
file size is between 2^32-block_size+1 and 2^32-1.
1.7 Unsquashfs, fix sparse file writing when holes are larger than
2^31-1.
1.8 Add external CFLAGS and LDFLAGS support to Makefile, and allow
build options to be specified on command line. Also don't
over-write passed in CFLAGS definition.
4.1 19 SEPT 2010 Major filesystem and tools improvements
1. Filesystem improvements:
1.1 Extended attribute support
1.2 New compression framework
1.3 Support for LZO compression
1.4 Support for LZMA compression (not yet in mainline)
2. Mksquashfs improvements:
1.1 Enhanced pseudo file support
1.2 New options for choosing compression algorithm used
1.3 New options for controlling extended attributes
1.4 Fix misalignment issues with memcpy etc. seen on ARM
1.5 Fix floating point error in progress_bar when max == 0
1.6 Removed use of get_nproc() call unavailable in ulibc
1.7 Reorganised help text
3. Unsquashfs improvements:
1.1 New options for controlling extended attributes
1.2 Fix misalignment issues with memcpy etc. seen on ARM
1.3 Fix floating point error in progress_bar when max == 0
1.4 Removed use of get_nproc() call unavailable in ulibc
4.0 5 APR 2009 Major filesystems improvements
1. Kernel code improvements:
1.1 Fixed little endian layout adopted. All swapping macros
removed, and in-line swapping added for big-endian
architectures.
1.2 Kernel code substantially improved and restructured.
1.3 Kernel code split into separate files along functional lines.
1.4 Vmalloc usage removed, and code changed to use separately
allocated 4K buffers
2. Unsquashfs improvements:
2.1 Support for 4.0 filesystems added.
2.2 Swapping macros rewritten.
2.3 Unsquashfs code restructured and split into separate files.
3. Mksquashfs improvements:
3.1 Swapping macros rewritten. Fixed little-endian layout allows
code to be optimised and only added at compile time for
big endian systems.
3.2 Support for pseudo files added.
3.4 26 AUG 2008 Performance improvements to Unsquashfs, Mksquashfs
and the kernel code. Plus many small bug fixes.
1. Kernel code improvements:
1.1 Internal Squashfs kernel metadata and fragment cache
implementations have been merged and optimised. Spinlocks are
now used, locks are held for smaller periods and wakeups have
been minimised. Small race condition fixed where if two or
more processes tried to read the same cache block
simultaneously they would both read and decompress it. 10-20%+
speed improvement has been seen on tests.
1.2 NFS export code rewritten following VFS changes in
linux-2.6.24.
1.3 New patches for linux-2.6.25, linux-2.6.26, and linux-2.6.27.
Fixed patch for linux-2.6.24.
1.4 Fixed small buffer_head leak in squashfs_read_data when
handling badly corrupted filesystems.
1.5 Fixed bug in get_dir_index_using_offset.
2. Unsquashfs improvements:
2.1 Unsquashfs has been parallelised. Filesystem reading, writing
and decompression is now multi-threaded. Up to 40% speed
improvement seen on tests.
2.2 Unsquashfs now has a progress bar. Use -no-progress to
disable it.
2.3 Fixed small bug where unistd.h wasn't being included on
some distributions, leading to lseek being used rather than
lseek64 - which meant on these distributions Unsquashfs
couldn't unsquash filesystems larger than 4GB.
3. Mksquashfs improvements:
3.1 Removed some small remaining parallelisation bottlenecks.
Depending on source filesystem, up to 10%+ speed improvement.
3.2 Progress bar improved, and moved to separate thread.
3.3 Sparse file handling bug in Mksquashfs 3.3 fixed.
3.4 Two rare appending restore bugs fixed (when ^C hit twice).
3.3 1 NOV 2007 Increase in block size, sparse file support,
Mksquashfs and Unsquashfs extended to use
pattern matching in exclude/extract files, plus
many more improvements and bug fixes.
1. Filesystem improvements:
1.1. Maximum block size has been increased to 1Mbyte, and the
default block size has been increased to 128 Kbytes.
This improves compression.
1.2. Sparse files are now supported. Sparse files are files
which have large areas of unallocated data commonly called
holes. These files are now detected by Squashfs and stored
more efficiently. This improves compression and read
performance for sparse files.
2. Mksquashfs improvements:
2.1. Exclude files have been extended to use wildcard pattern
matching and regular expressions. Support has also been
added for non-anchored excludes, which means it is
now possible to specify excludes which match anywhere
in the filesystem (i.e. leaf files), rather than always
having to specify exclude files starting from the root
directory (anchored excludes).
2.2. Recovery files are now created when appending to existing
Squashfs filesystems. This allows the original filesystem
to be recovered if Mksquashfs aborts unexpectedly
(i.e. power failure).
3. Unsquashfs improvements:
3.1. Multiple extract files can now be specified on the
command line, and the files/directories to be extracted can
now also be given in a file.
3.2. Extract files have been extended to use wildcard pattern
matching and regular expressions.
3.3. Filename printing has been enhanced and Unquashfs can
now display filenames with file attributes
('ls -l' style output).
3.4. A -stat option has been added which displays the filesystem
superblock information.
3.5. Unsquashfs now supports 1.x filesystems.
4. Miscellaneous improvements/bug fixes:
4.1. Squashfs kernel code improved to use SetPageError in
squashfs_readpage() if I/O error occurs.
4.2. Fixed Squashfs kernel code bug preventing file
seeking beyond 2GB.
4.3. Mksquashfs now detects file size changes between
first phase directory scan and second phase filesystem create.
It also deals better with file I/O errors.
3.2-r2 15 JAN 2007 Kernel patch update and progress bar bug fix
1. Kernel patches 2.6.19/2.6.20 have been updated to use
const structures and mutexes rather than older semaphores.
2. Minor SMP bug fixes.
3. Progress bar broken on x86-64. Fixed.
3.2 2 JAN 2007 NFS support, improvements to the Squashfs-tools, major
bug fixes, lots of small improvements/bug fixes, and new
kernel patches.
Improvements:
1. Squashfs filesystems can now be exported via NFS.
2. Unsquashfs now supports 2.x filesystems.
3. Mksquashfs now displays a progress bar.
4. Squashfs kernel code has been hardened against accidently or
maliciously corrupted Squashfs filesystems.
Bug fixes:
5. Race condition occurring on S390 in readpage() fixed.
6. Odd behaviour of MIPS memcpy in read_data() routine worked-around.
7. Missing cache_flush in Squashfs symlink_readpage() added.
3.1-r2 30 AUG 2006 Mksquashfs -sort bug fix
A code optimisation after testing unfortunately
broke sorting in Mksquashfs. This has been fixed.
3.1 19 AUG 2006 This release has some major improvements to
the squashfs-tools, a couple of major bug
fixes, lots of small improvements/bug fixes,
and new kernel patches.
1. Mksquashfs has been rewritten to be multi-threaded. It
has the following improvements
1.1. Parallel compression. By default as many compression and
fragment compression threads are created as there are available
processors. This significantly speeds up performance on SMP
systems.
1.2. File input and filesystem output is peformed in parallel on
separate threads to maximise I/O performance. Even on single
processor systems this speeds up performance by at least 10%.
1.3. Appending has been significantly improved, and files within the
filesystem being appended to are no longer scanned and
checksummed. This significantly improves append time for large
filesystems.
1.4. File duplicate checking has been optimised, and split into two
separate phases. Only files which are considered possible
duplicates after the first phase are checksummed and cached in
memory.
1.5 The use of swap memory was found to significantly impact
performance. The amount of memory used to cache files is now a
command line option, by default this is 512 Mbytes.
2. Unsquashfs has the following improvements
2.1 Unsquashfs now allows you to specify the filename or the
directory within the Squashfs filesystem that is to be
extracted, rather than always extracting the entire filesystem.
2.2 A new -force option has been added which forces Unsquashfs to
output to the destination directory even if files and directories
already exist in the destination directory. This allows you to
update an already existing directory tree, or to Unsquashfs to
a partially filled directory tree. Without the -force option
Unsquashfs will refuse to output.
3. The following major bug fixes have been made
3.1 A fragment table rounding bug has been fixed in Mksquashfs.
Previously if the number of fragments in the filesystem
were a multiple of 512, Mksquashfs would generate an
incorrect filesystem.
3.2 A rare SMP bug which occurred when simultaneously acccessing
multiply mounted Squashfs filesystems has been fixed.
4. Miscellaneous improvements/bug fixes
4.1 Kernel code stack usage has been reduced. This is to ensure
Squashfs works with 4K stacks.
4.2 Readdir (Squashfs kernel code) has been fixed to always
return 0, rather than the number of directories read. Squashfs
should now interact better with NFS.
4.3 Lseek bug in Mksquashfs when appending to larger than 4GB
filesystems fixed.
4.4 Squashfs 2.x initrds can now been mounted.
4.5 Unsquashfs exit status fixed.
4.6 New patches for linux-2.6.18 and linux-2.4.33.
3.0 15 MAR 2006 Major filesystem improvements
1. Filesystems are no longer limited to 4 GB. In
theory 2^64 or 4 exabytes is now supported.
2. Files are no longer limited to 4 GB. In theory the maximum
file size is 4 exabytes.
3. Metadata (inode table and directory tables) are no longer
restricted to 16 Mbytes.
4. Hardlinks are now suppported.
5. Nlink counts are now supported.
6. Readdir now returns '.' and '..' entries.
7. Special support for files larger than 256 MB has been added to
the Squashfs kernel code for faster read access.
8. Inode numbers are now stored within the inode rather than being
computed from inode location on disk (this is not so much an
improvement, but a change forced by the previously listed
improvements).
2.2-r2 8 SEPT 2005 Second release of 2.2, this release fixes a couple
of small bugs, a couple of small documentation
mistakes, and adds a patch for kernel 2.6.13.
1. Mksquashfs now deletes the output filesystem image file if an
error occurs whilst generating the filesystem. Previously on
error the image file was left empty or partially written.
2. Updated mksquashfs so that it doesn't allow you to generate
filesystems with block sizes smaller than 4K. Squashfs hasn't
supported block sizes less than 4K since 2.0-alpha.
3. Mksquashfs now ignores missing files/directories in sort files.
This was the original behaviour before 2.2.
4. Fixed small mistake in fs/Kconfig where the version was still
listed as 2.0.
5. Updated ACKNOWLEDGEMENTS file.
2.2 3 JUL 2005 This release has some small improvements, bug fixes
and patches for new kernels.
1. Sort routine re-worked and debugged from release 2.1. It now allows
you to give Mkisofs style sort files and checks for filenames that
don't match anything. Sort priority has also been changed to
conform to Mkisofs usage, highest priority files are now placed
at the start of the filesystem (this means they will be on the
inside of a CD or DVD).
2. New Configure options for embedded systems (memory constrained
systems). See INSTALL file for further details.
3. Directory index bug fixed where chars were treated as signed on
some architectures. A file would not be found in the rare case
that the filename started with a chracter greater than 127.
4. Bug introduced into the read_data() routine when sped up to use data
block queueing fixed. If the second or later block resulted in an
I/O error this was not checked.
5. Append bug introduced in 2.1 fixed. The code to compute the new
compressed and uncompressed directory parts after appending was
wrong.
6. Metadata block length read routine altered to not perform a
misaligned short read. This was to fix reading on an ARM7 running
uCLinux without a misaligned read interrupt handler.
7. Checkdata bug introduced in 2.1 fixed.
2.1-r2 15 DEC 2004 Code changed so it can be compiled with gcc 2.x
1. In some of the code added for release 2.1 I unknowingly used some
gcc extensions only supported by 3.x compilers. I have received
a couple of reports that the 2.1 release doesn't build on 2.x and so
people are clearly still using gcc 2.x. The code has been
rewritten to remove these extensions.
2.1 10 DEC 2004 Significantly improved directory handling plus numerous
other smaller improvements
1. Fast indexed directories implemented. These speed up directory
operations (ls, file lookup etc.) significantly for directories
larger than 8 KB.
2. All directories are now sorted in alphabetical order. This again
speeds up directory operations, and in some cases it also results in
a small compression improvement (greater data similarity between
files with alphabetically similar names).
3. Maximum directory size increased from 512 KB to 128 MB.
4. Duplicate fragment checking and appending optimised in mksquashfs,
depending on filesystem, this is now up to 25% faster.
5. Mksquashfs help information reformatted and reorganised.
6. The Squashfs version and release date is now printed at kernel
boot-time or module insertion. This addition will hopefully help
to reduce the growing problem where the Squashfs version supported
by a kernel is unknown and the kernel source is unavailable.
7. New PERFORMANCE.README file.
8. New -2.0 mksquashfs option.
9. CHANGES file reorganised.
10. README file reorganised, clarified and updated to include the 2.0
mksquashfs options.
11. New patch for Linux 2.6.9.
12. New patch for Linux 2.4.28.
2.0r2 29 AUG 2004 Workaround for kernel bug in kernels 2.6.8 and newer
added
1. New patch for kernel 2.6.8.1. This includes a workaround for a
kernel bug introduced in 2.6.7bk14, which is present in all later
versions of the kernel.
If you're using a 2.6.8 kernel or later then you must use this
2.6.8.1 patch. If you've experienced hangs or oopses using Squashfs
with a 2.6.8 or later kernel then you've hit this bug, and this
patch will fix it.
It is worth mentioning that this kernel bug potentially affects
other filesystems. If you receive odd results with other
filesystems you may be experiencing this bug with that filesystem.
I submitted a patch but this has not yet gone into the
kernel, hopefully the bug will be fixed in later kernels.
2.0 13 JULY 2004 A couple of new options, and some bug fixes
1. New mksquashfs -all-root, -root-owned, -force-uid, and -force-gid
options. These allow the uids/gids of files in the generated
filesystem to be specified, overriding the uids/gids in the
source filesystem.
2. Initrds are now supported for kernels 2.6.x.
3. amd64 bug fixes. If you use an amd64, please read the README-AMD64
file.
4. Check-data and gid bug fixes. With 2.0-alpha when mounting 1.x
filesystems in certain cases file gids were corrupted.
5. New patch for Linux 2.6.7.
2.0-ALPHA 21 MAY 2004 Filesystem changes and compression improvements
1. Squashfs 2.0 has added the concept of fragment blocks.
Files smaller than the file block size and optionally the
remainder of files that do not fit fully into a block (i.e. the
last 32K in a 96K file) are packed into shared fragments and
compressed together. This achieves on average 5 - 20% better
compression than Squashfs 1.x.
2. The maximum block size has been increased to 64K (in the ALPHA
version of Squashfs 2.0).
3. The maximum number of UIDs has been increased to 256 (from 48 in
1.x).
4. The maximum number of GIDs has been increased to 256 (from 15 in
1.x).
5. Removal of sleep_on() function call in 2.6.x patch, to allow Squashfs
to work on the Fedora rc2 kernel.
6. Numerous small bug fixes have been made.
1.3r3 18 JAN 2004 Third release of 1.3, this adds a new mksquashfs option,
some bug fixes, and extra patches for new kernels
1. New mksquashfs -ef exclude option. This option reads the exclude
dirs/files from an exclude file, one exclude dir/file per line. This
avoids the command line size limit when using the -e exclude option,
2. When appending to existing filesystems, if mksquashfs experiences a
fatal error (e.g. out of space when adding to the destination), the
original filesystem is restored,
3. Mksquashfs now builds standalone, without the kernel needing to be
patched.
4. Bug fix in the kernel squashfs filesystem, where the pages being
filled were not kmapped. This seems to only have caused problems
on an Apple G5,
5. New patch for Linux 2.4.24,
6. New patch for Linux 2.6.1, this replaces the patch for 2.6.0-test7.
1.3r2 14 OCT 2003 Second release of 1.3, bug fixes and extra patches for
new kernels
1. Bug fix in routine that adds files to the filesystem being
generated in mksquashfs. This bug was introduced in 1.3
(not enough testing...) when I rewrote it to handle files larger
than available memory. This bug caused a SEGV, so if you've ever
got that, it is now fixed,
2. Long running bug where ls -s and du reported wrong block size
fixed. I'm pretty sure this used to work many kernel versions ago
(2.4.7) but it broke somewhere along the line since then,
3. New patch for Linux 2.4.22,
4. New patch for 2.6.0-test7, this replaces the patch for 2.6.0-test1.
1.3 29 JUL 2003 FIFO/Socket support added plus optimisations and
improvements
1. FIFOs and Socket inodes are now supported,
2. Mksquashfs can now compress files larger than available
memory,
3. File duplicate check routine optimised,
4. Exit codes fixed in Mksquashfs,
5. Patch for Linux 2.4.21,
6. Patch for Linux 2.6.0-test1. Hopefully, this will work for
the next few releases of 2.6.0-testx, otherwise, I'll be
releasing a lot of updates to the 2.6.0 patch...
1.2 13 MAR 2003 Append feature and new mksquashfs options added
Mksquashfs can now add to existing squashfs filesystems. Three extra
options "-noappend", "-keep-as-directory", and "root-becomes"
have been added.
The append option with file duplicate detection, means squashfs can be
used as a simple versioning archiving filesystem. A squashfs
filesystem can be created with for example the linux-2.4.19 source.
Appending the linux-2.4.20 source will create a filesystem with the
two source trees, but only the changed files will take extra room,
the unchanged files will be detected as duplicates.
See the README file for usage changes.
1.1b 16 JAN 2003 Bug fix release
Fixed readpage deadlock bug. This was a rare deadlock bug that
happened when pushing pages into the page cache when using greater
than 4K blocks. I never got this bug when I tested the filesystem,
but two people emailed me on the same day about the problem!
I fixed it by using a page cache function that wasn't there when
I originally did the work, which was nice :-)
1.1 8 JAN 2003 Added features
1. Kernel squashfs can now mount different byte order filesystems.
2. Additional features added to mksquashfs. Mksquashfs now supports
exclude files and multiple source files/directories can be
specified. A nopad option has also been added, which
informs mksquashfs not to pad filesystems to a multiple of 4K.
See README for mksquashfs usage changes.
3. Greater than 2GB filesystems bug fix. Filesystems greater than 2GB
can now be created.
1.0c 14 NOV 2002 Bug fix release
Fixed bugs with initrds and device nodes
1.0 23 OCT 2002 Initial release
|