1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
|
Version: lbzip2-0.23
Focus: Minor feature enhancements
Date: 03-Mar-2010
Changes: In this release, if lbzip2 intends to exit with status 1 due to any
fatal error, but any SIGPIPE or SIGXFSZ with inherited SIG_DFL action
was generated for lbzip2 previously, then lbzip2 terminates by way of
one of said signals, after cleaning up any interrupted output file.
This should improve compatibility with GNU tar, when it spawns lbzip2
as a filter, and closes the pipe between them early, before it
receives an EOF from lbzip2.
Version: lbzip2-0.22
Focus: Minor bugfixes
Date: 18-Feb-2010
Changes: Building lbzip2 on Debian unstable discovered that the "lfs.sh" build
script, due to a typo, did not invoke the "getconf" utility in a
SUSv2-conformant way. This bug has been corrected.
Version: lbzip2-0.21
Focus: Minor bugfixes
Date: 17-Feb-2010
Changes: Code examination revealed that lbzip2-0.18 introduced a race between
the following two code paths: (1) the muxer thread displays an error
message when it encounters a write error, (2) the main thread, in
preparation to terminate the process, frees the output file name after
an INT or TERM signal is delivered to it. This bug had negligible
chance to occur, but it was fixed nonetheless.
Version: lbzip2-0.20
Focus: Code cleanup
Date: 28-Dec-2009
Changes: This release cleans up two theoretical portability problems. The input
file's sanity check in the multiple-workers decompressor relied on
character constants being encoded in ASCII; such character constants
were replaced by their ASCII octet values. "Flexible" array members
were replaced by unnamed memory regions following the affected
structures, since the "struct hack" idiom entails undefined behavior
in C89.
Version: lbzip2-0.19
Focus: Minor bugfixes
Date: 01-Dec-2009
Changes: This release works around a GNU/kFreeBSD standards-compliance problem.
GNU/kFreeBSD does not define some STREAMS-related errno macros
mandated by SUSv2. Consequently, lbzip2-0.18 cannot be built on
GNU/kFreeBSD. This version checks if those (and some other) macros are
defined before relying them.
Version: lbzip2-0.18
Focus: Major feature enhancements
Date: 29-Nov-2009
Changes: After adding sanity checks to both decompressors, the following
features were implemented: removal of input FILE operands; options
--keep and --force; copying of owner, group, permission bits, access
time, modification time to regular output files. Logging was cleaned
up and internally categorized into INFO, WARNING and FATAL levels; a
separate exit status was introduced for the case when a warning
message was printed. The decompressor robustness tests were
re-executed. The author has finally replaced bzip2 with lbzip2 on his
system.
Version: lbzip2-0.17
Focus: Code cleanup
Date: 29-Oct-2009
Changes: Uninitialized fields were accessed on the stack in a structure
assignment expression. Theoretically, this might have entailed read
accesses to trap representations. Since those fields weren't
initialized because they weren't used at all in the first place, this
portability bug was fixed by introducing a dedicated structure without
those fields.
Version: lbzip2-0.16
Focus: Minor bugfixes
Date: 25-Oct-2009
Changes: Non-regular input files passed on the command line are skipped
(symbolic links are followed, because lbzip2 doesn't remove input
files). Instead of exiting, lbzip2 skips input files passed on the
command line which it cannot open, or for which it cannot create the
corresponding output files. Standard output is closed also with
--stdout and file operands, not only in filter mode. Signal handling
has been cleaned up; lbzip2 can now be interrupted during skipping
many files in a row. Writing an error message to a broken pipe doesn't
result in an incomplete output file anymore.
Version: lbzip2-0.16rc1
Focus: Major feature enhancements
Date: 21-Oct-2009
Changes: With this release candidate, implementing features like settable
compression block size and working with file operands, lbzip2 can be
considered a multi-threaded, command-line compatible replacement for
bzip2 in most situations. Lbzip2 never deletes or overwrites files,
however. The workaround for an earlier GNU getconf bug was extended to
EGLIBC in the large file support build script. The development status
was degraded to beta.
Version: lbzip2-0.15
Focus: Minor bugfixes
Date: 05-Apr-2009
Changes: A bug has been fixed where the single-worker decompressor could
theoretically omit to decompress the last part of the compressed input
and wrongfully signify premature EOF instead. In this release,
logically independent buffer sizes are detached from each other.
Several buffer sizes should be more IO-friendly now. The splitter
block size of the multiple-workers decompressor was recalculated; an
effort was made to deduce it formally in comments. The
"malloc_trace.pl" script tracks peak memory usage. The documentation
was refined.
Version: lbzip2-0.14
Focus: Major bugfixes
Date: 15-Mar-2009
Changes: The single-worker decompressor's multiplexer, while passing back all
accrued released input slots to the splitter, wrote out at most one
decompressed sub-block per one iteration of its outermost loop. In
case of a slowly progressing muxer, this could lead to skyrocketing
memory allocation on part of the worker. Now the SWD's muxer writes
out all accrued decompressed sub-blocks per iteration. The
documentation has been updated. The build system has received a
minuscule portability improvement.
Version: lbzip2-0.13
Focus: Code cleanup
Date: 28-Jan-2009
Changes: Besides some code cleanup, various portability and other workarounds
were introduced to cope with bugs in C libraries, shells, and upstream
bzip2.
Version: lbzip2-0.12
Focus: Minor feature enhancements
Date: 26-Jan-2009
Changes: In this release the shell script test.sh tests the installed instance
of lbzip2, which can be different from the currently built one. pbzip2
is not a requirement for testing anymore, but test.sh will try to
measure its performance if it's available. The same applies to the
bzip2 module of 7za (from p7zip). Some GNU/Linux specific hints on
enabling SUSv2 conformance (which is requried for building and
testing) were added to the README.
Version: lbzip2-0.11
Focus: Minor feature enhancements
Date: 21-Jan-2009
Changes: This release adds support for testing by end-users, in a portable way.
Version: lbzip2-0.10
Focus: Minor bugfixes
Date: 18-Jan-2009
Changes: Testing on a 128-core HP SuperDome by a volunteering user showed a
known bottleneck in the multiple-workers decompressor to be
significant on many-core machines: whenever there were less input
blocks than cores, the work was distributed unevenly. Hence, the
splitter-to-workers queue of "scan and decompress" tasks was replaced
with two queues, a low priority, splitter-to-workers one of "scan"
tasks, and a high priority, workers-to-workers one of "decompress"
tasks. Alas, this also increased the number of context switches. The
new worker broadcast conditions were formally proven in the comments.
Version: lbzip2-0.09
Focus: Minor feature enhancements
Date: 06-Jan-2009
Changes: Now the Makefiles, with the help of the standard getconf utility,
select a programming environment, if there is one, in which large
files are supported.
Version: lbzip2-0.08
Focus: Code cleanup
Date: 04-Dec-2008
Changes: In the multiple-workers decompressor, the tail pointer of the
splitter-to-workers queue proved to be private to the splitter.
Accordingly, said pointer was eliminated as a shared resource,
simplifying the code.
Version: lbzip2-0.07
Focus: Documentation
Date: 29-Nov-2008
Changes: A manual page using the man macro package was added. The README file
describes the compressed output and why there is only one exit status
for all types of errors. A more portable Makefile was created. A
little code was cleaned up.
Version: lbzip2-0.06a
Focus: Documentation
Date: 29-Sep-2008
Changes: A class of valid bz2 files the multiple-workers decompressor (MWD)
possibly refuses has been described in the Bugs section of the README.
By concatenating empty bzip2 streams (each having a length of 14
bytes) and optionally inserting such a sequence before, after or
between non-empty bzip2 streams, the size of the input block that
unavoidably contains an entire bzip2 block header can be increased
without bound. This invalidates the assumption the MWD is based on.
However, neither bzip2 nor lbzip2 creates such files, and bz2 file
sets that do defeat the MWD when catenated should be rare.
Version: lbzip2-0.06
Focus: Minor feature enhancements
Date: 16-Sep-2008
Changes: When decompressing with a single worker thread, lbzip2 was previously
45% slower than standard bzip2. The new, dedicated single-worker
decompressor is only 3% slower, and provides input and output
buffering, which is useful in pipelines and on network file systems.
Hence using lbzip2 incurs virtually no performance penalty over bzip2
even on a single-core machine. A script was added to help automated
testing. Some thread notification conditions have been cleaned up. In
this release, lbzip2 compresses an empty file to a valid bzip2 stream
instead of an empty file.
Version: lbzip2-0.05
Focus: Major feature enhancements
Date: 10-Sep-2008
Changes: The decompressor was redesigned: all CPU-bound operations were moved
into the worker threads, so that now, besides the muxer, the splitter
is purely I/O-bound too. Lbzip2 supports tracing its memory allocation
with the new "-t" option. Both the compressor and decompressor were
retested.
Version: lbzip2-0.04
Focus: Major feature enhancements
Date: 01-Sep-2008
Changes: Decompression was extracted from the split-work-multiplex skeleton
into a separate module. Compression was added. The project has been
renamed to lbzip2. The reordering of processed sub-blocks happens
entirely in the multiplexer now, changing the time complexity from
O(log n) to O(1) inside the critical section between workers and the
muxer. The command line conforms to utility syntax guidelines. Lbzip2
queries the number of online processors if sysconf() supports it.
Block serial numbers have fixed 64 bit width. The README file was
updated. The development status has been advanced to Beta.
Version: lbunzip2-0.03
Focus: Major bugfixes
Date: 20-Aug-2008
Changes: After running lbunzip2 on the bz2 test material of CERT-FI 20469, a
bug was fixed where a worker (decompressor) thread could get into an
infinite loop, spinning until finally outrunning the multiplexer
thread, then consuming all available memory and exiting.
Version: lbunzip2-0.02
Focus: Major bugfixes
Date: 18-Aug-2008
Changes: Version 0.01 didn't throttle the decompressor threads when the
multiplexer thread was blocked on the write() system call, thus memory
consumption could grow indefinitely. This is fixed now. Some
performance testing was done on five multicore machines (Alpha,
Athlon, Itanium, Sparc, and Xeon). The README file was rewritten.
Version: lbunzip2-0.01
Focus: Initial freshmeat announcement
Date: 13-Aug-2008
Changes: Initial freshmeat announcement.
|
