File: zlibc.texi

package info (click to toggle)
zlibc 0.9k-4.3
  • links: PTS
  • area: main
  • in suites: bullseye, buster, sid, stretch
  • size: 624 kB
  • ctags: 461
  • sloc: sh: 3,195; ansic: 3,077; csh: 128; makefile: 60; sed: 4
file content (1243 lines) | stat: -rw-r--r-- 45,789 bytes parent folder | download | duplicates (4)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
\input texinfo @c -*-texinfo-*-
@c %**start of header
@setfilename zlibc.info
@settitle Zlibc
@dircategory Development
@c UPDATE THIS DATE WHENEVER YOU MAKE CHANGES!
@c For double-sided printing, uncomment:
@c @setchapternewpage odd
@setchapternewpage off
@c %**end of header

@c MANskip

@set EDITION 0.9k
@set VERSION 0.9k
@set UPDATED February 2005

@iftex
@finalout
@end iftex

@ifinfo
@format
START-INFO-DIR-ENTRY
* Zlibc: (zlibc).        Zlibc: transparently access compressed files.
END-INFO-DIR-ENTRY
@end format
@end ifinfo


@ifinfo
Zlibc Copyright (C) 1993-1999 Alain Knaff

Permission is granted to make and distribute verbatim copies of
this manual provided the copyright notice and this permission notice
are preserved on all copies.

@ignore
Permission is granted to process this file through TeX and print the
results, provided the printed document carries copying permission
notice identical to this one except for the removal of this paragraph
(this paragraph not being relevant to the printed manual).
@end ignore

Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one, and provided that the privacy
of any reader of the resulting derived work is respected. In particular
is it strictly forbidden to make this documentation available on a World
Wide Web server which deals cookies. However, keeping access statistics
is allowed.

Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation approved
by the Free Software Foundation.
@end ifinfo

@titlepage
@title Zlibc
@subtitle Transparently access compressed files.
@subtitle Edition @value{EDITION}, for Zlibc version @value{VERSION}
@subtitle @value{UPDATED}
@author by Alain Knaff
@page

@vskip 0pt plus 1filll
Copyright @copyright{} 1993-1999 Alain Knaff

Permission is granted to make and distribute verbatim copies of
this manual provided the copyright notice and this permission notice
are preserved on all copies.

Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one and provided that the privacy of
any reader of the resulting derived work is respected. In particular is
it strictly forbidden to make this documentation available on a World
Wide Web server which deals cookies. However, keeping access statistics
is allowed.

Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation approved
by the Free Software Foundation.

@end titlepage

@ifinfo
@node Top, Location, (dir), (dir)

@end ifinfo

@c MANend-skip
@c zlibc-conf.5skip

@ignore
@unnumbered Name
zlibc - transparently access compressed files.
@end ignore

@unnumbered Introduction

The zlibc package allows transparent on the fly uncompression of gzipped
files.  Your programs will be able to access any compressed file, just
as if they were uncompressed.  Zlibc will transparently uncompresses the
data from these files as soon as they are read, just as a compressed
filesystem would do.  No kernel patch, no recompilation of these
executables and no recompilation of the libraries is needed.

It is not (yet) possible execute compressed files with zlibc. However,
there is another package, called @code{tcx}, which is able to uncompress
executables on the fly.  On the other hand @code{tcx} isn't able to
uncompress data files on the fly.  Fortunately, the both zlibc and tcx
may coexist on the same machine without problems.

@menu
* Location::              Where to get zlibc and early bug fixes
* Installing zlibc::      How to install zlibc
* Using zlibc::           How to configure your system to use zlibc
* How it works::          The general principles behind zlibc
* Customization::         How to customize zlibc to your needs
* Porting zlibc::         How to port zlibc to new architectures
* Variable Index::        Variable Index
* Concept Index::         Concept Index
@end menu
@c MANskip
This documentation looks most pretty when printed or as html.  Indeed,
in the info version certain examples are difficult to read due to the
confusing quoting conventions of info.
@c MANend-skip

@include man-warning.texi


@cindex bugs
@cindex ALPHA patches
@cindex patches
@cindex diffs
@cindex mailing list


@node Location, Installing zlibc, Top, Top
@chapter Where to get zlibc

Zlibc can be found at the following places (and their mirrors):
@example
ftp://zlibc.linux.lu/zlibc-0.9k.tar.gz
ftp://www.tux.org/pub/knaff/zlibc/zlibc-0.9k.tar.gz
ftp://ibiblio.unc.edu/pub/Linux/libs/compression/zlibc-0.9k.tar.gz
ftp://ftp.gnu.org/gnu/zlibc/compression/zlibc-0.9k.tar.gz
@end example

Before reporting a bug, make sure that it has not yet been fixed in the
Alpha patches which can be found at:
@example
http://zlibc.linux.lu/
http://www.tux.org/pub/knaff/zlibc
@end example

These patches are named
@code{zlibc-}@var{version}@code{-}@var{ddmm}@code{.taz}, where version
stands for the base version, @var{dd} for the day and @var{mm} for the
month. Due to a lack of space, I usually leave only the most recent
patch.

There is an zlibc mailing list at zlibc @@ www.tux.org .  Please
send all bug reports to this list.  You may subscribe to the list by
sending a message with 'subscribe zlibc @@ www.tux.org' in its
body to majordomo @@ www.tux.org . (N.B. Please remove the spaces
around the "@@" both times. I left them there in order to fool
spambots.)  Announcements of new zlibc versions will also be sent to
the list, in addition to the linux announce newsgroups.  The mailing
list is archived at http://www.tux.org/hypermail/zlibc/latest

@node Installing zlibc, Using zlibc, Location, Top
@chapter Installing zlibc
@vindex LD_PRELOAD
@cindex Installation
@cindex Set-uid programs
@cindex Boot problems

@enumerate
@item 
If you install zlibc on Linux, make sure that your shared loader
(ld-linux.so.1/ld.so) understands @code{LD_PRELOAD}. (Best if
ld.so-1.8.5 or more recent)

@item
Type @code{./configure}. This runs the GNU autoconfigure script which
configures the @file{Makefile} and the @file{config.h} file.  You may
compile time configuration options to @code{./configure}, see for
details.

@item
Type @code{make} to compile zlibc.

@item
Type @code{make install} to install zlibc and associated programs to its
final target.

@item
To use this module, set the environment variable @code{LD_PRELOAD} to
point to the object. Example (sh syntax):

@example
      LD_PRELOAD=/usr/local/lib/uncompress.so
      export LD_PRELOAD
@end example

or (csh syntax):

@example
      setenv LD_PRELOAD /usr/local/lib/uncompress.so
@end example
On linux, use /lib/uncompress.so instead of /usr/local/lib/uncompress.so .

You might want to put these lines in your @file{.profile} or
@file{.cshrc} in order to have the uncompressing functions available all
the time.
@item
Compress your files using gzip and enjoy

@end enumerate

For security reasons, the dynamic loader disregards environmental
variables such as @code{LD_PRELOAD} when executing set uid programs.

However, on Linux, you can use zlibc with set uid programs too, by using
one of the two methods described below:

@enumerate
@item
You may ing the path to @file{uncompress.so} into
@file{/etc/ld.so.preload} instead of using @code{LD_PRELOAD}.

@strong{WARNING}: If you use @file{/etc/ld.so.preload}, be sure to
install @file{uncompress.so} on your root filesystem, for instance in
@code{/lib}, as is done by the default configuration.  Using a directory
which is not available at boot time, such as /usr/local/lib will cause
trouble at the next reboot!

It is also careful to remove zlibc from @file{/etc/ld.so.preload} when
installing a new version.  First test it out using @code{LD_PRELOAD},
and only if everything is ok, put it back into
@file{/etc/ld.so.preload}.

@item
If you have a version of @code{ld.so} which is more recent than
@code{1.9.0}, you can set @code{LD_PRELOAD} to just contain the basename
of @file{uncompress.so} without the directory.  In that case, the file is
found as long as it is in the shared library path (which usually
contains @file{/lib} and @file{/usr/lib})). Because the search is
restricted to the library search path, this also works for set-uid
programs.

Example (sh syntax):

@example
      LD_PRELOAD=uncompress.so
      export LD_PRELOAD
@end example

or (csh syntax):

@example
      setenv LD_PRELOAD uncompress.so
@end example

The advantage of this approach over @file{ld.so.preload} is that zlibc
can more easily be switched off in case something goes wrong.

@end enumerate

@node Using zlibc, How it works, Installing zlibc, Top
@chapter Using zlibc
@cindex Symbolic links

Once zlibc is installed, simply compress your biggest datafiles
using gzip. Your programs are now able to uncompress these files on the
fly whenever they need them.

@section Zlibc and links

@subsection Symbolic links

After compressing your datafiles, you also need to change any potential
symbolic links pointing to them.  Let's suppose that @file{x} is a
symlink to @file{tstfil}:

@example
> echo 'this is a test' >tstfil
> ln -s tstfil x
> ls -l
total 1
-rw-r--r--   1 alknaff  sirac          15 Feb 25 19:40 tstfil
lrwxrwxrwx   1 alknaff  sirac           8 Feb 25 19:40 x -> tstfil
@end example

After compressing it, you'll see the following listing:

@example
> gzip tstfil
> ls -l
total 1
pr--r--r--   1 alknaff  sirac          15 Feb 25 19:40 tstfil
lrwxrwxrwx   1 alknaff  sirac           8 Feb 25 19:40 x -> tstfil
@end example

@file{Tstfil} is now shown as a pipe by zlibc in order to warn programs
that they cannot seek in it.  Zlibc still shows it with its old name,
and you can directly look at its contents:
@example
> cat tstfil
this is a test
@end example

However, @file{tstfil} is not yet accessible using the symbolic link:
@example
> cat x
cat: x: No such file or directory
@end example

In order to make @file{tstfil} accessible using the link, you have to
destroy the link, and remake it:
@example
> rm x
/bin/rm: remove `x'? y
> ln -s tstfil x
> ls -l
total 1
pr--r--r--   1 alknaff  sirac          15 Feb 25 19:40 tstfil
lrwxrwxrwx   1 alknaff  sirac           8 Feb 25 19:44 x -> tstfil
> cat x
this is a test
@end example

@subsection Hard links

If you compress datafiles with hard links pointing to them, gzip refuses
to compress them. 

@example
> echo 'this is a test' >tstfil
> ln tstfil x
> ls -li
total 2
    166 -rw-r--r--   2 alknaff  sirac          15 Feb 25 19:46 tstfil
    166 -rw-r--r--   2 alknaff  sirac          15 Feb 25 19:46 x
> gzip tstfil
gzip: tstfil has 1 other link  -- unchanged
@end example

Thus you need to remove these hard links first, and remake them after
compressing the file.

@example
> rm x
/bin/rm: remove `x'? y
> gzip tstfil
> ln tstfil x
> ls -li
total 2
    167 pr--r--r--   2 alknaff  sirac          15 Feb 25 19:46 tstfil
    167 pr--r--r--   2 alknaff  sirac          15 Feb 25 19:46 x
> cat x
this is a test
@end example

@node How it works, Customization, Using zlibc, Top
@chapter How it works

Usually, programs don't make system calls directly, but instead call a
library function which performs the actual system calls. For instance,
to open a file, the program first calls the @code{open} library
function, and then this function makes the actual syscall.  Zlibc
overrides the @code{open} function and other related functions in order
to do the uncompression on the fly.

If the @code{open} system call fails because the file doesn't exist,
zlibc constructs the filename of a compressed file by appending
@code{.gz} to the filename supplied by the user program. If this
compressed file exists, it is opened and piped trough @code{gunzip}, and
the descriptor of the read end of this pipe is returned to the caller.

In some cases, the compressed file is first uncompressed into a
temporary file, and a read descriptor for this file is passed to the
caller.  This is necessary if the caller wants to call @code{lseek} on
the file or @code{mmap} it. A description of data files for which using
temporary is necessary can be given in the configuration files
@file{/usr/local/etc/zlibc.conf} (@file{/etc/zlibc.conf} on
Linux)@footnote{Actually the location of the system-wide include file
depends on the settings of @code{sysconfdir} and @code{prefix} during
@code{./configure} (@pxref{Compile-time configuration}).} and
@file{~/.zlibrc}. @xref{Configuration files}, for a detailed description
of their syntax.

Many user programs try to check the existence of a given file by other
system calls before actually opening it. That's why zlibc also
overrides these system calls. If for example the user program tries to
stat a file, this call is also intercepted.

The compressed file, which exists physically on the disk, is also called
'the real file', and the uncompressed file, whose existence is only
simulated by zlibc is called 'the virtual file'.

@node Customization, Porting zlibc, How it works, Top
@chapter Customization
@cindex Customization

The behavior of zlibc can be tailored using configuration files or
environment variables. This customization should normally not be needed,
as the compiled-in defaults are already pretty complete.


@menu
* Environmental variables::
* Configuration files::
* Compiled-in defaults::
* Compile-time configuration::
@end menu

@node Environmental variables, Configuration files, Customization, Customization
@section Environmental variables

Environmental variables come in two kinds: @emph{switch} variables have
a boolean value and can only be turned on or off, whereas @emph{string}
variables can have arbitrary strings as values.


@menu
* Switch variables::
* String variables::
@end menu


@node Switch variables, String variables, Environmental variables, Environmental variables
@subsection Switch variables
@vindex LD_ZLIB_VERBOSE
@cindex debugging zlibc
@vindex LD_ZLIB_UNLINK
@cindex removing files
@cindex preventing accidental removal of compressed files (env.var.)
@vindex LD_ZLIB_DISABLE
@vindex LD_ZLIB_READDIR_COMPR
@cindex showing compressed files in a directory listing (env.var.)

These variables represent a flag which can be turned on or off.  If
their value is @code{on} or @code{1} they are turned on, if their value
is @code{off} or @code{0} they are turned off. All other values are
ignored. If the same flag can be turned on or off using config files,
the environmental variable always has the priority.

@table @code
@item LD_ZLIB_VERBOSE 
If this variable is turned on, informational messages are printed on
many operations of zlibc.  Moreover, error messages are printed in order
to point out errors in the configuration files, if any. If this variable
is turned off, errors are silently ignored.

@item LD_ZLIB_UNLINK
If this variable is turned on, and if the user program tries to unlink a
virtual (uncompressed) file, zlibc translates this call into unlinking
the real file. If this variable is turned off, unlink calls on virtual
files are ignored.

@item LD_ZLIB_DISABLE
If this variable is turned on, zlibc is switched off.

@item LD_ZLIB_READDIR_COMPR
If this variable is turned on, the readdir function shows the real
(compressed) files instead of the virtual (uncompressed) files.
@end table

@node String variables, , Switch variables, Environmental variables
@subsection String variables
@vindex LD_ZLIB_TMP
@cindex directory for temporary files (env.var.)
@vindex LD_ZLIB_EXT
@cindex filename extension for compressed files (env.var.)
@vindex LD_ZLIB_UNCOMPRESSOR
@cindex command line for the uncompressing program (env.var.)
@vindex LD_ZLIB_CONFFILE
@cindex location of the configuration file (env.var.)

These variables have a string value, which represent a file, a directory
or a command.

@table @code
@item LD_ZLIB_TMP
This is the name of the directory where the temporary uncompressed files are 
put. The default is /tmp.

@item LD_ZLIB_EXT
This is the extension which is appended to a virtual file name in order
to obtain the real (compressed) file name. The default is @code{.gz}.

@item LD_ZLIB_UNCOMPRESSOR
This is the name of the program to be invoked to uncompress the data. Default
is @code{gzip -dc}.

@item LD_ZLIB_CONFFILE 
This is the name of an additional configuration file. If this variable
is defined and if the corresponding file exists, the configuration
described in this file overrides the configurations in @file{~/.zlibrc}
and in @file{/usr/local/etc/zlibc.conf} (@file{/etc/zlibc.conf} on
Linux).
@end table

@c zlibc.3skip
@c zlibc-conf.5end-skip

@c MANskip
@node Configuration files, Compiled-in defaults, Environmental variables, Customization
@section Configuration files
@cindex Processing order of configuration file.
@c MANend-skip
@ignore
@subsection Name
zlibc.conf - zlibc's configuration file
@include man-warning.texi
@subsection Description
@end ignore



To get its configuration, zlibc first looks into the file described by
the environment variable @code{LD_ZLIB_CONFFILE} (if any), then in
@file{~/.zlibrc} and finally in @file{/usr/local/etc/zlibc.conf}
(@file{/etc/zlibc.conf} on Linux @footnote{Actually the location of the
system-wide include file depends on the settings of @code{sysconfdir}
and @code{prefix} during @code{./configure} (@pxref{Compile-time
configuration}).}).  If the desired information is found in neither of
these files, the compiled-in defaults are used.  It is possible to
supply only part of the needed information in the configuration
files. In that case, the missing information is retrieved from the
compiled-in defaults. This allows you to have really small runtime
configuration files, which only list the differences between the desired
configuration and the compiled-in configuration.

If an error occurs while parsing one of the configuration files, the
offending file is skipped, and the search continues with the next
file. However, no error message is printed unless the environmental
variable @code{LD_ZLIB_VERBOSE} is turned on (i.e. set to @code{1} or
to @code{on} ).

If two files contain contradictory information, the information in the
file which is scanned first is retained (usually @file{~/.zlibrc}). If
any flags have been set or unset using environmental variables, these
settings override the flags specified in the configuration files.

The configuration files are read by each process. For each process, they
are read at most once, at the time when zlibc is first used (attempt to
access a compressed file). Afterwards they are cached in the process's
virtual memory.  Thus, changing zlibc configuration files doesn't
generally have any effect on already running processes.

@menu
* Overall structure::
* Commands section::
* Class section::
@end menu

@node Overall structure, Commands section, Configuration files, Configuration files
@subsection Overall structure
@cindex commands section
@cindex class section
@cindex Parsing order of configuration files.

The zlibrc files consist of two sections: A @emph{commands} section
(@ref{Commands line flags}) and a @emph{class definition} section
(@ref{Class section}).

The commands section describes how zlibc should behave depending on
the executable that it was called from.  Several commands are grouped
together into @emph{command classe}.

The class definition section describes how zlibc should behave depending
on the class of the command and the datafile opened.

The configuration file may contain comments in both sections: a comment
starts with a hash (@code{#}) and stops at the end of the line.

Dashes (@code{-}) and underbars (@code{_}) may be used indifferently in
all keywords.

@node Commands section, Class section, Overall structure, Configuration files
@subsection Commands section

@menu
* Commands section syntax::             What goes into the commands
                                        section
* Commands line flags::                 Which flags are available
@end menu

@node Commands section syntax, Commands line flags, Commands section, Commands section
@subsubsection General syntax of the commands section
@cindex commands section

The format for a line in the "commands" section is as follows:

@display
@code{commands "}@var{cmd1}@code{"} [ @dots{} @code{"}@var{cmdn}@code{"} ] @code{use} @var{flags} @code{"}@var{class}@code{"}

@end display


In this line, the @var{cmd1} @dots{} @var{cmdn} are the basenames of the
programs (commands) for which this line should apply.  The basename is
the name without the path, i.e. @code{ls} instead of
@code{/usr/bin/ls}. The command names should be enclosed in quotes.  You
may also use the keyword @code{default} (without quotes) to match all
commands.

The @var{flags} describe those aspects of zlibc's behaviour that are
independant from the datafile which is being accessed. These flags come
in pairs.  The flags need not to be listed on a single @code{commands}
line, they may occur in several places, even in several different
configuration files (for instance one in @code{/etc/zlibc.conf} and
another one in @code{~/.zlibrc}.

If two contradictory flags are found in the configuration files, the one
which is seen first is taken.

If on the other hand a certain flag is not found at all in the
configuration files, the compiled-in default for this flag is used. This
is usually the second flag of each pair, described below
(@pxref{Commands line flags}).

These flags can all be overridden by environmental variables.  When the
corresponding environmental variable is set to @code{1} or to @code{on},
the first flag is used, when it is set to @code{0} or to @code{off}.  If
the environmental variables is set to neither of these 4 values, it is
ignored).

The @var{class} names the commands class that these commands belong to.
If, for a given command, two @code{commands} lines give different
classes, the one which is seen first is taken. No union of classes is
made, the classes are always treated as a whole. Thus, if you want to
make a change to a command class, you need to describe it in its
entirety.

The following example says that the @code{tar}, @code{cpio}, @code{pax},
@code{cp} and @code{mv} show compressed files in a directory listing
(@code{readdir_compr} flag), and are of class @code{generic_safe}.
@example
commands "tar" "cpio" "pax" "cp" "mv" use readdir_compr "generic_safe"
@end example

The class @code{generic_safe} would then need to be described further in
the class section.

@node Commands line flags, , Commands section syntax, Commands section
@subsubsection Available commands line flags
@vindex disable
@vindex enable
@vindex readdir_compr
@vindex readdir_uncompr
@vindex verbose
@vindex silent

@cindex disabling zlibc (conf. file)
@cindex showing compressed files on a directory listing (conf. file)
@cindex debugging zlibc


This section describes the flags which can be used on a @code{commands}
line. All these flags come in pairs.

The table below describes each of these pairs.  The first word in the
header of each item is the non-default flag, the second word is the
default flag, and the third word is the environmental variable by which
you can override the settings from the configuration files. If this
environmental variable is turned on (set to 1), the non-default (first)
flag is taken, if it is turned off (set to 0), the default (second) flag
is taken.

@table @code
@item disable / enable / LD_ZLIB_DISAB
The @code{disable} flag disables zlibc for the programs on this commands
line.  This is useful for compression and uncompression utilities.
Without this flag, @code{gunzip} would not work anymore, because it
would think that the uncompressed file exists already, and it would
refuse to overwrite this file.

@item disable_child / enable_child / LD_ZLIB_DISAB_CHILD
The @code{disable_child} flag disables zlibc for the programs started by
programs on this commands line. This is implemented by removeing all
occurrences of @code{uncompress.so} from the LD_PRELOAD environment
variable. This function is useful for programs such as xemacs, in order
to make sure that all launched subprograms return results consistent
with emacs itself (directory listings, etc)

@item readdir_compr / readdir_uncompr / LD_ZLIB_READDIR
The readdir function shows the uncompressed files (i.e. with their
trailing @code{.gz} extension) when the @code{readdir_compr} flag is
set, and the compressed files otherwise.

@item verbose / silent / LD_ZLIB_VERBOSE
When @code{verbose} is set, zlibc prints informational messages.

@item unlink / no_unlink / LD_ZLIB_UNLINK
If the @code{unlink} flag is set, and if the user program tries to
unlink a virtual (uncompressed) file, the package translates this call
into unlinking the real file. If the @code{no_unlink} flag is set,
requests to unlink virtual (uncompressed) files are silently ignored.

@end table

@node Class section, , Commands section, Configuration files
@subsection Class section


A command class@footnote{These command classes are unrelated to the
classes in object oriented programming.} definition describes those
aspects of zlibc's behavior that depend on the name of the datafile
which is being accessed. Command classes are identified by a name which
is matched against the @var{class} parameter from the commands.  The
class name should be enclosed between quotes both in the commands line
and in the class definition.


The class section contains descriptions of different command classes
(i.e descriptions how datafiles should be uncompressed). Each class
definition begins with a line of the following format:

@display
@code{class "}@var{id}@code{"}
@end display

The class @var{id} is the same string as the one used in the commands
line. The remaining lines of a class definition are as follows:
@display
[ [ @var{criterion} ] [ @code{"}@var{name}@code{"} ]] @var{mode}
@end display

The following example shows the definition of the class used for X-servers.

@example
# X uses tmp files in its own directories.
class "X"
subdir "/usr/X11R6" usetmpfile
subdir "/usr/X386" usetmpfile
subdir "/usr/lib/X11" usetmpfile
showpipe
@end example

This says that all compressed files who are in a subdirectory below
@file{/usr/X11R6}, @file{/usr/X386} or @file{/usr/lib/X11} are
decompressed using temporary files (@code{usetmpfile}), and that files
from other directories are decompressed using pipes (@code{showpipe})


The following examples illustrates a command class, named @code{nopipe},
which always uses temporary files for decompression
@display
# generic class which uses temp files for all files.
class "nopipe"
usetmpfile
@end display

@menu
* Criteria::           To which data files does this line apply
* Mode::               How zlibc behaves for the files matching the
                       criterion
@end menu

@node Criteria, Mode, Class section, Class section
@subsubsection File selection Criteria
@cindex directory (selection criterion)
@cindex filename (selection criterion)
@cindex extension (selection criterion)
@cindex suffix (selection criterion)
@cindex basename (selection criterion)
@cindex doublespaced DOS filesystem

The @var{criterion } describes what parts of the filename should match:

@table @code
@item filename
The entire filename of the target data file must match
@var{name}.

@item basename
The basename (filename without directory) of the target data file must
match @var{name}.

@item directory
The data file must live in the directory @var{name}.  If the user
program opens the file with an absolute pathname, that filename is used
as is.  If on the other hand the user program uses a relative pathname,
zlibc uses the most direct path to the file (i.e. without symlinks).

@item subdirectory
The data file must live in the directory @var{name} or in one of its
subdirectories. If the user program opens the file with an absolute
pathname, that filename is used as is.  If on the other hand the user
program uses a relative pathname, zlibc uses the most direct path to the
file (i.e. without symlinks).

@item suffix
The data file's name must end in @var{name}.  This is useful for
selecting files according to their extension.

@item filesystem
The data file must live on the same filesystem as @var{name}. This
criterion can for example be used for example to disable zlibc on a
doublespaced filesystem (where zlibc would be redundant), or to switch
off uncompressed size reporting on an @code{ftpfs} filesystems
@footnote{uncompressed size reporting (@pxref{Size} would make @code{ls}
painfully slow on an @code{ftpfs} filesystem, as all files would have to
be downloaded before their size could be reported.}.

@item default
All data files match. All class definitions must have a default line,
and this default line must be the last line of the definition. The
default criterion needs no 'name' parameter.

@item all
All data files match. Unlike @code{default}, this line needs not to be
the last line of the class definitions. Thus it is possible to specify
several @code{all} lines for different aspects of @code{zlibc}
behaviour.
@end table

The lines of each class definition are scanned in sequence, and, for
each aspect@footnote{See @pxref{Mode}}, the first matching line is
adopted.  The class may be defined in another initialisation file, if
this file is scanned later. The @var{criterion} parameter may be omitted
if it can be deduced from the @var{name}. In that case, the following
heuristics are used:

@itemize @bullet
@item
@var{Name}s beginning with a dot are considered as @code{suffix}.

@item
@var{Name}s beginning with a slash, but not ending with a slash are
considered as @var{filename}.

@item
@var{Name}s beginning and ending with a slash are considered as
@code{subdirectory}.

@item
A missing @var{name} is considered as @code{default}.

@item
All the other @var{name}s are considered as @code{basename}.
@end itemize

@c With the exception of the @code{filesystem} criterion, these selection
@c criteria are applied on the filename @emph{as it was provided by the
@c user program}.  Names might not match if they are accessed by a
@c different path than the one specified in the config file. This is
@c especially true for criteria using the directory part of a
@c filename. Whenever possible, use the @code{suffix}, @code{basename} or
@code{filesystem} criteria, because these tend to be more predictable.

@node Mode, , Criteria, Class section
@subsubsection Mode

The @var{mode} describes the behavior of zlibc in certain
situations. These are:
@enumerate
@item
What to do when a file is accessed in readonly mode.
@item
What to do when a user program tries to append data to an existing file.
@item
What to do when a user program tries to create a new file.
@item
What to do when a user program tries to access an existing file and none
of the preceding situations apply (@pxref{Write}).
@item
Whether to show the size of the uncompressed file or the size of the
compressed file as a response to stat (@pxref{Size}).
@end enumerate

The mode for each off these 5 situations has to be described separately.
For each of these 5 situations, the scan through the class description
is done until the reaction of zlibc for this situation is defined.


@menu
* Readonly::     Behavior for programs wanting to only read files
* Append::              - for programs wanting to add more data to
                          existing files
* Creation::            - for programs wanting to create new files
* Write::               - for programs wanting to apply other
                          modifications to files
* Size::                - for programs wanting to know the size of a 
                          file
@end menu


@node Readonly, Append, Mode, Mode
@unnumberedsubsubsec Reaction to a readonly request

The reaction to a @code{readonly} request may be one of the following:

@table @code

@item show-pipe
The uncompressed data file is sent to the user program using a
pipe. This consumes very few resources, and it allows the decompression
to run in paralell with the user process, but it has the disadvantage
that the user program cannot use lseek.  To warn the user program of
this, the data file is shown as a named pipe (FIFO) when it is
stat'ed.

@item use-tmp-file
This is the default setting.  The data is uncompressed and put into a
temporary file. The user programs then reads its data from the temporary
file. This has the advantage that the user program may lseek, and the
disadvantage that more disk space is consumed, while the programming is
accessing the file.

@item hide-pipe
The data is sent through a pipe, but the file is shown as a regular file
("hidden") to the user program when it stats it. This might be needed
for programs which are picky about a file type, but who actually don't
need lseek.

@item leave-compressed
The virtual (uncompressed) file is shown as non-existent to stat, and
readdir shows the physical (compressed) file. For certain programs, this
is enough to disable zlibc on a file per file basis. This is useful, for
example, to make emacs use its own compression support (crypt.el)
instead of zlibc. Crypt.el is able to compress files when writing them
back, whereas zlibc isn't able to do so. The @code{leave-compressed}
doesn't work correctly with the @code{directory} and @code{subdirectory}
criteria.  Use the @code{filesystem} criterion instead.

@item dir-leave-compressed
The virtual (uncompressed) file is shown to stat, but readdir shows the
physical (compressed) file. This is useful to tell zlibc that we prefer
working on the physical file, but without making access to the virtual
file impossible. The @code{dir-leave-compressed} doesn't work with the
@code{dir} and @code{subdir} criteria. Use the @code{filesystem}
criterion instead.

@end table

@node Append, Creation, Readonly, Mode
@unnumberedsubsubsec Reaction to an append request

The reaction to an @code{append} request may be one of the following:

@table @code
@item append-compressed
When a user program tries to append data to a non-existant file, but the
corresponding compressed file exists, zlibc translates this request into
appending the compressed data to the compressed file.

@strong{WARNING-1:}

This works with gzip, and might not work with other (un)compressors!
This relies on gzip's feature to consider a concatenation of compressed
files as a compression of concatenated files.

@strong{WARNING-2:}

This is only reliable if you can guarantee that the file is not accessed
by some other program while the first one has it still open, or that it
won't be opened again (even by the same program) shortly after it has
been closed. Delays longer than a second should be ok. Don't enable
@code{append-compressed} if you expect the file to be written to by
several programs at once.

@item no-append-compressed
Don't append to a compressed file (the user program will get a file not
found error). This is the default behavior.

@end table

@node Creation, Write, Append, Mode
@unnumberedsubsubsec Reaction to a file creation request

The reaction to a @code{create} request may be one of the following:

@table @code
@item create-compressed
When a user program tries to create a new file whose name matches the
pattern, zlibc translates this request into creating a compressed
file. A file is considered to be created, if the @code{O_TRUNC} flag is
set, or if both @code{O_CREAT} and @code{O_EXCL} are set. This only
applies to files opened write-only.

@strong{WARNING-1:}

This is only reliable if you can guarantee that the newly
created file is not accessed by some other program while the first one
has it still open, or that it won't be opened again (even by the same
program) shortly after it has been closed.

@strong{WARNING-2:}

This should not be used if you expect the program to seek in this
file.

@item no-create-compressed
Newly created files are created uncompressed. This is the default.
@end table

@node Write, Size, Creation, Mode
@unnumberedsubsubsec Other write requests

The reaction to other @code{write} requests may be one of the following:

@table @code
@item uncompress-before-write
When a user program tries to write to a non-existing file, and when a
compressed file with a corresponding name exists, this file is
uncompressed in place (i.e. having the same name, but without the .gz
extension)

@strong{WARNING-1:}

This is only safe when you can guarantee that the file is not opened by
several programs at once. However, once the call to the open function
has returned, other programs may open this file safely. If a second
program tries to open the file during the open call of the first, this
second program gets a permission error.

@strong{WARNING-2:}

When using this option, be careful when opening files belonging to
another user, or files living in a directory where you have no write
access to. Using this option in a directory without write access will
result in a permission error. Using this option in a directory where you
do have write access will change the ownership of the file to you, even
if it belonged to another user initially.

@item no-uncompress-before-write
Compressed files are not uncompressed before writing to them, and zlibc
returns a "file not found" error. This is the default.

@end table

If several of these options apply for the same file,
@code{create-compressed} has priority over @code{append-compressed}
which has priority over @code{uncompress-before-write}.

@node Size, , Write, Mode
@unnumberedsubsubsec Size shown for compressed files
@cindex find
@cindex size of compressed files
@cindex slowness (of find due to reading the size)
@vindex show-compressed-size
@cindex ftpfs
@cindex ls slowness (on an ftpfs filesystem)
@cindex slowness (of ls, on an ftpfs filesystem)
@cindex hanging (of ls, on an ftpfs filesystem)


When an application calls @code{stat} to fetch the attributes of a file
(such as its permissions, size, type, etc.), zlibc stats the
corresponding compressed file instead.  After doing so, zlibc has to
adjust some of the values returned by stat, such as the type and the
size.  The type has to be adjusted for those files that should be shown
as pipes.  The size has to be adjusted because user programs are usually
interested in the amount of data that they can actually read from the
file (i.e. the size of the uncompressed file) rather than the size of
the physical file (i.e. the size of the compressed file).  However, in
order to find out the size of the uncompressed file, zlibc has to read
some data of the file, which may impact performance in situations where
many files are stat'ed.  This is for instance the case for @code{find},
or for @code{ls} on an @code{ftpfs} filesystem.  The following two
behaviors of the stat call may be specified:

@table @code
@item show-compressed-size
Stat returns the size of the compressed file.  This is less useful for
the application, but more efficient.  If you mount any @code{ftpfs}
filesystems, you may switch on @code{show-compressed-size} just for that
filesystem by using the @code{filesystem} criterion (@pxref{Criteria}).

@item show-uncompressed-size
Stat returns the size of the uncompressed file.  This is more useful for
the application, but less efficient.  This is the default behavior.

@end table

@ignore
@subsection See also
zlibc(3)
@end ignore

@c zlibc.3end-skip
@c zlibc-conf.5skip

@node Compiled-in defaults, Compile-time configuration, Configuration files, Customization
@section Compiled-in defaults

It is possible to operate zlibc entirely without configuration files.
In this case, it uses the @emph{compiled-in defaults}.  These are
generated at compile-time from the @file{zlibrc.sample} file.  This file
has the same syntax as the configuration files described above
(@pxref{Configuration files}).  If you want to change the compiled-in
defaults of zlibc, edit that file, and remake.

@node Compile-time configuration, , Compiled-in defaults, Customization
@section Compile-time configuration via GNU autoconf
@cindex configure options
@cindex temporary file directory
@cindex directory for temporary files
@cindex compile-time configuration

Before it can be compiled, zlibc must be configured using the GNU
autoconf script @code{./configure}.  In most circumstances, running
@code{./configure} without any parameters is enough. However, you may
customize zlibc using various options to @code{./configure}. The
following options are supported:
@table @code

@item --prefix @var{directory}
Prefix used for any directories used by zlibc.  By default, this is
@file{/usr/local}.  Zlibc is installed in @file{$prefix/lib}, looks for
its system wide configuration file in @file{$prefix/etc}.  Man pages are
installed in @file{$prefix/man}, info pages in @file{$prefix/info} etc.
On Linux, if you use zlibc via @file{/etc/ld.so.preload}, you should use
@file{/} as the prefix instead of the default @file{$prefix/lib}.

@item --sysconfdir @var{directory}
Directory containing the system-wide configuration file
@file{zlibc.conf}.  By default, this is derived from @code{prefix} (see
above).

@item --disable-runtime-conf
Disables run time configuration via environmental variables and via the
configuration files.  This may be needed in hyper secure environments.

@item --disable-env-conf
Disables run time configuration via environmental variables

@item --disable-have-proc
Tells zlibc not to use the /proc filesystem to find out the commandline
of the programs for which it runs, even if a working /proc is detected.

@item --disable-have-proc
Tells zlibc to use the /proc filesystem to find out the commandline of
the programs for which it runs, even if no working /proc is detected.

@item --with-compr-ext=@var{extension}
Uses @var{extension} as the filename extension of compressed files.  By
default, is @code{.gz}

@item --with-extlen=@var{length}
Allows to configure compressed filename extensions with at most
@var{length} character via runtime configuration.  By default is 5.

@item --with-tmpdir=@var{directory}
Uses @var{directory} to store the uncompressed files.  By default is
@code{/tmp}.  

@item --with-uncompressor=@var{uncompressor-command-line}
Defines how the program for uncompressing files should be invoked.  This
command should read the compressed file from stdin, and output the
uncompressed data to stdout By default is @code{gzip -dc}.

@end table

In addition to the above-listed options, the standard GNU autoconf
options apply.  Type @code{./configure --help} to get a complete list of
these.

@ignore
@subsection See also
zlibc.conf(5)
@end ignore

@c zlibc-conf.5end-skip
@c MANskip

@node Porting zlibc, Variable Index, Customization, Top
@chapter Porting zlibc

Zlibc has been tested on three variants of Unix so far: Linux, SunOs,
Solaris.  On all three platforms, zlibc has been compiled using GNU make
and gcc.  However, porting it to other platforms should be
straightforward as long as they verify both of the following conditions:

@enumerate
@item
The target platform should have shared library support.  The default
Makefile links zlibc by supplying the flags @code{-nostdlib -shared} to
gcc in addition to the usual @code{CFLAGS}.  If your target platform
needs different flags to make shared libraries, change the definition of
the variable @code{SHAREDCFLAGS} in the Makefile.
@item
The target platform should supply a way to perform system calls
directly without going through the library functions.  Indeed, this is
needed because we are redefining the library functions and thus the
default functions become unavailable.

Usually direct syscalls done by calling the syscall function.  For
instance, in order to perform @code{open("test", O_RDONLY)} directly,
@code{sycall(SYS_open, "test", O_RDONLY)} is called.

If your target platform uses a different way to perform direct
system calls, you need to supply different definitions for the real_xxx
functions in direct_sys.h.  This is the case for AIX.
@item
The target platform should provide a way to instruct the dynamic linker
to preload programs with extra object files not present in the shared
library.  Any symbols defined in that object file override equivalent
symbols from libc.  Usually this is done by pointing the
@code{LD_PRELOAD} variable to the object file to be used.
Unfortunately, I have found no way to achieve this on AIX.  Any ideas
about how to do this are welcome.
@end enumerate

Many platforms, including Solaris, provide several aliases for their
syscall stubs.  It would be interesting to leave one of them alone, and
use it as a "direct syscall" stub, but unfortunately, it turns out that
in sometimes both stubs are used by the library!  Thus, what would be an
advantage, becomes actually a disadvantage, because we will need to
override these aliases in addition to the canonical names.  This is done
in altnames.c.  Change it as needed.  In order to know the names of
these aliases, try the following command: @code{nm /lib/libc.so | grep
lstat}.  This lists any library symbols with @code{unlink} in their
name, which could be aliases.

If you have successfully ported zlibc to a new platform, could you drop
me a note, so that I can include support for that platform in my next
release.  Thanks

@node Variable Index, Concept Index, Porting zlibc, Top
@unnumbered Variable index
@printindex vr

@node Concept Index, , Variable Index, Top
@unnumbered Concept index
@printindex cp

@contents

@c MANend-skip

@bye