1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<TITLE>jigdo-file</TITLE><META NAME="GENERATOR" CONTENT="Modular DocBook
HTML Stylesheet Version 1.7"></HEAD>
<BODY CLASS="REFENTRY" BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#0000FF"
VLINK="#840084" ALINK="#0000FF">
<H1><A NAME="AEN1"></A>jigdo-file</H1>
<DIV><A NAME="AEN14"></A>
<H2>Name</H2>jigdo-file -- Prepare files for Jigsaw Download
(distribution of huge files, e.g. CD images).</DIV>
<DIV><A NAME="AEN17"></A>
<H2>Synopsis</H2>
<P><B>jigdo-file</B> { <VAR> COMMAND</VAR> } [<VAR>--image=<VAR
>cdrom.iso</VAR></VAR>] [<VAR>--jigdo=<VAR>cdrom.jigdo</VAR></VAR>]
[<VAR>--template=<VAR>cdrom.template</VAR></VAR>] [<VAR>--force</VAR>]
[MORE OPTIONS] [<VAR>FILES</VAR> | <VAR>--files-from=<VAR>f</VAR></VAR
>]<BR> Common COMMANDs: make-template, make-image, verify </P></DIV>
<DIV><A NAME="DESCRIPTION"></A>
<H2>DESCRIPTION</H2>
<P>Jigsaw Download, or short jigdo, is a scheme developed primarily to
make it easy to distribute huge filesystem images (e.g. CD (ISO9660) or
DVD (UDF) images) over the internet, but it could also be used for
other data which is awkward to handle due to its size, like audio/video
files or large software packages.</P>
<P>jigdo tries to ensure that the large file (always called <I>image</I
> from now on) is downloaded in small parts which can be stored on
different servers. People who want to download the image do so by
telling the <SPAN><SPAN>jigdo</SPAN>(1)</SPAN> <SPAN><I>(NOT
IMPLEMENTED YET)</I></SPAN> download tool to process one `<TT
>.jigdo</TT>' file; using it, <B>jigdo</B> downloads the parts and
reassembles the image. <B>jigdo-file</B> is used to prepare the files
for download.</P>
<P>What makes jigdo special is that the parts that are used to
reconstruct the image can have any size and content - they only need to
be contained in a contiguous region anywhere in the image.</P>
<P>For example, if you wish to distribute an ISO9660 image which
contains a snapshot of an FTP server, you can instruct <B
>jigdo-file</B> to prepare the download data in such a way that when
people use <B>jigdo</B> to download the image, <B>jigdo</B> actually
fetches the individual files from the FTP server and assembles them
into an exact copy of your image - during the download! (If the image
is not a filesystem dump, you can use <SPAN><SPAN>split</SPAN
>(1)</SPAN> to create the small parts that the image will be
reassembled from.)</P>
<P>You are completely free to choose where the individual parts of the
image are stored: They may be in entirely different directories on
different servers (e.g. because of storage/bandwidth constraints), but
this is invisible to the people downloading your image. The information
about available servers only needs to be added to the `<TT>.jigdo</TT
>' file by you before distributing it.</P>
<P>The `DETAILS' section below contains technical details on how jigdo
works. The `EXAMPLES' section lists a number of common scenarios and
may help you to get an idea of what jigdo is useful for.</P></DIV>
<DIV><A NAME="OPTIONS"></A>
<H2>OPTIONS</H2>
<P>Many options are specific to a particular <VAR>COMMAND</VAR>; the
ones below are general or used by several commands. Further options are
listed below with the individual commands. All options are silently
ignored if they are not applicable to the current command. For any <VAR
>BYTES</VAR> parameters to options, you can append one of the letters
`k', `M' or `G' to the amount you specify, to indicate kilobytes,
megabytes or gigabytes.</P>
<P></P>
<DIV>
<DL>
<DT><VAR>-h</VAR> <VAR>--help</VAR></DT>
<DD>
<P>Output short summary of commands and options.</P></DD>
<DT><VAR>-H</VAR> <VAR>--help-all</VAR></DT>
<DD>
<P>Output complete summary of commands and options.</P></DD>
<DT><VAR>-v</VAR> <VAR>--version</VAR></DT>
<DD>
<P>Output program version.</P></DD>
<DT><VAR>-i</VAR> <VAR>--image=<VAR>cdrom.iso</VAR></VAR></DT>
<DD>
<P>Specify location of the file containing the image. The image is
the large file that you want to distribute.</P></DD>
<DT><VAR>-j</VAR> <VAR>--jigdo=<VAR>cdrom.jigdo</VAR></VAR></DT>
<DD>
<P>Specify location of the Jigsaw Download description file. The
jigdo file is a human-readable file generated by <B>jigdo-file</B>,
to which you add information about all the servers you are going to
upload the files to. <B>jigdo</B> will download this file as the
first step of retrieving the image.</P></DD>
<DT><VAR>-t</VAR> <VAR>--template=<VAR>cdrom.template</VAR></VAR></DT
>
<DD>
<P>Specify location of the image `template' file. The template file
is a binary file generated by <B>jigdo-file</B>, it contains
information on how to reassemble the image and also (in compressed
form) all the data from the image which was not found in any of the
parts.</P>
<P>Depending on the command, each of these three files is used
sometimes for input, sometimes for output. If the file is to be used
for output for a particular command and the output file already
exists, <B>jigdo-file</B> exits with an error, unless <VAR
>--force</VAR> is present.</P>
<P>In most cases, you will only need to specify one out of <VAR
>-i</VAR> <VAR>-j</VAR> <VAR>-t</VAR>, because any missing filenames
will be deduced from the one you specify. This is done by first
stripping any extension from the supplied name and then appending
nothing (if deducing <VAR>--image</VAR>), `<TT>.jigdo</TT>' or `<TT
>.template</TT>'.</P></DD>
<DT><VAR>-r</VAR> <VAR>--report=default|noprogress|quiet|grep</VAR
></DT>
<DD>
<P>Control how verbose the program is, and what format the output
has: <VAR>noprogress</VAR> is the same as <VAR>default</VAR> except
that no `<SAMP>x% done</SAMP>' progress messages are printed. <VAR
>quiet</VAR> restricts the output to what is absolutely necessary,
mostly error messages. <VAR>grep</VAR> is only different from <VAR
>default</VAR> for the <B>make-template</B> command: It enables
output in a simple `<VAR><offset> <file></VAR>' format
which is useful when searching for binary files in other binary
files.</P></DD>
<DT><VAR>-f</VAR> <VAR>--force</VAR></DT>
<DD>
<P>Overwrite existent output files without complaining.</P></DD>
<DT><VAR>--no-force</VAR></DT>
<DD>
<P><SPAN><I>This is the default.</I></SPAN> Refuse to overwrite
existent output files.</P></DD>
<DT><VAR>-c</VAR> <VAR>--cache=<VAR>jigdo-cache.db</VAR></VAR></DT>
<DD>
<P><B>jigdo-file</B> usually needs to read the entire contents of all
the <VAR>FILES</VAR> you specify. If you use it repeatedly (e.g.
because you make a new CD image available daily), caching the file
information will increase the program's speed significantly. The
cache file is automatically created if it is not yet present. Data
is usually both read from and written to it.</P></DD>
<DT><VAR>--no-cache</VAR></DT>
<DD>
<P><SPAN><I>This is the default.</I></SPAN> Do not use a cache.</P
></DD>
<DT><VAR>--cache-expiry=<VAR>SECONDS</VAR></VAR></DT>
<DD>
<P>Set maximum age of cache entries. Any entries older than this will
be removed from the cache. The default is 30 days. You can append
one of the letters `h', `d', `w', `m', `y' to denote hours, days,
weeks, months or years, respectively. A value of `0' or `off'
disables expiry, so that all entries will stay in the cache forever.
See the section `CACHE FILES' below for more information.</P></DD>
<DT><VAR>--readbuffer=<VAR>BYTES</VAR></VAR></DT>
<DD>
<P>Set size of internal buffers. The default is 128k - if you have a
fast disc, increasing this value may make <B>jigdo-file</B> faster,
but in general, changing it is not necessary.</P></DD>
<DT><VAR>--md5-block-size=<VAR>BYTES</VAR></VAR></DT>
<DD>
<P><SPAN><I>Uninteresting internal parameter.</I></SPAN> Set size of
blocks into which files are subdivided. The default is 128k. If you
change it, any cache file will have to be regenerated. Internally,
<B>jigdo-file</B> may choose to use a slightly larger or smaller
value.</P></DD>
<DT><VAR>-T</VAR> <VAR>--files-from=<VAR>file</VAR></VAR></DT>
<DD>
<P>Read file and directory names from the specified file. If <VAR
>file</VAR> is `-', read names from standard input. Each line in the
file is taken as a name, so the names may contain spaces, but not
newline characters. An empty line causes <B>jigdo-file</B> to stop
reading from the file.</P>
<P><SPAN><SPAN>find</SPAN>(1)</SPAN> is a powerful tool for
generating file lists, but make sure to use `<B>find -type f</B>' if
possible - otherwise, if you instruct <B>find</B> to output both a
filename and a symlink to that filename, <B>jigdo-file</B> will read
the file contents twice.</P></DD>
<DT><VAR>--hex</VAR></DT>
<DD>
<P>Output checksums in hexadecimal instead of Base64-like format.
This should not be used with the <B>make-template</B> command,
because the resulting `<TT>.jigdo</TT>' file violates the `<TT
>.jigdo</TT>' file format. Its intended use is to make <B
>jigdo-file</B> more interoperable with other Unix shell utilities
like <SPAN><SPAN>md5sum</SPAN>(1)</SPAN>.</P></DD>
<DT><VAR>--no-hex</VAR></DT>
<DD>
<P><SPAN><I>This is the default.</I></SPAN> Use jigdo's own
Base64-like encoding of checksums.</P></DD>
<DT><VAR>--debug</VAR>[<SPAN>=<VAR>help</VAR>|=<VAR>all</VAR>|=<VAR
>UNIT,~UNIT...</VAR> </SPAN>]</DT>
<DD>
<P>Switch on or off debugging output. Just `--debug' is equivalent to
`--debug=all'. The argument is a comma-separated list of unit names
for which debugging output is to be enabled, or disabled if the name
is preceded by `~'. The special name `all' means all units. By
default, debugging output is switched off except for the units
`assert' and `general'. The exact list of available units for which
debugging can be switched on depends on whether jigdo was compiled
with debugging support - the list can be printed with
`--debug=help'.</P></DD>
<DT><VAR>FILES</VAR></DT>
<DD>
<P>Names of files or directories to use as input. These are the parts
that are contained in the image. In case one of the names is a
directory, the program recursively scans the directory and adds all
files contained in it. While doing this, it follows symbolic links,
but avoids symlink loops.</P>
<P>If one of the filenames starts with the character `-', you must
precede the list of files with `--'. A value of `-' has <SPAN><I
>no</I></SPAN> special meaning in this list, it stands for a file
whose name is a single hyphen.</P></DD></DL></DIV></DIV>
<DIV><A NAME="COMMANDS"></A>
<H2>COMMANDS</H2>
<P>The command name is the first non-option argument passed to <B
>jigdo-file</B>. Most commands have short abbreviations as well as long
names. <SPAN><I>The short command names should not be used in scripts -
there may be incompatible changes to them in the future!</I></SPAN></P
>
<DIV><A NAME="MAKE-TEMPLATE"></A>
<H3><B>make-template</B>, <B>mt</B></H3>
<P>Reads <VAR>image</VAR> and <VAR>FILES</VAR>, creates `<TT
>.jigdo</TT>' and `<TT>.template</TT>'. This is the main functionality
of <B>jigdo-file</B>.</P>
<P>It is possible to specify both <VAR>--image=-</VAR> and <VAR
>--files-from=-</VAR>. In this case, first the list of files is read
from standard input until an empty line is encountered. Everything
following it is assumed to be the image data. This can be useful if
you use <SPAN><SPAN>mkisofs</SPAN>(1)</SPAN> or similar programs that
can output the complete image on their standard output, because there
is no need to store the image on disc temporarily.</P>
<P>If a <VAR>FILES</VAR> argument contains the characters `<VAR
>//</VAR>' (Unix) or `<VAR>\.\</VAR>' (Windows), this has special
meaning. In the final jigdo file that users will download, each of the
parts is referenced in the `<VAR>[Parts]</VAR>' section with a URI of
the form `Label:some/filename'. (See `FORMAT OF .JIGDO FILES' below
for a detailed description.) The `<VAR>[Servers]</VAR>' section gives
a mapping of labels to servers on the internet, with lines like
`Label=http://myserver.org/jigdofiles/'. Using this information, <B
>jigdo</B> will create the final download URI for the part,
`http://myserver.org/jigdofiles/some/filename'. Specifying `<VAR
>//</VAR>' (or `<VAR>\.\</VAR>') in a file or directory name serves to
`cut off' the names at the right directory level. For example, if the
Unix path of one of your <VAR>FILES</VAR> is `/path/some/filename',
you can tell <B>jigdo-file</B> to cut off after the `/path' by passing
it the argument `/path//some/filename', or `/path//' if you want the
whole directory scanned. The path names need not be absolute;
`somedirectory//' is also possible.</P>
<P></P>
<DIV>
<DL>
<DT><VAR>--label <VAR>Label=/path</VAR></VAR></DT>
<DD>
<P>Specify a name to use as the label name for a path on disc.
(Influences the output jigdo file.) If you used `<VAR>//</VAR>' in
the <VAR>FILES</VAR> arguments as described above, <B
>jigdo-file</B> will by default pick label names automatically
(`A', `B' etc.). With this option, you can give labels more
meaningful names. Note that the label name will only be used if one
or more <VAR>FILES</VAR> begin with `/path//'.</P>
<P>Try to use label names that start with uppercase characters, to
disambiguate them clearly from protocol names like `http',
`ftp'.</P></DD>
<DT><VAR>--uri <VAR>Label=http://some.server.org/</VAR></VAR></DT>
<DD>
<P>By default, using <VAR>--label</VAR> as described above will
cause lines of the form `Label=file:/path/' to be written to the
`<VAR>[Servers]</VAR>' section of the output jigdo file. If you
want to override the `file:' URI so that the line reads
`Label=http://some.server.org/', you can do so by specifying <VAR
>--uri</VAR> along with <VAR>--label</VAR>. Giving just <VAR>--uri
<VAR>Label=...</VAR></VAR> without the corresponding <VAR>--label
<VAR>Label=...</VAR></VAR> has no effect, and even if you specify
both, an entry is only added to the `<VAR>[Servers]</VAR>' section
if the label is referenced by at least one `<VAR>[Parts]</VAR>'
entry.</P>
<P>The supplied value is not quoted by the program; if it contains
characters such as space or any of the characters <VAR>#"'\</VAR>
then you must quote it. (Under Unix, you may need to quote the
value twice to also protect it from the shell, e.g. <VAR>\\\\</VAR
> or <VAR>'\\'</VAR> to get a single backslash in the URI.)</P>
<P>The mapping specified with an <VAR>--uri</VAR> option is ignored
if it is already present in the output jigdo file.</P>
<P>Users of the Windows version may notice that the `<VAR>\</VAR>'
directory separators are converted into `<VAR>/</VAR>' in the
`file:' URIs that are generated by default. This is done to
increase cross-platform compatibility of `file:' - the <B
>print-missing</B> command of the Windows version will
automatically re-convert the characters when it prints the URIs. In
case you supply your own `file:' URIs under Windows using <VAR
>--uri</VAR>, you must also exchange `<VAR>/</VAR>' and `<VAR
>\</VAR>'.</P></DD>
<DT><VAR>-0</VAR> to <VAR>-9</VAR></DT>
<DD>
<P>Set amount of compression in the output template file, from <VAR
>-0</VAR> (no compression) to <VAR>-9</VAR> (maximum compression).
The default is <VAR>-9</VAR>, which can make the template
generation quite slow. The compression algorithm used is the same
as for <SPAN><SPAN>gzip</SPAN>(1)</SPAN>.</P></DD>
<DT><VAR>--min-length=<VAR>BYTES</VAR></VAR></DT>
<DD>
<P>Set minimum length of a part for <B>jigdo-file</B> to look for it
in the image. The default is 1k. Parts smaller than this will never
be found in the image, so their data will be included in the
template file. The search algorithm used requires such a minimum
length, otherwise template generation could become extremely slow.
If you know for sure that all your <VAR>FILES</VAR> are larger than
a certain amount, you can increase <B>jigdo-file</B>'s speed
slightly by specifying the amount with this option. There is a
hard-wired absolute minimum of 256 bytes - anything lower will
silently be set to 256.</P></DD>
<DT><VAR>--merge=<VAR>FILE</VAR></VAR></DT>
<DD>
<P>Include the contents of <VAR>FILE</VAR> in the output `<TT
>.jigdo</TT>' file. The file can contain data which you want added
to the output (for example, a `<VAR>[Servers]</VAR>' section with a
list of your servers as entries), or it can be the jigdo file
output by an earlier run of <B>jigdo-file</B>.</P>
<P>It is possible to specify the same file for input with <VAR
>--merge</VAR> and for output with <VAR>--jigdo</VAR>. However, you
will also need to use <VAR>--force</VAR> to make the program
overwrite the old version of the jigdo file with the new one. <VAR
>FILE</VAR> can be `-' for standard input.</P>
<P>When <SPAN><I>adding</I></SPAN> new information to the supplied
file, <B>jigdo-file</B> will not insert new lines into the `<VAR
>[Parts]</VAR>' section if an entry for the same MD5 checksum (but
not necessarily with the same URI!) already exists, and it will not
insert new lines into the `<VAR>[Servers]</VAR>' section if a
completely identical entry already exists.</P>
<P>When <SPAN><I>reading in</I></SPAN> the existing <VAR>FILE</VAR
>, the behaviour is slightly different: The program <SPAN><I
>preserves</I></SPAN> entries in the `<VAR>[Parts]</VAR>' section
with identical checksum, but different URIs. For completely
identical entries (same checksum and URI), only one entry is
preserved and the duplicates are removed. The `<VAR>[Servers]</VAR
>' section is left untouched.</P></DD>
<DT><VAR>--image-section</VAR></DT>
<DD>
<P><SPAN><I>This is the default.</I></SPAN> Causes <B>jigdo-file</B
> to add an `<VAR>[Image]</VAR>' section to the `<TT>.jigdo</TT>'
file.</P>
<P>As an exception, a new `<VAR>[Image]</VAR>' section is <SPAN><I
>not</I></SPAN> added if you use <VAR>--merge</VAR> and the file to
merge contains an `<VAR>[Image]</VAR>' section with a line which
reads `<VAR>Template-MD5Sum=</VAR>' (end of line after the `='). In
this case, the generated template data's MD5 checksum value is just
added after the `=' of the first line of this form in the file - no
whole new `<VAR>[Image]</VAR>' section is appended. This behaviour
is useful because it allows you to pass via <VAR>--merge</VAR> an
`<VAR>[Image]</VAR>' section with arbitrary content and then have
the MD5 checksum automatically added by <B>jigdo-file</B>. The
section `FORMAT OF .JIGDO FILES' below explains the `<VAR
>[Image]</VAR>' section contents in greater detail.</P></DD>
<DT><VAR>--no-image-section</VAR></DT>
<DD>
<P>Do <SPAN><I>not</I></SPAN> include an `<VAR>[Image]</VAR>'
section in the `<TT>.jigdo</TT>' file. You need to add one yourself
if you use this option. However, doing that is not easy (you also
need to add a `<VAR>Template-MD5Sum</VAR>' line with the correct
checksum, or <B>jigdo</B> will complain), so use of this option is
discouraged.</P></DD>
<DT><VAR>--servers-section</VAR></DT>
<DD>
<P><SPAN><I>This is the default.</I></SPAN> Causes <B>jigdo-file</B
> to add a `<VAR>[Servers]</VAR>' section to the `<TT>.jigdo</TT>'
file. This default section uses `file:' URIs, which allows for
immediate reassembly of the image from the local filesystem, and is
also useful if you want to edit the file manually and replace the
`file:' URIs with other URIs.</P></DD>
<DT><VAR>--no-servers-section</VAR></DT>
<DD>
<P>Do <SPAN><I>not</I></SPAN> add a `<VAR>[Servers]</VAR>' section
at the end of the `<TT>.jigdo</TT>' file. Useful e.g. if you are
going to append the section with a script.</P></DD>
<DT><VAR>--match-exec=<VAR>SHELLCOMMAND</VAR></VAR></DT>
<DD>
<P>Whenever a file is found in the image, execute the supplied
command string by passing it to a shell. <B>jigdo-file</B> sets up
a number of environment variables with information about the file
match. For example, if the file `<TT>/path//a/b/file</TT>' was
found in the image and `Label:a/b/file' is going to be written to
the `<TT>.jigdo</TT>' file:
<P></P><UL><LI>
<P><VAR>LABEL</VAR>="<VAR>Label</VAR>" - Name of the label for the
file. The example assumes that `<VAR>--label</VAR> <VAR
>Label=/path</VAR>' was specified by you. In the absence of such
an option, <VAR>LABEL</VAR> will be set but empty.</P></LI><LI>
<P><VAR>LABELPATH</VAR>="<VAR>/path/</VAR>" - The path
corresponding to the label, or in other words, the prefix of the
matched file's path that will <SPAN><I>not</I></SPAN> appear in
the output `<TT>.jigdo</TT>' file. Is set even without any `<VAR
>--label</VAR>' option present. Ends with a slash.</P></LI><LI>
<P><VAR>MATCHPATH</VAR>="<VAR>a/b/</VAR>" - The rest of the path,
without the leafname of the matched file. Is either empty or ends
with a slash.</P></LI><LI>
<P><VAR>LEAF</VAR>="<VAR>file</VAR>" - The leafname of the matched
file.</P></LI><LI>
<P><VAR>MD5SUM</VAR>="<VAR>lNVdUSqbo2yqm33webrhnw</VAR>" - The
md5sum of the matched file, in Base64-like format.</P></LI><LI>
<P><VAR>FILE</VAR>="<VAR>/path//a/b/file</VAR>" - For convenience,
the complete path of the file. The variable is always set to <VAR
>$LABELPATH$MATCHPATH$LEAF</VAR>.</P></LI></UL> </P>
<P>Please be careful to correctly quote the string passed to this
option, otherwise your supplied command will not work with
filenames that contain spaces. As an example, to create a backup of
hard links to the matched files, use the following option: <VAR
>--match-exec='mkdir -p "${LABEL:-.}/$MATCHPATH" && ln -f
"$FILE" "${LABEL:-.}/$MATCHPATH$LEAF"'</VAR> </P>
<P>By default, no command is executed. Use --match-exec="" to remove
a command string which was set with an earlier use of this
option.</P></DD></DL></DIV></DIV>
<DIV><A NAME="MAKE-IMAGE"></A>
<H3><B>make-image</B>, <B>mi</B></H3>
<P>Reads `<TT>.template</TT>' and <VAR>FILES</VAR>, creates <VAR
>image</VAR> (or `<TT>imagename.tmp</TT>'). Provides a rudimentary way
of reassembling images - <B>jigdo</B> is usually better suited for
this task. However, in contrast to <B>jigdo</B>, no `<TT>.jigdo</TT>'
file is required.</P>
<P>If the image is to be written to a file (and not to standard
output), it is possible to create the image in several steps, with
several invocations of `<B>jigdo-file make-image</B>', as follows: You
first invoke <B>jigdo-file</B>, specifying as many files as are
available at this time. The program scans the files, and those that
are contained in the image are copied to a temporary file, whose name
is formed by appending `<TT>.tmp</TT>' to the image filename.</P>
<P>For all further files which could be parts of the image, you repeat
this process. As soon as all parts are present, the temporary file
will be truncated slightly (to delete some administrative data that <B
>jigdo-file</B> appends at the end) and renamed to the final image
name. The possibility of reassembling the image in several steps is
especially useful for gathering files from removable media, e.g.
several older CDs.</P>
<P>Scripts using <B>make-image</B> can detect whether image creation is
complete by checking the exit status: 0 signals successful creation,
whereas 1 means that more files need to be supplied. Other errors
result in an exit status of 2 (`recoverable', e.g. file not found) or
3 (non-recoverable, e.g. write error).</P>
<P></P>
<DIV>
<DL>
<DT><VAR>--check-files</VAR></DT>
<DD>
<P><SPAN><I>This is the default.</I></SPAN> Whenever any part is
copied to the image, re-check its checksum against the checksum
stored in the template. It is recommended that you leave this
switched on, even if it slows down image creation a bit.</P></DD>
<DT><VAR>--no-check-files</VAR></DT>
<DD>
<P>Do not check files' checksums when copying them to the image.
This can be safely used when no cache file is used (which means
that files will be written to the image immediately after being
scanned) or the whole image is checked later with the <B>verify</B
> command.</P></DD></DL></DIV></DIV>
<DIV><A NAME="PRINT-MISSING"></A>
<H3><B>print-missing</B>, <B>pm</B></H3>
<P>Reads `<TT>.jigdo</TT>', `<TT>.template</TT>' and (if present) `<TT
>imagename.tmp</TT>', outputs a list of URIs still needed to
completely reassemble the image.</P>
<P>Together with the <B>make-image</B> command, this provides most of
the functionality of <B>jigdo</B> on the command line.</P>
<P>For each part that is not yet present in the temporary image file,
the file checksum is looked up in the `<VAR>[Parts]</VAR>' section of
the jigdo file. Any label in the corresponding entry is then expanded
according to the label definitions in the `<VAR>[Servers]</VAR>'
section and printed on standard output. <B>jigdo</B> allows you to
specify several alternative locations for each label in this section,
but <B>print-missing</B> will only output the first one for each
missing part.</P>
<P>If the checksum cannot be found in the `<VAR>[Parts]</VAR>' section
(this Should Not Happen unless you deleted that section), a lookup is
instead made for `MD5Sum:<VAR><checksum></VAR>', just like with
<B>jigdo</B>. (Thus, if you want to get rid of the `<VAR>[Parts]</VAR
>' section, you can do so if you rename each part to its own
checksum.)</P>
<P></P>
<DIV>
<DL>
<DT><VAR>--uri <VAR>Label=http://some.server.org/</VAR></VAR></DT>
<DD>
<P>Override the entries in the `<TT>.jigdo</TT>' file for any label
with a URI of your choice. With the example above, a `<VAR
>[Parts]</VAR>' entry of `Label:some/filename' will cause the line
`http://some.server.org/some/filename' to be printed.</P>
<P>The supplied value is not quoted by the program; if it contains
characters such as space or any of the characters <VAR>#"'\</VAR>
then you must quote it. (Under Unix, you may need to quote the
value twice to also protect it from the shell, e.g. <VAR>\\\\</VAR
> or <VAR>'\\'</VAR> to get a single backslash in the URI.)</P
></DD></DL></DIV></DIV>
<DIV><A NAME="PRINT-MISSING-ALL"></A>
<H3><B>print-missing-all</B>, <B>pma</B></H3>
<P>Just like <B>print-missing</B>, this command outputs a list of URIs
still needed to completely reassemble the image. However, <SPAN><I
>all</I></SPAN> alternative download locations are printed instead of
just one. In the output, the URIs for a file are separated from other
files' URIs with blank lines. The <VAR>--uri</VAR> option has the same
effect as for <B>print-missing</B>.</P></DIV>
<DIV><A NAME="VERIFY"></A>
<H3><B>verify</B>, <B>ver</B></H3>
<P>Reads <VAR>image</VAR> (presumably generated with <B>make-image</B
>) and `<TT>.template</TT>', checks for correct checksum of image.</P
>
<P>The template data does not only contain checksums of the individual
parts, but also of the image as a whole. <B>make-image</B> already
performs a number of internal checks, but if you like, you can
additionally check the image with this command.</P></DIV>
<DIV><A NAME="SCAN"></A>
<H3><B>scan</B>, <B>sc</B></H3>
<P>Reads all the <VAR>FILES</VAR> and enters them into the cache,
unless they are already cached. The <VAR>--cache</VAR> option must be
present for this command.</P></DIV>
<DIV><A NAME="MD5SUM"></A>
<H3><B>md5sum</B>, <B>md5</B></H3>
<P>Reads all the <VAR>FILES</VAR> and prints out MD5 checksums of their
contents. This command is quite similar to <SPAN><SPAN>md5sum</SPAN
>(1)</SPAN>, except that the checksum is output in the Base64-like
encoding which is also used elsewhere by <B>jigdo-file</B>.</P>
<P>The <VAR>FILES</VAR> arguments are processed in the same way as with
the other commands, which means that recursion automatically takes
place for any arguments that are directories, and that symbolic links
are not listed except when the file(s) they point to are not reachable
directly.</P>
<P>In the checksum list printed on standard output, only the part of
the filename following any `<VAR>//</VAR>' (or `<VAR>\.\</VAR>' on
Windows) is printed. Any <VAR>--cache</VAR> will be used for querying
files' MD5 checksums and/or writing the checksums of scanned files.</P
></DIV>
<DIV><A NAME="LIST-TEMPLATE"></A>
<H3><B>list-template</B>, <B>ls</B></H3>
<P>Reads a `<TT>.template</TT>' file and outputs low-level information
about the image and all parts contained in it, including offset,
length and checksum.</P>
<P>You can also use this command with temporary image files (by
specifying something like <VAR>--template=imagename.tmp</VAR>) - in
that case, the output also distinguishes between parts that have been
written to the image and parts that haven't.</P>
<P>The exact output format may change incompatibly between different
jigdo releases. The following different types of lines can be output.
`have-file' only occurs for `<TT>.tmp</TT>' files, indicating a file
that has already been successfully written to the temporary file:</P><PRE>in-template <VAR>offset-in-image length</VAR
>
need-file <VAR>offset-in-image length file-md5sum filestart-rsyncsum</VAR
>
have-file <VAR>offset-in-image length file-md5sum filestart-rsyncsum</VAR
>
image-info <VAR>image-length image-md5sum rsyncsum-size</VAR
></PRE></DIV></DIV>
<DIV><A NAME="DETAILS"></A>
<H2>DETAILS</H2>
<P>Jigsaw Download was created with the format of ISO9660 CD images in
mind - however, the following also applies to many other filesystem
formats, as well as to `tar' archives and uncompressed `zip' archives.
A CD image contains both information for organizing the filesystem
(header with disc name etc., ISO9660 directory data, data of extensions
such as Joliet or RockRidge, zero padding) and the files contained on
the CD. An important property that jigdo relies on is that each file is
stored in one contiguous section of the image; it is not split into two
or more parts.</P>
<P>When <B>jigdo-file</B> is given a number of files that might be
contained in an image, it detects whether any of the files are present
using a `rolling checksum' inspired by the one used by <SPAN><SPAN
>rsync</SPAN>(1)</SPAN>. The resulting data is written to the `<TT
>.template</TT>' file: If a section of the image could not be matched
(e.g. it was directory information), the data is compressed and written
directly to the template. However, if a matching file was found, its
data is omitted from the template. Instead, only a reference (an MD5
checksum of the file) is inserted in the template.</P>
<P>Note that the template data only contains binary data, it does not
contain any filenames or URIs, since it cannot be easily edited in case
any of these values need to be changed. All that information is stored
in the `<TT>.jigdo</TT>' file, a text file to which you can add URLs
for your server(s). The jigdo file provides a mapping for each MD5
checksum to one or more alternative download locations for the
corresponding part.</P>
<P>Apart from the mapping of MD5 sums to URIs, the jigdo file also
contains an URI pointing to a download location for the template file.
This way, the <B>jigdo</B> download tool only needs to be given one URI
(that of the `<TT>.jigdo</TT>' file) to be able to download and
reassemble the complete image.</P></DIV>
<DIV><A NAME="JIGDO-FORMAT"></A>
<H2>FORMAT OF .JIGDO FILES</H2>
<P>The overall format of `<TT>.jigdo</TT>' files follows that of `<TT
>.ini</TT>' files, as also used by the Gnome and KDE projects for some
data. The file is organized into sections, each of which is preceded by
a line reading `<VAR>[Sectionname]</VAR>'. Within each section, lines
have the form `Label=Value'. Such lines are also called `entries'
below. All `<TT>.jigdo</TT>' files use UTF-8 as their character
encoding.</P>
<P>Comments are introduced with the `<VAR>#</VAR>' character and extend
to the end of the line. Whitespace is ignored at line start and end as
well as to the left and right of section names and the `<VAR>=</VAR>'
in entries. Furthermore, the jigdo utilities split up the text of the
entry value (i.e. the part after the `<VAR>=</VAR>') into
whitespace-separated words, much like the Unix shell. Single <VAR
>''</VAR> and double <VAR>""</VAR> quotes can be used to prevent that
e.g. URIs containing whitespace are split apart. Similarly, characters
with special meaning (the characters <VAR>'"#\</VAR> and space/tab)
must be quoted with <VAR>\</VAR> to appear in the value. As with the
shell, there is a difference between <VAR>' '</VAR> and <VAR
>" "</VAR>: Within <VAR>' '</VAR>, the characters <VAR
>"#\</VAR> and whitespace lose their special meaning and become
ordinary characters, whereas within <VAR>" "</VAR>, only the
characters <VAR>'#</VAR> and whitespace lose their special meaning - in
other words, backslash escapes still work inside <VAR>" "</VAR>,
but not <VAR>' '</VAR>.</P>
<P>`<TT>.jigdo</TT>' files can optionally be compressed with <SPAN
><SPAN>gzip</SPAN>(1)</SPAN>. <B>jigdo-file</B> always outputs
uncompressed files, which you can compress yourself. <B>jigdo-lite</B>
supports single uncompressed and compressed files. <B>jigdo</B>
additionally supports any number of concatenated plaintext and gzipped
parts in the files - for example, you can compress a `<TT>.jigdo</TT>'
file and then add a couple of lines of uncompressed data to the end. In
all cases, the `<TT>.gz</TT>' extension should be removed from the
filename - the tools will determine automatically from the file
contents whether a file is compressed or not.</P>
<P>Below is a description of the individual section names used by
jigdo.</P>
<DIV><A NAME="JIGDO-SECTION"></A>
<H3>Jigdo section</H3><PRE>[Jigdo]
Version=1.1
Generator=jigdo-file/1.0.0</PRE>
<P>Information about the version of the jigdo file format used, and the
program that generated it. There should be one such section per `<TT
>.jigdo</TT>' file.</P></DIV>
<DIV><A NAME="IMAGE-SECTION"></A>
<H3>Image section</H3><PRE>[Image]
Filename=<VAR>"filename for saving on user's disc"</VAR
>
Template=<VAR>"URI where to fetch template file"</VAR
>
Template-MD5Sum=OQ8riqT1BuyzsrT9964A7g
ShortInfo=<VAR>single-line description of the image (200 characters max.)</VAR
>
Info=<VAR>long description (5000 characters max.)</VAR
></PRE>
<P>The value for the `Template' entry can be either an URL (absolute or
relative to the URL of the jigdo file) or a string of the form `<VAR
>Label</VAR>:<VAR>pathname</VAR>' (<SPAN><I>UNIMPLEMENTED</I></SPAN
>), as described below.</P>
<P>The `Template-MD5Sum' entry is added by <B>jigdo-file</B> and
specifies the MD5 checksum of the generated `<TT>.template</TT>' file.
It is used by <B>jigdo</B> to detect cases where the downloaded
template data is corrupted or belongs to a different image.</P>
<P>Unlike other entry values, the values of the `<VAR>ShortInfo</VAR>'
and `<VAR>Info</VAR>' entries are <SPAN><I>not</I></SPAN> split up
into words, instead all quoting is preserved.</P>
<P>The value of the `<VAR>Info</VAR>' entry is special in that <SPAN
><SPAN>jigdo</SPAN>(1)</SPAN> can optionally parse XML markup it
contains. If the markup has errors such as unbalanced/unsupported
tags, the string is displayed literally, without XML parsing.
Supported tags are <VAR><b></b></VAR> (bold), <VAR
><i></i></VAR> (italic), <VAR><tt></tt></VAR>
(typewriter font), <VAR><u></u></VAR> (underline), <VAR
><big></big></VAR> (larger font), <VAR
><small></small></VAR> (smaller font) and <VAR
><br/></VAR> (linebreak). Supported entities include <VAR
>&lt;</VAR> (`<'), <VAR>&gt;</VAR> (`>') and <VAR
>&amp;</VAR> (`&'). Note that the whole `<VAR>Info</VAR>'
entry must be on one line in the jigdo file.</P>
<P>This section may occur multiple times, but all except the first one
will be ignored. This is useful e.g. when creating a `<TT>.jigdo</TT
>' file for a DVD image when you already have `<TT>.jigdo</TT>' files
for CDs with the same content: You can simply `<VAR>[Include]</VAR>'
(see below) the CDs' jigdo files at the end of the DVD jigdo file,
after its `<VAR>[Image]</VAR>' section.</P></DIV>
<DIV><A NAME="PARTS-SECTION"></A>
<H3>Parts section</H3><PRE>[Parts]
xJNkjrq8NYMraeGavUpllw=LabelA:part0
GoTResP2EC6Lb_2wTsqOoQ=LabelA:part1
kyfebwu6clbYqqWUdFIyaw=LabelB:some/path/part2
-J9UAimo0Bqg9c0oOXI1mQ=http://some.where.com/part3</PRE>
<P>All lines in the section, which provides the mapping from MD5
checksums to URIs, have the same format: On the left side of the `<VAR
>=</VAR>' the checksum (encoded with a Base64-like encoding) is given,
and on the right a string corresponding to the part with this
checksum; either a complete URI or a string of the form `<VAR
>Label</VAR>:<VAR>pathname</VAR>', which is expanded into one or more
URIs by looking up the definition(s) for the <VAR>Label</VAR> in the
`<VAR>[Servers]</VAR>' section.</P>
<P>In case a particular MD5 checksum cannot be found in any `<VAR
>[Parts]</VAR>' section by <B>jigdo</B>, the program will perform a
lookup for `<VAR>MD5Sum:</VAR><VAR><checksum></VAR>', e.g. for
`<VAR>MD5Sum:xJNkjrq8NYMraeGavUpllw</VAR>' if you deleted the line for
`part0' above.</P>
<P>A checksum appearing multiple times in this section indicates
alternative download locations for the part.</P>
<P>There may be any number of `<VAR>[Parts]</VAR>' sections in the
file; they are all considered when looking up MD5 checksums.</P>
<P><B>jigdo-file</B> always puts the `<VAR>[Parts]</VAR>' section at
the end of the file, and it even rearranges any file specified with
<VAR>--merge</VAR> to have only one such section at the end. This is
done to allow <B>jigdo</B> to display the information from the `<VAR
>[Image]</VAR>' section while the rest of that file is still being
downloaded.</P></DIV>
<DIV><A NAME="SERVERS-SECTION"></A>
<H3>Servers section</H3><PRE>[Servers]
LabelA=http://myserver.org/
LabelA=ftp://mirror.myserver.org/
LabelB=LabelC:subdirectory/
LabelC=http://some.where.com/jigdo/</PRE>
<P>All lines in the section, which provides the mapping from server
labels to server locations, have the same format: On the left side of
the `<VAR>=</VAR>' the label name is given, and on the right the value
to expand the label name to.</P>
<P>A label name appearing multiple times in this section indicates
alternative download locations for the parts that use the label in the
`<VAR>[Parts]</VAR>' section. This notation makes it very easy to add
mirrors to the jigdo file.</P>
<P>As shown by the example above, the label values may themselves
reference other labels. In this case, the entry
`LabelB:some/path/part2' in the `<VAR>[Parts]</VAR>' section will
expand to `http://some.where.com/jigdo/subdirectory/some/path/part2'.
Loops in the label definitions result in undefined behaviour and must
be avoided.</P>
<P>There may be any number of `<VAR>[Servers]</VAR>' sections in the
file; they are all considered when looking up labels. Either of `<VAR
>[Parts]</VAR>' or `<VAR>[Servers]</VAR>', but not both, can be
omitted from the jigdo file.</P></DIV>
<DIV><A NAME="COMMENT-SECTION"></A>
<H3>Comment section</H3><PRE>[Comment]
Any text, except that lines must not begin with `['.</PRE>
<P>All text following a `<VAR>[Comment]</VAR>' or `<VAR>[comment]</VAR
>' line is ignored, up to the next line with a section label.</P
></DIV>
<DIV><A NAME="INCLUDE"></A>
<H3>Include directive</H3><PRE>[Include http://some.url/file.jigdo]</PRE
>
<P>Lines of this form cause the content of the specified jigdo file to
be downloaded and parsed just like the main jigdo file. The effect
will be the same as copying the included file's contents into the file
which contains the include directive.</P>
<P>The URL argument can be an absolute or relative URL. Relative URLs
are assumed to be relative to the URL of the jigdo file which contains
the include directive. Includes can be nested, but it is an error to
create a loop of include directives. It is <SPAN><I>not</I></SPAN>
possible to use URLs of the form `<VAR>Label</VAR>:<VAR>pathname</VAR
>'.</P>
<P>The URL cannot be quoted with "". Any `<VAR>]</VAR>' characters in
the argument must be escaped as `<VAR>%5D</VAR>', and any spaces as
`<VAR>%20</VAR>'.</P>
<P>Include directives are only supported by <B>jigdo</B>, they are
ignored by <B>jigdo-lite</B>.</P>
<P>An include directive terminates any previous section, but it does
not start a new one. In other words, a new section must always be
started after the include line, <B>jigdo</B> does not allow normal
entries to appear below the `<VAR>[Include]</VAR>'.</P></DIV></DIV>
<DIV><A NAME="CACHE-FILES"></A>
<H2>CACHE FILES</H2>
<P>Any file specified with the <VAR>--cache</VAR> option is used to
store information about the <VAR>FILES</VAR> presented to <B
>jigdo-file</B>. When querying the cache, a file is considered
unchanged (and the cached data is used) only if filename, file size and
last modification time (mtime) match exactly. For the filename match,
not the entire file name is used, but only the part following any `<VAR
>//</VAR>', so that any changes to the part before the `<VAR>//</VAR>'
will not invalidate the cache.</P>
<P>Old cache entries are removed from the cache if they have not been
read from or written to for the amount of time specified with <VAR
>--cache-expiry</VAR>. Entries are <SPAN><I>not</I></SPAN> immediately
removed from the cache if the file they refer to no longer exists -
this makes it possible to cache information about files on removable
media.</P>
<P>Cache expiry only takes place <SPAN><I>after</I></SPAN> <B
>jigdo-file</B> has done its main work - if any old entries are
accessed before expiry takes place, they will be kept. For example, if
the program is run using the default expiry time of 30 days, but
accesses a cache file with entries generated 2 months ago, then entries
in that cache <SPAN><I>will</I></SPAN> be considered, and only those
cache entries that were not needed during the program run will be
expired.</P>
<P>Due to a peculiarity of the underlying database library (libdb3),
cache files never shrink, they only grow. If a large number of entries
was expired from your cache file and you want it to shrink, you can
either just delete it (of course then everything will have to be
regenerated) or use the utilities accompanying libdb3 to dump and
restore the database, with a command like `<B>db3_dump <VAR
>old-cache.db</VAR> | db3_load <VAR>new-cache.db</VAR></B>'. <SPAN
><SPAN>For Debian, these programs are supplied in the package
`libdb3-util'.</SPAN></SPAN></P>
<P>If a different <VAR>--md5-block-size</VAR> is specified, the entire
file needs to be re-read to update its cache entry. If a different <VAR
>--min-length</VAR> is specified, only the first `md5-block-size' bytes
of the file need to be re-read.</P></DIV>
<DIV><A NAME="EXAMPLES"></A>
<H2>EXAMPLES</H2>
<DIV><A NAME="EX-CDIMAGE"></A>
<H3>Preparing your CD image for distribution</H3>
<P>You have created a CD image `<TT>image.iso</TT>' from some of the
files stored in the directory `<TT>/home/ftp</TT>' on your harddisc,
which is also available online as `ftp://mysite.org'. As you don't
want to waste space by effectively hosting the same data twice (once
as files on the FTP server, once inside the image), and you are fed up
with users' downloads aborting after 200MB and their restarting the
download dozens of times, you decide to use jigdo. How do you prepare
the image for download?</P>
<P>In fact, only one command is necessary: <A NAME="AEN755"></A
><BLOCKQUOTE>
<P><B>jigdo-file make-template --image=image.iso
--jigdo=/home/ftp/image.jigdo --template=/home/ftp/image.template
/home/ftp// --label Mysite=/home/ftp --uri
Mysite=ftp://mysite.org/</B></P></BLOCKQUOTE> </P>
<P>People can now point <B>jigdo</B> at `ftp://mysite.org/image.jigdo'
to download your image. The template file needs to be accessible as
`ftp://mysite.org/image.template'.</P>
<P>Note that nothing prevents you from doing the same for an FTP server
that isn't administrated by you - in that case, you only need to host
the `<TT>.jigdo</TT>' and `<TT>.template</TT>' files on your own
server/homepage.</P></DIV>
<DIV><A NAME="EX-LARGEFILE"></A>
<H3>Preparing an arbitrary large file for distribution</H3>
<P>We assume that you have a large file that is not a filesystem, e.g.
`<TT>movie.mpeg</TT>'. Because of space problems, you want to
distribute the data on two servers.</P>
<P>In this case, the parts of the image need to be generated
artificially with the <B>split</B> command. For example, to create
chunks of 4MB each, use `<B>split -b 4m movie.mpeg part</B>'. Copy the
resulting files `<TT>part<VAR>XX</VAR></TT>' into two directories `<TT
>1</TT>' and `<TT>2</TT>' that you create, according to how you want
the files distributed between the servers. Next, create the jigdo and
template files with `<B>jigdo-file make-template --image=movie.mpeg
1// 2//</B>'. You will need to edit the `<TT>.jigdo</TT>' file and
provide the right URIs for the two servers that you are going to
upload the `<TT>part<VAR>XX</VAR></TT>' files to.</P></DIV>
<DIV><A NAME="EX-CUSTOMIZE"></A>
<H3>Customized versions of images</H3>
<P>Because it is possible to assign a different URI for each part of an
image if necessary, jigdo is very flexible. Only one example is the
possibility of customized versions of images: Suppose that someone is
distributing a CD image, and that you want to make a few small changes
to it and redistribute your own version. You download the `<TT
>official.iso</TT>' CD image with <B>jigdo</B> (passing it the URL of
`<TT>official.jigdo</TT>'), write it to CD-R, make your changes (say,
adding files from the `<TT>myfiles</TT>' directory on your harddisc)
and produce your own version, `<TT>myversion.iso</TT>'. Next, you
instruct <B>jigdo-file</B> to create the jigdo and template files for
your modified image, using the command <A NAME="AEN787"></A
><BLOCKQUOTE>
<P><B>jigdo-file make-template --image=myversion.iso /mnt/cdrom/
myfiles// --label My=myfiles/ --uri My=http://my.homepage.net/
--merge=official.jigdo</B></P></BLOCKQUOTE> while `<TT
>official.iso</TT>' is mounted under `<TT>/mnt/cdrom</TT>'. By using
<VAR>--merge</VAR>, you have told <B>jigdo-file</B> to take the
contents of `<TT>official.jigdo</TT>', add to it a new `<VAR
>[Image]</VAR>' section for `<TT>myversion.iso</TT>' and write the
resulting jigdo file to `<TT>myversion.jigdo</TT>' - so now `<TT
>myversion.jigdo</TT>' offers two images for download, the original
version and your modified version. (If you do not want it to offer the
official version, edit it and remove the `<VAR>[Image]</VAR>' section
that lists `<TT>official.iso</TT>'.)</P>
<P>Now you can upload the `<TT>.jigdo</TT>' file, the `<TT
>.template</TT>' file and also the files in `<TT>myfiles</TT>' to
`http://my.homepage.net/'. Thus, for people to download your modified
image, you do <SPAN><I>not</I></SPAN> need to upload the complete
image contents to your web space, but only the changes you made!</P>
<P>(In case you only made very few changes, you could also omit the
`myfiles' parameter in the command above, then all your changes end up
in the new template file.)</P></DIV>
<DIV><A NAME="EX-COMBINE"></A>
<H3>Combining many jigdo-managed images into one</H3>
<P>It is also no problem to combine data from several sources that use
jigdo. For example, if of five different and unrelated servers each
one distributes a different CD image via jigdo, you can create a
customized DVD image that contains the data from all these CDs. When
people use <B>jigdo</B> to download your image, the individual files
on the DVD are fetched from the same sources as the original CDs.</P>
<P>Consequently, even though you will be distributing a 3.2GB file via
your web space, the actual amount of data that is stored on your
server will only be in the order of several MBs.</P></DIV></DIV>
<DIV><A NAME="BUGS"></A>
<H2>BUGS</H2>
<P>For certain contents of one of the input files, most notably a
sequence of zero bytes longer than <VAR>--min-length</VAR> at the start
of the file and an area of zeros preceding the file data in the image,
<B>jigdo-file make-template</B> may fail to find the file in the image.
Unfortunately, this restriction cannot be avoided because the program
could become very slow otherwise. If you use the <VAR>--debug</VAR>
option, all instances of <B>jigdo-file</B> discarding possible matches
are indicated by lines containing the word `<VAR>DROPPED</VAR>'.</P>
<P>In fact, not only all-zeroes files trigger this behaviour, but also
files which contain at their start a long sequence of short identical
strings. For example, both a file containing only `<VAR>a</VAR>'
characters and one containing `<VAR>abcabcabcabc</VAR>...' are
problematic.</P></DIV>
<DIV><A NAME="SEEALSO"></A>
<H2>SEE ALSO</H2>
<P> <SPAN><SPAN>jigdo</SPAN>(1)</SPAN> (NOT YET IMPLEMENTED), <SPAN
><SPAN>jigdo-lite</SPAN>(1)</SPAN>, <SPAN><SPAN>jigdo-mirror</SPAN
>(1)</SPAN>, <SPAN><SPAN>split</SPAN>(1)</SPAN> (or `<B>info split</B
>'), <SPAN><SPAN>find</SPAN>(1)</SPAN> (or `<B>info find</B>'), <SPAN
><SPAN>mkisofs</SPAN>(1)</SPAN>, <SPAN><SPAN>md5sum</SPAN>(1)</SPAN>
</P></DIV>
<DIV><A NAME="AUTHOR"></A>
<H2>AUTHOR</H2>
<P><A HREF="http://atterer.net/jigdo/" TARGET="_top">Jigsaw Download</A
> was written by Richard Atterer <CODE><<A HREF="mailto:jigdo
atterer.net">jigdo atterer.net</A>></CODE>, to make downloading of
CD ROM images for the Debian Linux distribution more convenient.</P
></DIV>
</BODY>
</HTML>
|
