1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
|
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="GENERATOR" CONTENT="Mozilla/4.03 [en] (X11; I; Linux 2.0.32 i686) [Netscape]">
<META NAME="Author" CONTENT="Lee McLoughlin and Zoe Leech">
<TITLE>MM - Mirror Master</TITLE>
<!--
Copyright (C) 1990 - 1997 Lee McLoughlin
Permission to use, copy, and distribute this software and its
documentation for any purpose with or without fee is hereby granted,
provided that the above copyright notice appear in all copies and
that both that copyright notice and this permission notice appear
in supporting documentation.
Permission to modify the software is granted, but not the right to
distribute the modified code. Modifications are to be distributed
as patches to released version.
This software is provided "as is" without express or implied warranty.
$Id: mm.html,v 2.9 1998/05/29 19:13:34 lmjm Exp lmjm $
$Log: mm.html,v $
Revision 2.9 1998/05/29 19:13:34 lmjm
id update for 2.9 release
-->
</HEAD>
<BODY TEXT="#000000" BGCOLOR="#FFFFFF" LINK="#0000EF" VLINK="#55188A" ALINK="#FF0000">
<CENTER>
<H1>
Mirror Master</H1></CENTER>
<CENTER>
<H1>
<I>Lee McLoughlin</I></H1></CENTER>
<CENTER><I><FONT SIZE=+0>and</FONT></I></CENTER>
<CENTER>
<H1>
<I><FONT SIZE=+3>Zoë Leech</FONT></I></H1></CENTER>
<CENTER>22 May 1998</CENTER>
<CENTER><TT><A HREF="mailto:lmjm@icparc.ic.ac.uk">lmjm@icparc.ic.ac.uk</A></TT></CENTER>
<CENTER><TT><A HREF="zl@icparc.ic.ac.uk">zl@icparc.ic.ac.uk</A></TT></CENTER>
<UL>
<LI>
<A HREF="#Warning">Warning</A></LI>
<LI>
<A HREF="#Introduction">Introduction</A></LI>
<LI>
<A HREF="#Description">Description</A></LI>
<LI>
<A HREF="#Flags">Flags</A></LI>
<LI>
<A HREF="#Configuration File">Configuration File</A></LI>
<LI>
<A HREF="#Examples">Examples</A></LI>
<LI>
<A HREF="#See Also">See Also</A></LI>
<LI>
<A HREF="#Author">Author</A></LI>
</UL>
<H2>
<A NAME="Warning"></A>Warning</H2>
mm is not for the faint-hearted
<H2>
<A NAME="Introduction"></A>Introduction</H2>
mm [flags] [configuration file]
<H2>
<A NAME="Description"></A>Description</H2>
<B>mm</B> is a <B>Perl </B>program designed to manage the running of multiple
mirrors. It will keep as many mirrors running in parallel as it can, up
to a user defined maximum, automatically starting another when one finishes.
It will also handle locking so that only one mirror of a package can occur
at a time and allows configuration of the minimum time between attempts
to mirror a package.
<P><B>mm</B> presumes that all the package details are stored in a directory
called <TT><FONT SIZE=+1>packages</FONT></TT>. All the packages for a given
site must be in a file whose name is the same as that of the site. So in
<TT><FONT SIZE=+1>packages/sunsite.org.uk</FONT></TT> will be all the package
details for the host sunsite.org.uk.
<H2>
<A NAME="Flags"></A>Flags</H2>
<TABLE>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="-t"></A><B>-t </B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Ignore the minimal retry timers. </TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="-o site:package"></A><B>-o
site:package </B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Only mirror the given site and package. </TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="-debug"></A><B>-debug </B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Enable debugging. If this argument is given more
than once the level of debugging will increase. Currently the maximum possible
level is four. </TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="-s"></A><B>-s </B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Enable status debugging. This will print out
the status of any subprocesses as they are spawned or as they exit. </TD>
</TR>
</TABLE>
Any unrecognized arguments are passed as arguments to <B>mirror </B>for
example:
<BR>
<B>mm -d mmin</B>
<BR>will pass <B>mirror </B>the <B>-d</B> flag (so mirror will run in debug
mode).
<BR>
<H2>
<A NAME="Configuration File"></A><B>Configuration File</B></H2>
The configuration file is parsed as a series of statements. Blank lines
and lines beginning with a hash are ignored. There are are two kinds of
statements: control statements and site:package details.
<P>Control statements are of the form:
<UL><B><I>keyword</I>=value</B></UL>
Whitespace around the equals sign is ignored.
<P>Statements are obeyed in sequence as they are reached.
<P>Here is a list of the keywords and their values and any defaults:
<TABLE>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="home"></A><B><I>home</I></B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Directory that mm chdirs to before doing any
work. The default is the current directory (i.e. .) </TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="max"></A><B><I>max</I></B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Maximum number of mirrors that can be running
in parallel at any one time. The default is 6. </TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="mirror"></A><B><I>mirror</I></B></TD>
<TD ALIGN=LEFT VALIGN=TOP>This is the way to call mirror.
<BR>The default is:
<BR><TT>exec ./mirror \$args -p'\$package' packages/\$site > logs/\$ site:\$pkg
2>&1</TT>
<P>$args, $package and $site are replaced with the appropriate values from
the package file. Any $pkg is replaced with a version of $package with
all characters that cause problems in filenames converted to underscore. </TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="cmd"></A><B><I>cmd</I></B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Run the given command. This can occur multiple
times. It can be used to delete temporary files or send mail.</TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="cmdin"></A><B><I>cmdin</I></B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Run the given command and use its output as mm
input. This can occur multiple times. For example
<BR>cmdin=./pkgs_to_mmin packages/*</TD>
</TR>
<TR VALIGN=TOP>
<TD ALIGN=LEFT VALIGN=TOP WIDTH="15%"><A NAME="skip"></A><B><I>skip</I></B></TD>
<TD ALIGN=LEFT VALIGN=TOP>Skip the given site:package when it occurs. </TD>
</TR>
</TABLE>
In addition to control statements there are also package details. Each
package details statement is of the form:
<BLOCKQUOTE><B>site:package min-restart-last-ok min-restart-last-notok
mirror-args</B></BLOCKQUOTE>
The site and package fields are matched against those in the files in the
packages directory. A mirror for this site:package will only be run if
the last mirror run was successful and was more than <TT>min-restart-last-ok</TT>
hours ago. If the last mirror of this was unsuccessful then <TT>min-restart-last-notok</TT>
will be used to decide if the mirror will be run. The information about
the last mirror run is stored in a file called <TT>mm.status</TT> in the
directory defined by <B><I><A HREF="#home">home</A></I></B>
<BR>
<BR>
<H2>
<A NAME="Examples"></A>Examples</H2>
Here is a <B>mm</B> <A HREF="#Configuration File">configuration file</A>
to manage the mirroring of two packages.
<BR>
<UL><TT># Configuration file for mm</TT>
<BR><TT>max=1</TT>
<BR><TT>cmd=/usr/bin/rm -f /public/mirror/tmp</TT>
<BR><TT>home=/public/mirror</TT>
<P><TT>ftp.cl.cam.ac.uk:JIPS-nrs 20 20</TT>
<BR><TT>hcrl.open.ac.uk:hcrl-psion 20 20</TT></UL>
<P>However maintaining this file where there are a large number of mirrors
can be a problem. To overcome this the program <B>pkgs_to_mmin</B>, was
<BR>designed to take a list of package files as input and turn these into
the necessary input for <B>mm</B>
<UL><B>pkgs_to_mmin [-y min-restart-last-ok] [-n min-restart-last-notok]
package-files-to-use</B></UL>
In this <B>mm</B> <A HREF="#Configuration File">configuration file</A>
pkgs_to_mmin is used to manage all the packages in the /public/mirror/packages
directory.
<BR>
<UL><TT># Configuration file for mm</TT>
<BR><TT>home=/public/mirror</TT>
<P><TT>cmdin=./pkgs_to_mmin packages/*</TT>
<BR> </UL>
<H2>
<A NAME="See Also"></A>See Also</H2>
perl(l), ftp(1), mirror
<H2>
<A NAME="Author"></A>Author</H2>
Written by Lee McLoughlin <lmjm@icparc.ic.ac.uk>.
</BODY>
</HTML>
|
