1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
|
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>gfprep</title>
<meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="refentry">
<a name="gfprep.1"></a><div class="titlepage"></div>
<div class="refnamediv">
<a name="name"></a><h2>Name</h2>
<p>gfprep — replicate Gfarm files in parallel</p>
</div>
<div class="refsynopsisdiv">
<a name="synopsis"></a><h2>Synopsis</h2>
<div class="cmdsynopsis"><p><code class="command">gfprep</code> [-nqvdpPBU] [-mx] [-X <em class="replaceable"><code>regexp</code></em>] [-S <em class="replaceable"><code>source-domainname</code></em>] [-h <em class="replaceable"><code>source-hostfile</code></em>] [-D <em class="replaceable"><code>destination-domainname</code></em>] [-H <em class="replaceable"><code>destination-hostfile</code></em>] [-j <em class="replaceable"><code>num-of-processes</code></em>] [-J <em class="replaceable"><code>num-of-processes</code></em>] [-M <em class="replaceable"><code>total-byte</code></em>] [-z <em class="replaceable"><code>minimum-byte</code></em>] [-Z <em class="replaceable"><code>maximum-byte</code></em>] [-w <em class="replaceable"><code>way-of-scheduling</code></em>] [-W <em class="replaceable"><code>kilobytes-for-threshold</code></em>] [-s <em class="replaceable"><code>KB/s-to-simulate</code></em>] [-F <em class="replaceable"><code>num-for-readahead</code></em>] [-I <em class="replaceable"><code>sec-to-update</code></em>] [-N <em class="replaceable"><code>num-of-replicas</code></em>] <em class="replaceable"><code>path</code></em> </p></div>
</div>
<div class="refsect1">
<a name="description"></a><h2>DESCRIPTION</h2>
<p>
<span class="command"><strong>gfprep</strong></span> creates replicas of Gfarm
files in parallel.
</p>
<p>
When the <em class="parameter"><code>path</code></em> specifies a
directory, files under the directory will be replicated recursively.
</p>
<p>
<em class="parameter"><code>Path</code></em> can be specified by a
Gfarm URL or a path on a mount point of gfarm2fs.
</p>
<p>
A set of source/destination hosts can be specified by a domain name
and/or a hostlist file. When both a domain name and a hostlist file
are specified, a set of hosts is determined by both conditions. When
a set of source hosts is specified, only files stored on the source
hosts are replicated.
</p>
<p>
Less busy file system nodes will be selected for source or destination.
The maximum number of parallel replications/copies per host is the
number of CPUs. Please see the manual page
of <span class="command"><strong>gfhost</strong></span> for information about the
number of CPUs.
</p>
<p>
<span class="command"><strong>gfprep</strong></span> also retrieves the directory
entries in parallel.
</p>
</div>
<div class="refsect1">
<a name="options"></a><h2>GFPREP OPTIONS</h2>
<p>
These are options only for <span class="command"><strong>gfprep</strong></span>.
</p>
<div class="variablelist"><dl class="variablelist">
<dt><span class="term"><code class="option">-N</code> <em class="parameter"><code>num-of-replicas</code></em></span></dt>
<dd>
<p>
Specifies the required number of file replicas. If there are a
sufficient number of replicas in the destination hosts, neither any
copying nor any removal will be done.
</p>
<p>
If this option is not specified, -N 1 is assumed.
</p>
</dd>
<dt><span class="term"><code class="option">-x</code></span></dt>
<dd><p>
Removes extra file replicas when a set of destination nodes has more
file replicas than specified by the -N option.
</p></dd>
<dt><span class="term"><code class="option">-m</code></span></dt>
<dd>
<p>
Migrates file replicas from a set of source nodes to destination
nodes. When specifying twice like -mm, it leaves the source file replica.
</p>
<p>
This option requires at least one of -S, -h, -D, and -H options.
</p>
<p>
This option and the -N option are mutually exclusive.
</p>
</dd>
</dl></div>
</div>
<div class="refsect1">
<a name="common-options"></a><h2>COMMON OPTIONS</h2>
<p>
There are several common options
with <span class="command"><strong>gfprep</strong></span>. (see
man <span class="command"><strong>gfpcopy</strong></span>)
</p>
</div>
<div class="refsect1">
<a name="examples"></a><h2>EXAMPLES</h2>
<p>
To replicate files under the directory recursively.
</p>
<div class="literallayout"><p><br>
$ gfprep -N 3 gfarm:///dir<br>
</p></div>
<div class="literallayout"><p><br>
$ cd /mnt/gfarm2fs<br>
$ gfprep -N 3 dir<br>
</p></div>
<p>
To replicate a file or remove surplus replicas of the file.
</p>
<div class="literallayout"><p><br>
$ gfprep -N 3 -x gfarm:///dir/file<br>
</p></div>
<p>
To migrate replicas from gfsd1.example.com to other nodes.
</p>
<div class="literallayout"><p><br>
$ gfprep -m -S gfsd1.example.com gfarm:///<br>
</p></div>
</div>
<div class="refsect1">
<a name="notes"></a><h2>NOTES</h2>
<p>
When the -L or -m option is not specified, a source node is not always
selected within a set of specified source file system nodes.
</p>
<p>
It is better to execute
<span class="command"><strong>gfprep</strong></span> command near the metadata
server. When you need to
execute <span class="command"><strong>gfprep</strong></span> command far from the
metadata server, increase the parallelism by the -j and -J options.
</p>
</div>
<div class="refsect1">
<a name="see-also"></a><h2>SEE ALSO</h2>
<p>
<span class="citerefentry"><span class="refentrytitle">gfrep</span>(1)</span>,
<span class="citerefentry"><span class="refentrytitle">gfpcopy</span>(1)</span>,
<span class="citerefentry"><span class="refentrytitle">gfhost</span>(1)</span>,
<span class="citerefentry"><span class="refentrytitle">gfarm2.conf</span>(5)</span>
</p>
</div>
</div></body>
</html>
|
