1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226
|
'\" t
.\" Title: gfpcopy
.\" Author: [FIXME: author] [see http://docbook.sf.net/el/author]
.\" Generator: DocBook XSL Stylesheets v1.78.1 <http://docbook.sf.net/>
.\" Date: 27 Aug 2015
.\" Manual: Gfarm
.\" Source: Gfarm
.\" Language: English
.\"
.TH "GFPCOPY" "1" "27 Aug 2015" "Gfarm" "Gfarm"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.\" -----------------------------------------------------------------
.\" * MAIN CONTENT STARTS HERE *
.\" -----------------------------------------------------------------
.SH "NAME"
gfpcopy \- copy Gfarm files in parallel
.SH "SYNOPSIS"
.HP \w'\fBgfpcopy\fR\ 'u
\fBgfpcopy\fR [\-nqvdpPU] [\-X\ \fIregexp\fR] [\-S\ \fIsource\-domainname\fR] [\-h\ \fIsource\-hostfile\fR] [\-D\ \fIdestination\-domainname\fR] [\-H\ \fIdestination\-hostfile\fR] [\-j\ \fInum\-of\-processes\fR] [\-J\ \fInum\-of\-processes\fR] [\-M\ \fIlimit\-byte\fR] [\-z\ \fIminimum\-byte\fR] [\-Z\ \fImaximum\-byte\fR] [\-w\ \fIway\-of\-scheduling\fR] [\-W\ \fIkilobytes\-for\-threshold\fR] [\-s\ \fIKB/s\-to\-simulate\fR] [\-F\ \fInum\-for\-readahead\fR] [\-b\ \fIbufsize\fR] [\-f] [\-e] [\-k] \fIsource\-path\fR \fIdestination\-path\fR
.SH "DESCRIPTION"
.PP
\fBgfpcopy\fR
copies files in parallel\&.
.PP
When the
\fIsource\-path\fR
is a directory, files under the directory will be copied recursively\&.
.PP
When the
\fIdestination\-path\fR
does not exist, the directory is created\&. When the
\fIdestination\-path\fR
exists, a directory of the same name as the
\fIsource\-path\fR
is created under the
\fIdestination\-path\fR\&.
.PP
A set of source/destination hosts can be specified by a domain name and/or a hostlist file\&. When both a domain name and a hostlist file are specified, a set of hosts is determined by both conditions\&. When a set of source hosts is specified, only files stored on the source hosts are copied\&.
.PP
\fBgfpcopy\fR
also retrieves the directory entries in parallel\&.
.SH "SOURCE PATH"
.PP
\fIsource\-path\fR
must be one of the following formats\&. Files on HPSS cannot be copied\&.
.PP
\fIpath\-name\fR
.RS 4
is a relative path or an absolute path of a local file system\&. When the path is a mount point on gfarm2fs, files are copied without passing through the gfarm2fs\&.
.RE
.PP
\fIgfarm:\&.\&.\&.\fR
.RS 4
is a Gfarm URL\&.
.RE
.PP
\fIfile:\&.\&.\&.\fR
.RS 4
is an URL of a local file system\&.
.RE
.SH "DESTINATION PATH"
.PP
\fIdestination\-path\fR
must be one of the following formats\&.
.PP
\fIpath\-name\fR
.RS 4
is a relative path or an absolute path of a local file system\&. When the path is a mount point on gfarm2fs, files are copied without passing through the gfarm2fs\&.
.RE
.PP
\fIgfarm:\&.\&.\&.\fR
.RS 4
is a Gfarm URL of a directory\&.
.RE
.PP
\fIfile:\&.\&.\&.\fR
.RS 4
is an URL of a directory on a local file system\&.
.RE
.PP
\fIhpss:\&.\&.\&.\fR
.RS 4
is an URL of a directory on HPSS\&. If the same directory as the
\fIsource\-path\fR
exists under this directory, the
\fIsource\-path\fR
cannot be copied\&. The differential copy is not supported for HPSS\&. Relative path such as "hpss:"\&. and "hpss:dir" can be specified\&.
.RE
.SH "GFPCOPY OPTIONS"
.PP
These are options only for
\fBgfpcopy\fR\&.
.PP
\fB\-b\fR \fIbufsize\fR
.RS 4
Specifies the buffer size in bytes to copy\&.
.sp
The default value is 64 KiB (64 * 1024)\&.
.RE
.PP
\fB\-f\fR
.RS 4
With the \-f option, existing files will be overwritten when the size is different or the modification time (mtime) is different from the source file\&.
.sp
Without the \-f option, existing files will be overwritten when they are older than the corresponding source files in the modification time\&.
.RE
.PP
\fB\-e\fR
.RS 4
Skips existing files in order to execute gfpcopy simultaneously\&.
.RE
.PP
\fB\-k\fR
.RS 4
Does not copy symbolic links\&.
.RE
.SH "COMMON OPTIONS"
.PP
There are several common options with
\fBgfprep\fR\&. (see man
\fBgfprep\fR)
.SH "EXAMPLES"
.PP
To copy files under the directory recursively\&.
.sp
.if n \{\
.RS 4
.\}
.nf
$ gfpcopy gfarm:///dir file:///tmp/dir
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
$ gfpcopy file:///tmp/dir gfarm:///dir
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
$ gfpcopy gfarm:///dir1 gfarm:///dir2
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
$ gfpcopy gfarm:///dir hpss:///tmp/dir
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
$ cd /mnt/gfarm2fs
$ gfpcopy dir /tmp/dir
.fi
.if n \{\
.RE
.\}
.PP
To copy a file\&.
.sp
.if n \{\
.RS 4
.\}
.nf
$ gfpcopy gfarm:///dir/file file:///dir
.fi
.if n \{\
.RE
.\}
.sp
.if n \{\
.RS 4
.\}
.nf
$ cd /mnt/gfarm2fs
$ gfpcopy file /tmp/dir
.fi
.if n \{\
.RE
.\}
.SH "NOTES"
.PP
To retrieve the directory entries efficiently, it is better to execute
\fBgfpcopy\fR
command near the metadata server\&. When you need to execute
\fBgfpcopy\fR
command far from the metadata server, increase the parallelism by the \-j and \-J options\&.
.SH "SEE ALSO"
.PP
\fBgfprep\fR(1),
\fBgfreg\fR(1),
\fBgfexport\fR(1),
\fBgfarm2.conf\fR(5)
|