1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
|
<!--
%\VignetteIndexEntry{Parallel Workers Running in Linux Containers}
%\VignetteAuthor{Henrik Bengtsson}
%\VignetteKeyword{R}
%\VignetteKeyword{package}
%\VignetteKeyword{vignette}
%\VignetteKeyword{Docker}
%\VignetteKeyword{Apptainer}
%\VignetteEngine{parallelly::selfonly}
-->
# Introduction
This vignette shows how to set up parallel workers running in Linux
containers, e.g. Docker (<https://www.docker.com/>), Apptainer
(<https://apptainer.org/>), and udocker
(<https://indigo-dc.github.io/udocker/>).
# Examples
## Example: Two parallel workers running in Docker
This example sets up two parallel workers running Docker image
'rocker/r-parallel' (<https://hub.docker.com/r/rocker/r-parallel>).
```r
library(parallelly)
cl <- makeClusterPSOCK(
rep("localhost", times = 2L),
## Launch Rscript inside Linux container via Docker
rscript = c(
"docker", "run", "--net=host", "rocker/r-parallel",
"Rscript"
),
## IMPORTANT: Because Docker runs inside a virtual machine (VM) on macOS
## and MS Windows (not Linux), when the R worker tries to connect back to
## the default 'localhost' it will fail, because the main R session is
## not running in the VM, but outside on the host. To reach the host on
## macOS and MS Windows, make sure to use master = "host.docker.internal"
master = if (.Platform$OS.type == "unix") NULL else "host.docker.internal",
)
print(cl)
#> Socket cluster with 2 nodes where 2 nodes are on host 'localhost'
#> (R version 4.3.3 (2024-02-29), platform x86_64-pc-linux-gnu)
```
## Example: Two parallel workers running in Apptainer
This example shows how to set up two parallel workers running Docker
image 'rocker/r-parallel'
(<https://hub.docker.com/r/rocker/r-parallel>) via Apptainer
(<<https://apptainer.org/>).
```r
library(parallelly)
cl <- makeClusterPSOCK(
rep("localhost", times = 2L),
## Launch Rscript inside Linux container via Apptainer
rscript = c(
"apptainer", "exec", "docker://rocker/r-parallel",
"Rscript"
)
)
print(cl)
#> Socket cluster with 2 nodes where 2 nodes are on host 'localhost'
#> (R version 3.6.1 (2019-07-05), platform x86_64-pc-linux-gnu)
```
## Example: Two parallel workers running in udocker
This example shows how to set up two parallel workers running Docker
image 'rocker/r-parallel'
(<https://hub.docker.com/r/rocker/r-parallel>) via udocker
(<https://indigo-dc.github.io/udocker/>).
```r
library(parallelly)
cl <- makeClusterPSOCK(
rep("localhost", times = 2L),
## Launch Rscript inside Linux container via Docker
rscript = c(
"udocker", "--quiet", "run", "rocker/r-parallel",
"Rscript"
)
)
print(cl)
#> Socket cluster with 2 nodes where 2 nodes are on host 'localhost'
#> (R version 3.6.1 (2019-07-05), platform x86_64-pc-linux-gnu)
```
|
