1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML3.2 EN">
<HTML>
<HEAD>
<META NAME="GENERATOR" CONTENT="DOCTEXT">
<TITLE>PetscOpenMPSpawn</TITLE>
</HEAD>
<BODY BGCOLOR="FFFFFF">
<A NAME="PetscOpenMPSpawn"><H1>PetscOpenMPSpawn</H1></A>
Initialize additional processes to be used as "worker" processes. This is not generally called by users. One should use -openmp_spawn_size <n> to indicate that you wish to have n-1 new MPI processes spawned for each current process.
<H3><FONT COLOR="#CC3333">Synopsis</FONT></H3>
<PRE>
#include "petsc.h"
PetscErrorCode PetscOpenMPSpawn(PetscMPIInt nodesize)
</PRE>
Not Collective (could make collective on MPI_COMM_WORLD, generate one huge <A HREF="../Sys/comm.html#comm">comm</A> and then split it up)
<P>
<H3><FONT COLOR="#CC3333">Input Parameter</FONT></H3>
<DT><B>nodesize </B> -<A HREF="../Sys/size.html#size">size</A> of each compute node that will share processors
<br>
<P>
<H3><FONT COLOR="#CC3333">Options Database</FONT></H3>
<DT><B>-openmp_spawn_size nodesize</B> -
Notes: This is only supported on systems with an MPI 2 implementation that includes the MPI_Comm_Spawn() routine.
<br>
<P>
<pre>
Comparison of two approaches for OpenMP usage (MPI started with N processes)
</pre>
<pre>
</pre>
<pre>
-openmp_spawn_size <n> requires MPI 2, results in n*N total processes with N directly used by application code
</pre>
<pre>
and n-1 worker processes (used by PETSc) for each application node.
</pre>
<pre>
You MUST launch MPI so that only ONE MPI process is created for each hardware node.
</pre>
<pre>
</pre>
<pre>
-openmp_merge_size <n> results in N total processes, N/n used by the application code and the rest worker processes
</pre>
<pre>
(used by PETSc)
</pre>
<pre>
You MUST launch MPI so that n MPI processes are created for each hardware node.
</pre>
<pre>
</pre>
<pre>
petscmpiexec -np 2 ./ex1 -openmp_spawn_size 3 gives 2 application nodes (and 4 PETSc worker nodes)
</pre>
<pre>
petscmpiexec -np 6 ./ex1 -openmp_merge_size 3 gives the SAME 2 application nodes and 4 PETSc worker nodes
</pre>
<pre>
This is what would use if each of the computers hardware nodes had 3 CPUs.
</pre>
<pre>
</pre>
<pre>
These are intended to be used in conjunction with USER OpenMP code. The user will have 1 process per
</pre>
<pre>
computer (hardware) node (where the computer node has p cpus), the user's code will use threads to fully
</pre>
<pre>
utilize all the CPUs on the node. The PETSc code will have p processes to fully use the compute node for
</pre>
<pre>
PETSc calculations. The user THREADS and PETSc PROCESSES will NEVER run at the same time so the p CPUs
</pre>
<pre>
are always working on p task, never more than p.
</pre>
<pre>
</pre>
<pre>
See <A HREF="../PC/PCOPENMP.html#PCOPENMP">PCOPENMP</A> for a PETSc preconditioner that can use this functionality
</pre>
<pre>
</pre>
<P>
<P>
<H3><FONT COLOR="#CC3333">See Also</FONT></H3>
<A HREF="../Sys/PetscFinalize.html#PetscFinalize">PetscFinalize</A>(), PetscInitializeFortran(), <A HREF="../Sys/PetscGetArgs.html#PetscGetArgs">PetscGetArgs</A>(), <A HREF="../Sys/PetscOpenMPFinalize.html#PetscOpenMPFinalize">PetscOpenMPFinalize</A>(), <A HREF="../Sys/PetscInitialize.html#PetscInitialize">PetscInitialize</A>(), <A HREF="../Sys/PetscOpenMPMerge.html#PetscOpenMPMerge">PetscOpenMPMerge</A>()
<BR>
<P>
<P><B><P><B><FONT COLOR="#CC3333">Level:</FONT></B>developer
<BR><FONT COLOR="#CC3333">Location:</FONT></B><A HREF="../../../src/sys/objects/mpinit.c.html#PetscOpenMPSpawn">src/sys/objects/mpinit.c</A>
<BR><A HREF="./index.html">Index of all Sys routines</A>
<BR><A HREF="../../index.html">Table of Contents for all manual pages</A>
<BR><A HREF="../singleindex.html">Index of all manual pages</A>
</BODY></HTML>
|
