1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML3.2 EN">
<HTML>
<HEAD> <link rel="canonical" href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Sys/PetscCUDAInitialize.html" />
<META NAME="GENERATOR" CONTENT="DOCTEXT">
<TITLE>PetscCUDAInitialize</TITLE>
</HEAD>
<BODY BGCOLOR="FFFFFF">
<div id="version" align=right><b>petsc-3.14.5 2021-03-03</b></div>
<div id="bugreport" align=right><a href="mailto:petsc-maint@mcs.anl.gov?subject=Typo or Error in Documentation &body=Please describe the typo or error in the documentation: petsc-3.14.5 v3.14.5 docs/manualpages/Sys/PetscCUDAInitialize.html "><small>Report Typos and Errors</small></a></div>
<A NAME="PetscCUDAInitialize"><H1>PetscCUDAInitialize</H1></A>
Initializes CUDA (eagerly in <A HREF="../Sys/PetscInitialize.html#PetscInitialize">PetscInitialize</A>() or soon after <A HREF="../Sys/PetscInitialize.html#PetscInitialize">PetscInitialize</A>()) and cuBLAS/cuSPARSE libraries on the device
<H3><FONT COLOR="#CC3333">Synopsis</FONT></H3>
<PRE>
PETSC_EXTERN <A HREF="../Sys/PetscErrorCode.html#PetscErrorCode">PetscErrorCode</A> <A HREF="../Sys/PetscCUDAInitialize.html#PetscCUDAInitialize">PetscCUDAInitialize</A>(<A HREF="../Sys/MPI_Comm.html#MPI_Comm">MPI_Comm</A> comm,<A HREF="../Sys/PetscInt.html#PetscInt">PetscInt</A> device)
</PRE>
Logically collective
<P>
<H3><FONT COLOR="#CC3333">Input Parameter</FONT></H3>
<TABLE border="0" cellpadding="0" cellspacing="0">
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>comm </B></TD><TD>- the MPI communicator that will utilize the devices
</TD></TR>
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>device </B></TD><TD>- the device assigned to current MPI process. Special values like <A HREF="../Sys/PETSC_DECIDE.html#PETSC_DECIDE">PETSC_DECIDE</A> or <A HREF="../Sys/PETSC_DEFAULT.html#PETSC_DEFAULT">PETSC_DEFAULT</A> have special meanings (see details below)
</TD></TR></TABLE>
<P>
<H3><FONT COLOR="#CC3333">Options Database</FONT></H3>
<TABLE border="0" cellpadding="0" cellspacing="0">
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>-cuda_device <device> </B></TD><TD>- the device assigned to current MPI rank. <device> is case-insensitive and can be:
NONE (or none, or -3) : the code will not use any device, otherwise it will error out;
<A HREF="../Sys/PETSC_DEFAULT.html#PETSC_DEFAULT">PETSC_DEFAULT</A>(or DEFAULT, or -2) : do not explicitly set device, i.e., use whatever device already set by user (probably before <A HREF="../Sys/PetscInitialize.html#PetscInitialize">PetscInitialize</A>()). Init device runtime etc;
<A HREF="../Sys/PETSC_DECIDE.html#PETSC_DECIDE">PETSC_DECIDE</A> (or DECIDE, or -1) : assign MPI ranks in comm to available devices in round-robin, and init device runtime etc on the selected device;
>= 0 integer : assign the device with this id to current MPI process. Error out if <device> is invalid. Init device runtime etc on this device;
With PETSC_{DECIDE, DEFAULT}, if there are actually no devices, the code can still run, but it will error out when trying to create device objects.
</TD></TR>
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>-cuda_view </B></TD><TD>- view information about the devices.
</TD></TR>
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>-cuda_synchronize </B></TD><TD>- wait at the end of asynchronize device calls so that their time gets credited to the current event. With -log_view, the default is true, otherwise false.
</TD></TR>
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>-log_view </B></TD><TD>- logging, however if alone or combined with `-cuda_set_device DEFAULT | DECIDE | >=0 int`, will init device; if combined with `-cuda_set_device none`, won't init device.
</TD></TR>
<TR><TD WIDTH=40></TD><TD ALIGN=LEFT VALIGN=TOP><B>-use_gpu_aware_mpi </B></TD><TD>- assume the MPI is device/GPU-aware when communicating data on devices. Default true.
</TD></TR></TABLE>
<P>
<P>
<H3><FONT COLOR="#CC3333">Notes</FONT></H3>
Unless the input parameter <device> = -3, this routine initializes the CUDA device. It also initializes the cuBLAS/cuSPARSE libraries, which
takes a lot of time. Initializing them early helps avoid skewing timings in -log_view.
<P>
If this routine is triggered by command line options, it is called in <A HREF="../Sys/PetscInitialize.html#PetscInitialize">PetscInitialize</A>(). If users want to directly call it, they should call it immediately after <A HREF="../Sys/PetscInitialize.html#PetscInitialize">PetscInitialize</A>().
<P>
If this is not called then the CUDA initialization is delayed until first creation of a CUDA object and this can affect the timing since they happen asynchronously on different nodes and take a lot of time.
<P>
.seealso: <A HREF="../Sys/PetscCUDAInitializeCheck.html#PetscCUDAInitializeCheck">PetscCUDAInitializeCheck</A>(), <A HREF="../Sys/PetscHIPInitialize.html#PetscHIPInitialize">PetscHIPInitialize</A>(), <A HREF="../Sys/PetscHIPInitializeCheck.html#PetscHIPInitializeCheck">PetscHIPInitializeCheck</A>()
<P><B></B><H3><FONT COLOR="#CC3333">Level</FONT></H3>beginner<BR>
<H3><FONT COLOR="#CC3333">Location</FONT></H3>
</B><A HREF="../../../src/sys/objects/cupminit.i.c.html#PetscCUDAInitialize">src/sys/objects/cupminit.inc</A>
<BR><A HREF="./index.html">Index of all Sys routines</A>
<BR><A HREF="../../index.html">Table of Contents for all manual pages</A>
<BR><A HREF="../singleindex.html">Index of all manual pages</A>
</BODY></HTML>
|
