1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
|
<!--#include virtual="header.txt"-->
<h1>Resource Binding</h1>
<ul>
<li><a href="#overview">Overview</a></li>
<li><a href="#srun">Srun --cpu-bind option</a></li>
<li><a href="#node">Node CpuBind Configuration</a></li>
<li><a href="#partition">Partition CpuBind Configuration</a></li>
<li><a href="#TaskPluginParam">TaskPluginParam Configuration</a></li>
</ul>
<h2 id="overview">Overview<a class="slurm_link" href="#overview"></a></h2>
<p>Slurm has a rich set of options to control the default
binding of tasks to resources.
For example, tasks can be bound to individual threads, cores, sockets, NUMA
or boards.
See the slurm.conf and srun man pages for more information about how these
options work.
This document focuses on how default binding configuration can be configured.
Default binding can be configured on a per-node, per-partition or global basis.
The highest priority will be that specified using the srun --cpu-bind option.
The next highest priority binding will be the node-specific binding if any
node in the job allocation has some CpuBind configuration parameter and all other
nodes in the job allocation either have the same or no CpuBind configuration
parameter.
The next highest priority binding will be the partition-specific CpuBind
configuration parameter (if any).
The lowest priority binding will be that specified by the TaskPluginParam
configuration parameter.</p>
<ol>
<li>Srun --cpu-bind option</li>
<li>Node CpuBind configuration parameter (if all nodes match)</li>
<li>Partition CpuBind configuration parameter</li>
<li>TaskPluginParam configuration parameter</li>
</ol>
<h2 id="srun">Srun --cpu-bind option<a class="slurm_link" href="#srun"></a></h2>
<p>The srun --cpu-bind option will always be used to control task binding.
If the --cpu-bind option only includes "verbose" rather than identifying
the entities to be bound to, then the verbose option will be used together
with the default entity based upon Slurm configuration parameters as described
below.</p>
<h2 id="node">Node CpuBind Configuration
<a class="slurm_link" href="#node"></a>
</h2>
<p>The next possible source of the resource binding information is the node's
configured CpuBind value, but only if every node has the same CpuBind value
(or no configured CpuBind value).
The node's CpuBind value is configured in the slurm.conf file.
Its value may be viewed or modified using the scontrol command.
To clear a node's CpuBind value use the command:</p>
<pre>
scontrol update NodeName=... CpuBind=off
</pre>
<p>If a node_features plugin is configured, typically to support booting Intel
KNL nodes into different NUMA and/or MCDRAM modes, the plugin can be configured
to modify the node's CpuBind option based upon the NUMA mode.
This is accomplished by specifying the NumaCpuBind parameter in the knl.conf
configuration file with pairs of NUMA modes and CpuBind options.
As soon as the node is booted into a new NUMA mode, the node's CpuBind option
is automatically modified.
For example, the following line in the knl.conf file</p>
<pre>
NumaCpuBind=a2a=core;snc2=thread
</pre>
<p>will set a node's CpuBind field to "core" when booted into "a2a" (all to all)
NUMA mode and to "thread" when booted into "snc2 NUMA mode.
Any NUMA mode not specified in the NumaCpuBind configuration file will result
in no change to the node's CpuBind field.</p>
<h2 id="partition">Partition CpuBind Configuration
<a class="slurm_link" href="#partition"></a>
</h2>
<p>The next possible source of the resource binding information is the
partition's configured CpuBind value.
The partition's CpuBind value is configured in the slurm.conf file.
Its value may be viewed or modified using the scontrol command.</p>
<h2 id="TaskPluginParam">TaskPluginParam Configuration
<a class="slurm_link" href="#TaskPluginParam"></a>
</h2>
<p>The last possible source of the resource binding information is the
TaskPluginParam configuration parameter from the slurm.conf file.</p>
<p style="text-align:center;">Last modified 24 June 2020</p>
<!--#include virtual="footer.txt"-->
|
