1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
|
.TH "topology.conf" "5" "Slurm Configuration File" "September 2024" "Slurm Configuration File"
.SH "NAME"
topology.conf \- Slurm configuration file for the topology plugins
.SH "DESCRIPTION"
\fBtopology.conf\fP is an ASCII file which describes the
cluster's network topology for optimized job resource allocation.
The file will always be located in the same directory as the \fBslurm.conf\fR.
.LP
Parameter names are case insensitive.
Any text following a "#" in the configuration file is treated
as a comment through the end of that line.
Changes to the configuration file take effect upon restart of
Slurm daemons, daemon receipt of the SIGHUP signal, or execution
of the command "scontrol reconfigure" unless otherwise noted.
.LP
.SH "topology/tree"
This plugin requires you to use the select/cons_tres plugin.
The network topology configuration, each line defining a switch name and
its children, either node names or switch names.
Slurm's hostlist expression parser is used, so the node and switch
names need not be consecutive (e.g. "Nodes=tux[0\-3,12,18\-20]"
and "Switches=s[0\-2,4\-8,12]" will parse fine).
An optional link speed may also be specified.
.LP
All nodes in the
network must be connected to at least one switch. The network must be fully
connected to use \fBTopologyParam=RouteTree\fR. Jobs can only span nodes
connected by the same switch fabric, even if there are available idle nodes
in other areas of the cluster.
.LP
The \fBtopology.conf\fP file for an Infiniband switch can be
automatically generated using the \fBslurmibtopology\fP tool found here:
<https://github.com/OleHolmNielsen/Slurm_tools/tree/master/slurmibtopology>.
.LP
The overall configuration parameters available for topology/tree include:
.TP
\fBSwitchName\fR
The name of a switch. This name is internal to Slurm and arbitrary.
Each switch should have a unique name.
This field must be specified and cannot be longer than 64 characters.
.IP
.TP
\fBSwitches\fR
Child switches of the named switch.
Either this option or the \fBNodes\fR option must be specified.
.IP
.TP
\fBNodes\fR
Child Nodes of the named leaf switch.
Either this option or the \fBSwitches\fR option must be specified.
.IP
.TP
\fBLinkSpeed\fR
An optional value specifying the performance of this communication link.
The units used are arbitrary and this information is currently not used.
It may be used in the future to optimize resource allocations.
.IP
.SH "topology/block"
.LP
The network topology configuration, each line defining a block name and
its children node names.
Slurm's hostlist expression parser is used, so the node
names need not be consecutive (e.g. "Nodes=tux[0\-3,12,18\-20]").
.LP
This topology plugin places emphasis on reducing fragmentation of the
cluster, allowing jobs to take advantage of lower\-latency connections
between smaller "blocks" of node, rather than starting jobs as quickly
as possible on the first available resources.
.LP
Defined blocks of nodes are paired with other contiguous blocks, to create
a higher level block of nodes. These larger blocks can then be paired with
other blocks at the same level for bigger and bigger blocks of contiguous
nodes with optimized communication between them. The enforced block sizes
are defined by \fBBlockSizes\fR.
.LP
The overall configuration parameters available for topology/block include:
.TP
\fBBlockName\fR
The name of a block. This name is internal to Slurm and arbitrary.
Each block should have a unique name.
This field must be specified.
.IP
.TP
\fBNodes\fR
Child Nodes of the named block.
This must be specified along with the \fBBlockName\fR.
.IP
.TP
\fBBlockSizes\fR
List of the planning base block size, alongside any
higher-level block sizes that would be enforced.
Each block must have at least the planning base block size count of nodes.
Successive BlockSizes must be a power of two larger than the prior values.
.IP
.SH "EXAMPLE"
.nf
##################################################################
# Slurm's network topology configuration file for use with the
# topology/tree plugin
##################################################################
SwitchName=s0 Nodes=dev[0\-5]
SwitchName=s1 Nodes=dev[6\-11]
SwitchName=s2 Nodes=dev[12\-17]
SwitchName=s3 Switches=s[0\-2]
.fi
.nf
##################################################################
# Slurm's network topology configuration file for use with the
# topology/block plugin
##################################################################
BlockName=block1 Nodes=node[1-32]
BlockName=block2 Nodes=node[33-64]
BlockName=block3 Nodes=node[65-96]
BlockName=block4 Nodes=node[97-128]
BlockSizes=30,120
.fi
.SH "COPYING"
Copyright (C) 2009 Lawrence Livermore National Security.
Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
.br
Copyright (C) 2010\-2023 SchedMD LLC.
.LP
This file is part of Slurm, a resource management program.
For details, see <https://slurm.schedmd.com/>.
.LP
Slurm is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option)
any later version.
.LP
Slurm is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
details.
.SH "SEE ALSO"
.LP
\fBslurm.conf\fR(5)
|
