1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.9.8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Hardware Locality (hwloc): Heterogeneous Memory</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr id="projectrow">
<td id="projectalign">
<div id="projectname">Hardware Locality (hwloc)<span id="projectnumber"> 2.12.2</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.8 -->
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
$(function() {
initMenu('',false,false,'search.php','Search');
});
/* @license-end */
</script>
<div id="main-nav"></div>
</div><!-- top -->
<div><div class="header">
<div class="headertitle"><div class="title">Heterogeneous Memory</div></div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>Heterogeneous memory hardware exposes different NUMA nodes for different memory technologies. On the image below, a dual-socket server has both HBM (high bandwidth memory) and usual DRAM connected to each socket, as well as some CXL memory connected to the entire machine.</p>
<div class="image">
<img src="heteromem.png" alt=""/>
</div>
<p>The hardware usually exposes "default" memory first because it is where "normal" data buffers should be allocated by default.</p>
<p>However there is no guarantee about whether HBM, NVM, CXL will appear second. Hence there is a need to explicit memory technologies and performance to help users decide where to allocate.</p>
<h1><a class="anchor" id="heteromem_memtiers"></a>
Memory Tiers and Default nodes</h1>
<p>hwloc builds <em>Memory Tiers</em> to identify different kinds of NUMA nodes. On the above machine, the first tier would contain both HBM NUMA nodes (L#1 and L#3), while the second tier would contain both DRAM nodes (L#0 and L#2), and the CXL memory (L#4) would be in the third tier. NUMA nodes are then annotated accordingly: </p><ul>
<li>
Each node object has its <code>subtype</code> field set to <code>HBM</code>, <code>DRAM</code> or <code>CXL-DRAM</code> (see other possible values in <a class="el" href="a00373.html#attributes_normal">Normal attributes</a>). </li>
<li>
Each node also has a string info attribute with name <code>MemoryTier</code> and value <code>0</code> for the first tier, <code>1</code> for the second, etc. </li>
</ul>
<p>Tiers are built using two kinds of information: </p><ul>
<li>
First hwloc looks into operating system information to find out whether a node is non-volatile, CXL, special-purpose, etc. </li>
<li>
Then it combines that knowledge with performance metrics exposed by the hardware to guess what's actually DRAM, HBM, etc. These metrics are also exposed in hwloc Memory Attributes, for instance bandwidth and latency, for read and write. See <a class="el" href="a00374.html#topoattrs_memattrs">Memory Attributes</a> and <a class="el" href="a00192.html">Comparing memory node attributes for finding where to allocate on</a> for more details. </li>
</ul>
<p>Once nodes with similar or different characteristics are identified, they are placed in tiers. Tiers are then sorted by bandwidth so that the highest bandwidth is ranked first, etc.</p>
<p>If hwloc fails to build tiers properly, see <code>HWLOC_MEMTIERS</code> and <code>HWLOC_MEMTIERS_GUESS</code> in <a class="el" href="a00369.html">Environment Variables</a>.</p>
<p><br />
</p>
<p>hwloc also tries to identify "default" memory nodes. They usually correspond the tier containing DRAM nodes. These are where normal data buffers should be allocated from, but they may also be used when placing tasks per NUMA domain (to hide NUMA nodes with overlapping localities, e.g. HBM and CXL in our example above).</p>
<h1><a class="anchor" id="heteromem_use_cli"></a>
Using Heterogeneous Memory from the command-line</h1>
<p>Specific kinds or tiers of memory may be specified in location filters when using NUMA nodes in hwloc command-line tools. For instance, binding memory on the first HBM node (<code>numa[hbm]:0</code>) is actually equivalent to binding on the second node (<code>numa:1</code>) on our example platform: </p><pre class="fragment">$ hwloc-bind --membind 'numa[hbm]:0' -- myprogram
$ hwloc-bind --membind 'numa:1' -- myprogram
</pre><p> To count DRAM nodes in the first CPU package, or all nodes: </p><pre class="fragment">$ hwloc-calc -N 'numa[dram]' package:0
1
$ hwloc-calc -N 'numa' package:0
2
</pre><p>To list all default NUMA nodes: </p><pre class="fragment">$ hwloc-calc --default-nodes all
0,2
</pre><p>To list all the physical indexes of Tier-0 NUMA nodes (HBM P#2 and P#3 not shown on the figure): </p><pre class="fragment">$ hwloc-calc -I 'numa[tier=0]' -p all
2,3
</pre><p> To find the memory kind of a NUMA node, one may look at its info attribute or use hwloc-calc: </p><pre class="fragment">$ hwloc-info --get-attr "info MemoryTier" numa:1
1
$ hwloc-calc -I memorytier numa:1
1
</pre><p>The number of tiers may be retrieved by looking at topology attributes in the root object, of by counting tiers inside it: </p><pre class="fragment">$ hwloc-info --get-attr "info MemoryTiersNr" topology
2
$ hwloc-calc --N memorytier all
2
</pre><p>hwloc-calc and hwloc-bind also have options such as <code>--local-memory</code> and <code>--best-memattr</code> to select the best NUMA node among the local ones. For instance, the following command-lines say that, among nodes near node:0 (DRAM L#0), the best one for latency is itself while the best one for bandwidth is node:1 (HBM L#1). </p><pre class="fragment">$ hwloc-calc --best-memattr latency node:0
0
$ hwloc-calc --best-memattr bandwidth node:0
1
</pre><h1><a class="anchor" id="heteromem_use_api"></a>
Using Heterogeneous Memory from the C API</h1>
<p>There are two major changes introduced by heterogeneous memory when looking at the hierarchical tree of objects. </p><ul>
<li>
First, there may be multiple memory children attached at the same place. For instance, each Package in the above image has two memory children, one for the DRAM NUMA node, and another one for the HBM node. </li>
<li>
Second, memory children may be attached at different levels. In the above image, CXL memory is attached to the root Machine object instead of below a Package. </li>
</ul>
<p>Hence, one may have to rethink the way it selects NUMA nodes.</p>
<h2><a class="anchor" id="heteromem_use_api_iterate"></a>
Iterating over the list of (heterogeneous) NUMA nodes</h2>
<p>A common need consists in iterating over the list of NUMA nodes (e.g. using <a class="el" href="a00167.html#ga8a90d48363f211740d1154056ea65ad9" title="Returns the next object of type type.">hwloc_get_next_obj_by_type()</a>). This is useful for counting some domains before partitioning a job, or for finding a node that is local to some objects. With heterogeneous memory, one should remember that multiple nodes may now have the same locality (HBM and DRAM above) or overlapping localities (e.g. DRAM and CXL above). </p><ul>
<li>
Checking NUMA node subtype or tier attributes is a good way to avoid this issue by ignoring nodes of different kinds. </li>
<li>
Another solution consists in ignoring nodes whose CPU set overlap the previously selected ones. For instance, in the above example, one could first select DRAM L#0 but ignore HBM L#1 (because it overlaps with DRAM L#0), then select DRAM L#2 but ignore HBM L#3 and CXL L#4 (overlap wih DRAM L#2). </li>
</ul>
<p><b>hwloc set of default nodes</b> (returned by <a class="el" href="a00192.html#gae7ff5d6f04d3bb71a3259cf1ff3afaed" title="Return the set of default NUMA nodes.">hwloc_topology_get_default_nodeset()</a>) <b>was designed for this purpose</b>: it ignores NUMA nodes with overlapping CPU set (only the first one is kept), and also tries to return nodes with similar subtypes.</p>
<p><br />
</p>
<p>It is also possible to iterate over the memory parents (e.g. Packages in our example) and select only one memory child for each of them. <a class="el" href="a00167.html#gae85786340b88e24835f8c403a1e2e54b" title="Return the depth of parents where memory objects are attached.">hwloc_get_memory_parents_depth()</a> may be used to find the depth of these parents. However this method only works if all memory parents are at the same level. It would fail in our example: the root Machine object also has a memory child (CXL), hence <a class="el" href="a00167.html#gae85786340b88e24835f8c403a1e2e54b" title="Return the depth of parents where memory objects are attached.">hwloc_get_memory_parents_depth()</a> would returns <a class="el" href="a00167.html#ggaf4e663cf42bbe20756b849c6293ef575ae99465995cacde6c210d5fc2e409798c" title="Objects of given type exist at different depth in the topology (only for Groups).">HWLOC_TYPE_DEPTH_MULTIPLE</a>.</p>
<h2><a class="anchor" id="heteromem_use_api_vertical"></a>
Iterating over local (heterogeneous) NUMA nodes</h2>
<p>Another common need is to find NUMA nodes that are local to some objects (e.g. a Core). A basic solution consists in looking at the Core nodeset and iterating over NUMA nodes to select those whose nodeset are included. A nicer solution is to walk up the tree to find ancestors with a memory child. With heterogeneous memory, multiple such ancestors may exist (Package and Machine in our example) and they may have multiple memory children.</p>
<p>Both these methods may be replaced with <a class="el" href="a00192.html#ga569e80c5be7ef27649b0ef5aa52ffcdc" title="Return an array of local NUMA nodes.">hwloc_get_local_numanode_objs()</a> which provides a convenient and flexible way to retrieve local NUMA nodes. One may then iterate over the returned array to select the appropriate one(s) depending on their subtype, tier or performance attributes.</p>
<p><br />
</p>
<p><a class="el" href="a00192.html#ga884d1f2ad745c2fa69c1583c82d28f10" title="Return the best target NUMA node for the given attribute and initiator.">hwloc_memattr_get_best_target()</a> is also a convenient way to select the best local NUMA node according to performance metrics. See also <a class="el" href="a00192.html">Comparing memory node attributes for finding where to allocate on</a>. </p>
</div></div><!-- contents -->
</div><!-- PageDoc -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by <a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.8
</small></address>
</body>
</html>
|
