1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.8"/>
<title>Hardware Locality (hwloc): Multi-node Topologies</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">Hardware Locality (hwloc)
 <span id="projectnumber">1.10.0</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.8 -->
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="index.html"><span>Main Page</span></a></li>
<li class="current"><a href="pages.html"><span>Related Pages</span></a></li>
<li><a href="modules.html"><span>Modules</span></a></li>
<li><a href="annotated.html"><span>Data Structures</span></a></li>
</ul>
</div>
</div><!-- top -->
<div class="header">
<div class="headertitle">
<div class="title">Multi-node Topologies </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>hwloc is usually used for consulting and manipulating single machine topologies. This includes large systems as long as a single instance of the operating system manages the entire system. However it is sometimes desirable to have multiple independent hosts inside the same topology, for instance when applying algorithms to an entire cluster topology. hwloc therefore offers the ability to agregate multiple host topologies into a single global one.</p>
<h1><a class="anchor" id="multinode_cpusets"></a>
Multi-node Objects Specifities</h1>
<p>A multi-node topology contains several single-node topologies. Those are assembled by making their own root objects (usually Machine object) children of higher objects. These higher objects include at least the root of the global topology (usually a System object). Some intermediate objects may also exists, for instance to represent switches in a large fabric.</p>
<p>There are actually three possible types of objects that have different properties with respect to cpusets, nodesets and binding. Indeed those cpusets and nodesets were designed for execution and memory binding within a single operating system. Binding on another system or across several different systems would be meaningless.</p>
<dl>
<dt>Local objects </dt>
<dd>Any object that corresponds to the local machine may be manipulated as usual. Obviously, if the multi-node topology does not contain the local machine topology, no such local object exists. </dd>
<dt>Objects from other nodes </dt>
<dd>Any object that comes from inside another node is represented as usual but its cpusets and nodesets should not be used for binding since binding on another system makes no sense. </dd>
<dt>Objects above single nodes </dt>
<dd>Any object above single-node topologies does not have any cpuset or nodeset pointer because binding across multiple systems makes no sense. This includes the glocal root object of a multi-node topology and possibly some intermediate objects between this global root and the local root of single-node topologies. </dd>
</dl>
<p>It is important to keep this in mind before binding using multi-node topologies. To make sure binding on an object is possible, one should first check that its cpuset or nodeset pointer is not NULL. Then, one should check whether the object is indeed local.</p>
<p>To find out which machine a given object corresponds to, one may look at the info attributes of the parent Machine object. The <code>HostName</code> info is usually available in Machine objects, it may be retrieved with the following code: </p><pre class="fragment"> hwloc_obj_t machine_obj;
obj = hwloc_get_ancestor_obj_by_type(topology, HWLOC_OBJ_MACHINE, obj);
if (machine_obj)
return hwloc_obj_get_info_by_name(machine_obj, "HostName");
else
return NULL;
</pre><p> The hwloc assembler scripts (see below) also add <code>AssemblerName</code> and <code>AssemblerIndex</code> info attributes to the Machine objects to identify the corresponding host name and index during assembly.</p>
<h1><a class="anchor" id="multinode_tools"></a>
Assembling topologies with command-line tools</h1>
<p>One way to manipulate multinode topologies is to retrieve other nodes' topologies as XML files and combine them as a global XML topology. It may then be loaded with <a class="el" href="a00077.html#ga879439b7ee99407ee911b3ac64e9a25e" title="Enable XML-file based topology. ">hwloc_topology_set_xml()</a> or with the HWLOC_XMLFILE environment variable.</p>
<p>The hwloc-assembler and hwloc-assembler-remote utilities offer the ability to combine XML topologies or remote nodes' topologies (see <a class="el" href="a00004.html">Command-Line Tools</a>).</p>
<h1><a class="anchor" id="multinode_interface"></a>
Assembling topologies with the programming interface</h1>
<p>The hwloc programming interface offers the ability to build multinode topologies using the <em>custom</em> interface. A new multinode topology has to be initialized with <a class="el" href="a00076.html#ga03fd4a16d8b9ee1ffc32b25fd2f6bdfa" title="Allocate a topology context. ">hwloc_topology_init()</a> and then set to custom with <a class="el" href="a00077.html#gad1f15795219e2c3ca6d44ba0db604c4f" title="Prepare the topology for custom assembly. ">hwloc_topology_set_custom()</a>. Topologies and objects mat then be assembled. Later, the custom topology is finalized as usual with <a class="el" href="a00076.html#gabdf58d87ad77f6615fccdfe0535ff826" title="Build the actual topology. ">hwloc_topology_load()</a>.</p>
<p>A custom topology starts with a single root object of type System. It may be modified by inserting a new child object with <a class="el" href="a00083.html#ga60c404d4f0544e1070f549af8be6bd73" title="Insert a new group object inside a custom topology. ">hwloc_custom_insert_group_object_by_parent()</a> or by duplicating another topology with <a class="el" href="a00083.html#ga58220f648a972140865d11940bccec81" title="Insert an existing topology inside a custom topology. ">hwloc_custom_insert_topology()</a>. Both of these operations require to specify the parent object in the custom topology where the insertion will take place. This parent may be either the root (returned by <a class="el" href="a00078.html#ga2d4b12fc187dfc53b35f2fa21d21044d" title="Returns the top-object of the topology-tree. ">hwloc_get_root_obj()</a>) or an already-inserted object (returned by <a class="el" href="a00083.html#ga60c404d4f0544e1070f549af8be6bd73" title="Insert a new group object inside a custom topology. ">hwloc_custom_insert_group_object_by_parent()</a>).</p>
<p>Ideally, any existing object in the custom topology could be the parent. However, special care should be taken when traversing the topology to find such an object because most links between objects (children, siblings, cousins) are not setup until <a class="el" href="a00076.html#gabdf58d87ad77f6615fccdfe0535ff826" title="Build the actual topology. ">hwloc_topology_load()</a> is invoked.</p>
<h1><a class="anchor" id="multinode_example"></a>
Example of assembly with the programming interface</h1>
<p>If the topologies of two hosts have been previously gathered in XML files <code>host1.xml</code> and <code>host2.xml</code>, the global topology may be assembled with the following code.</p>
<pre class="fragment">hwloc_topology_t host1, host2, global;
/* initialize global topology */
hwloc_topology_init(&global);
hwloc_topology_set_custom(global);
/* insert host1 entire topology below the global topology root */
hwloc_topology_init(&host1);
hwloc_topology_load(host1);
hwloc_custom_insert_topology(global, hwloc_get_root_obj(global),
host1, NULL);
hwloc_topology_destroy(host1);
/* insert host2 entire topology below the global topology root */
hwloc_topology_init(&host2);
hwloc_topology_load(host2);
hwloc_custom_insert_topology(global, hwloc_get_root_obj(global),
host2, NULL);
hwloc_topology_destroy(host2);
/* load and play with the global topology */
hwloc_topology_load(global);
...
</pre><p>If a intermediate object such as a switch should be inserted above one of the host topologies: </p><pre class="fragment">...
/* insert a switch object below the global topology root */
hwloc_obj_t sw =
hwloc_custom_insert_group_object_by_parent(global,
hwloc_get_root_obj(global), 0);
/* insert host2 entire topology below the switch */
hwloc_topology_init(&host2);
hwloc_topology_load(host2);
hwloc_custom_insert_topology(global, switch, host2, NULL);
hwloc_topology_destroy(host2);
/* load and play with the global topology */
hwloc_topology_load(global);
...
</pre> </div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Tue Oct 7 2014 11:56:52 for Hardware Locality (hwloc) by  <a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.8
</small></address>
</body>
</html>
|
