1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.12"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Hardware Locality (hwloc): Hardware Locality</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectalign" style="padding-left: 0.5em;">
<div id="projectname">Hardware Locality (hwloc)
 <span id="projectnumber">1.11.5</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.12 -->
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
$(function() {
initMenu('',false,false,'search.php','Search');
});
</script>
<div id="main-nav"></div>
</div><!-- top -->
<div class="header">
<div class="headertitle">
<div class="title">Hardware Locality </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h1 class="sub">Portable abstraction of hierarchical architectures for high-performance computing</h1>
<hr/>
<p>
<div class="section" id="introduction">
</p>
<h1><a class="anchor" id="Introduction"></a>
Introduction</h1>
<p>hwloc provides command line tools and a C API to obtain the hierarchical map of key computing elements, such as: NUMA memory nodes, shared caches, processor packages, processor cores, processing units (logical processors or "threads") and even I/O devices. hwloc also gathers various attributes such as cache and memory information, and is portable across a variety of different operating systems and platforms. Additionally it may assemble the topologies of multiple machines into a single one so as to let applications consult the topology of an entire fabric or cluster at once.</p>
<p>hwloc primarily aims at helping high-performance computing (HPC) applications, but is also applicable to any project seeking to exploit code and/or data locality on modern computing platforms.</p>
<p>hwloc supports the following operating systems:</p>
<ul>
<li>
Linux (including old kernels not having sysfs topology information, with knowledge of cpusets, offline CPUs, ScaleMP vSMP and Kerrighed support) on all supported hardware, including Intel Xeon Phi (KNL and KNC, either standalone or as a coprocessor) and NumaScale NumaConnect. </li>
<li>
Solaris </li>
<li>
AIX </li>
<li>
Darwin / OS X </li>
<li>
FreeBSD and its variants (such as kFreeBSD/GNU) </li>
<li>
NetBSD </li>
<li>
OSF/1 (a.k.a., Tru64) </li>
<li>
HP-UX </li>
<li>
Microsoft Windows </li>
<li>
IBM BlueGene/Q Compute Node Kernel (CNK) </li>
</ul>
<p>Since it uses standard Operating System information, hwloc's support is mostly independant from the processor type (x86, powerpc, ...) and just relies on the Operating System support. The only exception to this is kFreeBSD, which does not support topology information, and hwloc thus uses an x86-only CPUID-based backend (which can be used for other OSes too, see the <a class="el" href="a00333.html">Components and plugins</a> section).</p>
<p>To check whether hwloc works on a particular machine, just try to build it and run <code>lstopo</code> or <code>lstopo-no-graphics</code>. If some things do not look right (e.g. bogus or missing cache information), see <a class="el" href="index.html#bugs">Questions and Bugs</a> below.</p>
<p>hwloc only reports the number of processors on unsupported operating systems; no topology information is available.</p>
<p>For development and debugging purposes, hwloc also offers the ability to work on "fake" topologies:</p>
<ul>
<li>
Symmetrical tree of resources generated from a list of level arities, see <a class="el" href="a00330.html">Synthetic topologies</a>. </li>
<li>
Remote machine simulation through the gathering of topology as XML files, see <a class="el" href="a00329.html">Importing and exporting topologies from/to XML files</a>. </li>
</ul>
<p>hwloc can display the topology in a human-readable format, either in graphical mode (X11), or by exporting in one of several different formats, including: plain text, PDF, PNG, and FIG (see <a class="el" href="index.html#cli_examples">Command-line Examples</a> below). Note that some of the export formats require additional support libraries.</p>
<p>hwloc offers a programming interface for manipulating topologies and objects. It also brings a powerful CPU bitmap API that is used to describe topology objects location on physical/logical processors. See the <a class="el" href="index.html#interface">Programming Interface</a> below. It may also be used to binding applications onto certain cores or memory nodes. Several utility programs are also provided to ease command-line manipulation of topology objects, binding of processes, and so on.</p>
<p>Perl bindings are available from Bernd Kallies on <a href="http://search.cpan.org/~bka/Sys-Hwloc-0.10/">CPAN</a>.</p>
<p>Python bindings are available from Guy Streeter: </p><ul>
<li>
<a href="http://people.redhat.com/streeter/">Fedora RPM and tarball</a>. </li>
<li>
<a href="git://git.fedorahosted.org/python-hwloc.git">git tree</a> (<a href="http://git.fedorahosted.org/git/python-hwloc.git">html</a>). </li>
</ul>
<p>
</div><div class="section" id="installation">
</p>
<h1><a class="anchor" id="installation"></a>
Installation</h1>
<p>hwloc (<a href="http://www.open-mpi.org/projects/hwloc/">http://www.open-mpi.org/projects/hwloc/</a>) is available under the BSD license. It is hosted as a sub-project of the overall Open MPI project (<a href="http://www.open-mpi.org/">http://www.open-mpi.org/</a>). Note that hwloc does not require any functionality from Open MPI – it is a wholly separate (and much smaller!) project and code base. It just happens to be hosted as part of the overall Open MPI project.</p>
<p>Nightly development snapshots are available on the web site. Additionally, the code can be directly cloned from Git:</p>
<pre class="fragment">shell$ git clone https://github.com/open-mpi/hwloc.git
shell$ cd hwloc
shell$ ./autogen.sh
</pre><p>Note that GNU Autoconf >=2.63, Automake >=1.10 and Libtool >=2.2.6 are required when building from a Git clone.</p>
<p>Installation by itself is the fairly common GNU-based process:</p>
<pre class="fragment">shell$ ./configure --prefix=...
shell$ make
shell$ make install
</pre><p>The hwloc command-line tool "lstopo" produces human-readable topology maps, as mentioned above. It can also export maps to the "fig" file format. Support for PDF, Postscript, and PNG exporting is provided if the "Cairo" development package (usually <code>cairo-devel</code> or <code>libcairo2-dev</code>) can be found in "lstopo" when hwloc is configured and build.</p>
<p>The hwloc core may also benefit from the following development packages: </p><ul>
<li>
libnuma for memory binding and migration support on Linux (<code>numactl-devel</code> or <code>libnuma-dev</code> package). </li>
<li>
<p class="startli">libpciaccess for full I/O device discovery (<code>libpciaccess-devel</code> or <code>libpciaccess-dev</code> package). On Linux, PCI discovery may still be performed (without vendor/device names) even if libpciaccess cannot be used. </p>
<p class="endli"></p>
</li>
<li>
the AMD OpenCL implementation for OpenCL device discovery. </li>
<li>
the NVIDIA CUDA Toolkit for CUDA device discovery. </li>
<li>
the NVIDIA Management Library (NVML) for NVML device discovery. It is available within the NVIDIA GPU Deployment Kit from <a href="https://developer.nvidia.com/gpu-deployment-kit">https://developer.nvidia.com/gpu-deployment-kit</a> . </li>
<li>
the NV-CONTROL X extension library (NVCtrl) for NVIDIA display discovery. The relevant development package is usually <code>libXNVCtrl-devel</code> or <code>libxnvctrl-dev</code>. It is also available within nvidia-settings from <a href="ftp://download.nvidia.com/XFree86/nvidia-settings/">ftp://download.nvidia.com/XFree86/nvidia-settings/</a> and <a href="https://github.com/NVIDIA/nvidia-settings/">https://github.com/NVIDIA/nvidia-settings/</a> . </li>
<li>
libxml2 for full XML import/export support (otherwise, the internal minimalistic parser will only be able to import XML files that were exported by the same hwloc release). See <a class="el" href="a00329.html">Importing and exporting topologies from/to XML files</a> for details. The relevant development package is usually <code>libxml2-devel</code> or <code>libxml2-dev</code>. </li>
<li>
libudev on Linux for easier discovery of OS device information (otherwise hwloc will try to manually parse udev raw files). The relevant development package is usually <code>libudev-devel</code> or <code>libudev-dev</code>. </li>
<li>
libtool's ltdl library for dynamic plugin loading. The relevant development package is usually <code>libtool-ltdl-devel</code> or <code>libltdl-dev</code>. </li>
</ul>
<p>PCI and XML support may be statically built inside the main hwloc library, or as separate dynamically-loaded plugins (see the <a class="el" href="a00333.html">Components and plugins</a> section).</p>
<p>Note that because of the possibility of GPL taint, the <code>pciutils</code> library <code>libpci</code> will not be used (remember that hwloc is BSD-licensed).</p>
<p>Also note that if you install supplemental libraries in non-standard locations, hwloc's configure script may not be able to find them without some help. You may need to specify additional CPPFLAGS, LDFLAGS, or PKG_CONFIG_PATH values on the configure command line.</p>
<p>For example, if libpciaccess was installed into /opt/pciaccess, hwloc's configure script may not find it be default. Try adding PKG_CONFIG_PATH to the ./configure command line, like this:</p>
<pre class="fragment">./configure PKG_CONFIG_PATH=/opt/pciaccess/lib/pkgconfig ...
</pre><p>
</div><div class="section" id="cli_examples">
</p>
<h1><a class="anchor" id="cli_examples"></a>
Command-line Examples</h1>
<p>On a 4-package 2-core machine with hyper-threading, the <code>lstopo</code> tool may show the following graphical output:</p>
<div class="image">
<img src="dudley.png" alt="dudley.png"/>
</div>
<p>Here's the equivalent output in textual form:</p>
<pre class="fragment">Machine (16GB)
Package L#0 + L3 L#0 (4096KB)
L2 L#0 (1024KB) + L1 L#0 (16KB) + Core L#0
PU L#0 (P#0)
PU L#1 (P#8)
L2 L#1 (1024KB) + L1 L#1 (16KB) + Core L#1
PU L#2 (P#4)
PU L#3 (P#12)
Package L#1 + L3 L#1 (4096KB)
L2 L#2 (1024KB) + L1 L#2 (16KB) + Core L#2
PU L#4 (P#1)
PU L#5 (P#9)
L2 L#3 (1024KB) + L1 L#3 (16KB) + Core L#3
PU L#6 (P#5)
PU L#7 (P#13)
Package L#2 + L3 L#2 (4096KB)
L2 L#4 (1024KB) + L1 L#4 (16KB) + Core L#4
PU L#8 (P#2)
PU L#9 (P#10)
L2 L#5 (1024KB) + L1 L#5 (16KB) + Core L#5
PU L#10 (P#6)
PU L#11 (P#14)
Package L#3 + L3 L#3 (4096KB)
L2 L#6 (1024KB) + L1 L#6 (16KB) + Core L#6
PU L#12 (P#3)
PU L#13 (P#11)
L2 L#7 (1024KB) + L1 L#7 (16KB) + Core L#7
PU L#14 (P#7)
PU L#15 (P#15)
</pre><p>Note that there is also an equivalent output in XML that is meant for exporting/importing topologies but it is hardly readable to human-beings (see <a class="el" href="a00329.html">Importing and exporting topologies from/to XML files</a> for details).</p>
<p>On a 4-package 2-core Opteron NUMA machine, the <code>lstopo</code> tool may show the following graphical output:</p>
<div class="image">
<img src="hagrid.png" alt="hagrid.png"/>
</div>
<p>Here's the equivalent output in textual form:</p>
<pre class="fragment">Machine (32GB)
NUMANode L#0 (P#0 8190MB) + Package L#0
L2 L#0 (1024KB) + L1 L#0 (64KB) + Core L#0 + PU L#0 (P#0)
L2 L#1 (1024KB) + L1 L#1 (64KB) + Core L#1 + PU L#1 (P#1)
NUMANode L#1 (P#1 8192MB) + Package L#1
L2 L#2 (1024KB) + L1 L#2 (64KB) + Core L#2 + PU L#2 (P#2)
L2 L#3 (1024KB) + L1 L#3 (64KB) + Core L#3 + PU L#3 (P#3)
NUMANode L#2 (P#2 8192MB) + Package L#2
L2 L#4 (1024KB) + L1 L#4 (64KB) + Core L#4 + PU L#4 (P#4)
L2 L#5 (1024KB) + L1 L#5 (64KB) + Core L#5 + PU L#5 (P#5)
NUMANode L#3 (P#3 8192MB) + Package L#3
L2 L#6 (1024KB) + L1 L#6 (64KB) + Core L#6 + PU L#6 (P#6)
L2 L#7 (1024KB) + L1 L#7 (64KB) + Core L#7 + PU L#7 (P#7)
</pre><p>On a 2-package quad-core Xeon (pre-Nehalem, with 2 dual-core dies into each package):</p>
<div class="image">
<img src="emmett.png" alt="emmett.png"/>
</div>
<p>Here's the same output in textual form:</p>
<pre class="fragment">Machine (16GB)
Package L#0
L2 L#0 (4096KB)
L1 L#0 (32KB) + Core L#0 + PU L#0 (P#0)
L1 L#1 (32KB) + Core L#1 + PU L#1 (P#4)
L2 L#1 (4096KB)
L1 L#2 (32KB) + Core L#2 + PU L#2 (P#2)
L1 L#3 (32KB) + Core L#3 + PU L#3 (P#6)
Package L#1
L2 L#2 (4096KB)
L1 L#4 (32KB) + Core L#4 + PU L#4 (P#1)
L1 L#5 (32KB) + Core L#5 + PU L#5 (P#5)
L2 L#3 (4096KB)
L1 L#6 (32KB) + Core L#6 + PU L#6 (P#3)
L1 L#7 (32KB) + Core L#7 + PU L#7 (P#7)
</pre><p>
</div><div class="section" id="interface">
</p>
<h1><a class="anchor" id="interface"></a>
Programming Interface</h1>
<p>The basic interface is available in <a class="el" href="a00113_source.html" title="The hwloc API. ">hwloc.h</a>. Some higher-level functions are available in <a class="el" href="a00116_source.html" title="High-level hwloc traversal helpers. ">hwloc/helper.h</a> to reduce the need to manually manipulate objects and follow links between them. Documentation for all these is provided later in this document. Developers may also want to look at hwloc/inlines.h which contains the actual inline code of some <a class="el" href="a00113_source.html" title="The hwloc API. ">hwloc.h</a> routines, and at this document, which provides good higher-level topology traversal examples.</p>
<p>To precisely define the vocabulary used by hwloc, a <a class="el" href="a00321.html">Terms and Definitions</a> section is available and should probably be read first.</p>
<p>Each hwloc object contains a cpuset describing the list of processing units that it contains. These bitmaps may be used for <a class="el" href="a00170.html">CPU binding</a> and <a class="el" href="a00171.html">Memory binding</a>. hwloc offers an extensive bitmap manipulation interface in <a class="el" href="a00119_source.html" title="The bitmap API, for use in hwloc itself. ">hwloc/bitmap.h</a>.</p>
<p>Moreover, hwloc also comes with additional helpers for interoperability with several commonly used environments. See the <a class="el" href="a00331.html">Interoperability With Other Software</a> section for details.</p>
<p>The complete API documentation is available in a full set of HTML pages, man pages, and self-contained PDF files (formatted for both both US letter and A4 formats) in the source tarball in doc/doxygen-doc/.</p>
<p><b>NOTE:</b> If you are building the documentation from a Git clone, you will need to have Doxygen and pdflatex installed – the documentation will be built during the normal "make" process. The documentation is installed during "make install" to $prefix/share/doc/hwloc/ and your systems default man page tree (under $prefix, of course).</p>
<h2><a class="anchor" id="portability"></a>
Portability</h2>
<p>As shown in <a class="el" href="index.html#cli_examples">Command-line Examples</a>, hwloc can obtain information on a wide variety of hardware topologies. However, some platforms and/or operating system versions will only report a subset of this information. For example, on an PPC64-based system with 32 cores (each with 2 hardware threads) running a default 2.6.18-based kernel from RHEL 5.4, hwloc is only able to glean information about NUMA nodes and processor units (PUs). No information about caches, packages, or cores is available.</p>
<p>Similarly, Operating System have varying support for CPU and memory binding, e.g. while some Operating Systems provide interfaces for all kinds of CPU and memory bindings, some others provide only interfaces for a limited number of kinds of CPU and memory binding, and some do not provide any binding interface at all. Hwloc's binding functions would then simply return the ENOSYS error (Function not implemented), meaning that the underlying Operating System does not provide any interface for them. <a class="el" href="a00170.html">CPU binding</a> and <a class="el" href="a00171.html">Memory binding</a> provide more information on which hwloc binding functions should be preferred because interfaces for them are usually available on the supported Operating Systems.</p>
<p>Here's the graphical output from lstopo on this platform when Simultaneous Multi-Threading (SMT) is enabled:</p>
<div class="image">
<img src="ppc64-with-smt.png" alt="ppc64-with-smt.png"/>
</div>
<p>And here's the graphical output from lstopo on this platform when SMT is disabled:</p>
<div class="image">
<img src="ppc64-without-smt.png" alt="ppc64-without-smt.png"/>
</div>
<p>Notice that hwloc only sees half the PUs when SMT is disabled. PU #15, for example, seems to change location from NUMA node #0 to #1. In reality, no PUs "moved" – they were simply re-numbered when hwloc only saw half as many. Hence, PU #15 in the SMT-disabled picture probably corresponds to PU #30 in the SMT-enabled picture.</p>
<p>This same "PUs have disappeared" effect can be seen on other platforms – even platforms / OSs that provide much more information than the above PPC64 system. This is an unfortunate side-effect of how operating systems report information to hwloc.</p>
<p>Note that upgrading the Linux kernel on the same PPC64 system mentioned above to 2.6.34, hwloc is able to discover all the topology information. The following picture shows the entire topology layout when SMT is enabled:</p>
<div class="image">
<img src="ppc64-full-with-smt.png" alt="ppc64-full-with-smt.png"/>
</div>
<p>Developers using the hwloc API or XML output for portable applications should therefore be extremely careful to not make any assumptions about the structure of data that is returned. For example, per the above reported PPC topology, it is not safe to assume that PUs will always be descendants of cores.</p>
<p>Additionally, future hardware may insert new topology elements that are not available in this version of hwloc. Long-lived applications that are meant to span multiple different hardware platforms should also be careful about making structure assumptions. For example, there may someday be an element "lower" than a PU, or perhaps a new element may exist between a core and a PU.</p>
<h2><a class="anchor" id="interface_example"></a>
API Example</h2>
<p>The following small C example (available in the source tree as ``doc/examples/hwloc-hello.c'') prints the topology of the machine and performs some thread and memory binding. More examples are available in the doc/examples/ directory of the source tree.</p>
<div class="fragment"><div class="line"><span class="comment">/* Example hwloc API program.</span></div><div class="line"><span class="comment"> *</span></div><div class="line"><span class="comment"> * See other examples under doc/examples/ in the source tree</span></div><div class="line"><span class="comment"> * for more details.</span></div><div class="line"><span class="comment"> *</span></div><div class="line"><span class="comment"> * Copyright © 2009-2015 Inria. All rights reserved.</span></div><div class="line"><span class="comment"> * Copyright © 2009-2011 Université Bordeaux</span></div><div class="line"><span class="comment"> * Copyright © 2009-2010 Cisco Systems, Inc. All rights reserved.</span></div><div class="line"><span class="comment"> * See COPYING in top-level directory.</span></div><div class="line"><span class="comment"> *</span></div><div class="line"><span class="comment"> * hwloc-hello.c</span></div><div class="line"><span class="comment"> */</span></div><div class="line"></div><div class="line"><span class="preprocessor">#include <hwloc.h></span></div><div class="line"><span class="preprocessor">#include <errno.h></span></div><div class="line"><span class="preprocessor">#include <stdio.h></span></div><div class="line"><span class="preprocessor">#include <string.h></span></div><div class="line"></div><div class="line"><span class="keyword">static</span> <span class="keywordtype">void</span> print_children(<a class="code" href="a00165.html#ga9d1e76ee15a7dee158b786c30b6a6e38">hwloc_topology_t</a> topology, <a class="code" href="a00216.html">hwloc_obj_t</a> obj, </div><div class="line"> <span class="keywordtype">int</span> <a class="code" href="a00216.html#a9d82690370275d42d652eccdea5d3ee5">depth</a>)</div><div class="line">{</div><div class="line"> <span class="keywordtype">char</span> <a class="code" href="a00216.html#acc4f0803f244867e68fe0036800be5de">type</a>[32], <a class="code" href="a00216.html#accd40e29f71f19e88db62ea3df02adc8">attr</a>[1024];</div><div class="line"> <span class="keywordtype">unsigned</span> i;</div><div class="line"></div><div class="line"> <a class="code" href="a00168.html#gadb8765c260edea80c52cd06a76639ba4">hwloc_obj_type_snprintf</a>(type, <span class="keyword">sizeof</span>(type), obj, 0);</div><div class="line"> printf(<span class="stringliteral">"%*s%s"</span>, 2*depth, <span class="stringliteral">""</span>, type);</div><div class="line"> <span class="keywordflow">if</span> (obj-><a class="code" href="a00216.html#a61a7a80a68eaccbaaa28269e678c81a9">os_index</a> != (<span class="keywordtype">unsigned</span>) -1)</div><div class="line"> printf(<span class="stringliteral">"#%u"</span>, obj-><a class="code" href="a00216.html#a61a7a80a68eaccbaaa28269e678c81a9">os_index</a>);</div><div class="line"> <a class="code" href="a00168.html#ga870e876931c282a1c7aee2f031912ce3">hwloc_obj_attr_snprintf</a>(attr, <span class="keyword">sizeof</span>(attr), obj, <span class="stringliteral">" "</span>, 0);</div><div class="line"> <span class="keywordflow">if</span> (*attr)</div><div class="line"> printf(<span class="stringliteral">"(%s)"</span>, attr);</div><div class="line"> printf(<span class="stringliteral">"\n"</span>);</div><div class="line"> <span class="keywordflow">for</span> (i = 0; i < obj-><a class="code" href="a00216.html#aac3f6da35c9b57599909a44ce2b716c1">arity</a>; i++) {</div><div class="line"> print_children(topology, obj-><a class="code" href="a00216.html#a04d05403da37bfe17cd63b7c7dd07b1f">children</a>[i], depth + 1);</div><div class="line"> }</div><div class="line">}</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main(<span class="keywordtype">void</span>)</div><div class="line">{</div><div class="line"> <span class="keywordtype">int</span> <a class="code" href="a00216.html#a9d82690370275d42d652eccdea5d3ee5">depth</a>;</div><div class="line"> <span class="keywordtype">unsigned</span> i, n;</div><div class="line"> <span class="keywordtype">unsigned</span> <span class="keywordtype">long</span> size;</div><div class="line"> <span class="keywordtype">int</span> levels;</div><div class="line"> <span class="keywordtype">char</span> <span class="keywordtype">string</span>[128];</div><div class="line"> <span class="keywordtype">int</span> topodepth;</div><div class="line"> <a class="code" href="a00165.html#ga9d1e76ee15a7dee158b786c30b6a6e38">hwloc_topology_t</a> topology;</div><div class="line"> <a class="code" href="a00162.html#ga4bbf39b68b6f568fb92739e7c0ea7801">hwloc_cpuset_t</a> <a class="code" href="a00216.html#a67925e0f2c47f50408fbdb9bddd0790f">cpuset</a>;</div><div class="line"> <a class="code" href="a00216.html">hwloc_obj_t</a> obj;</div><div class="line"></div><div class="line"> <span class="comment">/* Allocate and initialize topology object. */</span></div><div class="line"> <a class="code" href="a00165.html#ga03fd4a16d8b9ee1ffc32b25fd2f6bdfa">hwloc_topology_init</a>(&topology);</div><div class="line"></div><div class="line"> <span class="comment">/* ... Optionally, put detection configuration here to ignore</span></div><div class="line"><span class="comment"> some objects types, define a synthetic topology, etc.... </span></div><div class="line"><span class="comment"></span></div><div class="line"><span class="comment"> The default is to detect all the objects of the machine that</span></div><div class="line"><span class="comment"> the caller is allowed to access. See Configure Topology</span></div><div class="line"><span class="comment"> Detection. */</span></div><div class="line"></div><div class="line"> <span class="comment">/* Perform the topology detection. */</span></div><div class="line"> <a class="code" href="a00165.html#gabdf58d87ad77f6615fccdfe0535ff826">hwloc_topology_load</a>(topology);</div><div class="line"></div><div class="line"> <span class="comment">/* Optionally, get some additional topology information</span></div><div class="line"><span class="comment"> in case we need the topology depth later. */</span></div><div class="line"> topodepth = <a class="code" href="a00167.html#gafa4f8dbc4c2e74c0da8019446353eed1">hwloc_topology_get_depth</a>(topology);</div><div class="line"></div><div class="line"> <span class="comment">/*****************************************************************</span></div><div class="line"><span class="comment"> * First example:</span></div><div class="line"><span class="comment"> * Walk the topology with an array style, from level 0 (always</span></div><div class="line"><span class="comment"> * the system level) to the lowest level (always the proc level).</span></div><div class="line"><span class="comment"> *****************************************************************/</span></div><div class="line"> <span class="keywordflow">for</span> (depth = 0; depth < topodepth; depth++) {</div><div class="line"> printf(<span class="stringliteral">"*** Objects at level %d\n"</span>, depth);</div><div class="line"> <span class="keywordflow">for</span> (i = 0; i < <a class="code" href="a00167.html#gab17065e3d53455973844568d9f21c72c">hwloc_get_nbobjs_by_depth</a>(topology, depth); </div><div class="line"> i++) {</div><div class="line"> <a class="code" href="a00168.html#gadb8765c260edea80c52cd06a76639ba4">hwloc_obj_type_snprintf</a>(<span class="keywordtype">string</span>, <span class="keyword">sizeof</span>(<span class="keywordtype">string</span>),</div><div class="line"> <a class="code" href="a00167.html#gabf8a98ad085460a4982cc7b74c344b71">hwloc_get_obj_by_depth</a>(topology, depth, i), 0);</div><div class="line"> printf(<span class="stringliteral">"Index %u: %s\n"</span>, i, <span class="keywordtype">string</span>);</div><div class="line"> }</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">/*****************************************************************</span></div><div class="line"><span class="comment"> * Second example:</span></div><div class="line"><span class="comment"> * Walk the topology with a tree style.</span></div><div class="line"><span class="comment"> *****************************************************************/</span></div><div class="line"> printf(<span class="stringliteral">"*** Printing overall tree\n"</span>);</div><div class="line"> print_children(topology, <a class="code" href="a00167.html#ga2d4b12fc187dfc53b35f2fa21d21044d">hwloc_get_root_obj</a>(topology), 0);</div><div class="line"></div><div class="line"> <span class="comment">/*****************************************************************</span></div><div class="line"><span class="comment"> * Third example:</span></div><div class="line"><span class="comment"> * Print the number of packages.</span></div><div class="line"><span class="comment"> *****************************************************************/</span></div><div class="line"> depth = <a class="code" href="a00167.html#ga8bec782e21be313750da70cf7428b374">hwloc_get_type_depth</a>(topology, <a class="code" href="a00163.html#ggacd37bb612667dc437d66bfb175a8dc55ab16ab8c0dbffc234921d86f3dfb63129">HWLOC_OBJ_PACKAGE</a>);</div><div class="line"> <span class="keywordflow">if</span> (depth == <a class="code" href="a00167.html#ggaf4e663cf42bbe20756b849c6293ef575a0565ab92ab72cb0cec91e23003294aad">HWLOC_TYPE_DEPTH_UNKNOWN</a>) {</div><div class="line"> printf(<span class="stringliteral">"*** The number of packages is unknown\n"</span>);</div><div class="line"> } <span class="keywordflow">else</span> {</div><div class="line"> printf(<span class="stringliteral">"*** %u package(s)\n"</span>,</div><div class="line"> <a class="code" href="a00167.html#gab17065e3d53455973844568d9f21c72c">hwloc_get_nbobjs_by_depth</a>(topology, depth));</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">/*****************************************************************</span></div><div class="line"><span class="comment"> * Fourth example:</span></div><div class="line"><span class="comment"> * Compute the amount of cache that the first logical processor</span></div><div class="line"><span class="comment"> * has above it.</span></div><div class="line"><span class="comment"> *****************************************************************/</span></div><div class="line"> levels = 0;</div><div class="line"> size = 0;</div><div class="line"> <span class="keywordflow">for</span> (obj = <a class="code" href="a00167.html#ga6f414dd80a2b943967a0ac92da3181a2">hwloc_get_obj_by_type</a>(topology, <a class="code" href="a00163.html#ggacd37bb612667dc437d66bfb175a8dc55abca6887e80cb291353b0a0c1da83f661">HWLOC_OBJ_PU</a>, 0);</div><div class="line"> obj;</div><div class="line"> obj = obj-><a class="code" href="a00216.html#adc494f6aed939992be1c55cca5822900">parent</a>)</div><div class="line"> <span class="keywordflow">if</span> (obj-><a class="code" href="a00216.html#acc4f0803f244867e68fe0036800be5de">type</a> == <a class="code" href="a00163.html#ggacd37bb612667dc437d66bfb175a8dc55a56ee0b7eca88f363b75b34fdde8c9ddc">HWLOC_OBJ_CACHE</a>) {</div><div class="line"> levels++;</div><div class="line"> size += obj-><a class="code" href="a00216.html#accd40e29f71f19e88db62ea3df02adc8">attr</a>-><a class="code" href="a00220.html#ab5a8ae3bf490e6b1071fea53f7382836">cache</a>.<a class="code" href="a00224.html#abe5e788943ed04302976740c829674c0">size</a>;</div><div class="line"> }</div><div class="line"> printf(<span class="stringliteral">"*** Logical processor 0 has %d caches totaling %luKB\n"</span>, </div><div class="line"> levels, size / 1024);</div><div class="line"></div><div class="line"> <span class="comment">/*****************************************************************</span></div><div class="line"><span class="comment"> * Fifth example:</span></div><div class="line"><span class="comment"> * Bind to only one thread of the last core of the machine.</span></div><div class="line"><span class="comment"> *</span></div><div class="line"><span class="comment"> * First find out where cores are, or else smaller sets of CPUs if</span></div><div class="line"><span class="comment"> * the OS doesn't have the notion of a "core".</span></div><div class="line"><span class="comment"> *****************************************************************/</span></div><div class="line"> depth = <a class="code" href="a00167.html#ga8125328e69eba709c33ea8055c12589b">hwloc_get_type_or_below_depth</a>(topology, <a class="code" href="a00163.html#ggacd37bb612667dc437d66bfb175a8dc55ac793958f330bca371aa1535de8aff45f">HWLOC_OBJ_CORE</a>);</div><div class="line"></div><div class="line"> <span class="comment">/* Get last core. */</span></div><div class="line"> obj = <a class="code" href="a00167.html#gabf8a98ad085460a4982cc7b74c344b71">hwloc_get_obj_by_depth</a>(topology, depth,</div><div class="line"> <a class="code" href="a00167.html#gab17065e3d53455973844568d9f21c72c">hwloc_get_nbobjs_by_depth</a>(topology, depth) - 1);</div><div class="line"> <span class="keywordflow">if</span> (obj) {</div><div class="line"> <span class="comment">/* Get a copy of its cpuset that we may modify. */</span></div><div class="line"> cpuset = <a class="code" href="a00186.html#gae679434c1a5f41d3560a8a7e2c1b0dee">hwloc_bitmap_dup</a>(obj-><a class="code" href="a00216.html#a67925e0f2c47f50408fbdb9bddd0790f">cpuset</a>);</div><div class="line"></div><div class="line"> <span class="comment">/* Get only one logical processor (in case the core is</span></div><div class="line"><span class="comment"> SMT/hyper-threaded). */</span></div><div class="line"> <a class="code" href="a00186.html#ga36d802130aa5d96bc1b5a90d2f0f3765">hwloc_bitmap_singlify</a>(cpuset);</div><div class="line"></div><div class="line"> <span class="comment">/* And try to bind ourself there. */</span></div><div class="line"> <span class="keywordflow">if</span> (<a class="code" href="a00170.html#ga80bc07473a8edf840cae17bd7ec21d48">hwloc_set_cpubind</a>(topology, cpuset, 0)) {</div><div class="line"> <span class="keywordtype">char</span> *str;</div><div class="line"> <span class="keywordtype">int</span> error = errno;</div><div class="line"> <a class="code" href="a00186.html#ga0fece972134fdecf2da9bc7a11dd827e">hwloc_bitmap_asprintf</a>(&str, obj-><a class="code" href="a00216.html#a67925e0f2c47f50408fbdb9bddd0790f">cpuset</a>);</div><div class="line"> printf(<span class="stringliteral">"Couldn't bind to cpuset %s: %s\n"</span>, str, strerror(error));</div><div class="line"> free(str);</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">/* Free our cpuset copy */</span></div><div class="line"> <a class="code" href="a00186.html#ga156130d85b3a0674d6e0e6770fe68fbe">hwloc_bitmap_free</a>(cpuset);</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">/*****************************************************************</span></div><div class="line"><span class="comment"> * Sixth example:</span></div><div class="line"><span class="comment"> * Allocate some memory on the last NUMA node, bind some existing</span></div><div class="line"><span class="comment"> * memory to the last NUMA node.</span></div><div class="line"><span class="comment"> *****************************************************************/</span></div><div class="line"> <span class="comment">/* Get last node. */</span></div><div class="line"> n = <a class="code" href="a00167.html#ga789a3f65aedff644be64a18526a03065">hwloc_get_nbobjs_by_type</a>(topology, <a class="code" href="a00163.html#ggacd37bb612667dc437d66bfb175a8dc55a9d917a3e5497950c6d8948b8e183db5a">HWLOC_OBJ_NUMANODE</a>);</div><div class="line"> <span class="keywordflow">if</span> (n) {</div><div class="line"> <span class="keywordtype">void</span> *m;</div><div class="line"> size = 1024*1024;</div><div class="line"></div><div class="line"> obj = <a class="code" href="a00167.html#ga6f414dd80a2b943967a0ac92da3181a2">hwloc_get_obj_by_type</a>(topology, <a class="code" href="a00163.html#ggacd37bb612667dc437d66bfb175a8dc55a9d917a3e5497950c6d8948b8e183db5a">HWLOC_OBJ_NUMANODE</a>, n - 1);</div><div class="line"> m = <a class="code" href="a00171.html#ga0ff3076f7f3633637699b809bcceceb1">hwloc_alloc_membind_nodeset</a>(topology, size, obj-><a class="code" href="a00216.html#a08f0d0e16c619a6e653526cbee4ffea3">nodeset</a>,</div><div class="line"> <a class="code" href="a00171.html#ggac9764f79505775d06407b40f5e4661e8ad811fa4b2a6002c4d63695a408ffde2c">HWLOC_MEMBIND_BIND</a>, 0);</div><div class="line"> <a class="code" href="a00171.html#ga32dbd4f54e9e4a7179f2dde37ffe6ad7">hwloc_free</a>(topology, m, size);</div><div class="line"></div><div class="line"> m = malloc(size);</div><div class="line"> <a class="code" href="a00171.html#ga7f11bd709ac0cb93af613e2dd84165ad">hwloc_set_area_membind_nodeset</a>(topology, m, size, obj-><a class="code" href="a00216.html#a08f0d0e16c619a6e653526cbee4ffea3">nodeset</a>,</div><div class="line"> <a class="code" href="a00171.html#ggac9764f79505775d06407b40f5e4661e8ad811fa4b2a6002c4d63695a408ffde2c">HWLOC_MEMBIND_BIND</a>, 0);</div><div class="line"> free(m);</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">/* Destroy topology object. */</span></div><div class="line"> <a class="code" href="a00165.html#ga9f34a640b6fd28d23699d4d084667b15">hwloc_topology_destroy</a>(topology);</div><div class="line"></div><div class="line"> <span class="keywordflow">return</span> 0;</div><div class="line">}</div></div><!-- fragment --><p>hwloc provides a <code>pkg-config</code> executable to obtain relevant compiler and linker flags. For example, it can be used thusly to compile applications that utilize the hwloc library (assuming GNU Make):</p>
<pre class="fragment">CFLAGS += $(pkg-config --cflags hwloc)
LDLIBS += $(pkg-config --libs hwloc)
cc hwloc-hello.c $(CFLAGS) -o hwloc-hello $(LDLIBS)
</pre><p>On a machine with 4GB of RAM and 2 processor packages – each package of which has two processing cores – the output from running <code>hwloc-hello</code> could be something like the following:</p>
<pre class="fragment">shell$ ./hwloc-hello
*** Objects at level 0
Index 0: Machine(3938MB)
*** Objects at level 1
Index 0: Package#0
Index 1: Package#1
*** Objects at level 2
Index 0: Core#0
Index 1: Core#1
Index 2: Core#3
Index 3: Core#2
*** Objects at level 3
Index 0: PU#0
Index 1: PU#1
Index 2: PU#2
Index 3: PU#3
*** Printing overall tree
Machine(3938MB)
Package#0
Core#0
PU#0
Core#1
PU#1
Package#1
Core#3
PU#2
Core#2
PU#3
*** 2 package(s)
shell$
</pre><p>
</div><div class="section" id="bugs">
</p>
<h1><a class="anchor" id="bugs"></a>
Questions and Bugs</h1>
<p>Questions should be sent to the devel mailing list (<a href="http://www.open-mpi.org/community/lists/hwloc.php">http://www.open-mpi.org/community/lists/hwloc.php</a>). Bug reports should be reported in the tracker (<a href="https://github.com/open-mpi/hwloc/issues">https://github.com/open-mpi/hwloc/issues</a>).</p>
<p>If hwloc discovers an incorrect topology for your machine, the very first thing you should check is to ensure that you have the most recent updates installed for your operating system. Indeed, most of hwloc topology discovery relies on hardware information retrieved through the operation system (e.g., via the /sys virtual filesystem of the Linux kernel). If upgrading your OS or Linux kernel does not solve your problem, you may also want to ensure that you are running the most recent version of the BIOS for your machine.</p>
<p>If those things fail, contact us on the mailing list for additional help. Please attach the output of lstopo after having given the <code>--enable-debug</code> option to ./configure and rebuilt completely, to get debugging output. Also attach the <code>/proc</code> + <code>/sys</code> tarball generated by the installed script <code>hwloc-gather-topology</code> when submitting problems about Linux, or send the output of <code>kstat cpu_info</code> in the Solaris case, or the output of <code>sysctl hw</code> in the Darwin or BSD cases.</p>
<p>
</div><div class="section" id="credits">
</p>
<h1><a class="anchor" id="history"></a>
History / Credits</h1>
<p>hwloc is the evolution and merger of the libtopology (<a href="http://runtime.bordeaux.inria.fr/libtopology/">http://runtime.bordeaux.inria.fr/libtopology/</a>) project and the Portable Linux Processor Affinity (PLPA) (<a href="http://www.open-mpi.org/projects/plpa/">http://www.open-mpi.org/projects/plpa/</a>) project. Because of functional and ideological overlap, these two code bases and ideas were merged and released under the name "hwloc" as an Open MPI sub-project.</p>
<p>libtopology was initially developed by the inria Runtime Team-Project (<a href="http://runtime.bordeaux.inria.fr/">http://runtime.bordeaux.inria.fr/</a>) (headed by Raymond Namyst (<a href="http://dept-info.labri.fr/~namyst/">http://dept-info.labri.fr/~namyst/</a>). PLPA was initially developed by the Open MPI development team as a sub-project. Both are now deprecated in favor of hwloc, which is distributed as an Open MPI sub-project.</p>
<p>
</div><div class="section" id="further_reading">
</p>
<h1><a class="anchor" id="further_read"></a>
Further Reading</h1>
<p>The documentation chapters include</p>
<ul>
<li>
<a class="el" href="a00321.html">Terms and Definitions</a> </li>
<li>
<a class="el" href="a00322.html">Command-Line Tools</a> </li>
<li>
<a class="el" href="a00323.html">Environment Variables</a> </li>
<li>
<a class="el" href="a00324.html">CPU and Memory Binding Overview</a> </li>
<li>
<a class="el" href="a00325.html">I/O Devices</a> </li>
<li>
<a class="el" href="a00326.html">Miscellaneous objects</a> </li>
<li>
<a class="el" href="a00327.html">Multi-node Topologies</a> </li>
<li>
<a class="el" href="a00328.html">Object attributes</a> </li>
<li>
<a class="el" href="a00329.html">Importing and exporting topologies from/to XML files</a> </li>
<li>
<a class="el" href="a00330.html">Synthetic topologies</a> </li>
<li>
<a class="el" href="a00331.html">Interoperability With Other Software</a> </li>
<li>
<a class="el" href="a00332.html">Thread Safety</a> </li>
<li>
<a class="el" href="a00333.html">Components and plugins</a> </li>
<li>
<a class="el" href="a00334.html">Embedding hwloc in Other Software</a> </li>
<li>
<a class="el" href="a00335.html">Frequently Asked Questions</a> </li>
</ul>
<p>Make sure to have had a look at those too!</p>
<p>
</div>
</p>
</div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by  <a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.12
</small></address>
</body>
</html>
|
