.TH "hwloc_intro" 3 "Thu Feb 11 2021" "Version 2.4.1" "Hardware Locality (hwloc)" \" -*- nroff -*-
.ad l
.nh
.SH NAME
hwloc_intro \- Hardware Locality (hwloc) Introduction 

.SH "Portable abstraction of hierarchical architectures for high-performance computing"
.PP
.PP
.PP
.PP
 See also \fBFurther Reading\fP  for links to more sections about hwloc concepts\&. 
.PP
 
.SH "hwloc Summary"
.PP
hwloc provides command line tools and a C API to obtain the hierarchical map of key computing elements within a node, such as: NUMA memory nodes, shared caches, processor packages, dies and cores, processing units (logical processors or 'threads') and even I/O devices\&. hwloc also gathers various attributes such as cache and memory information, and is portable across a variety of different operating systems and platforms\&.
.PP
hwloc primarily aims at helping high-performance computing (HPC) applications, but is also applicable to any project seeking to exploit code and/or data locality on modern computing platforms\&.
.PP
hwloc supports the following operating systems:
.PP
.PD 0
.IP "\(bu" 2
Linux (including old kernels not having sysfs topology information, with knowledge of cpusets, ScaleMP vSMP support, etc\&.) on all supported hardware, including Intel Xeon Phi and NumaScale NumaConnect\&. 
.IP "\(bu" 2
Solaris (with support for processor sets and logical domains) 
.IP "\(bu" 2
AIX 
.IP "\(bu" 2
Darwin / OS X 
.IP "\(bu" 2
FreeBSD and its variants (such as kFreeBSD/GNU) 
.IP "\(bu" 2
NetBSD 
.IP "\(bu" 2
HP-UX 
.IP "\(bu" 2
Microsoft Windows 
.IP "\(bu" 2
IBM BlueGene/Q Compute Node Kernel (CNK) 
.PP
.PP
Since it uses standard Operating System information, hwloc's support is mostly independant from the processor type (x86, powerpc, \&.\&.\&.) and just relies on the Operating System support\&. The main exception is BSD operating systems (NetBSD, FreeBSD, etc\&.) because they do not provide support topology information, hence hwloc uses an x86-only CPUID-based backend (which can be used for other OSes too, see the \fBComponents and plugins\fP section)\&.
.PP
To check whether hwloc works on a particular machine, just try to build it and run \fClstopo\fP or \fClstopo-no-graphics\fP\&. If some things do not look right (e\&.g\&. bogus or missing cache information), see \fBQuestions and Bugs\fP\&.
.PP
hwloc only reports the number of processors on unsupported operating systems; no topology information is available\&.
.PP
For development and debugging purposes, hwloc also offers the ability to work on 'fake' topologies:
.PP
.PD 0
.IP "\(bu" 2
Symmetrical tree of resources generated from a list of level arities, see \fBSynthetic topologies\fP\&. 
.IP "\(bu" 2
Remote machine simulation through the gathering of topology as XML files, see \fBImporting and exporting topologies from/to XML files\fP\&. 
.PP
.PP
hwloc can display the topology in a human-readable format, either in graphical mode (X11), or by exporting in one of several different formats, including: plain text, LaTeX tikzpicture, PDF, PNG, and FIG (see \fBCommand-line Examples\fP below)\&. Note that some of the export formats require additional support libraries\&.
.PP
hwloc offers a programming interface for manipulating topologies and objects\&. It also brings a powerful CPU bitmap API that is used to describe topology objects location on physical/logical processors\&. See the \fBProgramming Interface\fP below\&. It may also be used to binding applications onto certain cores or memory nodes\&. Several utility programs are also provided to ease command-line manipulation of topology objects, binding of processes, and so on\&.
.PP
Perl bindings are available from Bernd Kallies on \fCCPAN\fP\&.
.PP
Python bindings are available from Guy Streeter: 
.PD 0

.IP "\(bu" 2
\fCFedora RPM and tarball\fP\&. 
.IP "\(bu" 2
\fCgit tree\fP (\fChtml\fP)\&. 
.PP
.PP
 
.SH "hwloc Installation"
.PP
The generic installation procedure for both hwloc and netloc is described in \fBInstallation\fP\&.
.PP
The hwloc command-line tool 'lstopo' produces human-readable topology maps, as mentioned above\&. It can also export maps to the 'fig' file format\&. Support for PDF, Postscript, and PNG exporting is provided if the 'Cairo' development package (usually \fCcairo-devel\fP or \fClibcairo2-dev\fP) can be found in 'lstopo' when hwloc is configured and build\&.
.PP
The hwloc core may also benefit from the following development packages: 
.PD 0

.IP "\(bu" 2
libpciaccess for full I/O device discovery (\fClibpciaccess-devel\fP or \fClibpciaccess-dev\fP package)\&. On Linux, PCI discovery may still be performed (without vendor/device names) even if libpciaccess cannot be used\&. 
.PP

.IP "\(bu" 2
AMD or NVIDIA OpenCL implementations for OpenCL device discovery\&.  
.IP "\(bu" 2
the NVIDIA CUDA Toolkit for CUDA device discovery\&.  
.IP "\(bu" 2
the NVIDIA Management Library (NVML) for NVML device discovery\&. It is included in CUDA since version 8\&.0\&. Older NVML releases were available within the NVIDIA GPU Deployment Kit from https://developer.nvidia.com/gpu-deployment-kit \&.  
.IP "\(bu" 2
the NV-CONTROL X extension library (NVCtrl) for NVIDIA display discovery\&. The relevant development package is usually \fClibXNVCtrl-devel\fP or \fClibxnvctrl-dev\fP\&. It is also available within nvidia-settings from ftp://download.nvidia.com/XFree86/nvidia-settings/ and https://github.com/NVIDIA/nvidia-settings/ \&.  
.IP "\(bu" 2
the AMD ROCm SMI library for RSMI device discovery\&. The relevant development package is usually \fCrocm-smi-lib64\fP or \fClibrocm-smi-dev\fP\&.  
.IP "\(bu" 2
libxml2 for full XML import/export support (otherwise, the internal minimalistic parser will only be able to import XML files that were exported by the same hwloc release)\&. See \fBImporting and exporting topologies from/to XML files\fP for details\&. The relevant development package is usually \fClibxml2-devel\fP or \fClibxml2-dev\fP\&.  
.IP "\(bu" 2
libudev on Linux for easier discovery of OS device information (otherwise hwloc will try to manually parse udev raw files)\&. The relevant development package is usually \fClibudev-devel\fP or \fClibudev-dev\fP\&.  
.IP "\(bu" 2
libtool's ltdl library for dynamic plugin loading if the native dlopen cannot be used\&. The relevant development package is usually \fClibtool-ltdl-devel\fP or \fClibltdl-dev\fP\&.  
.PP
.PP
PCI and XML support may be statically built inside the main hwloc library, or as separate dynamically-loaded plugins (see the \fBComponents and plugins\fP section)\&.
.PP
Note that because of the possibility of GPL taint, the \fCpciutils\fP library \fClibpci\fP will not be used (remember that hwloc is BSD-licensed)\&.
.PP
 
.SH "Command-line Examples"
.PP
On a 4-package 2-core machine with hyper-threading, the \fClstopo\fP tool may show the following graphical output:
.PP
 
.PP
Here's the equivalent output in textual form:
.PP
.PP
.nf
Machine
  NUMANode L#0 (P#0)
  Package L#0 + L3 L#0 (4096KB)
    L2 L#0 (1024KB) + L1 L#0 (16KB) + Core L#0
      PU L#0 (P#0)
      PU L#1 (P#8)
    L2 L#1 (1024KB) + L1 L#1 (16KB) + Core L#1
      PU L#2 (P#4)
      PU L#3 (P#12)
  Package L#1 + L3 L#1 (4096KB)
    L2 L#2 (1024KB) + L1 L#2 (16KB) + Core L#2
      PU L#4 (P#1)
      PU L#5 (P#9)
    L2 L#3 (1024KB) + L1 L#3 (16KB) + Core L#3
      PU L#6 (P#5)
      PU L#7 (P#13)
  Package L#2 + L3 L#2 (4096KB)
    L2 L#4 (1024KB) + L1 L#4 (16KB) + Core L#4
      PU L#8 (P#2)
      PU L#9 (P#10)
    L2 L#5 (1024KB) + L1 L#5 (16KB) + Core L#5
      PU L#10 (P#6)
      PU L#11 (P#14)
  Package L#3 + L3 L#3 (4096KB)
    L2 L#6 (1024KB) + L1 L#6 (16KB) + Core L#6
      PU L#12 (P#3)
      PU L#13 (P#11)
    L2 L#7 (1024KB) + L1 L#7 (16KB) + Core L#7
      PU L#14 (P#7)
      PU L#15 (P#15)
.fi
.PP
.PP
Note that there is also an equivalent output in XML that is meant for exporting/importing topologies but it is hardly readable to human-beings (see \fBImporting and exporting topologies from/to XML files\fP for details)\&.
.PP
On a 4-package 2-core Opteron NUMA machine (with two core cores disallowed by the administrator), the \fClstopo\fP tool may show the following graphical output (with \fC--disallowed\fP for displaying disallowed objects):
.PP
 
.PP
Here's the equivalent output in textual form:
.PP
.PP
.nf
Machine (32GB total)
  Package L#0
    NUMANode L#0 (P#0 8190MB)
    L2 L#0 (1024KB) + L1 L#0 (64KB) + Core L#0 + PU L#0 (P#0)
    L2 L#1 (1024KB) + L1 L#1 (64KB) + Core L#1 + PU L#1 (P#1)
  Package L#1
    NUMANode L#1 (P#1 8192MB)
    L2 L#2 (1024KB) + L1 L#2 (64KB) + Core L#2 + PU L#2 (P#2)
    L2 L#3 (1024KB) + L1 L#3 (64KB) + Core L#3 + PU L#3 (P#3)
  Package L#2
    NUMANode L#2 (P#2 8192MB)
    L2 L#4 (1024KB) + L1 L#4 (64KB) + Core L#4 + PU L#4 (P#4)
    L2 L#5 (1024KB) + L1 L#5 (64KB) + Core L#5 + PU L#5 (P#5)
  Package L#3
    NUMANode L#3 (P#3 8192MB)
    L2 L#6 (1024KB) + L1 L#6 (64KB) + Core L#6 + PU L#6 (P#6)
    L2 L#7 (1024KB) + L1 L#7 (64KB) + Core L#7 + PU L#7 (P#7)
.fi
.PP
.PP
On a 2-package quad-core Xeon (pre-Nehalem, with 2 dual-core dies into each package):
.PP
 
.PP
Here's the same output in textual form:
.PP
.PP
.nf
Machine (total 16GB)
  NUMANode L#0 (P#0 16GB)
  Package L#0
    L2 L#0 (4096KB)
      L1 L#0 (32KB) + Core L#0 + PU L#0 (P#0)
      L1 L#1 (32KB) + Core L#1 + PU L#1 (P#4)
    L2 L#1 (4096KB)
      L1 L#2 (32KB) + Core L#2 + PU L#2 (P#2)
      L1 L#3 (32KB) + Core L#3 + PU L#3 (P#6)
  Package L#1
    L2 L#2 (4096KB)
      L1 L#4 (32KB) + Core L#4 + PU L#4 (P#1)
      L1 L#5 (32KB) + Core L#5 + PU L#5 (P#5)
    L2 L#3 (4096KB)
      L1 L#6 (32KB) + Core L#6 + PU L#6 (P#3)
      L1 L#7 (32KB) + Core L#7 + PU L#7 (P#7)
.fi
.PP
.PP
 
.SH "Programming Interface"
.PP
The basic interface is available in \fBhwloc\&.h\fP\&. Some higher-level functions are available in \fBhwloc/helper\&.h\fP to reduce the need to manually manipulate objects and follow links between them\&. Documentation for all these is provided later in this document\&. Developers may also want to look at hwloc/inlines\&.h which contains the actual inline code of some \fBhwloc\&.h\fP routines, and at this document, which provides good higher-level topology traversal examples\&.
.PP
To precisely define the vocabulary used by hwloc, a \fBTerms and Definitions\fP section is available and should probably be read first\&.
.PP
Each hwloc object contains a cpuset describing the list of processing units that it contains\&. These bitmaps may be used for \fBCPU binding\fP and \fBMemory binding\fP\&. hwloc offers an extensive bitmap manipulation interface in \fBhwloc/bitmap\&.h\fP\&.
.PP
Moreover, hwloc also comes with additional helpers for interoperability with several commonly used environments\&. See the \fBInteroperability With Other Software\fP section for details\&.
.PP
The complete API documentation is available in a full set of HTML pages, man pages, and self-contained PDF files (formatted for both both US letter and A4 formats) in the source tarball in doc/doxygen-doc/\&.
.PP
\fBNOTE:\fP If you are building the documentation from a Git clone, you will need to have Doxygen and pdflatex installed -- the documentation will be built during the normal 'make' process\&. The documentation is installed during 'make install' to $prefix/share/doc/hwloc/ and your systems default man page tree (under $prefix, of course)\&.
.SS "Portability"
Operating System have varying support for CPU and memory binding, e\&.g\&. while some Operating Systems provide interfaces for all kinds of CPU and memory bindings, some others provide only interfaces for a limited number of kinds of CPU and memory binding, and some do not provide any binding interface at all\&. Hwloc's binding functions would then simply return the ENOSYS error (Function not implemented), meaning that the underlying Operating System does not provide any interface for them\&. \fBCPU binding\fP and \fBMemory binding\fP provide more information on which hwloc binding functions should be preferred because interfaces for them are usually available on the supported Operating Systems\&.
.PP
Similarly, the ability of reporting topology information varies from one platform to another\&. As shown in \fBCommand-line Examples\fP, hwloc can obtain information on a wide variety of hardware topologies\&. However, some platforms and/or operating system versions will only report a subset of this information\&. For example, on an PPC64-based system with 8 cores (each with 2 hardware threads) running a default 2\&.6\&.18-based kernel from RHEL 5\&.4, hwloc is only able to glean information about NUMA nodes and processor units (PUs)\&. No information about caches, packages, or cores is available\&.
.PP
Here's the graphical output from lstopo on this platform when Simultaneous Multi-Threading (SMT) is enabled:
.PP
 
.PP
And here's the graphical output from lstopo on this platform when SMT is disabled:
.PP
 
.PP
Notice that hwloc only sees half the PUs when SMT is disabled\&. PU L#6, for example, seems to change location from NUMA node #0 to #1\&. In reality, no PUs 'moved' -- they were simply re-numbered when hwloc only saw half as many (see also Logical index in \fBIndexes and Sets\fP)\&. Hence, PU L#6 in the SMT-disabled picture probably corresponds to PU L#12 in the SMT-enabled picture\&.
.PP
This same 'PUs have disappeared' effect can be seen on other platforms -- even platforms / OSs that provide much more information than the above PPC64 system\&. This is an unfortunate side-effect of how operating systems report information to hwloc\&.
.PP
Note that upgrading the Linux kernel on the same PPC64 system mentioned above to 2\&.6\&.34, hwloc is able to discover all the topology information\&. The following picture shows the entire topology layout when SMT is enabled:
.PP
 
.PP
Developers using the hwloc API or XML output for portable applications should therefore be extremely careful to not make any assumptions about the structure of data that is returned\&. For example, per the above reported PPC topology, it is not safe to assume that PUs will always be descendants of cores\&.
.PP
Additionally, future hardware may insert new topology elements that are not available in this version of hwloc\&. Long-lived applications that are meant to span multiple different hardware platforms should also be careful about making structure assumptions\&. For example, a new element may someday exist between a core and a PU\&.
.SS "API Example"
The following small C example (available in the source tree as ``doc/examples/hwloc-hello\&.c'') prints the topology of the machine and performs some thread and memory binding\&. More examples are available in the doc/examples/ directory of the source tree\&.
.PP
.PP
.nf
/* Example hwloc API program\&.
 *
 * See other examples under doc/examples/ in the source tree
 * for more details\&.
 *
 * Copyright © 2009-2016 Inria\&.  All rights reserved\&.
 * Copyright © 2009-2011 Université Bordeaux
 * Copyright © 2009-2010 Cisco Systems, Inc\&.  All rights reserved\&.
 * See COPYING in top-level directory\&.
 *
 * hwloc-hello\&.c
 */

#include "hwloc\&.h"

#include <errno\&.h>
#include <stdio\&.h>
#include <string\&.h>

static void print_children(hwloc_topology_t topology, hwloc_obj_t obj,
                           int depth)
{
    char type[32], attr[1024];
    unsigned i;

    hwloc_obj_type_snprintf(type, sizeof(type), obj, 0);
    printf("%*s%s", 2*depth, "", type);
    if (obj->os_index != (unsigned) -1)
      printf("#%u", obj->os_index);
    hwloc_obj_attr_snprintf(attr, sizeof(attr), obj, " ", 0);
    if (*attr)
      printf("(%s)", attr);
    printf("\n");
    for (i = 0; i < obj->arity; i++) {
        print_children(topology, obj->children[i], depth + 1);
    }
}

int main(void)
{
    int depth;
    unsigned i, n;
    unsigned long size;
    int levels;
    char string[128];
    int topodepth;
    void *m;
    hwloc_topology_t topology;
    hwloc_cpuset_t cpuset;
    hwloc_obj_t obj;

    /* Allocate and initialize topology object\&. */
    hwloc_topology_init(&topology);

    /* \&.\&.\&. Optionally, put detection configuration here to ignore
       some objects types, define a synthetic topology, etc\&.\&.\&.\&.

       The default is to detect all the objects of the machine that
       the caller is allowed to access\&.  See Configure Topology
       Detection\&. */

    /* Perform the topology detection\&. */
    hwloc_topology_load(topology);

    /* Optionally, get some additional topology information
       in case we need the topology depth later\&. */
    topodepth = hwloc_topology_get_depth(topology);

    /*****************************************************************
     * First example:
     * Walk the topology with an array style, from level 0 (always
     * the system level) to the lowest level (always the proc level)\&.
     *****************************************************************/
    for (depth = 0; depth < topodepth; depth++) {
        printf("*** Objects at level %d\n", depth);
        for (i = 0; i < hwloc_get_nbobjs_by_depth(topology, depth);
             i++) {
            hwloc_obj_type_snprintf(string, sizeof(string),
                                    hwloc_get_obj_by_depth(topology, depth, i), 0);
            printf("Index %u: %s\n", i, string);
        }
    }

    /*****************************************************************
     * Second example:
     * Walk the topology with a tree style\&.
     *****************************************************************/
    printf("*** Printing overall tree\n");
    print_children(topology, hwloc_get_root_obj(topology), 0);

    /*****************************************************************
     * Third example:
     * Print the number of packages\&.
     *****************************************************************/
    depth = hwloc_get_type_depth(topology, HWLOC_OBJ_PACKAGE);
    if (depth == HWLOC_TYPE_DEPTH_UNKNOWN) {
        printf("*** The number of packages is unknown\n");
    } else {
        printf("*** %u package(s)\n",
               hwloc_get_nbobjs_by_depth(topology, depth));
    }

    /*****************************************************************
     * Fourth example:
     * Compute the amount of cache that the first logical processor
     * has above it\&.
     *****************************************************************/
    levels = 0;
    size = 0;
    for (obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_PU, 0);
         obj;
         obj = obj->parent)
      if (hwloc_obj_type_is_cache(obj->type)) {
        levels++;
        size += obj->attr->cache\&.size;
      }
    printf("*** Logical processor 0 has %d caches totaling %luKB\n",
           levels, size / 1024);

    /*****************************************************************
     * Fifth example:
     * Bind to only one thread of the last core of the machine\&.
     *
     * First find out where cores are, or else smaller sets of CPUs if
     * the OS doesn't have the notion of a "core"\&.
     *****************************************************************/
    depth = hwloc_get_type_or_below_depth(topology, HWLOC_OBJ_CORE);

    /* Get last core\&. */
    obj = hwloc_get_obj_by_depth(topology, depth,
                   hwloc_get_nbobjs_by_depth(topology, depth) - 1);
    if (obj) {
        /* Get a copy of its cpuset that we may modify\&. */
        cpuset = hwloc_bitmap_dup(obj->cpuset);

        /* Get only one logical processor (in case the core is
           SMT/hyper-threaded)\&. */
        hwloc_bitmap_singlify(cpuset);

        /* And try to bind ourself there\&. */
        if (hwloc_set_cpubind(topology, cpuset, 0)) {
            char *str;
            int error = errno;
            hwloc_bitmap_asprintf(&str, obj->cpuset);
            printf("Couldn't bind to cpuset %s: %s\n", str, strerror(error));
            free(str);
        }

        /* Free our cpuset copy */
        hwloc_bitmap_free(cpuset);
    }

    /*****************************************************************
     * Sixth example:
     * Allocate some memory on the last NUMA node, bind some existing
     * memory to the last NUMA node\&.
     *****************************************************************/
    /* Get last node\&. There's always at least one\&. */
    n = hwloc_get_nbobjs_by_type(topology, HWLOC_OBJ_NUMANODE);
    obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_NUMANODE, n - 1);

    size = 1024*1024;
    m = hwloc_alloc_membind(topology, size, obj->nodeset,
                            HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_BYNODESET);
    hwloc_free(topology, m, size);

    m = malloc(size);
    hwloc_set_area_membind(topology, m, size, obj->nodeset,
                           HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_BYNODESET);
    free(m);

    /* Destroy topology object\&. */
    hwloc_topology_destroy(topology);

    return 0;
}
.fi
.PP
.PP
hwloc provides a \fCpkg-config\fP executable to obtain relevant compiler and linker flags\&. For example, it can be used thusly to compile applications that utilize the hwloc library (assuming GNU Make):
.PP
.PP
.nf
CFLAGS += $(shell pkg-config --cflags hwloc)
LDLIBS += $(shell pkg-config --libs hwloc)

hwloc-hello: hwloc-hello.c
        $(CC) hwloc-hello.c $(CFLAGS) -o hwloc-hello $(LDLIBS)
.fi
.PP
.PP
On a machine 2 processor packages -- each package of which has two processing cores -- the output from running \fChwloc-hello\fP could be something like the following:
.PP
.PP
.nf
shell$ ./hwloc-hello
*** Objects at level 0
Index 0: Machine
*** Objects at level 1
Index 0: Package#0
Index 1: Package#1
*** Objects at level 2
Index 0: Core#0
Index 1: Core#1
Index 2: Core#3
Index 3: Core#2
*** Objects at level 3
Index 0: PU#0
Index 1: PU#1
Index 2: PU#2
Index 3: PU#3
*** Printing overall tree
Machine
  Package#0
    Core#0
      PU#0
    Core#1
      PU#1
  Package#1
    Core#3
      PU#2
    Core#2
      PU#3
*** 2 package(s)
*** Logical processor 0 has 0 caches totaling 0KB
shell$ 
.fi
.PP
.PP
 
.SH "History / Credits"
.PP
hwloc is the evolution and merger of the libtopology project and the Portable Linux Processor Affinity (PLPA) (https://www.open-mpi.org/projects/plpa/) project\&. Because of functional and ideological overlap, these two code bases and ideas were merged and released under the name 'hwloc' as an Open MPI sub-project\&.
.PP
libtopology was initially developed by the Inria Runtime Team-Project\&. PLPA was initially developed by the Open MPI development team as a sub-project\&. Both are now deprecated in favor of hwloc, which is distributed as an Open MPI sub-project\&.
.PP
 
.SH "Further Reading"
.PP
The documentation chapters include
.PP
.PD 0
.IP "\(bu" 2
\fBTerms and Definitions\fP 
.IP "\(bu" 2
\fBCommand-Line Tools\fP 
.IP "\(bu" 2
\fBEnvironment Variables\fP 
.IP "\(bu" 2
\fBCPU and Memory Binding Overview\fP 
.IP "\(bu" 2
\fBI/O Devices\fP 
.IP "\(bu" 2
\fBMiscellaneous objects\fP 
.IP "\(bu" 2
\fBObject attributes\fP 
.IP "\(bu" 2
\fBTopology Attributes: Distances, Memory Attributes and CPU Kinds\fP 
.IP "\(bu" 2
\fBImporting and exporting topologies from/to XML files\fP 
.IP "\(bu" 2
\fBSynthetic topologies\fP 
.IP "\(bu" 2
\fBInteroperability With Other Software\fP 
.IP "\(bu" 2
\fBThread Safety\fP 
.IP "\(bu" 2
\fBComponents and plugins\fP 
.IP "\(bu" 2
\fBEmbedding hwloc in Other Software\fP 
.IP "\(bu" 2
\fBFrequently Asked Questions\fP 
.IP "\(bu" 2
\fBUpgrading to the hwloc 2\&.0 API\fP 
.PP
.PP
Make sure to have had a look at those too!
.PP