1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
|
<!DOCTYPE html>
<html class="writer-html5" lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>4.15. Advice for packagers — Open MPI 5.0.7 documentation</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="../_static/css/theme.css" />
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/sphinx_highlight.js"></script>
<script src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="4.16. Advice for System Administrators" href="sysadmins.html" />
<link rel="prev" title="4.14. Updating or upgrading an Open MPI installation" href="updating.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home">
Open MPI
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../quickstart.html">1. Quick start</a></li>
<li class="toctree-l1"><a class="reference internal" href="../getting-help.html">2. Getting help</a></li>
<li class="toctree-l1"><a class="reference internal" href="../release-notes/index.html">3. Release notes</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">4. Building and installing Open MPI</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="quickstart.html">4.1. Quick start: Installing Open MPI</a></li>
<li class="toctree-l2"><a class="reference internal" href="downloading.html">4.2. Downloading Open MPI</a></li>
<li class="toctree-l2"><a class="reference internal" href="supported-systems.html">4.3. Supported Systems</a></li>
<li class="toctree-l2"><a class="reference internal" href="definitions.html">4.4. Definitions</a></li>
<li class="toctree-l2"><a class="reference internal" href="filesystem-requirements.html">4.5. Filesystem requirements</a></li>
<li class="toctree-l2"><a class="reference internal" href="vpath-builds.html">4.6. VPATH builds</a></li>
<li class="toctree-l2"><a class="reference internal" href="compilers-and-flags.html">4.7. Specifying compilers and flags</a></li>
<li class="toctree-l2"><a class="reference internal" href="required-support-libraries.html">4.8. Required support libraries</a></li>
<li class="toctree-l2"><a class="reference internal" href="configure-cli-options/index.html">4.9. <code class="docutils literal notranslate"><span class="pre">configure</span></code> command line options</a></li>
<li class="toctree-l2"><a class="reference internal" href="configure-output-summary.html">4.10. <code class="docutils literal notranslate"><span class="pre">configure</span></code> output summary</a></li>
<li class="toctree-l2"><a class="reference internal" href="make-targets.html">4.11. <code class="docutils literal notranslate"><span class="pre">make</span></code> targets</a></li>
<li class="toctree-l2"><a class="reference internal" href="installation-location.html">4.12. Installation location</a></li>
<li class="toctree-l2"><a class="reference internal" href="custom-components.html">4.13. Installing custom components</a></li>
<li class="toctree-l2"><a class="reference internal" href="updating.html">4.14. Updating or upgrading an Open MPI installation</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">4.15. Advice for packagers</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#do-not-use-open-mpi-s-internal-dependent-libraries">4.15.1. Do not use Open MPI’s internal dependent libraries</a></li>
<li class="toctree-l3"><a class="reference internal" href="#have-sphinx-installed">4.15.2. Have Sphinx installed</a></li>
<li class="toctree-l3"><a class="reference internal" href="#components-plugins-static-or-dso">4.15.3. Components (“plugins”): static or DSO?</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#why-build-components-as-dsos">4.15.3.1. Why build components as DSOs?</a></li>
<li class="toctree-l4"><a class="reference internal" href="#why-build-components-as-part-of-open-mpi-s-core-libraries">4.15.3.2. Why build components as part of Open MPI’s core libraries?</a></li>
<li class="toctree-l4"><a class="reference internal" href="#direct-controls-for-building-components-as-dsos-or-not">4.15.3.3. Direct controls for building components as DSOs or not</a></li>
<li class="toctree-l4"><a class="reference internal" href="#gnu-libtool-dependency-flattening">4.15.3.4. GNU Libtool dependency flattening</a></li>
<li class="toctree-l4"><a class="reference internal" href="#building-accelerator-support-as-dsos">4.15.3.5. Building accelerator support as DSOs</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="sysadmins.html">4.16. Advice for System Administrators</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../features/index.html">5. Open MPI-specific features</a></li>
<li class="toctree-l1"><a class="reference internal" href="../validate.html">6. Validating your installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../version-numbering.html">7. Version numbers and compatibility</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mca.html">8. The Modular Component Architecture (MCA)</a></li>
<li class="toctree-l1"><a class="reference internal" href="../building-apps/index.html">9. Building MPI applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="../launching-apps/index.html">10. Launching MPI applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="../tuning-apps/index.html">11. Run-time operation and tuning MPI applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="../app-debug/index.html">12. Debugging Open MPI Parallel Applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="../developers/index.html">13. Developer’s guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../contributing.html">14. Contributing to Open MPI</a></li>
<li class="toctree-l1"><a class="reference internal" href="../license/index.html">15. License</a></li>
<li class="toctree-l1"><a class="reference internal" href="../history.html">16. History of Open MPI</a></li>
<li class="toctree-l1"><a class="reference internal" href="../man-openmpi/index.html">17. Open MPI manual pages</a></li>
<li class="toctree-l1"><a class="reference internal" href="../man-openshmem/index.html">18. OpenSHMEM manual pages</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">Open MPI</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="index.html"><span class="section-number">4. </span>Building and installing Open MPI</a></li>
<li class="breadcrumb-item active"><span class="section-number">4.15. </span>Advice for packagers</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/installing-open-mpi/packagers.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<style>
.wy-table-responsive table td,.wy-table-responsive table th{white-space:normal}
</style><div class="section" id="advice-for-packagers">
<span id="label-install-packagers"></span><h1><span class="section-number">4.15. </span>Advice for packagers<a class="headerlink" href="#advice-for-packagers" title="Permalink to this heading"></a></h1>
<div class="section" id="do-not-use-open-mpi-s-internal-dependent-libraries">
<span id="label-install-packagers-do-not-use-internal"></span><h2><span class="section-number">4.15.1. </span>Do not use Open MPI’s internal dependent libraries<a class="headerlink" href="#do-not-use-open-mpi-s-internal-dependent-libraries" title="Permalink to this heading"></a></h2>
<p>The Open MPI community <strong>strongly</strong> suggests that binary Open MPI
packages should <em>not</em> include Hwloc, Libevent, PMIx, or PRRTE.
<a class="reference internal" href="required-support-libraries.html#label-install-required-support-libraries"><span class="std std-ref">Although several of these libraries are required by Open MPI</span></a> (and are therefore bundled
in the Open MPI source code distribution for end-user convenience),
binary Open MPI packages should limit themselves solely to Open MPI
artifacts. Specifically: ensure to configure and build Open MPI
against external installations of these required packages.</p>
<p>Packagers may therefore wish to configure Open MPI with something like
the following:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="c1"># Install Sphinx so that Open MPI can re-build its docs with the</span>
<span class="c1"># installed PRRTE's docs</span>
virtualalenv<span class="w"> </span>venv
.<span class="w"> </span>./venv/bin/activate
pip<span class="w"> </span>install<span class="w"> </span>docs/requirements.txt
./configure<span class="w"> </span>--with-libevent<span class="o">=</span>external<span class="w"> </span>--with-hwloc<span class="o">=</span>external<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--with-pmix<span class="o">=</span>external<span class="w"> </span>--with-prrte<span class="o">=</span>external<span class="w"> </span>...
</pre></div>
</div>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>Note the installation of the Sphinx tool so that Open
MPI can re-build its documentation with the external
PRRTE’s documentation.</p>
<p>Failure to do this will mean Open MPI’s documentation
will be correct for the version of PRRTE that is
bundled in the Open MPI distribution, but may not be
entirely correct for the version of PRRTE that you are
building against.</p>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">external</span></code> keywords will force Open MPI’s <code class="docutils literal notranslate"><span class="pre">configure</span></code> to
ignore all the bundled libraries and only look for external versions
of these support libraries. This also has the benefit of causing
<code class="docutils literal notranslate"><span class="pre">configure</span></code> to fail if it cannot find the required support libraries
outside of the Open MPI source tree — a good sanity check to
ensure that your package is correctly relying on the
independently-built and installed versions.</p>
<p><a class="reference internal" href="configure-cli-options/required-support-libraries.html#label-building-ompi-cli-options-required-support-libraries"><span class="std std-ref">See this section</span></a> for more
information about the required support library <code class="docutils literal notranslate"><span class="pre">--with-FOO</span></code> command
line options.</p>
</div>
<div class="section" id="have-sphinx-installed">
<h2><span class="section-number">4.15.2. </span>Have Sphinx installed<a class="headerlink" href="#have-sphinx-installed" title="Permalink to this heading"></a></h2>
<p>Since you should be (will be) installing Open MPI against an external
PRRTE and PMIx, you should have <a class="reference external" href="https://www.sphinx-doc.org/">Sphinx</a> installed before running Open MPI’s
<code class="docutils literal notranslate"><span class="pre">configure</span></code> script.</p>
<p>This will allow Open MPI to (re-)build its documentation according to
the PMIx and PRRTE that you are building against.</p>
<p>To be clear: the Open MPI distribution tarball comes with pre-built
documentation — rendered in HTML and nroff — that is
suitable for the versions of PRRTE and PMIx that are bundled in that
tarball.</p>
<p>However, if you are building Open MPI against not-bundled versions of
PRRTE / PMIx (as all packagers should be), Open MPI needs to re-build
its documentation with specific information from those external PRRTE
/ PMIx installs. For that, you need to have Sphinx installed before
running Open MPI’s <code class="docutils literal notranslate"><span class="pre">configure</span></code> script.</p>
</div>
<div class="section" id="components-plugins-static-or-dso">
<span id="label-install-packagers-dso-or-not"></span><h2><span class="section-number">4.15.3. </span>Components (“plugins”): static or DSO?<a class="headerlink" href="#components-plugins-static-or-dso" title="Permalink to this heading"></a></h2>
<p>Open MPI contains a large number of components (sometimes called
“plugins”) to effect different types of functionality in MPI. For
example, some components effect Open MPI’s networking functionality:
they may link against specialized libraries to provide
highly-optimized network access.</p>
<p>Open MPI can build its components as Dynamic Shared Objects (DSOs) or
statically included in core libraries (regardless of whether those
libraries are built as shared or static libraries).</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>As of Open MPI v5.0.7, <code class="docutils literal notranslate"><span class="pre">configure</span></code>’s global default is
to build all components as static (i.e., part of the Open
MPI core libraries, not as DSOs). Prior to Open MPI v5.0.0,
the global default behavior was to build most components as
DSOs.</p>
</div>
<div class="section" id="why-build-components-as-dsos">
<h3><span class="section-number">4.15.3.1. </span>Why build components as DSOs?<a class="headerlink" href="#why-build-components-as-dsos" title="Permalink to this heading"></a></h3>
<p>There are advantages to building components as DSOs:</p>
<ul class="simple">
<li><p>Open MPI’s core libraries — and therefore MPI applications
— will have very few dependencies. For example, if you build
Open MPI with support for a specific network stack, the libraries in
that network stack will be dependencies of the DSOs, not Open MPI’s
core libraries (or MPI applications).</p></li>
<li><p>Removing Open MPI functionality that you do not want is as simple as
removing a DSO from <code class="docutils literal notranslate"><span class="pre">$libdir/open-mpi</span></code>.</p></li>
</ul>
</div>
<div class="section" id="why-build-components-as-part-of-open-mpi-s-core-libraries">
<h3><span class="section-number">4.15.3.2. </span>Why build components as part of Open MPI’s core libraries?<a class="headerlink" href="#why-build-components-as-part-of-open-mpi-s-core-libraries" title="Permalink to this heading"></a></h3>
<p>The biggest advantage to building the components as part of Open MPI’s
core libraries is when running at (very) large scales when Open MPI is
installed on a network filesystem (vs. being installed on a local
filesystem).</p>
<p>For example, consider launching a single MPI process on each of 1,000
nodes. In this scenario, the following is accessed from the network
filesystem:</p>
<ol class="arabic simple">
<li><p>The MPI application</p></li>
<li><p>The core Open MPI libraries and their dependencies (e.g.,
<code class="docutils literal notranslate"><span class="pre">libmpi</span></code>)</p>
<ul class="simple">
<li><p>Depending on your configuration, this is probably on the order of
10-20 library files.</p></li>
</ul>
</li>
<li><p>All DSO component files and their dependencies</p>
<ul class="simple">
<li><p>Depending on your configuration, this can be 200+ component
files.</p></li>
</ul>
</li>
</ol>
<p>If all components are physically located in the libraries, then the
third step loads zero DSO component files. When using a networked
filesystem while launching at scale, this can translate to large
performance savings.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>If not using a networked filesystem, or if not launching at
scale, loading a large number of DSO files may not consume a
noticeable amount of time during MPI process launch. Put
simply: loading DSOs as indvidual files generally only
matters when using a networked filesystem while launching at
scale.</p>
</div>
</div>
<div class="section" id="direct-controls-for-building-components-as-dsos-or-not">
<h3><span class="section-number">4.15.3.3. </span>Direct controls for building components as DSOs or not<a class="headerlink" href="#direct-controls-for-building-components-as-dsos-or-not" title="Permalink to this heading"></a></h3>
<p>Open MPI v5.0.7 has two <code class="docutils literal notranslate"><span class="pre">configure</span></code>-time defaults regarding the
treatment of components that may be of interest to packagers:</p>
<ol class="arabic">
<li><p>Open MPI’s libraries default to building as shared libraries
(vs. static libraries). For example, on Linux, Open MPI will
default to building <code class="docutils literal notranslate"><span class="pre">libmpi.so</span></code> (vs. <code class="docutils literal notranslate"><span class="pre">libmpi.a</span></code>).</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>See the descriptions of <code class="docutils literal notranslate"><span class="pre">--disable-shared</span></code> and
<code class="docutils literal notranslate"><span class="pre">--enable-static</span></code> <a class="reference internal" href="configure-cli-options/installation.html#label-building-installation-cli-options"><span class="std std-ref">in this section</span></a> for more
details about how to change this default.</p>
<p>Also be sure to <a class="reference internal" href="../building-apps/building-static-apps.html#label-building-fully-static-apps"><span class="std std-ref">see this warning about building
static apps</span></a>.</p>
</div>
</li>
<li><p>Open MPI will default to including its components in its libraries
(as opposed to being compiled as dynamic shared objects, or DSOs).
For example, <code class="docutils literal notranslate"><span class="pre">libmpi.so</span></code> on Linux systems will contain the UCX
PML component, instead of the UCX PML being compiled into
<code class="docutils literal notranslate"><span class="pre">mca_pml_ucx.so</span></code> and dynamically opened at run time via
<code class="docutils literal notranslate"><span class="pre">dlopen(3)</span></code>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>See the descriptions of <code class="docutils literal notranslate"><span class="pre">--enable-mca-dso</span></code> and
<code class="docutils literal notranslate"><span class="pre">--enable-mca-static</span></code> <a class="reference internal" href="configure-cli-options/installation.html#label-building-installation-cli-options"><span class="std std-ref">in this section</span></a> for more
details about how to change this defaults.</p>
</div>
</li>
</ol>
<p>A side effect of these two defaults is that all the components
included in the Open MPI libraries will bring their dependencies with
them. For example (on Linux), if the XYZ PML component in the MPI
layer requires <code class="docutils literal notranslate"><span class="pre">libXYZ.so</span></code>, then these defaults mean that
<code class="docutils literal notranslate"><span class="pre">libmpi.so</span></code> will depend on <code class="docutils literal notranslate"><span class="pre">libXYZ.so</span></code>. This dependency will
likely be telegraphed into the Open MPI binary package that includes
<code class="docutils literal notranslate"><span class="pre">libmpi.so</span></code>.</p>
<p>Conversely, if the XYZ PML component was built as a DSO, then —
assuming no other parts of Open MPI require <code class="docutils literal notranslate"><span class="pre">libXYZ.so</span></code> —
<code class="docutils literal notranslate"><span class="pre">libmpi.so</span></code> would <em>not</em> be dependent on <code class="docutils literal notranslate"><span class="pre">libXYZ.so</span></code>. Instead, the
<code class="docutils literal notranslate"><span class="pre">mca_pml_xyz.so</span></code> DSO would have the dependency upon <code class="docutils literal notranslate"><span class="pre">libXYZ.so</span></code>.</p>
<p>Packagers can use these facts to potentially create multiple binary
Open MPI packages, each with different dependencies by, for example,
using <code class="docutils literal notranslate"><span class="pre">--enable-mca-dso</span></code> to selectively build some components as
DSOs and leave the others included in their respective Open MPI
libraries.</p>
<p><a class="reference internal" href="#label-install-packagers-building-accelerator-support-as-dsos"><span class="std std-ref">See the section on building accelerator support</span></a> for a
practical example where this can be useful.</p>
</div>
<div class="section" id="gnu-libtool-dependency-flattening">
<span id="label-install-packagers-gnu-libtool-dependency-flattening"></span><h3><span class="section-number">4.15.3.4. </span>GNU Libtool dependency flattening<a class="headerlink" href="#gnu-libtool-dependency-flattening" title="Permalink to this heading"></a></h3>
<p>When compiling Open MPI’s components statically as part of Open MPI’s
core libraries, <a class="reference external" href="https://www.gnu.org/software/libtool/">GNU Libtool</a>
— which is used as part of Open MPI’s build system — will
attempt to “flatten” dependencies.</p>
<p>For example, the <a class="reference internal" href="../man-openmpi/man1/ompi_info.1.html#man1-ompi-info"><span class="std std-ref">ompi_info(1)</span></a> command links
against the Open MPI core library <code class="docutils literal notranslate"><span class="pre">libopen-pal</span></code>. This library will
have dependencies on various HPC-class network stack libraries. For
simplicity, the discussion below assumes that Open MPI was built with
support for <a class="reference external" href="https://libfabric.org/">Libfabric</a> and <a class="reference external" href="https://openucx.org/">UCX</a>, and therefore <code class="docutils literal notranslate"><span class="pre">libopen-pal</span></code> has direct
dependencies on <code class="docutils literal notranslate"><span class="pre">libfabric</span></code> and <code class="docutils literal notranslate"><span class="pre">libucx</span></code>.</p>
<p>In this scenario, GNU Libtool will automatically attempt to “flatten”
these dependencies by linking <a class="reference internal" href="../man-openmpi/man1/ompi_info.1.html#man1-ompi-info"><span class="std std-ref">ompi_info(1)</span></a>
directly to <code class="docutils literal notranslate"><span class="pre">libfabric</span></code> and <code class="docutils literal notranslate"><span class="pre">libucx</span></code> (vs. letting <code class="docutils literal notranslate"><span class="pre">libopen-pal</span></code>
pull the dependencies in at run time).</p>
<ul class="simple">
<li><p>In some environments (e.g., Ubuntu 22.04), the compiler and/or
linker will automatically utilize the linker CLI flag
<code class="docutils literal notranslate"><span class="pre">-Wl,--as-needed</span></code>, which will effectively cause these dependencies
to <em>not</em> be flattened: <a class="reference internal" href="../man-openmpi/man1/ompi_info.1.html#man1-ompi-info"><span class="std std-ref">ompi_info(1)</span></a> will
<em>not</em> have a direct dependencies on either <code class="docutils literal notranslate"><span class="pre">libfabric</span></code> or
<code class="docutils literal notranslate"><span class="pre">libucx</span></code>.</p></li>
<li><p>In other environments (e.g., Fedora 38), the compiler and linker
will <em>not</em> utilize the <code class="docutils literal notranslate"><span class="pre">-Wl,--as-needed</span></code> linker CLI flag. As
such, <a class="reference internal" href="../man-openmpi/man1/ompi_info.1.html#man1-ompi-info"><span class="std std-ref">ompi_info(1)</span></a> will show direct
dependencies on <code class="docutils literal notranslate"><span class="pre">libfabric</span></code> and <code class="docutils literal notranslate"><span class="pre">libucx</span></code>.</p></li>
</ul>
<p><strong>Just to be clear:</strong> these flattened dependencies <em>are not a
problem</em>. Open MPI will function correctly with or without the
flattened dependencies. There is no performance impact associated
with having — or not having — the flattened dependencies.
We mention this situation here in the documentation simply because it
surprised some Open MPI downstream package managers to see that
<a class="reference internal" href="../man-openmpi/man1/ompi_info.1.html#man1-ompi-info"><span class="std std-ref">ompi_info(1)</span></a> in Open MPI v5.0.7 had more
shared library dependencies than it did in prior Open MPI releases.</p>
<p>If packagers want <a class="reference internal" href="../man-openmpi/man1/ompi_info.1.html#man1-ompi-info"><span class="std std-ref">ompi_info(1)</span></a> to not have
these flattened dependencies, use either of the following mechanisms:</p>
<ol class="arabic">
<li><p>Use <code class="docutils literal notranslate"><span class="pre">--enable-mca-dso</span></code> to force all components to be built as
DSOs (this was actually the default behavior before Open MPI v5.0.0).</p></li>
<li><p>Add <code class="docutils literal notranslate"><span class="pre">LDFLAGS=-Wl,--as-needed</span></code> to the <code class="docutils literal notranslate"><span class="pre">configure</span></code> command line
when building Open MPI.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The Open MPI community specifically chose not to
automatically utilize this linker flag for the following
reasons:</p>
<ol class="arabic simple">
<li><p>Having the flattened dependencies does not cause any
correctness or performance problems.</p></li>
<li><p>There’s multiple mechanisms (see above) for users or
packagers to change this behavior, if desired.</p></li>
<li><p>Certain environments have chosen to have — or
not have — this flattened dependency behavior.
It is not Open MPI’s place to override these choices.</p></li>
<li><p>In general, Open MPI’s <code class="docutils literal notranslate"><span class="pre">configure</span></code> script only
utilizes compiler and linker flags if they are
<em>needed</em>. All other flags should be the user’s /
packager’s choice.</p></li>
</ol>
</div>
</li>
</ol>
</div>
<div class="section" id="building-accelerator-support-as-dsos">
<span id="label-install-packagers-building-accelerator-support-as-dsos"></span><h3><span class="section-number">4.15.3.5. </span>Building accelerator support as DSOs<a class="headerlink" href="#building-accelerator-support-as-dsos" title="Permalink to this heading"></a></h3>
<p>If you are building a package that includes support for one or more
accelerators, it may be desirable to build accelerator-related
components as DSOs (see the <a class="reference internal" href="#label-install-packagers-dso-or-not"><span class="std std-ref">static or DSO?</span></a> section for details).</p>
<div class="tip admonition">
<p class="admonition-title">Rationale</p>
<p>Accelerator hardware is expensive, and may only be present on some
compute nodes in an HPC cluster. Specifically: there may not be
any accelerator hardware on “head” or compile nodes in an HPC
cluster. As such, invoking Open MPI commands on a “head” node with
an MPI that was built with static accelerator support but no
accelerator hardware may fail to launch because of run-time linker
issues (because the accelerator hardware support libraries are
likely not present).</p>
<p>Building Open MPI’s accelerator-related components as DSOs allows
Open MPI to <em>try</em> opening the accelerator components, but proceed
if those DSOs fail to open due to the lack of support libraries.</p>
</div>
<p>Use the <code class="docutils literal notranslate"><span class="pre">--enable-mca-dso</span></code> command line parameter to Open MPI’s
<code class="docutils literal notranslate"><span class="pre">configure</span></code> command can allow packagers to build all
accelerator-related components as DSO. For example:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="c1"># Build all the accelerator-related components as DSOs (all other</span>
<span class="c1"># components will default to being built in their respective</span>
<span class="c1"># libraries)</span>
shell$<span class="w"> </span>./configure<span class="w"> </span>--enable-mca-dso<span class="o">=</span>btl-smcuda,rcache-rgpusm,rcache-gpusm,accelerator
</pre></div>
</div>
<p>Per the example above, this allows packaging <code class="docutils literal notranslate"><span class="pre">$libdir</span></code> as part of
the “main” Open MPI binary package, but then packaging
<code class="docutils literal notranslate"><span class="pre">$libdir/openmpi/mca_accelerator_*.so</span></code> and the other named
components as sub-packages. These sub-packages may inherit
dependencies on the CUDA and/or ROCM packages, for example. The
“main” package can be installed on all nodes, and the
accelerator-specific subpackage can be installed on only the nodes
with accelerator hardware and support libraries.</p>
</div>
</div>
</div>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="updating.html" class="btn btn-neutral float-left" title="4.14. Updating or upgrading an Open MPI installation" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="sysadmins.html" class="btn btn-neutral float-right" title="4.16. Advice for System Administrators" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2003-2025, The Open MPI Community.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
|
