1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
|
<!DOCTYPE html>
<html class="writer-html5" lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>10.9. Launching with PBS / Torque — Open MPI 5.0.8 documentation</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="../_static/css/theme.css" />
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/sphinx_highlight.js"></script>
<script src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="10.10. Launching with Grid Engine" href="gridengine.html" />
<link rel="prev" title="10.8. Launching with LSF" href="lsf.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home">
Open MPI
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../quickstart.html">1. Quick start</a></li>
<li class="toctree-l1"><a class="reference internal" href="../getting-help.html">2. Getting help</a></li>
<li class="toctree-l1"><a class="reference internal" href="../release-notes/index.html">3. Release notes</a></li>
<li class="toctree-l1"><a class="reference internal" href="../installing-open-mpi/index.html">4. Building and installing Open MPI</a></li>
<li class="toctree-l1"><a class="reference internal" href="../features/index.html">5. Open MPI-specific features</a></li>
<li class="toctree-l1"><a class="reference internal" href="../validate.html">6. Validating your installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../version-numbering.html">7. Version numbers and compatibility</a></li>
<li class="toctree-l1"><a class="reference internal" href="../mca.html">8. The Modular Component Architecture (MCA)</a></li>
<li class="toctree-l1"><a class="reference internal" href="../building-apps/index.html">9. Building MPI applications</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">10. Launching MPI applications</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="quickstart.html">10.1. Quick start: Launching MPI applications</a></li>
<li class="toctree-l2"><a class="reference internal" href="prerequisites.html">10.2. Prerequisites</a></li>
<li class="toctree-l2"><a class="reference internal" href="pmix-and-prrte.html">10.3. The role of PMIx and PRRTE</a></li>
<li class="toctree-l2"><a class="reference internal" href="scheduling.html">10.4. Scheduling processes across hosts</a></li>
<li class="toctree-l2"><a class="reference internal" href="localhost.html">10.5. Launching only on the local node</a></li>
<li class="toctree-l2"><a class="reference internal" href="ssh.html">10.6. Launching with SSH</a></li>
<li class="toctree-l2"><a class="reference internal" href="slurm.html">10.7. Launching with Slurm</a></li>
<li class="toctree-l2"><a class="reference internal" href="lsf.html">10.8. Launching with LSF</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">10.9. Launching with PBS / Torque</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#verify-pbs-torque-support">10.9.1. Verify PBS/Torque support</a></li>
<li class="toctree-l3"><a class="reference internal" href="#launching">10.9.2. Launching</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="gridengine.html">10.10. Launching with Grid Engine</a></li>
<li class="toctree-l2"><a class="reference internal" href="unusual.html">10.11. Unusual jobs</a></li>
<li class="toctree-l2"><a class="reference internal" href="troubleshooting.html">10.12. Troubleshooting</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tuning-apps/index.html">11. Run-time operation and tuning MPI applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="../app-debug/index.html">12. Debugging Open MPI Parallel Applications</a></li>
<li class="toctree-l1"><a class="reference internal" href="../developers/index.html">13. Developer’s guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../contributing.html">14. Contributing to Open MPI</a></li>
<li class="toctree-l1"><a class="reference internal" href="../license/index.html">15. License</a></li>
<li class="toctree-l1"><a class="reference internal" href="../history.html">16. History of Open MPI</a></li>
<li class="toctree-l1"><a class="reference internal" href="../man-openmpi/index.html">17. Open MPI manual pages</a></li>
<li class="toctree-l1"><a class="reference internal" href="../man-openshmem/index.html">18. OpenSHMEM manual pages</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">Open MPI</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item"><a href="index.html"><span class="section-number">10. </span>Launching MPI applications</a></li>
<li class="breadcrumb-item active"><span class="section-number">10.9. </span>Launching with PBS / Torque</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/launching-apps/tm.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<style>
.wy-table-responsive table td,.wy-table-responsive table th{white-space:normal}
</style><div class="section" id="launching-with-pbs-torque">
<h1><span class="section-number">10.9. </span>Launching with PBS / Torque<a class="headerlink" href="#launching-with-pbs-torque" title="Permalink to this heading"></a></h1>
<p>Open MPI supports PBS, PBS Pro, Torque, and other related resource
managers.</p>
<div class="section" id="verify-pbs-torque-support">
<h2><span class="section-number">10.9.1. </span>Verify PBS/Torque support<a class="headerlink" href="#verify-pbs-torque-support" title="Permalink to this heading"></a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">prte_info</span></code> command can be used to determine whether or not an
installed Open MPI includes Torque/PBS Pro support:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>shell$ prte_info | grep ras
</pre></div>
</div>
<p>If the Open MPI installation includes support for PBS/Torque, you
should see a line similar to that below. Note the MCA version
information varies depending on which version of Open MPI is
installed.</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>MCA ras: tm (MCA v2.1.0, API v2.0.0, Component v3.0.0)
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>PRRTE is the software layer that provides run-time
environment support to Open MPI. Open MPI typically hides most
PMIx and PRRTE details from the end user, but this is one place
that Open MPI is unable to hide the fact that PRRTE provides this
functionality, not Open MPI. Hence, users need to use the
<code class="docutils literal notranslate"><span class="pre">prte_info</span></code> command to check for PBS/Torque support (not
<code class="docutils literal notranslate"><span class="pre">ompi_info</span></code>).</p>
</div>
</div>
<div class="section" id="launching">
<h2><span class="section-number">10.9.2. </span>Launching<a class="headerlink" href="#launching" title="Permalink to this heading"></a></h2>
<p>When properly configured, Open MPI obtains both the list of hosts and
how many processes to start on each host from Torque / PBS Pro
directly. Hence, it is unnecessary to specify the <code class="docutils literal notranslate"><span class="pre">--hostfile</span></code>,
<code class="docutils literal notranslate"><span class="pre">--host</span></code>, or <code class="docutils literal notranslate"><span class="pre">-n</span></code> options to <code class="docutils literal notranslate"><span class="pre">mpirun</span></code>. Open MPI will use
PBS/Torque-native mechanisms to launch and kill processes (<code class="docutils literal notranslate"><span class="pre">ssh</span></code> is
not required).</p>
<p>For example:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="c1"># Allocate a PBS job with 4 nodes</span>
shell$<span class="w"> </span>qsub<span class="w"> </span>-I<span class="w"> </span>-lnodes<span class="o">=</span><span class="m">4</span>
<span class="c1"># Now run an Open MPI job on all the nodes allocated by PBS/Torque</span>
shell$<span class="w"> </span>mpirun<span class="w"> </span>mpi-hello-world
</pre></div>
</div>
<p>This will run the MPI processes on the nodes that were allocated by
PBS/Torque. Or, if submitting a script:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>shell$<span class="w"> </span>cat<span class="w"> </span>my_script.sh
<span class="c1">#!/bin/sh</span>
mpirun<span class="w"> </span>mpi-hello-world
shell$<span class="w"> </span>qsub<span class="w"> </span>-l<span class="w"> </span><span class="nv">nodes</span><span class="o">=</span><span class="m">4</span><span class="w"> </span>my_script.sh
</pre></div>
</div>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Do not modify <code class="docutils literal notranslate"><span class="pre">$PBS_NODEFILE</span></code>!</p>
<p>We’ve had reports from some sites that system administrators modify
the <code class="docutils literal notranslate"><span class="pre">$PBS_NODEFILE</span></code> in each job according to local policies.
This will currently cause Open MPI to behave in an unpredictable
fashion. As long as no new hosts are added to the hostfile, it
<em>usually</em> means that Open MPI will incorrectly map processes to
hosts, but in some cases it can cause Open MPI to fail to launch
processes altogether.</p>
</div>
</div>
</div>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="lsf.html" class="btn btn-neutral float-left" title="10.8. Launching with LSF" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="gridengine.html" class="btn btn-neutral float-right" title="10.10. Launching with Grid Engine" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2003-2025, The Open MPI Community.
<span class="lastupdated">Last updated on 2025-05-30 16:41:43 UTC.
</span></p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
|
