1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
|
Development
===========
Prerequisites
-------------
You need to have the following software properly installed to develop
*MPI for Python*:
* `Python`_ 3.8 or above.
* The `Cython`_ compiler.
* A working `MPI`_ implementation like `MPICH`_ or `Open MPI`_,
preferably supporting MPI-4 and built with shared/dynamic libraries.
Optionally, consider installing the following packages:
* `NumPy`_ for enabling comprehensive testing of MPI communication.
* `CuPy`_ for enabling comprehensive testing with a GPU-aware MPI.
* `Sphinx`_ to build the documentation.
.. tip::
Most routine development tasks like building, installing in
editable mode, testing, and generating documentation can be
performed with the `spin`_ developer tool. Run :command:`spin` at
the top level source directory for a list of available subcommands.
.. _Python: https://www.python.org/
.. _Cython: https://cython.org/
.. _MPI: https://www.mpi-forum.org/
.. _MPICH: https://www.mpich.org/
.. _Open MPI: https://www.open-mpi.org/
.. _NumPy: https://numpy.org/
.. _CuPy: https://cupy.dev/
.. _Sphinx: https://www.sphinx-doc.org/
.. _spin: https://github.com/scientific-python/spin
Building
--------
*MPI for Python* uses **setuptools**-based build system that relies on
the :file:`setup.py` file. Some setuptools commands (e.g., *build*)
accept additional options:
.. cmdoption:: --mpi=
Lets you pass a section with MPI configuration within a special
configuration file. Alternatively, you can use the :envvar:`MPICFG`
environment variable.
.. cmdoption:: --mpicc=
Specify the path or name of the :program:`mpicc` C compiler wrapper.
Alternatively, use the :envvar:`MPICC` environment variable.
.. cmdoption:: --mpild=
Specify the full path or name for the MPI-aware C linker.
Alternatively, use the :envvar:`MPILD` environment variable. If
not set, the :program:`mpicc` C compiler wrapper is used for
linking.
.. cmdoption:: --configure
Runs exhaustive tests for checking about missing MPI types,
constants, and functions. This option should be passed in order to
build *MPI for Python* against old MPI-1, MPI-2, or MPI-3
implementations, possibly providing a subset of MPI-4.
If you use a MPI implementation providing a :program:`mpicc` C
compiler wrapper (e.g., MPICH or Open MPI), it will be used for
compilation and linking. This is the preferred and easiest way to
build *MPI for Python*.
If :program:`mpicc` is found in the executable search path
(:envvar:`PATH` environment variable), simply run the *build*
command::
$ python setup.py build
If :program:`mpicc` is not in your search path or the compiler wrapper
has a different name, you can run the *build* command specifying its
location, either via the :option:`--mpicc` command option or using the
:envvar:`MPICC` environment variable::
$ python setup.py build --mpicc=/path/to/mpicc
$ env MPICC=/path/to/mpicc python setup.py build
Alternatively, you can provide all the relevant information about your
MPI implementation by editing the :file:`mpi.cfg` file located in the
top level source directory. You can use the default section ``[mpi]``
or add a new custom section, for example ``[vendor_mpi]`` (see the
examples provided in the :file:`mpi.cfg` file as a starting point to
write your own section):
.. code-block:: ini
[mpi]
include_dirs = /usr/local/mpi/include
libraries = mpi
library_dirs = /usr/local/mpi/lib
runtime_library_dirs = /usr/local/mpi/lib
[vendor_mpi]
include_dirs = /opt/mpi/include ...
libraries = mpi ...
library_dirs = /opt/mpi/lib ...
runtime_library_dirs = /opt/mpi/lib ...
...
and then run the *build* command specifying you custom
configuration section::
$ python setup.py build --mpi=vendor_mpi
$ env MPICFG=vendor_mpi python setup.py build
Installing
----------
*MPI for Python* can be installed in editable mode::
$ python -m pip install --editable .
After modifying Cython sources, an in-place rebuild is needed::
$ python setup.py build --inplace
Testing
-------
To quickly test the installation::
$ mpiexec -n 5 python -m mpi4py.bench helloworld
Hello, World! I am process 0 of 5 on localhost.
Hello, World! I am process 1 of 5 on localhost.
Hello, World! I am process 2 of 5 on localhost.
Hello, World! I am process 3 of 5 on localhost.
Hello, World! I am process 4 of 5 on localhost.
$ mpiexec -n 5 python -m mpi4py.bench ringtest -l 10 -n 1048576
time for 10 loops = 0.00361614 seconds (5 processes, 1048576 bytes)
If you installed from a git clone or the source distribution, issuing
at the command line::
$ mpiexec -n 5 python demo/helloworld.py
will launch a five-process run of the Python interpreter and run the
demo script :file:`demo/helloworld.py` from the source distribution.
You can also run all the *unittest* scripts::
$ mpiexec -n 5 python test/main.py
or, if you have the `pytest`_ unit testing framework installed::
$ mpiexec -n 5 pytest
.. _pytest: https://docs.pytest.org/
|
