1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
|
.. _Controlling files in the distribution:
Controlling files in the distribution
=====================================
For the most common use cases, ``setuptools`` will automatically find out which
files are necessary for distributing the package. More precisely, the following
files are included in a source distribution by default:
- :term:`pure Python module <Pure Module>` files implied by the ``py-modules`` and ``packages``
configuration parameters in ``pyproject.toml`` and/or equivalent
in ``setup.cfg``/``setup.py``;
- C source files mentioned in the ``ext_modules`` or ``libraries``
``setup()`` arguments;
- Files that match the following glob patterns: ``tests/test*.py``,
``test/test*.py``;
- Scripts specified by the ``scripts-files`` configuration parameter
in ``pyproject.toml`` or ``scripts`` in ``setup.py``/``setup.cfg``;
- All files specified by the ``package-data`` and ``data-files``
configuration parameters in ``pyproject.toml`` and/or equivalent
in ``setup.cfg``/``setup.py``;
- The file specified by the ``license_file`` option in ``setup.cfg``;
- All files specified by the ``license-files`` configuration parameter
in ``pyproject.toml`` and/or equivalent in ``setup.cfg``/``setup.py``;
note that if you don't explicitly set this parameter, ``setuptools``
will include any files that match the following glob patterns:
``LICEN[CS]E*``, ``COPYING*``, ``NOTICE*``, ``AUTHORS**``;
- ``pyproject.toml``;
- ``setup.cfg``;
- ``setup.py``;
- ``README``, ``README.txt``, ``README.rst`` or ``README.md``;
- ``MANIFEST.in``
Please note that the list above is guaranteed to work with the last stable version
of ``setuptools``. The behavior of older versions might differ.
.. note::
.. versionadded:: v69.0.0
``setuptools`` will attempt to include type information files
by default in the distribution
(``.pyi`` and ``py.typed``, as specified in :pep:`561`),
as long as they are contained inside of a package directory
(for the time being there is no automatic support for top-level ``.pyi`` files).
*Please note however that this feature is* **EXPERIMENTAL** *and may change in
the future.*
If you have ``.pyi`` and ``py.typed`` files in your project, but do not
wish to distribute them, you can opt out by setting
:doc:`exclude-package-data </userguide/datafiles>` to remove them.
However, when building more complex packages (e.g. packages that include
non-Python files, or that need to use custom C headers), you might find that
not all files present in your project folder are included in package
:term:`distribution archive <Distribution Package>`.
If you are using a :wiki:`Revision Control System`, such as git_ or mercurial_,
and your source distributions only need to include files that you're
tracking in revision control, you can use a ``setuptools`` :ref:`plugin <Adding
Support for Revision Control Systems>`, such as :pypi:`setuptools-scm` or
:pypi:`setuptools-svn` to automatically include all tracked files into the ``sdist``.
.. _Using MANIFEST.in:
Alternatively, if you need finer control over the files (e.g. you don't want to
distribute :wiki:`CI/CD`-related files) or you need automatically generated files,
you can add a ``MANIFEST.in`` file at the root of your project,
to specify any files that the default file location algorithm doesn't catch.
This file contains instructions that tell ``setuptools`` which files exactly
should be part of the ``sdist`` (or not).
.. attention::
Please note that ``setuptools`` supports the ``MANIFEST.in``,
and not ``MANIFEST`` (no extension). Any documentation, tutorial or example
that recommends using ``MANIFEST`` (no extension) is likely outdated.
.. tip::
The ``MANIFEST.in`` file contains commands that allow you to discover and
manipulate lists of files. There are many commands that can be used with
different objectives, but you should try to not make your ``MANIFEST.in``
file too fine grained.
A good idea is to start with a ``graft`` command (to add all
files inside a set of directories) and then fine tune the file selection
by removing the excess or adding isolated files.
A :file:`MANIFEST.in` file consists of commands, one per line, instructing
setuptools to add or remove some set of files from the sdist. The commands
are:
========================================================= ==================================================================================================
Command Description
========================================================= ==================================================================================================
:samp:`include {pat1} {pat2} ...` Add all files matching any of the listed patterns
(Files must be given as paths relative to the root of the project)
:samp:`exclude {pat1} {pat2} ...` Remove all files matching any of the listed patterns
(Files must be given as paths relative to the root of the project)
:samp:`recursive-include {dir-pattern} {pat1} {pat2} ...` Add all files under directories matching ``dir-pattern`` that match any of the listed patterns
:samp:`recursive-exclude {dir-pattern} {pat1} {pat2} ...` Remove all files under directories matching ``dir-pattern`` that match any of the listed patterns
:samp:`global-include {pat1} {pat2} ...` Add all files anywhere in the source tree matching any of the listed patterns
:samp:`global-exclude {pat1} {pat2} ...` Remove all files anywhere in the source tree matching any of the listed patterns
:samp:`graft {dir-pattern}` Add all files under directories matching ``dir-pattern``
:samp:`prune {dir-pattern}` Remove all files under directories matching ``dir-pattern``
========================================================= ==================================================================================================
The patterns here are glob-style patterns: ``*`` matches zero or more regular
filename characters (on Unix, everything except forward slash; on Windows,
everything except backslash and colon); ``?`` matches a single regular filename
character, and ``[chars]`` matches any one of the characters between the square
brackets (which may contain character ranges, e.g., ``[a-z]`` or
``[a-fA-F0-9]``). Setuptools also has support for ``**`` matching
zero or more characters including forward slash, backslash, and colon.
Directory patterns are relative to the root of the project directory; e.g.,
``graft example*`` will include a directory named :file:`examples` in the
project root but will not include :file:`docs/examples/`.
File & directory names in :file:`MANIFEST.in` should be ``/``-separated;
setuptools will automatically convert the slashes to the local platform's
appropriate directory separator.
Commands are processed in the order they appear in the :file:`MANIFEST.in`
file. For example, given the commands:
.. code-block:: bash
graft tests
global-exclude *.py[cod]
the contents of the directory tree :file:`tests` will first be added to the
sdist, and then after that all files in the sdist with a ``.pyc``, ``.pyo``, or
``.pyd`` extension will be removed from the sdist. If the commands were in the
opposite order, then ``*.pyc`` files etc. would be only be removed from what
was already in the sdist before adding :file:`tests`, and if :file:`tests`
happened to contain any ``*.pyc`` files, they would end up included in the
sdist because the exclusion happened before they were included.
An example of ``MANIFEST.in`` for a simple project that organized according to a
:ref:`src-layout` is:
.. code-block:: bash
# MANIFEST.in -- just for illustration
graft src
graft tests
graft docs
# `-> adds all files inside a directory
include tox.ini
# `-> matches file paths relative to the root of the project
global-exclude *~ *.py[cod] *.so
# `-> matches file names (regardless of directory)
Once the correct files are present in the ``sdist``, they can then be used by
binary extensions during the build process, or included in the final
:term:`wheel <Wheel>` [#build-process]_ if you configure ``setuptools`` with
``include_package_data=True``.
.. important::
Please note that, when using ``include_package_data=True``, only files **inside
the package directory** are included in the final ``wheel``, by default.
So for example, if you create a :term:`Python project <Project>` that uses
:pypi:`setuptools-scm` and have a ``tests`` directory outside of the package
folder, the ``tests`` directory will be present in the ``sdist`` but not in the
``wheel`` [#wheel-vs-sdist]_.
See :doc:`/userguide/datafiles` for more information.
.. _Caching and Troubleshooting:
Caching and Troubleshooting
===========================
Setuptools automatically creates a few directories to host build artefacts and
cache files, such as ``build``, ``dist``, ``*.egg-info``. While cache is
useful to speed up incremental builds, in some edge cases it might become
stale. If you feel that caching is causing problems to your build, specially
after changes in configuration or in the directory/file structure., consider
removing ``build``, ``dist``, ``*.egg-info`` [#PKG-INFO]_ before rebuilding or
reinstalling your project.
----
.. [#build-process]
You can think about the build process as two stages: first the ``sdist``
will be created and then the ``wheel`` will be produced from that ``sdist``.
.. [#wheel-vs-sdist]
This happens because the ``sdist`` can contain files that are useful during
development or the build process itself, but not in runtime (e.g. tests,
docs, examples, etc...).
The ``wheel``, on the other hand, is a file format that has been optimized
and is ready to be unpacked into a running installation of Python or
:term:`Virtual Environment`.
Therefore it only contains items that are required during runtime.
.. [#PKG-INFO]
When working from an extracted sdist (e.g. for patching), you might also consider removing
the ``PKG-INFO`` file to force its recreation.
.. _git: https://git-scm.com
.. _mercurial: https://www.mercurial-scm.org
|
