1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
|
=======================================
Build System Support
=======================================
What is it?
-------------
Python packaging has come `a long way <https://bernat.tech/posts/pep-517-518/>`_.
The traditional ``setuptools`` way of packaging Python modules
uses a ``setup()`` function within the ``setup.py`` script. Commands such as
``python setup.py bdist`` or ``python setup.py bdist_wheel`` generate a
distribution bundle and ``python setup.py install`` installs the distribution.
This interface makes it difficult to choose other packaging tools without an
overhaul. Because ``setup.py`` scripts allow for arbitrary execution, it
is difficult to provide a reliable user experience across environments
and history.
`PEP 517 <https://www.python.org/dev/peps/pep-0517/>`_ came to
the rescue and specified a new standard for packaging and distributing Python
modules. Under PEP 517:
A ``pyproject.toml`` file is used to specify which program to use
to generate the distribution.
Two functions provided by the program, ``build_wheel(directory: str)``
and ``build_sdist(directory: str)``, create the distribution bundle in the
specified ``directory``.
The program may use its own configuration file or extend the ``.toml`` file.
The actual installation is done with ``pip install *.whl`` or
``pip install *.tar.gz``. If ``*.whl`` is available, ``pip`` will go ahead and copy
its files into the ``site-packages`` directory. If not, ``pip`` will look at
``pyproject.toml`` and decide which program to use to 'build from source'.
(Note that if there is no ``pyproject.toml`` file or the ``build-backend``
parameter is not defined, then the fall-back behaviour is to use ``setuptools``.)
With this standard, switching between packaging tools is a lot easier.
How to use it?
--------------
Start with a package that you want to distribute. You will need your source
files, a ``pyproject.toml`` file and a ``setup.cfg`` file::
~/meowpkg/
pyproject.toml
setup.cfg
meowpkg/
__init__.py
module.py
The ``pyproject.toml`` file specifies the build system (i.e. what is
being used to package your scripts and install from source). To use it with
``setuptools`` the content would be::
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
``build_meta`` implements ``setuptools``' build system support.
The ``setuptools`` package implements the ``build_sdist``
command and the ``wheel`` package implements the ``build_wheel``
command; the latter is a dependency of the former
exposed via :pep:`517` hooks.
Use ``setuptools``' :ref:`declarative config <declarative config>` to
specify the package information in ``setup.cfg``::
[metadata]
name = meowpkg
version = 0.0.1
description = a package that meows
[options]
packages = find:
.. _building:
Now generate the distribution. To build the package, use
`PyPA build <https://pypa-build.readthedocs.io/en/latest/>`_::
$ pip install -q build
$ python -m build
And now it's done! The ``.whl`` file and ``.tar.gz`` can then be distributed
and installed::
dist/
meowpkg-0.0.1.whl
meowpkg-0.0.1.tar.gz
$ pip install dist/meowpkg-0.0.1.whl
or::
$ pip install dist/meowpkg-0.0.1.tar.gz
.. _backend-wrapper:
Dynamic build dependencies and other ``build_meta`` tweaks
----------------------------------------------------------
With the changes introduced by :pep:`517` and :pep:`518`, the
``setup_requires`` configuration field was deprecated in ``setup.cfg`` and
``setup.py``, in favour of directly listing build dependencies in the
``requires`` field of the ``build-system`` table of ``pyproject.toml``.
This approach has a series of advantages and gives package managers and
installers the ability to inspect the build requirements in advance and
perform a series of optimisations.
However, some package authors might still need to dynamically inspect the final
user's machine before deciding these requirements. One way of doing that, as
specified by :pep:`517`, is to "tweak" ``setuptools.build_meta`` by using an
:pep:`in-tree backend <517#in-tree-build-backends>`.
.. tip:: Before implementing an *in-tree* backend, have a look at
:pep:`PEP 508 <508#environment-markers>`. Most of the time, dependencies
with **environment markers** are enough to differentiate operating systems
and platforms.
If you put the following configuration in your ``pyproject.toml``:
.. code-block:: toml
[build-system]
requires = ["setuptools"]
build-backend = "backend"
backend-path = ["_custom_build"]
then you can implement a thin wrapper around ``build_meta`` in
the ``_custom_build/backend.py`` file, as shown in the following example:
.. code-block:: python
from setuptools import build_meta as _orig
prepare_metadata_for_build_wheel = _orig.prepare_metadata_for_build_wheel
build_wheel = _orig.build_wheel
build_sdist = _orig.build_sdist
def get_requires_for_build_wheel(self, config_settings=None):
return _orig.get_requires_for_build_wheel(config_settings) + [...]
def get_requires_for_build_sdist(self, config_settings=None):
return _orig.get_requires_for_build_sdist(config_settings) + [...]
Note that you can override any of the functions specified in :pep:`PEP 517
<517#build-backend-interface>`, not only the ones responsible for gathering
requirements.
.. important:: Make sure your backend script is included in the :doc:`source
distribution </userguide/distribution>`, otherwise the build will fail.
This can be done by using a SCM_/VCS_ plugin (like :pypi:`setuptools-scm`
and :pypi:`setuptools-svn`), or by correctly setting up :ref:`MANIFEST.in
<manifest>`.
The generated ``.tar.gz`` and ``.whl`` files are compressed archives that
can be inspected as follows:
On POSIX systems, this can be done with ``tar -tf dist/*.tar.gz``
and ``unzip -l dist/*.whl``.
On Windows systems, you can rename the ``.whl`` to ``.zip`` to be able to
inspect it from File Explorer. You can also use the above ``tar`` command in a
command prompt to inspect the ``.tar.gz`` file. Alternatively, there are GUI programs
like `7-zip`_ that handle ``.tar.gz`` and ``.whl`` files.
In general, the backend script should be present in the ``.tar.gz`` (so the
project can be built from the source) but not in the ``.whl`` (otherwise the
backend script would end up being distributed alongside your package).
See ":doc:`/userguide/package_discovery`" for more details about package
files.
.. _SCM: https://en.wikipedia.org/wiki/Software_configuration_management
.. _VCS: https://en.wikipedia.org/wiki/Version_control
.. _7-zip: https://www.7-zip.org
|
