Description
What went wrong?
Whenever a pip version that's used doesn't support PEP 517/518, it just calls setup.py commands instead. This means that the dependencies declared in pyproject.toml don't get pre-installed.
This is what's been happening in Ansible Core CI since Friday.
How do we reproduce?
Click to expand the repro log
$ podman run --rm -it ubi8/ubi:8.6 sh -c 'set -x; dnf install -y python3-pip; python3 -V; pip3.6 install gssapi==1.8.0'
+ dnf install -y python3-pip
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered with an entitlement server. You can use subscription-manager to register.
Red Hat Universal Base Image 8 (RPMs) - BaseO 2.8 MB/s | 804 kB 00:00
Red Hat Universal Base Image 8 (RPMs) - AppSt 6.8 MB/s | 3.0 MB 00:00
Red Hat Universal Base Image 8 (RPMs) - CodeR 130 kB/s | 20 kB 00:00
Dependencies resolved.
==============================================================================
Package Arch Version Repository Size
==============================================================================
Installing:
python3-pip noarch 9.0.3-22.el8 ubi-8-appstream 20 k
Installing dependencies:
platform-python-pip noarch 9.0.3-22.el8 ubi-8-baseos 1.6 M
python3-setuptools noarch 39.2.0-6.el8 ubi-8-baseos 163 k
python36 x86_64 3.6.8-38.module+el8.5.0+12207+5c5719bc
ubi-8-appstream 19 k
Enabling module streams:
python36 3.6
Transaction Summary
==============================================================================
Install 4 Packages
Total download size: 1.8 M
Installed size: 7.2 M
Downloading Packages:
(1/4): python3-pip-9.0.3-22.el8.noarch.rpm 230 kB/s | 20 kB 00:00
(2/4): python36-3.6.8-38.module+el8.5.0+12207 465 kB/s | 19 kB 00:00
(3/4): python3-setuptools-39.2.0-6.el8.noarch 1.1 MB/s | 163 kB 00:00
(4/4): platform-python-pip-9.0.3-22.el8.noarc 4.7 MB/s | 1.6 MB 00:00
------------------------------------------------------------------------------
Total 5.1 MB/s | 1.8 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : python3-setuptools-39.2.0-6.el8.noarch 1/4
Installing : platform-python-pip-9.0.3-22.el8.noarch 2/4
Installing : python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x8 3/4
Running scriptlet: python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x8 3/4
Installing : python3-pip-9.0.3-22.el8.noarch 4/4
Running scriptlet: python3-pip-9.0.3-22.el8.noarch 4/4
Verifying : platform-python-pip-9.0.3-22.el8.noarch 1/4
Verifying : python3-setuptools-39.2.0-6.el8.noarch 2/4
Verifying : python3-pip-9.0.3-22.el8.noarch 3/4
Verifying : python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x8 4/4
Installed products updated.
Installed:
platform-python-pip-9.0.3-22.el8.noarch
python3-pip-9.0.3-22.el8.noarch
python3-setuptools-39.2.0-6.el8.noarch
python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x86_64
Complete!
+ python3 -V
Python 3.6.8
+ pip3.6 install gssapi==1.8.0
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3.6 install --user` instead.
Collecting gssapi==1.8.0
Downloading https://files.pythonhosted.org/packages/ed/dd/562f246e551d0e5e6dc1ab0e05bafb9e5708eb8f0e99b46050dde804356e/gssapi-1.8.0.tar.gz (94kB)
100% |████████████████████████████████| 102kB 2.7MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-o7sn4x2g/gssapi/setup.py", line 18, in <module>
from Cython.Build import cythonize # noqa: E402
ModuleNotFoundError: No module named 'Cython'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-o7sn4x2g/gssapi/Component versions (python-gssapi, Kerberos, OS / distro, etc.)
RHEL 8.6, Python 3.6, pip 9
Ideas
As a workaround, the packaging configuration could make use of the deprecated setup_requires when setuptools is detected to be below v58.3.0.
Alternatively, this package could drop the support for older Pythons that are known to bundle older pip versions.
The latest CPython 3.7 bundles pip v22.0.4: https://github.com/python/cpython/tree/3.7/Lib/ensurepip/_bundled. Though, its first stable release v3.7.0 only has pip v10.0.1 (https://github.com/python/cpython/tree/v3.7.0/Lib/ensurepip/_bundled) while PEP 517 started being implemented around pip v18.1.
We've had the same decisions to make in aiohttp around 3-4 years ago which was being complicated by Cython being an optional dependency so we've opted for shipping C-files in sdist. Although, we're now facing other problems with that due to newer Cython being needed for older releases to function under Python 3.11 that you seem to have attempted to address. FWIW, our case is easier because we can just ship wheels for GNU/Linux as well and gssapi doesn't do that.