Stop including setuptools and wheel in Python 3.12+ images

[edmorley]

In Python 3.12, the stdlib's ensurepip and venv modules were updated to no longer install setuptools alongside pip. This was viable since as of pip v22.1, for pre-PEP-517/518 packages pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend, with setuptools and wheel automatically installed), if the setuptools package isn't installed globally.

See:
python/cpython#95299
python/cpython@ece20db

As such, when this repo added support for Python 3.12, the explicit setuptools requirement was removed, as part of:
#833

However, that change didn't actually remove setuptools from the Python 3.12+ images, since get-pip implicitly installs setuptools and wheel if they are not already installed. Instead, the result was only that the setuptools version is now unconstrained and get-pip will pull in whatever is the latest version of setuptools at the time of the image being built.

eg:

$ docker run -q --rm python:3.12 pip list
Package    Version
---------- -------
pip        24.2
setuptools 72.2.0
wheel      0.44.0

This means that the Python 3.12 (and 3.13-rc) images here:
(a) are less aligned with the experience one gets when using ensurepip or venv from the stdlib (which no longer install setuptools),
(b) now have a floating setuptools version when they didn't before (which doesn't seem ideal from a determinism point of view, particularly given intentional breaking changes in recent setuptools major releases as part of an ongoing tech-debt clean-up effort).

I think it would be best if the packages in these images were consistent with the environment created by ensurepip / venv. Which for Python 3.12+ would mean not shipping with setuptools or wheel.

There are a few ways this could be done:

1. For Python 3.12+, pass --no-setuptools and --no-wheel when invoking get-pip.py.
2. By switching from get-pip to ensurepip (Switch from get-pip to ensurepip #951), which would (a) mean these images automatically match the behaviour of the stdlib, (b) mean a lot of the version pinning/handling code can be simplified (since ensurepip defaults to installing the bundled version, not the latest version from PyPI).
3. By waiting for when Stop installing setuptools and wheel on Python 3.12+ pypa/get-pip#218 is merged. Though at that point, there is even less reason for this repo to keep using get-pip instead of ensurepip, so we'd probably want to do (2) anyway.

See also:

• Stop installing setuptools and wheel by default on Python 3.12+ environments pypa/virtualenv#2487
• Stop installing setuptools and wheel on Python 3.12+ pypa/get-pip#218

[tianon]

Thank you for the very detailed analysis/breakdown ❣️

I agree we should fix this, and I really like the fix being removing more things we shouldn't be including by default (because they're not part of the official upstream distribution) -- ensurepip does seem like a good solution, but perhaps one we ought to be slightly more careful with, so maybe we do a little bit of both 1 and 2 (ie, do 1 right away for 3.12+ since it's going to be really trivial to add to the template and pretty low risk), and work on 2 in the meantime separately?

[yosifkit]

Opened #954 to get the quick fix for this issue. We can also continue to work toward the better fix of switching from get-pip to ensurepip (#951).