Allow overriding sys.platlibdir: add PYTHONPLATLIBDIR env var

[manisandro]

BPO	40854
Nosy	@vstinner, @manisandro
PRs	bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var #20605 [3.9] bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) #20725 bpo-40854: PYTHONPLATLIBDIR env var added to 3.9 #20735 bpo-40947: Replace PLATLIBDIR macro with config->platlibdir #20799 bpo-40947: getpath.c uses PyConfig.platlibdir #20807 [3.9] bpo-40947: getpath.c uses PyConfig.platlibdir (GH-20807) #20813

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-06-09.16:53:22.727>
created_at = <Date 2020-06-03.11:00:25.012>
labels = ['interpreter-core', '3.10']
title = 'Allow overriding sys.platlibdir: add PYTHONPLATLIBDIR env var'
updated_at = <Date 2020-06-11.16:27:04.621>
user = 'https://github.com/manisandro'

bugs.python.org fields:

activity = <Date 2020-06-11.16:27:04.621>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-06-09.16:53:22.727>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-06-03.11:00:25.012>
creator = 'smani'
dependencies = []
files = []
hgrepos = []
issue_num = 40854
keywords = ['patch']
message_count = 13.0
messages = ['370655', '370773', '370800', '370993', '370994', '370997', '371007', '371032', '371033', '371128', '371285', '371293', '371304']
nosy_count = 3.0
nosy_names = ['vstinner', 'python-dev', 'smani']
pr_nums = ['20605', '20725', '20735', '20799', '20807', '20813']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40854'
versions = ['Python 3.10']

[manisandro]

You can currently point the python interpreter to a different install say via

export PYTHONHOME=<Prefix>
export PYTHONPATH=<Prefix>/lib/python3.9
python3

With the newly added platlibdir [1], if python was configured with platlibdir=lib64, this will break because i.e. the site-packages dir as returned by sysconfig.get_paths() will use lib64 and not lib as the other install may be using.

This PR adds the possibility to override the platlibdir via environment variable.

Full rationale: [2].

[1] #8068
[2] https://src.fedoraproject.org/rpms/python3.9/pull-request/10

[vstinner]

Fedora and OpenSUSE Linux distributions were using a downstream patch which replaced /lib/ with /lib64/ for like 15 years.

Is this issue a new issue? Or did Fedora always had the issue for example?

Which Linux distribution are you using?

[manisandro]

I'm on Fedora. My use case is for the mingw-python package I maintain there, see [1] for the full details. I believe (though I haven't investigated) that the previous downstream lib64 patch behaved slightly differently, or that something else between python-3.8 and earlier and python-3.9 changed, in that I didn't have lib64 appear in the site packages path previously when invoking python as detailed in [1], whereas it now does.

[1] https://src.fedoraproject.org/rpms/python3.9/pull-request/10#comment-0

[vstinner]

New changeset 8f023a2 by Sandro Mani in branch 'master':
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605)
8f023a2

[vstinner]

I merged the patch in master, future Python 3.10. I would prefer to leave Python 3.9 unchanged, if possible. Can we close this issue?

[manisandro]

Many thanks! - I'll ask downstream to carry the patch for the 3.9 cycle, so I'm ok with closing.

[vstinner]

bpo-1294959 added sys.platlibdir and configure --with-platlibdir to use /usr/lib64 on Fedora and SuSE to Python 3.9.

The problem is that msg370655 use case seems to be a regression compared to Python 3.9. I now consider to backport the commit 8f023a2 to 3.9 since it looks like a fix part of bpo-1294959 rather than a bug fix.

I created PR 20725 to backport PyConfig.platlibdir and PYTHONPLATLIBDIR env var to Python 3.9.

[vstinner]

New changeset 8175064 by Victor Stinner in branch '3.9':
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) (GH-20725)
8175064

[vstinner]

New changeset 5edb832 by Victor Stinner in branch 'master':
bpo-40854: PYTHONPLATLIBDIR env var added to 3.9 (GH-20735)
5edb832

[vstinner]

Thanks Sandro Mani. The new env var should be part of the incoming Python 3.9.0 beta3.