Re-use already existing functionality to allow Python 2.7.x (both embedded and standalone) to locate the module path according to the shared library

[AliRizvi-Santiago]

BPO	35173
Nosy	@vstinner
Files	relocate-with-dllpath.patch: Patch to PC/getpathp.c to add usage of dllpath static variable

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-09-26.00:59:57.004>
created_at = <Date 2018-11-06.00:18:27.273>
labels = ['interpreter-core', 'type-feature', '3.8', '3.9']
title = 'Re-use already existing functionality to allow Python 2.7.x (both embedded and standalone) to locate the module path according to the shared library'
updated_at = <Date 2019-10-04.18:52:14.739>
user = 'https://bugs.python.org/AliRizvi-Santiago'

bugs.python.org fields:

activity = <Date 2019-10-04.18:52:14.739>
actor = 'Ali Rizvi-Santiago'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-26.00:59:57.004>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-11-06.00:18:27.273>
creator = 'Ali Rizvi-Santiago'
dependencies = []
files = ['47909']
hgrepos = []
issue_num = 35173
keywords = ['patch']
message_count = 3.0
messages = ['329322', '353249', '353970']
nosy_count = 2.0
nosy_names = ['vstinner', 'Ali Rizvi-Santiago']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue35173'
versions = ['Python 3.8', 'Python 3.9']

[AliRizvi-Santiago]

This is specific to the Windows platform as it's the only platform that uses the registry and other path hacks to identify the default module path. This patch is mostly intended for embedded Python, but is also applicable to a stand-alone Python.

A few years ago, I was looking at relocating my Python interpreter so that an embedded application (that I didn't have control over) would use the correct module path. While merging my code into CPython, I quickly noticed that Python already supported doing this with Py_ENABLE_SHARED but due to the implementation wasn't actually using it for some reason. The code that implements this is 10+ years old, so perhaps it was just an oversight or some other reason that I didn't know about.

Inside PC/getpathp.c there's a static variable, "dllpath", that is initialized with the path to the DLL that is being dynamically loaded when Py_ENABLE_SHARED is specified. Normally arg0 is used to locate the module path, but when embedding Python the .exe and thus arg0 is not involved. So, this patch uses "dllpath" by adding a final case to the calculation of the path by assigning it to "pythonhome" if the home was not able to be determined by any other means. This is done in 2 places within "calculatepath()".

This allows one to have multiple versions of Python (32-bit, 64-bit, older versions, etc) on the same Windows system and so a user should not need to copy the Python library into their System path or explicitly set any environment variables (unless truly desired of course). This should greatly simplify relocation of Python as the DLL and executable can be moved around without being dependant on any external invariants.

[vstinner]

Modules/getpath.c and PC/getpathp.c have been deeply reworked with the implementation of the PEP-587 in Python 3.8. Parameters which are already set (ex: Py_SetProgramName() or using the new PyConfig API) are no longer overriden. I added some tests on the "path configuration".

The most reliable way to configure the path configuration is now to use the new initialization API using PyConfig.
https://docs.python.org/dev/c-api/init_config.html

[AliRizvi-Santiago]

Sure. It was for Python2 anyways.

-Ali

On Wed, Sep 25, 2019 at 8:00 PM STINNER Victor <report@bugs.python.org>
wrote:

STINNER Victor <vstinner@python.org> added the comment:

Modules/getpath.c and PC/getpathp.c have been deeply reworked with the
implementation of the PEP-587 in Python 3.8. Parameters which are already
set (ex: Py_SetProgramName() or using the new PyConfig API) are no longer
overriden. I added some tests on the "path configuration".

The most reliable way to configure the path configuration is now to use
the new initialization API using PyConfig.
https://docs.python.org/dev/c-api/init_config.html

----------
nosy: +vstinner
resolution: -> fixed
stage: -> resolved
status: open -> closed
versions: +Python 3.8, Python 3.9 -Python 2.7

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue35173\>

---