Interpreter doesn't start when dynamic loading is disabled

[JeffreyArmstrong]

BPO	24769
Nosy	@brettcannon, @ncoghlan, @larryhastings, @rbtcollins, @termim, @encukou, @ericsnowcurrently
Files	issue24769.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/larryhastings'
closed_at = <Date 2015-08-25.02:55:23.405>
created_at = <Date 2015-08-01.03:17:42.505>
labels = ['build', 'release-blocker']
title = "Interpreter doesn't start when dynamic loading is disabled"
updated_at = <Date 2015-08-25.21:35:24.183>
user = 'https://bugs.python.org/JeffreyArmstrong'

bugs.python.org fields:

activity = <Date 2015-08-25.21:35:24.183>
actor = 'python-dev'
assignee = 'larry'
closed = True
closed_date = <Date 2015-08-25.02:55:23.405>
closer = 'larry'
components = ['Build']
creation = <Date 2015-08-01.03:17:42.505>
creator = 'Jeffrey.Armstrong'
dependencies = []
files = ['40156']
hgrepos = []
issue_num = 24769
keywords = ['patch']
message_count = 15.0
messages = ['247797', '248033', '248359', '248370', '248374', '248403', '248754', '248776', '249085', '249101', '249109', '249122', '249123', '249159', '249162']
nosy_count = 9.0
nosy_names = ['brett.cannon', 'ncoghlan', 'larry', 'rbcollins', 'termim', 'petr.viktorin', 'python-dev', 'eric.snow', 'Jeffrey.Armstrong']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue24769'
versions = ['Python 3.5', 'Python 3.6']

[JeffreyArmstrong]

When attempting to build Python without dynamic loading (HAVE_DYNAMIC_LOADING is not defined), the module "_imp" will not have the function "exec_dynamic." However, Lib/bootstrap.py seems to assume that "_imp.exec_dynamic" exists, causing the error:

----

./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
	echo "generate-posix-vars failed" ; \
	rm -f ./pybuilddir.txt ; \
	exit 1 ; \
fi
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1134, in _install
  File "<frozen importlib._bootstrap>", line 1114, in _setup
  File "<frozen importlib._bootstrap>", line 1082, in _builtin_from_name
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 748, in exec_module
AttributeError: module '_imp' has no attribute 'exec_dynamic'
Fatal Python error: Py_Initialize: importlib install failed

Current thread 0x00000000 (most recent call first):
ABNORMAL TERMINATION
generate-posix-vars failed
----

when trying to build. This error means that Python 3.5 will not be able to build in a purely static (no dynamic loading whatsoever) form.

This error was encountered in Python 3.5b4.

[ncoghlan]

This is an interesting find - thanks.

Adding Larry as 3.5 release manager to the cc here, as I think the right fix actually involves an API adjustment inside _imp.

One of the consequences of PEP-489 (multi-phase initialisation) was that builtin imports and extension module imports now share a lot more code, and that includes the fact that while we have both _imp.create_builtin and _imp.create_dynamic, they currently share a module level exec implementation in _imp.exec_dynamic.

However, the definition and implementation of _imp.exec_dynamic is still behind the same preprocessor guard as _imp.create_dynamic, hence the error reported here.

As far as a fix goes, rather than defining _imp.exec_dynamic unconditionally, my inclination is to say that merging the Python level APIs in addition to the underyling C level implementation represents a design mistake when we implemented the PEP, and there should be a separate _imp.exec_builtin function.

The fact that _imp.exec_dynamic just called _imp.exec_builtin internally would then be a pure CPython implementation detail, rather than something importlib specifically depended on.

[encukou]

Apologies for the delay; I was on vacation.

This was indeed a mistake in PEP-489 implementation. I agree with Nick on the solution.
Here is a patch that adds exec_builtin, with implementation shared with exec_dynamic.

[ericsnowcurrently]

Looks fine to me. Nick had suggested calling exec_builtin from exec_dynamic (to the same effect as your patch), but I don't consider that much of an issue. :)

[encukou]

Right. I think a common helper is cleaner than calling a clinic-generated wrapper.

[larryhastings]

Uh, Nick? You didn't add me to this bug.

[rbtcollins]

Patch looks good to me too. I think this needs to be put forward as a PR to bitbucket right? It looks Release Critical to me.

[brettcannon]

It's ultimately Larry's call, but I think it should go into 3.5.0.

[larryhastings]

Yes, I'll accept this into 3.5.0, please send a pull request.