unexpected system error with pep420 style namespace packages

[ronaldoussoren]

BPO	24192
Nosy	@brettcannon, @ronaldoussoren, @ericvsmith, @bitdancer, @ericsnowcurrently
Files	issue24192.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericsnowcurrently'
closed_at = <Date 2015-05-16.17:38:43.443>
created_at = <Date 2015-05-14.11:58:40.308>
labels = ['interpreter-core', 'type-bug', 'release-blocker']
title = 'unexpected system error with pep420 style namespace packages'
updated_at = <Date 2015-05-16.17:38:43.442>
user = 'https://github.com/ronaldoussoren'

bugs.python.org fields:

activity = <Date 2015-05-16.17:38:43.442>
actor = 'eric.snow'
assignee = 'eric.snow'
closed = True
closed_date = <Date 2015-05-16.17:38:43.443>
closer = 'eric.snow'
components = ['Interpreter Core']
creation = <Date 2015-05-14.11:58:40.308>
creator = 'ronaldoussoren'
dependencies = []
files = ['39386']
hgrepos = []
issue_num = 24192
keywords = ['patch']
message_count = 8.0
messages = ['243181', '243186', '243233', '243243', '243291', '243295', '243297', '243340']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'ronaldoussoren', 'eric.smith', 'r.david.murray', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24192'
versions = ['Python 3.5']

[ronaldoussoren]

The script below creates a basic PEP-420 style package with a single module in it ('package.sub') and tries to import that module

With 3.5 the script runs without problems and prints 42 (as expected).

With a 3.5 (fresh checkout as of 2015-05-14) I get an SystemError:

$ python3.5 demo.py
Traceback (most recent call last):
  File "demo.py", line 10, in <module>
    import package.mod
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load
  File "<frozen importlib._bootstrap>", line 947, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 575, in module_from_spec
  File "<frozen importlib._bootstrap>", line 519, in _init_module_attrs
SystemError: Parent module '' not loaded, cannot perform relative import

#######
import os

os.mkdir('path1')
os.mkdir('path1/package')
with open('path1/package/mod.py', 'w') as fp:
    fp.write('print(42)\n')

import site
site.addsitedir('path1')
import package.mod

[bitdancer]

I presume you meant that it works with 3.4?

[ericsnowcurrently]

The problem is right where the traceback says. Apparently there is a gap in the namespace package tests that I slipped through with my recent work to split out path-based import. I'll work up a patch.

[ericsnowcurrently]

Hmm, look like the test suite masks the issue due to the fact that importlib gets imported before running the applicable tests in test_namespace_pkgs.py. This causes _frozen_importlib.__package__ to get set properly, thus masking the problem.

The problem is the use of relative imports in importlib._bootstrap. The solution is to accomplish this in a different way.

[ericsnowcurrently]

Here's a fix. If I don't hear from anyone right away I'll push it in a few hours (or tomorrow morning).

[python-dev]

New changeset 46b2c99121f5 by Eric Snow in branch 'default':
Issue bpo-24192: Fix namespace package imports.
https://hg.python.org/cpython/rev/46b2c99121f5

[ronaldoussoren]

I can confirm that this fixes the problem I was having, both for the sample code I posted and for the real codebase I extracted this from.

Thanks.

[ericsnowcurrently]

Great! The buildbots are happy too. :)