Surprising name binding behavior of submodule imports needs documenting

[warsaw]

BPO	24029
Nosy	@warsaw, @brettcannon, @ericsnowcurrently
Files	issue24029-1.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2015-04-22.22:40:30.114>
created_at = <Date 2015-04-22.17:14:50.207>
labels = ['type-bug', 'docs']
title = 'Surprising name binding behavior of submodule imports needs documenting'
updated_at = <Date 2015-04-22.22:40:30.113>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2015-04-22.22:40:30.113>
actor = 'barry'
assignee = 'barry'
closed = True
closed_date = <Date 2015-04-22.22:40:30.114>
closer = 'barry'
components = ['Documentation']
creation = <Date 2015-04-22.17:14:50.207>
creator = 'barry'
dependencies = []
files = ['39175']
hgrepos = []
issue_num = 24029
keywords = []
message_count = 8.0
messages = ['241816', '241817', '241818', '241822', '241829', '241830', '241831', '241832']
nosy_count = 4.0
nosy_names = ['barry', 'brett.cannon', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24029'
versions = ['Python 3.4', 'Python 3.5']

[warsaw]

As described here: http://news.gmane.org/find-root.php?message_id=20150422115959.1ff2ee58%40limelight.wooz.org

Importing a submodule binds the submodule's name in the parent module's namespace. This is surprising, but it seems intentional and it's relied upon by existing code, e.g. asyncio/init.py in the stdlib.

It's also not documented afaict. It should be documented in the Language Reference's section on the import system. After a little more discussion on import-sig, I plan on doing that.

[warsaw]

From Guido:

It's definitely intentional, and it's fundamental to the package import
design. We've had many implementations of package import (remember "ni.py"?
last seen as "knee.py") and it was always there, because this is done as
part of submodule loading. For better or for worse (and because I didn't
know Java at the time :-) Python declares that if you write import foo.bar then later in your code you can use foo.bar to reference to the
bar submodule of package foo. And the way this is done is to make each
submodule an attribute of its parent package. This is done when the
submodule is first loaded, and because of the strict separation between
loading and importing, it is done no matter what form of import was used to
load bar.

I guess another thing to realize is that the globals of __init__.py are
also the attribute namespace of the package.

I'm not surprised it's in the reference manual -- that hasn't been updated
thoroughly in ages, and I sometimes cry when I see it. :-) So please do
clarify this for the benefit of future implementers.

[warsaw]

More rationale from the thread:

The surprising part is that it also happens for explicit relative
imports. I'm guessing that part was unintentional and simply not
noticed when PEP-328 was implemented.

No, that must also have been intentional, because even when you use
relative import, the module you imported knows its full name, and that full
name is used as its key in sys.modules. If someone else uses absolute
import for the same module they should still get the same module object.

[ericsnowcurrently]

Guido describes the global invariant for *all* the forms of importing a submodule, including explicit relative imports:

I just mean that for relative import
there is no need to bind the submodule to the parent, is there?

But there *is* a reason. The submodule must still be an attribute of the parent package, because of the invariant that if you have sys.modules['foo'] and sys.modules['foo.bar'], the latter must appear as the 'bar' attribute of the former. This is an invariant of module loading, and (I feel I'm repeating myself) the form of import used does not affect loading.

[warsaw]

Here's some new text for the Language Reference.

[ericsnowcurrently]

LGTM. You've covered all the key points and the example is good.

[warsaw]

Cool, thanks! I'll commit it and we can always clean it up/add to it later if needed.

[python-dev]

New changeset 3968e7a9614c by Barry Warsaw in branch '3.4':
Issue bpo-24029: Document the name binding behavior for submodule imports.
https://hg.python.org/cpython/rev/3968e7a9614c

New changeset 351ad8c4f3a6 by Barry Warsaw in branch '3.4':
Issue bpo-24029: Document the name binding behavior for submodule imports.
https://hg.python.org/cpython/rev/351ad8c4f3a6

New changeset 6295f207dfaa by Barry Warsaw in branch 'default':
Issue bpo-24029: Document the name binding behavior for submodule imports.
https://hg.python.org/cpython/rev/6295f207dfaa