Use of MutableMapping in os module slows down interpreter startup

[ericsnowcurrently]

BPO	19218
Nosy	@warsaw, @rhettinger, @pitrou, @vstinner, @tiran, @bitdancer, @ericsnowcurrently
Files	os-lazy-collections.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2013-10-13.00:53:34.686>
created_at = <Date 2013-10-10.18:08:01.373>
labels = ['type-feature', 'library']
title = 'Use of MutableMapping in os module slows down interpreter startup'
updated_at = <Date 2014-02-26.18:26:34.415>
user = 'https://github.com/ericsnowcurrently'

bugs.python.org fields:

activity = <Date 2014-02-26.18:26:34.415>
actor = 'r.david.murray'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2013-10-13.00:53:34.686>
closer = 'christian.heimes'
components = ['Library (Lib)']
creation = <Date 2013-10-10.18:08:01.373>
creator = 'eric.snow'
dependencies = []
files = ['32036']
hgrepos = []
issue_num = 19218
keywords = ['patch']
message_count = 14.0
messages = ['199402', '199404', '199405', '199406', '199449', '199450', '199520', '199521', '199642', '199644', '199647', '199654', '212282', '212285']
nosy_count = 9.0
nosy_names = ['barry', 'rhettinger', 'pitrou', 'vstinner', 'christian.heimes', 'Arfrever', 'r.david.murray', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19218'
versions = ['Python 3.4']

[ericsnowcurrently]

There has been some discussion on python-dev about improving interpreter startup time. Christian Heimes brought up how the os module imports collections (so _Environ can inherit from it) [1], and mentioned a couple of solutions [2].

[1]https://mail.python.org/pipermail/python-dev/2013-October/129312.html
[2]https://mail.python.org/pipermail/python-dev/2013-October/129367.html

[ericsnowcurrently]

Here is a patch for an alternate approach. It does not require re-implementing MutableMapping (could get out of sync) and does not require rearranging collections.abc. Instead it uses a metaclass to resolve the import issue lazily.

[tiran]

Nice trick :)

[ericsnowcurrently]

And when the lazy base class gets resolved, the metaclass gets to say "You didn't see anything and I was never here." :)

[pitrou]

-1 on such hacks. I much prefer the _abcoll approach.

[vstinner]

The collections module is loaded by the io module. You removed "from collections.abc import MutableMapping" from os.py. To be useful, you have also to rewrite the whole io module to remove all references to the collections.abc module, right?

[tiran]

The io module no longer imports collections.abc through the locale module.

Eric, I'm with Antoine. Your patch is too much of a clever hack and uses tricks that are dark magic. _abcoll is much simpler and easier to understand. Plus it can be used from other modules, too.

[ericsnowcurrently]

Not a problem. It is most definitely a hack. :) I put in up as an alternative to rearranging the collections module, but agree that doing so is preferable. I image that Raymond will make a decision on that one.

[rhettinger]

Christian, go ahead an rearrange the collections module.

Move Lib/collections/abc.py to Lib/_collections_abc.py.

Also, be sure to update source link on line 18 of Doc/library.collections.abc.rst

[python-dev]

New changeset 0b6052f2a8ee by Christian Heimes in branch 'default':
Issue bpo-19218: Rename collections.abc to _collections_abc in order to speed up interpreter start
http://hg.python.org/cpython/rev/0b6052f2a8ee

New changeset 7ea831581af4 by Christian Heimes in branch 'default':
Issue bpo-19218: Add facade collections.abc
http://hg.python.org/cpython/rev/7ea831581af4

New changeset 62b6ecd1e463 by Christian Heimes in branch 'default':
Issue bpo-19218: set __name__ of _collections_abc to collections.abc in order to fix tests and keep beautiful qualified names.
http://hg.python.org/cpython/rev/62b6ecd1e463