imp.py: load_package() appends to its own loop variable

[commodo]

BPO	30645
Nosy	@brettcannon, @ncoghlan, @bitdancer, @ericsnowcurrently, @SylvainDe, @commodo
PRs	bpo-30645: use an internal variable while iterating in imp.load_package() #2268 [3.6] bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) #2364 [3.5] bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) #2365

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-06-23.18:40:28.636>
created_at = <Date 2017-06-13.08:25:25.507>
labels = ['3.7', 'type-bug', 'library']
title = 'imp.py: load_package() appends to its own loop variable'
updated_at = <Date 2017-06-23.18:40:28.635>
user = 'https://github.com/commodo'

bugs.python.org fields:

activity = <Date 2017-06-23.18:40:28.635>
actor = 'brett.cannon'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-23.18:40:28.636>
closer = 'brett.cannon'
components = ['Library (Lib)']
creation = <Date 2017-06-13.08:25:25.507>
creator = 'Alexandru Ardelean'
dependencies = []
files = []
hgrepos = []
issue_num = 30645
keywords = []
message_count = 12.0
messages = ['295865', '295964', '295991', '295992', '295993', '296012', '296038', '296278', '296279', '296730', '296733', '296734']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'ncoghlan', 'r.david.murray', 'eric.snow', 'SylvainDe', 'Alexandru Ardelean']
pr_nums = ['2268', '2364', '2365']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30645'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

[commodo]

On OpenWrt, the Python & Python3 are shipped with bytecodes [also, bytecode compiling is disabled by default].
2 reasons for that: 1) that would fill up flash/RAM 2) bytecodes can be up to 10x faster than Python source code

I got a report of an issue with virtualenv on Python3, that had a weird path: the virtualenv /test would generate /test/lib/virtualenv/__init__.py/__init__.pyc

The fix is here in the patch file:
openwrt/packages@576c45e

[brettcannon]

Three things.

One, the imp module is deprecated, so please don't use it.

Two, all undocumented functions in the imp module are considered unsupported, so double-please don't use imp.load_package(). :)

Three, we can't look at that patch without the person supplying it signing Python's CLA as we risk running afoul of things legally.

[commodo]

Damn.

You guys are more corporate than... <scratching on forehead>... anyway.

Please tell me, how to send you my signed CLA.
Not sure if posting the link or PDF is fine here, but I'd be fine with that too.

To resolve your first 2 notes: I am not using the imp module, nor the imp.load_package() function.
I did see it was marked DEPRECATED.

But, virtualenv uses it in it's latest release [15.1.0].
This is from trunk:
https://github.com/pypa/virtualenv/blob/master/virtualenv_embedded/distutils-init.py#L17

But that's not the actual code that's used.
Seems virtualenv, has a twisted way of keeping Python code base64 encoded & zipped within the virtualenv.py script.
https://github.com/pypa/virtualenv/blob/master/virtualenv.py#L2117

So, maybe we can tell them not to use it, but let's see.
In my case, I just have a bug report [I was notified via email directly], that this is an issue with Python3 + virtualenv on LEDE & OpenWrt.
Python + virtualenv is not affected.

While I'm here, maybe I could use this opportunity to start sending a few more patches from the OpenWrt/LEDE package build.
But let's see how we resolve the legal matter first.

[SylvainDe]

Alexandru: You'll find the PSF Contributor Agreement at https://www.python.org/psf/contrib/contrib-form/ .

[commodo]

I already completed those steps.
Question is: what do I do with the output from that eSign ?
Or, how do I use it ?

It only mentions "Finally, click the "eSign" button to have the form sent to the PSF Secretary, Ewa Jodlowska."

It's unclear from there-on if my patch is acceptable, or whether I can submit it [or anything], wait for a reply ; basically, how to continue once the form is signed.

[bitdancer]

Yes, that's all you needed to do. She updated your status in the tracker (you now have the CLA signed '*' next to your name).

[brettcannon]

Yes, we're as corporate as an open source project that doesn't have enough funding to be able to defend against a lawsuit but is big enough that simply shutting down would really suck for millions of people. ;) Anyway, your CLA signing came through so now we just need a pull request on GitHub or a patch.

As for virtualenv still using a deprecated, hidden API, that's bad on them and should probably be reported to them directly.

[commodo]

FWIW, I took a look over the virtualenv repo.
There's an issue raised there
pypa/virtualenv#955

And a PR to address it
pypa/virtualenv#947

Though, it seems like they still want to keep imp.load_package() for a while, in order to keep compatibility with Python 2.6.

[commodo]

Admittedly, that PR refers to imp.load_module()
But, imp.load_modules() also calls imp.load_package()

[brettcannon]

New changeset c38e32a by Brett Cannon (Alexandru Ardelean) in branch 'master':
bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268)
c38e32a

[brettcannon]

New changeset 9db3ae0 by Brett Cannon in branch '3.6':
[3.6] bpo-30645: don't append to an inner loop path in imp.load_package() (GH-2268) (bpo-2364)
9db3ae0