Import dotted name as alias breaks with concurrency

[rshk]

BPO	30814
Nosy	@brettcannon, @ncoghlan, @vstinner, @ned-deily, @ericsnowcurrently, @serhiy-storchaka, @RazerM, @rshk
PRs	bpo-30814: Fixed a race condition when import a submodule from a package. #2580 [3.6] bpo-30814: Fixed a race condition when import a submodule from a package. (GH-2580). #2598 [3.5] bpo-30814: Add a test for concurrent relative import. (GH-2580). #2645 bpo-30891: importlib _find_and_load() try/except #2665 bpo-30814, bpo-30876: Add new import test files to projects. #2851 [3.6] bpo-30814, bpo-30876: Add new import test files to projects. (GH-2851). #2912

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-07-27.10:19:01.622>
created_at = <Date 2017-06-30.11:09:48.262>
labels = ['interpreter-core', 'type-bug', '3.7']
title = 'Import dotted name as alias breaks with concurrency'
updated_at = <Date 2017-07-27.10:19:01.621>
user = 'https://github.com/rshk'

bugs.python.org fields:

activity = <Date 2017-07-27.10:19:01.621>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-07-27.10:19:01.622>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2017-06-30.11:09:48.262>
creator = 'SamueleSanti'
dependencies = []
files = []
hgrepos = []
issue_num = 30814
keywords = ['3.6regression']
message_count = 18.0
messages = ['297380', '297475', '297482', '297679', '297698', '297704', '297803', '297806', '297966', '298050', '298065', '298951', '298958', '299005', '299018', '299019', '299296', '299302']
nosy_count = 8.0
nosy_names = ['brett.cannon', 'ncoghlan', 'vstinner', 'ned.deily', 'eric.snow', 'serhiy.storchaka', 'RazerM', 'SamueleSanti']
pr_nums = ['2580', '2598', '2645', '2665', '2851', '2912']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30814'
versions = ['Python 3.6', 'Python 3.7']

[rshk]

I noticed this weird behavior in Python 3.6:

Apparently import package.module as _alias fails in a threaded environment, sometimes raising an ImportError, seemingly at random.

Everything seems to be working perfectly fine:

• On 3.5 and lower
• When not defining an alias, eg. import package.module
• When using the from package import module as alias style

See this repo with code for reproducing the issue:

https://github.com/rshk/python36-import-alias-concurrency-bug

I tested this on the system Python 3.6.1 shipped with Archlinux (both inside and outside of a virtualenv), and in the official docker image for 3.6.

This all started here psycopg/psycopg2#550 as others were experiencing this issue too.

[serhiy-storchaka]

See also bpo-23203.

This was fixed in master in bpo-30024. Maybe it is worth to backport that change to maintained versions?

[serhiy-storchaka]

On other side, bpo-23203 fixed just a symptom. The issue is reproduced in master when replace "import foobar.submodule as foo" with "import foobar.submodule; foo = foobar.submodule".

The regression may be related to bpo-22557. Either the optimization introduced a bug, or it exposed an existing bug by changing timings.

[brettcannon]

My guess is the import of package has completed but package.submodule hasn't (I don't know what state it would be in, but obviously before the module is assigned as an attribute on the package), a context switch occurs, and then the assignment fails due to package.submodule not being set on package yet.

I wonder if package.submodule is in sys.modules when the AttributeError is triggered? I don't remember when the assignment of a submodule to an attriute on a package occurs, but this suggests it's rather late and that's why this is failing. Maybe the assignment is outside of the per-module lock being released and that is what's causing this?

Hey, at least importing in a thread doesn't deadlock anymore like it used to. ;)

[ncoghlan]

Documenting explicitly what I believe the expected order of module lock acquisition would be in this case:

Thread A acquires package.subpackage lock
Thread B blocks on package.subpackage lock
Thread A acquires package lock
Thread A releases package lock
Thread A releases package.subpackage lock
Thread B acquires & immediately releases package.subpackage lock

Looking at https://hg.python.org/cpython/rev/64f195790a3a#l4.367 (the commit for bpo-22557), there's a potentially suspect change in the scope of a "_PyImport_AcquireLock/_PyImport_ReleaseLock" pair inside PyImport_ImportModuleLevelObject

Specifically, I think we may now have a race condition at https://github.com/python/cpython/blob/master/Python/import.c#L1534, where two threads can *both* end up trying to initialize the same module, since we're no longer holding the global import lock around that "mod = PyDict_GetItem(interp->modules, abs_name);" call and the associated state updates where the first thread indicates that it is already initializing that module so the second thread should just wait for it to finish doing so.

[serhiy-storchaka]

I have came to the same conclusion.

PR 2580 adds a double check in _bootstrap._find_and_load() (and also moves the locking into it). It also simplifies the C code by removing the fast path for the case sys.module[name] is None.

[serhiy-storchaka]

New changeset b4baace by Serhiy Storchaka in branch 'master':
bpo-30814: Fixed a race condition when import a submodule from a package. (bpo-2580)
b4baace

[serhiy-storchaka]

New changeset 03b0e83 by Serhiy Storchaka in branch '3.6':
[3.6] bpo-30814: Fixed a race condition when import a submodule from a package. (GH-2580). (bpo-2598)
03b0e83

[brettcannon]

I just wanted to say thanks to everyone who helped to fix this bug. The locking situation in import is probably the trickiest part of it and has also been tweaked the most as of late and so solving these kinds of issues is tricky.

[vstinner]

test_concurrency() of test_import fails randomly on Windows: see bpo-30891.