PathFinder is twice on sys.meta_path

[htgoebel]

BPO	33128
Nosy	@brettcannon, @ncoghlan, @ned-deily, @ericsnowcurrently, @pablogsal, @miss-islington
PRs	bpo-33128 Fix duplicated call to importlib._install_external_importers  #6273 [3.7] bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273) #6592

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-04-25.02:58:44.339>
created_at = <Date 2018-03-23.23:30:41.727>
labels = ['interpreter-core', '3.8', 'type-bug', '3.7']
title = 'PathFinder is twice on sys.meta_path'
updated_at = <Date 2018-04-25.02:58:44.338>
user = 'https://bugs.python.org/htgoebel'

bugs.python.org fields:

activity = <Date 2018-04-25.02:58:44.338>
actor = 'ncoghlan'
assignee = 'none'
closed = True
closed_date = <Date 2018-04-25.02:58:44.339>
closer = 'ncoghlan'
components = ['Interpreter Core']
creation = <Date 2018-03-23.23:30:41.727>
creator = 'htgoebel'
dependencies = []
files = []
hgrepos = []
issue_num = 33128
keywords = ['patch', '3.7regression']
message_count = 8.0
messages = ['314340', '314348', '314556', '314557', '314628', '315717', '315718', '315719']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'htgoebel', 'ncoghlan', 'ned.deily', 'eric.snow', 'pablogsal', 'miss-islington']
pr_nums = ['6273', '6592']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33128'
versions = ['Python 3.7', 'Python 3.8']

[htgoebel]

As of Python 3.7.0b2 _frozen_importlib_external.PathFinder exists twice on sys.meta_path, and it is the same object:

$ python -S
Python 3.7.0b2 (default, Mar 22 2018, 20:09:00) 
[GCC 5.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.meta_path)
[<class '_frozen_importlib.BuiltinImporter'>,
<class '_frozen_importlib.FrozenImporter'>,
<class '_frozen_importlib_external.PathFinder'>,
<class '_frozen_importlib_external.PathFinder'>]
>>> print([id(p) for p in sys.meta_path])
[24427944, 24430216, 24517416, 24517416]
>>>

[ned-deily]

Thanks for the report. git bisect result:

1abcf67 is the first bad commit
commit 1abcf67
Author: Eric Snow <ericsnowcurrently@gmail.com>
Date: Tue May 23 21:46:51 2017 -0700

bpo-22257: Private C-API for core runtime initialization (PEP-432). (bpo-1772)

(patch by Nick Coghlan)

[pablogsal]

It seems that the problem is that _Py_InitializeEx_Private calls _Py_InitializeCore and _Py_InitializeMainInterpreter. The first one calls at the end initimport that in turns calls importlib._install_external_importers that sets the PathFinder. Then, _Py_InitializeMainInterpreter calls initexternalimport that in turns calls again importlib._install_external_importers and therefore we end with two PathFinders in sys.meta_path.

I think the solution is removing the call to initexternalimport in _Py_InitializeMainInterpreter as it has been alreade initialized by _Py_InitializeCore.

[pablogsal]

The same problem happens in new_interpreter as far as I understand.

[ncoghlan]

Calling initexternalimports InitalizeMainInterpreter and from new_interpreter is right, since we only want these importers on the metapath after we know sys.path has been configured appropriately.

It's the call from InitializeCore that's questionable, since we haven't finished setting up sys.path at that point.

[ncoghlan]

New changeset 0977091 by Nick Coghlan (Pablo Galindo) in branch 'master':
bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273)
0977091

[miss-islington]

New changeset 52a5a17 by Miss Islington (bot) in branch '3.7':
bpo-33128 Fix duplicated call to importlib._install_external_importers (GH-6273)
52a5a17

[ncoghlan]

Hartmut, thanks for the issue report, and Pablo, thanks for the PR to resolve it!