importlib: better error message when import fail during Python shutdown

[vstinner]

BPO	26637
Nosy	@brettcannon, @vstinner, @ericsnowcurrently
Files	importlib_shutdown.patch importlib_shutdown-2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-03-24.23:42:05.577>
created_at = <Date 2016-03-24.15:06:33.209>
labels = ['library']
title = 'importlib: better error message when import fail during Python shutdown'
updated_at = <Date 2016-03-25.07:57:50.818>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2016-03-25.07:57:50.818>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2016-03-24.23:42:05.577>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2016-03-24.15:06:33.209>
creator = 'vstinner'
dependencies = []
files = ['42275', '42277']
hgrepos = []
issue_num = 26637
keywords = ['patch']
message_count = 9.0
messages = ['262352', '262367', '262368', '262375', '262376', '262378', '262379', '262387', '262405']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'vstinner', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue26637'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

[vstinner]

Example of script.py:
-------------------------

class Bla:
    def __del__(self):
        try:
            import xxxx
        except Exception as exc:
            print("import error: [%s] %r" % (type(exc), exc))

bla = Bla()

---

Running this example logs a strange error:
-------------------------

$ python3.5 script.py
import error: [<class 'TypeError'>] TypeError("'NoneType' object is not iterable",)

---

The error comes from importlib._bootstrap._find_spec() which tries to iterator on sys.meta_path, whereas PyImport_Cleanup() was called and this function setted sys.meta_path to None.

Attached patch enhances _find_spec() to handle this case to return None. Error with the patch:
-------------------------

$ python3.5 script.py
import error: [<class 'ImportError'>] ImportError('sys.meta_path is None, Python is likely shutting down',)

---

[vstinner]

Updated patch with a dot in the comment :-)

Is ImportError the best exception in this case?

[brettcannon]

Either ImportError or TypeError. ImportError makes sense since this is directly related to import semantics while TypeError makes sense as we are saying None specifically is not allowed.

[vstinner]

I will reply to my own question: in fact, ImportError is more convenient. The "try/except ImportError" pattern is common, in some cases, it's acceptable to skip a whole module. For a direct example on import during Python shutdown, see the issue bpo-21925: "ResourceWarning sometimes doesn't display" which adds "try/except ImportError" to be able to log warnings.

@brett, Eric: So, do you like importlib_shutdown-2.patch?

[vstinner]

Oh. And is it ok to apply this change to Python 2.7 and 3.5?

[brettcannon]

It's technically a change in semantics as it shifts what exception is raised, so I wouldn't backport it.

[vstinner]

It's technically a change in semantics as it shifts what exception is raised, so I wouldn't backport it.

Ok, I'm fine with only changing Python 3.6.

[python-dev]

New changeset 14c56f697a85 by Victor Stinner in branch 'default':
Fix bug in __import__ during Python shutdown
https://hg.python.org/cpython/rev/14c56f697a85

[python-dev]

New changeset e2c482b60776 by Victor Stinner in branch 'default':
Issue bpo-26637: Fix test_io
https://hg.python.org/cpython/rev/e2c482b60776