site ignores ImportError when running sitecustomize and usercustomize

[vstinner]

BPO	26099
Nosy	@brettcannon, @vstinner
Files	site.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vstinner'
closed_at = <Date 2016-01-22.11:23:56.297>
created_at = <Date 2016-01-13.14:46:44.016>
labels = ['type-feature']
title = 'site ignores ImportError when running sitecustomize and usercustomize'
updated_at = <Date 2016-01-22.11:23:56.295>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2016-01-22.11:23:56.295>
actor = 'vstinner'
assignee = 'vstinner'
closed = True
closed_date = <Date 2016-01-22.11:23:56.297>
closer = 'vstinner'
components = []
creation = <Date 2016-01-13.14:46:44.016>
creator = 'vstinner'
dependencies = []
files = ['41603']
hgrepos = []
issue_num = 26099
keywords = ['patch']
message_count = 7.0
messages = ['258144', '258160', '258510', '258758', '258762', '258792', '258793']
nosy_count = 3.0
nosy_names = ['brett.cannon', 'vstinner', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26099'
versions = ['Python 3.6']

[vstinner]

If the code of sitecustomize raises an ImportError because the requested module doesn't exist, sitecustomize exception is silently ignored because site.py uses "try: import sitecustomize except ImportError: pass".

It's possible to log a warning since ImportError now has a name attribute since Python 3.3.

There is a similar issue on usercustomize.

Attached patch fixes the issue.

[brettcannon]

This is a change in semantics. It might be better to log an ImportWarning when the import fails and keep the current semantics (and be careful about importing warnings).

[vstinner]

Example of the bug:

----

$ cat Lib/sitecustomize.py
print("before")
import xxx
print("after")

haypo@smithers$ ./python -q -c pass
before
----

The line "after" is not executed and *no* error nor warning is raised. With the patch:

----

$ ./python -q -c pass
before
Error in sitecustomize; set PYTHONVERBOSE for traceback:
ImportError: No module named 'xxx'

$ PYTHONVERBOSE=1 ./python -q -c pass
(...)
# /home/haypo/prog/python/3.5/Lib/__pycache__/sitecustomize.cpython-35.pyc matches /home/haypo/prog/python/3.5/Lib/sitecustomize.py
# code object from '/home/haypo/prog/python/3.5/Lib/__pycache__/sitecustomize.cpython-35.pyc'
before
Traceback (most recent call last):
  File "/home/haypo/prog/python/3.5/Lib/site.py", line 508, in execsitecustomize
    import sitecustomize
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 662, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/haypo/prog/python/3.5/Lib/sitecustomize.py", line 2, in <module>
    import xxx
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'xxx'
# destroy sitecustomize
(...)

---

Brett Cannon: "This is a change in semantics."

Can you please elaborate? Even if importing sitecustomize changes, Python continue its execution. Only a warning is emited.

Brett Cannon: "It might be better to log an ImportWarning when the import fails and keep the current semantics (and be careful about importing warnings)."

With my patch, a message "Error in sitecustomize; set PYTHONVERBOSE for traceback:" is logged. I used the same behaviour then the code to handle site import.

Note: I found this issue when I worked on the PEP-511 to register a code transformer at startup. The bug is really annoying: the code transformers may or may not be registered depending if required modules can be important, I expect an error (or at least a warning).

[vstinner]

@brett: Ping? I'm going to push the fix to Python 3.6.

Tell me if you are ok to backport the change to Python 2.7 and 3.5 too.

[brettcannon]

Semantics are fine (I initially misread what you wanted to do; sorry).

And I wouldn't backport since it's an enhancement and not a bugfix.

[python-dev]

New changeset c873a479a6a3 by Victor Stinner in branch 'default':
site: error on sitecustomize import error
https://hg.python.org/cpython/rev/c873a479a6a3

[vstinner]

Brett: "Semantics are fine (I initially misread what you wanted to do; sorry)."

Ok, I pushed my change.

Brett: "And I wouldn't backport since it's an enhancement and not a bugfix."

Ok, fine.