quote bad module name in ImportError-like messages

[terryjreedy]

BPO	8754
Nosy	@brettcannon, @birkenfeld, @terryjreedy, @ncoghlan, @vstinner, @ezio-melotti, @merwok, @ericsnowcurrently
Files	import-repr.diff import-repr-lib.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2013-02-01.22:05:16.967>
created_at = <Date 2010-05-18.17:03:18.189>
labels = ['easy', 'type-bug', 'library']
title = 'quote bad module name in ImportError-like messages'
updated_at = <Date 2013-02-01.22:56:17.712>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2013-02-01.22:56:17.712>
actor = 'eric.araujo'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2013-02-01.22:05:16.967>
closer = 'brett.cannon'
components = ['Library (Lib)']
creation = <Date 2010-05-18.17:03:18.189>
creator = 'terry.reedy'
dependencies = []
files = ['19971', '23088']
hgrepos = []
issue_num = 8754
keywords = ['patch', 'easy', 'needs review']
message_count = 31.0
messages = ['105988', '105994', '106026', '106036', '121415', '121419', '121426', '121427', '121430', '121432', '121853', '123579', '123583', '123629', '124140', '124212', '124336', '124470', '124496', '126325', '126757', '130541', '131517', '131936', '131941', '143409', '152997', '154750', '155109', '181119', '181129']
nosy_count = 9.0
nosy_names = ['brett.cannon', 'georg.brandl', 'terry.reedy', 'ncoghlan', 'vstinner', 'ezio.melotti', 'eric.araujo', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8754'
versions = ['Python 3.3']

[terryjreedy]

ImportError messages should quote the name it cannot import since the actual problem may be whitespace in the name that is currently invisible in the message. In other words, display

ImportError: no module named 'bad name\r'

instead of

ImportError: no module named bad name

This defect lead to the current python-list thread
pickle unable to load collection

Peter Otten figured out that it was unable to load 'collections\r' rather than 'collections', which he demonstrated with

>>> >>> try: pickle.loads(garbled_data)
... except ImportError as e:
...     e
...
ImportError('No module named collections\r',)

The OP used 2.6, I tested 3.1, hence presume, after searching tracker issues, that this applies to 2.7 and 3.2 as well.

I marked this as a bug since the current message is wrong and misleading. I suspect the same may be true of a few other error messages, but I cannot think of any at the moment.

[brettcannon]

I guess it's a question of readability. Does::

ImportError: No module named mod

read better than::

ImportError: No module named 'mod'

In my eyes it doesn't by much, so switching to using repr() seems reasonable.

This can't be changed in released versions of Python as that could break someone's doctests. So making this only for 2.7 and 3.2.

[terryjreedy]

A refinement would be to only quote when there is whitespace in the name, but I do not know how well that works with unicode versus ascii.

[brettcannon]

It wouldn't matter (at least in Python 3) as str is unicode-aware. It's more about whether it's worth special-casing the output. I say no and just go with using the repr.

[merwok]

A question about process: Should every import bug be reported against core and library, so that importlib gets patched too?

[brettcannon]

I think everyone knows that importlib is there and might need to be patched. Plus I run importlib against the entire test suite already on occasion so changes which has a proper test will eventually get caught.

So just file it against core.

[merwok]

Thanks for the reply.

Funny thing, grep revealed a small inconsistency in this error message:

Python/pythonrun.c:415: * XXX Exception exceptions.ImportError: 'No module named sha'
Python/import.c:1821: "No module named %.200s", name);
Python/import.c:2722: "No module named %.200s", name);
Lib/test/test_multiprocessing.py:27:# message: "No module named _multiprocessing". _multiprocessing is not compiled
Lib/test/test_concurrent_futures.py:8:# message: "No module named _multiprocessing". _multiprocessing is not compiled
Lib/test/test_pydoc.py:194:badimport_pattern = "problem in %s - ImportError: No module named %s"
Lib/runpy.py:104: raise ImportError("No module named %s" % mod_name)
Lib/xml/etree/ElementTree.py:1492: "No module named expat; use SimpleXMLTreeBuilder instead"
Lib/modulefinder.py:189: self.msgout(4, "raise ImportError: No module named", qname)
Lib/modulefinder.py:190: raise ImportError("No module named " + qname)
Lib/modulefinder.py:202: self.msgout(4, "raise ImportError: No module named", mname)
Lib/modulefinder.py:203: raise ImportError("No module named " + mname)
Lib/modulefinder.py:219: raise ImportError("No module named " + subname)
Lib/importlib/_bootstrap.py:811: raise ImportError("no module named {}; "
Lib/importlib/_bootstrap.py:820: raise ImportError("No module named {0}".format(name))
Lib/unittest/test/test_loader.py:242: self.assertEqual(str(e), "No module named sdasfasfasdf")
Lib/unittest/test/test_loader.py:622: self.assertEqual(str(e), "No module named sdasfasfasdf")

Fixing the Python modules is easy, but I don’t know how to change PyErr_Format (or how to write C, for that matter :) Do you make offers like Alexandre, who has proposed to write the C part if someone provides a diff for the Python version?

[merwok]

I should have included only the one line that’s different:
Lib/importlib/_bootstrap.py:811: raise ImportError("no
(lower-case n)

[brettcannon]

PyErr_Format doesn't need to change, just it's argument. A call to PyObject_Repr() (w/ proper error checking) should be all that is needed.

And no, I don't make any "you do the Python, I'll do the C" deals because the Python part is the fun part. =)

[merwok]

A call to PyObject_Repr() (w/ proper error checking) should be all
that is needed.
Sadly out of reach for me.

And no, I don't make any "you do the Python, I'll do the C" deals
because the Python part is the fun part. =)
I understand :)

FTR, case inconsistency fixed by Brett in r86507.

[merwok]

Just for fun, I tried using my tiny understanding of C to write a patch. I am attaching my current result, which passes the test suite except for test_unittest:

FAIL: test_loadTestsFromName__unknown_module_name (unittest.test.test_loader.Test_TestLoader)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/wok/code/hg/cpython/3.2/fix-import-repr/Lib/unittest/test/test_loader.py", line 240, in test_loadTestsFromName__unknown_module_name
    loader.loadTestsFromName('sdasfasfasdf')
ImportError: No module named 0I��

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wok/code/hg/cpython/3.2/fix-import-repr/Lib/unittest/test/test_loader.py", line 242, in test_loadTestsFromName__unknown_module_name
    self.assertEqual(str(e), "No module named 'sdasfasfasdf'")
AssertionError: 'No module named 0I�\x01' != "No module named 'sdasfasfasdf'"
- No module named 0I��
+ No module named 'sdasfasfasdf'

======================================================================
FAIL: test_loadTestsFromNames__unknown_module_name (unittest.test.test_loader.Test_TestLoader)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/unittest/test/test_loader.py", line 620, in test_loadTestsFromNames__unknown_module_name
    loader.loadTestsFromNames(['sdasfasfasdf'])
ImportError: No module named HB��

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Lib/unittest/test/test_loader.py", line 622, in test_loadTestsFromNames__unknown_module_name
    self.assertEqual(str(e), "No module named 'sdasfasfasdf'")
AssertionError: 'No module named HB�\x01' != "No module named 'sdasfasfasdf'"
- No module named HB��
+ No module named 'sdasfasfasdf'

test_imp{,ort,ortlib} pass, though. Maybe someone will take my patch and fix the PyObject_Repr call and error checking. It was a fun bit of experimenting with help from kind people on IRC :)

[merwok]

I fixed my patch, thanks to a recent commit that showed me an example of PyErr_Format :) All tests now pass.