<unprintable AssertionError object> message in unittest tracebacks

[voidspace]

BPO	8313
Nosy	@gpshead, @amauryfa, @vstinner, @ezio-melotti, @voidspace
Files	unittest2-issue-8313.patch: Patch: more useful standard assertion messages on unicode comparisons traceback_unicode.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/voidspace'
closed_at = <Date 2010-05-05.12:46:29.813>
created_at = <Date 2010-04-05.10:56:28.021>
labels = ['type-bug', 'library', 'expert-unicode']
title = '<unprintable AssertionError object> message in unittest tracebacks'
updated_at = <Date 2011-12-23.15:45:17.636>
user = 'https://github.com/voidspace'

bugs.python.org fields:

activity = <Date 2011-12-23.15:45:17.636>
actor = 'michael.foord'
assignee = 'michael.foord'
closed = True
closed_date = <Date 2010-05-05.12:46:29.813>
closer = 'vstinner'
components = ['Library (Lib)', 'Unicode']
creation = <Date 2010-04-05.10:56:28.021>
creator = 'michael.foord'
dependencies = []
files = ['17201', '17202']
hgrepos = []
issue_num = 8313
keywords = ['patch']
message_count = 13.0
messages = ['102368', '104937', '104940', '104943', '104946', '104947', '104949', '104989', '105011', '105022', '150127', '150128', '150172']
nosy_count = 6.0
nosy_names = ['gregory.p.smith', 'amaury.forgeotdarc', 'vstinner', 'ezio.melotti', 'michael.foord', 'gthb']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8313'
versions = ['Python 2.7']

[voidspace]

>>> import unittest
>>> class Foo(unittest.TestCase):
...   def test_fffd(self): self.assertEqual(u'\ufffd', u'\ufffd\ufffd')
...
>>> unittest.main(exit=False)
F

======================================================================
FAIL: test_fffd (main.Foo)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "<stdin>", line 2, in test_fffd
AssertionError: <unprintable AssertionError object>

---

Ran 1 test in 0.001s

The problem with creating unicode tracebacks is that they could fail when being output on terminals not capable of showing the full range of unicode characters (the default terminal on Windows is CP1252).

This can already happen with Unicode messages that aren't part of the traceback.

Detecting the 'unprintable' message before calling into traceback and replacing it with the repr of the unicode is one possibility.

[gthb]

Replacing the message with its repr seems to me at least strongly preferable to the current “hide it all” behavior. :)

Better, msg.encode('ascii', 'backslashreplace') does what repr does with unencodable characters, but does not add the quotes, so the behavior is only different when it needs to be.

Better still, 'ascii' need not be hardcoded. I'm attaching a patch that sets the encoding from an environment variable, defaulting to 'ascii', and encodes the message with 'backslashreplace'. This makes unicode string equality assertions much more useful for me.

The encoding could also be configurable by some clean hook for test runners to use. unit2 could have a command-line parameter, and TextTestRunner could use stream.encoding if not None (or PYTHONIOENCODING on Python 3).

Ideally messages should not be forced to be 8-bit strings by the failure exception class, but I suppose that's a bigger change than you would want to make.

The downside of using backslashreplace (or repr, for that matter) is that it does not preserve lengths, so the diff markers can get misaligned. I find that an acceptable tradeoff, but 'replace' is another option that preserves lengths, at least more often.

[voidspace]

Sounds like a good solution - I'll look at this, thanks.

[amauryfa]

Very recently, bpo-8533 changed regrtest.py to use 'backslashreplace' when printing errors. This issue seems very similar

[vstinner]

The example raises an AssertionError(u'\n- \ufffd+ \ufffd\ufffd') which is converted to string by traceback.format_exception(). This function fails in _some_str() on str(value) instruction. You can reproduce the error with:

>>> str(AssertionError(u"\xe9"))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 0: ordinal not in range(128)

The problem with creating unicode tracebacks is that they could
fail when being output on terminals not capable of showing
the full range of unicode characters (the default terminal
on Windows is CP1252).

The problem is not related to the terminal encoding: str(value) uses Python default encoding (ASCII by default). Python3 is not concerned because str(AssertionError("\xe9")) doesn't raise any error: it returns "\xe9".

[vstinner]

Very recently, bpo-8533 changed regrtest.py to use
'backslashreplace' when printing errors. This issue seems
very similar

Issue bpo-8533 is not directly related because in this issue the error occurs before writing the traceback to the terminal.

[vstinner]

Attached patch fixes _some_str() function of the traceback module: encode unicode exception message to ASCII using backslashreplace error handler. ASCII is not the best choice, but str(unicode(...)) uses also ASCII (the default encoding) and we don't know the terminal encoding in traceback. We cannot do better here in Python2 (without breaking a lot of APIs...).

The right fix is to use Python3 which formats a traceback to unicode (unicode characters of the error message are kept unchanged). The choice of the encoding and error handler is made only at the end, when writing the output to the terminal, which is the right thing to do.

[vstinner]

The downside of using backslashreplace (or repr, for that matter) is
that it does not preserve lengths, so the diff markers can get
misaligned. I find that an acceptable tradeoff, but 'replace' is
another option that preserves lengths, at least more often.

'replace' loose important informations: if the test is about the unicode string content, we will be unable to see the error data.

Result of the first example with my patch (backslashreplace):
======================================================================
FAIL: test_fffd (main.Foo)
----------------------------------------------------------------------

Traceback (most recent call last):                                    
  File "x.py", line 3, in test_fffd                                   
    def test_fffd(self): self.assertEqual(u'\ufffd', u'\ufffd\ufffd') 
AssertionError:                                                       
- \ufffd+ \ufffd\ufffd                                                

Result of the first example with 'replace' error handler:
======================================================================
FAIL: test_fffd (main.Foo)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "x.py", line 3, in test_fffd
    def test_fffd(self): self.assertEqual(u'\ufffd', u'\ufffd\ufffd')
AssertionError:
- ?+ ??

(but this example is irrevelant because U+FFFD is the unicode replacement character :-D)

If nobody complains about my patch, I will commit it to Python trunk (only).

You can still reimplement fail() method to encode the message using a more revelant encoding and/or error handler.

[voidspace]

I would prefer to try str(...) first and only attempt to convert to unicode and do the backslash replace if the str(...) call fails.

[vstinner]

Commited: r80777 (trunk) and r80779 (2.6); blocked: r80778 (py3k).

Open a new issue if you would like to use something better than ASCII+backslashreplace in unittest (using runner stream encoding?).