regrtest: use backslashreplace error handler for stdout

[vstinner]

BPO	8533
Nosy	@pitrou, @vstinner, @florentx
Files	regrtest_stdout_backslashreplace.patch regrtest_traceback_stderr-2.patch regrtest_stdout_newline.patch py3k_regrtest_newline.patch py3k_regrtest_newline_more.patch py3k_also_no_unicode_error_on_direct_test_run.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-05-30.21:09:09.043>
created_at = <Date 2010-04-26.10:43:12.726>
labels = ['tests', 'expert-unicode']
title = 'regrtest: use backslashreplace error handler for stdout'
updated_at = <Date 2011-05-30.21:09:09.042>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2011-05-30.21:09:09.042>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2011-05-30.21:09:09.043>
closer = 'vstinner'
components = ['Tests', 'Unicode']
creation = <Date 2010-04-26.10:43:12.726>
creator = 'vstinner'
dependencies = []
files = ['17090', '17152', '17228', '18824', '18825', '19191']
hgrepos = []
issue_num = 8533
keywords = ['patch']
message_count = 21.0
messages = ['104212', '104223', '104229', '104231', '104670', '104773', '104783', '104794', '104824', '105087', '105180', '116003', '116004', '118248', '118253', '118256', '118395', '118396', '123605', '136813', '137330']
nosy_count = 5.0
nosy_names = ['pitrou', 'vstinner', 'ocean-city', 'flox', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue8533'
versions = ['Python 3.1', 'Python 3.2']

[vstinner]

If a test fails, regrtest writes the backtrace to sys.stdout. If the backtrace contains a non-ASCII characters, it's encoded using sys.stdout encoding.

In some conditions, sys.stdout is unable to encode some or all non-ASCII characters. Eg. if there is no locale set (empty environment or at least empty LANG variable value), sys.stdout.encoding="ascii".

If regrtest fails to display a test output (error backtrace), regrtest exits directly (don't execute next tests).

I propose to use backslashreplace error handler in sys.stdout, as done for sys.stderr to avoid this annoying issue.

Attached patch (for py3k) replace sys.stdout by a new file using backslashreplace, just before executing the tests.

I don't know if the issue concerns also Python2.

[pitrou]

A better resolution IMO would be to output tracebacks on stderr instead.

[vstinner]

A better resolution IMO would be to output tracebacks on stderr instead.

Yeah, that sounds easier and safer. Attached patch writes the tracebacks to stderr.

I don't remember how to reproduce this issue :-/ I guess that stdout/stderr encoding should be ASCII (eg. LANG="") and the traceback (eg. the error message) should contain a non-ASCII character.

[vstinner]

Write to stderr instead of stdout might change buildbot output order.

[vstinner]

regrtest_traceback_stderr.patch is not enough: support._run_suite() writes output to sys.stdout instead of sys.stderr. New version of the patch fixes that.

[vstinner]

Ok, let's try sys.stderr solution: commited in r80694 (py3k).

If it breaks buildbot outputs, I will revert it and try the second solution.

[pitrou]

Since it may reorder output, I think it's better revert the patch and try the other solution. However, I don't think you need to replace sys.stdout at all: just output the traceback more carefully.

[vstinner]

As "expected", the patch doesn't work: it randomize the output order :-(

I checked "sparc Ubuntu 3.x": the output order is correct. "test_xxx" lines are written to stdout, "FAIL: ..." + traceback are written to stderr, and the lines are written in the right order.

But it failed on "x86 Tiger 3.x" :
http://www.python.org/dev/buildbot/3.x/builders/x86 Tiger 3.x/builds/130/steps/test/logs/stdio
http://www.python.org/dev/buildbot/3.x/builders/x86 Tiger 3.x/builds/131/steps/test/logs/stdio

When a test is reexecuted in verbose mode, the output is written in red (why not, but it was not the case before my commit), and the stdout and stderr are in a "mixed".

--

antoine> just output the traceback more carefully.

Encode the traceback by hand would be possible, but it's more complex. A possible solution would be to write the output to a StringIO, encode unicode to bytes using the right encoding and backslashreplace error handler, and write the result to stdout. But I don't like buffering the output because the buildbot may hung for different reasons (search in the bug tracker for "test_multiprocessing" or "test_subprocess"...) and the buildbot master may consider the buildbot as dead (no new output since xxx seconds, whereas it's writing to a buffer).

--

I prefer my first simple idea: use backslashreplace error handler for stdout. Let's try: r80703. This commit reverts my previous commit, apply regrtest_stdout_backslashreplace.patch + a fix for multiprocessing mode (regrtest.py -j ...).

[vstinner]

Let's try: r80703

This one looks ok: the output order is kept and I didn't noticed anything special in the buildbot output.

Backported to 3.1 as r80711. Close the issue.

[vstinner]

Reopen: r80703 (and r80711) introduces a new bug: on Windows, there is an empty string between each line. It looks like a newline error.

replace_stdout() should set the newline argument to open(). But how can I get the newline attribute from sys.stdout? sys.stdout._writenl?

regrtest_stdout_newline.patch should fix the newline problem.

[vstinner]

I disabled my patch (replace_stdout function) on Windows until it gets fixed: r80905 (py3k) and r80906 (3.1)