test_syntax_error fails when run in the installed location

[doko42]

BPO	17756
Nosy	@birkenfeld, @terryjreedy, @doko42, @rbtcollins, @ezio-melotti, @voidspace

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-01-01.05:47:35.806>
created_at = <Date 2013-04-16.11:32:39.822>
labels = ['type-bug', 'tests']
title = 'test_syntax_error fails when run in the installed location'
updated_at = <Date 2016-01-01.05:47:35.792>
user = 'https://github.com/doko42'

bugs.python.org fields:

activity = <Date 2016-01-01.05:47:35.792>
actor = 'ezio.melotti'
assignee = 'none'
closed = True
closed_date = <Date 2016-01-01.05:47:35.806>
closer = 'ezio.melotti'
components = ['Tests']
creation = <Date 2013-04-16.11:32:39.822>
creator = 'doko'
dependencies = []
files = []
hgrepos = []
issue_num = 17756
keywords = []
message_count = 11.0
messages = ['187073', '199788', '217917', '217924', '218086', '218319', '218320', '218422', '218423', '251026', '257278']
nosy_count = 8.0
nosy_names = ['georg.brandl', 'terry.reedy', 'doko', 'rbcollins', 'ezio.melotti', 'michael.foord', 'python-dev', 'pitti']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17756'
versions = ['Python 3.3', 'Python 3.4']

[doko42]

no idea yet about that one ...

======================================================================
FAIL: test_syntax_error (test.test_code_module.TestInteractiveConsole)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/lib/python3.3/test/test_code_module.py", line 57, in test_syntax_error
    raise AssertionError("No syntax error from console")
AssertionError: No syntax error from console

[birkenfeld]

Can't reproduce with current 3.3 tip.

[doko42]

seen this again in our autopkg tester
https://jenkins.qa.ubuntu.com/view/Utopic/view/AutoPkgTest/job/utopic-adt-python3.4/12/

however now I can't reproduce this locally, and the test succeeds during the build.

[pitti]

I can reproduce this here. In that test, I added the following:

        with open('/tmp/debug', 'w') as f:
            for call in self.stderr.method_calls:
                f.write('call: %s\n' % str(call))

This gives:
=========== 8< ============

call: call.write('Python <MagicMock name=\'sys.version\' id=\'140431558254320\'> on <MagicMock name=\'sys.platform\' id=\'140431558262512\'>\nType "help", "copyright", "credits" or "license" for more information.\n(InteractiveConsole)\n')
call: call.write('Traceback (most recent call last):\n')
call: call.write('  File "/usr/lib/python3.4/code.py", line 90, in runcode\n')
call: call.write('    ')
call: call.write('exec(code, self.locals)')
call: call.write('\n')
call: call.write('  File "<console>", line 1, in <module>\n')
call: call.write('NameError')
call: call.write(': ')
call: call.write("name 'undefined' is not defined")
call: call.write('\n')
call: call.write('\n')
=========== 8< 

============

Observe that the test checks

            if 'NameError:' in ''.join(call[1]):

but the "NameError" and the ":" are in two different call lines. I don't know how self.stderr.method_calls is built, but this sounds like a race condition/timing or a stdin/out buffering issue? I. e. the original print that produces the "NameError:" certainly does that in two steps; perhaps the testsuite thing that repeatedly read()s on stderr catches these two as separate write()s then?

[doko42]

adding unittest developers

[voidspace]

It looks like the simplest fix would be to change "NameError:" to "NameError", as the problem is that they're (sometimes!?) on separate lines.

This still tests what we want to test.

[doko42]

sure, doing this. my follow-up question was if it is necessary to fix anything else in unittest.

[python-dev]

New changeset 20db5e9086d4 by doko in branch '3.4':

• Issue bpo-17756: Fix test_code test when run from the installed location.
  http://hg.python.org/cpython/rev/20db5e9086d4

New changeset 8885fc2e92b3 by doko in branch 'default':
Merge from 3.4:
http://hg.python.org/cpython/rev/8885fc2e92b3

[doko42]

not yet closing, to see if there are some stream buffering issues in mock

[terryjreedy]

In the absence of any further issue in over a year, can this be closed?

[ezio-melotti]

I'm going to close this since the reported issue has been fixed.

In the test sys.stderr was mocked, and Mock.method_calls simply records the calls done to the mock. Therefore the fact that 'NameError' and ':' were written on sys.stderr on two separate calls to write() is not a problem with mocks.