regrtest: capture stdout (-W) option is incompatible with refleak hunter (-R)

[vstinner]

BPO	27103
Nosy	@vstinner, @zware, @serhiy-storchaka
PRs	bpo-27103: regrtest disables -W if -R is used #1651 [3.6] bpo-27103: regrtest disables -W if -R is used (#1651) #1656 [3.5] bpo-27103: regrtest disables -W if -R is used #1660

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-06-28.01:06:44.592>
created_at = <Date 2016-05-24.15:20:30.033>
labels = ['3.7', 'tests']
title = 'regrtest: capture stdout (-W) option is incompatible with refleak hunter (-R)'
updated_at = <Date 2017-06-28.01:06:44.591>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2017-06-28.01:06:44.591>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-28.01:06:44.592>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2016-05-24.15:20:30.033>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 27103
keywords = []
message_count = 6.0
messages = ['266254', '293936', '293937', '293939', '293949', '297100']
nosy_count = 3.0
nosy_names = ['vstinner', 'zach.ware', 'serhiy.storchaka']
pr_nums = ['1651', '1656', '1660']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue27103'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

[vstinner]

The -R option of regrtest replaces sys.stdout with a io.StringIO object. Problem: the refleak are hunted after stdout is replaced, and so each print() increaes the total reference counter which is seen as a leak.

Example:

$ cat Lib/test/test_noop.py 
"noop"

$ ./python -u -m test -R 3:3 -W test_noop 
Run tests sequentially
0:00:00 [1/1] test_noop

---

Ran 0 tests in 0.000s

OK
beginning 6 repetitions
123456

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
.
(...)
test_noop leaked [9, 9, 9] references, sum=27
test_noop failed
1 test failed:
test_noop
Total duration: 0:00:01

[vstinner]

New changeset fcdd9b6 by Victor Stinner in branch 'master':
bpo-27103: regrtest disables -W if -R is used (bpo-1651)
fcdd9b6

[vstinner]

It seems like Python 2.7 is not affected.

I will apply my workaround to 3.5, 3.6 and master, since Zachary created a buildbot testing 2.7, 3.5, 3.6 and master branches.

[vstinner]

New changeset 2773add by Victor Stinner in branch '3.6':
bpo-27103: regrtest disables -W if -R is used (bpo-1651) (bpo-1656)
2773add

[vstinner]

New changeset 9503dd1 by Victor Stinner in branch '3.5':
bpo-27103: regrtest disables -W if -R is used (bpo-1660)
9503dd1

[vstinner]

While the root issue is not fixed, at least -W doesn't cause issues anymore with -R, so I consider that the "bug" was fixed ;-)