unittest.TestCase.subTest causes all subsequent tests to be skipped in failfast mode

[TreyCucco]

BPO	22894
Nosy	@pitrou, @rbtcollins, @ezio-melotti, @voidspace, @ethanfurman
Files	utbug.py: A python script demonstrating the bug subtests_failfast.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-11-23.15:06:40.650>
created_at = <Date 2014-11-17.22:18:09.098>
labels = ['type-bug', 'tests']
title = 'unittest.TestCase.subTest causes all subsequent tests to be skipped in failfast mode'
updated_at = <Date 2014-11-23.15:06:40.649>
user = 'https://bugs.python.org/TreyCucco'

bugs.python.org fields:

activity = <Date 2014-11-23.15:06:40.649>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2014-11-23.15:06:40.650>
closer = 'pitrou'
components = ['Tests']
creation = <Date 2014-11-17.22:18:09.098>
creator = 'Trey.Cucco'
dependencies = []
files = ['37218', '37238']
hgrepos = []
issue_num = 22894
keywords = ['patch']
message_count = 4.0
messages = ['231305', '231449', '231468', '231562']
nosy_count = 7.0
nosy_names = ['pitrou', 'rbcollins', 'ezio.melotti', 'michael.foord', 'ethan.furman', 'python-dev', 'Trey.Cucco']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22894'
versions = ['Python 3.4', 'Python 3.5']

[TreyCucco]

When running a test suite with the -f flag (--failfast), unittest seems to stop running tests once it exits a with self.subTest block.

In the attached script, run it without -f and test_b will run and fail. Run it with the -f flag and only the test_a test will run. test_b will not run and no errors will be reported.

I noticed this bug in 3.4.1 on OS X

[pitrou]

Here is a patch.

[voidspace]

Looks good, thanks for the quick response.

[python-dev]

New changeset 993e8f795194 by Antoine Pitrou in branch '3.4':
Issue bpo-22894: TestCase.subTest() would cause the test suite to be stopped when in failfast mode, even in the absence of failures.
https://hg.python.org/cpython/rev/993e8f795194

New changeset 04103cece49d by Antoine Pitrou in branch 'default':
Issue bpo-22894: TestCase.subTest() would cause the test suite to be stopped when in failfast mode, even in the absence of failures.
https://hg.python.org/cpython/rev/04103cece49d