test_idle: test_error() failed on aarch64 RHEL8 Refleaks 3.8

[vstinner]

BPO	41514
Nosy	@terryjreedy, @vstinner, @ambv, @miss-islington
PRs	bpo-41514: Fix buggy IDLE test #21808 [3.9] bpo-41514: Fix buggy IDLE test (GH-21808) #21809 [3.8] bpo-41514: Fix buggy IDLE test (GH-21808) #21810

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2020-08-10.19:44:33.637>
created_at = <Date 2020-08-10.12:02:17.078>
labels = ['type-bug', '3.8', '3.9', '3.10', 'expert-IDLE', 'tests']
title = 'test_idle: test_error() failed on aarch64 RHEL8 Refleaks 3.8'
updated_at = <Date 2020-08-10.19:44:33.636>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-08-10.19:44:33.636>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2020-08-10.19:44:33.637>
closer = 'terry.reedy'
components = ['IDLE', 'Tests']
creation = <Date 2020-08-10.12:02:17.078>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 41514
keywords = ['patch']
message_count = 7.0
messages = ['375106', '375110', '375112', '375116', '375119', '375120', '375144']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'vstinner', 'lukasz.langa', 'miss-islington']
pr_nums = ['21808', '21809', '21810']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41514'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[vstinner]

https://buildbot.python.org/all/#/builders/597/builds/178

test_error (idlelib.idle_test.test_run.HandleErrorTest) ... FAIL
(...)

======================================================================
FAIL: test_error (idlelib.idle_test.test_run.HandleErrorTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.8.cstratak-RHEL8-aarch64.refleak/build/Lib/unittest/mock.py", line 1325, in patched
    return func(*newargs, **newkeywargs)
  File "/home/buildbot/buildarea/3.8.cstratak-RHEL8-aarch64.refleak/build/Lib/idlelib/idle_test/test_run.py", line 352, in test_error
    eq(self.func.called, 2)
AssertionError: 4 != 2

I can reproduce the issue on 3.8, 3.9 and master branches:

14:00:18 vstinner@apu$ ./python -m test -m test_error -v test_idle test_idle
== CPython 3.8.5+ (heads/3.8:61f23cb62d, Aug 10 2020, 14:00:04) [GCC 10.2.1 20200723 (Red Hat 10.2.1-1)]
== Linux-5.7.11-200.fc32.x86_64-x86_64-with-glibc2.29 little-endian
== cwd: /home/vstinner/python/3.8/build/test_python_43858
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.68 Run tests sequentially
0:00:00 load avg: 1.68 [1/2] test_idle
test_error (idlelib.idle_test.test_run.HandleErrorTest) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.004s

OK
0:00:00 load avg: 1.68 [2/2] test_idle
test_error (idlelib.idle_test.test_run.HandleErrorTest) ... FAIL

======================================================================
FAIL: test_error (idlelib.idle_test.test_run.HandleErrorTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/3.8/Lib/unittest/mock.py", line 1325, in patched
    return func(*newargs, **newkeywargs)
  File "/home/vstinner/python/3.8/Lib/idlelib/idle_test/test_run.py", line 352, in test_error
    eq(self.func.called, 2)
AssertionError: 4 != 2

---

Ran 1 test in 0.007s

FAILED (failures=1)
test test_idle failed
test_idle failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_idle

Total duration: 722 ms
Tests result: FAILURE

It seems like the test was added recently:

commit f2e161c
Author: Terry Jan Reedy <tjreedy@udel.edu>
Date: Sun Aug 9 16:08:30 2020 -0400

bpo-41468: Improve and test IDLE run error exit  (GH-21798)

A message box pops up when an unexpected error stops the run process.  Tell users it is likely a random glitch, but report it if not.

[terryjreedy]

Your reproducer worked to reproduce the bug for me, make the cause obvious, and test the fix.

Łukasz, it would be good if this test fix made it into the release candidate, but probably not essential as not too many people will check it for refleaks.