IDLE: ColorDelegatorTest test_incremental_editing failures

[terryjreedy]

BPO	44282
Nosy	@terryjreedy, @vstinner, @taleinat, @miss-islington
PRs	bpo-41611: IDLE: Catch TclError exceptions in AutoCompleteWindow.winconfig_event() #26404 bpo-44282: fix occasional test_incremental_editing failures on buildbots #26491 [3.10] bpo-44282: Fix occasional test_incremental_editing failures on buildbots (GH-26491) #26499
Files	tkinter_testing_utils.py: Adaptive

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2021-06-03.03:17:41.798>
created_at = <Date 2021-06-01.20:38:25.128>
labels = ['expert-IDLE', 'type-bug', 'tests', '3.11']
title = 'IDLE: ColorDelegatorTest test_incremental_editing failures'
updated_at = <Date 2021-06-03.03:17:41.798>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2021-06-03.03:17:41.798>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2021-06-03.03:17:41.798>
closer = 'terry.reedy'
components = ['IDLE', 'Tests']
creation = <Date 2021-06-01.20:38:25.128>
creator = 'terry.reedy'
dependencies = []
files = ['50084']
hgrepos = []
issue_num = 44282
keywords = ['patch']
message_count = 14.0
messages = ['394870', '394914', '394918', '394919', '394921', '394925', '394936', '394937', '394939', '394941', '394942', '394959', '394962', '394973']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'vstinner', 'taleinat', 'miss-islington']
pr_nums = ['26404', '26491', '26499']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44282'
versions = ['Python 3.11']

[terryjreedy]

idlelib.idle_test.test_colordelagator.ColorDelegatorTest.test_incremental_editing has failed on at least on the following machines.

Pip32 CI Pipelines Win32
Gen3x https://buildbot.python.org/all/#/builders/464 x86 Gentoo Installed with X 3.x
GenPR https://buildbot.python.org/all/#/builders/465 x86 Gentoo Installed with X PR

Failures seen. Since tests are sequential without subtesting, failure means successors are not tested.

line keyword start want got
569 ('BUILTIN', '1.0'), ('1.0', '1.3') () Gen3x, x3; GenPR
573 ('BUILTIN', '1.0'), ('1.0', '1.3') () Gen3x, x5
586 ('BUILTIN', '1.0'), ('1.0', '1.3') () GenPR
597 ('KEYWORD', '1.0'), () ('1.0', '1.1') Pip32, x2

As far as I can tell, looking at Windows buildbots, the 1 failure on Win32 has not been repeated. If there have been any on CI machines, I have not been informed.

A few years ago, I believe Gentoo was the only *nix buildbot running tkinter and IDLE tests. It has a problem with a builtin at the beginning of the line about 10-20% of test runs. This are the majority of its failures in the last week. Perhaps there is a timing issue we can fix.

[vstinner]

Buildbots are very unstable these days. I propose to revert the change to give more time to IDLE developers to investigate the issue, and to be able to identify other regressions on buildbots.
https://pythondev.readthedocs.io/ci.html#revert-on-fail

I created #26487 to revert

[taleinat]

Please note that the referenced PR is not the correct one.

I'll add a link to the right one shortly.

[taleinat]

The latest PR where changes were made to IDLE's colorizer and its tests, including specifically adding the problematic test, is #70038.

[taleinat]

I'm investigating this now. If nothing obvious comes up, I'll revert that PR.