test_bad_getattr crashes on APPX test

[zooba]

BPO	40458
Nosy	@pfmoore, @vstinner, @tjguk, @ambv, @zware, @zooba, @miss-islington
PRs	bpo-40458: Increase reserved stack space to prevent overflow crash on Windows #19845 [3.8] bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845) #19941

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2020-09-24.20:39:51.631>
created_at = <Date 2020-04-30.23:10:27.627>
labels = ['3.8', 'OS-windows', 'type-crash', '3.9']
title = 'test_bad_getattr crashes on APPX test'
updated_at = <Date 2020-09-24.20:39:51.631>
user = 'https://github.com/zooba'

bugs.python.org fields:

activity = <Date 2020-09-24.20:39:51.631>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2020-09-24.20:39:51.631>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2020-04-30.23:10:27.627>
creator = 'steve.dower'
dependencies = []
files = []
hgrepos = []
issue_num = 40458
keywords = ['patch']
message_count = 8.0
messages = ['367803', '367880', '367881', '367888', '368060', '368185', '368186', '368189']
nosy_count = 7.0
nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'lukasz.langa', 'zach.ware', 'steve.dower', 'miss-islington']
pr_nums = ['19845', '19941']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue40458'
versions = ['Python 3.8', 'Python 3.9']

[zooba]

The Windows CI machines on Azure Pipelines run additional tests to check an "installed" layout and with the UWP entry point that's used for the Windows Store package.

These tests have been failing intermittently (though regularly) with a stack overflow crash in the PyPickler tests.

Example: https://dev.azure.com/Python/cpython/_build/results?buildId=62055&view=results

test_attribute_name_interning (test.test_pickle.PyPicklerTests) ... ok
File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
...

I assume this is due to having more code on the start at the start, and so the recursion limit isn't low enough. But it might also be worth checking this particular case to see whether there is unnecessary data being kept on the stack (e.g. in local C variables).

The crash occurs in both 3.8 and master, but not 3.7.

[zooba]

Turns out the stack reservation was different, which is why it was crashing.

As part of diagnosing it, I added the recursion count to faulthandler's output on Windows, but couldn't see if/where to do it for other platforms - Victor, any suggestions?

[zooba]

Also, it was really test_bad_getattr that was crashing.

[vstinner]

Also, it was really test_bad_getattr that was crashing.

Ah, an old friend. It seems like an issue with the maximum stack size. Here are my notes:
https://pythondev.readthedocs.io/unstable_tests.html#unlimited-recursion

Either reduce Python maximum stack depth, or increase the C stack size.

[zooba]

Yeah, I already got that part. If you check the PR, I added some better diagnostics to faulthandler for this case, but I don't see where I can add it for other platforms?