Support libffi implementations that cannot support invocations with 1024 arguments

[hoodmane]

BPO	47208
Nosy	@amauryfa, @abalkin, @tiran, @meadori, @hoodmane
PRs	bpo-47208: Allow vendors to override CTYPES_MAX_ARGCOUNT (GH-32297) #32297

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-04-04.00:27:44.616>
labels = ['ctypes', 'type-bug', 'tests', '3.11']
title = 'Support libffi implementations that cannot support invocations with 1024 arguments'
updated_at = <Date 2022-04-04.19:07:48.227>
user = 'https://github.com/hoodmane'

bugs.python.org fields:

activity = <Date 2022-04-04.19:07:48.227>
actor = 'hoodchatham'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests', 'ctypes']
creation = <Date 2022-04-04.00:27:44.616>
creator = 'hoodchatham'
dependencies = []
files = []
hgrepos = []
issue_num = 47208
keywords = ['patch']
message_count = 4.0
messages = ['416646', '416677', '416678', '416692']
nosy_count = 5.0
nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'christian.heimes', 'meador.inge', 'hoodchatham']
pr_nums = ['32297']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue47208'
versions = ['Python 3.11']

[hoodmane]

ctypes defines CTYPES_MAX_ARGCOUNT to be 1024:

cpython/Modules/_ctypes/ctypes.h

Line 21 in 6db2db9

#define CTYPES_MAX_ARGCOUNT 1024

If a function call is attempted with more than 1024 arguments, it will fail. The libffi emscripten port github.com/hoodmane/libffi-emscripten can only support function calls with at most 122 arguments due to limitations in Emscripten / Wasm:
emscripten-core/emscripten#16653

I propose to allow the libffi port to define FFI_MAX_CLOSURE_ARGS and if this is defined then use this number instead for CTYPES_MAX_ARGCOUNT.
libffi/libffi#703

The test test_callback_too_many_args should also be updated to respect the value of CTYPES_MAX_ARGCOUNT rather than hardcoding 1024 into the test.

[tiran]

New changeset d1b1c88 by Christian Heimes in branch 'main':
bpo-47208: Allow vendors to override CTYPES_MAX_ARGCOUNT (GH-32297)
d1b1c88

[tiran]

#76478 adds #ifndef CTYPES_MAX_ARGCOUNT and exposes the constant in the _ctypes extension module, so the test case can use the constant. It allows you to override CTYPES_MAX_ARGCOUNT until libffi upstream agrees on FFI_MAX_CLOSURE_ARGS.

[hoodmane]

Ideally the library should be able to define a different macro, e.g., CTYPES_CUSTOM_MAX_ARGCOUNT. This way the libffi port can define CTYPES_CUSTOM_MAX_ARGCOUNT and it will be compatible with older Python versions without causing Warning: "CTYPES_MAX_ARGCOUNT redefined`.

[furkanonder]

The issue seems to be solved, I think we can close it. @tiran