ctypes c_long & c_bool have incorrect PEP-3118 type codes

[pv]

BPO	10746
Nosy	@theller, @amauryfa, @mdickinson, @abalkin, @pitrou, @pv, @meadori, @csabella
PRs	bpo-10746: ctypes: Fix PEP 3118 type codes for c_long, c_bool, c_int #31 [3.6] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) #3241 [2.7] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) #3242
Files	001-ctypes-fix-pep-3118-type-codes-for-c-long-and-c-bool.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-09-02.16:24:58.195>
created_at = <Date 2010-12-20.23:37:20.797>
labels = ['ctypes', 'type-bug', '3.7']
title = 'ctypes c_long & c_bool have incorrect PEP-3118 type codes'
updated_at = <Date 2017-09-02.16:24:58.195>
user = 'https://github.com/pv'

bugs.python.org fields:

activity = <Date 2017-09-02.16:24:58.195>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2017-09-02.16:24:58.195>
closer = 'pitrou'
components = ['ctypes']
creation = <Date 2010-12-20.23:37:20.797>
creator = 'pv'
dependencies = []
files = ['20124']
hgrepos = []
issue_num = 10746
keywords = ['patch']
message_count = 10.0
messages = ['124411', '220819', '287795', '295247', '300951', '300952', '301015', '301016', '301019', '301165']
nosy_count = 8.0
nosy_names = ['theller', 'amaury.forgeotdarc', 'mark.dickinson', 'belopolsky', 'pitrou', 'pv', 'meador.inge', 'cheryl.sabella']
pr_nums = ['31', '3241', '3242']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10746'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[pv]

Currently on Python 3.x:

>>> import ctypes
>>> memoryview(ctypes.c_long()).format
'<l'

This is invalid on 64-bit platforms: the above means 32-bit little-endian float. The '<' endian specification turns on the "standard size" mode (similarly as for the struct module), which makes type character have a platform-independent meaning.

Unfortunately, the struct module format syntax does *not* allow specifying native-size non-native-endian items. So just replacing '<' by '^' cannot be in general be done.

Suggested fix attached. It adds a converter function that maps the platform-dependent ctypes codes to struct module standard-size codes; handling c_long and c_bool specially.

***

After this patch (and the one in http://bugs.python.org/issue10744 ):

>>> import numpy as np
>>> from ctypes import *
>>> class Point(Structure):
...     _fields_ = [("x", c_long), ("y", c_long)]
... 
>>> class StructWithArrays(Structure):
...     _fields_ = [("x", c_long * 3 * 2), ("y", Point * 4)]
... 
>>> x = StructWithArrays()
>>> y = np.asarray(x)
>>> y.dtype
dtype([('x', '<i8', (2, 3)), ('y', [('x', '<i8'), ('y', '<i8')], (4,))])
>>> y['x'] = [[1,2,3],[4,5,6]]
>>> y['y']['x'] = np.arange(4) + 10
>>> y['y']['y'] = np.arange(4) + 20
>>> x.x[0][0]
1
>>> x.x[0][1]
2
>>> x.x[0][2]
3
>>> x.y[0].x
10
>>> x.y[1].x
11
>>> x.y[0].y
20
>>> x.y[1].y
21

[BreamoreBoy]

Could someone do a patch review on this please.

[pv]

Converted patch to Github PR + some cleanup.

[csabella]

Added the names from the experts list to this ticket.

[pitrou]

New changeset 07f1658 by Antoine Pitrou (Pauli Virtanen) in branch 'master':
bpo-10746: Fix ctypes PEP-3118 type codes for c_long, c_bool, c_int (#31)
07f1658

[pitrou]

Pauli, do you want to submit backport PRs? This can be done using the cherry_picker script: https://devguide.python.org/committing/?highlight=cherry_picker#backporting-changes-to-an-older-version

[pv]

Created backport PR for 3.6: #3241
Which versions take backports?

[pitrou]

At this point, 2.7 and 3.6 do.

[pitrou]

New changeset 2d1653a by Antoine Pitrou (Pauli Virtanen) in branch '3.6':
[3.6] bpo-10746: Fix ctypes PEP-3118 type codes for c_long, c_bool, c_int (GH-31) (bpo-3241)
2d1653a

[pitrou]

New changeset 990b2d0 by Antoine Pitrou (Pauli Virtanen) in branch '2.7':
[2.7] bpo-10746: Fix ctypes PEP-3118 type codes for c_long, c_bool, c_int (GH-31) (bpo-3242)
990b2d0