bytearray partition bug

[BreamoreBoy]

BPO	20047
Nosy	@loewis, @brettcannon, @birkenfeld, @terryjreedy, @mdickinson, @pitrou, @ezio-melotti, @berkerpeksag, @serhiy-storchaka
PRs	bpo-20047: Make bytearray methods partition() and rpartition() rejecting #4158 [3.6] bpo-20047: Make bytearray methods partition() and rpartition() rejecting (GH-4158) #4162 [2.7] bpo-20047: Make bytearray methods partition() and rpartition() rejecting (GH-4158) #4163 [2.7] bpo-20047: Remove Objects/bytesobject.c from 2.7 #9268

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-10-29.10:26:39.904>
created_at = <Date 2013-12-21.19:52:41.350>
labels = ['interpreter-core', 'type-bug', '3.7']
title = 'bytearray partition bug'
updated_at = <Date 2018-09-13.23:16:44.290>
user = 'https://bugs.python.org/BreamoreBoy'

bugs.python.org fields:

activity = <Date 2018-09-13.23:16:44.290>
actor = 'berker.peksag'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-10-29.10:26:39.904>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2013-12-21.19:52:41.350>
creator = 'BreamoreBoy'
dependencies = []
files = []
hgrepos = []
issue_num = 20047
keywords = ['patch']
message_count = 10.0
messages = ['206773', '206774', '206775', '206792', '206793', '305167', '305170', '305183', '305184', '325306']
nosy_count = 9.0
nosy_names = ['loewis', 'brett.cannon', 'georg.brandl', 'terry.reedy', 'mark.dickinson', 'pitrou', 'ezio.melotti', 'berker.peksag', 'serhiy.storchaka']
pr_nums = ['4158', '4162', '4163', '9268']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20047'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[BreamoreBoy]

If partition is called with a single byte it works correctly but if called with the equivalent integer it returns the same bytearray with two empty arrays as follows.

py> ba = bytearray(range(8))
py> ba
bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07')
py> 3 in ba
True
py> ba.find(3) == ba.index(3) == ba.find(b'\x03')
True
py> ba.partition(b'\x03')
(bytearray(b'\x00\x01\x02'), bytearray(b'\x03'), bytearray(b'\x04\x05\x06
\x07'))
py> ba.partition(3)
(bytearray(b'\x00\x01\x02\x03\x04\x05\x06\x07'), bytearray(b''), bytearray
(b''))

More background on the thread starting here https://mail.python.org/pipermail/python-list/2013-December/663111.html which refers to bpo-12170.

[serhiy-storchaka]

Similar bug was in 3.2:

>>> ba = bytearray(range(8))
>>> ba[2:6]
bytearray(b'\x02\x03\x04\x05')
>>> ba[2:6] = 2
>>> ba
bytearray(b'\x00\x01\x00\x00\x06\x07')

Now it is fixed.

[serhiy-storchaka]

Bytearray slice assignment bug was fixed in bpo-8401.

[terryjreedy]

Whatever the change, bytes and bytearray should act the same.

>>> b = bytes(range(8))
>>> b
b'\x00\x01\x02\x03\x04\x05\x06\x07'
>>> b.partition(3)
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    b.partition(3)
TypeError: expected bytes, bytearray or buffer compatible object

As noted in the thread, ba.partition(a) apparently is executed as ba.partition(bytearray(a)) if a is not a bytearray (or maybe not a buffer object). bytearray(3) == bytearray((0,0,0)) and the latter is not in ba and hence the output given is 'correct'.

[BreamoreBoy]

I believe that all methods should act the same, but they don't as a result of the work done in bpo-12170. E.g. find will accept integer input but split will not. Given this comment at the top of test_bytes.py "XXX This is a mess. Common tests should be moved to buffer_tests.py, which itself ought to be unified with string_tests.py (and the latter should be modernized).", it looks like a thorough review of the code and tests is in order.

[terryjreedy]

To answer Mark, even though no longer nosy: In general, sequence methods .count, .index, and .__contains__ take sequence members and only members as arguments. Unicode sequences are exceptional because codepoints are not Python objects, so string subsequences must be used instead. Byte-like sequences are also exceptional in that both members and subsequences are accepted for these methods.

String-like sequence methods .split and .partition take subsequences as arguments. I think the doc should make this clearer.

[serhiy-storchaka]

New changeset a231428 by Serhiy Storchaka in branch 'master':
bpo-20047: Make bytearray methods partition() and rpartition() rejecting (bpo-4158)
a231428

[serhiy-storchaka]

New changeset 9ea5a3a by Serhiy Storchaka in branch '3.6':
[3.6] bpo-20047: Make bytearray methods partition() and rpartition() rejecting (GH-4158) (bpo-4162)
9ea5a3a

[serhiy-storchaka]

New changeset 107f3cc by Serhiy Storchaka in branch '2.7':
[2.7] bpo-20047: Make bytearray methods partition() and rpartition() rejecting (GH-4158) (bpo-4163)
107f3cc

[berkerpeksag]

New changeset 0b9fe17 by Berker Peksag (Zackery Spytz) in branch '2.7':
[2.7] bpo-20047: Remove Objects/bytesobject.c from 2.7 (GH-9268)
0b9fe17