Custom iterator to memoryview - performance improvement

[dxflores]

BPO	41732
Nosy	@gvanrossum, @vstinner, @corona10, @dxflores
PRs	bpo-41732: add iterator to memoryview #22119

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-09-08.09:19:26.875>
created_at = <Date 2020-09-06.09:06:52.941>
labels = ['3.10']
title = 'Custom iterator to memoryview - performance improvement'
updated_at = <Date 2020-09-08.10:11:31.178>
user = 'https://github.com/dxflores'

bugs.python.org fields:

activity = <Date 2020-09-08.10:11:31.178>
actor = 'dxflores'
assignee = 'none'
closed = True
closed_date = <Date 2020-09-08.09:19:26.875>
closer = 'corona10'
components = []
creation = <Date 2020-09-06.09:06:52.941>
creator = 'dxflores'
dependencies = []
files = []
hgrepos = []
issue_num = 41732
keywords = ['patch']
message_count = 6.0
messages = ['376460', '376549', '376550', '376558', '376559', '376561']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'vstinner', 'corona10', 'dxflores']
pr_nums = ['22119']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41732'
versions = ['Python 3.10']

[dxflores]

Hi,

Here's a patch that adds an iterator to memoryview, which makes iterating over memoryview objs considerably faster.

The following values result from a compilation with debug ON.

------------------------------------------------------------------------

Without patch:

cpython: ./python.exe -m timeit -s "a=memoryview(b'x'*1000000)" "for x in a: pass"
5 loops, best of 5: 98 msec per loop

cpython: ./python.exe -m timeit -s "a=b'x'*1000000" "for x in a: pass"
5 loops, best of 5: 68.6 msec per loop

With patch:

cpython: ./python.exe -m timeit -s "a=memoryview(b'x'*1000000)" "for x in a: pass"
5 loops, best of 5: 68.1 msec per loop

cpython: ./python.exe -m timeit -s "a=b'x'*1000000" "for x in a: pass"
5 loops, best of 5: 70 msec per loop

------------------------------------------------------------------------

Please let me know your thoughts regarding it.

Diogo

[corona10]

New changeset b0ac5d7 by dxflores in branch 'master':
bpo-41732: add iterator to memoryview (GH-22119)
b0ac5d7

[skrah]

It would have been nice to have had a chance to look at this, but
apparently those days are over.

[corona10]

Thanks, Diogo for working on this issue.
And thank you for the mentoring guido!

[vstinner]

Thanks Diogo Flores, it's a nice optimization!

Stefan Krah: "It would have been nice to have had a chance to look at this, but apparently those days are over."

The PR has been approved by two core developers. It's a nice optimization, I don't see what is the problem here.

[dxflores]

Thank you Dong-hee Na and Victor.

@Stefan Krah: I am sorry that you didn't had the chace to try out the patch before it got merged, however iff you are curious about the any aspect of the implementation (or the motivation behind it) I am happy to discuss it in depth.