-
-
Notifications
You must be signed in to change notification settings - Fork 33.7k
gh-129005: Remove copies from _pyio using take_bytes #141539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Memory usage now matches that of _io for large files
|
Optimization enabled by gh-139871 (Trying to keep that one issue referencing all the things which I use it for; planning to expand the What's New with a single "here's all the modules that got faster" list once optimize) |
| return None | ||
| if n < 0 or n > len(b): | ||
| raise ValueError(f"readinto returned {n} outside buffer size {len(b)}") | ||
| del b[n:] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a difference between b.resize(n) and del b[n:]?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.resize() could grow while the del b[n:] never will. There's some code which does a = bytearray(b'12'); del a[3:] (intentionally del a slice past the end) as part of its buffer management. Generally I prefer resize because there's a lot less code involved than the slice code (and the slice code eventually calls resize anyways)
vstinner
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…1539) Memory usage now matches that of _io for large files.
Memory usage now matches that of _io for large files.
For reference; this test goes from 8.5sec overall duration on my AMD 64 bit Arch Linux box to 7 seconds.