Limit read size in File::read_to_end loop#110655
Conversation
This works around performance issues on Windows by limiting reads the size of reads when the expected size is known.
rustbot
added
S-waiting-on-review
|
Hey! It looks like you've submitted a new PR for the library teams! If this PR contains changes to any Examples of
|
joshtriplett
left a comment
joshtriplett
left a comment
There was a problem hiding this comment.
Please don't add this amount of additional space when reading (for instance) an 8 byte file. If we need a bit of slack to make sure we've gotten EOF, that's fine, but let's not add 1024 bytes or more to an 8 byte read.
|
@bors r+ rollup=never |
bors
added
S-waiting-on-bors
|
☀️ Test successful - checks-actions |
|
Finished benchmarking commit (9de7d91): comparison URL. Overall result: ✅ improvements - no action needed@rustbot label: -perf-regression Instruction countThis is a highly reliable metric that was used to determine the overall result at the top of this comment.
Max RSS (memory usage)ResultsThis is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
CyclesResultsThis is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
|
…ChrisDenton delay uncapping the max_read_size in File::read_to_end In rust-lang#130600 (comment) I realized that we're likely still passing too-large buffers to the OS, at least once at the end. Previous issues and PRs: * rust-lang#110650 * rust-lang#110655 * rust-lang#118222 r? ChrisDenton
…ChrisDenton delay uncapping the max_read_size in File::read_to_end In rust-lang#130600 (comment) I realized that we're likely still passing too-large buffers to the OS, at least once at the end. Previous issues and PRs: * rust-lang#110650 * rust-lang#110655 * rust-lang#118222 r? ChrisDenton
…ChrisDenton delay uncapping the max_read_size in File::read_to_end In rust-lang#130600 (comment) I realized that we're likely still passing too-large buffers to the OS, at least once at the end. Previous issues and PRs: * rust-lang#110650 * rust-lang#110655 * rust-lang#118222 r? ChrisDenton
Rollup merge of rust-lang#130670 - the8472:read-to-end-heuristics, r=ChrisDenton delay uncapping the max_read_size in File::read_to_end In rust-lang#130600 (comment) I realized that we're likely still passing too-large buffers to the OS, at least once at the end. Previous issues and PRs: * rust-lang#110650 * rust-lang#110655 * rust-lang#118222 r? ChrisDenton
6 participants
Fixes #110650.
Windows file reads have perf overhead that's proportional to the buffer size. When we have a reasonable expectation that we know the file size, we can set a reasonable upper bound for the size of the buffer in one read call.