Hermit: fix `readdir()` - #158649

jounathaen · 2026-07-01T10:24:53Z

On Hermit, the sys_getdents implementation was flawed, as it wasn't stateful. So the caller had to provide a buffer large enough to hold all dirents of the directory in question. Else it returns EINVAL. The current workaround used in Rust is to grow the buffer until it is sufficiently large.

hermit-os/kernel#1738 fixes the behavior of sys_getdents. Hermit now keeps track of the directory position, and each new call to the sys_getdents provides the next entries.

But this results in the current implementation in Rust Std being flawed: Because Hermit now returns a proper readcount, read_dir will only ever iterate over the first 512 bytes of directory entries, even if not all dirent64s are read.

This PR fixes this with a proper implementation of the getdents64 algorithm.

Remark:
This breaks compatibility with Hermit versions 0.11 and previous. We have discussed this in the Hermit team and have come to the conclusion that this is ok. We have released two newer "major" versions since, and Hermit is still in the development phase where we don't guarantee any stability and backwards compatibility in the kernel itself.

This is also how it is done on other platforms.

This optimization was already partially in place, but not used. Other platforms already do this.

rustbot · 2026-07-01T10:24:57Z

Thanks for the pull request, and welcome! The Rust Project is excited to review your changes, and you should hear from @joboet (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: @ChrisDenton, libs
@ChrisDenton, libs expanded to 13 candidates
Random selection from 7 candidates

joboet · 2026-07-02T12:45:27Z

+    // The buffer.
+    buf: Box<[MaybeUninit<u8>]>,


This seems risky, the buffer must have a valid alignment for dirent64 (which has a u64 and a i64 field, and thus needs alignment), but a Box<[MaybeUninit<u8>]> doesn't guarantee any particular alignment.

Sorry that it took me so long to react. This is a very valid point. I've now changed the type to Box<[MaybeUninit<dirent64>]>, which should ensure the correct alignment. This gives a little less flexibility in the sizing of the buffer, but should be fine in this context.

joboet · 2026-07-02T12:47:17Z

-    }
+    let root = path.to_path_buf();
+    let inner = Arc::new(InnerReadDir { root });
+    let buf = GetdentsBuffer::with_capacity(DEFAULT_BUF_SIZE);


Can you guarantee that DEFAULT_BUF_SIZE is enough to hold any direntry? In any case, I'd decouple this definition (or rewrite it to max(DEFAULT_BUF_SIZE, MAX_DIRENTRY_SIZE)) to make sure that changes to DEFAULT_BUF_SIZE don't affect the correctness of ReadDir.

Very good point. I've set it to usize::max(DEFAULT_BUF_SIZE, size_of::<dirent64>()), as the dirent64 already is sized in the hermit-abi to be large enough to hold the max filename size.

rustbot · 2026-07-23T15:53:18Z

Some changes occurred in match checking

cc @Nadrieril

⚠️ #[miri::intrinsic_fallback_is_spec] must only be used if the function actively checks for all UB cases,
and explores the possible non-determinism of the intrinsic.

cc @rust-lang/miri

Some changes occurred to the intrinsics. Make sure the CTFE / Miri interpreter
gets adapted for the changes, if necessary.

cc @rust-lang/miri, @RalfJung, @oli-obk, @lcnr

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Warning

If you are changing how CI LLVM is built or linked, make sure to bump
src/bootstrap/download-ci-llvm-stamp.

cc @jieyouxu

These commits modify the library/Cargo.lock file. Unintentional changes to library/Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred to the CTFE machinery

cc @RalfJung, @oli-obk, @lcnr

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

rustc_codegen_gcc is developed in its own repository. If possible, consider making this change to rust-lang/rustc_codegen_gcc instead.

cc @antoyo, @GuillaumeGomez

Any special-casing of Miri in the standard library requires review.

cc @rust-lang/miri

miri is developed in its own repository. If the Miri part of this change can be broken out, consider making this change to rust-lang/miri instead. However, if Miri needs adjusting for rustc changes, just ignore this message.

cc @rust-lang/miri

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

jounathaen · 2026-07-23T15:56:34Z

Sorry, my rebase has gone wrong. I'm currently fixing this.

Previously, we read all entries into a huge initialized buffer up front, which does not work correctly since on Hermit 0.12, getdents64 behaves more reasonable and does no longer fail on buffers which cannot hold all entries. Additionally, for each new entry, we started searching for the current position from the start instead of just saving the position directly. The new design uses a fixed-size uninitialized buffer that is read into as necessary. Co-authored-by: Martin Kröning <martin.kroening@eonerc.rwth-aachen.de>

While Hermit does not return these yet, it will do so in the future.

rustbot · 2026-07-23T15:59:35Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

jounathaen · 2026-07-23T16:08:55Z

I guess that all the CCs and the labels are a false alarm. My apologies for the noise.

joboet · 2026-07-29T13:46:30Z

Looks reasonable!
@bors r+ rollup

rust-bors · 2026-07-29T13:46:33Z

📌 Commit 0229444 has been approved by joboet

It is now in the queue for this repository.

Hermit: fix `readdir()` On Hermit, the `sys_getdents` implementation was flawed, as it wasn't stateful. So the caller had to provide a buffer large enough to hold all `dirents` of the directory in question. Else it returns `EINVAL`. The current workaround used in Rust is to grow the buffer until it is sufficiently large. hermit-os/kernel#1738 fixes the behavior of `sys_getdents`. Hermit now keeps track of the directory position, and each new call to the `sys_getdents` provides the next entries. But this results in the current implementation in Rust Std being flawed: Because Hermit now returns a proper _readcount_, `read_dir` will only ever iterate over the first 512 bytes of directory entries, even if not all `dirent64`s are read. This PR fixes this with a proper implementation of the `getdents64` algorithm. Remark: This breaks compatibility with Hermit versions 0.11 and previous. We have discussed this in the Hermit team and have come to the conclusion that this is ok. We have released two newer "major" versions since, and Hermit is still in the development phase where we don't guarantee any stability and backwards compatibility in the kernel itself.

Rollup of 17 pull requests Successful merges: - #159014 ([rustdoc] Do not take `doc(cfg())` into account when filtering doctests) - #159130 (a bit optimize four-digit chunks in integer formatting) - #159592 (core: implement bounded random sampling) - #159898 (Add intrinsic-test alias and set sample rate) - #158247 (hermit/fs: Return `unsupported()` instead of `from_raw_os_error(22)`) - #158649 (Hermit: fix `readdir()` ) - #159049 (Avoid ICE in From/TryFrom cast suggestion when encountering HRTBs) - #160053 (test: add test suite for the 85681 issue) - #160087 (Add regression test for nested associated-type projection ICE) - #160090 (rustc_resolve: Further reduce mutability in resolver) - #160099 (Resolver: split module resolutions into local and external resolutions) - #160106 (Add suggestions for `must_implement_one_of`) - #160117 (Remove unnecessary format usage) - #160134 (Work around Wine bug 60084 by calling WSAStartup at most once) - #160142 (bootstrap: remove use-lld config alias) - #160148 (Rename `errors.rs` file to `diagnostics.rs` (15/N)) - #160151 (Mark a doctest as requiring unwinding)

Hermit: fix `readdir()` On Hermit, the `sys_getdents` implementation was flawed, as it wasn't stateful. So the caller had to provide a buffer large enough to hold all `dirents` of the directory in question. Else it returns `EINVAL`. The current workaround used in Rust is to grow the buffer until it is sufficiently large. hermit-os/kernel#1738 fixes the behavior of `sys_getdents`. Hermit now keeps track of the directory position, and each new call to the `sys_getdents` provides the next entries. But this results in the current implementation in Rust Std being flawed: Because Hermit now returns a proper _readcount_, `read_dir` will only ever iterate over the first 512 bytes of directory entries, even if not all `dirent64`s are read. This PR fixes this with a proper implementation of the `getdents64` algorithm. Remark: This breaks compatibility with Hermit versions 0.11 and previous. We have discussed this in the Hermit team and have come to the conclusion that this is ok. We have released two newer "major" versions since, and Hermit is still in the development phase where we don't guarantee any stability and backwards compatibility in the kernel itself.

Rollup of 18 pull requests Successful merges: - #159130 (a bit optimize four-digit chunks in integer formatting) - #159592 (core: implement bounded random sampling) - #159898 (Add intrinsic-test alias and set sample rate) - #158247 (hermit/fs: Return `unsupported()` instead of `from_raw_os_error(22)`) - #158649 (Hermit: fix `readdir()` ) - #159049 (Avoid ICE in From/TryFrom cast suggestion when encountering HRTBs) - #160053 (test: add test suite for the 85681 issue) - #160087 (Add regression test for nested associated-type projection ICE) - #160090 (rustc_resolve: Further reduce mutability in resolver) - #160099 (Resolver: split module resolutions into local and external resolutions) - #160106 (Add suggestions for `must_implement_one_of`) - #160117 (Remove unnecessary format usage) - #160134 (Work around Wine bug 60084 by calling WSAStartup at most once) - #160139 (iter: specialize Take::count using advance_by) - #160142 (bootstrap: remove use-lld config alias) - #160148 (Rename `errors.rs` file to `diagnostics.rs` (15/N)) - #160151 (Mark a doctest as requiring unwinding) - #160166 (Use correct feature gates for `f16`/`f128` `From` impls)

Hermit: fix `readdir()` On Hermit, the `sys_getdents` implementation was flawed, as it wasn't stateful. So the caller had to provide a buffer large enough to hold all `dirents` of the directory in question. Else it returns `EINVAL`. The current workaround used in Rust is to grow the buffer until it is sufficiently large. hermit-os/kernel#1738 fixes the behavior of `sys_getdents`. Hermit now keeps track of the directory position, and each new call to the `sys_getdents` provides the next entries. But this results in the current implementation in Rust Std being flawed: Because Hermit now returns a proper _readcount_, `read_dir` will only ever iterate over the first 512 bytes of directory entries, even if not all `dirent64`s are read. This PR fixes this with a proper implementation of the `getdents64` algorithm. Remark: This breaks compatibility with Hermit versions 0.11 and previous. We have discussed this in the Hermit team and have come to the conclusion that this is ok. We have released two newer "major" versions since, and Hermit is still in the development phase where we don't guarantee any stability and backwards compatibility in the kernel itself.

…uwer Rollup of 18 pull requests Successful merges: - #159898 (Add intrinsic-test alias and set sample rate) - #158247 (hermit/fs: Return `unsupported()` instead of `from_raw_os_error(22)`) - #158649 (Hermit: fix `readdir()` ) - #158693 (Add type-check to offload intrinisc calls) - #159049 (Avoid ICE in From/TryFrom cast suggestion when encountering HRTBs) - #159411 ([rustdoc] Correctly handle output options with --show-coverage) - #160053 (test: add test suite for the 85681 issue) - #160087 (Add regression test for nested associated-type projection ICE) - #160090 (rustc_resolve: Further reduce mutability in resolver) - #160099 (Resolver: split module resolutions into local and external resolutions) - #160106 (Add suggestions for `must_implement_one_of`) - #160117 (Remove unnecessary format usage) - #160134 (Work around Wine bug 60084 by calling WSAStartup at most once) - #160139 (iter: specialize Take::count using advance_by) - #160142 (bootstrap: remove use-lld config alias) - #160148 (Rename `errors.rs` file to `diagnostics.rs` (15/N)) - #160151 (Mark a doctest as requiring unwinding) - #160166 (Use correct feature gates for `f16`/`f128` `From` impls)

Rollup of 20 pull requests Successful merges: - #157669 (cfi: add diag mode support) - #158247 (hermit/fs: Return `unsupported()` instead of `from_raw_os_error(22)`) - #158649 (Hermit: fix `readdir()` ) - #158693 (Add type-check to offload intrinisc calls) - #159049 (Avoid ICE in From/TryFrom cast suggestion when encountering HRTBs) - #159411 ([rustdoc] Correctly handle output options with --show-coverage) - #160053 (test: add test suite for the 85681 issue) - #160087 (Add regression test for nested associated-type projection ICE) - #160090 (rustc_resolve: Further reduce mutability in resolver) - #160099 (Resolver: split module resolutions into local and external resolutions) - #160101 (Add missing `needs-unwind` annotation to `add-spawn-hook-reentrancy-159923` test) - #160106 (Add suggestions for `must_implement_one_of`) - #160117 (Remove unnecessary format usage) - #160134 (Work around Wine bug 60084 by calling WSAStartup at most once) - #160139 (iter: specialize Take::count using advance_by) - #160142 (bootstrap: remove use-lld config alias) - #160148 (Rename `errors.rs` file to `diagnostics.rs` (15/N)) - #160151 (Mark a doctest as requiring unwinding) - #160166 (Use correct feature gates for `f16`/`f128` `From` impls) - #160178 (Remove unused `va_start` intrinsic)

Rollup merge of #158649 - hermit-os:hermit-readdir, r=joboet Hermit: fix `readdir()` On Hermit, the `sys_getdents` implementation was flawed, as it wasn't stateful. So the caller had to provide a buffer large enough to hold all `dirents` of the directory in question. Else it returns `EINVAL`. The current workaround used in Rust is to grow the buffer until it is sufficiently large. hermit-os/kernel#1738 fixes the behavior of `sys_getdents`. Hermit now keeps track of the directory position, and each new call to the `sys_getdents` provides the next entries. But this results in the current implementation in Rust Std being flawed: Because Hermit now returns a proper _readcount_, `read_dir` will only ever iterate over the first 512 bytes of directory entries, even if not all `dirent64`s are read. This PR fixes this with a proper implementation of the `getdents64` algorithm. Remark: This breaks compatibility with Hermit versions 0.11 and previous. We have discussed this in the Hermit team and have come to the conclusion that this is ok. We have released two newer "major" versions since, and Hermit is still in the development phase where we don't guarantee any stability and backwards compatibility in the kernel itself.

mkroening added 4 commits July 1, 2026 11:18

Hermit: Don't cast i32 to i32

3d40189

Hermit: Inline InnerReadDir::new

87900d9

Hermit: Avoid unsoundness around dirent64

149dc77

This is also how it is done on other platforms.

Hermit: Avoid cloning InnerReadDir::root

736fa96

This optimization was already partially in place, but not used. Other platforms already do this.

rustbot assigned joboet Jul 1, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 1, 2026

joboet reviewed Jul 2, 2026

View reviewed changes

jounathaen force-pushed the hermit-readdir branch from a2a9878 to bcf9197 Compare July 23, 2026 15:50

This comment has been minimized.

Sign in to view

jounathaen force-pushed the hermit-readdir branch from bcf9197 to 6856aad Compare July 23, 2026 15:53

This comment has been minimized.

Sign in to view

jounathaen and others added 2 commits July 23, 2026 17:58

Hermit: Filter out . and .. from getdents64

0229444

While Hermit does not return these yet, it will do so in the future.

jounathaen force-pushed the hermit-readdir branch from 6856aad to 0229444 Compare July 23, 2026 15:59

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 29, 2026

JonathanBrouwer mentioned this pull request Jul 29, 2026

Rollup of 7 pull requests #160158

Closed

jhpratt mentioned this pull request Jul 29, 2026

Rollup of 17 pull requests #160168

Closed

jhpratt mentioned this pull request Jul 29, 2026

Rollup of 18 pull requests #160174

Closed

JonathanBrouwer mentioned this pull request Jul 29, 2026

Rollup of 18 pull requests #160182

Closed

jhpratt mentioned this pull request Jul 30, 2026

Rollup of 20 pull requests #160190

Merged

rust-bors Bot merged commit b014a91 into rust-lang:main Jul 30, 2026
13 checks passed

rustbot added this to the 1.99.0 milestone Jul 30, 2026

mkroening deleted the hermit-readdir branch July 30, 2026 11:55

mkroening mentioned this pull request Aug 9, 2026

build(deps): update Rust toolchain to nightly-2026-08-01 hermit-os/hermit-rs#1050

Merged

Navigation Menu

Uh oh!

Uh oh!

Hermit: fix `readdir()` - #158649