std: use `readdir` on nearly all UNIX platforms by joboet · Pull Request #158727 · rust-lang/rust

joboet · 2026-07-03T10:49:20Z

POSIX.1-2024 formalised what was already guaranteed by a lot of implementations and required readdir to be thread-safe as long as an individual DIR* is not accessed concurrently (which ReadDir::next ensures by taking a mutable reference). But since our read_dir implementation predates that standard, we currently only utilise readdir on the platforms that guarantee thread-safety in their documentation. On other implementations – notably macOS – we use the readdir_r function, which was always required to be thread-safe but is problematic because it cannot handle directory entries with names longer than NAME_MAX.

However, even the first POSIX issue, POSIX.1-1994, specified that the data in the returned dirent

is not overwritten by another call to readdir() on a different directory stream.

and that guarantee together with the requirement that the underlying syscalls need to be thread-safe already because of readdir_r result in readdir being thread-safe on nearly all implementations, even if they predate POSIX.1-2024. Given the now formalised guarantee I think it safe to assume that currently thread-safe implementations will not be changed in a way that violates thread-safety.

CC T-libs, do you agree?
@rustbot label +I-libs-nominated

I thus looked at the readdir implementation of all the UNIXes currently utilising readdir_r to check for thread-safety:

On the implementations where I couldn't confirm thread-safety ReadDir will still use readdir_r, but I've changed the code so that this edge-case is limited to ReadDir::next. All other platforms now use readdir.

rustbot · 2026-07-03T10:49:23Z

miri is developed in its own repository. If possible, consider making this change to rust-lang/miri instead.

cc @rust-lang/miri

rustbot · 2026-07-03T10:49:25Z

r? @JohnTitor

rustbot has assigned @JohnTitor.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: @ChrisDenton, libs
@ChrisDenton, libs expanded to 13 candidates
Random selection from 7 candidates

jonathanpallant · 2026-07-03T11:05:11Z

@joboet - you've linked to VxWorks docs under the QNX entry?

https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.lib_ref/topic/r/readdir.html says:

The readdir() function is safe to use in multithreaded environments as long as the directory stream, DIR *dirp, isn't shared between threads.

https://www.qnx.com/developers/docs/7.1/com.qnx.doc.neutrino.lib_ref/topic/r/readdir.html says the same thing

joboet · 2026-07-03T11:28:35Z

Whoops, I got the RTOSs confused 😉 ... thanks for the link!

RalfJung

Miri changes LGTM.

At some point we can probably remove readdir_r from macOS then given that it is generally considered a deprecated API, the only reason we support it is because std used it.

View changes since this review

ivmarkov · 2026-07-07T20:26:06Z

ESP-IDF: readdir is not thread-safe (at least on FAT) due to caching file metadata in the VFS context without locks. opendir does the same thing, too?! And the cache is never invalidated, even when the file is deleted?! Honestly, this is just broken... CC @ivmarkov @MabezDev @SergioGasquez

@joboet I completely agree with your analysis that FAT readdir in ESP-IDF V5.4.0+ is completely broken (LittleFS and Spiffs seem OK).

I've filed the issue upstream.
In the meantime, and as per the PR, ESP-IDF should stay on readdir_r.

Thanks a lot for spotting this!

EDIT: I haven't put an effort to actually reproduce it on real hardware yet; only static analysis (confirmed by Fable though). So still a slim chance we might be missing something. Very slim. :-)

nia-e · 2026-07-08T16:15:37Z

Per today's t-libs meeting, we like this, +1. The possibility was also raised for those other platforms of invoking the underlying syscall directly if they're stable, but that's probably a question for a different PR. Thanks ^^

RalfJung · 2026-08-02T19:03:52Z

@JohnTitor reminder that this is waiting for review :)

JohnTitor

Sorry for the delay!
Makes sense to me, thanks for the detailed description!
r=me once the conflict is resolved

View changes since this review

Co-authored-by: Ralf Jung <post@ralfj.de>

rustbot · 2026-08-03T12:03:00Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

JohnTitor · 2026-08-03T20:50:28Z

@bors r+

rust-bors · 2026-08-03T20:50:31Z

📌 Commit 7f43ef4 has been approved by JohnTitor

It is now in the queue for this repository.

@rustbot

std: use `readdir` on nearly all UNIX platforms POSIX.1-2024 [formalised](https://pubs.opengroup.org/onlinepubs/9799919799/functions/readdir.html) what was already guaranteed by a lot of implementations and required `readdir` to be thread-safe as long as an individual `DIR*` is not accessed concurrently (which `ReadDir::next` ensures by taking a mutable reference). But since our `read_dir` implementation predates that standard, we currently only utilise `readdir` on the platforms that guarantee thread-safety in their documentation. On other implementations – notably macOS – we use the `readdir_r` function, which was always required to be thread-safe but is problematic because it cannot handle directory entries with names longer than `NAME_MAX`. However, even the first POSIX issue, POSIX.1-1994, specified that the data in the returned `dirent` > is not overwritten by another call to readdir() on a different directory stream. and that guarantee together with the requirement that the underlying syscalls need to be thread-safe already because of `readdir_r` result in `readdir` being thread-safe on nearly all implementations, even if they predate POSIX.1-2024. Given the now formalised guarantee I think it safe to assume that currently thread-safe implementations will not be changed in a way that violates thread-safety. CC T-libs, do you agree? @rustbot label +I-libs-nominated I thus looked at the `readdir` implementation of all the UNIXes currently utilising `readdir_r` to check for thread-safety: - [x] [Apple platforms](https://github.com/apple-oss-distributions/Libc/blob/8a5571058ea8cf099279d8df1602958327b1d400/gen.subproj/readdir.c), starting with Mac OS X 10.0. While the original implementation did not use any synchronisation, it only modifies data of the `DIR*` it acts on. - [x] [Cygwin](https://github.com/cygwin/cygwin/blob/854594be786023d7e92c9352d8f54e64f9fb36eb/winsup/cygwin/dir.cc): same thing. - [x] [Dragonfly](https://github.com/DragonFlyBSD/DragonFlyBSD/blob/2e3a87ce41bfdf759c1936e31cd8cb7be0a9c1cc/lib/libc/gen/readdir.c) uses locks. - [x] [L4Re](https://github.com/kernkonzept/l4re-core/blob/b6f7495bb8b010c6d0f2d44a25c61885a6ac7ac1/libc/musl/contrib/musl/src/dirent/readdir.c): only modifies the passed `DIR*`. - [ ] LynxOS: closed-source, the [documentation](https://www-f9.ijs.si/~rok/detectors/doc/LynxOS-2.5.0/LynxOS_Documentation.html) says that `readdir` is not reentrant. CC @rfatykhov-lynx - [x] [Managarm](https://github.com/managarm/mlibc/blob/368a00fa3ab482a76fbc2fb04afc188c6ff2407b/options/posix/generic/dirent.cpp): only modifies the passed `DIR*`. - [x] [NetBSD](https://github.com/NetBSD/src/blob/3604f3d8178e005847b7c7a1c4e4fa20838cecb8/lib/libc/gen/readdir.c): either uses locks depending on configuration, or only modifies the passed `DIR* `. - [x] [OpenBSD](https://github.com/openbsd/src/blob/394336142320c7c38e2361c4293df4c86626668e/lib/libc/gen/readdir.c): uses locks. - [x] [Unikraft](https://github.com/unikraft/unikraft/blob/be744898b6947824e367e01765703401e08ce3c5/lib/nolibc/musl-imported/src/dirent/readdir.c) - [ ] VxWorks: closed-source, the [documentation](https://archive.org/details/manualzilla-id-5786163) does not make any reference to thread-safety. CC @biabbas @hax0kartik - [x] QNX: the [documentation](https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.lib_ref/topic/r/readdir.html) gives the same guarantee as POSIX.1-2024. - [x] [NuttX](https://github.com/apache/nuttx/blob/9a4114a9d3e06e47cab2c15e00b2b29c17c10b10/libs/libc/dirent/lib_readdir.c): only modifies the passed `DIR*`. - [x] 3DS: the underlying [directory read implementation in libctru](https://github.com/devkitPro/libctru/blob/36fe1ada5b7ebe53ba4decda36d764a55f8fefb6/libctru/source/services/fs.c#L1835-L1850) is thread-safe - [ ] RTEMS: I'm unable to find the implementation. CC @thesummer - [ ] QuRT: CC @androm3da - [ ] ESP-IDF: `readdir` is *not* thread-safe (at least on FAT) due to [caching file metadata in the VFS context without locks](https://github.com/espressif/esp-idf/blob/fa8039b5cadb6e85dd830ff8c2c4bd73b6538aee/components/fatfs/vfs/vfs_fat.c#L976). [`opendir`](https://github.com/espressif/esp-idf/blob/fa8039b5cadb6e85dd830ff8c2c4bd73b6538aee/components/fatfs/vfs/vfs_fat.c#L924) does the same thing, too?! And the cache is never invalidated, even when the file is deleted?! Honestly, this is just broken... CC @ivmarkov @MabezDev @SergioGasquez - [x] [Emscripten](https://github.com/emscripten-core/emscripten/blob/f351c42755ad56323f0bb3b2195ca50e35650496/system/lib/libc/musl/src/dirent/readdir.c): derived from musl, only modifies the current `DIR*`. On the implementations where I couldn't confirm thread-safety `ReadDir` will still use `readdir_r`, but I've changed the code so that this edge-case is limited to `ReadDir::next`. All other platforms now use `readdir`.

…uwer Rollup of 26 pull requests Successful merges: - #153749 (Account for ownership mismatch on argument that doesn't meet bound) - #158727 (std: use `readdir` on nearly all UNIX platforms) - #159130 (a bit optimize four-digit chunks in integer formatting) - #159326 (Deny multiple EII impls on a single item) - #159535 (Optimize slice::contains for bytewise types) - #159595 (Promote loongarch32-unknown-none* to Tier 2) - #160184 (Add -Zinstrument-mcount={fentry-nop-record,fentry-record}) - #160320 (point at trait definition when it is used as a derive macro) - #160369 (When suggesting method names, prefer *exact* doc aliases over similar names) - #160406 (`DepKind` cleanups) - #160424 (Use `thread::available_parallelism` as the default limit for backend parallelism) - #159303 (Fix ICE for direct inline const generic defaults) - #159977 (Add regression test for bool indexing codegen) - #160011 (remove InterpError::map_err_info) - #160165 (reject `...` without pattern post-expansion) - #160295 (Fix rustdoc ICE when checking if a generic arg can be elided) - #160305 (Linkify C-SKY targets in `platform-support.md`) - #160314 (fix borrowck ICE for consts with fn pointer type) - #160322 (ElaborateBoxDeref: remove unnecessary projection) - #160338 (Add regression test for supertrait associated type normalization through dyn) - #160340 (Add regression test for unused_parens on contract clauses) - #160371 (Add doc aliases for transpositions `read_exact_buf` and `read_exact_buf_at`) - #160384 (Add PR body notes for Cargo lock file maintenance) - #160412 (Move duplicate-names check for #[rustc_must_implement_one_of] to attribute parser) - #160435 (bump tracing-tree) - #160449 (Fix lookup of object files)

Zalathar · 2026-08-04T03:32:16Z

Possibly failed in rollup? #160469 (comment)

@bors r-
@bors try jobs=x86_64-msvc-ext2

rust-bors · 2026-08-04T03:32:22Z

This pull request was unapproved.

This PR was contained in a rollup (#160469), which was unapproved.

View changes since this unapproval

std: use `readdir` on nearly all UNIX platforms try-job: x86_64-msvc-ext2

rust-bors · 2026-08-04T05:10:37Z

💔 Test for 8471650 failed: CI. Failed job:

try - x86_64-msvc-ext2 (web logs, enhanced plaintext logs)

RalfJung · 2026-08-04T05:52:00Z

Apparently Miri tests are failing on x86_64-apple-darwin. PR CI passed, meaning aarch64-apple-darwin seems okay... I did not realize that those targets sometimes use different symbols.

joboet · 2026-08-04T14:12:10Z

Apparently Miri tests are failing on x86_64-apple-darwin. PR CI passed, meaning aarch64-apple-darwin seems okay... I did not realize that those targets sometimes use different symbols.

Oh yeah, I forgot about those... the $INODE64 suffix is used for legacy reasons on all non-AArch64 kernels.

joboet · 2026-08-04T14:13:16Z

@bors r=@JohnTitor

rust-bors · 2026-08-04T14:13:19Z

📌 Commit 8b2acf4 has been approved by JohnTitor

It is now in the queue for this repository.

🌲 The tree is currently closed for pull requests below priority 10. This pull request will be tested once the tree is reopened.

Reason for tree closure: manually handling queue due to backlog

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 3, 2026

rustbot assigned JohnTitor Jul 3, 2026

joboet force-pushed the more_readdir branch from 5ca7084 to 127a08c Compare July 3, 2026 10:58

This comment has been minimized.

Sign in to view

joboet force-pushed the more_readdir branch from 127a08c to 8eca20a Compare July 3, 2026 11:21

RalfJung approved these changes Jul 3, 2026

View reviewed changes

Comment thread src/tools/miri/tests/pass-dep/libc/libc-fs.rs

joboet force-pushed the more_readdir branch from 20bea8c to 308f79b Compare July 3, 2026 11:48

rustbot added the I-libs-nominated Nominated for discussion during a libs team meeting. label Jul 3, 2026

ivmarkov mentioned this pull request Jul 7, 2026

#10220's fix (e17da6f9f9, v5.4.0+) is deeply broken (IDFGH-17926) espressif/esp-idf#18802

Closed

3 tasks

nia-e removed the I-libs-nominated Nominated for discussion during a libs team meeting. label Jul 8, 2026

This comment has been minimized.

Sign in to view

joboet force-pushed the more_readdir branch from 308f79b to c8f509c Compare July 9, 2026 11:36

This comment has been minimized.

Sign in to view

JohnTitor approved these changes Aug 3, 2026

View reviewed changes

JohnTitor added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 3, 2026

joboet and others added 3 commits August 3, 2026 13:57

std: use readdir on nearly all UNIX platforms

c9ad1eb

miri: allow readdir on macOS

0d9b392

miri: explain why readdir_r is only tested on macOS

7f43ef4

Co-authored-by: Ralf Jung <post@ralfj.de>

joboet force-pushed the more_readdir branch from c8f509c to 7f43ef4 Compare August 3, 2026 12:02

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 3, 2026

JonathanBrouwer mentioned this pull request Aug 3, 2026

Rollup of 26 pull requests #160469

Closed

rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 4, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Aug 4, 2026

Auto merge of #158727 - joboet:more_readdir, r=<try>

8471650

std: use `readdir` on nearly all UNIX platforms try-job: x86_64-msvc-ext2

This comment has been minimized.

Sign in to view

miri: support the $INODE64 version of readdir used on non-AArch64

8b2acf4

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 4, 2026

Uh oh!

Uh oh!

Conversation

joboet commented Jul 3, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jul 3, 2026

Uh oh!

rustbot commented Jul 3, 2026

Uh oh!

This comment has been minimized.

jonathanpallant commented Jul 3, 2026

Uh oh!

joboet commented Jul 3, 2026

Uh oh!

RalfJung left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ivmarkov commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nia-e commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

RalfJung commented Aug 2, 2026

Uh oh!

JohnTitor left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rustbot commented Aug 3, 2026

Uh oh!

JohnTitor commented Aug 3, 2026

Uh oh!

rust-bors Bot commented Aug 3, 2026

Uh oh!

Zalathar commented Aug 4, 2026

Uh oh!

rust-bors Bot commented Aug 4, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Aug 4, 2026

Uh oh!

This comment has been minimized.

RalfJung commented Aug 4, 2026

Uh oh!

joboet commented Aug 4, 2026

Uh oh!

joboet commented Aug 4, 2026

Uh oh!

rust-bors Bot commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

joboet commented Jul 3, 2026 •

edited by rustbot

Loading

RalfJung left a comment •

edited by rustbot

Loading

ivmarkov commented Jul 7, 2026 •

edited

Loading

nia-e commented Jul 8, 2026 •

edited

Loading

JohnTitor left a comment •

edited by rustbot

Loading