Bun.write: poll instead of spin when a nonblocking stdout pipe returns EAGAIN - #35953
Bun.write: poll instead of spin when a nonblocking stdout pipe returns EAGAIN#35953robobun wants to merge 6 commits into
Conversation
When fd 1 is a pipe that has been flipped to O_NONBLOCK (process.stdout.write
does this on the shared open file description via a dup), a Bun.write that
overflows the pipe buffer reaches the async WriteFile path with
could_block=false, and the EAGAIN handler there spun on a cached result
without re-issuing write() or polling: every pool worker pegged a core with
the returned promise never settling.
- do_write: EAGAIN implies the fd is pollable; set could_block and
wait_for_writable instead of looping.
- run_with_fd: derive could_block from the store's mode (stdio stores set
mode but not seekable), and dup a caller-supplied pollable fd so
concurrent WriteFile instances each own a distinct fd number for the
shared IO-thread epoll/kqueue.
- write_{string,bytes}_to_file_fast: only hand off to the async path when
nothing has been written yet; after a partial write, poll(POLLOUT) and
continue so bytes already committed are not re-sent.
|
Updated 1:24 PM PT - Jul 26th, 2026
❌ @autofix-ci[bot], your commit 82c7c82 has 1 failures in
Add 🧪 To try this PR locally: bunx bun-pr 35953That installs a local version of the PR into your bun-35953 --bun |
||||||||||||||||||||||||||||||||||||||||||||||||||||||
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 18 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughChangesPOSIX I/O retry handling
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
test/js/bun/io/bun-write.test.js:762-769— The PR description states "fd count before/after 160 such writes is unchanged", but this test only asserts{length, stderr, exitCode, signalCode}— there is no fd-count check. Sincerun_with_fdnow dups caller-supplied pollable fds and relies onis_allowed_to_close's newopened_fd != fdbranch to release them, consider having the child comparereaddirSync(process.platform === 'darwin' ? '/dev/fd' : '/proc/self/fd').lengthbefore and after thePromise.alland emit the delta on stderr so a regression in that predicate would fail CI.Extended reasoning...
What the gap is
This PR introduces a new resource-acquisition path: when the destination is a caller-supplied pollable fd (e.g.
Bun.stdoutbacked by a pipe),run_with_fdnow callsbun_sys::dup(fd)so each concurrentWriteFileowns a private fd number for the epoll/kqueue interest set. The paired release lives in the newly-extendedis_allowed_to_close:PathOrFileDescriptor::Fd(fd) => { self.opened_fd != Fd::INVALID && self.opened_fd != fd }
The PR description's Verification section states "fd count before/after 160 such writes is unchanged", but the added test at
test/js/bun/io/bun-write.test.js:762-769asserts only{length, stderr, exitCode, signalCode}. There is no fd-count assertion anywhere in the child script or the parent — the claim is a manual verification.Why it matters
REVIEW.md is explicit on both points:
"Verified manually", unnamed "existing tests", and benchmarks don't count, even for one-liners.
Pair every acquisition with its release at the acquisition site … a struct gaining an owning field wires its release into … ALL lifecycle exits … in the same commit.
If the
opened_fd != fdcomparison ever regresses (e.g. someone refactorsis_allowed_to_closeback topathlike.is_path(), oropened_fdgets reset to the store fd beforeon_finish), every largeBun.write(Bun.stdout, ...)to a nonblocking pipe leaks one fd and no test catches it.Step-by-step: what a regression would look like
- Child runs
process.stdout.write("x")→ fd 1's open file description is nowO_NONBLOCK. - Child fires 8 ×
Bun.write(Bun.stdout, large)(≥ 256 KiB each → thread-poolWriteFilepath). - For each,
run_with_fdseescaller_supplied_fd && could_block→bun_sys::dup(1)→opened_fd = 4, 5, 6, …. - Each write completes;
on_finishcallsdo_close(is_allowed_to_close()). - Today:
opened_fd (4) != store fd (1)→true→ dup is closed. ✅ - Regressed: predicate returns
false→ dup is never closed → fd 4, 5, 6, … leak. - Test still passes:
stdout.length,stderr,exitCode,signalCodeare all unchanged by an fd leak.
Suggested fix
Have the child sample the fd table around the
Promise.alland emit it on stderr (harness already hasgetFDCount(), but the child is a-escript so inline it):const fdDir = process.platform === 'darwin' ? '/dev/fd' : '/proc/self/fd'; const before = require('fs').readdirSync(fdDir).length; const wrote = await Promise.all(ps); const after = require('fs').readdirSync(fdDir).length; process.stderr.write("wrote=" + wrote.reduce((a, b) => a + b, 0) + " fdDelta=" + (after - before));
Then assert
stderr: "wrote=" + (expected - 1) + " fdDelta=0"in the parent. This backs the PR description's claim with an automated assertion and guards the new acquisition/release pair going forward.Severity
nit — the primary behavioral change (spin-wedge → poll → completes) is covered by an automated test that fails on the unfixed build with
SIGKILL. This is a coverage-hardening ask for the secondary dup lifecycle; the release logic is straightforward and correct as written. - Child runs
-
🟡
src/runtime/webcore/blob/write_file.rs:480-491— The dup-for-polling guardif self.could_block && caller_supplied_fdis evaluated usingcould_block = file.mode != 0 && ..., butBun.file(rawFd)for a non-stdio fd never populatesmode(it stays 0 viaFile::init's..Default::default()), so the dup is skipped. Thendo_writeflipscould_block = trueon the first EAGAIN and callswait_for_writable()on the original caller-supplied fd — two concurrentBun.write(Bun.file(pipeFd), ...)calls hit the exact EEXIST/udata-clobber collision the dup was added to prevent. Consider dupping unconditionally whencaller_supplied_fd, or dupping lazily indo_writebefore the firstwait_for_writable()whenopened_fdstill equals the store fd.Extended reasoning...
What the bug is
The PR adds a
dup()inrun_with_fdso that eachWriteFiletargeting a caller-supplied pollable fd owns a private fd number for epoll/kqueue registration. The guard is:if self.could_block && caller_supplied_fd { match bun_sys::dup(fd) { ... self.opened_fd = duped ... } }
where
could_blockwas just computed asfile.mode != 0 && !bun_sys::is_regular_file(file.mode). This works forBun.stdout/Bun.stderrbecause their stores are constructed via the stdio ctor whichfstat()s up front and populatesmode. But forBun.file(rawFd)whererawFd > 2,modeis never populated.The code path
Bun.file(fd)forfd > 2→find_or_create_file_from_path(Blob.rs:3754-3776) —fd.stdio_tag()isNone, so it falls through toStore::init_file(PathOrFileDescriptor::Fd(fd), None)→File::init(webcore_types.rs:788):pub fn init(pathlike: PathOrFileDescriptor, mime_type: Option<MimeType>) -> File { File { pathlike, mime_type: ..., ..Default::default() } // mode: 0 }
Nothing along the
write_file_internal→WriteFile::run_with_fdpath stats the destination, somodestays 0 when the guard executes.Step-by-step
Take two concurrent
Bun.write(Bun.file(pipeFd), Buffer.alloc(256*1024))calls wherepipeFdis the write end of anO_NONBLOCKpipe (fd 7, say):- Both reach
run_with_fd.caller_supplied_fd = true,file.mode == 0→could_block = (0 != 0 && ...) = false. - The dup guard
if false && truefails;opened_fdstays 7 for both. - Both call
do_write_loop→do_write→sys::write(7, ...). The first write partially succeeds; a subsequent one returnsEAGAIN. do_write's EAGAIN arm now setsself.could_block = trueand callsself.wait_for_writable(), which scheduleson_request_writable→io::Action::Writable { fd: 7, poll: &mut self.io_poll, ... }.- Both
WriteFileinstances register fd 7 with the IO thread's epoll set. On Linux the secondEPOLL_CTL_ADDreturnsEEXIST(surfaces as a rejected promise); on kqueue the second registration overwrites the first'sudata, so the losing instance's poll never fires and its promise never settles.
This is exactly bug (3) from the PR description, just for a store whose
modewas never populated.Why existing code doesn't prevent it
The dup is gated on the initial value of
could_block, butdo_writecan flipcould_blocktotrueafterrun_with_fdhas already decided not to dup.is_allowed_to_closeis safe here (it returnsfalsesinceopened_fd == store fd, so the caller's fd is not accidentally closed), but that just means the fd isn't leaked — it doesn't help with the epoll collision.Impact and severity
This is not a regression: before this PR, the same scenario hit the infinite-spin bug (the old
EAGAIN && !could_block => continuere-matched a cached result forever), which is strictly worse. After this PR, single writes toBun.file(nonStdioPipeFd)work where they didn't before, and only the concurrent case can still fail (with a clean error on Linux rather than a wedge). The trigger — a user-owned nonblocking pipe/socket fd > 2 with concurrent ≥256 KiBBun.writecalls — is narrow. Marking as a nit, but per REVIEW.md's "fix the whole class in the same PR" rule the sibling case is worth closing here since the fix is trivial.Fix
Either:
- dup unconditionally when
caller_supplied_fd(an extradupon a regular-file fd is cheap and never registered anyway), or - dup lazily in
do_writeright before the firstwait_for_writable()whenself.opened_fdstill equals the store's fd.
- Both reach
…adFile::do_read; assert fdDelta in test
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/runtime/webcore/Blob.rs`:
- Around line 5445-5461: The EAGAIN handling in write_string_to_file_fast at
src/runtime/webcore/Blob.rs#L5445-L5461 and write_bytes_to_file_fast at
src/runtime/webcore/Blob.rs#L5531-L5545 must not block with poll(..., -1) after
partial writes. Remove the synchronous POLLOUT polling and route any EAGAIN,
including after partial progress, through the existing async path so the
main-thread Bun.write() fast path yields instead of blocking.
In `@src/runtime/webcore/blob/write_file.rs`:
- Around line 447-477: Update the caller_supplied_fd handling in run_with_fd so
the private dup occurs only when the fd-backed blob has could_block true;
regular-file descriptors must retain the existing non-polling path without
dup/close overhead. Verify that file.mode is populated from the earlier fstat
for fd-backed blobs before this logic runs, and preserve do_write’s RETRY
behavior when mode remains unknown.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 924d5b67-6b70-4328-92f5-16e910d967b3
📒 Files selected for processing (4)
src/runtime/webcore/Blob.rssrc/runtime/webcore/blob/read_file.rssrc/runtime/webcore/blob/write_file.rstest/js/bun/io/bun-write.test.js
There was a problem hiding this comment.
Beyond the inline timeout nit: the fast-path poll(POLLOUT, -1) after a partial write (coderabbitai's unresolved comment on Blob.rs:5461) was examined — falling through to needs_async there would re-send bytes already committed, and the payload is <256 KiB by construction, so the bounded block is the correct tradeoff here. The unconditional dup on caller-supplied fds (coderabbitai's other comment) is deliberate per the earlier thread — Bun.file(rawFd) stores have mode == 0, so gating on could_block would leave those un-duped and reintroduce the epoll EEXIST collision.
Extended reasoning...
The two unresolved coderabbitai comments overlap with concerns already worked through on this PR. The poll(-1)-blocks-main-thread finding was raised by a finder and refuted by verifiers: the fast path only reaches that branch after a partial write on a <256 KiB payload, and the alternative (setting needs_async and returning) hands the full original buffer to WriteFile, duplicating whatever was already written — that's the bug the branch exists to avoid. The unconditional-dup-for-regular-file-fds observation is the intentional flip side of the fix I asked for earlier (dup regardless of the upfront could_block guess so mode==0 stores are covered); a dup+close per Bun.write to a raw fd is cheap relative to the write itself. Leaving approval to a human given this reshapes fd ownership and is_allowed_to_close semantics on a hot I/O path.
|
CI status: build #82571 finished with 195/196 lanes passing. The only failure is the The flaky-retry annotation lists Ready for review; the binary-size baseline needs a refresh independent of this change. |
|
Heads-up for the rebase: #37128 routes import { openSync, readSync, constants } from "node:fs";
// fifo created with mkfifo beforehand
const rfd = openSync(fifo, constants.O_RDONLY | constants.O_NONBLOCK);
const wfd = openSync(fifo, constants.O_WRONLY | constants.O_NONBLOCK);
const done = Bun.write(Bun.file(wfd), Buffer.alloc(1024 * 1024, "A"));
// drain rfd from a setInterval; `done` never resolves, one pool thread sits at 100% CPU,
// and the reader only ever receives the first partial write (8192 bytes here).Closed #35956 today as superseded by #37128; its |
2 participants
What's wrong
Once
process.stdout.writehas run (which dups fd 1 and setsO_NONBLOCKon the dup;O_NONBLOCKlives on the shared open file description, so fd 1 itself becomes nonblocking), anyBun.write(Bun.stdout, ...)that overflows the pipe buffer wedges the process:strace shows all pool threads running with zero syscalls, main parked in
epoll_wait:Cause
Three bugs chained:
WriteFile::run_with_fdderivedcould_blockfrom the file store'sseekable, but the stdio stores (Bun.stdout/Bun.stderr) only setmode, socould_blockstayedfalsefor a pipe.WriteFile::do_writeissuedsys::write()once before itsloop { match result { ... } }, and theEAGAIN && !could_block => continuearm re-matched that same cached result forever without re-issuing the syscall: a busy spin with zero forward progress.ReadFile::do_readhad the identical stale-result loop.WriteFiletried to register fd 1 with the shared IO-thread epoll; the secondEPOLL_CTL_ADDreturnedEEXIST.Separately, the synchronous fast path's
needs_asyncfallback re-sent the entire payload after a partial write, duplicating whatever had already been committed. This was masked by (2) because the async path never made progress.Fix
src/runtime/webcore/blob/write_file.rs:do_write:write(2)on a regular file never returnsEAGAIN, so reaching the retry arm means the fd is pollable regardless of the initial guess. Setcould_block = trueandwait_for_writable(); drop the stale-result loop.run_with_fd: derivecould_blockfrommode != 0 && !is_regular_file(mode). When the destination is a caller-supplied fd, dup it so thisWriteFileowns a private fd number for the same open file description (coversBun.file(rawFd)whose store hasmode == 0as well as stdio); the dup is closed viais_allowed_to_closeon finish.src/runtime/webcore/blob/read_file.rs: apply the same drop-the-loop reshaping todo_read.src/runtime/webcore/Blob.rs,write_{string,bytes}_to_file_fast: only setneeds_asyncwhen nothing has been written yet. After a partial write,poll(POLLOUT)and continue the write loop so bytes already committed are not re-sent.Verification
New test in
test/js/bun/io/bun-write.test.jsspawns a child that touchesprocess.stdout.writeand then fires two rounds of 16 × 64 KiB (< 256 KiB, exercises the fast-pathEAGAINfallback) plus 4 × 256 KiB (>= 256 KiB, goes straight to the thread-pool path)Bun.write(Bun.stdout, ...)calls into a pipe. Without this change the child is killed by the spawn timeout with every pool worker spinning (exitCode: 137, signalCode: "SIGKILL"); with it the child exits 0, all promises resolve with their full byte counts, the parent receives exactly1 + 2 × (16·64 KiB + 4·256 KiB)bytes, and the fd count sampled around the second round is unchanged (guards the dup-close pairing).Windows is unaffected (the async write path there is libuv's
uv_fs_write).Related: #35949 keeps stdio writes on the synchronous fast path when the destination is a regular file (ordering), and #32890 stops
Bun.writefromftruncateing an fd destination for an empty source; this change covers the pipe-destination wedge. The FileSink'sO_NONBLOCKon an inherited-stdio dup, and a single sequential owner for fd 1/2 acrossBun.write/console.log/process.stdout, remain as separate follow-ups.no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/io/bun-write.test.js