console/worker_threads: keep fd 1/2 blocking across worker start; poll on EAGAIN in the console writer - #36066
console/worker_threads: keep fd 1/2 blocking across worker start; poll on EAGAIN in the console writer#36066robobun wants to merge 6 commits into
Conversation
…l on EAGAIN in the console writer
Starting a node:worker_threads Worker was silently making the main
thread's console.log lossy on a pipe: the worker's stdio rebind called
Object.defineProperty(process, "stdout"/"stderr", {...}), and JSC's
defineOwnProperty reifies a static PropertyCallback slot before
replacing it. Reification ran the fd-backed stream constructor
(Bun.file(1).writer()), which dup()s fd 1 and sets O_NONBLOCK on the
dup. O_NONBLOCK lives on the open file description, which the dup
shares with the process-wide fd 1, so every thread's fd 1 was now
nonblocking for the rest of the process. The main thread's native
console writer (fd_write_all_quiet) treated the resulting EAGAIN as a
terminal error and discarded the unwritten tail, so a burst into a
slow pipe reader dropped most of its lines with exit 0.
Three independent fixes, each closing one seam of that chain:
- worker_threads: rebind stdio with plain assignment instead of
Object.defineProperty. put() replaces a PropertyCallback slot without
reifying it and yields the same {writable,enumerable,configurable}
descriptor, so the fd-backed constructor is never run in a worker.
- sys: fd_write_all_quiet polls for POLLOUT on EAGAIN instead of giving
up. Anything sharing the open file description (a worker, a parent
shell, libuv in a co-process) can flip O_NONBLOCK at any time;
blocking on writability is what Node's writer does.
- FileSink: record the fd returned by open_for_writing in self.fd.
setup() handed the fd to the writer but left self.fd at INVALID, so
Bun__ForceFileSinkToBeSynchronousForProcessObjectStdio's
update_nonblocking(self.fd, false) was a no-op and get_fd() returned
-1 on the Bun.file(fd).writer() path.
WalkthroughChangesWorker stdio rebinding now uses assignments, FileSink records started descriptors, Unix writes retry after readiness polling, and POSIX tests cover blocking state and complete pipe output. stdio write handling
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Found 2 issues this PR may fix:
🤖 Generated with Claude Code |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
Avoids leaving a closed fd number in self.fd when start()/start_sync() fails (the error arms close the fd and return). The sink is deref'd immediately on that path today so it was not observable, but this keeps the pre-existing invariant that self.fd is INVALID whenever setup() returns Err.
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/js/node/worker_threads.ts`:
- Around line 449-466: Update the stdio replacement logic around process.stdout,
process.stderr, and process.stdin to use Object.defineProperty with
Node-compatible writable descriptors or setters, rather than plain assignment.
Ensure each worker-local stream actually replaces the cache-backed getter,
including the port-backed and immediately-EOF stdin paths.
In `@src/sys/lib.rs`:
- Around line 9414-9424: Update the write error handling around the #[cfg(unix)]
Err(e) if e.is_retry() arm so EINTR is treated as retryable before entering the
EAGAIN polling path. Ensure an interrupted write retries the write operation and
preserves the unwritten tail instead of reaching the final Err(_) => return
false path, while retaining existing polling behavior for EAGAIN.
In `@test/js/node/process/process-stdio.test.ts`:
- Around line 168-172: Restrict the libcPathForDlopen() guard in the
“stdout/stderr vs O_NONBLOCK on a pipe” describe block to the platforms
supported by that helper, Linux and Darwin, rather than using isPosix. Preserve
the empty-string fallback for unsupported platforms so the skipped tests do not
evaluate the throwing lookup.
- Around line 268-274: Update the process stdio test’s stderr-reading logic to
accumulate chunks until a newline-delimited byte count is complete, safely
handle done reads, and parse only the framed header. Preserve any bytes read
after the header by initializing stderrRest with the remaining data (including
rest.join("\n")). Move closeSync(w) into the existing finally cleanup so it also
runs when spawn fails, using the established guarded close pattern.
- Around line 176-179: Replace the fixed three-argument fcntl binding in the
dlopen setup and the nonblock helper with a non-variadic binding that matches
the exact F_SETFL ABI, using an FFIType.i64 flag argument where supported, or
route the flag update through a non-variadic syscall binding. Ensure the
existing F_GETFL/F_SETFL flow changes only O_NONBLOCK reliably across supported
architectures.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: ed47fc71-0c59-48cd-b24d-6bd9567a46de
📒 Files selected for processing (4)
src/js/node/worker_threads.tssrc/runtime/webcore/FileSink.rssrc/sys/lib.rstest/js/node/process/process-stdio.test.ts
… EINTR retry on macOS, gate on linux|darwin, frame stderr header, close w on spawn failure
…c symbol-map ASAN leak pipe(2) is not variadic so a fixed-arg dlopen binding is ABI-correct on every target; only the spawned children need the cc-compiled fcntl wrapper (and they exit, so ASAN does not report their allocations).
There was a problem hiding this comment.
No new issues found; the earlier variadic-fcntl and isPosix/FreeBSD concerns are addressed in the current diff. Deferring to a human given this touches the console output hot path, FileSink fd tracking, and worker stdio bootstrap together, with noted overlap against #35956/#33560.
What was reviewed:
fd_write_all_quiet: EINTR now retried before the EAGAIN→posix::pollarm;posix::pollitself is EINTR-retried, andpoll(-1)returns POLLERR/POLLHUP on a dead reader so the next write hits the terminalErr(_)path rather than hanging.FileSink::setupnow recordsself.fdon both the Windowsstart_syncand the POSIXstartsuccess arms;self.fdis read-only elsewhere (no double-close introduced).- Test:
cc-compiled non-variadicfcntlwrappers replace the fixed-arg dlopen binding;libcPathForDlopen()is now only reached inside a test callback underisLinux || isMacOS;tempDirWithFilesat describe-body level is Windows-safe; stderr header is framed to\nandwis closed infinally.
Extended reasoning...
Overview
Four files: src/js/node/worker_threads.ts (swap Object.defineProperty for plain assignment in setupWorkerStdio so the lazy fd-backed process.stdout/stderr/stdin PropertyCallback is not reified), src/runtime/webcore/FileSink.rs (record the dup'd/opened fd in self.fd after writer.start()/start_sync() succeed), src/sys/lib.rs (fd_write_all_quiet retries EINTR and polls POLLOUT on EAGAIN instead of dropping the tail), and three new POSIX-gated tests in test/js/node/process/process-stdio.test.ts.
Security risks
None identified. No new user-controlled input reaches a parser or filesystem path; the FFI usage is test-only in a spawned child. The runtime change makes console.log block on a full nonblocking pipe, which is the Node-matching behavior and not an availability regression relative to silently dropping output.
Level of scrutiny
High. fd_write_all_quiet backs the native console/output writer and the crash reporter; changing its error handling to an unbounded poll(-1) is correct for the stdio-pipe case but is a semantic shift on a very hot path. The FileSink.rs self.fd.set(fd) line overlaps a sibling PR (#35956) and enables a previously-dead update_nonblocking branch. The worker_threads change relies on a JSC-specific detail (put() vs defineOwnProperty on a static PropertyCallback slot) that was empirically verified in-thread but is subtle enough to warrant a maintainer glance.
Other factors
All prior review threads (CodeRabbit and my earlier inline findings on the Apple-arm64 variadic fcntl ABI and the FreeBSD libcPathForDlopen() gate) are resolved in the current revision: the tests now compile non-variadic C wrappers via bun:ffi cc, gate on isLinux || isMacOS, frame the stderr header, and close the write-end fd in finally. The PR's own evidence footer notes the platform-specific tests were not run locally and are deferred to CI, and the description flags overlap with #35956 and #33560 — both reasons a human should sign off on landing order.
|
Updated 2:30 AM PT - Jul 27th, 2026
✅ @robobun, your commit f49470f0d8b079e3645fd0a3a8e0fb056b6f204d passed in 🧪 To try this PR locally: bunx bun-pr 36066That installs a local version of the PR into your bun-36066 --bun |
|
Closing: #37128 lists this PR as superseded and covers all three seams here. worker_threads assigns process.stdout/stderr instead of using defineProperty, the console writer polls on EAGAIN instead of dropping, and the stdio force-sync / O_NONBLOCK handling is replaced outright. Its process-stdio tests include the same three cases (fd flags after materialising process.stdout, console.log into a full O_NONBLOCK pipe, idle worker), and the idle-worker repro from this PR leaves fd 1 and 2 blocking on its current head. |
Repro
The worker never writes a byte. Its mere creation flips the process-wide fd 1 to
O_NONBLOCK, after which the main thread's nativeconsole.logsilently drops onEAGAINwhen piped to a slow reader (| tee,| grep, CI log collectors behind any worker pool such as piscina/tinypool).strace signature from the worker's JS thread, before any user eval runs:
No thread ever issues a restoring
F_SETFL.Cause
O_NONBLOCKis an open-file-description flag, not an fd flag.dup()makes a new fd number pointing at the same description, and worker threads share the fd table, so anF_SETFLon the worker's dup is a mode change of the parent's fd 1.The flip is reached because the worker's stdio rebind (
setupWorkerStdio) usesObject.defineProperty(process, "stdout"/"stderr", {...}).process.stdout/stderrare staticPropertyCallbackslots, and JSC'sdefineOwnPropertyreifies a lazy static property before defining over it. Reification runsconstructStdout→getStdioWriteStream→Bun.file(1).writer()→FileSink::setup()→open_for_writing, which dups fd 1 and callsset_nonblockingon the dup. The worker immediately discards that fd-backed stream (the port-backed writable replaces it), but theO_NONBLOCKon the shared description is the residue.The intended undo lives in
Bun__ForceFileSinkToBeSynchronousForProcessObjectStdio, which is gated onself.fd != INVALID.FileSink::setup()never wroteself.fd(the fd went only toself.writer), so that branch is dead code.void process.stdouton a pipe therefore leaves fd 1 nonblocking even on the main thread with no worker involved.The drop is
fd_write_all_quiet(the native console / output writer) treating anyErr(_), includingEAGAIN, as terminal and discarding the unwritten tail.Fix
Three small changes, each closing one seam:
src/js/node/worker_threads.ts: rebind worker stdio with plain assignment (process.stdout = ...) instead ofObject.defineProperty.put()replaces aPropertyCallbackslot without reifying it and yields the same{writable,enumerable,configurable}=truedescriptor, so the fd-backed constructor is never run in a worker and fd 1/2 are never touched.src/sys/lib.rs:fd_write_all_quietpollsPOLLOUTand retries onEAGAINinstead of giving up. Anything sharing the open file description (a worker, a parent shell, libuv in a co-process, node itself) can flip the flag at any time; blocking on writability is what makes this class safe in Node.src/runtime/webcore/FileSink.rs:setup()records the dup'd/opened fd inself.fd, so the process-stdio force-sync hook'supdate_nonblocking(self.fd, false)actually runs, and_getFd()returns the real fd on theBun.file(fd).writer()path.Verification
New tests in
test/js/node/process/process-stdio.test.ts(POSIX):reading process.stdout / process.stderr leaves fd 1/2 blocking: fcntl(F_GETFL) on fd 1/2 before/after the lazy getter fires.starting a node:worker_threads Worker leaves fd 1/2 blocking: same, across an idle worker coming online.console.log delivers every byte when fd 1 is O_NONBLOCK and the pipe is full: a rawpipe(2)under the test's control; the child fills it toEAGAIN, signals on stderr, thenconsole.logs markers into the still-full pipe. Without the fix the markers are dropped (payload"", 5/5); with the fix the writer polls until the parent drains and all markers arrive (5/5).All three fail on
mainand pass with this change. The original 20000-line worker repro delivers 20000/20000.Related
self.fdinFileSink::setup()(and fixes a separateBun.writeEAGAIN spin); the one-line overlap here will drop out on rebase if that lands first.fd_write_all_quietEAGAIN poll; this PR adds the worker trigger and a deterministic pipe-full test alongside it.definePropertythat reaches the existingsetup()/writer seams.Fixes #21516.
no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/node/process/process-stdio.test.ts