Skip to content

Bun.write(file, Bun.stdin): splice a pipe source into any destination on Linux - #36692

Open
robobun wants to merge 6 commits into
mainfrom
farm/50f97965/bun-write-stdin-pipe
Open

Bun.write(file, Bun.stdin): splice a pipe source into any destination on Linux#36692
robobun wants to merge 6 commits into
mainfrom
farm/50f97965/bun-write-stdin-pipe

Conversation

@robobun

@robobun robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Fixes #14054.

Repro

echo "hello world" | bun -e 'await Bun.write(Bun.file("foo.txt"), Bun.stdin)'
# error: Non-regular files aren't supported yet
#  syscall: "fstat", errno: 95

Cause

The Linux file-to-file copy dispatch in CopyFile::run_async has three fast-path predicates before falling through to "Non-regular files aren't supported yet":

  • ISREG(src) && ISREG(dest)copy_file_range
  • ISFIFO(src) && ISFIFO(dest)splice
  • ISREG|ISCHR|ISSOCK(src)sendfile

A FIFO source with a regular-file destination matches none of them.

Fix

splice(2) only requires one end to be a pipe, so drop the ISFIFO(dest) guard and take the splice branch for any FIFO source. do_copy_file_range already falls back to a read/write loop on EINVAL/ENOTSUP/ENOSYS if the destination can't be spliced to, so no new error handling is needed.

On macOS, also treat ENOTSUP from fcopyfile as a fallback trigger alongside the existing EBADF case, since the original report was macOS with ENOTSUP.

Verification

$ head -c 5000000 /dev/urandom > src.bin
$ cat src.bin | bun-debug -e 'console.log(await Bun.write(Bun.file("out.bin"), Bun.stdin))'
5000000
$ cmp src.bin out.bin && echo ok
ok

The new test in bun-write.test.js fails on released bun with Non-regular files aren't supported yet and passes with this change.


no test proof · iteration 5 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/io/bun-write.test.js

@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

Status: diff is green; CI red is unrelated. Ready for review.

Reproduced with echo hello | bun -e 'await Bun.write(Bun.file("foo.txt"), Bun.stdin)' on Linux (errno 95, Non-regular files aren't supported yet). Both new tests fail on released bun, pass on this branch.

Build 87572 (final): 194/196 passed. bun-write.test.js is not in any failure list; it passed on every darwin and linux lane. The two red jobs are unrelated to this change:

  • step-failed-outside-runner exit 255 on one darwin 14 aarch64 shard (agent/infra, not a test)
  • test/cli/install/bun-upgrade.test.ts on windows-aarch64 (pre-existing on main)

Picked up along the way:

  • do_copy_file_range no longer ftruncates a user-provided destination fd, so a >> redirect keeps its existing bytes.
  • macOS fcopyfile success path now records read_len so Bun.write() resolves the byte count when clonefile was skipped.

@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 11:59 PM PT - Aug 1st, 2026

@robobun, your commit dc5c7f3 has 2 failures in Build #87572 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 36692

That installs a local version of the PR into your bun-36692 executable, so you can run:

bun-36692 --bun

@github-actions github-actions Bot added the claude label Aug 1, 2026
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The file copy path now handles macOS ENOTSUP failures and Linux/Android FIFO sources more broadly. A regression test validates streaming 200,000 bytes from Bun.stdin into a regular file.

Changes

Pipe-backed file copying

Layer / File(s) Summary
Platform copy fallback handling
src/runtime/webcore/blob/copy_file.rs
macOS now falls back for ENOTSUP and EBADF. Linux and Android use splice for FIFO sources, with existing fallback handling.
stdin-to-file regression coverage
test/js/bun/io/bun-write.test.js
A non-Windows test validates Bun.write output, byte count, file size, empty child output, and process exit status.

Possibly related PRs

  • oven-sh/bun#36590: Both changes update pipe and FIFO streaming behavior in copy_file.rs and bun-write.test.js.
  • oven-sh/bun#35832: Both changes address pipe-based reads in src/runtime/webcore/blob.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main Linux change: enabling Bun.write to splice pipe input into any destination.
Description check ✅ Passed The description explains the issue, cause, fix, macOS behavior, verification steps, and regression test coverage.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/runtime/webcore/blob/copy_file.rs`:
- Around line 550-552: Update the ENOTSUP fallback branch in the macOS copy flow
to assign the checked, max-length-adjusted total_written value to self.read_len
before returning. Preserve the existing read/write loop and ensure successful
pipe copies resolve the actual byte count through then().

In `@test/js/bun/io/bun-write.test.js`:
- Around line 452-458: In the regression test comment preceding the Bun.write
pipe case, retain only the issue URL comment and remove the explanatory lines
describing the stdin pipe behavior and shell workaround.
- Around line 462-478: Update the assertion around Bun.file(out).size to also
read the destination contents and compare them with a new Uint8Array(size),
while preserving the existing stdout, resolved byte count, and written-length
checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 28b65e52-f21b-4c41-8423-f459188318d8

📥 Commits

Reviewing files that changed from the base of the PR and between f91d5c9 and 26214ae.

📒 Files selected for processing (2)
  • src/runtime/webcore/blob/copy_file.rs
  • test/js/bun/io/bun-write.test.js

Comment thread src/runtime/webcore/blob/copy_file.rs Outdated
Comment thread test/js/bun/io/bun-write.test.js
Comment thread test/js/bun/io/bun-write.test.js
Comment thread src/runtime/webcore/blob/copy_file.rs Outdated
Comment thread src/runtime/webcore/blob/copy_file.rs Outdated
Comment thread src/runtime/webcore/blob/copy_file.rs Outdated
Comment thread src/runtime/webcore/blob/copy_file.rs

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/runtime/webcore/blob/copy_file.rs:842-846 — Dropping the ISFIFO(dest) guard makes echo data | bun -e 'await Bun.write(Bun.stdout, Bun.stdin)' >> log.txt newly reach do_copy_file_range::<Splice, false>() on Linux, where splice(2) returns EINVAL for an O_APPEND target and the fallback read/write loop then unconditionally ftruncates the destination — silently destroying the pre-existing contents of log.txt. Before this PR the same command hit unsupported_non_regular_file_error() (a clean rejection), so this is a regression from error to silent data loss. Skip the ftruncate when the destination is a user-provided fd (or has O_APPEND set), or key CLEAR_APPEND_IF_INVALID on F_GETFL & O_APPEND rather than is_atty.

    Extended reasoning...

    What the bug is

    Removing the && ISFIFO(self.destination_file_store.mode) clause routes any FIFO-source Bun.write through do_copy_file_range::<Splice, _>() regardless of destination type. That newly exposes an existing ftruncate footgun in the EINVAL fallback for a case that previously errored cleanly: writing a piped Bun.stdin into Bun.stdout when stdout has been redirected with >> (O_APPEND) to a non-empty regular file.

    Code path

    For echo data | bun -e 'await Bun.write(Bun.stdout, Bun.stdin)' >> log.txt on Linux:

    1. Bun.stdout's store is built by stdio_stores::build_store (BunObject.rs:2999-3013, BunObject.rs:3053): is_atty = matches!(stdout_descriptor_type(), Terminal)Some(false) for a >> redirect; mode = fstat(1).st_modeS_IFREG.
    2. run_async: both source and dest are PathOrFileDescriptor::Fd, so no open() runs; fstat(0) → FIFO. The first branch (ISREG(src) && ISREG(dest)) is false. The new second branch matches on ISFIFO(stat.st_mode) alone — before this PR it also required ISFIFO(dest.mode), which is false for S_IFREG.
    3. is_atty.unwrap_or(false) == falsedo_copy_file_range::<{ TryWith::Splice }, false>(), i.e. CLEAR_APPEND_IF_INVALID = false.
    4. splice(0, NULL, 1, NULL, 65536, 0)EINVAL. The splice(2) man page and kernel fs/splice.c:do_splice() are explicit: if (out->f_flags & O_APPEND) return -EINVAL; — "The target file is opened in append mode."
    5. In the E::EINVAL arm, CLEAR_APPEND_IF_INVALID is a const-generic false, so the fcntl(F_SETFL, flags ^ O_APPEND) retry block is dead code. total_written == 0, so it drops into copy_file_using_read_write_loop, which uses Syscall::write() — and write() honors O_APPEND, so N bytes land at offset M..M+N where M is the pre-existing file size.
    6. On Ok(()) the fallback unconditionally calls libc::ftruncate(dest_fd, total_written) = ftruncate(1, N), discarding everything from offset N onward — the pre-existing content and (if M > 0) the appended bytes themselves.

    The promise resolves successfully with N, but the destination file is silently corrupted.

    Why existing safeguards don't help

    The O_APPEND-strip retry is gated on CLEAR_APPEND_IF_INVALID, which is keyed on destination_file_store.is_atty. That's Some(true) only when stdout is a terminal, and a shell >> redirect to a regular file is not a terminal, so the retry never fires. The comment added at line 844 ("falls back to a read/write loop on EINVAL … if the destination cannot be spliced to") is accurate about the fallback but omits that the fallback then ftruncates. The ftruncate is safe when Bun opened the destination itself (OPEN_DESTINATION_FLAGS includes O_TRUNC, so the file starts empty), but destructive for a user-provided O_APPEND fd.

    Step-by-step proof

    Suppose log.txt already contains 100 bytes and stdin delivers "data\n" (5 bytes):

    • splice(0→1) → EINVAL (fd 1 has O_APPEND from >>).
    • CLEAR_APPEND_IF_INVALID=false → skip the append-strip retry.
    • total_written==0 → read/write loop: read(0) gets 5 bytes, write(1, buf, 5) — with O_APPEND — writes at offset 100..105. total_written = 5.
    • ftruncate(1, 5) shrinks log.txt from 105 bytes to 5 bytes: the original 100 bytes and the appended 5 bytes at 100..105 are gone; only whatever was at offsets 0..5 survives.
    • then() resolves the promise to 5.

    Before this PR: FIFO src + REG dest matched none of the three predicates (ISFIFO(dest)=false, and ISREG|ISCHR|ISSOCK(FIFO)=false), so it fell through to unsupported_non_regular_file_error() — "Non-regular files aren't supported yet", errno 95. A clean rejection, not data loss.

    The trigger isn't exotic: >> is standard shell append, and make(1) sets O_APPEND on stdout by default (which is exactly what the kylelaker.com comment already in the code describes for the is_atty=true case).

    Suggested fix

    Either:

    • Skip the ftruncate in do_copy_file_range's fallback arms when the destination fd was user-provided (matches!(self.destination_file_store.pathlike, PathOrFileDescriptor::Fd(_))) — Bun didn't open it with O_TRUNC, so it must not assume the file starts at offset 0; or
    • Key CLEAR_APPEND_IF_INVALID on fcntl(dest_fd, F_GETFL) & O_APPEND (checked once, up-front) rather than is_atty, so the append flag is stripped before the retry regardless of terminal-ness.

    The first is more conservative (doesn't silently clear a flag on an fd the user owns). Note the same ftruncate-on-user-fd hazard pre-exists for the ISREG(src) branch (copy_file_range also EINVALs on O_APPEND), so the underlying defect isn't new — but this PR newly exposes it for the FIFO-source path that previously errored cleanly, per REVIEW.md's "regression from clean error to silent corruption is blocking".

Comment thread src/runtime/webcore/blob/copy_file.rs
Comment thread src/runtime/webcore/blob/copy_file.rs Outdated
Comment thread src/runtime/webcore/blob/copy_file.rs Outdated
@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

Re the O_APPEND finding (FIFO source + >> redirected stdout): fixed in 4060eb9. do_copy_file_range now gates the read/write-loop fallback's ftruncate on destination_file_store.pathlike being a Path, i.e. an fd Bun opened with O_TRUNC. A user-provided fd (Bun.stdout, Bun.file(fd)) is never truncated, so write(2) honours O_APPEND and the pre-existing bytes survive. Added a test that seeds a file, pipes into Bun.write(Bun.stdout, Bun.stdin) with >>, and checks the result is appended.

Also folded in the fcopyfile-success read_len fix in the same commit.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/runtime/webcore/blob/copy_file.rs:840-842 — On Linux, dropping the ISFIFO(dest) guard lets a FIFO source reach the splice branch with a user-supplied regular-file fd opened O_APPEND (e.g. Bun.write(Bun.file(fs.openSync('log','a')), Bun.stdin)). splice(2) returns EINVAL for an O_APPEND target; CLEAR_APPEND_IF_INVALID is keyed on is_atty (false here) so the strip-and-retry is skipped; the read/write fallback appends M bytes after the existing N; then the unconditional ftruncate(dest_fd, M) keeps only the first M bytes of the original content and discards everything just written — while the promise resolves successfully with M. Before this PR the same call matched no fast-path predicate and cleanly rejected with "Non-regular files aren't supported yet", so this is a regression from a diagnosable error to silent data loss. Simplest fix: skip the post-fallback ftruncate when destination_file_store.pathlike is Fd (we didn't open it with O_TRUNC, so writes cannot be assumed to start at offset 0).

    Extended reasoning...

    What the bug is

    Removing the ISFIFO(dest) guard on the splice branch (line 842) means a FIFO source now reaches do_copy_file_range::<Splice, _> for any destination, including a user-supplied regular-file fd opened with O_APPEND. On that path, splice(2) returns EINVAL (per the man page: "The target file is opened in append mode"), the EINVAL fallback appends via plain write(), and then unconditionally ftruncates the destination to total_written — an offset that assumes writes started at 0. When the fd was opened O_APPEND and already had N bytes, that assumption is wrong: the M newly-written bytes went to offset N, and ftruncate(fd, M) keeps bytes 0..M of the original content, discarding everything that was just written (and part of the original if M < N). The promise then resolves successfully with M.

    Step-by-step trace

    Concrete example on Linux:

    const fs = require('fs');
    fs.writeFileSync('log.txt', 'AAAAAAAAAA');           // 10 bytes
    const fd = fs.openSync('log.txt', 'a');              // O_APPEND
    // stdin is a pipe carrying "BBB" (3 bytes)
    await Bun.write(Bun.file(fd), Bun.stdin);            // resolves 3
    1. run_async: source is fd 0 (FIFO), dest is a user fd → both fds set from pathlike.fd(); no do_open_file (so no O_TRUNC).
    2. fstat(0)S_ISFIFO. Dest is_atty is false for a regular file → do_copy_file_range::<Splice, false>(), i.e. CLEAR_APPEND_IF_INVALID = false.
    3. splice(pipe_fd, regular_file_fd_with_O_APPEND)EINVAL (splice(2) man page: "The target file is opened in append mode").
    4. EINVAL arm: CLEAR_APPEND_IF_INVALID is false, so the fcntl(F_SETFL, flags ^ O_APPEND) retry is skipped. total_written == 0, so it falls into copy_file_using_read_write_loop.
    5. The read/write loop uses plain Syscall::write(), which honors O_APPEND: 3 bytes of "BBB" are written at offset 10. File is now AAAAAAAAAABBB (13 bytes), total_written = 3.
    6. On Ok(()), the fallback unconditionally does libc::ftruncate(dest_fd, 3). File becomes AAA — the first 3 bytes of the original content. All newly-written bytes are gone, and 7 of the original 10 bytes are gone.
    7. The scopeguard sets self.read_len = 3; then() resolves the promise with 3.

    Result: the user sees a successful resolve with the byte count they piped in, but the file contains neither the appended data nor the original content.

    Why this is new to this PR

    Before this change, the splice branch was gated on ISFIFO(src) && ISFIFO(dest). With a FIFO source and a regular-file destination:

    • ISREG(src) && ISREG(dest) — no (src is FIFO)
    • ISFIFO(src) && ISFIFO(dest) — no (dest is regular)
    • ISREG|ISCHR|ISSOCK(src) — no (src is FIFO)

    …so the call fell through to unsupported_non_regular_file_error() and rejected with "Non-regular files aren't supported yet". That's exactly the error #14054 reports. So this input has gone from a diagnosable rejection to silent data corruption + false success signal.

    The ftruncate-after-fallback in the EINVAL arm itself is pre-existing code, but it was previously reachable via this branch only for FIFO destinations, where ftruncate fails harmlessly with EINVAL and is discarded via let _ =. Regular-file destinations reaching it is what this PR introduces.

    (This is not reachable via the other two branches: copy_file_range on an O_APPEND out_fd returns EBADF, which hits the catch-all error arm without ftruncate; and sendfile requires ISREG|ISCHR|ISSOCK source, which a FIFO is not.)

    Why nothing else prevents it

    CLEAR_APPEND_IF_INVALID is the mechanism that already exists for exactly this problem — it strips O_APPEND and retries — but it's keyed on destination_file_store.is_atty, which is false for a regular-file fd. So the strip-and-retry never runs on this path.

    The destination fd came from the user via PathOrFileDescriptor::Fd, so do_open_file::<Destination> (which uses OPEN_DESTINATION_FLAGS including O_TRUNC) is never called. Nothing else in run_async truncates or seeks the fd before the copy.

    Impact

    Silent data loss with a false success signal on a one-line user-reachable input. Per REVIEW.md, "Never swallow a failure or signal success on one" — this is exactly that class. The trigger is admittedly unusual (Bun.write semantics are "replace", so passing an append-mode fd is odd), but before this PR it errored cleanly, and now it corrupts.

    Fix

    Either of these is sufficient; (a) is the smaller diff:

    (a) Skip the post-fallback ftruncate in the EINVAL (and ENOSYS|EXDEV|ENOTSUP, and !can_use_copy_file_range_syscall()) arms when matches!(self.destination_file_store.pathlike, PathOrFileDescriptor::Fd(_)). We only opened the destination with O_TRUNC when it was a Path; for a user-supplied fd we cannot assume writes started at offset 0, so truncating to total_written is never safe.

    (b) Extend CLEAR_APPEND_IF_INVALID to also cover ISREG(destination_file_store.mode) (or just pass true for the splice branch now that the dest can be anything), so O_APPEND is stripped before the retry and the subsequent ftruncate assumption holds.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both earlier findings are addressed — the fcopyfile fallback and success paths now set read_len, and the read/write fallback no longer ftruncates a user-provided fd. No new issues found, but the scope has grown from a one-line predicate drop into cross-platform I/O behavior changes (Linux splice dispatch, macOS read_len accounting, bun_opened_dest ftruncate gating), and the macOS path of the two new tests hasn't been exercised locally, so this should get a human sign-off once CI is green.

What was reviewed

  • Linux FIFO→any-dest splice path, including the O_APPEND EINVAL fallback with CLEAR_APPEND_IF_INVALID=false — reaches the read/write loop and skips ftruncate via bun_opened_dest.
  • macOS run_async block: the new read_len = stat_size.min(max_length) only fires when stat.st_size != 0 && read_len == 0, so it doesn't clobber the pipe-fallback's total_written.
  • bun_opened_dest gate applied consistently at all three read/write-fallback ftruncate sites in do_copy_file_range.
Extended reasoning...

Overview

The PR fixes #14054 by dropping the ISFIFO(dest) guard on the Linux splice branch so a pipe source can be spliced into any destination, and adds ENOTSUP to the macOS fcopyfile fallback triggers. Through review it also picked up: (a) setting self.read_len in the macOS read/write-loop fallback and after fcopyfile success, and (b) gating the Linux read/write-fallback ftruncate on whether Bun opened the destination fd. Two new tests cover pipe→file and pipe→O_APPEND-file.

Security risks

None. This is local file I/O dispatch; no untrusted input parsing, auth, or network surface.

Level of scrutiny

Moderate-to-high. CopyFile::run_async is the core path for every Bun.write(BunFile, BunFile) on POSIX, and the changes are platform-gated (#[cfg(linux)] / #[cfg(macos)]) with behavior that differs per kernel. The bun_opened_dest gate is a real semantic change: previously the read/write fallback unconditionally ftruncated the destination, which was harmless when Bun opened it with O_TRUNC but wrong for user fds — dropping it is correct but changes observable behavior for anyone who was (probably accidentally) relying on it.

Other factors

I've reviewed this PR twice already; both findings (macOS fallback discarding total_written, and the sibling fcopyfile-success arm leaving read_len=0) were addressed in a047bf1 and 4060eb9. The author also independently handled the O_APPEND/ftruncate interaction and added a targeted test. The remaining uncertainty is the macOS behavior of the second new test — fcopyfile(pipe, regular-O_APPEND, COPYFILE_DATA) needs to fail cleanly with ENOTSUP/EBADF without touching the destination for the append assertion to hold; the PR body notes tests were not run locally on macOS. That plus the scope expansion beyond the original predicate fix is why I'm deferring rather than approving.

@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

Verified the macOS assumption on a fleet box (Darwin 24.1.0 arm64): fcopyfile(pipe_fd, regular_O_APPEND_fd, COPYFILE_DATA) returns -1 with errno 45 (ENOTSUP) and leaves the destination untouched, so the EBADF | ENOTSUP fallback fires and the read/write loop (which honours O_APPEND) handles both new tests on macOS.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All prior feedback addressed — I found nothing new on this pass. Deferring to a human primarily because the macOS side now carries a rider fix for a pre-existing behavior (Bun.write(file, file) resolving to 0 whenever fcopyfile succeeds after clonefile was skipped/failed), which is correct but a user-visible change worth a maintainer's eye.

What was reviewed:

  • Linux splice branch with FIFO src → regular-file dest, and → O_APPEND dest (EINVAL → read/write loop, no ftruncate on user fd).
  • bun_opened_dest gate vs. the earlier preallocate_file — preallocate only fires for ISREG(src) with known size, so skipping ftruncate on a user fd doesn't strand preallocated space in the FIFO-source paths this PR opens up.
  • macOS read_len after the helper: the read_len == 0 guard correctly leaves the fallback-set value alone; stat_size.min(max_length) matches the post-ftruncate size for the offset==0 cases.
Extended reasoning...

Overview

Two files. copy_file.rs (Linux/macOS syscall dispatch for Bun.write(Blob, Blob)): drops the ISFIFO(dest) requirement so a pipe source splices into any destination; gates the read/write-fallback ftruncate on Bun having opened the destination itself; adds ENOTSUP to the macOS fcopyfile fallback trigger; propagates total_written to read_len in that fallback; and sets read_len from stat.st_size after the macOS helper when fcopyfile itself succeeded. bun-write.test.js: two subprocess tests (pipe→file, pipe→>>-appended stdout) covering both platforms.

Security risks

None. No untrusted-input parsing; the only new decision point is matches!(pathlike, Path(_)) on internal state.

Level of scrutiny

Medium-high — this is core Bun.write I/O dispatch across two platforms with kernel-semantics reasoning (splice one-end-pipe rule, O_APPEND vs. splice EINVAL, fcopyfile ENOTSUP on pipes). Not mechanical, but narrow: ~40 changed Rust lines in one function pair, and every branch traced above lands on an existing fallback.

Other factors

Three review rounds are already resolved on the thread (coderabbit's read_len finding, my two read_len findings, comment-cop, and the O_APPEND/ftruncate finding that produced the bun_opened_dest gate). The author empirically verified fcopyfile(pipe, O_APPEND regular)ENOTSUP on Darwin 24.1, so the second test's macOS path is grounded. What keeps me from approving outright is the fcopyfile-success read_len assignment: it's the right fix for a latent bug I flagged, but it changes the resolved value of a common macOS path (clonefile EEXIST/EXDEV → fcopyfile) from 0 to the byte count, and no existing test asserted that value — a maintainer should be aware it's riding along.

@Jarred-Sumner Jarred-Sumner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use bu_sys instead of libc. Rebase against main.

robobun added 6 commits August 2, 2026 03:34
… on Linux

The Linux file-to-file copy dispatch only took the splice(2) branch when
both source and destination were FIFOs. A pipe source with a regular-file
destination (echo ... | bun -e 'await Bun.write(Bun.file(p), Bun.stdin)')
matched none of the fast-path predicates and fell through to the
"Non-regular files aren't supported yet" error.

splice(2) only requires one end to be a pipe, so take that branch for any
FIFO source; do_copy_file_range already falls back to a read/write loop on
EINVAL/ENOTSUP/ENOSYS if the destination can't be spliced to. On macOS,
also treat ENOTSUP from fcopyfile as a fallback trigger alongside EBADF.

Fixes #14054
The EBADF/ENOTSUP fallback in do_fcopy_file_with_read_write_loop_fallback
computed total_written but never stored it to self.read_len, so a
successful pipe copy on macOS would resolve Bun.write() to 0. Also assert
the destination bytes (not just the size) in the new test, and trim the
test comment.
…r fcopyfile

When the Linux fast-path falls back to a read/write loop (splice/sendfile/
copy_file_range rejected the pair), it ftruncates the destination to the
written length. That is only correct for files Bun opened with O_TRUNC; on
a user-provided fd such as Bun.stdout redirected with >>, write(2) honours
O_APPEND so the bytes land at EOF and the trailing ftruncate would destroy
the pre-existing contents. Gate the ftruncate on the destination pathlike
being a Path.

Also record the byte count after fcopyfile() succeeds on macOS so
Bun.write() resolves the copied length when clonefile was skipped or
failed, mirroring the clonefile and FreeBSD paths.
@robobun
robobun force-pushed the farm/50f97965/bun-write-stdin-pipe branch from 2334703 to dc5c7f3 Compare August 2, 2026 03:40
@robobun

robobun commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main and switched the ftruncate calls the diff touches (three Linux read/write-fallback sites plus the macOS post-fcopyfile site) from libc::ftruncate/bun_sys::darwin::ftruncate to bun_sys::ftruncate in dc5c7f3. Tests still pass locally; fail-before still holds against released bun.

Comment thread src/runtime/webcore/blob/copy_file.rs
Jarred-Sumner pushed a commit that referenced this pull request Aug 4, 2026
… fd (#36758)

## Repro

```sh
head -c 200000 /dev/zero | tr '\0' S > /tmp/src.bin
printf 'DDDDDDDDDDDDDDDDDDDDDDDDDDDDDD' > /tmp/dst.bin
BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1 bun -e '
  const fs = require("fs");
  const fd = fs.openSync("/tmp/dst.bin", "r+");
  console.log(await Bun.write(Bun.file(fd).slice(0, 5), Bun.file("/tmp/src.bin")));
  fs.closeSync(fd);
'
wc -c /tmp/dst.bin
# Linux (fallback): 200000 (expected 30; whole source over-copied then ftruncated)
# macOS:            5      (expected 30; fcopyfile rewrote from 0 then ftruncate(5))
```

Same for `Bun.stdout` under `>>` redirection: the pre-existing bytes in
the redirected file are gone after the write.

## Cause

`CopyFile::run_async` dispatched fd-backed destinations through the same
path as path destinations.

- **macOS:** `fcopyfile(COPYFILE_DATA)` rewrites the destination from
offset 0, and when `stat.st_size > max_length` Bun then calls
`ftruncate(dest, max_length)` to trim the over-copied tail. Both are
safe when Bun opened the destination with `O_CREAT|O_TRUNC`, but for a
caller-supplied fd they discard whatever the file already contained.
- **FreeBSD** and the **Linux read/write fallback** (kernel < 4.5,
`EXDEV`, `ENOTSUP`, `EINVAL`, or `BUN_CONFIG_DISABLE_COPY_FILE_RANGE`):
`copy_file_using_read_write_loop` reads the source to EOF regardless of
`max_length` and then `ftruncate(dest, total_written)` sets the file
length, with the same effect.

The Linux `copy_file_range`/`sendfile`/`splice` happy path already
copies exactly `max_length` bytes and does not truncate, so this only
shows up there when the kernel syscall is unavailable for the fd pair.

## Fix

Add `read_write_loop_capped`, a bounded `read()`/`write()` loop that
transfers exactly `cap` bytes (or to EOF) and never touches the
destination's length. Route every fd-backed destination through it:

- macOS/FreeBSD `run_async`: take the `fcopyfile`/`ftruncate` branch
only when `destination_file_store.pathlike` is a path.
- Linux `do_copy_file_range`: the three identical fallback sites are
factored into `read_write_fallback`, which keeps the existing
copy-to-EOF + `ftruncate` for path destinations and uses the bounded
loop for fd destinations.

## Verification

Two tests in `test/js/bun/io/bun-write.test.js` under
`Bun.write(Bun.file(fd), Bun.file(path)) does not truncate the fd` spawn
a child with `BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1` so they fail-before
on every POSIX lane:

```
=== system bun ===
(fail) preserves bytes past the slice window in an r+ fd
  resolved: "200000", content: 200000 x "S"   (expected "5", "SSSSS" + 25 x "D")
(fail) preserves pre-existing bytes when stdout is redirected with >>
  resolved: "1000", content: 1000 bytes        (expected "100", 110 bytes)

=== this branch ===
(pass) preserves bytes past the slice window in an r+ fd
(pass) preserves pre-existing bytes when stdout is redirected with >>
```

The existing `Bun.write(Bun.stdout, Bun.stdin) copies the whole pipe`
test is widened from Linux-only to every POSIX lane, covering the
bounded loop's EOF-termination branch on macOS/FreeBSD. `cargo check -p
bun_runtime --target {aarch64,x86_64}-apple-darwin` and `--target
x86_64-unknown-freebsd` pass.

## Related

Found during review of #36692 (which routes a FIFO source through
`splice` on Linux; the `bun_opened_dest` guard it adds to the same three
fallback sites is subsumed by `read_write_fallback` here). #32859
reworks the same block to honour source slices and #33715 sets
`read_len` after `fcopyfile`; both are complementary and will need a
small rebase against whichever lands first.

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 4 · Platform-specific test(s) that do not
run on this machine. Deferring to CI, which covers all platforms:
test/js/bun/io/bun-write.test.js

<!-- robobun:evidence:end -->

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
springmin pushed a commit to springmin/bun that referenced this pull request Aug 4, 2026
… fd (oven-sh#36758)

## Repro

```sh
head -c 200000 /dev/zero | tr '\0' S > /tmp/src.bin
printf 'DDDDDDDDDDDDDDDDDDDDDDDDDDDDDD' > /tmp/dst.bin
BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1 bun -e '
  const fs = require("fs");
  const fd = fs.openSync("/tmp/dst.bin", "r+");
  console.log(await Bun.write(Bun.file(fd).slice(0, 5), Bun.file("/tmp/src.bin")));
  fs.closeSync(fd);
'
wc -c /tmp/dst.bin
# Linux (fallback): 200000 (expected 30; whole source over-copied then ftruncated)
# macOS:            5      (expected 30; fcopyfile rewrote from 0 then ftruncate(5))
```

Same for `Bun.stdout` under `>>` redirection: the pre-existing bytes in
the redirected file are gone after the write.

## Cause

`CopyFile::run_async` dispatched fd-backed destinations through the same
path as path destinations.

- **macOS:** `fcopyfile(COPYFILE_DATA)` rewrites the destination from
offset 0, and when `stat.st_size > max_length` Bun then calls
`ftruncate(dest, max_length)` to trim the over-copied tail. Both are
safe when Bun opened the destination with `O_CREAT|O_TRUNC`, but for a
caller-supplied fd they discard whatever the file already contained.
- **FreeBSD** and the **Linux read/write fallback** (kernel < 4.5,
`EXDEV`, `ENOTSUP`, `EINVAL`, or `BUN_CONFIG_DISABLE_COPY_FILE_RANGE`):
`copy_file_using_read_write_loop` reads the source to EOF regardless of
`max_length` and then `ftruncate(dest, total_written)` sets the file
length, with the same effect.

The Linux `copy_file_range`/`sendfile`/`splice` happy path already
copies exactly `max_length` bytes and does not truncate, so this only
shows up there when the kernel syscall is unavailable for the fd pair.

## Fix

Add `read_write_loop_capped`, a bounded `read()`/`write()` loop that
transfers exactly `cap` bytes (or to EOF) and never touches the
destination's length. Route every fd-backed destination through it:

- macOS/FreeBSD `run_async`: take the `fcopyfile`/`ftruncate` branch
only when `destination_file_store.pathlike` is a path.
- Linux `do_copy_file_range`: the three identical fallback sites are
factored into `read_write_fallback`, which keeps the existing
copy-to-EOF + `ftruncate` for path destinations and uses the bounded
loop for fd destinations.

## Verification

Two tests in `test/js/bun/io/bun-write.test.js` under
`Bun.write(Bun.file(fd), Bun.file(path)) does not truncate the fd` spawn
a child with `BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1` so they fail-before
on every POSIX lane:

```
=== system bun ===
(fail) preserves bytes past the slice window in an r+ fd
  resolved: "200000", content: 200000 x "S"   (expected "5", "SSSSS" + 25 x "D")
(fail) preserves pre-existing bytes when stdout is redirected with >>
  resolved: "1000", content: 1000 bytes        (expected "100", 110 bytes)

=== this branch ===
(pass) preserves bytes past the slice window in an r+ fd
(pass) preserves pre-existing bytes when stdout is redirected with >>
```

The existing `Bun.write(Bun.stdout, Bun.stdin) copies the whole pipe`
test is widened from Linux-only to every POSIX lane, covering the
bounded loop's EOF-termination branch on macOS/FreeBSD. `cargo check -p
bun_runtime --target {aarch64,x86_64}-apple-darwin` and `--target
x86_64-unknown-freebsd` pass.

## Related

Found during review of oven-sh#36692 (which routes a FIFO source through
`splice` on Linux; the `bun_opened_dest` guard it adds to the same three
fallback sites is subsumed by `read_write_fallback` here). oven-sh#32859
reworks the same block to honour source slices and oven-sh#33715 sets
`read_len` after `fcopyfile`; both are complementary and will need a
small rebase against whichever lands first.

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 4 · Platform-specific test(s) that do not
run on this machine. Deferring to CI, which covers all platforms:
test/js/bun/io/bun-write.test.js

<!-- robobun:evidence:end -->

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Writing stdin to a file errors with ENOTSUP

2 participants