Skip to content

Bun.write: honour the destination window on the Linux read/write fallback - #37856

Open
robobun wants to merge 1 commit into
mainfrom
farm/14ae63c4/bun-write-fallback-dest-window
Open

Bun.write: honour the destination window on the Linux read/write fallback#37856
robobun wants to merge 1 commit into
mainfrom
farm/14ae63c4/bun-write-fallback-dest-window

Conversation

@robobun

@robobun robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • Bun.write(Bun.file("dest.bin").slice(0, 5000), Bun.file("src.bin")) with a 300000 byte source writes 5000 bytes on the copy_file_range path but 300000 bytes on the Linux read/write fallback. Reproduced on 1.4.0 and main.
  • The fallback is taken with BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1 or when copy_file_range returns EXDEV, ENOSYS or ENOTSUP, so the same call copies a different number of bytes depending on whether the two files share a filesystem.
  • Cause: for a path destination the fallback reused the fs.copyFile read loop, which treats the size it is given as a hint and reads to EOF, then truncated the destination to whatever it had copied. The window was never applied.
  • Every other arm of the copy (kernel syscall loop, fd-destination fallback, macOS, FreeBSD, Windows) already stops at or truncates to the window. This was the only arm that did not.

Fix

  • The path-destination fallback now uses the same capped 64 KiB read/write loop as the fd-destination arm, so every arm of the copy stops at the window.
  • The ftruncate after the copy stays for destinations Bun opened itself: the file may have been preallocated to the source's stat size, which is too long if the source shrank during the copy. It is a no-op otherwise.
  • The POSIX_FADV_SEQUENTIAL hint (sys: hint POSIX_FADV_SEQUENTIAL on the copy-file read/write fallback #34825) lived in the old loop, so it moves into the capped loop; per the numbers in sys: hint POSIX_FADV_SEQUENTIAL on the copy-file read/write fallback #34825 that keeps the 64 KiB loop on par with the old 8 MiB buffer, and the existing hint test still passes.
  • Verification: 7 new Linux tests, 3 of which fail on the released bun and all of which pass with this build. They reach the fallback through the env var, not through a real cross-filesystem copy.

Background

  • Bun.write(dest, Bun.file(src)) copies file to file inside the runtime. Slicing the destination, Bun.file(path).slice(0, n), hands the copy a window of n bytes that every platform arm is expected to apply.
  • On Linux the copy first tries copy_file_range (an in-kernel copy) and drops to a userspace read()/write() loop when the kernel refuses: EXDEV across filesystems, ENOSYS or ENOTSUP where unsupported, or the env var above.
  • The fallback has two arms: a destination Bun opened from a path, which may be preallocated to the expected length before the copy (hence the trailing ftruncate), and a file descriptor supplied by the caller. Only the path arm was affected.
  • POSIX_FADV_SEQUENTIAL tells the kernel the source will be read front to back so it can read ahead, which is what lets a small buffer keep pace with a large one.
  • A source with st_size == 0 (procfs, for example) gives the copy no length up front: an unsliced destination must still read to EOF, a sliced one stops at n.
Original description

Repro

const fs = require("fs");
fs.writeFileSync("src.bin", Buffer.alloc(300000, "S"));
const n = await Bun.write(Bun.file("dest.bin").slice(0, 5000), Bun.file("src.bin"));
console.log(n, fs.statSync("dest.bin").size);
$ bun repro.js                                        # copy_file_range
5000 5000
$ BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1 bun repro.js   # read/write fallback
300000 300000

The env var is only the easiest way in. The same fallback is taken when copy_file_range returns EXDEV, ENOSYS or ENOTSUP, so on a 6.x kernel the same call copies 5000 bytes when both files are on one filesystem and 300000 when they are not (reproduced here with overlayfs -> /dev/shm, and /proc/version -> tmpdir, both with no env var set). Release 1.4.0 and current main behave the same.

Cause

read_write_fallback in src/runtime/webcore/blob/copy_file.rs sent a path destination through NodeFS::copy_file_using_read_write_loop(.., stat_size = cap, ..). That helper is written for fs.copyFile, where the stat size is a hint: it reads stat_size bytes and then, unless it hit EOF, keeps reading until it does. So the cap was ignored whenever the source was longer than it, and the ftruncate(dest, total) that followed then pinned the file at the full source length.

Every other implementation of the copy honours the window already: the Linux copy_file_range/sendfile/splice loop stops at remain, the fd-destination arm of this same function uses read_write_loop_capped (#36758), macOS (clonefile, fcopyfile) and FreeBSD truncate the destination to max_length, and CopyFileWindows::on_complete truncates to size. This was the one arm that did not, and whichever cap the caller ends up passing (the destination window today, a source slice if #31515 lands) CopyFile needs every arm to apply it the same way.

Fix

Route the path destination through read_write_loop_capped as well, so read_write_fallback is one bounded loop plus, for a destination Bun opened itself, the existing ftruncate(dest, total). The truncate is kept because run_async may have fallocated the destination to the stat'd length before the copy, which is longer than the copy if the source shrank in between; it is a no-op otherwise. The fixing line is the read_write_loop_capped call replacing the copy_file_using_read_write_loop call.

copy_file_using_read_write_loop was also where the fallback picked up its POSIX_FADV_SEQUENTIAL hint (#34825), so the hint moves into read_write_loop_capped (same cfg as the original) and the fd arm gets it too; the existing LD_PRELOAD test for the hint still passes. Per the numbers in #34825, a 64 KiB loop with the hint is on par with the 8 MiB slab that helper used for large files; a 256 MiB fallback copy with this debug build took 0.6 to 0.9 s here against 1.0 s or more for the release slab.

Unchanged: the fd-destination arm (already bounded), the kernel syscall loop, and the macOS/FreeBSD/Windows arms (all truncate to the window). cargo check -p bun_runtime passes for x86_64-apple-darwin and x86_64-unknown-freebsd, which share read_write_loop_capped.

Verification

New describe in test/js/bun/io/bun-write.test.js (Linux block), 7 cases: the window on both the copy_file_range path and the fallback, a window past EOF on both, a 2.5 MiB window (above the 2 MiB fallocate threshold, so the kept ftruncate is exercised), and an st_size == 0 source (/proc/version) both unsliced (must still read to EOF) and sliced.

USE_SYSTEM_BUN=1 bun test test/js/bun/io/bun-write.test.js -t "copies n bytes"   # 3 fail: fallback, fallback preallocated, fallback st_size == 0 source
bun bd test test/js/bun/io/bun-write.test.js -t "copies n bytes"                  # 7 pass

The rest of the file passes with this build apart from should work when copyFileRange is not available > on large files, which exceeds the 5 s timeout on this machine on unmodified main too (and starves whichever concurrent siblings are in flight); #37792 is the fix for that test.

Related

While checking blast radius: on the normal copy_file_range path (and the macOS/Windows arms), reading f.size or awaiting f.exists() on the destination before Bun.write(f, Bun.file(src)) caps the copy at the destination's previous size, because resolve_size() stores the stat size in the same field slice() uses and write_file_with_source_destination passes that field as the window. That is a call-site problem in Blob.rs which this PR does not change (after this PR the fallback matches the other arms there too); it is being tracked separately. #35815 is about sliced destinations with a non-zero offset and leaves slice(0, n) as is.

…back

Bun.write(Bun.file(path).slice(0, n), Bun.file(src)) copies n bytes via
copy_file_range, but when that degrades to the read()/write() fallback
(EXDEV, ENOSYS, ENOTSUP, BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1) a path
destination went through copy_file_using_read_write_loop, which treats
its size argument as a hint and drains the source to EOF, and the
ftruncate that followed pinned the file at the full source length.

Route path destinations through the same bounded loop the fd arm uses,
keeping the ftruncate for the case where run_async fallocate'd the file
longer than what was copied. The POSIX_FADV_SEQUENTIAL hint moves into
the bounded loop so the fallback keeps it.
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 4 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 829ad488-891f-48fb-b783-b87af8be350f

📥 Commits

Reviewing files that changed from the base of the PR and between 9a543cc and f35d99d.

📒 Files selected for processing (2)
  • src/runtime/webcore/blob/copy_file.rs
  • test/js/bun/io/bun-write.test.js

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on release 1.4.0 and current main (Linux 6.17) with the snippet in the description, both via BUN_CONFIG_DISABLE_COPY_FILE_RANGE=1 and via a plain cross-filesystem copy with no env var. Fix is in read_write_fallback (src/runtime/webcore/blob/copy_file.rs); the three new fallback window cases in test/js/bun/io/bun-write.test.js fail on the unfixed build and pass with it. Waiting on CI.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. The fix is small and the test matrix is thorough, but since it reroutes the Linux Bun.write fallback through a different copy loop (64 KiB + fadvise instead of the 8 MiB-slab helper) on a path reachable in production via EXDEV, a maintainer sign-off on that trade-off would be worthwhile.

Checked: read_write_loop_capped still reads to EOF when cap == MAX_SIZE (the /proc/version unsliced case); ftruncate is still gated on bun_opened_dest and skipped on error, matching the old behavior; the posix_fadvise cfg matches node_fs.rs exactly and is compiled out on macOS; node_fs import is still live via DEFAULT_PERMISSION and the macOS/FreeBSD arms.

Extended reasoning...

Overview

The PR fixes read_write_fallback in src/runtime/webcore/blob/copy_file.rs (Linux/Android only). Previously, when the destination was a path Bun opened itself, the fallback delegated to NodeFS::copy_file_using_read_write_loop, which treats its size argument as a hint and reads to EOF — so a .slice(0, n) destination window was ignored whenever copy_file_range was unavailable (EXDEV, ENOSYS, ENOTSUP, or the debug env var). The fix routes both the path- and fd-destination arms through the existing read_write_loop_capped, keeping the ftruncate for the path arm to trim any fallocate overshoot. The POSIX_FADV_SEQUENTIAL hint that lived in the removed helper is moved into read_write_loop_capped under the same cfg(linux|android|freebsd). Seven new test cases in bun-write.test.js cover both syscall paths, window-past-EOF, the preallocate threshold, and st_size==0 sources.

Security risks

None. No untrusted input parsing, no path handling changes, no new syscalls beyond an advisory posix_fadvise hint that was already issued on this path via the removed helper.

Level of scrutiny

Medium-high. This is core Bun.write file-copy runtime code, and the fallback is reachable in production on any cross-filesystem copy (EXDEV). The correctness change itself is straightforward — one function call swapped for another that already exists and is already used by the sibling arm — but it also swaps the large-file buffer strategy (8 MiB slab → 64 KiB loop with readahead hint). The PR description benchmarks this as neutral-to-better, referencing #34825, but that's a performance claim on a hot-ish path that a maintainer should confirm.

Other factors

I verified the node_fs import remains live (used by DEFAULT_PERMISSION, the macOS fcopyfile fallback, the FreeBSD arm, and Windows), that the new posix_fadvise cfg exactly matches the one in node_fs.rs:4723, and that the unknown-size / read-to-EOF semantics are preserved (cap = MAX_SIZE → loop until read() returns 0). The ftruncate gating and error-path skip are unchanged from before. FreeBSD's fd-destination arm now also gets the fadvise hint via do_read_write_loop_capped, which is new but harmless. Tests follow harness conventions (tempDir, bunEnv spread, combined-object assert before exitCode, it.each matrix) and the author confirmed fail-before with USE_SYSTEM_BUN=1. No CODEOWNERS cover these paths.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 9:05 AM PT - Aug 12th, 2026

@robobun, your commit f35d99d is building: #93296

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 1:05 PM PT - Aug 12th, 2026

@robobun, your commit f35d99d has some failures in Build #93296 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 37856

That installs a local version of the PR into your bun-37856 executable, so you can run:

bun-37856 --bun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant