Skip to content

Bun.write: don't cap a file-to-file copy at the destination's cached size - #37877

Open
robobun wants to merge 1 commit into
mainfrom
farm/cd01cad1/bun-write-dest-cached-size
Open

Bun.write: don't cap a file-to-file copy at the destination's cached size#37877
robobun wants to merge 1 commit into
mainfrom
farm/cd01cad1/bun-write-dest-cached-size

Conversation

@robobun

@robobun robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • Bun.write(Bun.file(dest), Bun.file(src)) copies only as many bytes as dest used to have, once anything has read dest's size (.size, exists(), toHaveLength(), structured clone). With the usual exists() guard on a missing file, the copy leaves an empty file behind.
  • Only file to file is affected; string, buffer and Blob sources never look at the destination's size. Reproduced on 1.4.0 on Linux; the macOS and Windows copy paths consume the same value, and Windows also padded a shorter copy back out to the old length.
  • Cause: a file-backed blob keeps one size field for two things, a slice() window and the cached stat size. The copy treated whatever was in it as a window, so a destination that had merely been stat'ed was written as if it were dest.slice(0, oldSize).
  • Fixes Calling BunFile.exists makes Bun.write write nothing #4930
  • Fixes Bun.write do short write when copying file #22456

Fix

  • The blob now records whether its size came from slice(). The copy is bounded by the destination only in that case; otherwise it is bounded by the source, as a fresh Bun.file(dest) or a path destination already was.
  • Structured clone puts a size on the wire only for sliced blobs and treats a received size as a window, so a clone of a stat'ed whole-file blob stays unbounded and a clone of a real slice keeps its window. No wire format change.
  • A destination the caller sliced behaves exactly as before; only the cases that lost data change.
  • Verification: new tests prime the destination each of the six ways, plus the missing-file guard, an fd destination, a shrinking copy and sliced destinations. 14 of 17 fail on the release bun, all pass with the fix. The returned byte count is asserted only on Linux; macOS and Windows resolve with 0 over an existing file regardless of this change (Bun.write: fix file-to-file copy resolving 0 bytes on Windows and macOS overwrite #33715).

Background

  • Bun.file(path) is a lazy blob: it holds a path or fd and does not stat the file until asked. .size, exists(), toHaveLength() and structured clone all stat it and cache the size on that blob object.
  • blob.slice(start, end) is a new blob over a byte window of the same file. As a Bun.write destination it means "write into these bytes only", and existing tests rely on it capping a copy.
  • Before the stat, the size field holds MAX_SIZE, the sentinel for unknown. The per-platform copy routines (Linux, macOS, Windows) take a max length, and MAX_SIZE there means the source decides; all three were being handed the destination's size field.
Original description

Bun.write(Bun.file(dest), Bun.file(src)) copies only as many bytes as dest used to have once anything has resolved dest's size. The common shape is an exists() guard, which leaves an empty file behind (#4930, open since 1.0); the other is overwriting a shorter file after exists() / .size (#22456).

Repro

import fs from "fs";
fs.writeFileSync("src.bin", Buffer.alloc(300000, "S"));
fs.writeFileSync("dest.bin", "short");

const f = Bun.file("dest.bin");
f.size;                                               // or: await f.exists()
console.log(await Bun.write(f, Bun.file("src.bin"))); // 5, expected 300000
console.log(fs.statSync("dest.bin").size);            // 5, expected 300000

const out = Bun.file("missing.bin");
await out.exists();                                   // false
await Bun.write(out, Bun.file("src.bin"));
console.log(fs.statSync("missing.bin").size);         // 0: the #4930 shape

Without the .size / exists() line both copies are complete. Only the file to file path is affected; string, buffer and Blob sources go through WriteFile, which never looks at the destination's size. Reproduces on 1.4.0 on Linux; the macOS (clonefile / fcopyfile + ftruncate) and Windows (CopyFileWindows::on_complete truncate) arms consume the same value. On Windows the cached size also works the other way round: copying a shorter file into a longer one padded it back out to the old length.

Cause

Blob.size holds two different things for a file-backed blob: the window set by slice(), and, once resolve_size() has run, the file's stat size. .size, exists(), expect(f).toHaveLength() and structured-clone serialization all run resolve_size() on the blob itself. write_file_with_source_destination (src/runtime/webcore/Blob.rs) then handed destination_blob.size to CopyFile::create / CopyFileWindows::init as the copy length, so a destination whose stat size merely got cached was copied into as if the caller had written Bun.file(dest).slice(0, oldSize).

Fix

Add Blob::size_is_explicit, set by slice() (and by dupe() / the other field-copy), and have the copy take the destination window only when it is set; otherwise it passes MAX_SIZE and the copy is bounded by the source like a fresh Bun.file(dest) or a path destination already is. slice() with no bounds on an unresolved file blob produces MAX_SIZE, which is not a window, so that case stays unbounded too.

Structured clone writes the v4 size field only for sliced blobs (the format already defines MAX_SIZE there as "receiver stats it itself") and sets the flag when it reads a concrete one. Without that, a clone of a primed whole-file blob would arrive looking like a slice of the old length, and a clone of a real slice would lose its window under the new check. No wire format change.

An explicitly sliced destination behaves exactly as before (bun-write.test.js already relies on Bun.file(fd).slice(0, 5) and Bun.stdout.slice(0, 100) capping the copy), so unlike the attempts that changed what a destination slice means, this only changes the cases that were losing data. It is also independent of which size ends up being consulted later: #31515 / #32859 (copy the source slice instead) would make the call-site hunk here redundant, and #33360 / #33659 add the same field for the read paths and the getters; whichever of those lands second has a one-line conflict on the field declaration.

Verification

test/js/bun/io/bun-write.test.js, new describe next to the existing file to file test: the six ways of priming the destination (.size, exists(), toHaveLength(), serializing it, writing to a clone of a primed blob, writing to a whole-file slice()) against a shorter and an empty existing file, the missing-destination exists() guard from #4930, an fd destination, a shorter source over a longer destination, and guards that slice(0, 4) and structuredClone(slice(0, 4)) destinations are still bounded. The resolved byte count is only asserted on Linux because uv_fs_copyfile and fcopyfile over an existing file resolve with 0 independently of this (#33715).

$ USE_SYSTEM_BUN=1 bun test test/js/bun/io/bun-write.test.js -t "not capped by the destination"
 3 pass, 14 fail
$ bun bd test test/js/bun/io/bun-write.test.js -t "not capped by the destination"
 17 pass, 0 fail

Also green with the debug build: the rest of bun-write.test.js (the subprocess tests there only time out when run concurrently under ASAN here, same as on main), structured-clone-blob-file.test.ts, blob.test.ts, blob-write.test.ts, blob-cow.test.ts, bun-file*.test.ts, bun-serve-file.test.ts and the serve.test.ts Content-Range suite. cargo check -p bun_runtime passes for x86_64-pc-windows-msvc.

Fixes #4930
Fixes #22456

…size

Bun.write(Bun.file(dest), Bun.file(src)) passed the destination blob's
size to CopyFile / CopyFileWindows as the number of bytes to copy. For an
unsliced Bun.file() that field is only the stat size cached by .size,
exists(), toHaveLength() or structuredClone(), so once any of those had
run the copy was cut to the destination's previous length, and the
exists() guard pattern produced an empty file.

Record on the blob whether size is a window the caller asked for with
slice(); the copy only takes the window in that case and otherwise copies
the whole source. Structured clones carry the window only for sliced
blobs, so a clone of a primed whole-file blob stays a whole-file blob and
a clone of a slice stays a slice.

Fixes #4930
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 24 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: d8b6d6ea-2b40-451d-83f7-34e9c948da8f

📥 Commits

Reviewing files that changed from the base of the PR and between 9a543cc and 5f4d4c8.

📒 Files selected for processing (3)
  • src/jsc/webcore_types.rs
  • src/runtime/webcore/Blob.rs
  • test/js/bun/io/bun-write.test.js

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Reproduced on released 1.4.0 (Linux) with the repro in the description and with the original scripts from #4930 and #22456: after .size / exists() on the destination, Bun.write(dest, Bun.file(src)) copies dest's old length (0 bytes for a missing file). Both scripts produce the full copy with this branch.

Fix: src/runtime/webcore/Blob.rs (write_file_with_source_destination) only passes the destination window to the copy when it came from slice(), tracked by the new Blob::size_is_explicit. Tests: test/js/bun/io/bun-write.test.js, 14 of the 17 new cases fail on 1.4.0 and all pass with the fix.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 2:05 PM PT - Aug 12th, 2026

@robobun, your commit 5f4d4c8 has 3 failures in Build #93369 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 37877

That installs a local version of the PR into your bun-37877 executable, so you can run:

bun-37877 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it adds a new discriminator field to the core Blob struct and changes file→file copy semantics across all three platforms — and the description notes several overlapping in-flight PRs (#31515 / #32859 / #33360 / #33659) touching the same size semantics — a human look at the approach would still be worthwhile.

What was reviewed:

  • All three Blob literal sites (Default, dupe_with_content_type, and the inline copy at Blob.rs:3316) initialize the new field; Rust would fail the build otherwise.
  • get_slice_from unconditionally overwrites the dupe()'d flag with len != MAX_SIZE, so a whole-file slice() on an unresolved blob correctly stays unbounded and a bounded re-slice stays bounded.
  • Structured-clone serialize now writes MAX_SIZE for unsliced blobs and deserialize sets the flag on a concrete size — no wire-format change, and the round-trip guard tests cover both directions.
  • The Windows and POSIX arms of write_file_with_source_destination both consume the new max_length.
Extended reasoning...

Overview

The PR fixes #4930 / #22456: Bun.write(Bun.file(dest), Bun.file(src)) was capped to the destination's cached stat size once anything (.size, exists(), toHaveLength, structured clone) had run resolve_size() on the destination blob. It adds Blob::size_is_explicit: Cell<bool> in src/jsc/webcore_types.rs, sets it in slice() / dupe() / the one other field-copy site / structured-clone deserialize, gates the v4 structured-clone size field on it, and has write_file_with_source_destination in src/runtime/webcore/Blob.rs pass MAX_SIZE to CopyFile::create / CopyFileWindows::init unless the destination was explicitly sliced. 17 new test cases in test/js/bun/io/bun-write.test.js cover the primer matrix, the #4930 missing-destination shape, an fd destination, the Windows over-truncate direction, and guards that a real slice(0, 4) (and its clone) still bounds the copy.

Security risks

None identified. This is a data-loss correctness fix in the file-copy path; no untrusted-input parsing, auth, or privilege boundaries are touched. The new flag defaults to false (unbounded copy, i.e. bounded by the source), which is the pre-existing behavior for a fresh Bun.file(dest).

Level of scrutiny

Medium-high. The diff itself is small and the mechanism is a single boolean discriminator, but it changes semantics in the core Blob type that flows through every file-backed I/O path on Linux, macOS, and Windows, and it adjusts what structured-clone puts on the wire (within the existing v4 format). The PR description explicitly calls out four other open PRs converging on the same Blob.size ambiguity with different approaches; a maintainer should confirm this is the direction they want relative to those.

Other factors

The bug hunting system found nothing. I confirmed every Blob { .. } literal in the tree carries the new field (grep on the adjacent is_jsdom_file: line), that get_slice_from overwrites the flag after dupe() so the parent's value can't leak through incorrectly, and that both the #[cfg(windows)] and non-Windows arms consume max_length. Test coverage is thorough (six primers × two existing-file states, plus fd, missing-file, shrink, and slice-still-bounds guards) and follows harness conventions (tempDir, Buffer.alloc over .repeat, isLinux gate on the resolved byte count with a stated reason). The existing Bun.file(fd).slice(0, 5) and Bun.stdout.slice(0, 100) tests in the same file continue to exercise the bounded path. Given the cross-platform reach and the coordination question with the sibling PRs, deferring to a human reviewer rather than auto-approving.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calling BunFile.exists makes Bun.write write nothing Bun.write do short write when copying file

1 participant