Skip to content

fix(webcore): close reader when sliced Bun.file stream reaches max_size - #27213

Closed
robobun wants to merge 2 commits into
mainfrom
claude/fix-sliced-bunfile-stream-hang
Closed

fix(webcore): close reader when sliced Bun.file stream reaches max_size#27213
robobun wants to merge 2 commits into
mainfrom
claude/fix-sliced-bunfile-stream-hang

Conversation

@robobun

@robobun robobun commented Feb 20, 2026

Copy link
Copy Markdown
Collaborator

What

Bun.file(path).slice(start, end).stream() hangs forever when the underlying file is larger than 640 KiB.

import { writeFileSync } from "fs";
writeFileSync("zeroes", Buffer.alloc(768 * 1024));
await Bun.readableStreamToText(Bun.file("zeroes").slice(0, 1).stream()); // never resolves

Fixes #18192

Why

FileReader.onReadChunk() tracks max_size for sliced file blobs. When total_readed >= max_size it returned false to stop the low-level read loop, but never closed the underlying BufferedReader. That leaves reader.isDone() == false, so the next onPull() returns .pending — and since regular files are not pollable, nothing ever resolves that pending promise.

The separate if (buf.len == 0) check meant to close the reader was dead code: at that point buf has been truncated to @min(max_size - total_readed, buf.len) where both operands are >= 1.

The 640 KiB threshold comes from PipeReader's 256 KiB pipeReadBuffer and its 128 KiB flush cutoff: the second onPull starts reading at offset 512 KiB, and if more than 128 KiB remain before EOF, onReadChunk is called with .progress (hitting the broken early return) instead of reaching the EOF path that would mark the reader done.

How

In FileReader.onReadChunk():

  • Set close = true on the total_readed >= max_size early return so the deferred reader.close() actually runs.
  • Replace the dead buf.len == 0 check with total_readed >= max_size so the chunk that satisfies the slice is delivered as the final one (close = true; hasMore = false).
  • Fold close into was_done so the async/pending path (Windows file reads, pipes) delivers *_and_done and returns false instead of continuing to read from a closed fd.
  • Return false at the end of the function when close is set, so the synchronous POSIX read loop does not issue another pread against the now-closed fd.

As a side effect the reader now stops after the first chunk that satisfies max_size instead of reading (and discarding) the rest of the file.

Verification

  • bun bd test test/regression/issue/18192.test.ts → 6 pass
  • USE_SYSTEM_BUN=1 bun test test/regression/issue/18192.test.ts → 4 fail (hang on every file > 640 KiB)
  • bun bd test test/js/web/fetch/blob.test.ts test/js/bun/util/bun-file-fd-read.test.ts → all pass
  • bun run zig:check-all → all targets compile

@robobun

robobun commented Feb 20, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 3:01 AM PT - May 12th, 2026

@autofix-ci[bot], your commit 42907fc has 3 failures in Build #53660 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 27213

That installs a local version of the PR into your bun-27213 executable, so you can run:

bun-27213 --bun

@coderabbitai

coderabbitai Bot commented Feb 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: b4ed5e1b-32ce-44bc-9cc6-d016c8d26be4

📥 Commits

Reviewing files that changed from the base of the PR and between 5710239 and 42907fc.

📒 Files selected for processing (1)
  • test/regression/issue/18192.test.ts

Walkthrough

This PR enforces slice max_size during FileReader streaming by tracking bytes read, scheduling closure after the final chunk, and updating completion signalling. It adds regression tests that stream sliced files (including >640KiB) to verify no hangs and correct byte output.

Changes

FileReader stream max_size fix and regression tests

Layer / File(s) Summary
FileReader max_size handling and early termination
src/runtime/webcore/FileReader.zig
The onReadChunk callback now treats max_size as a hard upper bound: it returns immediately when the limit is already satisfied, clamps and counts delivered bytes, marks the final chunk and schedules reader.close(), updates was_done to include the close path, and prevents further reads after close is scheduled.
Regression tests for sliced file streaming
test/regression/issue/18192.test.ts
Adds a test file with harness imports and two concurrent test cases: one asserts slice(0, 1) returns a single byte without hanging across multiple file sizes; the other writes a 1MiB deterministic buffer and validates multiple slice(start, end) streamed ranges byte-for-byte.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely describes the main fix: closing the reader when a sliced Bun.file stream reaches max_size, which directly addresses the root cause of the hang.
Description check ✅ Passed The PR description comprehensively covers both required sections: 'What' explains the bug and the fix, 'Why' provides detailed root cause analysis, and 'How' describes implementation changes with 'Verification' showing testing effort.
Linked Issues check ✅ Passed All objectives from issue #18192 are met: the hang when streaming sliced files > 640 KiB is fixed, closure and final-chunk semantics are restored, and both sync/async read paths are handled correctly.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the sliced file stream hang: FileReader.onReadChunk modifications address the root cause, and the regression test validates the specific issue without introducing unrelated changes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

FileReader.onReadChunk() returned false once total_readed >= max_size
without closing the underlying BufferedReader, leaving reader.isDone()
== false. For regular files (non-pollable), the next onPull() would
then park on a pending promise that nothing ever resolves, hanging the
stream whenever the backing file exceeds 640 KiB (256K pipeReadBuffer +
256K + 128K flush cutoff).

- Set close = true on both the early-return path and when the current
  chunk reaches max_size, so the deferred reader.close() actually runs.
- Fold close into was_done so the async/pending path (Windows, pipes)
  delivers the final chunk as *_and_done and returns false to stop the
  low-level read loop instead of reading from a closed fd.
- Return false at the end of onReadChunk when close is set, so the
  synchronous POSIX read loop does not issue another pread against the
  now-closed fd.

Fixes #18192
@robobun
robobun force-pushed the claude/fix-sliced-bunfile-stream-hang branch from df6a1da to 5710239 Compare May 12, 2026 08:31
@robobun robobun changed the title fix: close reader when sliced Bun.file stream reaches max_size fix(webcore): close reader when sliced Bun.file stream reaches max_size May 12, 2026
@robobun

robobun commented May 12, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto current main (the file had moved from src/bun.js/webcore/ to src/runtime/webcore/) and extended the fix:

  • The previous revision only set hasMore = false when the slice was satisfied; it also needs close = true there so the reader is marked done immediately rather than after one more read.
  • Added was_done = reader.isDone() or close in the pending-promise path so Windows (where file reads are async via libuv and always land in that branch) delivers *_and_done and stops, instead of scheduling another uv_fs_read against a closed handle — this is what caused the previous Windows CI failures.
  • Added an explicit if (close) return false before the fall-through return so the POSIX readWithFn loop doesn't pread the closed fd.
  • Tests now spawn a subprocess so a regression manifests as a clean timeout rather than wedging the test runner.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any bugs, but this activates the previously-dead defer reader.close() path inside onReadChunk (re-entrantly firing onReaderDone while still in the read loop / onPull), and that interaction plays out differently across the read_inside_on_pull, pending-promise, and fall-through paths on POSIX vs Windows — worth a human pass on the close/done ordering, and CI is still showing build-zig failures on the reported commit.

Extended reasoning...

Overview

This PR fixes #18192: Bun.file(path).slice(start, end).stream() hangs forever when the underlying file is larger than ~640 KiB. The change is confined to FileReader.onReadChunk() in src/runtime/webcore/FileReader.zig, plus a new regression test at test/regression/issue/18192.test.ts.

The fix sets close = true when total_readed >= max_size (both on the early-return and on the chunk that satisfies the slice), folds close into was_done for the pending-promise delivery path, and adds if (close) return false before the fall-through return so the low-level read loop stops. The previous if (buf.len == 0) check that was supposed to do this was provably dead code (it sat inside if (buf.len > 0) after a @min of two >= 1 operands).

Security risks

None. This is read-side stream termination logic for local file blobs; no parsing, auth, or untrusted input handling is involved. The change strictly reduces I/O (stops reading once the slice is satisfied instead of reading and discarding the rest of the file).

Level of scrutiny

High. This is core runtime I/O code on a hot path (Bun.file().stream()), and the function being modified is a state machine with three distinct delivery modes (read_inside_on_pull, pending.state == .pending, and the buffered fall-through) that behave differently on POSIX (synchronous pread loop) vs Windows (async libuv). Critically, the defer if (close) this.reader.close() mechanism existed before but was never actually triggered — the only prior site that set close = true was the dead buf.len == 0 check. So this PR is the first time that deferred close actually fires, and on POSIX reader.close()closeHandle()done()finish() + vtable.onReaderDone() runs re-entrantly before onReadChunk's caller resumes. That means FileReader.onReaderDone() (which can call parent().onClose()) executes while onPull is still on the stack with read_inside_on_pull = .js, and then onPull proceeds to return .into_array_and_done. The author's local testing covers this and the related blob/fd-read suites pass, but the close/done ordering across all three paths deserves a human reviewer's eyes.

Other factors

  • The PR was rebased after the file moved from src/bun.js/webcore/ to src/runtime/webcore/, and the robobun CI comment still reports build-zig/build-cpp failures across ~16 platforms for commit 5710239. These are likely the pre-rebase state (patch targeting a moved file) and an autofix commit 42907fc landed afterward, but green CI should be confirmed before merge.
  • The was_done = reader.isDone() or close change correctly covers the pending-promise path (Windows async file reads), but the read_inside_on_pull path still relies on reader.isDone() becoming true synchronously via the deferred close() — which it does on POSIX per PipeReader.zig:111/245-263, but is worth a second look.
  • Regression tests are solid: they exercise the 640 KiB boundary and several slice ranges, and run in subprocesses so a hang surfaces as a clean timeout rather than wedging the runner.

@robobun

robobun commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #31680. Since the Rust port, src/runtime/webcore/FileReader.zig is a non-compiled porting reference — this diff no longer changes the shipped binary. #31680 applies the same window-exhaustion fix to the live implementation (FileReader.rs), extends it to the already-exhausted early return (zero-length windows), and guards the posix drain loops against reading a closed reader's fd.

@robobun robobun closed this Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream on sliced Bunfile doesn't work

1 participant