Skip to content

fetch: deliver response body bytes as a borrowed slice; reserve Content-Length for buffered consumers - #36570

Merged
Jarred-Sumner merged 32 commits into
mainfrom
farm/a9cc8c76/fetch-buffer-handoff-swap
Jul 31, 2026
Merged

fetch: deliver response body bytes as a borrowed slice; reserve Content-Length for buffered consumers#36570
Jarred-Sumner merged 32 commits into
mainfrom
farm/a9cc8c76/fetch-buffer-handoff-swap

Conversation

@robobun

@robobun robobun commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Two changes to the fetch response-body path, together bringing buffered .arrayBuffer() peak to ~1x payload and the 50-concurrent streaming window from ~78 to ~10 MB/stream.

1. Buffered consumers (.arrayBuffer()/.bytes()/.text()) peak at ~1x payload

FetchTasklet::callback fires per socket read, so scheduled_response_buffer accumulated the body via extend_from_slice doubling, leaving up to ~2x over-capacity on the Vec the ArrayBuffer adopts and spiking RSS to ~2-3x. A new is_buffering_body flag (set by on_start_buffering_callback, cleared under the mutex when a ByteStream attaches) gates a one-time reserve_exact(Content-Length) (capped at 256 MiB) so the buffer grows once.

129 MiB body, RSS after await res.arrayBuffer():
  release linux:    377 -> 139 MiB over baseline
  debug+ASAN linux: 420 -> 139 MiB over baseline

2. Body bytes delivered as &[u8]; response_buffer intermediate removed

Previously every caller supplied a *mut MutableString sink that the HTTP client wrote decoded body bytes into, then the callback copied that into its own destination and reset() both buffers (retaining their high-water capacity for the connection's life). Under 50-way contention those retained capacities dominate the per-stream window.

  • InternalState now owns decoded_body: MutableString (was a NonNull<MutableString> back-pointer). Chunked/decompress write there; uncompressed Content-Length appends the recv-buffer slice.
  • HTTPClientResult.body is &'a [u8] (was Option<&'a mut MutableString>). send_progress_update_* lifts decoded_body onto the stack, hands out decoded_body.list.as_slice(), and after the callback returns re-seats it if capacity is under 512 KiB (else drops it).
  • AsyncHTTP::init/init_sync drop the response_buffer parameter; FetchTasklet and S3HttpDownloadStreamingTask drop their response_buffer field entirely and copy result.body straight into their destination under their own mutex.
  • Non-streaming consumers (NetworkTask, S3HttpSimpleTask, RemoteImageDownload, send_sync) keep a local response_buffer and extend_from_slice(result.body) in their callback.
  • FetchTasklet::on_body_received releases scheduled_response_buffer capacity above 512 KiB after each streaming drain instead of clear().
CONCURRENCY=50 MB=64 fetch-to-fetch (bench/fetch/streaming-backpressure.mjs):
  release canary:         peakRssMB 3896, elapsedMs 4727
  debug+ASAN (this PR):   peakRssMB  808, elapsedMs 13761

(Debug is ~3x slower as expected; release numbers to follow from CI artifacts.)

Net: 16 files, +~230/-~440.

Verification

$ USE_SYSTEM_BUN=1 bun test test/js/web/fetch/fetch-leak.test.ts -t "peaks at ~1x"
{"bodyMB":129,"rssBeforeMB":29,"rssAfterMB":406}
(fail) Received: 377

$ bun bd test test/js/web/fetch/fetch-leak.test.ts -t "peaks at ~1x"
{"bodyMB":129,"rssBeforeMB":324,"rssAfterMB":464}
(pass)

body.test.ts (448), fetch-backpressure.test.ts h1/h2 (20), fetch-redirect.test.ts (30), and bun-install.test.ts (178; 14 network-gated fails match release) all pass.


no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/web/fetch/fetch-leak.test.ts

Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

This PR replaces caller-owned HTTP response buffers with internal decoded storage and callback body ownership. It updates HTTP consumers and call sites. Fetch buffering adds capped preallocation and cleanup rules. Tests measure RSS for a 129 MiB arrayBuffer() response.

Changes

HTTP response ownership and buffering

Layer / File(s) Summary
Owned decoded storage and result delivery
src/http/InternalState.rs, src/http/lib.rs
Decoded bodies use internal storage. Results expose borrowed bytes and terminal owned bytes. Retries and redirects no longer require external body-buffer ownership.
AsyncHTTP API
src/http/AsyncHTTP.rs
AsyncHTTP removes response-buffer initialization state. send_sync receives the destination buffer directly.
Fetch buffering
src/runtime/webcore/fetch/FetchTasklet.rs
FetchTasklet tracks buffering state, caps Content-Length preallocation at 256 MiB, appends callback data, and replaces oversized buffers.
Installer and S3 consumers
src/install/NetworkTask.rs, src/runtime/webcore/s3/*
Consumers copy callback body data into task-owned buffers and separate streaming from buffered handling.
HTTP call sites
src/install/npm.rs, src/runtime/cli/*, src/standalone_graph/StandaloneModuleGraph.rs
Synchronous callers pass buffers to send_sync. Asynchronous callers omit the removed response-buffer argument.
Memory regression coverage
test/js/web/fetch/fetch-buffer-peak-fixture.ts, test/js/web/fetch/fetch-leak.test.ts
The fixture measures RSS for arrayBuffer(). The regression test validates a 129 MiB response and RSS bounds.

Possibly related PRs

  • oven-sh/bun#35093: Both PRs modify FetchTasklet response-body completion behavior.
  • oven-sh/bun#36271: Both PRs modify InternalState and HTTPClient response-body buffering and decompression.
  • oven-sh/bun#36318: Both PRs use MutableString response-buffer helpers.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the two main changes: borrowed response-body slices and Content-Length reservation for buffered consumers.
Description check ✅ Passed The description explains the changes, performance impact, implementation details, and verification results, including platform-specific test limitations.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Found 3 issues this PR may fix:

  1. Memory leak when downloading file (both @google-cloud/storage and Bun's S3) #20487 - Memory leak when downloading files via fetch; the repro uses await fetch(...).arrayBuffer() showing RSS growth proportional to download size, which this PR's pre-reservation and capacity-dropping directly addresses
  2. Doing a large amount of fetch requests results in a memory leak #20912 - Bulk fetch requests (9.5k) cause RSS to grow to 2.46GB; the excessive memory from amortized-doubling buffers across many fetches would be significantly reduced
  3. perf: bun fetch() client drives bun node:http server at ~half the throughput oha does (loadgen artifact, not server regression) #35436 - Client-side fetch() throughput bottleneck; reducing buffer copies (swap instead of copy on first chunk) and avoiding over-allocation directly improves fetch client performance

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #20487
Fixes #20912
Fixes #35436

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. fetch: hand off buffered response bodies instead of copying them #31676 - Both modify FetchTasklet::callback to swap instead of copy the first chunk in the buffered-body handoff path, targeting the same goal of reducing peak memory when consuming fetch response bodies via arrayBuffer(); PR fetch: deliver response body bytes as a borrowed slice; reserve Content-Length for buffered consumers #36570 subsumes fetch: hand off buffered response bodies instead of copying them #31676's optimization and adds Content-Length-based pre-reservation on top.

🤖 Generated with Claude Code

@robobun

robobun commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

Overlaps with the swap in #31676. That PR also right-sizes the gzip libdeflate fast path in InternalState.rs, which this one doesn't touch; this one adds the reserve_exact(Content-Length) step, which is what actually brings the uncompressed peak to ~1x (the swap alone is a near no-op because callback() fires per socket read, so scheduled_response_buffer is the thing that grows by doubling). Happy to rebase either way.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/runtime/webcore/fetch/FetchTasklet.rs:2446-2453 — The reserve_exact runs for every non-Ignore receive mode, including streaming (res.body / getReader()). After on_start_streaming_http_response_body_callback takes scheduled_response_buffer (leaving cap=0), the next HTTP-thread callback swaps and reserves ~Content-Length again, and the on_body_received scopeguard's reset() is Vec::clear() so that capacity is held for the rest of the download. Streaming a 200 MiB body now keeps a ~200 MiB scratch allocation resident for the whole stream (plus the over-capacity DrainResult::Owned Vec) — gate the reservation on the buffered path (e.g. skip once http_.enable_response_body_streaming() has fired).

    Extended reasoning...

    What the bug is

    The new pre-reservation in FetchTasklet::callback fires on the else branch at line 2428, which covers every non-Ignore BodyReceiveModeAutoPause, Paused, and BufferAll alike. The streaming consumer path (res.body / getReader()) runs under AutoPause (on_start_streaming_http_response_body_callback transitions Paused → AutoPause at line 1597), so it hits the reserve too. That means a caller who chose streaming precisely to avoid holding the whole body in memory now gets a Content-Length-sized scratch allocation held for the entire download.

    Code path

    Step-by-step for a 200 MiB Content-Length body consumed via res.body.getReader():

    1. First HTTP-thread callback (headers + first packets arrive, before the user has chosen buffered vs streaming): scheduled.list.capacity() == 0 → swap the incoming packet Vec in → body_size is ContentLength(200 MiB), which is > scheduled.capacity()try_reserve_exact grows scheduled_response_buffer to ~200 MiB.
    2. User touches res.bodyon_start_streaming_http_response_body_callback (FetchTasklet.rs:1615-1622): scheduled_response_buffer is non-empty, so core::mem::take hands the 200 MiB-capacity Vec (containing only a few packets of len) to DrainResult::Owned, and leaves scheduled_response_buffer at capacity 0.
    3. Next HTTP-thread callback: scheduled.list.capacity() == 0 again → swap the ~64 KB packet in → 200 MiB > ~64 KBtry_reserve_exact reserves ~200 MiB again.
    4. JS-thread on_body_received (line 719, has_more branch): borrows scheduled_response_buffer.list.as_slice() for bytes.on_data(), then the scopeguard at lines 655-660 calls (*scheduled_buf).reset(). MutableString::reset() (src/bun_core/string/MutableString.rs:252) is self.list.clear(), which retains capacity. So scheduled_response_buffer stays at ~200 MiB capacity for every subsequent packet until the stream ends.

    body_size stays ContentLength(n) on every callback (line 2404 re-reads it from result.body_size, which http/lib.rs to_result() recomputes from the parsed header each time), so step 3's condition keeps being true whenever capacity is 0.

    Why nothing prevents it

    • The reserve is not gated on BodyReceiveMode::BufferAll or on http_.response_body_streaming — only Ignore is special-cased (line 2413).
    • on_start_buffering_callback (the .arrayBuffer() path) does not take scheduled_response_buffer, so the double-reservation in step 3 is specific to the streaming drain callback, which core::mem::takes it.
    • The on_body_received reset uses clear(), not = MutableString::default(), so the second reservation persists.

    Impact

    Before this PR, streaming a 200 MiB Content-Length body kept scheduled_response_buffer at roughly the max coalesced packet burst (~64 KB, bounded by AutoPause). After this PR, it holds a ~200 MiB (capped at 256 MiB) allocation for the entire download — plus the over-capacity DrainResult::Owned Vec parked in reader.context.buffer from step 2. That is a straight regression of the memory profile for the API users reach for on large bodies specifically to avoid this. The PR title says "peak at ~1x payload" for the buffered path; the streaming path went from ~0x to ~1x.

    Fix

    Gate the try_reserve_exact on the buffered path — e.g. only reserve when the receive mode is BufferAll, or skip it once http_.enable_response_body_streaming() has been called / self.is_waiting_body and streaming has attached. The swap-instead-of-copy is fine for both modes; only the Content-Length reservation needs the gate.

Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
@robobun

robobun commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

Good catch on the streaming regression. Gated the reserve_exact on BodyReceiveMode::BufferAll in 2d2f3ab; the swap stays for both modes since it only ever moves a packet-sized Vec. Streaming a 128 MiB body under debug+ASAN now matches main (peakDelta 143 vs 147 MiB, both pre-existing streaming overhead), and the buffered .arrayBuffer() test still passes at ~1.08x.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/web/fetch/fetch-leak.test.ts`:
- Around line 967-1028: Expand the fetch memory regression coverage around the
existing large-body test to exercise the sibling consumers bytes() and text()
with method-appropriate RSS assertions, and add a controlled case where
Content-Length exceeds the 256 MiB cap. Replace the current explanatory comments
with the relevant issue URL, or remove them if no issue URL exists, while
preserving the existing arrayBuffer() behavior and test setup.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: d295f165-9911-4933-b80c-36b7c894e2e9

📥 Commits

Reviewing files that changed from the base of the PR and between 5b7c3ca and 2d2f3ab.

📒 Files selected for processing (3)
  • src/runtime/webcore/fetch/FetchTasklet.rs
  • test/js/web/fetch/fetch-buffer-peak-fixture.ts
  • test/js/web/fetch/fetch-leak.test.ts

Comment thread test/js/web/fetch/fetch-leak.test.ts

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline finding, I also checked: the core::mem::swap is safe w.r.t. the HTTP client's body_out_str — it's a *mut MutableString pointing at the response_buffer field, so swapping the field contents leaves the pointer valid and the client writes into the (now-empty) swapped-in Vec on the next packet. The !success branch still clears response_buffer on every path (reset if has_more, drop-capacity otherwise). Deferring to a human on the overlap with #31676 and the un-branched 1.5x RSS threshold in the new test.

Extended reasoning...

The inline nit (BufferAll gate not excluding the unobserved-ReadableStream setter at drop_backpressure_if_unobserved) is real — MutableString::reset() is Vec::clear(), so the reserved capacity would sit idle at len≈0 for the whole download on that path. It's bounded (256 MiB cap) and the triggering pattern is unusual, so it's correctly filed as a nit rather than a blocker. Separately confirmed the swap doesn't invalidate body_out_str (field-address aliasing, not heap-pointer) and that the !success reset is preserved. This is a hot-path change with an overlapping open PR and an RSS-threshold test that isn't branched on isASAN/isDebug, so a maintainer should look.

Comment thread src/runtime/webcore/fetch/FetchTasklet.rs Outdated
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs
robobun added 5 commits July 31, 2026 13:58
…ffer()/bytes() peak at ~1x payload

FetchTasklet::callback fires per socket read (signals.body_receive_mode
is always set for fetch), so scheduled_response_buffer accumulates the
body via extend_from_slice and grows by amortized doubling. The
resulting over-capacity Vec is what the ArrayBuffer/Uint8Array adopts,
and the intermediate reallocations push peak RSS to ~2-3x the payload.

At the handoff, swap the first chunk into scheduled instead of copying,
then reserve_exact the known Content-Length once (capped at 256 MiB) so
the buffer grows to the final size in one step. On completion drop
response_buffer's capacity instead of clearing.

129 MiB body, RSS after await res.arrayBuffer():
  release linux:    377 -> 139 MiB over baseline
  debug+ASAN linux: 420 -> 139 MiB over baseline
Streaming consumers drain scheduled_response_buffer per chunk; reserving
the full body there would hold a body-sized scratch allocation for the
whole download.
…mode

drop_backpressure_if_unobserved also sets BufferAll while still delivering
per chunk to a ByteStream; reserving there would hold an idle body-sized
allocation in scheduled_response_buffer. Track the actual buffered-consumer
path with an AtomicBool written by on_start_buffering_callback.
@robobun
robobun force-pushed the farm/a9cc8c76/fetch-buffer-handoff-swap branch from 4f61bf5 to a8b1e0a Compare July 31, 2026 14:01

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/runtime/webcore/fetch/FetchTasklet.rs:2450-2458 — The is_buffering_body gate still doesn't exclude every path where a ByteStream is attached: ValueBufferer::buffer_locked_body_value (Body.rs:2582-2585, reached via new HTMLRewriter().transform(await fetch(url))) calls on_start_buffering_callback (setting is_buffering_body = true) and then immediately materializes a ByteStream via to_readable_stream()on_readable_stream_available. From there on_body_received delivers each chunk to the ByteStream and reset()s (i.e. clear()s, capacity retained) scheduled_response_buffer, so this reserve grows it to min(Content-Length, 256 MiB) and it sits idle at len≈0 for the whole download — the same regression the drop_backpressure_if_unobserved comment flagged, now reached through the flag its fix introduced. The predicate the reserve actually needs is "no ByteStream attached" (e.g. also require self.readable_stream_ref unset, or have on_readable_stream_available/on_start_streaming clear is_buffering_body).

    Extended reasoning...

    What the bug is

    Commit 4f61bf5 replaced the BufferAll gate with a new is_buffering_body: AtomicBool on the assumption that only on_start_buffering_callback sets it, and that firing implies the body will accumulate in scheduled_response_buffer. That does hold for the primary caller (Body.rs:413-417, the .arrayBuffer()/.text()/.bytes() path) and for Bun.write(file, response) (Blob.rs, which sets on_receive_value and never materializes a stream). But on_start_buffering has a third caller: ValueBufferer::buffer_locked_body_value at Body.rs:2582-2585, and that one does materialize a ByteStream immediately after setting the flag.

    Step-by-step trace

    For new HTMLRewriter().on(...).transform(await fetch(url)) on a fresh Response with Content-Length: 129 MiB:

    1. html_rewriter.rs:724-751get_body_readable_stream() returns None (Body.rs:1671-1687 only reads the JS cache and locked.readable, both unset for a fresh fetch Response) → ValueBufferer::run(value, None)buffer_locked_body_value(value, None).
    2. Body.rs:2482-2497: locked.readable is None and owned_readable_stream is Nonereadable_stream = None, fall through.
    3. Body.rs:2579: locked.task.is_some() (the FetchTasklet ptr, set at FetchTasklet.rs:1731) → true.
    4. Body.rs:2582-2585: on_start_buffering(task) = FetchTasklet::on_start_buffering_callbackis_buffering_body.store(true) and set_receive_mode_terminal(BufferAll).
    5. Body.rs:2588-2590: value.to_readable_stream()locked_to_native_stream → calls on_start_streaming_http_response_body_callback (FetchTasklet.rs:1620 mem::takes scheduled_response_buffer, leaving it at capacity 0) and then on_readable_stream_available (Body.rs:917-918 → FetchTasklet.rs:1581 sets this.readable_stream_ref).
    6. Body.rs:2596: recursive call now finds locked.readable set → Source::Bytes branch (2519) → sets byte_stream.pipe.
    7. Next HTTP-thread callback() at line 2438: scheduled.list.capacity() == 0 → swap; then is_buffering_body == true and ContentLength(129 MiB) and 129 MiB > captry_reserve_exact grows scheduled_response_buffer to ~129 MiB (bounded by SCHEDULED_PRERESERVE_MAX = 256 MiB).
    8. Every subsequent JS-thread on_body_received at FetchTasklet.rs:717-736 sees readable_stream_ref set → delivers the chunk to the ByteStream via temporary_chunk, returns without buffer_reset.set(false), so the scopeguard at line 663 runs (*scheduled_buf).reset() = Vec::clear() (MutableString.rs:252-254) — len → 0, capacity retained.

    Why the gate doesn't prevent it

    is_buffering_body was introduced to mean "on_start_buffering_callback fired", but that callback does not uniquely imply "body accumulates in scheduled_response_buffer": one of its three callers immediately materializes a ByteStream and routes chunks there instead. Nothing clears is_buffering_body when readable_stream_ref is later set. The predicate the reserve actually needs — as the earlier review comment already suggested — is "no ByteStream attached".

    Impact

    On main, scheduled_response_buffer capacity in this path stays at ~one socket read (a few tens of KB); with this PR it grows to min(Content-Length, 256 MiB) and sits idle at len≈0 for the entire download while the ValueBufferer's stream_buffer independently accumulates the same bytes. For HTMLRewriter().transform(await fetch(url)) on a large body that is a per-fetch memory regression of up to 256 MiB versus main — exactly the failure mode the earlier drop_backpressure_if_unobserved review identified, reached through a different setter of the very flag that comment's fix introduced. Since the PR's whole purpose is reducing peak memory, regressing it on a real user-facing path should block.

    Fix

    Either tighten the predicate to also require no ByteStream is attached:

    if task_ref.is_buffering_body.load(Ordering::Acquire)
        && task_ref.readable_stream_ref.get().is_none()
    {

    or clear the flag when a stream is attached (in on_readable_stream_available or on_start_streaming_http_response_body_callback):

    this.is_buffering_body.store(false, Ordering::Release);

    The second option also handles the case where the ByteStream is attached mid-download after buffering has begun.

ValueBufferer (HTMLRewriter.transform) calls on_start_buffering then
immediately materializes a ByteStream; without this the reserve would
hold an idle body-sized capacity on scheduled_response_buffer while the
stream drains it per chunk.
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs
Comment thread src/http/lib.rs
Comment thread src/http/lib.rs
Comment thread src/http/lib.rs
Comment thread src/http/lib.rs
Comment thread src/http/lib.rs
Comment thread src/http/lib.rs
Comment thread src/runtime/webcore/fetch/FetchTasklet.rs
@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun two things before review:

  1. cargo clippy is failing on this head (https://github.com/oven-sh/bun/actions/runs/30666683045/job/91275258496) — please fix the lints.
  2. Rebase onto current main (f68e504afetch/S3: replace ResumableSink with proper JSSinks #36087 merged) and force-push so this is measured against the merged backpressure work.

After the rebase, please re-run and post the two measurements this PR is judged on, since the slice-passing rework changed the receive path after the last numbers were taken (those were on 51103b419):

  • buffered peak: fetch().arrayBuffer() at 128/256/384 MB, peak RSS ratio to payload (target ≤ ~1.15× up to the reserve cap);
  • streaming window: bench/fetch/streaming-backpressure.mjs fetch-to-fetch with CONCURRENCY=50 MB=256 on bun-fuzz, peak RSS + elapsed vs canary and node 26.5.1 (target ≈ 350 MB or better, throughput not below the pre-rebase run).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/runtime/webcore/fetch/FetchTasklet.rs (1)

2600-2623: 🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win

Avoid reserving before adopting an owned terminal chunk.

When body_owned is the first non-empty chunk, scheduled.list is empty. Lines 2608-2614 first allocate the capped Content-Length capacity. Line 2622 then replaces that vector with body_owned. Both allocations are live during the assignment, which can restore a roughly 2× payload peak for a single terminal callback.

Skip the reservation when the direct adoption path applies.

Proposed fix
             let scheduled = &mut task_ref.scheduled_response_buffer;
+            let can_adopt_owned_body =
+                body.is_empty() && !body_owned.is_empty() && scheduled.list.is_empty();
 
-            if task_ref.is_buffering_body.load(Ordering::Acquire) {
+            if !can_adopt_owned_body
+                && task_ref.is_buffering_body.load(Ordering::Acquire)
+            {

Add a regression case for a response that completes in one callback with a large Content-Length.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/runtime/webcore/fetch/FetchTasklet.rs` around lines 2600 - 2623, Update
the reservation guard in the success path around scheduled_response_buffer so
Content-Length pre-reservation is skipped when body_owned is non-empty and
scheduled.list is empty, allowing the terminal chunk to be adopted directly
without a second allocation. Preserve reservation behavior for other buffering
cases, and add a regression test covering a large Content-Length response
completed in one callback.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/runtime/webcore/fetch/FetchTasklet.rs`:
- Around line 2600-2623: Update the reservation guard in the success path around
scheduled_response_buffer so Content-Length pre-reservation is skipped when
body_owned is non-empty and scheduled.list is empty, allowing the terminal chunk
to be adopted directly without a second allocation. Preserve reservation
behavior for other buffering cases, and add a regression test covering a large
Content-Length response completed in one callback.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 92709fdd-d2a1-43d3-a3ab-7aff01bb2e2a

📥 Commits

Reviewing files that changed from the base of the PR and between cf1f11f and 48b9d2a.

📒 Files selected for processing (7)
  • src/http/AsyncHTTP.rs
  • src/http/lib.rs
  • src/install/NetworkTask.rs
  • src/runtime/cli/run_command.rs
  • src/runtime/webcore/fetch/FetchTasklet.rs
  • src/runtime/webcore/s3/download_stream.rs
  • src/runtime/webcore/s3/simple_request.rs

@robobun

robobun commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 3:07 PM PT - Jul 31st, 2026

@robobun, your commit 637d54b6b61ffa50cdd672ab01b9e1c8d1a59607 passed in Build #86442! 🎉


🧪   To try this PR locally:

bunx bun-pr 36570

That installs a local version of the PR into your bun-36570 executable, so you can run:

bun-36570 --bun

@robobun

robobun commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

Clippy fixed in 7ba280e. Branch is already based on f68e504 (git merge-base HEAD origin/main = f68e504).

Buffered peak (fetch().arrayBuffer(), RSS delta / payload)

Measured on linux-x64 at 637d54b, debug+ASAN build vs release canary 1498d7b:

body canary this PR
128 MiB 2.23x 1.07x
256 MiB 2.34x 1.04x
384 MiB 2.79x 1.02x

All under the 1.15x target, including 384 MiB (past the 256 MiB reserve cap; the cap only bounds how far ahead of received bytes the reservation runs, so larger bodies still grow tight).

Streaming window (CONCURRENCY=50 MB=256 fetch-to-fetch)

linux-x64 container, same binaries:

peakRssMB elapsedMs MB/stream
release canary 1498d7b 13858 22454 ~277
this PR (debug+ASAN) 802 53292 ~9.5

Window flat vs payload (802 MB at MB=64 too). Debug+ASAN is ~2.4x slower, which is the expected debug-build overhead, not a throughput regression in the hot path.

I can't reach bun-fuzz from this container (not on the farm-tailscale routes I have), so no release-build apples-to-apples yet. Once the linux-x64 release artifact lands from build 86444 I'll pull it and re-run here, or if you can kick ~/run50.sh on bun-fuzz against this head that's the cleaner number.

@Jarred-Sumner
Jarred-Sumner merged commit 5f65d37 into main Jul 31, 2026
54 checks passed
@Jarred-Sumner
Jarred-Sumner deleted the farm/a9cc8c76/fetch-buffer-handoff-swap branch July 31, 2026 23:47
Jarred-Sumner pushed a commit that referenced this pull request Aug 1, 2026
#36588)

Shrinks the `node:http` proxy memory window
(`pipeline(Readable.fromWeb((await fetch(upstream)).body), res)`) by
keeping the upstream socket paused while `ByteStream` still has spilled
bytes.

## Repro

```
MB=64 CONCURRENCY=50 STALL=0 <bin> bench/fetch/streaming-backpressure.mjs download-proxy
```

| | peak RSS | elapsed |
|---|---|---|
| main (`df49a6e1c`, after #36570) | 277 MB | 4.13 s |
| this PR | **226 MB** | **3.92 s** |
| node 26.3 | 204 MB | 5.80 s |

At `CONCURRENCY=40 MB=32` (the shape the RSS test uses): main 241 MB,
this PR 197 MB.

## Cause

`ByteStream::on_data` called `signal_drained()` after filling a pending
pull view even when the tail of the chunk spilled into `buffer`. That
resumed the upstream socket with data still queued; the next recv landed
with no reader waiting (`pipeline` was paused on `res.write() == false`)
and went straight to `ByteStream.buffer`. Instrumented on main at C=50,
`on_data` hit the no-pending-pull append path 1686 times; with this
change, zero.

`FetchTasklet.scheduled_response_buffer` growth from the same pattern
was already capped by #36570 (which drops the staging allocation once it
passes `DECODED_BODY_RETAIN_CAP`), so the `ByteStream.buffer` overflow
is what remains.

## Fix

`on_data` only calls `signal_drained()` when the whole chunk fit the
pull view. When it spilled, the next `on_pull` signals once it drains
`buffer`, so the producer stays paused until the reader has caught up.

## Tests

`test/js/web/fetch/fetch-backpressure.test.ts` gains a
`Readable.fromWeb` stalled-pipe drain test (all lanes) and a
release-Linux-only RSS bound for the `download-proxy` shape (fails on
main with peak ~241 MB, passes here at ~197 MB). ASAN quarantine
dominates the per-conn buffer delta, so the RSS bound is skipped under
debug/ASAN; the drain test covers the path there.

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 1 · Platform-specific test(s) that do not
run on this machine. Deferring to CI, which covers all platforms:
test/js/web/fetch/fetch-backpressure.test.ts

<!-- robobun:evidence:end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants