s3: emit HTTP suffix/open-ended Range for S3File.slice(-n) and slice(a) by robobun · Pull Request #33687 · oven-sh/bun

robobun · 2026-07-07T16:39:21Z

Repro

using server = Bun.serve({
  port: 0,
  fetch(req) {
    console.log(req.headers.get("range"));
    // RFC 7233: first-byte-pos past the object length is 416
    return new Response(
      '<?xml version="1.0"?><Error><Code>InvalidRange</Code><Message>not satisfiable</Message></Error>',
      { status: 416 },
    );
  },
});
const c = new Bun.S3Client({ endpoint: server.url.href, bucket: "b", accessKeyId: "x", secretAccessKey: "x" });
await c.file("k").slice(-5).bytes();
// Range: bytes=4503599627370490-4503599627370494
// S3Error: not satisfiable  code=InvalidRange

s3.file(k).slice(-n).bytes() is the Blob idiom for reading the tail of an object (footers, zip central directories). It always fails against AWS S3 / MinIO / R2 with InvalidRange. slice(a) with no end sends bytes=a-4503599627370494, which only works because servers clamp the last-byte-pos.

Cause

Blob::get_slice applies the W3C relative clamp against self.size, which for a lazy S3 blob is the MAX_SIZE sentinel (2^52-1). slice(-5) becomes relative_start = MAX_SIZE - 5, and that number is serialised straight into the Range header. The in-memory Blob path is correct because its size is always known.

Fix

When slicing an S3 blob whose size is still the sentinel and no end is given, encode the two forms HTTP already supports without knowing the size:

slice(-n) stores offset = MAX_SIZE, size = n and is serialised as bytes=-n
slice(a) keeps size = MAX_SIZE and is serialised as bytes=a-

The duplicated range-builder in download_slice / download_stream is extracted into one range_header() helper that handles all three forms. get_slice_from now takes the final (offset, size) pair so the caller owns the arithmetic.

Bun.file() has the same sentinel leak (the sibling fix in #33601 resolves the size via stat); S3 cannot stat synchronously, so the encoding approach is used instead. Scoped to is_s3() so file-backed slice behaviour is unchanged here.

Verification

USE_SYSTEM_BUN=1 bun test test/js/bun/s3/s3-slice-range.test.ts   # 16 fail (InvalidRange / bytes=...4503599627370494)
bun bd test test/js/bun/s3/s3-slice-range.test.ts                 # 32 pass
bun bd test test/js/web/fetch/blob.test.ts                        # 26 pass

The new test covers .bytes() / .text() / .arrayBuffer() / .stream() against an RFC 7233 mock server and asserts both the exact Range header sent and the bytes returned.

[review] gate passed · iteration 5 · 3 files touched

fails on main (without fix)

ASAN without fix: 20 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" "test/js/bun/s3/s3-slice-range.test.ts"
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (3c7052031)

test/js/bun/s3/s3-slice-range.test.ts:
75 |     stdout: "pipe",
76 |     stderr: "pipe",
77 |   });
78 |   const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
79 |   if (exitCode !== 0) {
80 |     throw new Error(`exit ${exitCode}\nstdout: ${stdout}\nstderr: ${stderr}`);
                   ^
error: exit 1
stdout: {"range":"bytes=4503599627370490-4503599627370494"}

stderr: S3Error: The requested range is not satisfiable
 path: "k",
 code: "InvalidRange"


Bun v1.4.0-debug+3c7052031 (Linux x64)

      at run (/workspace/bun/test/js/bun/s3/s3-slice-range.test.ts:80:15)
      at async <anonymous> (/workspace/bun/test/js/bun/s3/s3-slice-range.test.ts:92:20)
(fail
... (truncated)

release without fix: all passed
bun test v1.4.0-canary.1 (b6fa2d411)

test/js/bun/s3/s3-slice-range.test.ts:
(pass) S3File.slice() Range header > via .bytes() > slice(0) with no end fetches the whole object (no Range header) [28.46ms]
(pass) S3File.slice() Range header > via .bytes() > slice(6, 10) still sends an absolute range [32.08ms]
(pass) S3File.slice() Range header > via .bytes() > slice(0, 5) still sends an absolute range [33.30ms]
(pass) S3File.slice() Range header > via .bytes() > slice() with no args fetches the whole object (no Range header) [34.45ms]
(pass) S3File.slice() Range header > via .bytes() > slice(-5) sends a suffix range [42.84ms]
(pass) S3File.slice() Range header > via .bytes() > slice(200) with no end sends an open-ended range [42.50ms]
(pass) S3File.slice() Range header > via .bytes() > slice(-1) sends a suffix range [47.15ms]
(pass) S3File.slice() Range header > via .bytes() > slice(-500) sends a suffix range [45.99ms]
(pass) S3File.slice() Range header > via .text() > slice(-1) sends a suffix range [42.84ms]
(pass) S3File.slice() Range header > via .text() > slice(0) with no end fetches the whole object (no Range header) [36.44ms]
(pass) S3File.slice() Range header > 
... (truncated)

passes on PR (with fix)

ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" "test/js/bun/s3/s3-slice-range.test.ts"
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (3c7052031)

test/js/bun/s3/s3-slice-range.test.ts:
(pass) S3File.slice() Range header > via .bytes() > slice(-1) sends a suffix range [1310.60ms]
(pass) S3File.slice() Range header > via .bytes() > slice(-500) sends a suffix range [1301.28ms]
(pass) S3File.slice() Range header > via .bytes() > slice(6, 10) still sends an absolute range [1310.35ms]
(pass) S3File.slice() Range header > via .bytes() > slice(200) with no end sends an open-ended range [1358.66ms]
(pass) S3File.slice() Range header > via .bytes() > slice(-5) sends a suffix range [1377.08ms]
(pass) S3File.slice() Range header > via .bytes() > slice(0, 5) still sends an absolute range [1279.33ms]
(pass) S3File.slice() Range header > via .bytes() > slice(0) with no end
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
[configured] bun-profile → bun (stripped) in 712ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/6] gen generated_host_exports.rs
generated_host_exports.rs: 90 exports (host=3, lazy=10, generic=77, rust=0); 254 extern-C blocks audited
[1/6] cargo bun_bin → libbun_rust.a (--target x86_64-unknown-linux-gnu)
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: component rust-std is up to date

  nightly-2026-05-06-x86_64-unknown-linux-gnu unchanged - rustc 1.97.0-nightly (e95e73209 2026-05-05)

info: checking for self-update (current version: 1.29.0)
�[1m�[92m   Compiling�[0m bun_core v0.0.0 (/workspace/bun/src/bun_core)
�[1m�[92m   Compiling�[0m bun_paths v0.0.0 (/workspace/bun/src/paths)
�[1m�
... (truncated)

diff hotspot

src/runtime/webcore/Blob.rs           | 116 ++++++++++++++------
 src/runtime/webcore/s3/client.rs      |  65 ++++++-----
 test/js/bun/s3/s3-slice-range.test.ts | 198 ++++++++++++++++++++++++++++++++++
 3 files changed, 311 insertions(+), 68 deletions(-)

gate history · 2 passed · 0 rejected · iteration 5

evidence per changed file

file                                   reads  edits  tests
src/runtime/webcore/Blob.rs               18     14      0
src/runtime/webcore/s3/client.rs           3      3      0
test/js/bun/s3/s3-slice-range.test.ts      5     10      0

robobun · 2026-07-07T16:39:32Z

^{Updated 3:08 PM PT - Jul 9th, 2026}

❌ @robobun, your commit 3c70520 has 1 failures in Build #71150 (All Failures):

test/js/sql/postgres-binary-array-bounds.test.ts - code 1 on 🪟 2019 x64

🧪 To try this PR locally:

bunx bun-pr 33687

That installs a local version of the PR into your bun-33687 executable, so you can run:

bun-33687 --bun

coderabbitai · 2026-07-07T16:44:53Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8df5e992-6b3e-4a00-a739-dd961ae46be3

📥 Commits

Reviewing files that changed from the base of the PR and between 3bef55f and b6fa2d4.

📒 Files selected for processing (1)

src/runtime/webcore/Blob.rs

Walkthrough

Blob slicing now uses absolute offset and size values, with special handling for S3 blobs of unknown size. S3 range header construction is centralized in a shared helper, and a new test suite exercises slice behavior against a mock S3 endpoint.

Changes

Blob Slicing and S3 range handling

Layer / File(s)	Summary
get_slice_from contract change `src/runtime/webcore/Blob.rs`	`BlobExt::get_slice_from` accepts absolute offset and size values and copies them directly onto the duplicated blob.
get_slice offset computation `src/runtime/webcore/Blob.rs`	`get_slice` parses optional start and end values, wraps negative indices against the current blob length, handles S3 unknown-size slice cases, and passes computed offset and size into `get_slice_from`. The source-destination write path slices from the source blob's current offset.
S3 range header helper `src/runtime/webcore/s3/client.rs`	`s3/client.rs` adds `range_header(offset, size)` and uses it from `download_slice` and `download_stream` instead of inline range construction.
S3 slice range test coverage `test/js/bun/s3/s3-slice-range.test.ts`	`s3-slice-range.test.ts` adds a mock S3 subprocess, a spawn helper, and assertions for suffix, open-ended, inclusive, chained, empty, and extreme slice cases.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main fix: emitting valid HTTP Range headers for S3File.slice suffix and open-ended slices.
Description check	✅ Passed	It covers the PR purpose and verification, though it uses custom headings instead of the template's exact section names.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/runtime/webcore/Blob.rs`:
- Around line 2112-2122: The S3 slice handling in Blob::slice is treating a
positive start at the exact MAX_SIZE boundary as the suffix sentinel, which can
cause an empty boundary slice to download the full object. Update the
unresolved-S3 branch to clamp via relative_start, and explicitly short-circuit
the case where relative_start equals this_size before assigning the sentinel
offset. Keep the offset/size sentinel logic in sync with the self.is_s3(),
start_raw, and end_raw path so exact-limit boundaries succeed and one-past-limit
still fails.
- Line 4959: The Blob cloning path is creating an empty slice because
get_slice_from now uses its size argument directly, so the source window length
is being lost. Update the get_slice_from call in the Blob source-window cloning
logic to pass the source blob’s current size instead of 0, preserving the
original slice length when constructing the new view.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8fb1b304-766a-4942-a27a-2a65a274be17

📥 Commits

Reviewing files that changed from the base of the PR and between 3f5d816 and d371973.

📒 Files selected for processing (3)

src/runtime/webcore/Blob.rs
src/runtime/webcore/s3/client.rs
test/js/bun/s3/s3-slice-range.test.ts

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/runtime/webcore/Blob.rs`:
- Around line 2107-2153: The wildcard branch in Blob::slice range handling can
collide with the MAX_SIZE suffix sentinel and return the wrong bytes for chained
slices like slice(F).slice(-F). Update the matching logic around the
default_offset/span fallback in Blob.rs to clamp or otherwise guard against
producing offset == MAX_SIZE for S3 unresolved-size blobs, similar to the
existing Some(s) if s >= 0 arm. Also add a regression test in the S3 slice range
tests for slice(700).slice(-700) that asserts InvalidRange rather than a
successful response.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5583135d-0790-4d22-9a26-9caca4fc6560

📥 Commits

Reviewing files that changed from the base of the PR and between d371973 and f1e5e50.

📒 Files selected for processing (3)

src/runtime/webcore/Blob.rs
src/runtime/webcore/s3/client.rs
test/js/bun/s3/s3-slice-range.test.ts

robobun · 2026-07-07T18:43:47Z

CI status after rebase onto main (3bef55f + autofix b6fa2d4): the diff is green on every lane that ran s3-slice-range.test.ts. The remaining red is unrelated flake:

build 71138: test/regression/issue/26030.test.ts on alpine aarch64, MySQL docker container application not healthy after 1m0s
build 71150 (retrigger): test/js/sql/postgres-binary-array-bounds.test.ts on windows x64, ERR_POSTGRES_CONNECTION_REFUSED

Neither touches Blob/S3. Locally: 46 tests pass with the fix, 20 fail without (InvalidRange / bytes=...4503599627370494). coderabbit reviewed the rebased diff with no actionable comments and all review threads are resolved. Ready for a maintainer.

S3File.slice(-n) computed the negative offset against the unresolved MAX_SIZE sentinel (2^52-1) and sent Range: bytes=4503599627370490-..., which every real S3 endpoint rejects with 416 InvalidRange. slice(a) with no end sent bytes=a-4503599627370494, working only because servers clamp the last-byte-pos. When slicing an S3 blob whose size is still the lazy sentinel and no end is given, encode slice(-n) as an HTTP suffix range (bytes=-n) and slice(a) as an open-ended range (bytes=a-). The suffix form is stored as offset=MAX_SIZE, size=n and decoded by a shared range_header helper used by both download_slice and download_stream. The suffix encoding is restricted to a top-level file (self.offset == 0, n < MAX_SIZE) so chained slices and -Infinity fall through to the generic arithmetic, and default_offset is capped below MAX_SIZE so ordinary arithmetic can never land exactly on the sentinel. Re-slicing a suffix parent that still ends at the parent's end is encoded as a shorter suffix; one that stops short or is empty is pushed past the sentinel so the server 416s instead of returning wrong bytes. get_slice_from now takes the final (offset, size) pair directly instead of relative indices; its other caller in write_file_with_source_destination is updated to pass the same values the old arithmetic produced.

github-actions Bot added the claude label Jul 7, 2026

coderabbitai Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread src/runtime/webcore/Blob.rs

Comment thread src/runtime/webcore/Blob.rs Outdated

claude Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread src/runtime/webcore/Blob.rs Outdated

Comment thread src/runtime/webcore/s3/client.rs Outdated

claude Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread src/runtime/webcore/Blob.rs Outdated

claude Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread src/runtime/webcore/s3/client.rs

coderabbitai Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread src/runtime/webcore/Blob.rs

robobun force-pushed the farm/6ced184f/s3-slice-suffix-range branch from 691a48a to 3bef55f Compare July 9, 2026 19:27

autofix-ci Bot and others added 2 commits July 9, 2026 19:29

[autofix.ci] apply automated fixes

b6fa2d4

ci: retrigger

3c70520

Navigation Menu

Uh oh!

s3: emit HTTP suffix/open-ended Range for S3File.slice(-n) and slice(a) - #33687

s3: emit HTTP suffix/open-ended Range for S3File.slice(-n) and slice(a)#33687
robobun wants to merge 3 commits into
mainfrom
farm/6ced184f/s3-slice-suffix-range

robobun commented Jul 7, 2026 •

edited

Loading

Uh oh!

robobun commented Jul 7, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

robobun commented Jul 7, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

robobun commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Repro

Cause

Fix

Verification

Uh oh!

robobun commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

robobun commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

robobun commented Jul 7, 2026 •

edited

Loading

robobun commented Jul 7, 2026 •

edited

Loading

coderabbitai Bot commented Jul 7, 2026 •

edited

Loading

robobun commented Jul 7, 2026 •

edited

Loading