Skip to content

bundler: chain inline input sourcemaps through to output - #30539

Open
robobun wants to merge 29 commits into
mainfrom
farm/eb1afa62/chain-input-sourcemaps
Open

bundler: chain inline input sourcemaps through to output#30539
robobun wants to merge 29 commits into
mainfrom
farm/eb1afa62/chain-input-sourcemaps

Conversation

@robobun

@robobun robobun commented May 12, 2026

Copy link
Copy Markdown
Collaborator

Closes #30536.
Also fixes #6173 — plugin onLoad returns with an inline sourcemap are fed through the same scanner, so the plugin case rides on the same pipeline.

Summary

When Bun.build({ sourcemap: 'inline' }) bundles an input .js file that carries an inline //# sourceMappingURL=data:application/json;base64,… comment (e.g. .vue/.svelte compilers emitting an intermediate .js with a chain back to the authored .ts/.vue), the final output's sourcemap chain now resolves through to the authored source. Before this change the deepest sources[] entry was the intermediate .js; after, it's the authored source, and sourcesContent[] holds the clean authored bytes without the trailing comment. Same fix covers onLoad plugin returns (#6173) because the scanner runs on source.contents regardless of where they came from.

Repro

// inner.ts — the "original" source
export const x = 5;

// Produce inner.js with an inline sourcemap → inner.ts
writeFileSync('inner.js', src + `\n//# sourceMappingURL=data:application/json;base64,${base64Map}\n`);

const result = await Bun.build({
  entrypoints: ['entry.ts'],
  outdir: 'out',
  sourcemap: 'inline',
});
// Decode out/entry.js's inline sourcemap.

Before: sources = ["../inner.js", "../entry.ts"]. sourcesContent[0] is the raw inner.js bytes including the literal //# sourceMappingURL=… comment. Chain stops at the intermediate.

After: sources = ["../inner.js", "../inner.ts", "../entry.ts"]. sourcesContent[1] is the clean inner.ts, no embedded comment. Mappings that live in the intermediate resolve through to the authored line/column.

Cause

The lexer already detects //# sourceMappingURL= but nothing reads it back — the bundler's linker emits the output map directly from source.contents + the intermediate's path, discarding any chain.

Fix

Thread the inline map end-to-end:

  • src/sourcemap/InputSourceMap.rs (new, ~220 lines): owns the parsed inner ParsedSourceMap + per-source contents. parse_from_source scans for a trailing //# sourceMappingURL=data:... anchored on the last line (Source Map spec — in-body markers in template literals must not hijack the lookup). Supports both ;base64, and raw data:application/json,... payloads. Malformed input → None (silent fallback); OOM propagates via handle_oom.
  • src/bundler/ParseTask.rs: after parsing JS, scan the source bytes. Gated on source_map != .None && loader.can_have_source_map() && !source.contents.is_empty() so no-sourcemap builds pay nothing; external .map references are out of scope. The scan runs on source.contents regardless of where the contents came from (file read or plugin onLoad return), so this also covers the plugin case from Support sourcemaps in onLoad plugins #6173.
  • src/bundler/Graph.rs, bundle_v2.rs: new InputFile.input_source_map: Option<Box<InputSourceMap>> column; ownership moves from the parse result onto the file and drains in deinit_without_freeing_arena (MultiArrayList's Drop is slab-only).
  • src/sourcemap/Chunk.rs + src/js_printer/lib.rs: Builder + Options grow an optional input_source_map. In add_source_mapping, the (line, col) we'd have emitted against the intermediate is translated via map.find_mapping:
    • hit → source_index = 1 + inner.source_index, inner (line, col)
    • miss → slot 0 (the intermediate) so unmapped tokens land in a real file
  • src/bundler/LinkerContext.rs: each outer source in sources[] expands to [intermediate, inner_0 … inner_N-1] and sourcesContent[] matches slot-for-slot. Chunk stitching uses base + chunk.end_state.source_index as the absolute end so per-chunk mappings whose source_index varies across their length compose correctly. Gated on self.dev_server.is_none() because Bake's SourceMapStore::join_vlq stitcher hard-codes one sources[] slot per file and would corrupt on expansion.

Malformed or unrecognized inline maps fall back cleanly — the build still succeeds with the pre-fix behavior (tested).

Verification

bun bd test test/bundler/bun-build-api.test.ts → 53 pass / 0 fail (8 new + 45 existing). 8 new cases under describe("Bun.build chains inline input sourcemaps", …):

  • inline base64 data: URL — authored source surfaces in bundled map
  • inline raw data:application/json,… URL — authored source surfaces
  • inline map with multiple inner sources (e.g. .vue ?script + ?template) — all surface
  • malformed inline payload — build succeeds, silently falls back
  • inline VLQ source_index >= sources.len — rejected, no slot aliasing
  • //# sourceMappingURL= marker inside a template literal (but no trailing comment) — ignored, no hijack
  • external .map filename reference (non-data:) — unchanged behavior
  • plugin onLoad return carrying inline map — regression guard for Support sourcemaps in onLoad plugins #6173

Also ran the full bun-build-api.test.ts and adjacent bundler suites — no regressions.

Build script fix

scripts/build/bun.ts gets a small fix unrelated to the bundler but triggered by this PR's larger release-mode build: added strippedExe as an orderOnlyInputs ninja dep for the release smoke-test rule, so the test doesn't race the strip bun step. Without this, the gate hit an intermittent Permission denied on build/release/bun during the smoke test.

Rebase notes (latest)

Rebased across ~1250 commits of main. Three substantive adaptations:

  • The build-script smoke-test ordering fix this PR carried is gone: main adopted it upstream (emitPostLink centralizes the strip-before-smoke-test invariant, crediting bundler: chain inline input sourcemaps through to output #30539), so that commit was dropped and the PR no longer touches scripts/build/bun.ts.
  • The unsafe lifetime erasure in js_printer is gone: main parameterized Chunk::Builder over a lifetime, so input_source_map is now a plain Option<&'a InputSourceMap> borrow with no transmute.
  • Main replaced the JSON parser with a tape-based reader (EObjectJSON/EArrayJSON rows) and made mapping::parse return a Result; InputSourceMap::parse_internal now reads version/mappings/sources/sourcesContent through the tape accessors (ObjectJSON::get, JsonValue::as_str/as_array). Without this the container lookups silently failed and no chaining occurred.

Verified after the rebase: all 9 chain tests pass, and the 9 sourcemap tests in bun-build-api.test.ts (including main's new C0-control-chars and non-ASCII-columns cases) pass.

@robobun

robobun commented May 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 7:05 AM PT - Aug 15th, 2026

@robobun, your commit 2d79d79 is building: #97866

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. Support sourcemaps in onLoad plugins #6173 - This PR chains inline //# sourceMappingURL=data:... comments through to the output sourcemap, which is exactly what onLoad plugins need when they return transformed JS with an embedded inline sourcemap.

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #6173

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented May 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Parses trailing inline data:application/json //# sourceMappingURL= maps, stores owned InputSourceMap on inputs, threads them into the JS printer and SourceMap chunk builder, remaps mappings through inner maps, and expands emitted sources/sourcesContent so authored inner sources appear in final bundled maps.

Changes

Inline sourcemap chaining

Layer / File(s) Summary
InputSourceMap parser module
src/sourcemap/InputSourceMap.zig, src/sourcemap/sourcemap.zig
New InputSourceMap type owns a parsed sourcemap with sources, sourcesContent, and parsed mappings; parse(json_bytes) validates v3.0 structure and allocates owned copies; parseFromSource(source) extracts and decodes trailing inline data:application/json URLs with base64 and raw payload support.
Parse-phase detection and storage
src/bundler/Graph.zig, src/bundler/ParseTask.zig
Graph.InputFile and ParseTask.Result.Success gain optional input_source_map fields; parser conditionally calls InputSourceMap.parseFromSource when sourcemaps are enabled, the loader may supply maps, and input contents are non-empty; dev-server HMR error paths deinit transient maps.
Bundler ownership and lifecycle
src/bundler/bundle_v2.zig
On parse success result.input_source_map is moved into graph.input_files.items(.input_source_map) (deinit any prior map); on parse failure or overwrite transient maps are deinitialized; deinitWithoutFreeingArena deinitializes stored input maps before tearing down AST and input_files.
Sourcemap builder remapping
src/js_printer/js_printer.zig, src/sourcemap/Chunk.zig
js_printer.Options and SourceMap.Chunk.Builder gain input_source_map; addSourceMapping optionally looks up intermediate (line,column) in the inner map and emits source_index = 1 + inner_index plus the inner map's zero-based original coordinates when a covering mapping exists, otherwise falls back to the intermediate slot-0 coordinates.
Linker emission expansion
src/bundler/LinkerContext.zig
generateSourceMapForChunk expands sources/sourcesContent per input: emit the intermediate input first then any inner external_source_names and their sourcesContent (emit null for missing content); update mapping source-index stitching across appended chunks to include inner-source offsets; pass input_source_map into printer except for DevServer.
Test coverage
test/bundler/bun-build-api.test.ts
Adds a suite validating base64/raw inline maps, multiple inner sources, malformed inline payload graceful fallback, out-of-range inner indexes rejection, trailing-line-only detection (ignore in-body markers), non-chaining of external .map references, and a plugin regression test for onLoad-returned inline maps.
Build smoke-test ordering
scripts/build/bun.ts
Passes strippedExe to smoke-test rules and marks it as an order-only input so the smoke test runs after strip output is produced.
🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Out of Scope Changes check ❓ Inconclusive All changes are tightly scoped to inline sourcemap chaining. The build script changes (scripts/build/bun.ts) only add smoke-test ordering dependencies unrelated to this feature but appear to be a pre-existing CI infrastructure fix. One unrelated change present: scripts/build/bun.ts smoke test serialization. Clarify whether scripts/build/bun.ts smoke-test ordering changes are intentional or should be separated into a different PR for review clarity.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main feature: threading inline input sourcemaps through the bundler to the output, which accurately reflects the primary changes across all modified files.
Linked Issues check ✅ Passed The PR fully addresses both linked issues: #30536 (detect inline sourceMappingURL comments in file inputs and compose chains through to output) and #6173 (support sourcemaps from plugin onLoad returns). All core requirements are implemented in the Zig reference implementation with comprehensive tests.
Description check ✅ Passed The description clearly explains the PR purpose, implementation, scope, edge cases, and verification results, although it does not use the template headings exactly.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/bundler/bundle_v2.zig`:
- Around line 3675-3677: The assignment to
graph.input_files.items(.input_source_map)[result.source.index.get()] =
result.input_source_map leaks the previous ParsedSourceMap and can leak the
freshly parsed map on early failure; before overwriting the graph slot,
deinit/free the existing map (if any) stored in
graph.input_files.items(.input_source_map)[...], then move
result.input_source_map into that slot, and ensure you null out or mark
result.input_source_map as moved; additionally add matching cleanup in the error
path around runResolutionForParseTask() so that if result transitions from
.success to .err the newly allocated ParsedSourceMap and its sourcesContent are
deallocated (use defer or explicit deinit in the same scope where
result.input_source_map is allocated) to avoid leaks.

In `@src/bundler/ParseTask.zig`:
- Around line 1320-1325: The inline source-map parsing currently runs for
plugin/virtual sources too; update the conditional that computes
input_source_map (the if using transpiler.options.source_map,
loader.canHaveSourceMap(), and source.contents) to also require that the input
is file-backed by checking source.path.isFile() (or that source.path.namespace
== "file") so only file-backed inputs attempt
bun.SourceMap.InputSourceMap.parseFromSource; leave the other checks
(transpiler.options.source_map and loader.canHaveSourceMap and non-empty
source.contents) intact.

In `@src/sourcemap/InputSourceMap.zig`:
- Around line 37-145: The parse function currently treats allocator failures
(e.g., at allocator.alloc, allocator.dupe, and similar catch sites such as
source_paths_slice, sources_content_slice, and the dupes inside the loops) as
generic parse failures by using `catch return null`; update each such `catch` to
inspect the error and call `bun.handleOom()` for `error.OutOfMemory` and
otherwise `return null` — e.g. replace `... catch return null` with `... catch
|err| if (err == error.OutOfMemory) bun.handleOom() else return null` at every
allocation/dupe site in InputSourceMap.parse (including the other occurrences
around lines ~203-209) so OOMs trigger Bun’s fatal path while preserving null
for real decode/validation failures.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3d64f326-1bc4-4bfb-98df-db68570a30e3

📥 Commits

Reviewing files that changed from the base of the PR and between 2043f9c and a7ffbe3.

📒 Files selected for processing (9)
  • src/bundler/Graph.zig
  • src/bundler/LinkerContext.zig
  • src/bundler/ParseTask.zig
  • src/bundler/bundle_v2.zig
  • src/js_printer/js_printer.zig
  • src/sourcemap/Chunk.zig
  • src/sourcemap/InputSourceMap.zig
  • src/sourcemap/sourcemap.zig
  • test/bundler/bun-build-api.test.ts

Comment thread src/bundler/bundle_v2.zig Outdated
Comment thread src/bundler/ParseTask.zig Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/bundler/bun-build-api.test.ts`:
- Around line 1297-1299: The test decodes base64 sourcemap payloads using m![1]
without checking that the regex match succeeded; add an explicit guard before
decoding at each occurrence (the variable m in the bun-build-api.test cases) —
e.g., assert expect(m).toBeTruthy() or if (!m) fail with a clear message, then
use m[1] to Base64-decode and JSON.parse; apply the same fix to all four
locations where m![1] is used (the matches at lines matching the
sourceMappingURL extraction).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0d06802d-137b-46ff-afe3-f7d3c764486e

📥 Commits

Reviewing files that changed from the base of the PR and between a7ffbe3 and 192b355.

📒 Files selected for processing (1)
  • test/bundler/bun-build-api.test.ts

Comment thread test/bundler/bun-build-api.test.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/sourcemap/InputSourceMap.zig`:
- Around line 197-212: The current findSourceMappingURL searches the whole file
for "\n//# sourceMappingURL=" which can miss a trailing inline comment or match
markers inside later content; change it to first trim trailing whitespace from
source, compute the start of the final line (use std.mem.lastIndexOfScalar to
find the last '\n' or 0), then only search for the needle inside that final-line
slice (e.g. search source[lineStart..] for "//# sourceMappingURL="). If found,
compute start/end relative to the original buffer and return the trimmed URL as
before; otherwise return null. Ensure you update uses of needle, found, start,
end and keep using bun.strings.trim for trimming.
- Around line 79-81: The parser currently treats the "version" field as
optional; in InputSourceMap.zig change the logic that handles
json.get("version") so that absence returns error.InvalidSourceMap and presence
is still validated (i.e., ensure version.data is .e_number and
version.data.e_number.value == 3.0); locate the block referencing
json.get("version") and replace the optional branch with an explicit existence
check that errors when missing and otherwise performs the same type/value
validation.

In `@test/bundler/bun-build-api.test.ts`:
- Around line 1357-1361: The test currently only asserts an inline source map
exists by checking text for "sourceMappingURL=data:...base64"; update the
assertion to decode and validate the actual fallback source used in the
malformed-map case: extract the base64 payload from the data URL in the variable
text (from Bun.file(result.outputs[0].path).text()), base64-decode and
JSON.parse it, then assert that the parsed source map's sources or
sourcesContent includes the expected intermediate/fallback source (e.g., the
intermediate filename or its source text) so the mapping truly targets the
documented fallback source instead of just existing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 292ecd2b-9daf-426c-9bc5-9fd0149be05d

📥 Commits

Reviewing files that changed from the base of the PR and between 192b355 and af3f78d.

📒 Files selected for processing (3)
  • src/bundler/bundle_v2.zig
  • src/sourcemap/InputSourceMap.zig
  • test/bundler/bun-build-api.test.ts

Comment thread src/sourcemap/InputSourceMap.zig Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated
Comment thread test/bundler/bun-build-api.test.ts Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated
Comment thread src/bundler/ParseTask.zig Outdated
Comment thread src/bundler/LinkerContext.zig Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
src/sourcemap/InputSourceMap.zig (1)

214-219: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Match trailing inline sourcemaps after code on the final line.

This only accepts a comment-only last line. Valid one-line outputs like code;//# sourceMappingURL=data:... still return null, so chaining is skipped for minified/plugin-generated intermediates.

Suggested fix
-    const needle = "//# sourceMappingURL=";
-    if (!bun.strings.hasPrefixComptime(last_line, needle)) return null;
-    return bun.strings.trim(last_line[needle.len..], " \r\t");
+    const needle = "//# sourceMappingURL=";
+    const found = std.mem.lastIndexOf(u8, last_line, needle) orelse return null;
+    return bun.strings.trim(last_line[found + needle.len ..], " \r\t");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/sourcemap/InputSourceMap.zig` around lines 214 - 219, The current logic
in InputSourceMap.zig only accepts a sourceMappingURL if the final line starts
with the needle, so cases like "code;//# sourceMappingURL=..." are missed;
update the check to search for needle anywhere in last_line (use a
substring/index search instead of hasPrefixComptime) and, when found, slice
last_line at the found index (needle.len after the index) and then trim that
substring before returning; reference the existing variables last_line_start,
last_line, and needle to locate and replace the prefix-only hasPrefixComptime
logic with an index-based search (e.g., std.mem.indexOf or similar) and handle
the not-found case by returning null.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/bundler/LinkerContext.zig`:
- Around line 744-755: The code currently only sets path.pretty for file-backed
inputs (in the branch checking path.isFile()) but always serializes path.pretty,
which loses non-file/virtual/plugin module names; update the serialization so
that when path.isFile() is false you use outer_path.text (or set path.pretty =
outer_path.text beforehand) before calling js_printer.quoteForJSON and pushing
to joiner (symbols: outer_path, path, path.isFile(), path.pretty,
js_printer.quoteForJSON, joiner, MutableString.init, leading_comma) so
plugin/virtual module source names are preserved in emitted sources[].

In `@test/bundler/bun-build-api.test.ts`:
- Around line 1250-1253: The test currently grabs the first sourceMappingURL
match from the file (via text.match and m) which can pick up in-body markers;
change the extraction to target only the final inline sourcemap line by either
using a regex anchored to the end of the string (e.g. match the sourceMappingURL
line followed by optional whitespace to string end with the /m flag) or by
finding all matches and using the last one before base64 decoding; update the
occurrences that use Bun.file(...).text(), text.match(...), m and parsed at the
noted spots (including the other listed locations) to use this trailing-line
approach so the parsed JSON always comes from the final inline sourcemap.

---

Duplicate comments:
In `@src/sourcemap/InputSourceMap.zig`:
- Around line 214-219: The current logic in InputSourceMap.zig only accepts a
sourceMappingURL if the final line starts with the needle, so cases like
"code;//# sourceMappingURL=..." are missed; update the check to search for
needle anywhere in last_line (use a substring/index search instead of
hasPrefixComptime) and, when found, slice last_line at the found index
(needle.len after the index) and then trim that substring before returning;
reference the existing variables last_line_start, last_line, and needle to
locate and replace the prefix-only hasPrefixComptime logic with an index-based
search (e.g., std.mem.indexOf or similar) and handle the not-found case by
returning null.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7927bb9d-82d1-4fb6-a98a-185ceb0904d3

📥 Commits

Reviewing files that changed from the base of the PR and between af3f78d and b93356c.

📒 Files selected for processing (4)
  • src/bundler/LinkerContext.zig
  • src/bundler/ParseTask.zig
  • src/sourcemap/InputSourceMap.zig
  • test/bundler/bun-build-api.test.ts

Comment thread src/bundler/LinkerContext.zig Outdated
Comment thread test/bundler/bun-build-api.test.ts Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated
Comment thread src/sourcemap/InputSourceMap.zig Outdated
@robobun
robobun force-pushed the farm/eb1afa62/chain-input-sourcemaps branch from c4223fc to 03f82da Compare May 16, 2026 01:21
Comment thread test/bundler/bun-build-api.test.ts Outdated
Comment thread test/bundler/bun-build-api.test.ts Outdated
Comment thread src/sourcemap/InputSourceMap.rs Outdated
@robobun
robobun force-pushed the farm/eb1afa62/chain-input-sourcemaps branch from 06c646d to 528da29 Compare May 19, 2026 09:50

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues found, but this is a substantial bundler feature (~1k lines across parse/link/print + a new sourcemap parser handling untrusted input) with non-trivial slot-index arithmetic and ownership threading, plus an unrelated build-script change — worth a human pass before merge, and CI is still red on aarch64-musl build-rust.

Extended reasoning...

Overview

This PR threads inline //# sourceMappingURL=data:... input sourcemaps through Bun.build's output map. It adds a new parser module (InputSourceMap.{zig,rs}, ~270 lines each), a new SoA column on Graph::InputFile, ownership transfer in ParseTask/bundle_v2, per-mapping remapping in Chunk::Builder, and per-source slot expansion + chunk-stitching index arithmetic in LinkerContext::generate_source_map_for_chunk. 8 new tests in bun-build-api.test.ts. There's also an unrelated scripts/build/bun.ts change adding an order-only ninja dep to serialize strip against the smoke test.

Security risks

The new parser ingests untrusted bytes (any data: URL embedded in any bundled .js, including plugin onLoad returns). Failure modes are designed to fall back to None (silent no-chain), and earlier review rounds caught and fixed the source-index-aliasing case (out-of-range VLQ indices now hit .fail via the real sources_count). I don't see injection or auth surface, but the slot-expansion arithmetic in LinkerContext (base + chunk.end_state.source_index) and the dev-server gate are exactly the kind of subtle index bookkeeping that benefits from a second pair of eyes.

Level of scrutiny

High. This is production-critical bundler output (sourcemaps affect every debugging/error-reporting workflow), the change spans 19 files in two languages, and it carries several design decisions a maintainer should sign off on: keeping the intermediate as slot 0 alongside inner sources (vs. replacing it), gating the entire feature off for the Bake DevServer path, and deferring sourceRoot / external-.map / plugin-namespace path normalization to follow-ups.

Other factors

The PR went through several automated review rounds; all prior inline findings (dead errdefers, leak on Success→err overwrite, dev-server corruption, ban-words, port-fidelity trim) are marked resolved. The Rust port landed in 30171d1 and the tests are now active (not .todo). However, the most recent CI status (Build #56048, commit 528da29) still shows build-rust failing on aarch64-musl, and the scripts/build/bun.ts change is acknowledged as out-of-scope for the headline feature.

@robobun
robobun force-pushed the farm/eb1afa62/chain-input-sourcemaps branch from c74b504 to e21f069 Compare May 20, 2026 05:44
Comment thread src/bundler/Graph.rs
Comment thread src/sourcemap/InputSourceMap.rs Outdated
Comment thread src/bundler/ParseTask.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found on this pass, but this is a substantial feature touching the bundler's sourcemap stitching across both the Zig reference and the Rust port (new InputSourceMap module, per-chunk source_index offset arithmetic in LinkerContext/Chunk, ownership threading through Graph/ParseTask/bundle_v2, plus an unsafe 'static lifetime erasure in get_source_map_builder) — worth a human pass on the index-stitching math and the Rust lifecycle before merge.

Extended reasoning...

Overview

This PR adds inline-input-sourcemap chaining to Bun.build: a new ~270-line InputSourceMap module (both .zig reference and .rs port) plus threading through Graph/ParseTask/bundle_v2/LinkerContext/Chunk/js_printer so that a trailing //# sourceMappingURL=data:... on an input file expands the output map's sources[]/sourcesContent[] and remaps VLQ mappings through the inner map. It also bundles an unrelated scripts/build/bun.ts ninja order-only-dep fix and 8 new tests in bun-build-api.test.ts.

Security risks

Low. The new parser ingests untrusted inline sourcemap payloads from input files / plugin returns, but malformed input falls through to None/null (silent fallback), OOM goes through handle_oom/bun.outOfMemory, and the out-of-range source_index aliasing concern was fixed by passing the real sources_count to Mapping.parse. No auth, crypto, network, or filesystem-write surface.

Level of scrutiny

High. This is core bundler output-correctness code: the per-chunk source_index expansion and the prev_end_state.source_index = mapping_source_index + chunk.end_state.source_index stitching in generateSourceMapForChunk are subtle — an off-by-one silently corrupts sourcemaps for every downstream user. The Rust port also includes an unsafe { &*(r as *const _) } lifetime erasure to 'static in get_source_map_builder, manual MultiArrayList column draining in deinit_without_freeing_arena, and a Stacked-Borrows-sensitive hoist in ParseTask.rs. The PR went through ~15 inline fixes over the week (dead errdefer, leak paths, DevServer gating, ban-words, E0063, UB), which speaks to its complexity.

Other factors

All prior inline comments (mine and CodeRabbit's) are resolved. Test coverage is good (8 targeted cases including malformed/edge inputs). However, several design decisions were explicitly deferred to follow-ups (DevServer slot expansion, sourceRoot, version strictness, external .map resolution, path.pretty for non-file namespaces), and the unrelated build-script change is bundled in. Given the cross-cutting scope and the index-arithmetic subtlety, a human reviewer should sign off on the stitching math and the Rust ownership/lifetime story.

@robobun
robobun force-pushed the farm/eb1afa62/chain-input-sourcemaps branch from 8e56af7 to f0a63f6 Compare June 2, 2026 01:43
Comment thread src/sourcemap/Chunk.rs

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues found, but this is a substantial feature touching bundler-core sourcemap stitching (slot-expansion index arithmetic in LinkerContext, per-token remapping in Chunk::Builder) plus an unsafe lifetime transmute in js_printer — worth a human pass before merge.

Extended reasoning...

Overview

This PR threads inline //# sourceMappingURL=data:... comments on bundler input files through to the output sourcemap, so authored sources (.vue/.svelte/.ts) surface in the final map instead of the intermediate .js. It adds a new ~270-line InputSourceMap module (both .rs and reference .zig), plumbs an input_source_map column through Graph::InputFile / ParseTask::Success / bundle_v2, adds remapping logic to Chunk::Builder::add_source_mapping, rewrites the sources[]/sourcesContent[] emission and chunk-stitching arithmetic in LinkerContext::generate_source_map_for_chunk, and adds a new input_source_map field to js_printer::Options with an unsafe transmute to erase its lifetime to 'static. It also carries an unrelated scripts/build/bun.ts fix (order-only ninja dep so the smoke test doesn't race strip). 19 files changed; 8 new tests in bun-build-api.test.ts.

Security risks

None apparent. The new code parses untrusted inline sourcemap JSON/base64 from input files, but malformed input falls back to None (no chain) rather than erroring; out-of-range VLQ source indices are bounded by sources_count so they can't alias neighboring slots; the URL scanner is anchored to the last line so in-body markers can't hijack. No filesystem reads of external .map files (explicitly out of scope). The unsafe transmute extends a borrow lifetime, not raw memory access.

Level of scrutiny

High. This is production bundler-core code that affects every Bun.build with sourcemap != none. The slot-expansion arithmetic (base + chunk.end_state.source_index, 1 + inner.source_index, expansion counts) is delicate — the review history shows it was wrong in multiple subtle ways (DevServer corruption, out-of-range aliasing, hint-coordinate-space mismatch) before being fixed. The unsafe lifetime erasure in js_printer/lib.rs:7995-8013 deserves a human eye on whether the SAFETY comment's invariant (graph slot outlives every add_source_mapping call) actually holds across all printer entry points. The DevServer gate (self.dev_server.is_none()) is a deliberate feature scope-down that a maintainer should sign off on.

Other factors

The PR went through ~10 rounds of bug fixes during review (dead-errdefer leaks, stacked-borrows UB, E0063 build break, ban-words violation, slot-overwrite leaks, DevServer corruption gate, hint-poisoning perf regression) — all now resolved, which speaks well of the final state but also confirms this is not mechanical code. All prior inline comments (mine and CodeRabbit's) are resolved. The unrelated scripts/build/bun.ts change is small and well-commented but bundling it here is a scope question for the maintainer. Test coverage is good (8 targeted cases including malformed/edge inputs).

@robobun

robobun commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator Author

CI on 3e0f8bb8 is green on every lane that exercises this diff — cargo clippy ✅, debian-13-x64-asan-test-bun ✅ (the 8 chain tests run under ASAN here), all four Windows test lanes ✅. The only red checks are three darwin *-test-bun lanes, all marked Expired (Buildkite agent reclaimed the job before it finished) — infra flake unrelated to the change. I've already spent my one retrigger earlier in this branch, so I'm not re-rolling again. Ready for a human merge pass.

@robobun

robobun commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator Author

CI status after the rebase (head 8020783, build #61103): every test lane fails on exactly one test, test/cli/install/bunx.test.ts > should handle package that requires node 24, which runs bunx --bun @angular/cli@latest --help against the live npm registry and now gets exit 3. This is fleet-wide, not PR-specific: every recent Buildkite build on every branch (e.g. #61109, #61110, #61111, #61114) fails on the same file, so a registry-side @angular/cli release broke the test everywhere. That file is untouched by this PR, and this PR's own tests pass on their shards. I already used my one retrigger on this branch; re-running won't help until the bunx test is fixed on main. The diff itself is green: full bun-build-api.test.ts is 54 pass / 0 fail locally on this head.

@robobun
robobun force-pushed the farm/eb1afa62/chain-input-sourcemaps branch from 3e0f8bb to 8020783 Compare June 6, 2026 09:46
Comment thread src/bundler/bundle_v2.rs
Comment thread src/js_printer/lib.rs
Comment thread src/sourcemap/Chunk.rs
Comment thread src/sourcemap/Chunk.rs
Comment thread src/sourcemap/Chunk.rs
Comment thread src/sourcemap/Chunk.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
Comment thread src/sourcemap/InputSourceMap.rs
@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

On the comment-lint findings: commit 7d009dd trims the explanatory comments across all flagged files to their load-bearing core, a net -116 comment lines. What remains is short (2-4 lines) and documents cross-file invariants a reviewer cannot recover from any single file: the sources[] slot layout shared by Chunk::Builder and LinkerContext, the chunk-relative vs absolute source_index stitching, the intermediate-vs-authored coordinate space for the line-table hint, the DevServer gating, the slab-only SoA drop requiring an explicit drain, and the adversarial-input caps. None of these annotate workarounds; they describe the contract between files, so I'm keeping them rather than trimming further. I'll mark the open lint threads resolved accordingly.

Comment thread src/sourcemap/InputSourceMap.rs Outdated
@robobun
robobun force-pushed the farm/eb1afa62/chain-input-sourcemaps branch from 3249a95 to 27ec2ea Compare August 15, 2026 08:22
Comment thread src/bundler/LinkerContext.rs
Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/sourcemap/Chunk.rs
…es through

The DevServer stitcher never consumes the parsed map, so the per-reparse
scan was dead work on the HMR path. URL-schemed inner sources[] entries
(webpack:///...) must not be path-joined; emit them verbatim per spec.
Comment thread src/bundler/LinkerContext.rs
Comment thread src/bundler/ParseTask.rs
Comment thread src/bundler/ParseTask.rs
Comment thread src/bundler/LinkerContext.rs Outdated
Comment thread src/sourcemap/InputSourceMap.rs
A plugin onResolve custom-namespace path has no on-disk directory, so
joining inner names against dirname(text) produced bogus labels.
Comment on lines +1253 to +1254
// A non-file intermediate (plugin virtual module) has no directory
// to resolve against; emit inner names verbatim.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

CI on 2d79d79 (build #97866, now complete) has zero test failures: 176 jobs passed, including every lane that runs the bun-build-inline-sourcemap-chain suite (11 tests). The only non-green entries are infrastructure: the windows x64 verify-baseline step was canceled (no exit code, never failed), and the two darwin 14 aarch64 test-bun jobs expired waiting for an agent, the same darwin agent scarcity this PR has hit before. I've already used my one retrigger on this branch, so re-running those two darwin jobs (or merging past a lone agent-expiry) is a maintainer call. All review findings are addressed or explicitly deferred with the reviewer's agreement, and all review threads are resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun.build ignores inline //# sourceMappingURL= comments on input files (file-input sibling of #6173) Support sourcemaps in onLoad plugins

1 participant