Skip to content

bundler: escape what is spliced into JS chunks (ASCII for bun), and fix the column count source map shifts use after non-ASCII text - #38666

Open
robobun wants to merge 6 commits into
mainfrom
farm/830724a2/html-import-bun-target-ascii
Open

bundler: escape what is spliced into JS chunks (ASCII for bun), and fix the column count source map shifts use after non-ASCII text#38666
robobun wants to merge 6 commits into
mainfrom
farm/830724a2/html-import-bun-target-ascii

Conversation

@robobun

@robobun robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • bun build --target=bun of a server file that imports an HTML file with a non-ASCII name (import h from "./é.html") writes a bundle that bun cannot load: SyntaxError: Invalid character '\u00a9'. With --minify it loads, but the manifest's index/path values come back as ./é.html, and with --sourcemap every stack frame after the manifest maps to the wrong position (s.ts:6:6 for both frames of a throw at 4:13 called from 6:1; the same build with an ASCII name is right since bundler: shift source maps past the spliced HTML import manifest #38429).
  • The same happens to chunk and asset paths: --target=bun --splitting with import("./é.ts") fails at runtime with Cannot find module './é-630mkta0.js', and a file-loader import of ./é.txt evaluates to ./é-hvs6h2nf.txt. For every target, an asset named q"b.txt is emitted as var q_b_default = "./q"b-t6x98h26.txt";, which does not parse (the bug bundler: JS-escape file-loader asset paths at chunk assembly #34188 is open for).
  • Three causes:
    • generate_server_html_module (src/bundler/bundle_v2.rs:6843) builds the manifest module's AST but never sets its target, so it keeps the parser default Browser. The printer picks its escaping from the per-file target, so var é_default = ... is printed raw while every reference to it elsewhere in the chunk is printed as \u{e9}_default. computeChunks also classifies files by this target: with --splitting, the shared server chunk holding the manifest got flagged as browser output and listed in the manifest itself.
    • IntermediateOutput::code_with_source_map_shifts (src/bundler/Chunk.rs) copies each chunk/asset path into the chunk unescaped, although the placeholder it replaces is the body of a "..." literal, and HTMLImportManifest::write_escaped_json escaped the manifest only for quotes (ascii_only = false). In a chunk whose first line says // @bun, which the runtime loads as Latin-1 (String::clone_latin1 in src/runtime/jsc_hooks.rs), the raw UTF-8 é (C3 A9) is then read as é.
    • LineColumnOffset::advance (src/sourcemap/lib.rs:158) jumps from one newline/non-ASCII byte to the next and only counts the character it stopped at plus the trailing ASCII run; the ASCII between stops is never added to columns. The manifest above (5 é, 384 bytes) advanced the shift by 111 columns instead of 379. This is independent of the target: in a browser build with --minify-whitespace, a non-ASCII string anywhere before an asset or chunk placeholder on the line moves the recorded splice position left, and the mappings between the two get shifted although they sit before the splice (repro in the details below, mappings off by 5 columns). Ported unchanged from the Zig version.

Fix

  • generate_server_html_module sets the manifest module's target to the importing file's target. The hunk is byte-identical to the one the open bundler: keep the import kind of server-side HTML imports so import() and require() of the manifest link #38621 and bundler: give the browser side of a server build its own copy of the runtime #38376 carry for their own symptoms (bundler: keep the import kind of server-side HTML imports so import() and require() of the manifest link #38621 also adds a test for the --splitting classification), so bundle_v2.rs merges cleanly in any order; it is needed here because the identifier escaping depends on it.
  • code_with_source_map_shifts picks a SpliceEscape per chunk: CSS and HTML chunks are written as before; in JS chunks every spliced path goes through write_pre_quoted_string_inner with the arguments the printer uses for a "..." literal, with ascii_only set by the predicate postProcessJSChunk uses to emit the pragma (entry file's target is_bun()), and the same flag goes to write_escaped_json. Every placeholder in a JS chunk is the body of a "..." literal (import paths, require()/import() arguments, asset strings, __jsonParse("...")), so "./\xE9-630mkta0.js", "./q\"b-....txt" and \u00E9 inside the manifest evaluate to the strings the raw bytes were meant to be. Paths without quotes, backslashes or control characters come out byte-identical for browser and node (checked CLI and API builds of ASCII names against a build of main); this covers what bundler: JS-escape file-loader asset paths at chunk assembly #34188 fixes in the same two passes, with the printer's escaper instead of a separate one.
  • Both passes resolve the path through one helper (spliced_path_parts). The sizing pass used to measure the path before the separator normalization the writing pass applies, which was harmless while both copied the bytes through but over-counted once the path was escaped (a Windows \ counts as two bytes, then gets normalized to / and written as one), leaving NUL bytes at the end of the chunk; test/regression/issue/31575.test.ts caught that on the Windows lanes of the first push. The manifest is likewise sized with the same function that writes it (through bun_io::DiscardingWriter) instead of a Display adapter; EscapedJson, the HTMLImportManifest struct and the html_import_manifest facade module existed only for that adapter and are removed. The shift is advanced over the bytes actually written, so the recorded width is the escaped width, which is also what the Latin-1 load sees.
  • The metafile resolves its chunk references through the same routine, with the first chunk standing in, so the escape mode chosen for that chunk was applied inside JSON ("./mod\xFCl\xE9-....js", which JSON.parse rejects; caught in review). The escape mode is now passed in explicitly: code() derives it from the chunk, and MetafileBuilder calls code_for_metafile(), which escapes as JSON string content. That is what the release writes for non-ASCII names (raw UTF-8, checked byte for byte on the repro) and additionally makes a quote in a chunk name valid JSON, which the release gets wrong.
  • advance adds i - offset (the ASCII run the search skipped) before handling the character at i. For a newline this is immediately reset, as before; for non-ASCII it makes the count match what the printer and source-map consumers count (UTF-16 units). Callers that pass ASCII-only input take the same path as before.
  • Verification:
    • test/bundler/html-import-manifest.test.ts: non-ascii-file-name-loads-under-target-bun (fails on main on the ASCII check and with the SyntaxError), non-ascii-file-name-source-map-columns (fails on main with error: ./sidé.html and frames 5:6/5:6 instead of 3:13/5:1).
    • test/bundler/bundler_bun.test.ts bun/NonAsciiChunkAndAssetPaths: fails on main (Cannot find module).
    • test/bundler/bundler_loader.test.ts {bun,node,browser}/loader-file-path-with-quote (POSIX only, Windows cannot create the file): on main the output contains "./q"b- for all three targets.
    • test/bundler/metafile.test.ts "metafile escapes resolved chunk paths as JSON" (POSIX only): a dynamically imported q"modülé.js chunk in a target: "bun" build; the metafile is unparseable on main (raw quote) and on the intermediate version of this branch (\xFC), and parses with the chunk path intact now.
    • test/bundler/bun-build-api.test.ts "generated columns before and after a spliced asset path on a line with non-ASCII text": browser build; on main the mapping for the function between the "é" and the asset splice is recorded 5 columns left of the function (101 vs 106), the ones before and after are right both ways.
    • Also ran against the debug build: the rest of those four files, bundler_html*.test.ts, bundler_splitting.test.ts, bundler_naming.test.ts, metafile.test.ts, bundler_banner.test.ts, bundler_compile.test.ts, bundler_edgecase.test.ts, esbuild/loader.test.ts, bun-serve-html-manifest.test.ts, test/regression/issue/{27465,25628,31575}.test.ts; cargo fmt --check and clippy on bun_bundler/bun_sourcemap. On a Windows x64 debug build: 31575.test.ts, the two html-import tests and bun-build-api.test.ts pass, and manual builds with nested entry points (relative paths between directories), a --public-path containing a quote, and --target=bun produce chunks without NUL bytes that run. (Bun.build can be called thousands of times in bun-build-api.test.ts exceeds its 180 s budget in this debug/ASAN container with and without the change, about 550 ms per build either way; compile/HelloWorldWithProcessVersionsBun fails on every debug build and is tracked separately.)
    • The first Windows run also showed the manifest's input fields as ../C:\...\sidé.html; that is the separate Windows bug bundler: emit posix-relative paths in the HTML-import manifest on Windows #34557 is for, so the new test does not assert input.
  • Left as is: the /* é.html */ path comment in front of each module still contains the name verbatim. It is harmless to the load, it shows as é only in error excerpts of builds without source maps, and escaping it would make the file itself less readable.

Background

  • // @bun pragma: bun build --target=bun puts this on the first line of every JS chunk. When bun later runs such a file it skips the transpiler and hands the bytes to JavaScriptCore as a Latin-1 string (one character per byte), which is only correct if the file is pure ASCII. The printer guarantees that for the code it prints (ASCII_ONLY in src/js_printer, selected per file from ast.target): non-ASCII in strings becomes \xE9/\u...., in identifiers \u{e9}. Standalone executables do not go through this path (a --compile build with a non-ASCII asset name works on main); the escaped form means the same thing there, and the one above still prints /$bunfs/root/é-....bin and reads the file.
  • Placeholders: while a chunk is printed, output paths are not known yet, so import paths, asset strings and the manifest argument are printed as 25-byte unique keys. After printing, break_output_into_pieces cuts the chunk at those keys and code_with_source_map_shifts writes the final text, resolving each key to a path (or, for the HTML import, to the manifest JSON). This runs once to size the buffer and once to fill it, which is why both passes have to agree byte for byte.
  • Shifts: the source map was produced against the text with the keys in it. For each substitution the writer records before (position in the printed text, advanced over the key) and after (position in the final text, advanced over the replacement); SourceMapPieces::finalize adds after - before to every mapping located past before on that line. Both positions are computed with LineColumnOffset::advance over the text in between, so an under-count there moves before left and applies the shift to mappings that precede the splice.
  • HTML import manifest: in a server build, import h from "./x.html" bundles the HTML as a browser build and binds h to a generated module whose body is __jsonParse("<key>"); the key is replaced by the JSON listing the browser outputs. generate_server_html_module builds that module directly instead of going through a parse task, which is where the target assignment was missed.
Repro from the report, before and after
d=$(mktemp -d); cd "$d"
printf '<script type=module src=./c.ts></script>' > 'é.html'
echo 'console.log(1)' > c.ts
printf 'import h from "./é.html";\nexport const k = [h];\nfunction boom() {\n  throw new Error("L4");\n}\nboom();\n' > s.ts

bun build ./s.ts --target=bun --outdir out && bun out/s.js
# before: SyntaxError: Invalid character '\u00a9'
# after:  error: L4 (the bundle loads and runs s.ts)

bun build ./s.ts --target=bun --minify --sourcemap=inline --outdir out2 && bun out2/s.js 2>&1 | grep -o 's\.ts:[0-9:]*'
# before: s.ts:6:6  s.ts:6:6
# after:  s.ts:4:13 s.ts:6:1   (identical to the same build with an ASCII name)

Manifest line before (raw, inside a // @bun file) and after:

var é_default = __jsonParse("{\"index\":\"./é.html\",...
var \u{e9}_default = __jsonParse("{\"index\":\"./\u00E9.html\",...

Chunk and asset paths:

printf 'export const v = "from é";\n' > 'é.ts'
printf 'const m = await import("./é.ts");\nconsole.log(m.v);\n' > entry.ts
bun build ./entry.ts --target=bun --splitting --outdir out3 && bun out3/entry.js
# before: error: Cannot find module './é-630mkta0.js' from '.../out3/entry.js'
# after:  from é            (entry.js contains import("./\xE9-630mkta0.js"))

advance under-count in a browser build (no // @bun involved):

cat > a.ts <<'EOF'
export function first() { return "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; }
export function second() { return "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; }
export function third() { return "é"; }
EOF
printf '<svg/>' > logo.svg
printf 'import { first, second, third } from "./a";\nimport logo from "./logo.svg";\nconsole.log(first(), second(), third(), logo);\n' > entry.ts
bun build ./entry.ts --target=browser --minify-whitespace --sourcemap=external --outdir out4

Decoded mappings for a.ts line 2 and 3 (function second, function third), generated column recorded vs. where the token is:

before:  second -> 80  (token at 86)    third -> 167 (token at 173)    entry.ts console.log -> 239 (correct)
after:   second -> 86                   third -> 173                   entry.ts console.log -> 239

The shift for ./logo-7cr55x2a.svg (19 bytes replacing a 25-byte key, so -6) was applied to everything past column ~46 instead of past the key, because everything in front of the é was left out of the recorded position.


no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/bundler/bun-build-api.test.ts test/bundler/bundler_loader.test.ts test/bundler/metafile.test.ts

…n front of non-ASCII in source map shifts

Chunks printed for bun start with "// @Bun", which the runtime loads as
Latin-1 without re-parsing, so the printer escapes everything in them to
ASCII. The paths and HTML import manifests spliced in afterwards were
written as raw UTF-8, and the manifest module itself was printed as
browser code because generate_server_html_module left its target at the
parser default. A non-ASCII file name therefore produced a bundle that
failed to load (or resolved mojibake paths).

LineColumnOffset::advance skipped the ASCII run in front of every
newline or non-ASCII character it found, so a shift recorded after any
non-ASCII text on the same line was too small and the mappings around
the splice point were off.
@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 7:59 PM PT - Aug 14th, 2026

@robobun, your commit 9590770fb616178b4faeca31db168d9c050403b6 passed in Build #96786! 🎉


🧪   To try this PR locally:

bunx bun-pr 38666

That installs a local version of the PR into your bun-38666 executable, so you can run:

bun-38666 --bun

@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Status: reproduced both reported symptoms on a build of current main (with #38429): bun build --target=bun of a server file importing é.html fails to load with SyntaxError: Invalid character '\u00a9', and with --minify --sourcemap=inline both frames map to s.ts:6:6 (ASCII name: 4:13 / 6:1). Same root cause also reproduced for --splitting chunk paths and file-loader asset paths with non-ASCII names, for a quote in an asset name on every target, and for the LineColumnOffset::advance under-count in a plain browser build.

Fix and tests are in this PR (description has the details and repro transcript). The six new tests fail on that build and pass with the branch, on Linux and, for the ones that can run there, on Windows; the metafile regression found in review is fixed in aede9d4. All review threads are resolved. CI on 12f0113 (build 96742): every lane that exercises this change is green, including the two Windows failures from the first push. The only test red on every retry was test/bake/deinitialization.test.ts (dev server teardown segfault on Windows x64), which also fails on main (5 of the last 40 main builds) and does not touch anything in this diff; it is reported to main-break triage. The rest were retried-and-passed flakes in unrelated areas. Re-running CI once for a clean build.

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5f69c6c7-0f20-464b-8bd1-9210d89b73b2

📥 Commits

Reviewing files that changed from the base of the PR and between 9cff2a1 and aede9d4.

📒 Files selected for processing (11)
  • src/bundler/Chunk.rs
  • src/bundler/HTMLImportManifest.rs
  • src/bundler/bundle_v2.rs
  • src/bundler/lib.rs
  • src/bundler/linker_context/MetafileBuilder.rs
  • src/sourcemap/lib.rs
  • test/bundler/bun-build-api.test.ts
  • test/bundler/bundler_bun.test.ts
  • test/bundler/bundler_loader.test.ts
  • test/bundler/html-import-manifest.test.ts
  • test/bundler/metafile.test.ts
💤 Files with no reviewable changes (1)
  • src/bundler/lib.rs

Walkthrough

The bundler centralizes splice escaping and path writing for raw output, JavaScript, Bun ASCII-only output, and metafile JSON. HTML manifests and metafiles use dedicated generation paths. Source-map offset accounting and regression tests cover non-ASCII and quoted paths.

Changes

Splice output generation

Layer / File(s) Summary
Splice escaping and output generation
src/bundler/Chunk.rs
SpliceEscape selects the output encoding. Chunk, standalone, HTML manifest, and metafile generation share path sizing, escaping, writing, and source-map shift handling.
Manifest and metafile integration
src/bundler/HTMLImportManifest.rs, src/bundler/bundle_v2.rs, src/bundler/lib.rs, src/bundler/linker_context/MetafileBuilder.rs
HTML manifest JSON writing accepts the ASCII-only policy. Generated HTML modules inherit the importing target. Metafiles use code_for_metafile.
Source-map offset correction
src/sourcemap/lib.rs, test/bundler/bun-build-api.test.ts, test/bundler/html-import-manifest.test.ts
Column advancement counts skipped ASCII bytes. Tests validate mappings around non-ASCII text and spliced paths.
Path escaping regression coverage
test/bundler/bundler_bun.test.ts, test/bundler/bundler_loader.test.ts, test/bundler/html-import-manifest.test.ts, test/bundler/metafile.test.ts
Tests cover Unicode and quoted paths in bundled JavaScript, file-loader output, HTML manifests, and metafile JSON.

Possibly related PRs

  • oven-sh/bun#36982: Related HTML bundling and sourcemap handling changes in different code paths.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main fixes for JavaScript path escaping and source map column shifts.
Description check ✅ Passed The description explains the problem, implementation, verification steps, regression coverage, and platform-specific limitations in detail.

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. bundler: keep the import kind of server-side HTML imports so import() and require() of the manifest link #38621 - Carries the byte-identical ast_for_html_entrypoint.target = target; fix in generate_server_html_module for the same reason: the HTML manifest module keeps the parser default Browser target instead of the importing side's.
  2. bundler: give the browser side of a server build its own copy of the runtime #38376 - Also contains the same one-line ast_for_html_entrypoint.target = target; hunk with the same comment and rationale, alongside its larger per-target runtime-copy change.
  3. bundler: JS-escape file-loader asset paths at chunk assembly #34188 - Rewrites the same IntermediateOutput::get_size / code_with_source_map_shifts passes in src/bundler/Chunk.rs to escape spliced asset paths instead of copying them raw; differs only in escape set and gating.

🤖 Generated with Claude Code

…e same way in both assembly passes

The sizing pass used to measure the path before the Windows separator
normalization the writing pass applies, which was harmless while both
copied the bytes through but over-counted once the path was escaped
(each backslash counted twice), leaving NUL bytes at the end of the
chunk on Windows. Both passes now resolve the path through one helper.

Escaping now applies to every JS chunk, not only to chunks built for
bun: the placeholder is always the body of a double-quoted literal, so
a quote in an asset name broke the output for every target. ASCII-only
escaping stays specific to chunks that carry the bun pragma.

The manifest test no longer asserts the manifest's input paths, which
are a separate Windows problem.
Comment thread src/bundler/Chunk.rs Outdated
Comment thread src/bundler/Chunk.rs Outdated
Comment thread src/bundler/Chunk.rs Outdated
Comment thread src/bundler/Chunk.rs Outdated
Comment thread src/bundler/HTMLImportManifest.rs Outdated
Comment thread src/bundler/bundle_v2.rs Outdated
Comment thread src/bundler/Chunk.rs
Comment thread src/bundler/Chunk.rs
Comment thread src/bundler/HTMLImportManifest.rs
@robobun robobun changed the title bundler: splice ASCII into chunks built for bun and fix the column count source map shifts use after non-ASCII text bundler: escape what is spliced into JS chunks (ASCII for bun), and fix the column count source map shifts use after non-ASCII text Aug 14, 2026
@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

On the three PRs listed above:

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/bundler/Chunk.rs:478-491SpliceEscape::for_chunk is also applied to the metafile JSON: MetafileBuilder::generate calls intermediate.code(..., &chunks[0], ...) on the assembled metafile, and chunks[0] is always a JS chunk, so under --target=bun this returns JsString { ascii_only: true } and write_spliced_path escapes the replacement path with \xNN (json=false at js_printer/lib.rs:1037) — not a valid JSON escape. Repro: await import("./modülé.ts") built with --target=bun --splitting --metafile=meta.json → the metafile's imports[].path becomes "./mod\xFCl\xE9-<hash>.js" and JSON.parse rejects it (before this PR the path was written raw, valid UTF-8). The metafile call site needs SpliceEscape::Raw (or a json=true variant).

    Extended reasoning...

    What the bug is

    code_with_source_map_shifts now derives its escape mode from the chunk argument via SpliceEscape::for_chunk(chunk, linker_graph) (Chunk.rs:780). That is correct for the per-chunk call sites in generateChunksInParallel, but there is one more caller: MetafileBuilder::generate (linker_context/MetafileBuilder.rs:442) resolves unique-key placeholders in the assembled metafile JSON by calling intermediate.code(..., &chunks[0], chunks, ...). chunks[0] is a stand-in — the metafile is not a JS chunk — but for_chunk treats it as one.

    Code path

    1. computeChunks sorts JS chunks first (computeChunks.rs), so chunks[0] is always a JS entry chunk. Under --target=bun, linker_graph.ast.items_target()[chunks[0].entry_point.source_index()].is_bun() is true, so SpliceEscape::for_chunk returns JsString { ascii_only: true }.
    2. computeCrossChunkDependencies rewrites each dynamic-import record that points at another chunk to record.path.text = other_chunk.unique_key and sets record.source_index = INVALID (computeCrossChunkDependencies.rs:177-178).
    3. MetafileBuilder reads parse_graph.ast.items_import_records() and, because record.source_index.is_valid() is false, falls through to record.path.text (MetafileBuilder.rs:300-311) and JSON-quotes it via format_json_string_utf8. The 25-byte unique key is pure ASCII, so it survives quoting byte-for-byte.
    4. break_output_into_pieces (MetafileBuilder.rs:433) finds that key inside the JSON "..." literal and produces a QueryKind::Chunk piece for it.
    5. write_spliced_path (Chunk.rs:598-609) runs the replacement final_rel_path through write_pre_quoted_string_inner::<_, Utf8>(path, writer, b'"', ascii_only=true, json=false). For c <= 0xFF && !json (js_printer/lib.rs:1037-1039) that emits \xNN, which RFC 8259 §7 does not permit in a JSON string.

    Step-by-step repro

    // entry.ts
    const m = await import("./modülé.ts");
    console.log(m.v);
    // modülé.ts
    export const v = 1;
    

    bun build ./entry.ts --target=bun --splitting --metafile=meta.json --outdir out

    • The dynamic import becomes a separate chunk modülé-<hash>.js; its import record's path.text is rewritten to the unique key and source_index cleared.
    • The metafile's outputs["out/entry.js"].imports entry for it is written as "path": "<unique key>", which break_output_into_pieces splits.
    • With escape = JsString { ascii_only: true }, ü (U+00FC) is written as \xFC and é (U+00E9) as \xE9, giving "path": "./mod\xFCl\xE9-<hash>.js".
    • JSON.parse(await Bun.file("meta.json").text()) throws SyntaxError: JSON Parse error at the \x.

    Before this PR, code_with_source_map_shifts did remain[..].copy_from_slice(cheap_normalizer[i]) — the raw UTF-8 bytes — which is valid inside a JSON string (only ", \, and C0 controls need escaping, and those cannot appear in a chunk file name). So this is a regression.

    Even without --target=bun, chunks[0] is still a JS chunk so for_chunk returns JsString { ascii_only: false }, which would emit \v / \x07 for control characters — also invalid JSON — but that case requires a control character in a chunk name and is much less likely in practice than the non-ASCII case above.

    Why nothing else prevents it

    SpliceEscape::for_chunk keys off chunk.content.is_javascript(); there is no variant for "the surrounding text is JSON" (or "raw, as before"), and the metafile call site cannot supply one. The PR description says "CSS and HTML chunks are written exactly as before", which is true for postProcess{CSS,HTML}Chunk, but this call site is neither. The PR's new tests cover JS chunk output (bun/NonAsciiChunkAndAssetPaths uses --splitting but not metafile) and the PR ran metafile.test.ts, but that suite has no non-ASCII chunk names.

    Impact

    --metafile is a documented flag; a build with --target=bun --splitting --metafile and any dynamically-imported file whose name contains a Latin-1 non-ASCII character produces a metafile that no JSON parser accepts. This is exactly the class of input (--target=bun + non-ASCII names) the PR sets out to fix, so it is in scope for this PR rather than pre-existing.

    Fix

    The metafile's placeholders sit inside JSON "..." literals, not JS ones, so the two correct choices are:

    • SpliceEscape::Raw for this call site (byte-identical to the pre-PR behavior; UTF-8 is valid in a JSON string, and format_json_string_utf8 already escaped anything that needed it in the surrounding text), or
    • a SpliceEscape::JsonString variant that calls write_pre_quoted_string_inner with json = true (emits ü instead of \xFC).

    Either way the escape mode has to be supplied by the caller here rather than derived from the dummy &chunks[0] — e.g. thread an Option<SpliceEscape> into code()/code_with_source_map_shifts() that overrides for_chunk, and have MetafileBuilder::generate pass Some(SpliceEscape::Raw). Add a metafile test with a non-ASCII dynamically-imported chunk name that JSON.parses the result.

The metafile resolves its chunk references through the same assembly
routine as the chunks, with the first chunk standing in, so the escape
mode picked for that chunk (a JS string, ASCII-only for bun) was applied
inside JSON. The metafile now asks for JSON escaping explicitly.
Comment thread src/bundler/Chunk.rs Outdated
Comment thread src/bundler/linker_context/MetafileBuilder.rs Outdated
Comment thread src/bundler/Chunk.rs
@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

The metafile finding from the review above was real: the metafile resolves its chunk references through the same routine with chunks[0] standing in, so the escape mode picked for that chunk was applied inside JSON ("./mod\xFCl\xE9-....js", which JSON.parse rejects). Fixed in aede9d4: the escape mode is now passed in explicitly, code() derives it from the chunk, and the metafile goes through code_for_metafile(), which escapes as JSON string content. On the repro the metafile is byte-identical to what the release writes; test/bundler/metafile.test.ts gets a test with a q"modülé.js chunk that fails both on the release (raw quote) and on the intermediate state of this branch (\xFC). The description is updated accordingly. Nothing else outstanding on my side; waiting on CI for 12f0113.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it reworks the two-pass chunk assembly (sizing and writing must agree byte-for-byte) across four output modes and overlaps with three other open PRs (#38621, #38376, #34188), a human look would still be worthwhile.

What was reviewed:

  • SpliceEscape::for_chunk uses the same items_target()[entry_point.source_index()].is_bun() predicate as postProcessJSChunk for the pragma, so ascii_only matches when // @bun is emitted.
  • spliced_path_parts normalizes separators before both passes, and both passes size/write via the same write_spliced_pathwrite_pre_quoted_string_inner, so the escaped byte counts agree.
  • LineColumnOffset::advance now adds i - offset for the skipped ASCII run before handling the stop character; the newline arm resets columns afterward so the addition is a no-op there, and the trailing-remain path is unchanged.
  • code_for_metafile forces SpliceEscape::JsonString regardless of chunks[0]'s content, so a bun-target first chunk no longer leaks \xNN into the metafile JSON.
Extended reasoning...

Overview

This PR fixes three related bundler bugs around non-ASCII and quote characters in file names when they are spliced into already-printed chunks: (1) the generated HTML manifest module was not tagged with the importing side's target, so its identifier was printed unescaped in a // @bun chunk; (2) chunk/asset paths and the manifest JSON were spliced raw into "..." literals, breaking Latin-1 loading for bun and producing unparseable output for quotes on every target; (3) LineColumnOffset::advance under-counted columns by dropping the ASCII run before each non-ASCII stop, mis-placing the source-map shift boundary. It touches Chunk.rs (new SpliceEscape enum, shared spliced_path_parts/write_spliced_path helpers, code_for_metafile), HTMLImportManifest.rs (adds ascii_only param, deletes the Display/EscapedJson scaffolding), bundle_v2.rs (one-line target assignment), MetafileBuilder.rs, sourcemap/lib.rs, and adds six tests across five test files.

Security risks

None identified. The change escapes more (paths inside string literals and JSON), not less; input is file names the user already controls, and the escaper is the existing write_pre_quoted_string_inner used by the printer. No auth, network, or untrusted-input parsing paths are touched.

Level of scrutiny

High. code_with_source_map_shifts is the two-pass assembly of every output chunk: the sizing pass allocates exactly count bytes and the writing pass fills them, ending with debug_assert!(remain.is_empty()). Any divergence between the passes either leaves NUL bytes (over-count) or panics/UB (under-count into a fixed buffer). The PR itself documents that the first push over-counted on Windows (pre-normalization path measured, post-normalization path written) and that review caught the metafile picking up the JS chunk's escape mode. Both are fixed, and the shared-helper design (spliced_path_parts + write_spliced_path + DiscardingWriter) structurally prevents the class, but the surface area — JS/CSS/HTML/standalone/metafile × ascii-only/not × Windows/POSIX paths — is large enough that a maintainer should confirm the escape-mode selection is right for every caller and that CSS url() splices genuinely want Raw.

Other factors

  • The bundle_v2.rs hunk is deliberately byte-identical to #38621 and #38376, and the Chunk.rs change supersedes #34188's approach. A human should decide landing order and whether #34188's extra tests should be folded in.
  • All comment-cop bot threads are resolved (comments were shortened or justified as doc comments).
  • Test coverage is thorough: each of the three root causes has a test that fails on main, plus the metafile regression found in review; Windows-incompatible file names are correctly skipIf(isWindows).
  • The code_with_source_map_shifts visibility change from pub(crate) to private is safe — the only external caller (MetafileBuilder) now goes through code_for_metafile.
  • The file_path_buf scratch buffer is now used in the sizing pass too; it was already acquired at the top of the function, so no new allocation.

@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

On the one open question in the review above (whether CSS and HTML chunks should really get SpliceEscape::Raw): Raw is exactly what those chunks got before this PR, byte for byte, so this PR neither improves nor changes them. Their placeholders sit in url(...), @import "...", src="..." and href="...", each of which needs its own escaping rules; that is a separate change (the same boundary #34188 drew) and deliberately not part of this one, which is about the text that lands inside JS and JSON string literals.

Current state for whoever picks this up: head is 9590770 (an empty retrigger on top of 12f0113, because build 96742 was red only on test/bake/deinitialization.test.ts, a Windows dev-server teardown crash that also fails on main and is reported separately); all review threads are resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants