Skip to content

fix(transpiler): scanImports drops dynamic import() with non-ASCII specifier - #34015

Open
robobun wants to merge 1 commit into
mainfrom
farm/ad00a32c/scan-imports-nonascii-dynamic
Open

fix(transpiler): scanImports drops dynamic import() with non-ASCII specifier#34015
robobun wants to merge 1 commit into
mainfrom
farm/ad00a32c/scan-imports-nonascii-dynamic

Conversation

@robobun

@robobun robobun commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

What

Bun.Transpiler.scanImports() silently dropped every dynamic import() whose specifier contains a non-ASCII character, while scan() and static imports with the same specifier were reported.

const t = new Bun.Transpiler({ loader: "ts" });
t.scanImports(`import("./café");`)        // []   <- dropped
t.scanImports(`import("./caf\u00e9");`)   // []   <- dropped even with pure-ASCII source
t.scan(`import("./café");`).imports       // [{ kind: "dynamic-import", ... }]
t.scanImports(`import a from "./café";a`) // [{ kind: "import-statement", ... }]

Why

The SCAN_ONLY branch of parse_import_expr only called add_import_record when e_string.is_utf8() was true. Any non-ASCII code point makes the lexer store the literal as UTF-16, so the guard failed and the branch fell through without recording the import.

The full parse records the same import via transpose_import, which calls EString::slice() without the guard. slice() already handles both encodings (it transcodes UTF-16 to UTF-8 via string()).

Fix

Drop the is_utf8() guard; slice() handles both encodings.

The test asserts scanImports() matches scan().imports for non-ASCII dynamic imports rather than hardcoding path bytes, since the path text is currently Latin-1-decoded in named_imports_to_js (pre-existing, affects static imports and scan() too, tracked separately).


[stamp-90s] gate passed · iteration 0 · 2 files touched

fails on main (without fix)
ASAN without fix: 1 failed, 22 skipped
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/bundler/transpiler/transpiler.test.js
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (b4f409584)

test/bundler/transpiler/transpiler.test.js:
(pass) Bun.Transpiler > handles errors when parsing macros [5.70ms]
(pass) Bun.Transpiler > normalizes \r\n [6.57ms]
1
(pass) Bun.Transpiler > doesn't hang indefinitely #2746 [4.25ms]
(pass) Bun.Transpiler > property access inlining > bails out with spread [6.77ms]
(pass) Bun.Transpiler > property access inlining > bails out with multiple items [2.71ms]
(pass) Bun.Transpiler > property access inlining > works [2.43ms]
(pass) Bun.Transpiler > property access inlining > works nested [2.33ms]
(pass) Bun.Transpiler > TypeScript > import Foo = Baz.Bar [2.87ms]
(pass) Bun.Transpiler > TypeScript > ternary should parse correctly when parsing typescript fails [2.31ms]
(pass) B
... (truncated)

release without fix: 1 failed, 22 skipped
bun test v1.4.0-canary.1 (1498d7b77)

test/bundler/transpiler/transpiler.test.js:
(pass) Bun.Transpiler > handles errors when parsing macros [0.16ms]
(pass) Bun.Transpiler > normalizes \r\n [0.20ms]
1
(pass) Bun.Transpiler > doesn't hang indefinitely #2746 [0.13ms]
(pass) Bun.Transpiler > property access inlining > bails out with spread [0.13ms]
(pass) Bun.Transpiler > property access inlining > bails out with multiple items [0.03ms]
(pass) Bun.Transpiler > property access inlining > works [0.03ms]
(pass) Bun.Transpiler > property access inlining > works nested [0.03ms]
(pass) Bun.Transpiler > TypeScript > import Foo = Baz.Bar [0.05ms]
(pass) Bun.Transpiler > TypeScript > ternary should parse correctly when parsing typescript fails [0.06ms]
(pass) Bun.Transpiler > TypeScript > contextual keywords used as plain identifiers keep their statements [0.27ms]
(pass) Bun.Transpiler > TypeScript > does not crash when export default abstract is an expression followed by a class [0.24ms]
(pass) Bun.Transpiler > TypeScript > scope tracking stays balanced when a contextual keyword starts a larger expression [0.24ms]
(pass) Bun.Transpiler > TypeScript > scope tracking stays balan
... (truncated)
passes on PR (with fix)
ASAN with fix: 22 skipped
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/bundler/transpiler/transpiler.test.js
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (b4f409584)

test/bundler/transpiler/transpiler.test.js:
(pass) Bun.Transpiler > handles errors when parsing macros [5.40ms]
(pass) Bun.Transpiler > normalizes \r\n [6.72ms]
1
(pass) Bun.Transpiler > doesn't hang indefinitely #2746 [4.48ms]
(pass) Bun.Transpiler > property access inlining > bails out with spread [6.80ms]
(pass) Bun.Transpiler > property access inlining > bails out with multiple items [2.68ms]
(pass) Bun.Transpiler > property access inlining > works [2.43ms]
(pass) Bun.Transpiler > property access inlining > works nested [2.26ms]
(pass) Bun.Transpiler > TypeScript > import Foo = Baz.Bar [2.84ms]
(pass) Bun.Transpiler > TypeScript > ternary should parse correctly when parsing typescript fails [2.59ms]
(pass) B
... (truncated)

release with fix: 22 skipped
$ bun scripts/build.ts --profile=release
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
[configured] bun-profile → bun (stripped) in 818ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/21] gen generated_host_exports.rs
generated_host_exports.rs: 91 exports (host=3, lazy=10, generic=78, rust=0); 243 extern-C blocks audited
[2/21] gen JS modules (bundle-modules)
Preprocess modules (6758ms)
Bundle modules (36ms)
Postprocesss modules (31ms)
Bundle Functions (744ms)
Generate Code (86ms)

[7.67s] Bundled "src/js" for production
  1913 kb
  162 internal modules
  12 native modules
  90 internal functions across 19 files
[2/6] cargo bun_bin → libbun_rust.a (--target x86_64-unknown-linux-gnu)
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: component rust-std is up to date

  nightly
... (truncated)
diff hotspot
src/js_parser/parse/parse_import_export.rs |  2 +-
 test/bundler/transpiler/transpiler.test.js | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 1 deletion(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                        reads  edits  tests
src/js_parser/parse/parse_import_export.rs      1      1      0
test/bundler/transpiler/transpiler.test.js      1      2      0

…ecifier

The SCAN_ONLY branch of parse_import_expr only recorded a dynamic import
when the string literal was stored as UTF-8 bytes. Any non-ASCII code
point (including one produced by a \u escape in pure-ASCII source) makes
the lexer store the literal as UTF-16, so is_utf8() is false and the
branch fell through without calling add_import_record.

EString::slice() already transcodes UTF-16 to UTF-8 (same call the full
parse uses in transpose_import), so the is_utf8() guard is unnecessary.
@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 10 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 04e87eed-b519-40a1-9e3d-e334970209fe

📥 Commits

Reviewing files that changed from the base of the PR and between b16ca17 and b4f4095.

📒 Files selected for processing (2)
  • src/js_parser/parse/parse_import_export.rs
  • test/bundler/transpiler/transpiler.test.js

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 2:12 AM PT - Jul 12th, 2026

@robobun, your commit b4f4095 has 3 failures in Build #72106 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34015

That installs a local version of the PR into your bun-34015 executable, so you can run:

bun-34015 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — one-line guard removal that aligns the SCAN_ONLY path with transpose_import, plus a parity test.

What was reviewed:

  • Verified EString::slice() (src/ast/e.rs:1920) transcodes UTF-16 via string(), so dropping is_utf8() is safe.
  • Confirmed transpose_import (src/js_parser/p.rs:1087) already calls slice() without the guard — this just brings scanImports to parity.
  • Test covers literal non-ASCII, \u/\u{} escapes, CJK, emoji, and an ASCII control case; asserts scanImports == scan().imports rather than hardcoding path bytes.
Extended reasoning...

Overview

Removes the e_string.is_utf8() guard in the SCAN_ONLY branch of parse_import_expr (src/js_parser/parse/parse_import_export.rs:75), leaving only is_present(). Adds a test in test/bundler/transpiler/transpiler.test.js that iterates six dynamic-import specifiers (non-ASCII literal, \u00e9, \u{e9}, CJK, emoji, plain ASCII) and asserts scanImports() returns exactly one dynamic-import entry that matches scan().imports.

Security risks

None. This is parser bookkeeping — the only effect is that a dynamic import specifier containing non-ASCII characters is now recorded in the import-record list during scan-only mode, matching what the full parse already does. No untrusted-size arithmetic, no allocation-size changes driven by user input, no auth/crypto/permissions.

Level of scrutiny

Low. The diff is a single-token deletion whose correctness is directly verifiable against existing code: EString::slice() (src/ast/e.rs:1920-1923) calls string(), which branches on is_utf8() and transcodes UTF-16 via strings::to_utf8_alloc when false — so the removed guard was redundant. The full-parse path (transpose_import, src/js_parser/p.rs:1086-1087) already calls str_.slice(self.arena) with no is_utf8() check, so this change brings the two paths to parity rather than introducing new behavior.

Other factors

  • The is_present() guard is retained, so empty-string specifiers are still skipped (no behavior change there).
  • The test wisely asserts parity between scanImports() and scan().imports with expect.any(String) for the path, sidestepping the pre-existing Latin-1 decoding quirk in named_imports_to_js that the PR description explicitly calls out as tracked separately. Including ./plain as a control case guards against the loop trivially passing.
  • No CODEOWNERS entry matches src/js_parser/ or the transpiler test file.
  • No prior human review comments or outstanding requests on the PR.

@robobun

robobun commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator Author

CI build #72106: the only hard failure is test/integration/next-pages/test/dev-server-ssr-100.test.ts on darwin 26 aarch64, where bun install SIGTERM'd during Next.js dependency install. That test also fails on unrelated build #72137 (webkit-mimalloc branch) and is listed in this build's flaky annotation for debian 13 aarch64.

The changed code path (SCAN_ONLY branch of parse_import_expr) is only reachable via Bun.Transpiler.scanImports(); bun install does not exercise it. transpiler.test.js passed on all lanes. Diff is ready for review.

@robobun

robobun commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator Author

The dev-server-ssr-100.test.ts RED on build 72106 is unrelated to this diff: a transient npm hang on one tart agent timed the test out, and five deliberate crash reports from run-crash-handler.test.ts / native-plugin.test.ts got pinned on it so it lost its retries. Addressed in #34024.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant