Handle 16-bit module sources in the error preview, the new-position fix-up and the REPL

[robobun]

First of a four PR stack split out of #33866 (1: this, 2: coverage line table, 3: disk sources decoded as UTF-8, 4: #33866 itself). Each part is a bug on main today with its own failing test; this one has no dependency on the others.

Problem

• A module whose text contains any non-ASCII character is held by JSC as a 16-bit string. Today that happens to every --compile executable with a non-ASCII preserved (/*! */) comment and to anything else loaded through String::clone_utf8 (plain bun build --target=bun output is still read as Latin-1 on main and joins them with Decode module text that comes from disk or bundler output as UTF-8 #38714). Three readers of module text assumed 8-bit:
  • src/jsc/bindings/ZigException.cpp: the source preview printed under an uncaught error was only built for 8-bit sources (it read them through span8()), so these modules print error: boom with no excerpt at all.
  • src/jsc/bindings/ErrorStackFrame.cpp, adjustPositionBackwards: when moving a constructor call's position back to its new keyword crosses a line it has to read the source; on a 16-bit source it hit an ASSERT_NOT_REACHED (a no-op in release) and reset the frame to 1:1, so the stack says at code (app:1:1).
  • src/jsc/bindings/bindings.cpp, Bun__REPL__evaluate: the program was decoded with WTF::String::fromUTF8, which returns a null string for ill-formed input. Preserved comments pass through the REPL's transpile verbatim, so one stray byte in one made .load silently define nothing.

Fix

• The preview indexes the WTF::String (works for both widths) and no longer requires 8-bit; adjustPositionBackwards drops the 8-bit check, since the offsets it walks are code units in either width; the REPL decodes with the same Bun::toString the module loader uses, which substitutes U+FFFD.
• Verification, both fail on main and pass here:
  • test/bundler/bundler_compile.test.ts compile/NoSourceMapNonAsciiSource: a compiled fixture with a non-ASCII preserved comment and a new (class ...)("boom") whose argument list is four lines below new. On main the output is error: boom followed by at code (/$bunfs/root/out:1:1); now it prints the excerpt (with the comment decoded) and :5:9, exactly what the ASCII twin of the fixture has always printed. The excerpt's context-line labels are not asserted; they are off by one for bundled modules today, independently of this change; error printer: fix the code frame lines above errors thrown from vm/eval sources #38244 is fixing that.
  • test/js/bun/repl/repl.test.ts: .load of a file containing /*! <0xE9> */ inside a function. On main the file defines nothing (ReferenceError); now it evaluates and Function.prototype.toString shows the U+FFFD.
  • Also run: the rest of bundler_compile's source map cases, the .load REPL tests, inspect-error.test.js (its two "minified file" cases fail on debug builds before and after this change: an extra internal frame), stack.test.ts, bundler_bun.test.ts.

Background

• JSC strings are either 8-bit (one Latin-1 character per byte) or 16-bit (UTF-16 code units). WTF::String::operator[] and length() work on either; span8() is only valid for 8-bit strings. Source positions JSC reports (divot, columns) are code unit offsets, so the same arithmetic is correct for both widths.
• The source preview is the N | code excerpt Bun prints above error: ... for an uncaught exception when no source map is involved; it is produced from the module text JSC holds.
• JSC places the position of a new X(...) at the argument list or the end of the callee; Bun moves it back to the new keyword so stacks point where V8's would. The move only needs the source text when new is on an earlier line.

---

no test proof · iteration 10 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/bundler/bundler_compile.test.ts test/js/bun/repl/repl.test.ts

[coderabbitai]

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6a6d379c-f9a7-4335-914b-15dab51e39e4

📥 Commits

Reviewing files that changed from the base of the PR and between ada2a67 and 6f9b751.

📒 Files selected for processing (5)

• src/jsc/bindings/ErrorStackFrame.cpp
• src/jsc/bindings/ZigException.cpp
• src/jsc/bindings/bindings.cpp
• test/bundler/bundler_compile.test.ts
• test/js/bun/repl/repl.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

I reviewed this PR and didn't find any bugs. Because it touches native C++ across three JSC-binding sites (stack-frame position fix-up, source-line preview, REPL source decoding) with string-width subtleties, a human look would still be worthwhile.

What was reviewed

• Bun::toStringView → toZigString(StringView) handles 16-bit via taggedUTF16Ptr, so the newly-reachable 16-bit source_lines[] path is well-formed; the existing provider->ref() still owns the backing storage.
• Bun::toString(const char*, size_t) → BunString__fromBytes → fromUTF8ReplacingInvalidSequences for the U+FFFD substitution; the sourceLen > 0 guard covers the ASSERT(length > 0) in BunString__fromBytes.
• transferToWTFString() on the WTFStringImpl tag balances the leaked +1 from BunString__fromBytes (ref+deref), so no leak/UAF on the REPL source string.
• Both new tests assert the specific pre-fix failure modes (missing excerpt / 1:1 frame; .load defining nothing) and would fail on main.

Extended reasoning...

Overview

The PR removes 8-bit-only assumptions from three readers of module source text in the JSC C++ bindings:

• src/jsc/bindings/ErrorStackFrame.cpp adjustPositionBackwards: deletes the !source.is8Bit() early-return that reset the frame to 1:1. The remaining code already indexes via source[i] (StringView::operator[]), which is width-agnostic; JSC divot/column offsets are code units in either width, so the arithmetic is unchanged.
• src/jsc/bindings/ZigException.cpp populateStackFramePosition: drops the sourceString.is8Bit() gate on the source-preview block and replaces the raw span8().data() byte pointer with sourceString[offset]. The variable rename (byte→code-unit) is cosmetic. The Bun::toStringView(sourceString.substring(...)) calls and provider->ref() lifetime handling are untouched.
• src/jsc/bindings/bindings.cpp Bun__REPL__evaluate: swaps WTF::String::fromUTF8 (returns null on ill-formed input) for Bun::toString(bytes, len).transferToWTFString(), which routes through BunString__fromBytes → fromUTF8ReplacingInvalidSequences.

Two new tests: a compiled fixture with a non-ASCII preserved comment and a multi-line new (class …) (covers both the preview and the position fix-up), and a REPL .load of a file with a stray 0xE9 in a preserved comment.

Security risks

None identified. The changes widen accepted inputs on already-trusted code paths (module source held by JSC, transpiled REPL program). No new user-controlled length arithmetic or allocation; indexing is bounded by sourceString.length() as before.

Level of scrutiny

Moderate-to-high. This is native C++ in the JSC bindings — the review guide's most-blocked category — and involves string-width representation (Latin-1 vs UTF-16). I verified: toZigString(StringView) tags 16-bit pointers so the Rust consumer sees the width; the compile test's toInclude("| /*! © café 中 */") assertion proves the full 16-bit print path decodes correctly; transferToWTFString() balances the ref left by BunString__fromBytes (constructs WTF::String(impl) = +1, then deref() = -1); and the sourceLen > 0 guard covers the ASSERT(length > 0) inside BunString__fromBytes.

Other factors

The PR description is unusually thorough (mechanism, before/after, verification against main, related suites run). The changes are mechanical — deleting a wrong guard and swapping to width-agnostic accessors — with no lifetime or ownership changes. I'm deferring rather than approving only because native JSC binding changes across three files with encoding subtleties are the kind a maintainer familiar with the Rust-side ZigString/BunString consumers should confirm end-to-end.

[robobun]

CI for 6f9b751 (#96691): 166 jobs green. The Windows lanes were cancelled because no agent could be created for them (fleet-wide at the moment), and the x64-asan lane's failures are all timeouts of unrelated leak and stress tests (require-cache, setInterval, fs readdir x100, sourcetextmodule-leak, html-rewriter-leak) on an overloaded agent; this diff touches the error excerpt, the frame position fix-up and REPL decoding, none of which those tests exercise. Reported for triage; not retriggering while the fleet is saturated.

[alii]

Read the whole of populateStackFramePosition and adjustPositionBackwards against a 16-bit source, traced every offset back to JSC (all code units: divot, startOffset, lineColumn), checked operator[] bounds at each use, the toStringView-into-provider lifetime (provider is ref'd, Rust reads the tagged UTF-16 pointer), and the REPL decoder (BunString__fromBytes, transferToWTFString net one ref, empty and Dead cases same as before). Both fixtures reproduce on a main build for the stated reason: .load defines nothing, and the compile fixture hits the ErrorStackFrame assert. No remaining is8Bit/span8 gate on module text in ZigException.cpp, ErrorStackFrame.cpp, ErrorStackTrace.cpp or NodeVM.cpp. Fine to land from my side.

Two body nits: bun build --target=bun output is not 16-bit on main (the // @Bun arms load it as Latin-1 until #38714), only --compile and anything through clone_utf8 are; and the context-line skip in the preview that the test avoids asserting is described as tracked separately with no link, worth adding one.

[robobun]

Thanks. Both body nits fixed: the Problem bullet now says only --compile and clone_utf8 loads are 16-bit on main (plain --target=bun output joins them with #38714), and the context-line note links #38244, which is fixing the excerpt.