Skip to content

module loader: mark has_loaded before transpiler-cache early return - #35482

Closed
robobun wants to merge 2 commits into
mainfrom
farm/cfc64862/fix-has-loaded-cache-hit
Closed

module loader: mark has_loaded before transpiler-cache early return#35482
robobun wants to merge 2 commits into
mainfrom
farm/cfc64862/fix-has-loaded-cache-hit

Conversation

@robobun

@robobun robobun commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

What

When the entry point is >= 4 KiB (the runtime transpiler cache's MINIMUM_CACHE_SIZE) and a cache entry exists for it, an ESM import x from "./file.c" (or any other unknown extension) is parsed as TSX instead of going through the file loader. The first run of the program works; every subsequent run fails with:

1 | #include <stdio.h>
    ^
error: Unexpected #include
    at .../impl.c:1:1

Repro

mkdir /tmp/repro && cd /tmp/repro
printf '#include <stdio.h>\n' > impl.c
printf 'import s from "./impl.c"; console.log(typeof s, s);\n//%.0s' {1..5000} > entry.ts
bun entry.ts   # string /tmp/repro/impl.c
bun entry.ts   # error: Unexpected #include at .../impl.c:1:1

Cause

transpile_source_code_inner sets vm.has_loaded = true only after the printer runs, but several success paths return earlier: the bytecode/already-bundled arm, the empty .cjs/.cts arm, and the runtime-transpiler-cache hit. On a warm run the cache-hit return fires before has_loaded is set.

transpile_file's synchronous_loader block uses has_loaded || is_in_preload to decide between Loader::File (unknown extension inside the module graph) and Loader::Tsx ("potentially the main module"). With has_loaded still false, the .c import takes the Tsx branch.

Fix

Move the has_loaded = true assignment to immediately after the parse-error check, so every successful parse of the entry point (cache hit or not) sets it before any early return.

Test

Added a case to test/cli/run/transpiler-cache.test.ts that runs a >4 KiB entry importing a .c file twice against an isolated cache dir, asserting the file-loader result on both the cold and warm run and that the cache was populated in between.

When the entry point is >= MINIMUM_CACHE_SIZE (4 KiB) and the runtime
transpiler cache hits, transpile_source_code_inner returns early before
reaching the has_loaded = true assignment. The synchronous_loader block
in transpile_file then falls through to the "potentially the main
module" Tsx fallback for any subsequent unknown-extension import, so
`import x from "./file.c"` is parsed as JS instead of using the file
loader on every run after the first.

Move the has_loaded assignment to just after the parse-error check so
it is set before any of the successful early returns (bytecode cache,
empty .cjs/.cts, transpiler-cache hit, async module queue).
@robobun

robobun commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 4:23 PM PT - Jul 24th, 2026

@autofix-ci[bot], your commit 1443a0d is building: #79784

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: b3de42c5-18ec-4669-a133-696820b58f00

📥 Commits

Reviewing files that changed from the base of the PR and between 028f7a3 and 1443a0d.

📒 Files selected for processing (2)
  • src/runtime/jsc_hooks.rs
  • test/cli/run/transpiler-cache.test.ts

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. Runtime transpiler cache causes unknown-extension ESM imports to use the TSX loader on a warm cache #34437 - Describes the exact same bug: runtime transpiler cache hit on the entry point skips setting has_loaded, causing unknown-extension ESM imports (e.g. .svg, .c) to fall back to the TSX loader on warm runs

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #34437

🤖 Generated with Claude Code

@robobun

robobun commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #33371, which is the same fix (move has_loaded = true before the early-return paths in transpile_source_code_inner) with broader test coverage: it also covers the require.extensions case on a cache hit and the // @bun already-bundled entry-point case. That PR is green on CI.

@robobun robobun closed this Jul 24, 2026
@robobun
robobun deleted the farm/cfc64862/fix-has-loaded-cache-hit branch July 24, 2026 23:25
@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. Set has_loaded on every path that produces the entry point's source #33371 - Also fixes has_loaded not being set on all code paths in transpile_source_code_inner, addressing the same loader selection bug for unknown-extension imports

🤖 Generated with Claude Code

// On the warm run the entry point is restored from cache; the `.c` import
// must still go through the file loader.
const warm = run("warm run (cache hit)");
expect(warm.stderr).not.toContain("Unexpected #include");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 This assertion can never fail: if the warm run printed Unexpected #include, the subprocess would exit non-zero and run() would already have thrown at if (!result.success) throw new Error(...) before this line is reached. The next line's expect(warm.stdout).toBe("file-loader-ok") is the load-bearing check — either drop this line, or have run() return {stdout, stderr, exitCode} and assert the combined object so this becomes reachable.

Extended reasoning...

What the issue is

The assertion expect(warm.stderr).not.toContain("Unexpected #include") is vacuous — there is no execution path on which it can fail. REVIEW.md's test rules say "Every assertion must be able to fail, and assert the strongest invariant. Hunt vacuous patterns."

Why it can't fail

The local run() helper does:

if (!result.success) throw new Error(`${label}: ${stderr}\n${stdout}`);
return { stdout, stderr };

So the only way execution reaches the .not.toContain(...) line is if result.success is true (exit code 0). But Unexpected #include is the parse error produced when Bun tries to parse impl.c as TSX via a static ESM import — a fatal module-load error that always causes a non-zero exit. There is no realistic path where Bun exits 0 and prints that parse-error text to stderr.

Step-by-step proof

  1. Regression case (unfixed build, warm run): import src from "./impl.c" is parsed as TSX → error: Unexpected #include on stderr → process exits non-zero → result.success is false → run() throws Error("warm run (cache hit): error: Unexpected #include ..."). The test fails at the throw, before line 257 is evaluated.
  2. Fixed build, warm run: file loader is used → stdout is file-loader-ok, stderr has no parse error → exit 0 → run() returns → line 257 evaluates expect(<no error text>).not.toContain("Unexpected #include") → trivially passes.
  3. There is no third branch: a static-import parse error cannot coexist with exit 0.

Therefore in every reachable state the assertion passes; it contributes nothing to the test's ability to catch the regression.

Impact

Low. The test as a whole is not vacuous — on regression it still fails via the throw inside run() (which helpfully embeds stderr, so the failure message even contains "Unexpected #include"), and expect(warm.stdout).toBe("file-loader-ok") on the next line is a strict positive assertion. Only this single line is dead weight. It doesn't hurt correctness or coverage; it just reads as though it's guarding something when it isn't.

How to fix

Two options:

  • Delete the line. The throw + the .toBe("file-loader-ok") assertion already cover both the negative and positive contract.
  • Make it reachable. Change run() to return { stdout, stderr, exitCode } without throwing, then assert on a combined object per REVIEW.md's subprocess guidance ("assert a combined { stdout, stderr, exitCode } object"), e.g. expect({ stdout: warm.stdout, exitCode: warm.exitCode }).toEqual({ stdout: "file-loader-ok", exitCode: 0 }). That way a regression produces a diff showing the actual stderr instead of a thrown Error.

Comment thread src/runtime/jsc_hooks.rs
Comment on lines +2660 to +2661
// `Tsx` to `File`; leaving it unset on a cache hit made a second
// run of `bun entry.ts` parse `import x from "./a.c"` as JS.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The trailing clause of this comment ("leaving it unset on a cache hit made a second run of bun entry.ts parse import x from \"./a.c\" as JS") is past-tense bug history — REVIEW.md's Comments section says that belongs in the PR description, not the source. Keep the first five lines (the placement invariant and the cross-file dependency on transpile_file's synchronous_loader block) and drop just the trailing repro clause. Same applies to the 5-line block at test/cli/run/transpiler-cache.test.ts:227-231, which narrates the historical failure the test name already states as an invariant.

Extended reasoning...

What the finding is

Two of the new comments in this PR mix durable invariant documentation with past-tense bug narration, which REVIEW.md's Comments section explicitly separates:

Comments carry only durable non-obvious content: invariants, ownership/lifetime contracts, SAFETY justifications, deliberate deviations from upstream. No narrating what the code does, no bug history — that belongs in the PR description. Regression tests get exactly one comment: the issue URL.

Location 1 — src/runtime/jsc_hooks.rs:2655-2661

The first ~5 lines are exactly what the rule wants kept: they state the placement invariant ("before any of the early Ok(...) returns below") and the non-local, cross-file dependency ("has_loaded is what flips the loader fallback for unknown extensions — transpile_file's synchronous_loader block — from Tsx to File"). That's durable, non-obvious, and prevents a future refactor from re-hoisting the assignment.

The trailing clause, however, is bug history:

leaving it unset on a cache hit made a second run of bun entry.ts parse import x from "./a.c" as JS.

That's a past-tense description of the pre-fix failure mode — precisely what the PR description's Repro and Cause sections already record verbatim. Dropping it loses nothing; the invariant half of the comment already tells a future reader everything they need to keep the assignment where it is.

Location 2 — test/cli/run/transpiler-cache.test.ts:227-231

The 5-line block:

The runtime-transpiler-cache early return used to skip marking the VM's has_loaded flag, so on a warm run … an ESM import x from "./file.c" fell into the "potentially the main module" loader fallback and tried to parse the C source as TSX …

is entirely past-tense narration of what was broken. The test name — "cache hit on the entry point does not change loader selection for unknown-extension imports" — already states the invariant, and the two surviving inline comments (line 233 "Entry file large enough to be cached, importing an unknown-extension file" and the pre-warm-run comment "the .c import must still go through the file loader") already cover the non-obvious setup constraints (why 50 KiB of filler, why a .c import, why two runs).

Addressing the counter-argument ("local convention allows explanatory comments")

One reading holds that this file's convention permits multi-line explanatory blocks — the ignores files under the minimum cache size test explains the 4 KiB threshold, the --feature flag test annotates each stage, and the rejects cached module records test documents the on-disk cache-entry byte layout. But each of those is present-tense reference material (a threshold's rationale, a serialized layout, per-stage expected state) — durable content a reader needs to modify the test. None of them narrates a historical bug in the past tense. The distinction REVIEW.md draws is not "long comments vs short comments" but "invariants vs bug history", and the new block falls on the bug-history side ("used to skip", "fell into", "tried to parse").

The counter-argument also notes there is no GitHub issue URL to substitute. That's fine — the rule doesn't require one; it says regression tests get at most the issue URL. With no issue, the setup comments already present at line 233 and before the warm run are sufficient, or the block can be rephrased as a one-line present-tense invariant if the author prefers.

Suggested edit

jsc_hooks.rs — end the comment after "…from Tsx to File.":

// Mark the entry point as loaded as soon as it has parsed
// successfully, before any of the early `Ok(...)` returns below
// (bytecode, empty-cjs, runtime-transpiler-cache hit, etc.).
// `has_loaded` is what flips the loader fallback for unknown
// extensions (`transpile_file`'s `synchronous_loader` block) from
// `Tsx` to `File`.

transpiler-cache.test.ts — drop lines 227-231; the test name plus the two remaining inline comments already carry the durable content.

Severity

Nit. This is comment hygiene enforced by an explicit repo rule, but merging as-is causes no behavioral, correctness, or maintenance failure — it's a one-clause trim in the source and a block deletion in the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant