Skip to content

bake: load files outside the bundle by their Windows path during prerendering - #39092

Open
robobun wants to merge 4 commits into
mainfrom
farm/bda95807/bake-prod-windows-abs-import
Open

bake: load files outside the bundle by their Windows path during prerendering#39092
robobun wants to merge 4 commits into
mainfrom
farm/bda95807/bake-prod-windows-abs-import

Conversation

@robobun

@robobun robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • On Windows, bun build --app fails to prerender a route that loads a file the bundler did not see, such as await import(join(import.meta.dir, "../extra/banner.mjs")) inside a page component. The build prints error: EINVAL reading "\\" followed by Route "pages\index.tsx" cannot be pre-rendered to a static page. and exits 1. The same project builds on Linux and macOS. Reproduced with the current canary on Windows Server 2019.
  • import.meta.dir is inlined as the source file's directory, so on Windows the specifier is a drive path (C:\app\extra\banner.mjs). BakeProdResolve (src/runtime/bake/production.rs) turns every specifier into a key with join_abs::<Posix>(dirname(referrer key), specifier); a drive path is not posix-absolute, so it is appended under the referrer's bundle directory: bake:/_bun/C:/app/extra/banner.mjs.
  • That key misses the module map, so bakeModuleLoaderFetch (src/runtime/bake/BakeGlobalObject.cpp) strips bake: and calls BakeToWindowsPath (production.rs), which used to_w_path_normalize_auto_extend and returned an extended-length path, \\?\_bun\C:\app\.... The regular loader cuts a specifier at its first ? as a query string (normalize_specifier_for_loader in src/runtime/jsc_hooks.rs), so the file it tried to read was \\. Because of this second defect, fixing the key alone would still have failed.
  • Both functions date from the commit that added this escape hatch (SSG stuff #20998) and were carried over unchanged by the Rust port; the hatch has never worked on Windows.

Fix

  • BakeProdResolve now resolves with Windows path semantics whenever the referrer's key or the specifier is a disk path (drive-qualified or UNC), using the same join_abs the rest of the runtime uses for path.resolve, and spells the result into the key namespace the way file: URLs do: C:\app\extra\banner.mjs becomes bake:/C:/app/extra/banner.mjs, \\server\share\x.mjs becomes bake://server/share/x.mjs. Bundled modules (bake:/_bun/...) take the unchanged posix join, and non-Windows builds are unchanged apart from a renamed local (the new code is #[cfg(windows)]).
  • BakeToWindowsPath now inverts that spelling (/C:/app/... -> C:\app\..., //server/share/... -> \\server\share\...) and returns a plain Win32 path, which is what the loader can read.
  • Why this shape is correct:
    • The module loader resolves the key returned for an import() a second time (referrer bake:/, specifier = the key's path), so the key spelling must be a fixed point of BakeProdResolve. /C:/... decodes to a disk path and resolves back to itself; //server/share/... is already a UNC path and does the same; bundle paths have no volume and keep taking the posix join. The debug log in the details below shows the keys that come out.
    • Static imports inside the file loaded from disk arrive with its bake:/C:/... key as the referrer. Decoding the referrer before taking dirname keeps them on disk as well (./detail.mjs, ../extra2/sibling.mjs in the test and log below); with the posix join they would only have worked by accident for drive paths and not at all for UNC.
    • Every key stays under bake:/, which is the prefix all three loader hooks dispatch on.
    • A specifier whose resolved form does not fit a path buffer cannot name a file; that throws, in the same way the existing non-relative import check in BakeProdResolve does, rather than indexing out of the buffer.
  • Verified with the new test in test/bake/dev/production.test.ts ("a route can import a file outside the bundle while rendering"). It builds a page that imports the same out-of-bundle file by a path.join specifier and by an unnormalized one (.. segment, mixed separators on Windows), where that file has a relative import of its own, and checks the rendered HTML shows both exports and that the two import()s returned one module instance.
    • Windows x64, current canary: fails with the EINVAL reading "\\" output above. Windows x64, debug build of this branch: passes, and the whole file passes (10 tests).
    • Linux (debug, ASAN): the whole file passes with --timeout 120000 (every full-build test in this file needs more than the 5s local default on a debug build here; CI passes its own timeout). The new test also passes on Linux without the fix: posix disk paths were never affected, so the failing-before evidence for this change is the Windows run, and the CI Windows lanes are what exercise it.
  • bake: check for exceptions in the production build's module helpers #38949 skips an equivalent test on Windows and points at this bug; once both land that skip can be removed. bake: release the strings production.rs returns to the module loader hooks #38960 changes how BakeGlobalObject.cpp consumes the strings these functions return; BakeToWindowsPath still returns an owned WTFStringImpl (clone_utf8 instead of clone_utf16), so that change stays correct on top of this one. Neither touches this hunk of production.rs.

Background

  • bun build --app bundles the app's server code into chunks, then prerenders the static routes in a dedicated JS global (Bake::GlobalObject, created by BakeCreateProdGlobal) whose module loader hooks serve the chunks from memory. Module keys are bake: plus a posix-style absolute path; a chunk written to dist/_bun/abc.js is the module bake:/_bun/abc.js, and imports between chunks are resolved by joining the relative specifier onto the referrer's key.
  • The escape hatch: when a key is not in the module map, bakeModuleLoaderFetch strips bake: and hands the remainder to the regular module loader, which reads it from disk. That is how a route can load a file at render time that the bundler did not see. On POSIX the remainder is already the disk path. BakeToWindowsPath exists to convert the remainder into a Windows path before the loader sees it.
  • join_abs in bun_paths is path.resolve parameterized by platform. The Windows variant understands drive letters and UNC volumes (\\server\share); the posix variant only treats a leading / as absolute, which is what the key namespace needs for bundle paths.
  • \\?\ is the Win32 extended-length path prefix; it is meant for direct file system calls, never for module specifiers, which the loader parses for ?query suffixes like ?raw.
Keys produced by the debug build on Windows (BUN_DEBUG_production=1), fixture with a drive path, an unnormalized spelling, a lowercase drive letter, and UNC spellings through the C$ admin share

pages/index.tsx imports extra/banner.mjs via join(import.meta.dir, "..", "extra", "banner.mjs"), via [import.meta.dir, "..", "extra", "banner.mjs"].join("/") and via the lowercased join; pages/unc.tsx imports it via \\localhost\C$\tmp\bakefix\extra\banner.mjs and via //localhost/C$/tmp/bakefix/extra/../extra/banner.mjs. banner.mjs itself imports ./detail.mjs and ../extra2/sibling.mjs.

[production] BakeProdLoad: bake:/C:/tmp/bakefix/extra/banner.mjs
[production] BakeProdLoad: bake://localhost/C$/tmp/bakefix/extra/banner.mjs
[production] BakeProdLoad: bake:/C:/tmp/bakefix/extra/detail.mjs
[production] BakeProdLoad: bake:/C:/tmp/bakefix/extra2/sibling.mjs
[production] BakeProdLoad: bake://localhost/C$/tmp/bakefix/extra/detail.mjs
[production] BakeProdLoad: bake://localhost/C$/tmp/bakefix/extra2/sibling.mjs

Rendered output (a === b and a === c are the unnormalized and lowercase spellings; unc === uncFwd the two UNC spellings):

dist/index.html:     <p>banner from disk | detail from disk | sibling via .. | true | true | C:\tmp\bakefix\pages</p>
dist/unc/index.html: <p>banner from disk | detail from disk | sibling via .. | true</p>

The UNC case is not in the committed test because it depends on the administrative share being reachable on the runner.

…endering

A production route that does import(join(import.meta.dir, "x.mjs")) at
render time hands BakeProdResolve a drive path. The posix join used for
bundle keys does not recognize it as absolute and appends it to the
referrer's bundle directory, and BakeToWindowsPath then returned a \\?\
prefixed path, which the module loader cuts at the ? as a query string,
so the build failed with EINVAL reading "\\".

Resolve against disk paths with Windows path semantics and key them the
way file: URLs spell them (bake:/C:/a/x.mjs, bake://server/share/x.mjs),
and have BakeToWindowsPath turn such a key back into a plain Win32 path
for the loader. Bundle keys keep the posix join.
@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 36 seconds

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4c7cbd4f-f04c-4d47-9304-1bee7cf1aeb6

📥 Commits

Reviewing files that changed from the base of the PR and between 88a6398 and d39457b.

📒 Files selected for processing (2)
  • src/runtime/bake/production.rs
  • test/bake/dev/production.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 10:05 AM PT - Aug 15th, 2026

@robobun, your commit d39457b has some failures in Build #98258 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 39092

That installs a local version of the PR into your bun-39092 executable, so you can run:

bun-39092 --bun

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Status

  • Reproduced on Windows Server 2019 (x64) with the current canary: the fixture in test/bake/dev/production.test.ts ("a route can import a file outside the bundle while rendering") fails with error: EINVAL reading "\\" and exit code 1; with this branch's debug build it passes, as does the rest of the file.
  • Linux never had the bug (posix disk paths already resolve), so the new test passes there with and without the change; the Windows CI lanes are the ones that exercise the fix.
  • Fix: bake: load files outside the bundle by their Windows path during prerendering #39092 (this PR), src/runtime/bake/production.rs only.

Comment thread test/bake/dev/production.test.ts
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs Outdated
Comment thread src/runtime/bake/production.rs
Comment thread src/runtime/bake/production.rs

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed after the follow-up commits and didn't find any bugs — the earlier stdout: "pipe" nit is addressed (now "ignore"), and the Rust changes look internally consistent. Because this introduces a new Windows key-spelling scheme (bake:/C:/..., bake://server/share/...) with a fixed-point invariant that only Windows CI exercises, a human look at the design would still be worthwhile.

What was reviewed:

  • resolve_disk_key fixed-point: /C:/... and //server/... decode → is_disk_path → re-encode to the same key; bundle keys (/_bun/...) fall through to the unchanged posix join.
  • BakeToWindowsPath inverse: /C:/aC:\\a, //server/s/x\\\\server\\s\\x; no more \\\\?\\ prefix that the loader would split on ?.
  • Overflow path uses join_abs_string_buf_checked and throws + returns BunString::dead(), matching the existing non-relative-import error pattern in BakeProdResolve.
  • Non-Windows: only a local rename (after_schemereferrer_key_path) and #[cfg(unix)]#[cfg(not(windows))] on the panic arm; behavior unchanged.
Extended reasoning...

Overview

The PR fixes bun build --app prerendering on Windows when a route dynamically imports a file the bundler never saw (via import(join(import.meta.dir, ...))). Two extern "C" functions in src/runtime/bake/production.rs are changed: BakeProdResolve gains a #[cfg(windows)] branch that recognizes drive-qualified and UNC paths and resolves them with join_abs::<platform::Windows>, encoding the result as bake:/C:/... or bake://server/share/...; BakeToWindowsPath is rewritten to invert that encoding into a plain Win32 path (previously it produced \\\\?\\..., which the module loader then truncated at the first ?). Three small #[cfg(windows)] helpers (key_path_to_disk_path, is_disk_path, resolve_disk_key) are added. A new test in test/bake/dev/production.test.ts builds a fixture that imports an out-of-bundle .mjs (which itself has a relative import) via two different specifier spellings and asserts the rendered HTML.

Security risks

None identified. The input is a module specifier during a local build's prerender step; the resolved path is handed to the regular module loader, which is the same loader that already reads arbitrary user paths during bun run. The overflow case (join_abs_string_buf_checked returning None) throws instead of indexing past the buffer.

Level of scrutiny

Moderate-to-high. The change is small and entirely #[cfg(windows)]-gated (non-Windows behavior is byte-identical apart from a renamed local), but it encodes a non-obvious invariant: the key spelling must be a fixed point of BakeProdResolve because JSC's module loader re-resolves the returned key with referrer bake:/. The PR description walks through why /C:/... and //server/... satisfy this, and I traced it (key_path_to_disk_path strips the leading / before a drive letter; UNC //server/share already has a Windows volume so is_disk_path is true; both re-encode to themselves). This is the kind of platform-specific path logic where a maintainer familiar with the bake loader hooks should confirm the encoding choice is the one they want long-term.

Other factors

  • My prior inline nit (undrained stdout: "pipe") was addressed in b6f36de — the test now uses stdout: "ignore". That thread is resolved.
  • The comment-cop bot fired again on the two remaining doc comments (lines ~1322 and ~1357) after the last trim commit. Those comments are 2 and 4 lines respectively and document the encoding invariant rather than justify a workaround, so I don't consider them blocking, but the author may want to trim once more or accept the bot noise.
  • The new test passes on Linux with or without the fix (posix disk paths were never affected), so the failing-before evidence is Windows-only. The PR description reports it was verified on Windows Server 2019 against both the current canary (fails) and this branch's debug build (passes).
  • No CODEOWNERS entry covers src/runtime/bake/.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant