bundler: bundle template-literal require()/import() via a __glob lookup map - #35680
bundler: bundle template-literal require()/import() via a __glob lookup map#35680robobun wants to merge 9 commits into
Conversation
…up map
When the argument to require() or import() is a relative-path template
literal, walk the filesystem for files matching the static shape, add an
import record for each match, and rewrite the call to
__glob({ "./bin/a/x.node": () => require("./bin/a/x.node"), ... })(arg)
so the dispatch happens at runtime against bundled modules instead of
falling through to import.meta.require. This is the same approach esbuild
takes for glob-style imports.
The shape extractor is extended to look through string concatenation and
through a const binding whose initializer is itself a template literal or
concat chain, which is how native-addon loaders such as tigerbeetle-node
pick a platform-specific .node file:
const filename = `./bin/${arch}-${platform}${abi}/client.node`;
return require(filename);
With --compile, each matched .node file goes through the existing Napi
loader path and is embedded in the binary, so the compiled executable no
longer fails with "Cannot find module './bin/.../client.node' from
'/$bunfs/root/...'".
Zero matches fall through to the existing runtime-require path (same as
before), let/var bindings are not looked through (reassignment would
invalidate the shape), bare specifiers and shapes containing glob
metacharacters are skipped, and a 256-match cap keeps a pathological
pattern from exploding the bundle.
Fixes #9951
|
Updated 11:05 PM PT - Jul 25th, 2026
❌ @robobun, your commit 9018c2b has 1 failures in
🧪 To try this PR locally: bunx bun-pr 35680That installs a local version of the PR into your bun-35680 --bun |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Warning Review limit reached
Next review available in: 5 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (14)
Comment |
|
Found 7 issues this PR may fix:
🤖 Generated with Claude Code |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…OT_FOUND code, symlinks, snapshots - Flatten rope EStrings before reading in shape extraction (adjacent literals folded under minify-syntax would otherwise lose segments). - Bound append_dynamic_specifier_shape recursion at depth 32 so a self-referential const cannot stack-overflow the bundler. - Skip the glob path for import() when a second argument is present so import attributes / loader overrides are preserved, and compute HANDLES_IMPORT_ERRORS per call-site kind. - Set .code = 'MODULE_NOT_FOUND' on the __glob miss error so catch-by-code loaders keep their fallback behavior. - Only rewrite backslash to slash on Windows; on POSIX it is a legal filename byte. - Walker: follow_symlinks=false (matches esbuild, avoids pnpm cycles). - Guard against empty source_dir. - Trim doc comments. - Update bundler output-hash / debugId snapshots shifted by the new __glob helper in runtime.js. - Add tests for rope concat, self-referential const, import() with options, and an end-to-end --compile case with .node assets.
append_estring_rope now treats a NUL byte in the literal content as opaque so it cannot collide with the placeholder marker (filesystem paths cannot contain NUL anyway). Also update the SelfReferentialConst test's comment to match the remove-before-recurse behavior and pin the emitted map with an onAfterBundle check.
3628109 to
c63a10c
Compare
glob_shape_is_eligible now also rejects a literal backslash in the static content (bun_glob treats it as an escape), and append_estring_rope truncates the buffer back to its entry length when a later rope segment contains NUL so the caller's nothing-appended contract holds.
1dddbf8 to
3faf857
Compare
…er_shape Snapshot buf.len() at the top of append_dynamic_specifier_shape and truncate back to it whenever the match evaluates to false, so every arm honours the contract without per-arm discipline. This closes the ETemplate case where a NUL in a part's tail could leave the head and earlier placeholders in buf.
There was a problem hiding this comment.
No new findings on 9018c2b — the uniform buf.truncate(start) at the tail of append_dynamic_specifier_shape now enforces the nothing-appended-on-false contract for every arm (the EIdentifier early-return is fine since the recursive call's entry length equals this frame's). Deferring to a maintainer: this is a ~330-line new bundler feature with user-visible semantics (filesystem walking during parse, a new __glob runtime helper, the const-indirection extension beyond esbuild, the 256-match cap) that warrants human sign-off on the design.
What was reviewed
- Shape extraction in
p.rs: NUL-in-literal rejection, backslash metachar, rope segments, depth cap, remove-before-recurse cycle guard, and the truncate-on-false invariant acrossEString/ETemplate/EBinary/EIdentifier. glob_shape_is_eligible/glob_shape_to_patternagainstbun_glob's actual metachar set and the**/*-vs-*boundary rule.parser_glob_resolvermid-walk error handling (returns empty rather than partial) and Windows separator normalization.Imports::ALL_SORTED/ALL_SORTED_INDEXre-derivation for__glob— matches the unit test's expectations.
Extended reasoning...
Overview
This PR adds esbuild-style glob resolution for template-literal / string-concat require() and import() arguments to bun build. It touches the JS parser (src/js_parser/p.rs, ~330 new lines implementing shape extraction, eligibility, pattern conversion, and the __glob({...})(arg) rewrite), the bundler (ParseTask.rs wires parser_glob_resolver, options.rs implements the filesystem walker), the runtime-imports table (ast/runtime.rs), runtime.js (new __glob helper), and adds a 341-line test file plus snapshot/hash updates in three existing tests.
Security risks
Low but non-zero. The new code walks the filesystem during parse using a pattern derived from user source code, but the pattern is constrained to ./ or ../ prefixes relative to the source file's own directory, glob metacharacters (including \\) in the literal parts abort the rewrite, and a 256-match cap bounds the walk. The __glob runtime helper does a plain own-property lookup on a bundler-generated object literal, so prototype pollution isn't a concern there. No auth/crypto/permissions code is touched.
Level of scrutiny
High. This is production-critical bundler + parser code introducing new user-visible semantics: previously-unbundled dynamic requires now pull additional files into the graph, which changes what bun build --compile embeds. It also extends esbuild's behavior (looking through const bindings), which is an API design decision. Filesystem I/O now happens inside the parse worker via an injected function pointer — architecturally reasonable given the crate-tier constraint, but a maintainer should confirm this is the intended layering.
Other factors
Five prior automated review rounds surfaced edge cases (literal NUL confused with the placeholder marker, \\ missing from the metachar reject list, partial-append-on-Ok(false) in the rope helper and the ETemplate arm, a stale test comment); all were addressed in c63a10c / 3faf857 / 9018c2b and are marked resolved. Test coverage is thorough (15 itBundled cases including --compile end-to-end, negative cases for let/bare specifiers/import-attributes/literal-NUL, and the tigerbeetle loader pattern). The snapshot-hash churn in bun-build-api.test.ts.snap, html-import-manifest.test.ts, and the two debugId updates are the expected consequence of runtime.js gaining __glob. Given the scope and the design choices involved (match cap, const-indirection, error-message wording, whether zero-matches should warn), this needs a maintainer's eyes rather than bot approval.
1 participant
What
bun buildnow bundlesrequire()/import()calls whose argument is a relative-path template literal (or string concatenation, or aconstbinding initialized with one of those). Every file matching the static shape is added as an import record and the call is rewritten to a runtime lookup:This is the same approach esbuild takes for glob-style imports, with one extension: shape extraction also looks through a
constbinding whose initializer is a template or concat chain, so the common native-addon loader patternis handled as well.
Why
Packages that pick a platform-specific
.nodefile at runtime (tigerbeetle-node, duckdb, sharp, libsql, tailwindcss-oxide, lzma-native, ...) fail afterbun build --compilebecause the bundler can't statically resolve the dynamic specifier and the.nodefile is never embedded:With this change, each matched
.nodefile goes through the existingLoader::Napipath and is embedded in the compiled binary, and the generated__globmap dispatches to the right one at runtime.Semantics
./or../prefix) are eligible; bare package specifiers are left to runtimerequire.let/varbindings are not looked through (reassignment would invalidate the shape); onlyconst./becomes**/*; elsewhere it becomes*(same as esbuild), so./mods/${x}.jsstays bounded to one directory.Verification
test/bundler/bundler_glob_require.test.tscovers: direct template, concat,constindirection,let(negative), zero matches,import(), subdirectory matches, the full tigerbeetle-style loader pattern, and bare specifiers (negative).Fixes #9951
[review] gate passed · iteration 2 · 15 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 3 passed · 0 rejected · iteration 2
evidence per changed file