Skip to content

install: isolated linker honors active bun link - #30289

Open
robobun wants to merge 1 commit into
mainfrom
farm/779760c6/isolated-linker-bun-link
Open

install: isolated linker honors active bun link#30289
robobun wants to merge 1 commit into
mainfrom
farm/779760c6/isolated-linker-bun-link

Conversation

@robobun

@robobun robobun commented May 5, 2026

Copy link
Copy Markdown
Collaborator

Closes #30287. Supersedes #29615 (same approach, originally in Zig; re-ported to the Rust installer after #30412 landed mid-PR).

Problem

Under the isolated linker, bun link in a producer and bun link <pkg> in a consumer had no effect when the consumer's dep resolves via npm or a catalog. The consumer's node_modules/.bun/<pkg>@<ver>/node_modules/<pkg> was materialized from the registry tarball cache; the producer's working tree was never read, so edits in the producer never reached the consumer. The hoisted linker honored the link. Silent footgun.

Repro

$ mkdir -p /tmp/prod && cd /tmp/prod
$ echo '{"name":"lodash","version":"4.17.21","main":"index.js"}' > package.json
$ echo 'module.exports = { from: "PRODUCER" };' > index.js
$ bun link
Success! Registered "lodash"

$ mkdir -p /tmp/cons && cd /tmp/cons
$ printf '[install]\nlinker = "isolated"\n' > bunfig.toml
$ echo '{"name":"c","dependencies":{"lodash":"^4.17.0"}}' > package.json
$ bun link lodash
$ bun install
$ head -1 node_modules/lodash/index.js
# registry tarball content — not "{ from: \"PRODUCER\" }"

Fix (Rust installer: isolated_install.rs, Installer.rs, PackageManagerDirectories.rs)

One readdir of the global link dir at install start (populate_linked_names_cache, main thread, before workers) seeds a name set on the PackageManager. The probe is read-only: the link dir path is computed without touching the filesystem (global_dir_path, same precedence open_global_dir uses) and a missing dir just leaves the cache empty, so installs on machines that never ran bun link create nothing under the global dir. The link dir is shared with bun add -g: hoisted global installs land as real directories (skipped by the symlink filter), and isolated-linker global installs land as symlinks into the global dir's own .bun store, so candidates are discriminated by their immediate readlink target: bun link writes a single absolute symlink to the producer tree, while isolated top-level entries are relative .bun/... symlinks (POSIX) or junctions into the link dir (Windows), in every globalStore mode (with globalStore on, the chain resolves into <cache>/links/, so a resolved-path check alone is insufficient). Without that rejection a bun add -g <pkg> --linker=isolated would silently substitute the global version into every consumer project listing <pkg> as a direct dep. The override is also gated on !options.global: a global install is the link dir itself, so bun add -g of a link-registered name installs registry bytes (hoisted parity); every subsequent linked_package_path() is a hashmap check with no syscalls on POSIX (zero cost when no packages are linked — the CI / most-dev-machines case). Windows keeps a linked_names_any_on_windows fast-path flag and re-probes per hit via GetFileAttributesW + a dangling-junction check.

The override is scoped by resolution, not name: a linked_pkg_ids bitset (built once on the main thread, read-only for workers) marks the package ids that root/workspace direct dependencies resolve to and whose name is link-registered. If the lockfile also contains the linked name at another version transitively (direct no-deps@1.0.0 plus no-deps@1.0.1 via one-dep), that copy keeps its registry bytes — matching the hoisted linker, which only replaces the top-level node_modules/<name>. (Version-matching the producer's package.json against the locked resolution was rejected: a WIP producer's version rarely equals the locked one, so it would silently disable the override — the original bug's failure mode.) For a marked entry, when the user didn't pick --backend=symlink:

  • Skip the cache-fetch dance. has_active_link is OR'd into needs_install and short-circuits to start_task before the cache-subpath/download enqueue, so the registry is never consulted for linked entries (the canonical workflow: the producer isn't published yet).
  • Detach stale GVS symlinks, then wipe and rebuild. The worker's override block lstat/unlinks a leftover node_modules/.bun/<storepath> symlink into <cache>/links/ (propagating non-ENOENT failures so a blocked unlink can't route writes into the shared cache), delete_trees the project-local entry so producer-deleted files don't persist, and FileCopiers the producer tree with default excludes (.git/.hg/.svn/CVS/node_modules dirs; .DS_Store, lockfiles, .npmrc, bunfig.toml, .env.production, etc.).
  • Link-overridden entries are GVS-ineligible. The eligibility DFS forces entry_hash = 0 for linked_pkg_ids members (the bitset is empty under --backend=symlink, matching the override guards), so mutable producer content can never land in, or be deleted from, the shared content-addressed <cache>/links/ store.
  • Copyfile only. A lifecycle script rewriting files inside .bun/<storepath>/... would propagate back into the producer through a shared hardlink inode.
  • bun unlink recovery. bun unlink has no uninstall step, so the override drops a .bun-link marker inside the materialized body (same convention as the .bun-tag files). needs_install treats marker-present-without-active-link as a rebuild, covering entries that stay project-local after unlink (trusted dependency, or globalStore off, which is the default); entries that regain global-store eligibility rebuild via the global existence miss anyway. Mirrors the PatchInfo::Remove un-patch recovery.

Thread-safety: workers use the read-only linked_package_path (&PackageManager; cache + global_link_dir_path are immutable post-populate per the Task::run SAFETY contract); the main thread uses linked_package_path_mut (lazy global-dir init) via installer.manager_mut() per the loop's BACKREF convention.

Verification

test/cli/install/isolated-install.test.ts, describe("bun link integration") — all fail without the fix:

  • npm-resolved dep honors active bun link (marker file proves producer body)
  • catalog-resolved dep honors active bun link
  • producer rebuild propagates on reinstall
  • bun unlink then reinstall restores the registry body (and removes the marker)
  • no link registered → registry tarball (negative control)
  • --backend=symlink bypasses the override
  • dangling link (producer deleted without bun unlink) falls back to the registry
  • multi-version: only the direct-dep resolution gets the producer body; the transitive no-deps@1.0.1 copy (via one-dep) keeps its registry bytes

Three pack-parity tests (package.json#files whitelist === bun pm pack --dry-run) are test.todo, see follow-ups.

Scope / follow-ups

  1. Pack-parity materialization: the Zig draft routed file selection through collectPublishablePaths (shared with bun pm pack); the Rust port ships default-excludes only. Porting that helper (plus bundledDependencies / optional-bin semantics) unlocks the three test.todos.
  2. Windows @scope/name flattening in populate_linked_names_cache (currently served by the per-call attribute probe; Name::slice_u8() already exposes the UTF-8 transcode, so the WTF-16 limitation noted in earlier reviews no longer applies).

Credit to @Kniggishood for the original approach in #29615.

Rebase note

Rebased onto current main (single commit; net diff is the 5 src/install/*.rs files plus the test file). Latest rebase resolved one trivial conflict: #39157's Timings enum binding landed where the linked_pkg_ids bitset build sits in isolated_install.rs; kept both, bitset directly before the build_store() call. The prior rebase resolved #38333's extraction of the inline store build into build_store(); every other hunk (eligibility carve-out, installer field, needs_install terms, fetch skip) merged cleanly into the refactored layout.

Earlier rebase resolutions
  • Content conflicts: the needs_install early-start comment in isolated_install.rs (kept the has_active_link skip-the-fetch block, re-applied main's adjacent comment); the PackageManager struct field list (kept the linked_names fields alongside main's now-pub(crate) on_wake, dropping the removed ci_mode); the isolated-install.test.ts import block (unioned both sides) and the test body (kept main's new hoist describe next to the bun link integration block).
  • API drift from main: Fd::from_std_dir was removed (now Dir::fd()); DirEntry::name::as_zstr() became private (added a small name_zstr helper that copies the dirent name into a NUL-terminated buffer); FileCopier::init_with_skip tripped the newly-enabled -D unreachable-pub (now pub(crate)).

Full suite green locally (78 pass, 3 todo, 0 fail).


no test proof · iteration 63 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/isolated-install.test.ts

@github-actions github-actions Bot added the claude label May 5, 2026
@robobun

robobun commented May 5, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 3:57 PM PT - Aug 15th, 2026

@robobun, your commit 4101de8 is building: #98790

@github-actions

github-actions Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. install: isolated linker honors active bun link #29615 - Same feature (isolated linker honors active bun link) by the original author; this PR explicitly supersedes it with a rebase and review feedback fixes

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds publishable-paths computation and a linked-package-name cache, uses them so isolated installs honor active bun link by materializing only publishable files from linked producers; updates copier/hardlink init signatures and tests to validate parity with bun pm pack --dry-run.

Changes

bun link + publishable-paths for isolated installs

Layer / File(s) Summary
Data Shape / Types
src/install/PackageManager.zig, src/install/PackageManager/WorkspacePackageJSONCache.zig, src/cli/pack_command.zig
Add linked_names: bun.StringHashMapUnmanaged(void) and linked_names_populated: bool to PackageManager; add lock: bun.Mutex to workspace JSON cache; add exported PublishablePaths type.
Pack / Selection Implementation
src/cli/pack_command.zig
Add collectPublishablePaths(parent_allocator, root_dir, json_root) returning an arena-owned, sentinel-terminated POSIX-relative slice computed from bin entries and package.json#files rules (globs, negations, defaults) and always includes package.json.
Link Cache & Resolution
src/install/PackageManager/PackageManagerDirectories.zig, src/install/PackageManager.zig
Add populateLinkedNamesCache(this) to scan global link directory and populate linked_names (flatten @scope/name on non-Windows). Make linkedPackagePath cache-first when populated (short-circuit null when missing), otherwise fall back to platform lstat/getFileAttributes checks; export linkedPackagePath and populateLinkedNamesCache.
Installer Integration / Flow
src/install/isolated_install.zig, src/install/isolated_install/Installer.zig
Call manager.populateLinkedNamesCache() on main thread before workers. Detect has_active_link via linkedPackagePath() (except .symlink backend), include it in needs_install, and when true start install immediately using the linked producer as source. Compute publishable paths and materialize only that subset. Adjust staging/final-dir deletion logic for global vs non-global linked entries.
Materializer helpers
src/install/isolated_install/Installer.zig
Add linkedHardlinkPaths(...) that hardlinks publishable POSIX subpaths using linkatZ (create parents, skip ENOENT, handle EEXIST by deleting and retrying, allow EXDEV to bubble) and linkedCopyPaths(...) for copyfile fallback (create parents, open/truncate, preserve mode best-effort).
Materializer API (wiring)
src/install/isolated_install/FileCopier.zig, src/install/isolated_install/Hardlinker.zig, src/install/PackageManager/patchPackage.zig
Extend FileCopier.init(...) and Hardlinker.init(...) to accept skip_filenames: []const bun.OSPathSlice and forward to Walker.walk; update patchPackage.zig call site to pass the new arg. In folder/root flows, pass empty exclude lists (&.{}) instead of excluding node_modules.
Thread-safety note
src/install/PackageManager/WorkspacePackageJSONCache.zig
Add lock: bun.Mutex and documentation that callers must hold the mutex when using returned *MapEntry pointers across threads (no internal locking added in shown functions).
Tests / Capabilities
test/cli/install/isolated-install.test.ts, test/internal/ban-limits.json
Add bun link integration test suite (hermetic env helpers, recursive listing) verifying isolated linker honors bun link, producer edits propagate, --backend=symlink bypass, installed entry file set matches bun pm pack --dry-run, enforces package.json#files, plus nested-directory exclusion regression. Increment ban-limits counters.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Code changes fully implement the objective from issue #30287: seed a linked-names cache at install start, skip download/extract for linked packages, materialize from producer via collectPublishablePaths respecting package.json#files, and add per-backend copy/hardlink logic.
Out of Scope Changes check ✅ Passed All code changes are in-scope: cache implementation, linked-package detection, materialization logic, test coverage for the feature, and necessary helper additions like skip_filenames parameter in FileCopier/Hardlinker.
Title check ✅ Passed The title clearly and concisely describes the primary change: making the isolated linker honor active bun links.
Description check ✅ Passed The description explains the problem, implementation, verification, test coverage, and follow-ups, despite using different headings from the template.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/cli/pack_command.zig`:
- Around line 958-960: The pack step is losing the optional flag for binary
entries: when adding bins to pack_queue you mark them as optional
(pack_queue.add with .{ .path = bin.path, .optional = true }) but
PublishablePaths.paths (and later pack()) returns only plain paths so missing
optional bins are treated as required and cause linker/copy failures; either
filter out non-existent optional bin entries at the point you add them (e.g.,
check file existence before calling pack_queue.add) or change the return type
from PublishablePaths.paths to include the optional metadata and propagate that
richer type through pack() and any consumers so optional bins are skipped if
missing. Update the code paths that consume pack_queue entries and
PublishablePaths.paths (including functions pack() and any code in the 1010-1015
region) to respect the optional flag.
- Around line 968-999: The code currently treats a present-but-not-array "files"
field as if it were absent and falls back to iterateProjectTree, causing
inconsistency with pack(); instead, detect the malformed manifest and abort the
command: in the branch handling json_root.get("files") where files.asArray() is
false, replace the fallback logic with a manifest error return (or call the
existing manifest validation error/reporting helper) so the function (the
pack_command path that calls iterateIncludedProjectTree / iterateProjectTree and
ultimately pack()) fails fast on non-array "files" entries rather than widening
the published file set.
- Around line 945-1021: collectPublishablePaths currently never walks vendored
bundled dependencies, so packages relying on bundledDependencies are incomplete;
modify collectPublishablePaths to mirror pack() by detecting the package
manifest's bundledDependencies (e.g. json_root.get("bundledDependencies") or the
equivalent field you expect) and invoke the existing iterateBundledDeps function
to add those files into pack_queue (similar to how bins and files are
processed). Locate collectPublishablePaths and after processing bins/files
(before final iterateProjectTree fallback), call iterateBundledDeps(allocator,
&pack_queue, root_dir, json_root, .silent) or the correct iterateBundledDeps
signature used elsewhere, ensuring bundled deps are queued into pack_queue so
they appear in the returned paths array. Ensure errors are propagated
consistently (use try) and that iterateBundledDeps runs with the same
allocator/arena so added paths live in the returned arena.

In `@src/install/isolated_install.zig`:
- Around line 1589-1598: The code currently computes has_active_link but only
forces a rebuild and doesn't prevent a linked producer from being considered for
the global virtual store; modify the eligibility logic so that when
has_active_link is true the package is treated as ineligible for the global
store before the entry_hash/eligibility pass runs (i.e., ensure
entryUsesGlobalStore or the variable checked by the entry_hash pass is set to
false for that pkg_name). Update the places where has_active_link is computed
(the block around PackageInstall.supported_method / manager.linkedPackagePath)
and the other noted blocks (the analogous checks at the later occurrences) so
that linked producers are excluded from global-store eligibility rather than
merely triggering rebuilds.

In `@src/install/PackageManager/PackageManagerDirectories.zig`:
- Around line 518-540: The no-cache fallback calls globalLinkDirPath() which can
abort via Global.exit(1); change the fallback to use the same
non-fatal/open-and-check logic as populateLinkedNamesCache() so missing or
unreadable global link state returns null instead of exiting. Specifically, in
the branch after use_cache (the second call sites of globalLinkDirPath() and
subsequent bun.path.joinAbsStringBufZ/joined handling), replace the direct
globalLinkDirPath() usage with a safe lookup that attempts to open/read the
global link dir (reusing the open/path error handling from
populateLinkedNamesCache()), and if that fails return null; otherwise proceed to
join the path and perform the same Windows
(getFileAttributes/is_directory/is_reparse_point) and non-Windows
lstat/ISDIR/ISLNK checks as before.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3dabccb5-e09c-4647-8054-6e19c551cf87

📥 Commits

Reviewing files that changed from the base of the PR and between b009453 and 088bcaa.

📒 Files selected for processing (10)
  • src/cli/pack_command.zig
  • src/install/PackageManager.zig
  • src/install/PackageManager/PackageManagerDirectories.zig
  • src/install/PackageManager/WorkspacePackageJSONCache.zig
  • src/install/PackageManager/patchPackage.zig
  • src/install/isolated_install.zig
  • src/install/isolated_install/FileCopier.zig
  • src/install/isolated_install/Hardlinker.zig
  • src/install/isolated_install/Installer.zig
  • test/cli/install/isolated-install.test.ts

Comment thread src/runtime/cli/pack_command.zig Outdated
Comment thread src/runtime/cli/pack_command.zig Outdated
Comment thread src/runtime/cli/pack_command.zig Outdated
Comment thread src/install/isolated_install.zig Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.zig Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.zig Outdated
Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread test/cli/install/isolated-install.test.ts Outdated
Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.zig Outdated
Comment thread test/cli/install/isolated-install.test.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/install/isolated_install/Installer.zig`:
- Around line 763-775: The EXDEV fallback currently jumps from
linkedHardlinkPaths() to linkedCopyPaths() (in the linked_initial_method
.hardlink branch) without removing any files already created by the partial
hardlink pass, which risks truncating producer inodes; update the .err handling
for linkedHardlinkPaths (and the analogous spots) so that on err.getErrno() ==
.XDEV you first delete/cleanup any partially created destination entries (the
staging files/links pointed to by dest or whatever structure linkedHardlinkPaths
populated) before continuing to :backend .copyfile; ensure the cleanup logic is
invoked in the same branches referenced (linked_initial_method .hardlink, the
EXDEV checks around linkedHardlinkPaths) so linkedCopyPaths()/FileCopier.copy()
runs against a clean destination.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0c866757-c107-4f52-9b41-c37234d1fc08

📥 Commits

Reviewing files that changed from the base of the PR and between 2f448c6 and 93bd311.

📒 Files selected for processing (3)
  • src/install/PackageManager/PackageManagerDirectories.zig
  • src/install/isolated_install/Installer.zig
  • test/internal/ban-limits.json

Comment thread src/install/isolated_install/Installer.zig Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/install/isolated_install/Installer.zig`:
- Around line 795-809: The fallback is constructing src by starting from
initTopLevelDirLongPath() then appendJoin(producer_path), which yields an
incorrect path when producer_path is already absolute; change the src
construction to use the absolute producer path directly (i.e., initialize src
from producer_path or from linkedPackagePath() result) so Hardlinker is given
the real linked producer tree; update the code around the src variable creation
(used when calling Hardlinker.init with folder_dir, src, dest,
linked_skip_files, linked_skip_dirs) to avoid prepending the project root.

In `@test/cli/install/isolated-install.test.ts`:
- Around line 2187-2219: The test currently only asserts that the isolated
installer used the registry body (bodyDir) but doesn't verify the consumer's
node_modules points at the linked producer; add an assertion that
node_modules/no-deps exposes the producer tree by checking the presence of the
producer marker (use packageDir and join to build join(packageDir,
"node_modules", "no-deps", "marker.js") and assert existsSync(...) is true or
assert the top-level node_modules/no-deps is a symlink to the producer); update
the test after the existing bodyDir checks (symbols: packageDir, producer,
bodyDir, join, existsSync) so the test fails if --backend=symlink stopped
pointing consumers at the linked producer.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 20b800db-75ce-4e1a-8044-9b3cc2f6ba8e

📥 Commits

Reviewing files that changed from the base of the PR and between 93bd311 and e8ea27e.

📒 Files selected for processing (3)
  • src/install/PackageManager/PackageManagerDirectories.zig
  • src/install/isolated_install/Installer.zig
  • test/cli/install/isolated-install.test.ts

Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread test/cli/install/isolated-install.test.ts Outdated
Comment thread src/runtime/cli/pack_command.zig Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.zig Outdated
Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread test/cli/install/isolated-install.test.ts
Comment thread test/internal/ban-limits.json Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.zig Outdated
Comment thread src/install/isolated_install.zig Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.zig Outdated
Comment thread src/install/isolated_install/Installer.zig Outdated
@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from 8d8aa19 to 19dc6ef Compare May 15, 2026 05:18
@robobun

robobun commented May 15, 2026

Copy link
Copy Markdown
Collaborator Author

The Rust rewrite (#30412) landed while this PR was open. All the install-side changes here are in .zig files that are no longer compiled into the binary — isolated_install.rs, isolated_install/Installer.rs, and PackageManager/PackageManagerDirectories.rs are what run now, and they do not contain the link-override logic. The test failures on this CI run are the tests verifying the fix; they fail because the Rust installer ignores bun link registrations.

The logic to port (from the .zig counterparts to .rs):

  1. populate_linked_names_cache — readdir global link dir, @scope/name flatten, DT_UNKNOWN lstat fallback, target-resolution
  2. linked_package_path — cache-first; Windows GetFileAttributesW fallback with dangling-link guard; linked_names_any_on_windows fast path for zero-link machines
  3. linked_names + linked_names_populated + linked_names_any_on_windows fields on PackageManager
  4. Call populate_linked_names_cache once on the main thread at the top of install_isolated_packages
  5. has_active_link check and link-override block in the per-entry installer loop (materialize from producer via collect_publishable_paths, EXDEV→copyfile, stale-GVS-symlink detachment, per-task log)
  6. Eligibility-DFS carve-out: linked packages project-local (entry_hash = 0), gated on supported_method != .symlink so GVS hit rate stays intact under --backend=symlink

The test suite under the bun link integration describe block in test/cli/install/isolated-install.test.ts is ready — it exercises all six scenarios and fails cleanly on main today.

I do not have the iteration budget for this port in this session; flagging for a maintainer familiar with both sides of the rewrite.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/install/isolated_install.rs:2277-2282 — The Rust port omits the if has_active_link { installer.start_task(entry_id); continue; } short-circuit that the Zig version adds at isolated_install.zig:1709-1716. has_active_link is OR'd into needs_install here, but after the !needs_install block closes at line 2347, control falls straight through to cache_subpath_z resolution and enqueue_package_for_download — so a linked package still hits the registry even though the worker (Installer.rs:1068+) will source the body from the producer dir and never read the downloaded bytes. This is the PR description's first design bullet ("Skip the cache-fetch dance") not ported to Rust; add the same short-circuit immediately after line 2347.

    Extended reasoning...

    What the bug is

    The Zig side of this PR adds, immediately after the if (!needs_install) { … continue; } block in isolated_install.zig:

    // `link_package` will source from the producer dir via
    // `linkedPackagePath`; skip the cache-fetch dance entirely
    // (mirrors how `.folder` is handled — no registry traffic
    // needed when the body comes from an on-disk producer).
    if (has_active_link) {
        installer.startTask(entry_id);
        continue;
    }

    This is the PR description's first fix bullet: "Skip the cache-fetch dance. The body comes from the on-disk producer, not the registry, so the download/extract task for that entry never runs."

    The Rust port (isolated_install.rs) only does half of this: it computes has_active_link (lines 2229-2241) and ORs it into needs_install (line 2280) so that an existing store entry doesn't short-circuit the override. But it does not add the corresponding short-circuit between the !needs_install block and the cache-subpath / download-enqueue logic.

    Code path

    At isolated_install.rs:2320-2452:

    1. Line 2280: needs_install = … || has_active_link || …true for a linked package.
    2. Lines 2320-2347: if !needs_install { … continue; } — skipped because needs_install is true.
    3. Line 2351 (immediately after, no intervening check): let cache_subpath_z = match pkg_res_tag { ResolutionTag::Npm => cached_npm_package_folder_name(…), … }.
    4. Line 2389: missing_from_cache is computed by checking whether <cache>/<pkg>@<ver>/package.json exists.
    5. If the tarball isn't already cached (cold install / fresh machine / private package not yet published), line 2452 calls manager.enqueue_package_for_download(…) with TaskCallbackContext::IsolatedPackageInstallContext(entry_id).

    The download/extract runs to completion, and only then does the callback fire installer.start_task(entry_id). The worker's Task.run (Installer.rs:1068-1170) then re-checks linked_package_path, finds the producer, and copies from it — never reading the downloaded/extracted bytes.

    Why existing code doesn't prevent it

    There is simply no branch between lines 2347 and 2351 that consults has_active_link. The variable is computed and used once (in the needs_install chain) but the purpose of that boolean — to make the entry skip both the warm-hit fast path and the registry fetch — is only half-implemented. The Zig reference and the PR description both document the missing half explicitly. The robobun comment on this PR even calls out item 5 as needing porting ("has_active_link check and link-override block in the per-entry installer loop"); the override block landed in Installer.rs but the main-thread scheduling half didn't fully land in isolated_install.rs.

    Step-by-step proof

    On a machine with a fresh tarball cache (or BUN_INSTALL_CACHE_DIR pointed at an empty dir, as the PR's own hermetic tests do):

    1. Producer at /tmp/prod with {"name":"lodash","version":"4.17.21"}; user runs bun link.
    2. Consumer (isolated linker) depends on lodash@^4.17.0. User runs bun install.
    3. populate_linked_names_cache() puts "lodash" in linked_names.
    4. Main loop reaches lodash@4.17.21: has_active_link = true (line 2238), needs_install = … || true = true.
    5. !needs_install is false → skip lines 2320-2347.
    6. Line 2351: cache_subpath_z = "lodash@4.17.21". Line 2389: <cache>/lodash@4.17.21/package.json doesn't exist → missing_from_cache = true.
    7. Line 2452: enqueue_package_for_download("lodash", …, 4.17.21, registry_url, IsolatedPackageInstallContext(entry_id), …).
    8. Bun fetches lodash-4.17.21.tgz from the registry, extracts it into <cache>/lodash@4.17.21/, and only then fires the callback → start_task(entry_id).
    9. Worker enters the override block in Installer.rs, opens /tmp/prod via the global-link symlink, and FileCopiers the producer tree into the store entry. The freshly-extracted <cache>/lodash@4.17.21/ is never opened.

    In Zig, step 6 never happens: if (has_active_link) { installer.startTask(entry_id); continue; } jumps straight to step 9.

    Impact

    • Wasted network/disk/latency on every cold install of a linked package — exactly what the PR's first design bullet says it eliminates. The new bun link integration tests in this PR all use hermeticEnv with a fresh BUN_INSTALL_CACHE_DIR, so every test run downloads no-deps@1.0.0 from Verdaccio unnecessarily.
    • Correctness in offline / unpublished scenarios: if the registry is unreachable (offline dev, air-gapped CI) or the linked package hasn't been published yet (the canonical bun link use case — developing a package before its first publish), enqueue_package_for_download fails and on_package_download_error fires instead of start_task. The linked package never installs even though its producer is sitting on disk. The Zig version handles this correctly; the Rust version regresses it.

    Fix

    Insert immediately after line 2347 (mirroring isolated_install.zig:1709-1716):

    // `link_package` will source from the producer dir via
    // `linked_package_path`; skip the cache-fetch dance entirely
    // (mirrors how `.folder` is handled — no registry traffic
    // needed when the body comes from an on-disk producer).
    if has_active_link {
        installer.start_task(entry_id);
        continue;
    }

Comment thread src/install/isolated_install/Installer.rs Outdated
Comment thread src/install/isolated_install/Installer.rs

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/install/isolated_install.rs:2282 — The Rust port is missing the if has_active_link { installer.start_task(entry_id); continue; } early dispatch that the Zig reference inserts at isolated_install.zig:1713–1716 between the !needs_install block and the cache-subpath/download logic. As a result, when has_active_link is true and the tarball isn't cached, control falls through to enqueue_package_for_download; if that download fails (offline / 404 / private-registry auth), the entry is marked Done with TaskError::Download and the worker override never runs — the on-disk producer is never consulted. Fix: insert the dispatch between line 2349 and the cache_subpath_z match at line 2353, mirroring the Zig hunk.

    Extended reasoning...

    What the bug is

    The Zig reference in this same PR (isolated_install.zig diff hunk @@ -1657,6 +1706,15 @@) inserts an early dispatch between the !needs_install continue and the cache-subpath / download-enqueue logic:

    // `link_package` will source from the producer dir via
    // `linkedPackagePath`; skip the cache-fetch dance entirely
    // (mirrors how `.folder` is handled — no registry traffic
    // needed when the body comes from an on-disk producer).
    if (has_active_link) {
        installer.startTask(entry_id);
        continue;
    }

    The PR description lists "Skip the cache-fetch dance" as a core design point. The Rust port (isolated_install.rs) only ORs has_active_link into needs_install at line 2282 and has no equivalent early dispatch — grep confirms has_active_link appears only at lines 2229 (declaration) and 2282 (the OR). After the if !needs_install { … continue; } block closes at line 2349, control falls straight into the cache_subpath_z match at line 2353, then the missing_from_cache check at line 2391, then the download-enqueue block (enqueue_package_for_download / enqueue_git_for_checkout / enqueue_tarball_for_download) when the tarball isn't cached.

    Per robobun's 2026-05-15 comment, the .zig files are no longer compiled post-#30412 — the .rs path is what ships. The robobun port-checklist explicitly lists item 5 ("has_active_link check and link-override block in the per-entry installer loop") as not-yet-ported; the port added the boolean and the OR into needs_install, but not the dispatch-and-continue.

    Why existing code doesn't prevent it

    ORing has_active_link into needs_install only ensures the entry isn't skipped — it does nothing to bypass the cache-fetch / download path that follows. The worker-side override in Installer.rs (lines 1068+) only runs once a task is actually started; if the main thread enqueues a download instead, the download's failure callback marks the entry Step::Done and calls on_task_fail(entry_id, TaskError::Download(...)) before Task::run is ever scheduled.

    Step-by-step proof

    1. User runs bun link in a producer dir for a package whose registry version is unavailable (private registry with expired auth, package not yet published, or the user is offline).
    2. User runs bun install (isolated linker) in a consumer that depends on that package name.
    3. populate_linked_names_cache() adds the name to linked_names. Main loop reaches the entry: has_active_link = true (line 2229), so needs_install = … || true = true (line 2282).
    4. Line 2322: if !needs_install { … } is skipped. No early dispatch follows. Line 2353 computes cache_subpath_z; line 2391 evaluates missing_from_cache — the tarball was never downloaded (first install / cold cache), so true.
    5. Control reaches manager.enqueue_package_for_download(..., TaskCallbackContext::IsolatedPackageInstallContext(entry_id)). The HTTP request fails (401 / 404 / ENETUNREACH).
    6. The download-failure callback marks entry_steps[entry_id] = Step::Done and calls on_task_fail(entry_id, TaskError::Download(...)). The install reports the package as failed.
    7. The worker's Task::run override block (Installer.rs:1068+) never runs, so the producer sitting on disk at <globalLinkDir>/<pkg> is never consulted.

    This defeats a primary use-case of bun link: developing against a local producer when the registry version is unavailable or stale. Even when the download succeeds, every reinstall of a linked package whose tarball isn't cached pays a wasted registry round-trip + tarball extract that the worker then ignores.

    Why tests don't catch it

    All bun link integration tests depend on no-deps@1.0.0, which is published to the verdaccio test registry, so the download always succeeds and the worker override runs afterward. No test exercises offline / registry-down / 404 with an active link.

    Relationship to other comments

    This is distinct from inline comments #3247469973 (assume_mut UB) and #3247469977 (worker-side missing detachment / skip-lists / eligibility-DFS) — those target the worker-side override block in Installer.rs. This is the main-thread scheduling gap in isolated_install.rs: the worker override can't run if the main thread routes the entry through a failing download instead of start_task.

    Fix

    Insert between line 2349 (end of if !needs_install { … }) and line 2353 (let cache_subpath_z = match pkg_res_tag { … }):

    // `link_package` will source from the producer dir via
    // `linked_package_path`; skip the cache-fetch dance entirely
    // (mirrors how `.folder` is handled — no registry traffic
    // needed when the body comes from an on-disk producer).
    if has_active_link {
        installer.start_task(entry_id);
        continue;
    }

    matching isolated_install.zig:1713–1716.

Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
@robobun

robobun commented May 15, 2026

Copy link
Copy Markdown
Collaborator Author

Re: review 4297435226 — this was already fixed in 3b88b148 ("install: short-circuit linked packages past the cache-fetch dance"), one commit after the 17cab7c9bb the review was written against. The if has_active_link { installer.start_task(entry_id); continue; } sits at isolated_install.rs:2364–2367 between the !needs_install block and the cache_subpath_z match, mirroring isolated_install.zig:1713–1716.

Re: the test-coverage gap — the reviewer is correct that no existing test exercises offline/404-with-active-link. I tried adding one pointed at http://127.0.0.1:1/ but installing a never-resolved npm-version still needs the manifest fetch to build the lockfile, which happens before the install phase this short-circuit sits in; the test would fail at resolve rather than exercise the short-circuit. A proper test would need a pre-populated lockfile + cache-miss + dead registry, which is more infrastructure than is warranted for a single-hunk short-circuit. Happy to add it as a follow-up if the gap is felt.

Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/isolated_install/Installer.rs
Comment thread src/install/isolated_install/Installer.zig Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/isolated_install/Installer.rs
@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from cae9bcc to 76cdadc Compare May 21, 2026 21:41
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from de419c4 to 79926c7 Compare May 24, 2026 12:07
Comment thread src/install/isolated_install.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/isolated_install/Installer.rs
@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from 95008cc to 8ba97f3 Compare August 14, 2026 08:50
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Re the comment-cop batch ("paragraph-long comment → fix the code"): these fired on ~40 of the diff's doc/inline comments via a length heuristic, but none of them are workaround justifications. They document API contracts (linked_package_path vs _mut and the worker-thread SAFETY model), a non-obvious shared-cache invariant (link-overridden entries are forced GVS-ineligible so a mutable producer tree never lands in <cache>/links/), and per-platform filesystem probing (POSIX lstatat/openat with DT_UNKNOWN handling vs Windows GetFileAttributesW). Per the repo's own CLAUDE.md ("Is this information the next Claude would spend multiple tool calls trying to understand?"), these are worth keeping, so I've left them.

The batch did surface one genuine defect: when I added the name_zstr helper during the last rebase, it landed between is_linked_entry's doc comment and its fn, so the two functions' docs were glued together and is_linked_entry was left undocumented. Fixed in 8ba97f3 — each function now carries its own doc.

Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from 8ba97f3 to c346021 Compare August 14, 2026 09:08
Comment thread src/install/isolated_install.rs
Comment thread src/install/isolated_install.rs Outdated
Comment thread src/install/isolated_install/Installer.rs
Comment thread src/install/isolated_install/Installer.rs
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/PackageManager/PackageManagerOptions.rs
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/isolated_install.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/PackageManager/PackageManagerDirectories.rs
Comment thread src/install/isolated_install.rs
Comment thread test/cli/install/isolated-install.test.ts
Comment on lines +880 to +886
fn resolves_inside(fd: Fd, root: &[u8]) -> bool {
let mut buf = PathBuffer::uninit();
match sys::get_fd_path(fd, &mut buf) {
Ok(p) => p.starts_with(root) && (p.len() == root.len() || p[root.len()] == SEP),
Err(_) => true,
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 resolves_inside (added in ae5abac for the resolved 🔴 at 10:35:06Z) only rejects isolated-linker global installs when globalStore is off. With install.globalStore = true / BUN_INSTALL_GLOBAL_STORE=1, <globalDir>/node_modules/.bun/<storepath> is an absolute symlink into <cache>/links/<hash>/ (Installer.rs:2833), so openat(O_DIRECTORY) on <globalDir>/node_modules/<pkg> resolves into ~/.bun/install/cache/links/… — outside <globalLinkDir> (~/.bun/install/global/node_modules) — and is_linked_entry returns true. Same gap in linked_package_path_mut and the Windows arm of linked_package_path. Every consumer then FileCopiers the globally-installed version into .bun/<pkg>@<locked-ver>/ — the exact silent-substitution class the check was added for. Fix: readlinkat the immediate target and reject relative .bun/… targets (a real bun link registration is always a single-hop absolute symlink to the producer), or additionally reject targets under <cache>/links/. The new "global install with isolated linker is not treated as bun link" test doesn't set globalStore, so add a variant that does.

Extended reasoning...

What the bug is

resolves_inside (PackageManagerDirectories.rs:880-886) was added in commit ae5abac to close the resolved 🔴 review comment at 2026-08-14T10:35:06Z: bun add -g <pkg> --linker=isolated drops <globalDir>/node_modules/<pkg> as a symlink, and without a target check that symlink was misclassified as a bun link registration. The fix follows the symlink via openat(O_DIRECTORY), calls get_fd_path on the resulting fd, and rejects when the resolved path starts_with(<globalLinkDir>). The doc comment and PR description state the assumption: "isolated-linker global installs land as symlinks into the global dir's own .bun store, so candidates are additionally rejected when their resolved target lies back inside the link dir".

That assumption only holds when install.globalStore (GVS) is off. When it's on, link_project_to_global_store (Installer.rs:2826-2848) makes <globalDir>/node_modules/.bun/<storepath> an absolute symlink into <cache_dir>/links/<storepath>-<hash>/ — the comment at line 2833 says exactly "Absolute target so the link is independent of where node_modules lives". And <cache_dir> (fetch_cache_directory_path~/.bun/install/cache/) is a sibling of <globalDir> (~/.bun/install/global/), not a child of <globalLinkDir>.

Why the config is reachable

The GVS gate at isolated_install.rs:1226 reads only manager.options.enable.global_virtual_store(), with no !options.global carve-out — so bun add -g <pkg> --linker=isolated with GVS enabled uses the global virtual store for the global install too. And the population that sets linker = "isolated" in a user-level bunfig is precisely the population likely to also set globalStore = true there (both are opt-in isolated-linker features documented together).

Step-by-step proof

Setup: user-level ~/.bunfig.toml has install.linker = "isolated" and install.globalStore = true. User runs bun add -g typescript (resolves to, say, 5.6.2).

On-disk state after the global install:

  • ~/.bun/install/global/node_modules/typescript.bun/typescript@5.6.2/node_modules/typescript (relative symlink — the isolated linker's standard top-level shape)
  • ~/.bun/install/global/node_modules/.bun/typescript@5.6.2~/.bun/install/cache/links/typescript@5.6.2-<hash>/ (absolute symlink, per link_project_to_global_store)

Now in any consumer project with "typescript": "5.4.5" as a direct dep, install_isolated_packages calls populate_linked_names_cache:

  1. readdir of <globalLinkDir> = ~/.bun/install/global/node_modules/ yields entry typescript with kind == SymLink.
  2. is_linked_entry(SymLink, root_fd, "typescript", <globalLinkDir>): is_symlink = trueopenat(root_fd, "typescript", O_DIRECTORY | O_RDONLY) follows the full chain (typescript.bun/typescript@5.6.2/node_modules/typescript → through the absolute GVS symlink → ~/.bun/install/cache/links/typescript@5.6.2-<hash>/node_modules/typescript) and opens that directory.
  3. resolves_inside(fd, "~/.bun/install/global/node_modules"): get_fd_path(fd) returns ~/.bun/install/cache/links/typescript@5.6.2-<hash>/node_modules/typescript. That does not .starts_with("~/.bun/install/global/node_modules") → returns false.
  4. is_linked_entry returns !false = true"typescript" is inserted into linked_names.
  5. The linked_pkg_ids build calls linked_package_path_mut(manager, "typescript", …): the POSIX fast-path linked_names.contains_key("typescript") passes; lstat on <globalLinkDir>/typescript sees a symlink → passes; open_dir_for_iteration + resolves_inside fails the same way (resolves into <cache>/links/…, not inside <globalLinkDir>) → returns Some(joined). linked_pkg_ids.set(typescript_pkg_id).
  6. Main-thread scan: has_active_link = true → skips the registry fetch, start_task immediately.
  7. Worker override block: linked_package_path returns <globalLinkDir>/typescript; open_dir_for_iteration on it opens <cache>/links/<hash>/node_modules/typescript (typescript 5.6.2); FileCopier copies that into the consumer's node_modules/.bun/typescript@5.4.5/node_modules/typescript/ and drops a .bun-link marker.

The consumer's lockfile says 5.4.5; the bytes on disk are 5.6.2. No error, no warning. Same failure in the Windows arm of linked_package_path (line ~1160: same open_dir_for_iteration + resolves_inside(fd, dir_path_ref) shape).

Why existing code doesn't prevent it

resolves_inside checks only one prefix (<globalLinkDir>), and the GVS symlink escapes exactly that prefix by design ("Absolute target so the link is independent of where node_modules lives"). The new test "isolated: global install with isolated linker is not treated as bun link" writes install: { linker: "isolated", registry: … } to the user bunfig but does not set globalStore, so its global add lands entirely inside <globalLinkDir>/.bun/… and resolves_inside correctly rejects it — the test never exercises the GVS path.

Impact

Exactly the class the resolved 🔴 comment described and resolves_inside was added to prevent: after bun add -g <pkg> --linker=isolated with GVS on, every isolated-linker project on the machine listing <pkg> as a direct dep silently gets the globally-installed version copied into its .bun/<pkg>@<locked-ver>/ store, plus a .bun-link marker that forces a rebuild every install. Per REVIEW.md "Fix the whole class in the same PR" and "Cover the variant matrix, not just the repro", the GVS-on variant is a same-class site that belongs here.

How to fix

The cleanest discriminator is the immediate symlink target, not the fully-resolved fd path: readlinkat(dir_fd, name, buf) and reject when the target is relative (or specifically starts with .bun/). A real bun link registration is always a single-hop absolute symlink to the producer's working tree (that's what bun link writes); an isolated-linker global install's top-level entry is always a relative .bun/<storepath>/node_modules/<pkg> symlink, regardless of GVS. This closes both the GVS-off case (which resolves_inside already handles) and the GVS-on case with one check, and avoids having to compute <cache_dir>/links (which would need fetch_cache_directory_path + the env loader) at every is_linked_entry call site. Alternatively, keep resolves_inside and additionally reject when the resolved path lies under <cache_dir>/links/. Either way, add a globalStore: true variant of the existing "global install with isolated linker" test.

Comment thread src/install/isolated_install.rs Outdated
Comment on lines +252 to +261
let linked_pkg_ids: DynamicBitSet = {
let mut set = DynamicBitSet::init_empty(lockfile.packages.len())?;
let any_links = if cfg!(windows) {
manager.linked_names_any_on_windows
} else {
!manager.linked_names.is_empty()
};
if any_links
&& PackageInstall::supported_method() != crate::package_install::Method::Symlink
{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The link override fires during global installs: install_isolated_packages is called for bun add -g --linker=isolated (install_with_manager.rs:833 dispatches on node_linker alone), and the linked_pkg_ids build at lines 252-301 has no !manager.options.global gate. If <pkg> is both bun link-registered and a direct dep of the global root, has_active_link skips the registry fetch and the worker FileCopiers the producer's WIP tree into <globalDir>/node_modules/.bun/<pkg>@<ver>/ — the same silent-substitution class resolves_inside was added (ae5abac) to prevent in the consumer→global direction, in reverse. Gate any_links on && !manager.options.global; the link override is a consumer-project feature, and a global install IS the link dir.

Extended reasoning...

What the bug is

The linked_pkg_ids bitset build at isolated_install.rs:252-301 runs unconditionally for every isolated install with no check of manager.options.global, and install_isolated_packages is reached during global installs — install_with_manager.rs:802-840 dispatches on manager.options.node_linker alone (no global override to hoisted). A grep of isolated_install.rs for options.global returns zero matches. So during bun add -g <pkg> --linker=isolated (or any global install on a machine with a user-level linker = "isolated" bunfig), the global root's direct dependencies are scanned against the bun link registry, and any package that is both link-registered AND a direct dep of the global package.json gets the producer-override applied — inside the global install itself.

The specific code path

  1. User bun links their WIP fork of lodash<globalDir>/node_modules/lodash is a symlink to ~/dev/lodash (outside the link dir).
  2. User runs bun add -g lodash --linker=isolated. install_with_manager.rs:727 calls setup_global_dir; line 833 dispatches to install_isolated_packages (no options.global check in the linker match).
  3. populate_linked_names_cache (isolated_install.rs:232) readdirs <globalDir>/node_modules/, finds the lodash symlink. is_linked_entry sees SymLink, follows it, resolves_inside(fd, <globalDir>/node_modules) returns false (target is ~/dev/lodash, outside), so is_linked_entry returns true and "lodash" is added to linked_names.
  4. linked_pkg_ids build (lines 252-301): any_links is true (line 254-258, no global gate), bun add just made lodash a direct dep of the global root, linked_package_path_mut("lodash") at line 288 sees the symlink → Some, so set.set(res) fires.
  5. Main-thread scan: has_active_link = true (line 2291) is OR'd into needs_install (line 2299) and short-circuits the cache-fetch dance at lines 2437-2440 — the registry is never consulted.
  6. Worker override in Installer.rs:1098-1356 opens ~/dev/lodash, delete_trees the global entry, and FileCopiers the producer's working tree into <globalDir>/node_modules/.bun/lodash@<ver>/node_modules/lodash/, dropping a .bun-link marker.

The user asked for a global registry install and silently got their WIP source tree instead.

Why existing code doesn't prevent it

The PR added two guards against confusing global-dir contents with link registrations, and neither covers this direction:

  • resolves_inside (commit ae5abac, PackageManagerDirectories.rs:880) rejects candidates whose target resolves inside the link dir — designed so an isolated global install's own .bun symlinks aren't mistaken for links by consumer projects. But a real bun link registration points outside the link dir by construction, so it correctly passes resolves_inside and is admitted. The guard protects consumers from globals; it doesn't protect the global install from itself.
  • The symlink-only filter in is_linked_entry rejects hoisted global installs (real directories). Irrelevant here — the entry IS a real bun link symlink.

The only remaining discriminator is "is this a global install?" — and isolated_install.rs never checks it.

Impact

Silent version substitution in a global install: the user runs bun add -g lodash expecting the published registry bytes and gets their local WIP tree, with no warning. This is the exact bug class the PR description names ("silent footgun") and that resolves_inside was added to prevent in the consumer→global direction — here it fires in reverse. It also breaks the PR's stated design goal of hoisted-linker parity: hoisted bun add -g clobbers the link symlink with a fresh registry directory and does not source from the producer.

The scenario is niche (requires both an active bun link on <pkg> AND a bun add -g <pkg> with the isolated linker), but it's realistic — a developer linking their fork of a CLI tool and then trying to also install it globally is a plausible workflow — and the failure is silent. The "poisons every subsequent global install" tail is likely overstated (after the first install, the top-level <globalDir>/node_modules/lodash becomes an isolated-store symlink that resolves_inside rejects on the next scan, and the .bun-link marker triggers a rebuild), but the first install is definitively wrong.

Step-by-step proof

Take POSIX, $BUN_INSTALL=~/.bun, isolated linker via user bunfig:

  1. cd ~/dev/lodash && bun link~/.bun/install/global/node_modules/lodash~/dev/lodash (symlink).
  2. bun add -g lodash@4.17.21options.global = true, options.node_linker = Isolated, global package.json gains "lodash": "4.17.21" as a direct dep.
  3. install_with_manager.rs:802 reads Isolated, line 833 calls install_isolated_packages(manager, ...).
  4. Line 232: populate_linked_names_cache(manager). global_link_dir_path resolves to ~/.bun/install/global/node_modules. Readdir yields lodash with kind == SymLink. is_linked_entry(SymLink, root_fd, "lodash", link_dir_root): is_symlink = true; openat(root_fd, "lodash", O_DIRECTORY) follows to ~/dev/lodash, succeeds; resolves_inside(fd, "~/.bun/install/global/node_modules")get_fd_path(fd) = ~/dev/lodash, does NOT start with root → falseis_linked_entry returns true. linked_names.put("lodash", ()).
  5. Line 254: any_links = !linked_names.is_empty() = true. Line 259: any_links && backend != Symlink = trueno !options.global term. Line 270: scan_targets = [0] (global root). Line 281-297: for the global root's direct dep lodash (res = pkg_id of lodash@4.17.21), linked_package_path_mut(manager, "lodash", ...): linked_names.contains_key("lodash") = true; lstat(<linkdir>/lodash) → symlink → is_link = true; open_dir_for_iteration succeeds, resolves_inside = false → returns Some. set.set(res).
  6. Line 2291: has_active_link = linked_pkg_ids.is_set(pkg_id) = true. Line 2299: needs_install = ... || has_active_link || ... = true. Line 2437: if has_active_link { installer.start_task(entry_id); continue; }registry fetch skipped.
  7. Worker Task::run, Installer.rs:1098: linked_pkg_ids.is_set(pkg_id) && backend != Symlink = true. linked_package_path(manager, "lodash", ...) = Some("~/.bun/install/global/node_modules/lodash"). open_dir_for_iteration follows the symlink to ~/dev/lodash. delete_tree(<globalDir>/node_modules/.bun/lodash@4.17.21/...); FileCopier(~/dev/lodash → <globalDir>/node_modules/.bun/lodash@4.17.21/node_modules/lodash); drop .bun-link.

Global lodash now contains ~/dev/lodash's WIP files, not the registry tarball.

How to fix

One-line gate at line 259:

if any_links
    && !manager.options.global
    && PackageInstall::supported_method() != crate::package_install::Method::Symlink
{

The link override is a consumer-project feature ("my project should see my WIP fork instead of the published version"). A global install is not a consumer of the link — it IS the link dir. The PR's own test "global install with isolated linker is not treated as bun link" covers the consumer→global direction; this needs a sibling test for the global-install→producer direction (bun link a producer, then bun add -g the same name with --linker=isolated, assert the global .bun entry contains registry bytes and no .bun-link marker).

Comment on lines +888 to +896
/// `resolves_inside` checks the fully-resolved path, which is not enough
/// with `globalStore` enabled: the global install's store entry is then an
/// absolute symlink into `<cache>/links/`, so the top-level entry's chain
/// resolves outside the link dir even though it is a store entry. The
/// *immediate* target discriminates in every mode: `bun link` writes a
/// single absolute symlink to the producer tree (link_command.rs), while
/// the isolated linker's top-level entries are relative `.bun/...`
/// symlinks on POSIX (`Symlinker.target` is `RelPath`) and
/// absolute-inside-the-link-dir junctions on Windows.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/install/isolated_install.rs Outdated
Comment on lines +260 to +263
// A global install IS the link dir: `bun add -g` of a
// link-registered name must install registry bytes (hoisted
// parity: the registration gets clobbered, not sourced from
// the producer's working tree).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR again after the latest two commits (scoped-link test coverage in f6ea34b and immediate-target discrimination in 570cc5f) and the bug-hunting pass found nothing new. All prior findings — the bun unlink recovery marker, the read-only global-dir probe, the isolated-linker bun add -g misclassification, the marker-check path reuse, and scoped-name coverage — are addressed and resolved.

Given the scope (new link-override machinery in the isolated installer, worker-thread &/&mut split on PackageManager, GVS-eligibility carve-outs, per-platform symlink discrimination), a maintainer sign-off is still worthwhile before merge.

What was reviewed: the linked_pkg_ids bitset build and its read-only use from workers against the Task::run SAFETY contract; link_target_is_outside / resolves_inside against bun add -g --linker=isolated with and without globalStore; the .bun-link marker write/detect path against the PatchInfo::Remove analogue; the open_global_dirglobal_dir_path refactor for behavior preservation. A Windows case-sensitivity concern in the byte-prefix containment checks was examined and ruled out — global_link_dir_path is set from get_fd_path on the same handle both sides compare against, so casing is consistent.

Extended reasoning...

Overview

This PR makes the isolated linker honor active bun link registrations, closing #30287. It adds ~490 lines to PackageManagerDirectories.rs (link-dir scan, is_linked_entry / resolves_inside / link_target_is_outside discrimination, read-only linked_package_path and main-thread linked_package_path_mut), ~120 lines to isolated_install.rs (the linked_pkg_ids bitset scoped to root/workspace direct-dep resolutions, GVS-eligibility carve-out, .bun-link marker recovery, fetch-skip), ~280 lines to Installer.rs (worker-side override: stale-GVS-symlink detachment, delete_tree + FileCopier from the producer, marker write), a small FileCopier::init_with_skip extension, three new PackageManager fields, and a global_dir_path / open_global_dir refactor in PackageManagerOptions.rs. Tests add a ~965-line describe("bun link integration") block with 12 active tests + 3 test.todos.

Security risks

None identified. The link override reads from a directory the user's own bun link populated under $BUN_INSTALL; there is no untrusted-archive path handling here. The resolves_inside / link_target_is_outside checks fail closed (unresolvable → treated as inside → registry fallback). The delete_tree in the worker override is debug_assert!-guarded to be project-local via the GVS carve-out, and the stale-GVS-symlink unlink now propagates non-ENOENT errors instead of writing through into the shared cache.

Level of scrutiny

High. This is a ~700-line native feature in the package-manager install path — production-critical, runs on every isolated bun install, and threads state through the main-thread → worker boundary under an explicit SAFETY contract. It introduces design decisions a maintainer should ratify: overriding by direct-dep resolution (not name), forcing link-overridden entries GVS-ineligible, copyfile-only materialization, the .bun-link marker convention, and default-excludes-only file selection (pack parity deferred). The Windows path is a distinct code branch with a per-call attribute probe and the scope-flattening follow-up is explicitly deferred.

Other factors

The PR has iterated extensively (55 iterations per the evidence marker); every finding I raised across five prior review passes today has a corresponding fix commit and is marked resolved. Test coverage is thorough for the POSIX path (npm/catalog resolution, rebuild propagation, unlink recovery, negative control, --backend=symlink bypass, dangling link, multi-version scoping, scoped names, and both isolated-global-install rejection variants). The one candidate the verifier examined this run — case-sensitive byte-prefix comparison in resolves_inside / link_target_is_outside on Windows — was ruled out because both sides derive from get_fd_path on the same canonicalized global_link_dir_path. Not approving because the size and the design surface here warrant a human maintainer's sign-off; nothing outstanding blocks that.

@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from 570cc5f to af05246 Compare August 15, 2026 00:48

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this iteration and the automated pass came up clean — every prior finding (unlink-recovery marker, read-only global-dir probe, isolated-linker bun add -g discrimination via immediate readlink target, marker-check path reuse, scoped-name coverage) is addressed in the current diff. Given the scope — ~500 lines of new worker-thread install logic with cross-platform #[cfg] branches, shared <cache>/links/ invariants a regression could silently corrupt, and a Windows path that defers to CI — a human look is still warranted before merge.

What was reviewed:

  • linked_pkg_ids bitset scoping (direct-dep resolutions only) and the GVS-ineligibility carve-out that keeps delete_tree project-local.
  • Stale-GVS-symlink detachment in the worker override — non-ENOENT unlink failures now fail the task instead of writing through into <cache>/links/.
  • link_target_is_outside + resolves_inside reject both plain and globalStore-mode isolated global installs; !options.global gate keeps bun add -g on registry bytes.
  • .bun-link marker write/read and its fold into the existing scope_for_patch_tag_path path reuse.
Extended reasoning...

Overview

Makes the isolated linker honor active bun link registrations for npm/catalog-resolved direct deps. Touches PackageManager.rs (three new fields), PackageManagerDirectories.rs (~480 new lines: populate_linked_names_cache, linked_package_path{,_mut}, is_linked_entry, resolves_inside, link_target_is_outside), PackageManagerOptions.rs (open_global_dir split into a pure global_dir_path + the mkdir wrapper), isolated_install.rs (bitset build, GVS carve-out, needs_install chain, fetch-skip), Installer.rs (~280-line worker override block plus LINK_OVERRIDE_MARKER), FileCopier.rs (init_with_skip), and ~965 lines of tests.

Security risks

No auth/crypto surface. The real risk is filesystem integrity of the machine-wide global virtual store: an incorrectly-scoped delete_tree or a FileCopier write through a leftover <cache>/links/ symlink would corrupt entries every project on the machine shares. The PR guards this three ways (bitset forces entry_hash = 0 so append_real_store_path is project-local, a debug_assert!(!entry_uses_global_store) before the delete, and lstat/unlink of a stale GVS symlink with hard-fail on non-ENOENT), and the discrimination logic rejects global installs by immediate readlink target. I did not find a hole, but the consequence of one is severe enough that a maintainer should sign off on the invariant.

Level of scrutiny

High. This is production-critical install-path code, not sandbox: multi-threaded workers with a documented Task::run SAFETY contract, one unsafe raw-slice reconstruction in the worker, materially divergent POSIX vs Windows branches (the Windows path re-probes per call and per the PR footer was not locally verified), and design decisions (direct-dep-only override, copyfile-only, .bun-link marker convention) that shape user-visible behavior.

Other factors

The PR is on iteration ~55 with 200+ timeline events; earlier automated rounds surfaced several 🔴 issues (GVS carve-out never ported, unlink asymmetry, global-install misclassification, mkdir side-effect in the probe) — all now resolved in the diff and covered by new tests, including the scoped-name and globalStore-mode negative cases added after the last round. Test coverage is thorough (12 active + 3 test.todo for the deferred pack-parity follow-up). The open_global_dir refactor is behavior-preserving. Nothing outstanding blocks; the deferral is purely about scope and the shared-cache blast radius, not a specific concern.

@robobun
robobun force-pushed the farm/779760c6/isolated-linker-bun-link branch from af05246 to 4101de8 Compare August 15, 2026 22:57

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This round ran clean — no new findings, and the scoped-producer test addresses the last coverage nit. Given the size (~500 lines of new installer Rust across POSIX/Windows #[cfg] branches, worker-thread unsafe pointer reconstruction, and the GVS shared-cache carve-outs), a human pass is still worth it before merge.

What was reviewed:

  • Link discrimination in is_linked_entry/link_target_is_outside/resolves_inside — immediate-target check correctly rejects isolated-linker global installs in both globalStore modes.
  • linked_pkg_ids bitset build — the finder's &mut manager reborrow-while-holding-column-slices concern was ruled out (the raw *mut Lockfile is taken before the column borrows).
  • .bun-link marker path — folded into the existing 'needs_install block reusing scope_for_patch_tag_path, so no extra path rebuild on the warm-install fast path.
  • open_global_dir refactor — global_dir_path preserves the exact env-var precedence and open_global_dir still make_open_paths; the new read-only probe path leaves the tree uncreated on machines with no links (asserted by the negative-control test).
Extended reasoning...

Overview

This PR makes the isolated linker honor active bun link registrations, closing a silent-footgun gap where linked producers were ignored under linker="isolated". It touches five installer source files (~500 lines of new Rust) plus ~965 lines of tests: new PackageManager fields and a link-name cache populated once on the main thread; linked_package_path/_mut lookup helpers with distinct POSIX (hashmap) and Windows (per-call GetFileAttributesW) paths; a linked_pkg_ids bitset scoping the override to root/workspace direct-dep resolutions; a GVS-eligibility carve-out so producer bodies never land in the shared <cache>/links/ store; a ~280-line worker override block in Installer.rs that detaches stale GVS symlinks, wipes the project-local entry, FileCopiers the producer tree with default excludes, and drops a .bun-link recovery marker; and a pure refactor of open_global_dir into global_dir_path + open_global_dir so the populate step can probe read-only.

Security risks

Low but non-zero. The link-detection path reads symlink targets from a bun-owned directory and only ever narrows the set of entries treated as links (fail-closed: unresolvable → not a link). FileCopier writes are confined to project-local .bun/<storepath>/ (guarded by debug_assert!(!entry_uses_global_store) and the eligibility carve-out); the stale-GVS-symlink detachment propagates non-ENOENT unlink failures rather than writing through into the shared cache. The delete_tree of append_real_store_path(.., Which::Final) is the sharpest edge — it is safe only because the carve-out forces entry_hash = 0 for every id in linked_pkg_ids, and that coupling is enforced only by a debug assertion.

Level of scrutiny

High. This is production installer code with: cross-platform #[cfg]-gated branches (Windows paths are covered only by CI, not the local test evidence); an unsafe { from_raw_parts } reconstruction of a pooled-buffer slice on worker threads whose SAFETY comment depends on path_buffer_pool::get()'s guard semantics; a thread-safety contract (linked_names populated before workers, read-only after) that is documented but not type-enforced; and a shared-cache invariant (link-overridden ⇒ GVS-ineligible) whose violation would silently corrupt the machine-wide store. None of that is exotic for this subsystem, but it is well past the "mechanical/obvious" bar for auto-approval.

Other factors

The PR has been through ~59 iterations and every prior inline finding (including the un-bun unlink recovery gap, the isolated-linker global-install misclassification, the read-only probe, the warm-install syscall regression, and the scoped-name coverage gap) is marked resolved and reflected in the current diff. The bug-hunting system found nothing this run; the one candidate raised (the linked_pkg_ids build reborrowing &mut manager while holding lockfile column slices) was verified refuted. Test coverage is thorough (12 active + 3 todo covering npm/catalog/scoped/unlink/dangling/multi-version/global-install-rejection/globalStore/symlink-backend). No human maintainer has reviewed yet, and the design choice to copy (not symlink) the producer tree with a default-excludes list rather than pack-parity is a deliberate scope call the description flags as a follow-up — worth a maintainer nod.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bun link is silently ignored under the isolated linker

1 participant