install: don't mark a placed peer optional when saving a migrated bun.lock - #37289
install: don't mark a placed peer optional when saving a migrated bun.lock#37289robobun wants to merge 5 commits into
Conversation
….lock After migrating a package-lock.json, the bun.lock writer checks every non-optional peer dependency for a resolution in the hoisted tree and lists peers without one in optionalPeers so the saved lockfile stays parseable. The lookup walked up from the parent tree node's path instead of the package entry's own path, so a peer whose resolution is placed nested under the package itself (the entry "gamma/delta" for gamma's peer delta) was missed and marked optional even though its resolution was written two lines below. Reloading that lockfile turns the peer into an optional peer, and since bbe3f6a optional-peer resolution slots are re-derived from the rebuilt tree during cleaning. A nested-only placement is not visible from the dependent's position, so the entry was dropped: the next bun install --frozen-lockfile on an unchanged tree failed with "lockfile had changes, but lockfile is frozen", and a plain install silently rewrote the lockfile. Pass the package entry's own tree path (its packages key, the same string the parser passes to find_resolution when binding that entry's dependencies), so writer and parser agree on which placements a dependency can see.
|
Updated 2:34 PM PT - Aug 9th, 2026
❌ @robobun, your commit 60cb27f has 1 failures in
🧪 To try this PR locally: bunx bun-pr 37289That installs a local version of the PR into your bun-37289 --bun |
WalkthroughPackage serialization now writes complete package tree paths and passes them to package metadata and peer dependency resolution. A migration regression test covers nested file dependency peers, frozen installs, and repeat-install lockfile stability. ChangesLockfile package path resolution
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/cli/install/migration/migrate.test.ts`:
- Around line 488-494: Shorten the comment above the gamma/delta fixture to two
concise lines that preserve only the invariant: delta is a regular peer
dependency with nested-only placement and must not appear in gamma’s
optionalPeers. Remove the detailed reload, resolution-slot, and frozen-lockfile
explanation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: ed1a95f3-7517-40ce-afa7-44881af39ddf
📒 Files selected for processing (2)
src/install/lockfile/bun.lock.rstest/cli/install/migration/migrate.test.ts
There was a problem hiding this comment.
LGTM — targeted fix aligning the writer's peer-resolution probe with the parser's.
What was reviewed:
- The entry-key write is byte-identical to before (both halves already went through
format_json_string_utf8withquote: false, and/is not JSON-escaped;relative_pathalready carried interior slashes through the same escaper for depth>1). find_resolutionnow starts from the entry's own key instead of its parent's — the probe set is a strict superset (addsentry_path/dep_namefirst, then walks up as before), so peers can only stop being marked optional, never start.write_workspace_depschange is a parameter rename only; both callers already passed the workspace's own tree path (b""for root,pkg_names[id]otherwise).- New test is hermetic (all
file:deps, no network), uses the sharedinstall()helper which drains stderr concurrently, and asserts the fixed-point invariant directly.
Extended reasoning...
Overview
The PR fixes a lockfile round-trip bug in src/install/lockfile/bun.lock.rs: when serializing a binary/migrated lockfile to bun.lock, the writer checks each non-optional peer for a resolution via PkgMap::find_resolution and marks it optionalPeers if none is found. That lookup was seeded with the parent tree node's relative_path instead of the package entry's own key, so a peer placed only nested under the package (e.g. "gamma/delta") was invisible and wrongly marked optional. On reload, #35681's re-derive logic then dropped the entry, breaking --frozen-lockfile on an unchanged tree. The fix builds entry_path = relative_path[/]dep_name once per entry (also used as the JSON key, replacing three writes with one) and passes it to write_package_info_object. The write_workspace_deps hunk is a parameter rename with no caller changes. A new concurrent test in migrate.test.ts reproduces the migration → frozen-lockfile → reinstall sequence.
Security risks
None. This is lockfile-writer path-resolution logic operating on already-parsed internal data structures; no untrusted input handling, no filesystem writes beyond the lockfile, no network. The entry-key output is byte-identical to before.
Level of scrutiny
Medium. Lockfile serialization is user-visible and affects --frozen-lockfile in CI, so a regression would be widely felt. However, the change is narrow and provably monotone: find_resolution walks up the tree from its seed path, and the new seed (entry_path) is exactly one level deeper than the old seed (relative_path), so every probe the old code did is still done — the fix only adds the entry_path/dep_name probe first. A peer previously found is still found; only previously-missed nested placements are now seen. The written key is unchanged (verified that / passes through format_json_string_utf8 unescaped, and relative_path already contained slashes for depth>1 entries going through the same escaper).
Other factors
The test is well-constructed: hermetic (tempDir, all file: dependencies, no registry), uses the file's existing install() helper (which drains stderr and awaits exit concurrently), runs as test.concurrent, and asserts the strongest invariant — byte-identical lockfile after re-install plus --frozen-lockfile exit 0. The PR description reports passing runs across the migration, bun-lock, hoist, workspace, isolated, and peer/optional test suites. All comment-cop and CodeRabbit feedback (comment length) has been addressed and resolved. entry_path_buf is declared once outside the loop and clear()ed per iteration, avoiding per-entry allocation.
|
CI on 60cb27f: the only hard failure is test/napi/napi.test.ts on macOS 14 x64, which is pre-existing on main and reported separately. The remaining red entries are retry-passed flakes on lanes this diff does not touch (install tests on Windows aarch64 and macOS, http2 and fetch-backpressure on macOS, and parallel-batch timeouts that passed when rerun alone). The lockfile change and the new migration round-trip test are green on every lane. |
|
Closing in favor of #38333, which carries the same |
…ilter/--catalog, nested overrides, transitive update, and workspace fixes (#38333) Brings Bun's package manager to parity with pnpm for monorepo workflows, and fixes the bugs found while checking every command against pnpm's implementation, pnpm's test suites, pnpm's open issue tracker, pnpm's docs, npm's arborist fixtures, and — for `bun update` — running real pnpm and Bun side by side on the same projects. ### What does this PR do? #### New commands - **`bun dedupe [--check]`** — collapses duplicate versions in `bun.lock` onto the smallest set that still satisfies every dependent's range, using only versions already in the lockfile, then installs. Never downgrades a direct dependency unless that is the only way to drop a version; keeps patched versions (and anything needed to reach them, and says so); refuses to run on a lockfile that is behind `package.json`. `--check` exits 1 without writing. - **`bun prune [--production | --omit=…] [--dry-run] [--filter <ws>]`** — removes everything in `node_modules` that the lockfile does not put there; `--production` leaves exactly what `bun install --production` would. Hoisted and isolated layouts, Windows junctions and shims, workspace links, bundled deps; refuses when `package.json` and `bun.lock` disagree or when `node_modules` was laid out by a different linker; understands turbo-pruned checkouts. - **`bun pm licenses [--json] [--prod|--dev] [--long] [--filter <ws>]`** — installed packages grouped by license, with a `(dev)` marker and `paths`/`license`/`description` in `--json`. - **`bun audit fix [--latest] [--dry-run] [--json]`** — moves each vulnerable package to the lowest safe version its dependents accept, per installed instance; rewrites exact pins when that is the only way; `--latest` also rewrites your own declared ranges (root, workspace, catalog) so a semver-major fix can be taken, and every blocked or unfixable item is followed by the command that resolves it (`bun audit fix --latest`, `bun audit --ignore GHSA-…`); re-audits the tree it actually installed and reports/exits from that second response (npm's `_submitQuickAudit`), so an advisory that starts at the version it moved to is not missed; works across registries; security fixes bypass `minimumReleaseAge` with an annotation. `bun audit --json` honors `--audit-level`/`--ignore` for its exit code; `--omit` is honored by `audit` and `licenses`. #### `bun update` semantics (pnpm's model) - A bare `bun update` re-resolves **transitive** packages too — every edge moves to the newest version its own range (or dist-tag) allows, per dependent, so `bun.lock` no longer stays stale after an update; overrides/catalogs changed since the last install are honored. From a workspace member or `--filter`, only what the selected workspaces reach is re-resolved; from the root, everything. - `bun update <name>` reaches any depth, matches `npm:` aliases by real name, updates in place, never adds to `package.json`, and errors on a name nothing selected depends on. `-r`/`--filter` fan a named update out across workspaces. `--latest` never downgrades a locked version that is ahead of the tag, and `update <name> --latest` also refreshes that package's own dependencies. - Plain updates keep dist-tag literals and non-caret ranges (`*`, `1.x`, `^1 || ^2`) exactly as written and only move the lockfile; `--latest` rewrites them to the resolved version as before. `bun update -i` applies only what you selected. New: positional patterns (`bun update '@types/*'`), `--dev`/`--prod`/`--no-optional`, `-L`, `bun up`. - `package.json` is written after resolution and `bun.lock`'s declared ranges, overrides and catalogs are re-derived from the final `package.json`, replacing the per-command literal rewriting; a no-op update leaves the file byte-identical. #### Overrides - **Nested overrides** (#6608): npm's nested objects, yarn's `a/b` paths and pnpm's `a>b` selectors, applied to the direct parent→child edge; **version-scoped targets** (`"lodash@<4.17.21": "4.17.21"`, the shape `pnpm audit --fix` writes), matched against the dependent's declared range as pnpm does. Rules persist inside the `overrides` section and the file is stamped `lockfileVersion: 3` **only when such rules exist** — existing lockfiles are byte-identical. Flat overrides additionally fix `$ref` to workspace-member deps, catalog-valued rules going stale, and warn on pnpm's `-` / `pkg@` forms. #### Workspaces and filters - `bun add|remove|update … --filter <ws>` (also `-F`, also `bun install <pkg> --filter`) edits the selected workspaces' `package.json` files and **links only those workspaces**, like `bun install --filter`. Filters gain pnpm's relation selectors (`foo...`, `...foo`, `foo^...`, `...^foo`) and `{dir}` subtrees, for the install family **and** `bun run --filter`; `--filter` may precede the subcommand; every command warns about patterns that match nothing; `add`/`remove` no longer select the root implicitly. - `bun add <pkg> --catalog[=name]` reuses an existing catalog entry, keeps a range an explicit version fits, catalogs the range a package already declares, decides per target, and refuses workspace names and local paths; a plain `bun add` uses a default-catalog entry when one exists. A package defined in both `catalog` and `catalogs.default` is an error. `catalog:` peers of registry packages bind to the importer's copy instead of the root catalog. - `--frozen-lockfile` / `bun ci` on turbo-pruned monorepos: pruned-away workspaces are tolerated, a survivor depending on a pruned workspace is an error, catalog subsets are accepted, and an overrides/catalogs change is a frozen failure. #### One output vocabulary Every command here prints the install family's shapes: header, glyph rows (`+`/`-`/`↑`, dedupe's `↳ name old → new`), exactly one noun-first summary line with counts and a duration (`2 duplicate versions removed, 3 packages installed (checked 5 packages) [12ms]`, `N packages removed (checked C) [t]`), no-ops that say what was checked, remedies printed as copy-pasteable command lines, warnings as `warn:`, `--silent` printing nothing, and errors with their remedy together on stderr. Transitive and named updates render as the summary's `↑` rows (once per package; `--dry-run` prints the same rows plus `N packages would be updated`); dedupe reports after the install it triggers, so lifecycle-script output never splits it. A lockfile whose bytes did not change is no longer rewritten (`Saved lockfile` only prints on a real write; `--lockfile-only` no-ops print `Done! Checked N packages (no changes)`). This came out of running every command against fixtures and comparing with `install`/`add`/`remove` (95 findings, all fixed). #### Config precedence A project's `bunfig.toml` now beats any `.npmrc` (project or user-level) for the same key (npmrc files → bunfig's set fields → CLI); npmrc-only settings such as `//host/:_authToken` still attach to bunfig-declared registries, matched by host and path regardless of how either file spells the trailing slash. #### Lockfile migration - `package-lock.json`: rebuilt around a reachability walk that derives each resolution from the entry itself. Fixes `git+https://github.com/…` resolutions being written unparseably (the next install threw the lockfile away), root `bundleDependencies` migrating to an **empty** lockfile, lockfileVersion 1 (and npm's upcoming 4) making `bun install` exit 1 instead of resolving fresh, dependency-level bundles, `dependencies`+`optionalDependencies` double edges, unreferenced entries aborting the migration, duplicate packages for identical `name@version` at nested paths, lost `optionalPeers`, lost integrity when a bundled copy was seen first, and `overrides` not being carried over. All 57 of arborist's v2/v3 fixture projects are vendored and migrated under snapshot. - `pnpm-lock.yaml` v9: bare-hash `patchedDependencies`, snapshot aliases, `catalog:default`, recorded tarball URLs, git `path:`, multi-document files, `runtime:` entries, named registries, peer-suffixed keys chosen per importer, injected workspaces, manifest-only importer deps. #### Isolated linker - An existing store entry whose dependencies re-resolved (override, dedupe, update) now has its links refreshed on the next install (measured cost below). `bun prune` builds the same store the installer builds, so stale `name@version+<peerhash>` variants left by peer bumps are removed and a kept package's real entry never is (whether it was installed with full or `--production` features); on the hoisted linker, dedupe / audit fix / update delete the nested copies whose rows they collapsed instead of leaving the old copy loadable. - Blocked entries resume through per-entry intrusive waiter lists instead of a scan of every store entry after each completion (robobun's #25983/#28425 attempted this). Measured on the reporter's repro from #25799 (2,259 store entries) and a synthetic 6,425-entry monorepo, PR build vs merge-base build: main-thread CPU in the link phase drops 0.85 → 0.33 s and 5.5 → 0.85 s (the removed work grows quadratically); wall time is unchanged with spare cores and 16% / 26% faster pinned to one CPU, the CI/Docker shape in those reports. (The minute-long installs originally reported were peer resolution, fixed before this PR's base.) - Two pre-existing leaks surfaced by the new LSan-enabled tests are fixed: the header buffer of every authenticated registry request, and the per-entry lifecycle-script lists. #### Other bug fixes `bun add x@npm:pkg` writes a range; `bun add --trust a b` no longer drops `b` when `a` was already trusted; `bun add x --dev` no longer rewrote every group in `bun.lock`; `catalog:` peer hoisting; `dependency::Version::eql` treated all `catalog:` specifiers as equal; a `file:` package whose dependencies reach itself (its own name, an `npm:` alias under its own name, two link targets depending on each other — the shapes a `package-lock.json` migration produces, and #25202's `workspace:.` self-reference) hung `bun install` forever in the hoisting tree — the migration shapes now install, and #25202's literal shape now terminates with `Workspace dependency "foo" not found` rather than installing as npm does; a peer of a `file:` package that was only placed nested was also written to `optionalPeers` in a migrated bun.lock, so the next `--frozen-lockfile` failed and a plain install rewrote the lockfile; `catalog:` literals in `bun update`; alias output in the install summary; help/completions for everything above. #### Behavior changes to note in the release notes - `bun update` moves transitive packages; `bun update <name>` no longer adds an undeclared package (exit 1); `--production`/`--prod` on update means "only update `dependencies` and `optionalDependencies`" (a group filter like `--dev`, not the install flag) and `-r`+names with no match is an error; `-i` updates only the selection. - Project `bunfig.toml` overrides any `.npmrc`. - `bun install <pkg> --filter x` edits `x` (not the root); `bun add y --filter x` no longer installs a package named `x`; `add`/`remove --filter '*'` no longer includes the root. - A plain `bun add x` in a workspace whose default catalog lists `x` writes `catalog:`; `audit fix` may rewrite exact pins; `--frozen-lockfile --lockfile-only` writes nothing; overrides/catalog changes fail frozen installs. - One-time lockfile churn after upgrading for projects with `catalog:` peers or dead `pkg@range` override rows; lockfiles that use nested/scoped overrides are v3 and unreadable by older Bun (only when opted in). Turborepo, Nx and Dependabot have been checked; the needed upstream changes are open (nrwl/nx#36666 covers v2 and v3; vercel/turborepo#13740 accepts v3 and preserves the object rows through prune — turborepo main today parses v2 and rejects v3; dependabot needs nothing). Note v2 itself only exists on the 1.4 line. - `bun audit --json` keeps npm's contract: `--audit-level`/`--ignore` decide the exit code, the JSON document is the full registry report (closes #31013 as won't-change). Automatic removal of stale `node_modules` entries on plain `bun install` (#32974) is separate from this PR: `bun prune` is the manual form, and dedupe / audit fix / update now clean up the nested copies they collapse on the hoisted linker; #32974 should reuse prune's planner, and #29512 (sbom) is sequenced after this so it can build on `reachable.rs` instead of carrying its own walk. - Deliberately kept where we differ from pnpm: root `bun update` covers the whole workspace; dependents whose ranges allow follow a moved version (one copy, not two); `--latest` works on transitive names; `--no-save` touches neither file; prune deletion failures exit 1; audit requests stay per-registry. #### Performance Measured on a 1,113-package Next/Prisma/MUI app (PR build vs a PR build of the merge base, interleaved, plus canary and 1.3.14): every hoisted cell is within noise except no-op install, +0.8 ms (+2%, identical syscalls); isolated no-op is +1.9 ms (+3.9%) — the deliberate cost of re-checking existing entries' links every install rather than persisting a stamp file. Everything added is otherwise off the plain-install path (gated on the feature being used or on a diff), and id-indexed sets are bitsets. ### How did you verify your code works? ~1,100 new or ported test cases across the install suites (designed behavior, cases ported from pnpm's suites, pinning tests for pnpm bugs this implementation is immune to, arborist's fixtures, and the CI review findings), all `toStrictEqual`; the whole `test/cli/install` directory passes locally and the existing suites are unchanged except where a pre-existing expectation was deliberately changed (each listed above). `bun update` was additionally verified with a rerunnable differential harness that runs pnpm 11 and this branch on 26 scenario families against one registry and diffs the resulting resolutions edge by edge — after this PR only the deliberate differences above remain. Ecosystem: Turborepo, Nx and Dependabot were checked against the new lockfile output. Co-authored work absorbed with credit: @kjanat's #38190 (alias handling, co-author on the commit), @charpeni's #31143 and @crystalin's #34407 (both superseded), and the tests of the earlier `bun update` PRs (#31752 by @zlotnika, #33127, #36381, #36729, #38224). robobun's #34688 (folder-dependency cycles; its tests are lifted, co-author on the commit) and #37289 (migrated optionalPeers; its test is lifted, co-author on the commit) were fixed independently here and are closed by this PR. #28422's quadratic scan is fixed here as well (already closed). Related but not closed — `bun prune` gives these a manual fix while the automatic-cleanup asks stay open: #8662, #26305, #29793, #21216, #16176. Also related: #10930, #26970, #26751. Fixes #1343 Fixes #3605 Fixes #14719 Fixes #24122 Fixes #18612 Fixes #20238 Fixes #25826 Fixes #23615 Fixes #26973 Fixes #20593 Closes #31013 Fixes #28959 Fixes #28402 Fixes #27897 Fixes #26675 Fixes #10949 Fixes #18504 Fixes #13388 Fixes #24523 Fixes #6608 Fixes #19059 Fixes #16569 Fixes #8262 Fixes #11901 Fixes #13469 Fixes #25202 Closes #29664 Closes #31143 Closes #34407 Closes #34688 Closes #37289 Closes #38190 --------- Co-authored-by: Kaj Kowalski <info@kajkowalski.nl> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: robobun <117481402+robobun@users.noreply.github.com>
|
Thanks for this PR, @robobun — closing it in favor of #38333, which reworks the whole Your repro: right after migrating, $ bun install # migrates package-lock.json -> bun.lock, no optionalPeers
$ bun install --frozen-lockfile # exits 0 (was: lockfile had changes, but lockfile is frozen)
$ bun install # bun.lock is byte-identical
It'll be in the next canary ( (This comment was written by Claude, on behalf of the Bun team.) |
|
Sounds good, thanks. Confirmed #38333 is merged with the same writer-side lookup (walking up from each entry's own key) and the round-trip test, so this is covered. One note for anyone landing here from the frozen-lockfile symptom: the second shape of this bug (a file: package using optionalDependencies plus peerDependenciesMeta, no foreign lockfile involved) is a different mechanism, a duplicate same-name tree entry from the folder-dependency hoist shortcut, and is tracked in #33156. |
Repro
bun installright after a package-lock.json migration is not a fixed point: the very nextbun install --frozen-lockfileon an unchanged tree fails, and a plain re-install rewrites bun.lock.Any project migrated from package-lock.json with this shape (a peer dependency of a
file:package whose resolution is only placed nested, never hoisted to the root) fails its next--frozen-lockfileCI run.Cause
When saving a lockfile loaded in the binary format (which includes the package-lock.json migration path), the writer checks every non-optional peer dependency for a resolution in the hoisted tree, and lists peers without one in
optionalPeersso the saved file stays parseable. That lookup (PkgMap::find_resolution) walked up from the parent tree node'srelative_pathinstead of from the package entry's own path. For the entry"gamma"it probed"delta"(root) instead of"gamma/delta"first, so a peer placed nested under the package itself was missed and marked optional, even though its resolution was written two lines below:On reload the parser adds
Behavior::OPTIONALto anything inoptionalPeers, and since bbe3f6a (#35681) optional-peer resolution slots are re-derived from the rebuilt tree during cleaning instead of carried over. A nested-only placement is not visible from the dependent's position, so the re-derive leaves the slot empty and the"gamma/delta"entry is dropped: the frozen check fails, and a plain install rewrites the lockfile. Before #35681 the stale optional marking round-tripped unnoticed; #35681 exposed it.Fix
Pass the package entry's own tree path (its
packageskey) towrite_package_info_object, the exact string the parser passes tofind_resolutionwhen binding that entry's dependencies, so writer and parser agree on which placements a peer can see. The new probe set is a superset of the old one (it checks one level deeper first), so a peer can only stop being marked optional, never start.A non-optional peer with genuinely no placement anywhere is still marked
optionalPeers, which keeps such lockfiles loadable, and that state round-trips stably.Verification
New test in
test/cli/install/migration/migrate.test.tscovering the repro above: migration output must not mark the placed peer optional,--frozen-lockfilemust pass on the unchanged tree, and a re-install must leave bun.lock byte-identical. Fails before this change (the lockfile contains"optionalPeers": ["delta"]and the frozen install exits 1), passes after.Also ran
migrate.test.ts,bun-lock.test.ts(including the #35681 optional-peer tests),bun-lockb.test.ts,migrate-bun-lockb-v2.test.ts,pnpm-lock-migration.test.ts,yarn-lock-migration.test.ts,hoist.test.ts,bun-workspaces.test.ts,isolated-install.test.ts,overrides.test.ts,lockfile-only.test.ts, and the peer/optional filters ofbun-install-registry.test.ts: no failures.There is a second, independent way to break the same
install && install --frozen-lockfileinvariant, via the optionalDependencies + peerDependenciesMeta idiom on afile:package (a duplicate same-name tree entry from the folder-dependency hoist shortcut). That one is already fixed by theTree.rsdedupe hunk in #33156, so it is intentionally not duplicated here; details posted on that PR.no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/migration/migrate.test.ts