Skip to content

install: keep the pnpm block in package.json when migrating from pnpm - #38775

Open
robobun wants to merge 5 commits into
mainfrom
farm/45e63abb/pnpm-migration-keep-pnpm-block
Open

install: keep the pnpm block in package.json when migrating from pnpm#38775
robobun wants to merge 5 commits into
mainfrom
farm/45e63abb/pnpm-migration-keep-pnpm-block

Conversation

@robobun

@robobun robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • bun install / bun pm migrate on a project with a pnpm-lock.yaml copies pnpm.overrides and pnpm.patchedDependencies to the root of package.json and then removes them from the "pnpm" block (update_package_json_after_migration, src/install/pnpm.rs; the block itself is removed when nothing else is in it). Since install: pnpm parity — dedupe, prune, pm licenses, audit fix, add --filter/--catalog, nested overrides, transitive update, and workspace fixes #38333 this is announced as moved pnpm.overrides to overrides in package.json; the removal itself dates back to the original migration (implement pnpm migration #22262).
  • pnpm reads its configuration only from the "pnpm" block and ignores the root-level fields. After one bun install is committed, pnpm install --frozen-lockfile fails for everyone still on pnpm with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, and a plain pnpm install drops the overrides: section from pnpm-lock.yaml, so the pins stop applying.
  • bun never reads the "pnpm" block (OverrideMap and Package only read the root-level overrides / resolutions / patchedDependencies; bun-workspaces.test.ts pins that root pnpm.overrides is ignored), so leaving the block in place costs nothing.
  • Found while testing the fix: bun update run directly on a pnpm project read freed memory (ASAN heap-use-after-free in E::String::eql_bytes, reached from PackageJSONEditor::edit_update_entries). The migration replaced source.contents of the cached root package.json entry but kept the edited tree, whose strings point into the buffer that was just freed (and whose new nodes live in the Store, which the next initialize_store() resets). Any bun update that triggers the migration hits this whenever the migration rewrites package.json, which includes every pnpm workspace (the pnpm-workspace.yaml packages list is written into workspaces).
  • This is the crash in first time running bun in a pnpm monorepo: bun update --interactive #23694 (bun update --interactive as the first bun command in a pnpm monorepo, segfault in printJSON under updatePackageJsonAfterMigration): update -i loads the lockfile once to list outdated packages (first migration, package.json rewritten, cached tree left dangling), edits the root package.json through that cached tree, then installs, which migrates again because bun.lock is still not on disk and prints the dangling tree. Reproduced with the debug build: ASAN reports the use-after-free at the editing step (update_package_json_files_from_updates), the release build gets as far as the second migration's print.

Fix

  • pnpm.overrides / pnpm.patchedDependencies are copied and the "pnpm" block is left as it was. The block-pruning code is deleted.
  • copy_object gives the root a separate object, so rewriting bare patch keys to name@version (rewrite_bare_patch_keys) and merging in pnpm-workspace.yaml entries afterwards cannot show up inside the "pnpm" block (the Expr returned by get aliases the same object).
  • The four identical merge-or-create blocks (package.json and pnpm-workspace.yaml, overrides and patchedDependencies) now share copy_into_root. The message reads copied ... in package.json; nothing it lists is modified at its source.
  • The rewritten package.json is produced with print_package_json_into_cache_entry followed by reparse_root, the same sequence bun add / bun remove / bun add --catalog use, so the cache entry's tree matches the contents bun just wrote. This is what fixes the use-after-free, and it is also needed for the copies above, which Expr::init places in the resettable Store. install: import pnpm-workspace.yaml even without a migratable pnpm-lock.yaml #38754 (open: import pnpm-workspace.yaml without a lockfile) adds the same write-back to this function independently and keeps the move semantics; the two PRs overlap only in that tail. Whichever lands second needs a small rebase, and install: import pnpm-workspace.yaml even without a migratable pnpm-lock.yaml #38754's new moved ... test strings become copied ... once this is in.
  • Same class, same function (raised in review): pnpm-workspace.yaml was parsed into an arena local to the block that read the file, while the catalog / catalogs / overrides / patchedDependencies objects taken from it are printed after that block ends. Quoted yaml scalars ('@scope/pkg': '^1.0.0', the usual spelling for scoped names) are copied into that arena (yaml.rs, NodeScalar::to_expr), so they were read after the arena was destroyed; unquoted ones point into the file contents and were fine. The yaml is now parsed into the function's bump, which outlives the print. install: import pnpm-workspace.yaml even without a migratable pnpm-lock.yaml #38754 carries the same one-line fix. ASAN does not flag this one reliably (mimalloc keeps the destroyed heap's pages mapped), so the test for it pins the output rather than proving fail-before.
  • Docs: the migration section of docs/pm/cli/install.mdx now says the fields are copied and the "pnpm" block is kept.
  • Verified with test/cli/install/migration/pnpm-lock-v9.test.ts:
    • the pnpm block in package.json is left as it was (block with other settings, and block containing only the migrated keys) and pnpm-workspace.yaml overrides go into the root copy, not into the pnpm block: fail on main (block removed), pass with this change.
    • bare hash whose path is only in package.json pnpm.patchedDependencies now also checks the bare key survives in the block while the root gets no-deps@1.0.1.
    • bun update straight from pnpm-lock.yaml edits the package.json the migration rewrote: on main the debug build exits 1 with the ASAN report above; passes with this change.
    • bun update -i in a pnpm workspace migrates twice and keeps package.json intact (the first time running bun in a pnpm monorepo: bun update --interactive #23694 shape, driven through piped stdin like the tests in bun-update-transitive.test.ts): on main the debug build exits 1 with a use-after-free in update_package_json_files_from_updates; passes with this change. update -i still runs the migration twice, so the copied ... line prints twice there; the second pass merges identical values and leaves the file as the first pass wrote it.
    • quoted pnpm-workspace.yaml scalars are written to package.json: quoted keys and values in the yaml overrides and catalog end up in package.json (passes both ways, see above).
    • Existing assertions in this file and in test/cli/install/nested-overrides.test.ts that encoded the removal were updated.
  • Also ran: pnpm-lock-v9.test.ts (82 pass), nested-overrides.test.ts (144 pass), pnpm-lock-migration, migrate, lockfile-only, pnpm-migration, pnpm-comprehensive, pnpm-migration-complete (all pass; the last one is a single 15-spawn test that sits close to the 5 s default timeout on a loaded machine with a debug build, unrelated to this change).

Background

  • pnpm 9 keeps overrides and patched dependencies under the "pnpm" key of package.json; pnpm 10 also accepts them in pnpm-workspace.yaml, and the lockfile records the merged set. bun uses npm's root-level overrides and its own root-level patchedDependencies, which is why the migration writes root-level copies. The migration leaves pnpm-lock.yaml and pnpm-workspace.yaml untouched; the "pnpm" block was the one place it edited pnpm's own configuration.
  • WorkspacePackageJSONCache holds, per package.json, the file contents (source) and a parsed tree (root) whose string nodes borrow from those contents. install_with_manager re-parses the root from source after the lockfile is loaded, but bun update's editor and a few diagnostics use the cached root directly, so a writer that changes source has to rebuild root (reparse_root) or keep the old buffer alive (stale_contents); print_package_json_into_cache_entry does the latter and callers follow it with the former.
  • Expr::init allocates AST nodes in a thread-local Store that initialize_store() resets (poisoned in debug builds); nodes that have to outlive that are allocated in an arena (Expr::allocate) or, as here, the tree is re-parsed from the printed text.
  • A bare patchedDependencies key ("no-deps") is legal for pnpm; bun.lock keys patches by name@version, so the migration rewrites bare keys in the root copy using the version the lockfile resolved.
Reproduction on bun 1.4.0 (release)
package.json: {"name":"r","private":true,"dependencies":{"a":"1.0.0"},
               "pnpm":{"overrides":{"b":"1.0.0"},"onlyBuiltDependencies":["a"]}}
pnpm-lock.yaml: lockfileVersion: '9.0' / overrides: {b: 1.0.0} / importers: {.: {}}

$ bun pm migrate
$ cat package.json
{
  ...
  "pnpm": {
    "onlyBuiltDependencies": ["a"]      <- overrides removed from the pnpm block
  },
  "overrides": { "b": "1.0.0" }         <- added at the root, which pnpm does not read
}

With this change the pnpm block is unchanged and the root-level overrides is added; the line printed is copied pnpm.overrides to overrides in package.json.

ASAN report from the new bun update test on the unfixed code
==23017==ERROR: AddressSanitizer: heap-use-after-free
READ of size 8
    #4 bun_core::string::immutable::eql_long
    #5 <bun_ast::e::EString>::eql_bytes
    #6 <bun_ast::e::Object>::as_property
    #8 bun_install::...::package_json_editor::edit_update_entries  PackageJSONEditor.rs:463
    #11 bun_install::...::package_json_write_back::edit_cwd
    #14 bun_install::...::install_with_manager
freed by thread T0 here:
    #13 core::ptr::drop_glue::<alloc::borrow::Cow<[u8]>>
    #14 bun_install::pnpm::update_package_json_after_migration   (the `source.contents = ...` assignment)
    #15 bun_install::pnpm::migrate_pnpm_lockfile

Fixes #23694

The pnpm-lock.yaml migration copied pnpm.overrides and
pnpm.patchedDependencies to the root of package.json and then deleted
them from the pnpm block (deleting the block itself when nothing else
was in it). pnpm only reads its configuration from that block, so after
one bun install, pnpm install --frozen-lockfile failed for everyone
still on pnpm with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, and a plain pnpm
install dropped the overrides from pnpm-lock.yaml.

The entries are now copied and the block is left as it was. The copy
written to the root is a separate object, so rewriting bare patch keys
to name@version and merging in pnpm-workspace.yaml entries no longer
shows up inside the block. The four merge-into-root blocks share one
helper, and the message says "copied" instead of "moved".

The rewritten package.json now goes through
print_package_json_into_cache_entry and reparse_root like the other
package.json editors. Before, the cache entry kept the edited tree,
which pointed into the contents buffer that had just been freed and
into Store nodes that the next initialize_store() resets, so bun update
run directly on a pnpm project read freed memory when it edited the
root package.json after the migration.
@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: b4891181-7ed0-4d58-92ce-c7f9c12d6e05

📥 Commits

Reviewing files that changed from the base of the PR and between 4bf3f36 and d40cca5.

📒 Files selected for processing (4)
  • docs/pm/cli/install.mdx
  • src/install/pnpm.rs
  • test/cli/install/migration/pnpm-lock-v9.test.ts
  • test/cli/install/nested-overrides.test.ts

Walkthrough

Changes

The pnpm migration now copies overrides and patchedDependencies to root-level package.json fields while preserving the pnpm block. Tests cover direct, interactive, workspace, repeated, and frozen migrations. Documentation describes the updated behavior and cleanup steps.

pnpm migration

Layer / File(s) Summary
Copy and merge migration configuration
src/install/pnpm.rs
The migration copies and merges pnpm overrides and patched dependencies, preserves the original pnpm fields, updates diagnostics, and reparses cached package.json content.
Validate copied configuration and repeated migrations
test/cli/install/migration/pnpm-lock-v9.test.ts, test/cli/install/nested-overrides.test.ts
Tests cover direct and interactive updates, workspaces, frozen installs, repeated migration, preserved pnpm fields, copied diagnostics, and nested override cases.
Document retained pnpm configuration
docs/pm/cli/install.mdx
The migration documentation describes copied root-level fields, retained pnpm configuration, and cleanup of pnpm files and fields.

Possibly related PRs

  • oven-sh/bun#38333: Both PRs modify pnpm migration handling and related migration tests.
  • oven-sh/bun#38754: Both PRs modify pnpm migration logic and package.json cache handling.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main migration change: preserving the pnpm block in package.json.
Description check ✅ Passed The description explains the problem, fix, background, and verification in detail, although it uses different headings from the template.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 11:59 PM PT - Aug 14th, 2026

@robobun, your commit cb7af7477b7de75f7f629998dff2a034dbcee1de passed in Build #97096! 🎉


🧪   To try this PR locally:

bunx bun-pr 38775

That installs a local version of the PR into your bun-38775 executable, so you can run:

bun-38775 --bun

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on bun 1.4.0 with bun pm migrate on a package.json whose pnpm block holds overrides (the block came back without them), and the bun update / bun update -i use-after-free with the debug build (ASAN report in the description; the -i shape is #23694). Fix and tests are on this branch (latest d40cca5); waiting on CI.

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. first time running bun in a pnpm monorepo: bun update --interactive #23694 - Segfault whose stack goes straight through updatePackageJsonAfterMigrationjs_printer.printJSON on first bun update --interactive in a pnpm monorepo, which is the stale-AST/freed-source.contents use-after-free this PR replaces with print_package_json_into_cache_entry + reparse_root.

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #23694

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it changes user-facing bun pm migrate behavior (leaving the pnpm block in place instead of pruning it) and reworks the package.json cache-entry write path to fix a use-after-free, a human look would still be worthwhile.

Checked: copy_object gives the root a distinct property list so rewrite_bare_patch_keys and the workspace-yaml merge cannot mutate the pnpm block; copy_into_root preserves the old merge-vs-create semantics and now correctly skips announcing when the existing root field is not an object; the print_package_json_into_cache_entry + reparse_root sequence matches bun add/bun remove/add_catalog; wrote_workspaces still gates the write via copied.

Extended reasoning...

Overview

This PR changes update_package_json_after_migration in src/install/pnpm.rs so that migrating from pnpm copies pnpm.overrides / pnpm.patchedDependencies to root-level fields without removing them from the "pnpm" block (previously the block was pruned and deleted when empty). It also replaces the hand-rolled BufferPrinter + source.contents = Cow::Owned(...) sequence — which freed the old contents while the cached root tree still borrowed them — with the shared print_package_json_into_cache_entry + reparse_root pattern. Four near-identical merge-or-create blocks are collapsed into copy_into_root, and copy_object is added so edits to the root copy cannot alias back into the pnpm block. Docs and two test files are updated; three new tests cover the preserved block, the yaml-merge aliasing case, and the bun update UAF.

Security risks

None identified. No untrusted-input parsing paths change; the modified code operates on the already-parsed root package.json and pnpm-workspace.yaml trees.

Level of scrutiny

Medium-high. The functional half is a deliberate product decision (non-destructive migration so pnpm install --frozen-lockfile keeps working for teammates still on pnpm) — reasonable and well-argued, but a user-visible behavior change to bun install that a maintainer should sign off on. The memory-safety half touches AST-node lifetime rules (Store reset, source.contents borrowing, stale_contents retention) and, while the fix reuses the exact pattern from add_remove_with_filter.rs / add_catalog.rs / updatePackageJSONAndInstall.rs, it is the kind of change the repo review guide flags for careful reading.

Other factors

I traced shallow_clone_prop and confirmed copy_object builds a fresh PropertyList, so rewrite_bare_patch_keys reassigning prop.key on the copy cannot touch the original block's list. copy_into_root's Ok(false) branch (existing root field is not an object) is a slight behavior change from the old code — previously the message was printed and the pnpm block was still pruned even though nothing was merged; now nothing is announced or written. That is an improvement in a degenerate case, not a regression. The old silent return Ok(()) on print_json failure becomes a Global::crash() via the shared helper, matching every other caller. No dead imports remain (bun_js_printer is no longer referenced in pnpm.rs). Test coverage looks thorough, including the ASAN repro for bun update and a check that the bare patch key survives in the block while the root gets the versioned key.

Comment thread src/install/pnpm.rs Outdated
Comment thread src/install/pnpm.rs Outdated
Comment thread src/install/pnpm.rs Outdated
Comment thread src/install/pnpm.rs Outdated
Comment thread src/install/pnpm.rs Outdated
Comment thread src/install/pnpm.rs Outdated
@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Confirmed #23694 is this use-after-free: bun update -i loads the lockfile once to list the outdated packages (first migration rewrites package.json and leaves the cached tree pointing at the freed contents), edits the root package.json through that tree, then installs, which migrates again because bun.lock is still not on disk. With main's pnpm.rs the debug build reports the use-after-free at the editing step; the issue's release build got one step further and crashed printing the tree in the second migration, which is the stack in the report. Added bun update -i in a pnpm workspace migrates twice and keeps package.json intact to pnpm-lock-v9.test.ts (fails on main, passes here) and Fixes #23694 to the description. The comments flagged above are down to one line each.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Independent reproduction of the same two problems landed on the same fix, so no second PR: branch farm/a6cc97c2/pnpm-migration-keep-pnpm-config (1719e7a) copies instead of removing, builds the root objects separately, and switches the write-out to print_package_json_into_cache_entry + reparse_root; the bun update use-after-free reproduced there the same way (EString::eql_bytes under edit_update_entries).

Two notes for whoever merges:

  • install: import pnpm-workspace.yaml even without a migratable pnpm-lock.yaml #38754 rewrites the same function (update_package_json_after_migration) and also adds the print + reparse step, so whichever of the two lands second needs a rebase.
  • One thing this PR leaves as is and install: import pnpm-workspace.yaml even without a migratable pnpm-lock.yaml #38754 fixes: the pnpm-workspace.yaml tree is parsed into an Arena that is dropped at the end of the read block, but quoted yaml scalars ('packages/*', react: '^18.0.0') are copied into that arena (NodeScalar::to_expr, YamlString::List), and the catalog / overrides / patchedDependencies objects taken from the tree are printed after the block ends. Hoisting the arena to function scope is enough (that branch does it); the pnpm.overrides are copied, not removed test there uses a quoted yaml value to cover the path.

Comment thread src/install/pnpm.rs
…ackage.json print

Quoted yaml scalars are copied into the arena the yaml is parsed with.
That arena was local to the block that read the file, while the
catalog, overrides and patchedDependencies objects taken from the tree
are printed into package.json after the block ends.
@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Re the note above: the block-scoped yaml arena is fixed here too as of cb7af74 (the yaml is parsed into the function's bump), with quoted pnpm-workspace.yaml scalars are written to package.json covering quoted keys and values. Description updated; the only remaining overlap with #38754 is the tail of update_package_json_after_migration.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The arena-lifetime issue I flagged is fixed in cb7af74 (the yaml parse now uses the function-scope bump, and quoted pnpm-workspace.yaml scalars are written to package.json covers it). No further issues found this pass. Given this reworks arena/Store lifetimes in update_package_json_after_migration, changes the migration's user-visible contract for the pnpm block, and overlaps with #38754, a human look is still worthwhile.

What was reviewed:

  • &bump now backs the yaml parse; bump at pnpm.rs:2348 outlives print_package_json_into_cache_entry, so quoted-scalar Exprs stay valid through the print.
  • copy_object builds a fresh E::Object so rewrite_bare_patch_keys and workspace-yaml merges cannot mutate the retained pnpm block via the aliased get result.
  • copy_into_root preserves the old merge-vs-create semantics, including returning false (no "copied" line) when a non-object root field already exists.
  • print_package_json_into_cache_entry stashes the old contents in stale_contents and reparse_root follows immediately, matching the pattern bun add/--catalog use.
Extended reasoning...

Overview

The PR touches src/install/pnpm.rs (update_package_json_after_migration and two new helpers), docs/pm/cli/install.mdx, and two test files. It (1) stops deleting pnpm.overrides/pnpm.patchedDependencies from the "pnpm" block during migration and copies them to root-level fields instead, (2) fixes a use-after-free where the migration replaced source.contents on the cached root package.json entry without rebuilding root (crashes bun update/bun update -i on first run in a pnpm project — #23694), (3) fixes a second UAF I flagged last pass where pnpm-workspace.yaml was parsed into a block-scoped arena whose quoted-scalar allocations were read after drop, and (4) collapses four copy-pasted merge blocks into copy_into_root plus a copy_object that severs the alias between the root copy and the pnpm block.

Security risks

None identified. No untrusted-input parsing was added; the yaml/json inputs were already being parsed on this path. The change reduces memory-safety risk by replacing a hand-rolled source.contents = Cow::Owned(...) (which left entry.root dangling) with the shared print_package_json_into_cache_entry + reparse_root sequence, and by extending the yaml arena's lifetime to cover its consumers.

Level of scrutiny

Medium-high. This is package-manager code on the migration path with two memory-lifetime fixes whose correctness depends on bun_alloc::Arena scope, the resettable Expr Store, and WorkspacePackageJSONCache invariants (stale_contents, reparse_root). The behavior change (keep the pnpm block) is a product decision — well-argued in the description, but still a user-visible contract change. The refactor is a net simplification (~130 lines deleted) and matches an in-tree helper pattern, but the aliasing subtlety that motivates copy_object is exactly the kind of thing a maintainer should sign off on.

Other factors

My previous inline finding was addressed in cb7af74 with a dedicated test using single- and double-quoted yaml scalars. Test coverage is thorough: new tests for both UAF repros (including the #23694 update -i shape driven via piped stdin), the block-kept invariant in both the mixed-keys and only-migrated-keys cases, the workspace-yaml-merges-into-root-not-pnpm-block case, and existing movedcopied assertions updated across two files. The comment-cop bot flags were resolved (comments are now single-line). The PR description notes a rebase will be needed against #38754 whichever lands second. CI build #97096 is in progress per the robobun comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

first time running bun in a pnpm monorepo: bun update --interactive

2 participants