diff --git a/packages/cli/src/ai-context/references/configure-playwright-checks.md b/packages/cli/src/ai-context/references/configure-playwright-checks.md index 515f0dd5..436d1602 100644 --- a/packages/cli/src/ai-context/references/configure-playwright-checks.md +++ b/packages/cli/src/ai-context/references/configure-playwright-checks.md @@ -16,7 +16,7 @@ - For pnpm projects, the workspace-root pnpmfile (`.pnpmfile.cjs` / `.pnpmfile.mjs`) is bundled automatically so the remote install can reproduce the lockfile — but only when the lockfile records a pnpmfile checksum (run `pnpm install` with the pnpmfile in place to record one; without it the pnpmfile is silently not bundled). The pnpmfile must also be self-contained: it may only load side-effect-free Node.js builtins (such as `path`, `crypto`, `util`) via literal `require('...')`/`import` specifiers, and must not reference `process`, `__dirname`, `import.meta`, or dynamically computed module paths. A pnpmfile that is not self-contained is skipped with a warning, and the remote install may re-resolve dependencies instead of using the lockfile. The pnpmfile is uploaded, so avoid embedding secrets in it. - In a workspace (monorepo) whose code bundle covers only part of the workspace, the bundled lockfile is pruned automatically so the remote install only fetches what the bundle needs, instead of failing on dependencies of workspace members that were left out. - Supported for `pnpm-lock.yaml` (v6/9), `package-lock.json` (v2/3), the text `bun.lock`, and Yarn Berry `yarn.lock`. Yarn Classic v1 lockfiles are not supported; for bun's binary `bun.lockb`, regenerate a text lockfile with `bun install --save-text-lockfile`. - - The workspace's package manager binary must be available on the machine running the CLI. For pnpm and npm, pruning reuses cached registry metadata and only contacts the registry for packages missing from the cache; yarn Berry and bun regenerate offline. + - The workspace's package manager binary must be available on the machine running the CLI. pnpm resolves against the workspace's own store plus cached registry metadata, npm reuses its cached metadata, and both only contact the registry for misses; yarn Berry and bun regenerate offline. - When pruning was needed but cannot run or cannot verify its result, the original lockfile ships unchanged and the CLI prints a note explaining why — follow it (a timed-out prune's note names `CHECKLY_LOCKFILE_PRUNE_TIMEOUT=`, which raises the default 30 s budget). Skips are silent when there is nothing to prune, or when pruning is disabled via `CHECKLY_LOCKFILE_PRUNE=0`; silent skip reasons, a timed-out prune's partial package manager output, and per-entry embed match details are visible under `DEBUG='checkly:cli:services:check-parser:*'` and `DEBUG='checkly:cli:services:embedded-packages'`. - For pnpm projects with `patchedDependencies`, patches belonging to unbundled workspace members are filtered out automatically. A note naming stale patch declarations means the lockfile is out of date with the config — refresh it with a regular install. - Checkly caches installed dependencies between runs, keyed off the workspace's dependency inputs (the lock file, every workspace member's `package.json` and `.npmrc` — whether or not the member is in the bundle) plus the bundle's own install inputs, so the key can change without a file edit, e.g. when a different set of workspace members lands in the bundle. To force a reinstall for deployed checks, set `runner.cache.install.version` (a string or a safe integer) at the top level of `checkly.config.ts` — not per check — and change its value whenever the cache should be invalidated; deployed, scheduled checks pick up the change on the next `checkly deploy`. If the config still declares the deprecated `caching.dependencyCache.version`, remove it — setting both fails config loading. Unset or empty values leave the cache key unchanged, so a dynamic value such as `version: process.env.DEPENDENCY_CACHE_VERSION` is safe when the variable is not always set. For a one-off reinstall during an ad-hoc run, use the `--refresh-cache` flag on `checkly test`, `checkly pw-test`, `checkly trigger`, or `checkly checks run` instead. diff --git a/packages/cli/src/services/check-parser/__tests__/bundler.spec.ts b/packages/cli/src/services/check-parser/__tests__/bundler.spec.ts index 1f146771..47eb5722 100644 --- a/packages/cli/src/services/check-parser/__tests__/bundler.spec.ts +++ b/packages/cli/src/services/check-parser/__tests__/bundler.spec.ts @@ -527,6 +527,7 @@ describe('Bundler.finalize() embedded package materialization', () => { ].join('\n')) return Object.assign(Object.create(new PNpmDetector()), { lockfileOnlyInstallCommand: () => new Runnable('node', [scriptPath]), + storeDirCommand: () => undefined, }) } @@ -833,6 +834,7 @@ describe('Bundler.finalize() patch filtering', () => { ].join('\n')) return Object.assign(Object.create(new PNpmDetector()), { lockfileOnlyInstallCommand: () => new Runnable('node', [scriptPath]), + storeDirCommand: () => undefined, }) } @@ -1231,6 +1233,7 @@ describe('Bundler.finalize() package pruning', () => { await fs.writeFile(scriptPath, lines.join('\n')) return Object.assign(Object.create(new PNpmDetector()), { lockfileOnlyInstallCommand: () => new Runnable('node', [scriptPath]), + storeDirCommand: () => undefined, }) } diff --git a/packages/cli/src/services/check-parser/__tests__/lockfile-pruner.spec.ts b/packages/cli/src/services/check-parser/__tests__/lockfile-pruner.spec.ts index 8c622712..617307f6 100644 --- a/packages/cli/src/services/check-parser/__tests__/lockfile-pruner.spec.ts +++ b/packages/cli/src/services/check-parser/__tests__/lockfile-pruner.spec.ts @@ -1,4 +1,5 @@ import { spawnSync } from 'node:child_process' +import { rmSync } from 'node:fs' import fs from 'node:fs/promises' import os from 'node:os' import path from 'node:path' @@ -15,11 +16,20 @@ import { } from '../lockfile-pruner.js' import { createFauxPackageFiles } from '../faux-package.js' import { UNPRINTABLE_URL } from '../../embedded-packages/diagnostics.js' -import { BunDetector, NpmDetector, PackageManager, PNpmDetector, Runnable, YarnDetector } from '../package-files/package-manager.js' +import { + BunDetector, + LockfileOnlyInstallOptions, + NpmDetector, + PackageManager, + PNpmDetector, + Runnable, + YarnDetector, +} from '../package-files/package-manager.js' import { Package, Workspace } from '../package-files/workspace.js' import { Err, Ok } from '../package-files/result.js' import { File } from '../parser.js' import { captureStderr } from '../../../testing/capture-stderr.js' +import { shellQuote } from '../../shell.js' const PNPM_FIXTURE_ROOT = path.join(__dirname, 'lockfile-pruner-fixtures', 'pnpm-workspace') // Same shape as PNPM_FIXTURE_ROOT, plus two patched dependencies: `ms` is @@ -97,11 +107,33 @@ const testEnv = (): NodeJS.ProcessEnv => ({ }) // A PackageManager whose lockfile-only install command is replaced, for -// exercising failure paths without a real package manager. +// exercising failure paths without a real package manager. The store +// lookup is disabled too, so no test on this stub spawns a real pnpm. const stubPackageManager = (runnable: Runnable | undefined): PackageManager => { return Object.assign(Object.create(new PNpmDetector()), { lockfileOnlyInstallCommand: () => runnable, + storeDirCommand: () => undefined, + }) +} + +// Like stubPackageManager, but with a store lookup command of its own, and +// recording the options the install command was requested with so a test +// can see what the lookup fed into it. A requested store is appended to +// the install's arguments (a trailing argument node ignores), so the +// command the pruner runs and reports is distinguishable from the +// unpinned one. +const stubPackageManagerWithStoreProbe = (install: Runnable, probe: Runnable) => { + const installOptions: LockfileOnlyInstallOptions[] = [] + const packageManager: PackageManager = Object.assign(Object.create(new PNpmDetector()), { + lockfileOnlyInstallCommand: (options: LockfileOnlyInstallOptions = {}) => { + installOptions.push(options) + return options.storeDir === undefined + ? install + : new Runnable(install.executable, [...install.args, `store=${options.storeDir}`]) + }, + storeDirCommand: () => probe, }) + return { packageManager, installOptions } } describe('lockfile-pruner', () => { @@ -368,6 +400,282 @@ describe('lockfile-pruner', () => { }) }) + describe('store directory lookup', () => { + // A lookup that prints like `pnpm store path` does: the VERSIONED + // store directory, as the last line. + const printingProbe = (...lines: string[]) => new Runnable('node', [ + '-e', `process.stdout.write(${JSON.stringify(lines.join('\n') + '\n')})`, + ]) + const noopInstall = () => new Runnable('node', ['-e', '']) + const versionedStorePath = async () => path.join(await makeTempDir(), 'store', 'v10') + + it('pins the install to the parent of the printed versioned store path', async () => { + const { workspace, files } = makePnpmScenario() + const storePath = await versionedStorePath() + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe( + noopInstall(), printingProbe(storePath), + ) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + expect(result.status).not.toEqual('failed') + // The capability check asks without options; the install itself is + // requested with the store. The parent is passed so that an install + // whose pnpm major differs from the lookup's still lands in the + // workspace's store (pnpm appends its own version segment). + expect(installOptions).toEqual([{}, { storeDir: path.dirname(storePath) }]) + }) + + it('takes the last non-empty line, so a warning printed ahead of the path is ignored', async () => { + const { workspace, files } = makePnpmScenario() + const storePath = await versionedStorePath() + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe( + noopInstall(), + printingProbe('WARN the "pnpm" field in package.json is no longer read', storePath, ''), + ) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + expect(result.status).not.toEqual('failed') + expect(installOptions.at(-1)).toEqual({ storeDir: path.dirname(storePath) }) + }) + + it.each([ + ['a relative path', path.join('store', 'v10')], + ['an absolute path without a version segment', path.resolve(os.tmpdir(), 'store')], + ['nothing', ''], + ])('skips notably when the lookup prints %s', async (_, printed) => { + const { workspace, files } = makePnpmScenario() + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe( + noopInstall(), printingProbe(printed), + ) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + expect(result).toMatchObject({ + status: 'skipped', + reason: expect.stringContaining('store directory could not be determined'), + notable: true, + }) + // Only the capability check ran; nothing was installed. + expect(installOptions).toEqual([{}]) + }) + + it('skips notably when the lookup fails, quoting its output', async () => { + const { workspace, files } = makePnpmScenario() + const probe = new Runnable('node', ['-e', `process.stderr.write('ERR_PNPM_BOOM\\n'); process.exit(1)`]) + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe(noopInstall(), probe) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + expect(result).toMatchObject({ + status: 'skipped', + reason: expect.stringContaining('store directory could not be determined'), + notable: true, + }) + expect(result.reason).toContain('failed: ERR_PNPM_BOOM') + expect(installOptions).toEqual([{}]) + }) + + it('skips notably when the lookup executable does not exist', async () => { + const { workspace, files } = makePnpmScenario() + const { packageManager } = stubPackageManagerWithStoreProbe( + noopInstall(), + new Runnable('checkly-no-such-executable-xyz', ['store', 'path']), + ) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + expect(result).toMatchObject({ + status: 'skipped', + reason: expect.stringContaining('checkly-no-such-executable-xyz is not installed'), + notable: true, + }) + }) + + it('fails, not skips, when the workspace directory is gone by the time the lookup runs', async () => { + // execa reports an invalid working directory as ENOENT too, which + // must not be mistaken for a missing executable. storeDirCommand() + // is consulted after the bundle is materialized and right before + // the lookup spawns, so removing the root there hits exactly that + // window. + const root = await makeTempDir() + await fs.cp(PNPM_FIXTURE_ROOT, root, { recursive: true }) + const { workspace, files } = makePnpmScenario(root) + const packageManager: PackageManager = Object.assign(Object.create(new PNpmDetector()), { + lockfileOnlyInstallCommand: () => noopInstall(), + storeDirCommand: () => { + rmSync(root, { recursive: true, force: true }) + return new Runnable('checkly-no-such-executable-xyz', ['store', 'path']) + }, + }) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + expect(result).toMatchObject({ + status: 'failed', + reason: `the workspace directory '${root}' disappeared`, + }) + }) + + it('redacts the lookup output it logs at debug level', async () => { + const { workspace, files } = makePnpmScenario() + // A misconfigured npmrc makes pnpm echo registry URLs, credentials + // included, when the lookup fails; the debug log must scrub them + // like the timed-out path does. + const probe = new Runnable('node', ['-e', ` + process.stderr.write('ERR_PNPM_REGISTRY ' + process.env.PRUNE_TEST_URL + '\\n') + process.exit(1) + `]) + const { packageManager } = stubPackageManagerWithStoreProbe(noopInstall(), probe) + const previouslyEnabled = Debug.disable() + Debug.enable('checkly:cli:services:check-parser:lockfile-pruner') + try { + let result: PruneBundledLockfileResult | undefined + const written = await captureStderr(async () => { + result = await pruneBundledLockfile({ + workspace, + packageManager, + files, + env: { ...testEnv(), PRUNE_TEST_URL: 'https://user:secret@registry.example/pkg?token=abc' }, + }) + }) + expect(result).toMatchObject({ status: 'skipped', notable: true }) + expect(result?.reason).not.toContain('secret') + const debugOutput = written.join('') + expect(debugOutput).toContain('ERR_PNPM_REGISTRY https://registry.example') + expect(debugOutput).not.toContain('secret') + expect(debugOutput).not.toContain('token=abc') + } finally { + Debug.enable(previouslyEnabled) + } + }) + + it('fails when the lookup times out', async () => { + const { workspace, files } = makePnpmScenario() + const probe = new Runnable('node', ['-e', 'setInterval(() => {}, 1000)']) + const { packageManager } = stubPackageManagerWithStoreProbe(noopInstall(), probe) + const result = await pruneBundledLockfile({ + workspace, packageManager, files, timeoutMs: 1_000, env: testEnv(), + }) + expect(result).toMatchObject({ + status: 'failed', + reason: expect.stringContaining('timed out after 1s; set CHECKLY_LOCKFILE_PRUNE_TIMEOUT= to raise it'), + }) + }, 30_000) + + it('fails when the lookup uses up the time budget the install needs', async () => { + const { workspace, files } = makePnpmScenario() + const storePath = await versionedStorePath() + // Answers correctly, but only after most of the budget is gone. The + // delay sits well inside the budget so that node's own startup + // latency on a loaded host cannot turn this into a probe timeout. + const probe = new Runnable('node', ['-e', ` + setTimeout(() => process.stdout.write(${JSON.stringify(storePath + '\n')}), 2_200) + `]) + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe(noopInstall(), probe) + const result = await pruneBundledLockfile({ + workspace, packageManager, files, timeoutMs: 3_000, env: testEnv(), + }) + expect(result).toMatchObject({ + status: 'failed', + reason: expect.stringContaining('determining the pnpm store directory used up the prune time budget'), + }) + expect(installOptions).toEqual([{}]) + }, 30_000) + + it('runs the lookup and the install without a timeout when the budget is unlimited', async () => { + const { workspace, files } = makePnpmScenario() + const storePath = await versionedStorePath() + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe( + noopInstall(), printingProbe(storePath), + ) + // timeoutMs 0 is "no timeout"; it must reach the install rather + // than be clamped to a 1 ms budget for either spawn. + const result = await pruneBundledLockfile({ workspace, packageManager, files, timeoutMs: 0, env: testEnv() }) + expect(result.status).not.toEqual('failed') + expect(installOptions).toEqual([{}, { storeDir: path.dirname(storePath) }]) + }) + + it('names the store-pinned install command when the install fails', async () => { + const { workspace, files } = makePnpmScenario() + const storePath = await versionedStorePath() + const failingInstall = new Runnable('node', ['-e', 'process.exit(2)']) + const { packageManager } = stubPackageManagerWithStoreProbe(failingInstall, printingProbe(storePath)) + const result = await pruneBundledLockfile({ workspace, packageManager, files, env: testEnv() }) + // The reason must come from the command returned by the post-lookup + // call (carrying the store), not from the capability check's. + expect(result).toMatchObject({ + status: 'failed', + // Quoted the way the pruner renders the command: a Windows path + // carries backslashes, which the shell quoting wraps in quotes. + reason: expect.stringContaining(`${shellQuote(`store=${path.dirname(storePath)}`)} failed:`), + }) + }) + + it('rejects a budget below the install floor before running the lookup', async () => { + const { workspace, files } = makePnpmScenario() + const { packageManager, installOptions } = stubPackageManagerWithStoreProbe( + noopInstall(), + printingProbe(await versionedStorePath()), + ) + const result = await pruneBundledLockfile({ workspace, packageManager, files, timeoutMs: 500, env: testEnv() }) + expect(result).toMatchObject({ + status: 'failed', + reason: expect.stringContaining('below the minimum needed to run pnpm'), + }) + expect(installOptions).toEqual([{}]) + }) + + // The real-pnpm pair below plants a storeDir in both places pnpm could + // read one from: the workspace's own pnpm-workspace.yaml (what the + // lookup, run in the workspace root, must honor) and the materialized + // copy in the prune temp dir (what an unpinned install would honor). + // pnpm creates its versioned store directory on every install, + // lockfile-only included, so which of the two directories appears + // tells which store the install used. The prune's own outcome is not + // asserted: an empty store legitimately falls back on a machine whose + // metadata cache is stale. + const plantStoreDirDecoys = async () => { + const root = await makeTempDir() + await fs.cp(PNPM_FIXTURE_ROOT, root, { recursive: true }) + const workspaceStore = path.join(await makeTempDir(), 'workspace-store') + const decoyStore = path.join(await makeTempDir(), 'decoy-store') + const workspaceYamlPath = path.join(root, 'pnpm-workspace.yaml') + const workspaceYaml = await fs.readFile(workspaceYamlPath, 'utf8') + await fs.writeFile(workspaceYamlPath, `${workspaceYaml}storeDir: ${workspaceStore}\n`) + const { workspace, files } = makePnpmScenario(root) + files.set('pnpm-workspace.yaml', { + filePath: workspaceYamlPath, + physical: false, + content: `${workspaceYaml}storeDir: ${decoyStore}\n`, + }) + return { workspace, files, workspaceStore, decoyStore } + } + const versionedStoreCreatedIn = async (store: string) => { + const entries = await fs.readdir(store) + expect(entries).toEqual([expect.stringMatching(/^v\d+$/)]) + } + + it('resolves from the store of the workspace, not one chosen for the temp dir, with real pnpm', async () => { + const { workspace, files, workspaceStore, decoyStore } = await plantStoreDirDecoys() + await pruneBundledLockfile({ + workspace, + packageManager: new PNpmDetector(), + files, + env: testEnv(), + }) + await versionedStoreCreatedIn(workspaceStore) + await expect(fs.access(decoyStore)).rejects.toThrow() + }, 60_000) + + it('decoy control: pnpm honors the materialized storeDir when the store is not pinned', async () => { + // Positive control for the test above: proves the planted channel + // actually reaches the pnpm on PATH, so the pinning test cannot + // pass vacuously. + const { workspace, files, workspaceStore, decoyStore } = await plantStoreDirDecoys() + const unpinned: PackageManager = Object.assign(Object.create(new PNpmDetector()), { + storeDirCommand: () => undefined, + }) + await pruneBundledLockfile({ + workspace, + packageManager: unpinned, + files, + env: testEnv(), + }) + await versionedStoreCreatedIn(decoyStore) + await expect(fs.access(workspaceStore)).rejects.toThrow() + }, 60_000) + }) + it('fails when the command times out and debug-logs the partial output', async () => { const { workspace, files } = makePnpmScenario() // The timeout reason stays a one-liner; the child's partial output is @@ -590,10 +898,90 @@ describe('lockfile-pruner', () => { }) expect(result).toMatchObject({ status: 'failed', - reason: expect.stringContaining('not present in the original'), + reason: expect.stringContaining('not present in the original: safe-buffer@5.2.1 (is the lockfile'), }) }) + it('names a bounded sample of the unexpected entries, once each, and lists all at debug level', async () => { + const { workspace, files } = makePnpmScenario() + // A real re-resolution records a new package under both `packages` + // and `snapshots`, the latter with a peer suffix; the reason must + // name it once. Nine packages exceeds the eight the reason shows. + const names = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'].map(name => `fresh-${name}@1.0.0`) + const packagesBlock = names.map(name => ` ${name}:\n resolution: {integrity: sha512-x}`).join('\n') + const snapshotsBlock = names.map(name => ` ${name}(peer-dep@2.0.0): {}`).join('\n') + // The anchors carry no newline on purpose: a Windows checkout can + // hold the fixture with CRLF, and the helper fails the script when an + // anchor is absent, so the injection can never silently no-op. + const script = rewriteLockfileScript('pnpm-lock.yaml', + ['packages:', `packages:\n${packagesBlock}`], + ['snapshots:', `snapshots:\n${snapshotsBlock}`], + ) + const previouslyEnabled = Debug.disable() + Debug.enable('checkly:cli:services:check-parser:lockfile-pruner') + try { + let result: PruneBundledLockfileResult | undefined + const written = await captureStderr(async () => { + result = await pruneBundledLockfile({ + workspace, + packageManager: stubPackageManager(new Runnable('node', ['-e', script])), + files, + env: testEnv(), + }) + }) + expect(result).toMatchObject({ + status: 'failed', + reason: 'the regenerated lockfile resolves entries not present in the original: ' + + `${names.slice(0, 8).join(', ')} and 1 more (is the lockfile out of date with package.json?)`, + }) + const debugOutput = written.join('') + expect(debugOutput).toContain(`9 unexpected resolution(s) in the regenerated lockfile: ${names.join(', ')}`) + } finally { + Debug.enable(previouslyEnabled) + } + }) + + it('redacts URL-form entries in the reason and in the debug list', async () => { + const { workspace, files } = makePnpmScenario() + // Tarball and git dependencies are keyed by their URL, which can carry + // registry credentials. A plain entry follows the URL one, so the + // separator between them must survive redaction. + const script = ` + const fs = require('fs') + const content = fs.readFileSync('pnpm-lock.yaml', 'utf8') + fs.writeFileSync('pnpm-lock.yaml', content + + '\\n ' + process.env.PRUNE_TEST_KEY + ':\\n resolution: {integrity: sha512-x}\\n' + + ' after-dep@1.0.0:\\n resolution: {integrity: sha512-x}\\n') + ` + const previouslyEnabled = Debug.disable() + Debug.enable('checkly:cli:services:check-parser:lockfile-pruner') + try { + let result: PruneBundledLockfileResult | undefined + const written = await captureStderr(async () => { + result = await pruneBundledLockfile({ + workspace, + packageManager: stubPackageManager(new Runnable('node', ['-e', script])), + files, + env: { ...testEnv(), PRUNE_TEST_KEY: 'tarball-dep@https://user:secret@registry.example/T0KEN/dep.tgz' }, + }) + }) + expect(result).toMatchObject({ + status: 'failed', + reason: expect.stringContaining('not present in the original: tarball-dep@https://registry.example, after-dep@1.0.0 ('), + }) + const debugOutput = written.join('') + expect(debugOutput).toContain( + '2 unexpected resolution(s) in the regenerated lockfile: tarball-dep@https://registry.example, after-dep@1.0.0', + ) + for (const output of [result?.reason ?? '', debugOutput]) { + expect(output).not.toContain('secret') + expect(output).not.toContain('T0KEN') + } + } finally { + Debug.enable(previouslyEnabled) + } + }) + it('fails when the lockfile format version changes', async () => { const { workspace, files } = makePnpmScenario() const script = ` @@ -941,7 +1329,7 @@ describe('lockfile-pruner', () => { }) expect(result).toMatchObject({ status: 'failed', - reason: expect.stringContaining('not present in the original'), + reason: expect.stringContaining('not present in the original: node_modules/ms@2.0.0 ('), }) }) @@ -1148,7 +1536,7 @@ describe('lockfile-pruner', () => { }) expect(result).toMatchObject({ status: 'failed', - reason: expect.stringContaining('not present in the original'), + reason: expect.stringContaining('not present in the original: ms@2.1.3 ('), }) }) @@ -1637,7 +2025,7 @@ describe('lockfile-pruner', () => { }) expect(result).toMatchObject({ status: 'failed', - reason: expect.stringContaining('not present in the original'), + reason: expect.stringContaining('not present in the original: ms@npm:2.1.3 ('), }) }) diff --git a/packages/cli/src/services/check-parser/lockfile-pruner.ts b/packages/cli/src/services/check-parser/lockfile-pruner.ts index 53126398..edbc9c54 100644 --- a/packages/cli/src/services/check-parser/lockfile-pruner.ts +++ b/packages/cli/src/services/check-parser/lockfile-pruner.ts @@ -4,17 +4,17 @@ import { tmpdir } from 'node:os' import path from 'node:path' import Debug from 'debug' -import { execa } from 'execa' +import { execa, type Result } from 'execa' import JSON5 from 'json5' import { parse as parseYaml } from 'yaml' import { createFauxPackageFiles } from './faux-package.js' import { isPnpmfilePath } from './package-files/pnpmfile.js' import { lineage } from './package-files/walk.js' -import { PackageManager, PathLookup } from './package-files/package-manager.js' +import { PackageManager, PathLookup, Runnable } from './package-files/package-manager.js' import { Package, Workspace } from './package-files/workspace.js' import { File, VirtualFile } from './parser.js' -import { redactUrl } from '../embedded-packages/diagnostics.js' +import { capList, redactUrl } from '../embedded-packages/diagnostics.js' import { pathToPosix } from '../util.js' const debug = Debug('checkly:cli:services:check-parser:lockfile-pruner') @@ -89,10 +89,71 @@ export type PruneBundledLockfileResult = // resolutions — so waiting minutes buys nothing. const DEFAULT_TIMEOUT_MS = 30_000 -// The yarn version probe shares the install's budget; if it leaves the -// install less than this, the prune is abandoned with a provisioning -// message rather than spawning an install doomed to time out. -const YARN_PROBE_MIN_INSTALL_BUDGET_MS = 1_000 +// A pre-install probe (yarn's version check, pnpm's store lookup) shares +// the install's budget, see PruneBudget; if it leaves the install less than +// this, the prune is abandoned with a message naming the probe rather than +// spawning an install doomed to time out. +const PROBE_MIN_INSTALL_BUDGET_MS = 1_000 + +/** What a prune spawn yields: execa's non-rejecting result with string streams. */ +type ChildResult = Result<{ reject: false }> + +/** + * The prune's time budget, shared between the pre-install probes (yarn's + * version check, pnpm's store lookup) and the install itself, so the prune + * cannot block for longer than the documented timeout in total — a stalled + * first-use corepack download would otherwise be paid once per spawn. A + * total of 0 means "no timeout" (execa's own semantics), so nothing is + * ever exhausted then. + */ +class PruneBudget { + #spentMs = 0 + + constructor (readonly totalMs: number) {} + + /** + * execa timeout for the next spawn. Never 0 for a bounded budget: execa + * would read that as "no timeout". + */ + get remainingMs (): number { + if (this.totalMs === 0) { + return 0 + } + return Math.max(this.totalMs - this.#spentMs, 1) + } + + /** + * Whether a bounded budget has less than the install floor left, in + * which case spawning the install would only produce a misleading + * ~zero-length timeout. + */ + get belowFloor (): boolean { + return this.totalMs > 0 && this.totalMs - this.#spentMs < PROBE_MIN_INSTALL_BUDGET_MS + } + + /** + * Spawns a child within the remaining budget and charges its wall-clock + * time. The timeout is read before the clock starts, so the first spawn + * reports exactly the configured budget when it times out. Returned + * alongside the result for that message. + */ + async spawn ( + runnable: Runnable, + options: { cwd: string, env: NodeJS.ProcessEnv }, + ): Promise<{ result: ChildResult, timeoutMs: number }> { + const timeoutMs = this.remainingMs + const startedAt = Date.now() + const result = await execa(runnable.executable, runnable.args, { + cwd: options.cwd, + env: options.env, + extendEnv: false, + timeout: timeoutMs, + reject: false, + }) + this.#spentMs += Date.now() - startedAt + return { result, timeoutMs } + } +} const MAX_FAILURE_DETAIL_LENGTH = 400 @@ -319,8 +380,10 @@ interface LockfileSnapshot { /** Dependency edges by a format-specific unique key. */ edges: Map /** - * Resolution entries: key → recorded version (or empty when the key - * itself pins the version, as in pnpm). Used for the subset check. + * Resolution entries for the subset check, keyed per format so that the + * key pins the exact resolution (pnpm and npm include the version, yarn + * and bun the whole serialized entry), mapped to a short `name@version` + * display form for messages. */ resolutions: Map /** Importer directories, relative to the root ('.' for the root itself). */ @@ -398,7 +461,10 @@ function parseLockfileSnapshot (content: string, lockfileName: string): Lockfile // is what the subset check needs. for (const section of ['packages', 'snapshots']) { for (const key of Object.keys(doc[section] ?? {})) { - snapshot.resolutions.set(`${section}\0${key}`, '') + // The display name drops the peer suffix (`foo@1.0.0(bar@2.0.0)`) + // that the `snapshots` key carries and the `packages` key does not, + // so a re-resolved package is named once across the two sections. + snapshot.resolutions.set(`${section}\0${key}`, key.replace(/\(.*$/, '')) } } @@ -446,7 +512,11 @@ function parseLockfileSnapshot (content: string, lockfileName: string): Lockfile isLink: entry.link === true, }) if (entry.link !== true) { - snapshot.resolutions.set(key, String(entry.version ?? entry.resolved ?? '')) + // Paths are unique per lockfile, so the version can ride along in + // the key to make a changed version an unknown entry. + const version = String(entry.version ?? entry.resolved ?? '') + const pinned = version === '' ? key : `${key}@${version}` + snapshot.resolutions.set(pinned, pinned) } } @@ -541,7 +611,8 @@ function parseLockfileSnapshot (content: string, lockfileName: string): Lockfile // still fail the check. Serialization is stable because both sides are // parsed from bun's own deterministic output by this same function. for (const tuple of Object.values(packages)) { - snapshot.resolutions.set(JSON.stringify(tuple), '') + const name = Array.isArray(tuple) && typeof tuple[0] === 'string' ? tuple[0] : JSON.stringify(tuple) + snapshot.resolutions.set(JSON.stringify(tuple), name) } return snapshot @@ -656,7 +727,7 @@ function parseLockfileSnapshot (content: string, lockfileName: string): Lockfile // dependencies) must still fail the check. Serialization is stable // because both sides are parsed from yarn's own deterministic // output by this same function. - snapshot.resolutions.set(JSON.stringify(entry), '') + snapshot.resolutions.set(JSON.stringify(entry), resolution) continue } // Workspace entries are the importers: their resolution carries the @@ -742,11 +813,27 @@ function verifyPrunedLockfile ( // lockfile was out of date with the bundled manifests and the package // manager resolved something fresh from the registry — versions the user // never installed or tested with. - for (const [key, version] of regenerated.resolutions) { - if (original.resolutions.get(key) !== version) { - return `the regenerated lockfile resolves entries not present in the original ` - + `(is the lockfile out of date with package.json?)` + // Collected by display name: pnpm records a re-resolved package under + // both `packages` and `snapshots`, which would otherwise list it twice. + const unexpected = new Set() + for (const [key, name] of regenerated.resolutions) { + if (!original.resolutions.has(key)) { + unexpected.add(name) + } + } + if (unexpected.size > 0) { + // Names can be URLs (tarball and git dependencies), so each is + // redacted on its own, after deduplication (two entries can redact to + // the same text) and before joining (a URL's redaction would swallow + // the separator after it). Only the length cap applies to the joined + // sample; the complete list goes to the debug channel. + const listed = [...unexpected].map(name => redactDetail(name)) + if (debug.enabled) { + debug('%d unexpected resolution(s) in the regenerated lockfile: %s', listed.length, listed.join(', ')) } + return `the regenerated lockfile resolves entries not present in the original: ` + + capDetail(capList(listed, ', ', ' and ')) + + ` (is the lockfile out of date with package.json?)` } // Every dependency edge that was a workspace link and that still exists @@ -932,20 +1019,28 @@ export function redactDetail (detail: string): string { } function sanitizeDetail (detail: string): string { - const redacted = redactDetail(detail) - if (redacted.length <= MAX_FAILURE_DETAIL_LENGTH) { - return redacted + return capDetail(redactDetail(detail)) +} + +/** The length cap alone, for text that is already redacted. */ +function capDetail (detail: string): string { + if (detail.length <= MAX_FAILURE_DETAIL_LENGTH) { + return detail } - return `${redacted.slice(0, MAX_FAILURE_DETAIL_LENGTH)}…` + return `${detail.slice(0, MAX_FAILURE_DETAIL_LENGTH)}…` } /** - * Logs a timed-out child's partial stdout/stderr at debug level. The tail - * is kept because the most recent output is where a stalled request shows; - * the user-facing reason stays a one-liner, so this is the only place the - * output survives. + * Logs a child's stdout/stderr at debug level, redacted. Only the tail is + * kept: for a timed-out child the most recent output is where a stalled + * request shows, and a probe's answer is its last line. The user-facing + * reason stays a one-liner, so this is the only place the output survives. */ -function debugTimedOutChildOutput (label: string, output: { stdout?: string, stderr?: string }): void { +function debugChildOutput ( + label: string, + output: { stdout?: string, stderr?: string }, + { timedOut }: { timedOut: boolean }, +): void { if (!debug.enabled) { return } @@ -959,10 +1054,20 @@ function debugTimedOutChildOutput (label: string, output: { stdout?: string, std : `…${redacted.slice(-MAX_DEBUG_OUTPUT_LENGTH)}` // The output goes in as an argument, not the format string, so `%o` // and `%%` sequences in it are printed verbatim. - debug('%s timed out; partial %s:\n%s', label, stream, tail) + debug(timedOut ? '%s timed out; partial %s:\n%s' : '%s %s:\n%s', label, stream, tail) } } +/** + * The first non-empty stream of a failed child, for the failure reason: + * stderr carries the error for most package managers, stdout for some, + * and execa's own message covers a child that never wrote at all. + */ +function firstChildOutput (result: ChildResult): string { + return [result.stderr, result.stdout, result.shortMessage] + .find(value => typeof value === 'string' && value.trim() !== '') ?? 'unknown error' +} + // Larger files are not plausible manifests; the cap also keeps a scan of a // shared temp root from slurping an arbitrarily large unrelated file. const MAX_ANCESTOR_MANIFEST_BYTES = 4 * 1024 * 1024 @@ -999,6 +1104,26 @@ function executableMissing (executable: string): PruneBundledLockfileResult { } } +/** + * Classifies a failed spawn whose executable may simply be absent. On + * Windows a missing executable never surfaces as a spawn ENOENT: execa + * resolves the command itself (via which-command) and wraps an + * unresolvable command in cmd.exe, so the child "runs" and exits non-zero + * with cmd.exe's not-recognized message. Classify after the fact with a + * PATH probe — safe against probe/spawn resolution differences, because + * the command has already failed either way and only the reporting is at + * stake. Executables given as a path are left to the spawn's own error + * detail. Returns undefined when the executable exists, i.e. the failure + * is the command's own. + */ +async function missingFromPath (executable: string): Promise { + if (path.basename(executable) !== executable) { + return undefined + } + const executablePath = await new PathLookup().lookupPath(executable) + return executablePath === undefined ? executableMissing(executable) : undefined +} + /** * Walks from `startDir` to the filesystem root looking for a package.json * that declares npm workspaces. The caller treats any hit as "this location @@ -1074,7 +1199,9 @@ export async function pruneBundledLockfile ( // unsupported package manager should always skip notably, never surface // a lockfile read error as a 'failed' warning that implies pruning was // attempted. - const runnable = packageManager.lockfileOnlyInstallCommand() + // Reassigned once the store directory is known (see below), so every + // message past that point names the command that actually ran. + let runnable = packageManager.lockfileOnlyInstallCommand() if (runnable === undefined) { return { status: 'skipped', @@ -1203,50 +1330,39 @@ export async function pruneBundledLockfile ( // below. Probe and install share the executable, cwd and env, so their // version resolution cannot diverge. const childEnv = buildChildEnv(env) - // The probe shares the install's time budget so the yarn path cannot - // block for longer than the documented timeout in total (a stalled - // first-use corepack download would otherwise be paid twice). - // timeoutMs === 0 means "no timeout" to execa, so the whole budget - // dance is skipped in that case. - let installTimeoutMs = timeoutMs - if (packageManager.name === 'yarn') { - // A whole budget below the install floor can never succeed (the yarn - // path spends a probe plus an install), so reject it up front — - // before the probe, whose own outcome under such a budget would be a - // misleading "timed out" rather than this caller-misconfiguration. - if (timeoutMs > 0 && timeoutMs < YARN_PROBE_MIN_INSTALL_BUDGET_MS) { - return { - status: 'failed', - reason: `the prune timeout (${timeoutMs}ms) is below the minimum needed to run yarn`, - } + const budget = new PruneBudget(timeoutMs) + const storeDirCommand = packageManager.storeDirCommand() + // A whole budget below the install floor can never succeed when a probe + // precedes the install (the yarn version check, the store lookup), so + // reject it up front — before the probe, whose own outcome under such + // a budget would be a misleading "timed out" rather than this + // caller-misconfiguration. + if ((packageManager.name === 'yarn' || storeDirCommand !== undefined) && budget.belowFloor) { + return { + status: 'failed', + reason: `the prune timeout (${timeoutMs}ms) is below the minimum needed to run ${packageManager.name}`, } - const probeStartedAt = Date.now() - const probe = await execa(runnable.executable, ['--version'], { - cwd: tempDir, - env: childEnv, - extendEnv: false, - timeout: timeoutMs, - reject: false, - }) + } + if (packageManager.name === 'yarn') { + const { result: probe, timeoutMs: probeTimeoutMs } = await budget.spawn( + new Runnable(runnable.executable, ['--version']), + { cwd: tempDir, env: childEnv }, + ) if (probe.timedOut) { // The probe consumed the whole budget; spawning the install with // the ~zero remainder would only produce a confusing second kill. - debugTimedOutChildOutput(`${runnable.executable} --version`, probe) - return { status: 'failed', reason: `${runnable.executable} timed out after ${formatBudget(timeoutMs)}; ${TIMEOUT_HINT}` } + debugChildOutput(`${runnable.executable} --version`, probe, { timedOut: true }) + return { status: 'failed', reason: `${runnable.executable} timed out after ${formatBudget(probeTimeoutMs)}; ${TIMEOUT_HINT}` } } - if (timeoutMs > 0) { - const remaining = timeoutMs - (Date.now() - probeStartedAt) - if (remaining < YARN_PROBE_MIN_INSTALL_BUDGET_MS) { - // The probe (typically a slow first-use corepack toolchain - // download) left too little for the install; a 1 ms install would - // be a misleading second timeout, so say what actually happened. - return { - status: 'failed', - reason: 'provisioning the yarn toolchain used up the prune time budget before the' - + ` lockfile could be regenerated; pre-install yarn or ${TIMEOUT_HINT}`, - } + if (budget.belowFloor) { + // The probe (typically a slow first-use corepack toolchain + // download) left too little for the install; a 1 ms install would + // be a misleading second timeout, so say what actually happened. + return { + status: 'failed', + reason: 'provisioning the yarn toolchain used up the prune time budget before the' + + ` lockfile could be regenerated; pre-install yarn or ${TIMEOUT_HINT}`, } - installTimeoutMs = remaining } const probeVersion = probe.failed ? '' : probe.stdout?.trim() ?? '' const major = Number.parseInt(probeVersion, 10) @@ -1289,18 +1405,77 @@ export async function pruneBundledLockfile ( } } + // Pins the store the install resolves from to the workspace's own, + // which pnpm would not pick for a temp dir on another filesystem (the + // rationale is with PNpmDetector.lockfileOnlyInstallCommand). The + // lookup runs in the WORKSPACE ROOT so it reads the workspace's own + // config and applies pnpm's same-mount rule to the real project dir. + // An unusable answer is a notable skip, not a failure: running against + // an empty store is the very thing being avoided, and the remedy is + // the package manager's installation or configuration, not the + // lockfile. + if (storeDirCommand !== undefined) { + const display = storeDirCommand.unsafeDisplayCommand + const { result: probe, timeoutMs: probeTimeoutMs } = await budget.spawn( + storeDirCommand, + { cwd: workspace.root.path, env: childEnv }, + ) + if (probe.timedOut) { + debugChildOutput(display, probe, { timedOut: true }) + return { status: 'failed', reason: `${display} timed out after ${formatBudget(probeTimeoutMs)}; ${TIMEOUT_HINT}` } + } + debugChildOutput(display, probe, { timedOut: false }) + if (probe.code === 'ENOENT') { + // execa also reports ENOENT for an invalid working directory, so + // rule out a vanished workspace root before blaming the executable. + if (!await directoryExists(workspace.root.path)) { + return { status: 'failed', reason: `the workspace directory '${workspace.root.path}' disappeared` } + } + return executableMissing(storeDirCommand.executable) + } + if (budget.belowFloor) { + return { + status: 'failed', + reason: `determining the ${packageManager.name} store directory used up the prune time budget before the` + + ` lockfile could be regenerated; pre-install ${packageManager.name} or ${TIMEOUT_HINT}`, + } + } + if (probe.failed || probe.exitCode !== 0) { + const missing = await missingFromPath(storeDirCommand.executable) + if (missing !== undefined) { + return missing + } + return { + status: 'skipped', + reason: `the ${packageManager.name} store directory could not be determined` + + ` (${display} failed: ${sanitizeDetail(firstChildOutput(probe))})`, + notable: true, + } + } + // The path is the last non-empty line: pnpm 10's default reporter + // can print warnings to stdout ahead of it. It must be absolute and + // versioned (`/v10`) — anything else means the command's + // output changed shape, and guessing would risk an empty store. The + // PARENT is what gets pinned: pnpm appends its own version segment, + // and the install's pnpm may be of another major than the lookup's. + const printed = (probe.stdout ?? '').split(/\r?\n/).map(line => line.trim()).filter(line => line !== '').at(-1) + if (printed === undefined || !path.isAbsolute(printed) || !/^v\d+$/.test(path.basename(printed))) { + return { + status: 'skipped', + reason: `the ${packageManager.name} store directory could not be determined (${display} printed` + + ` ${printed === undefined ? 'nothing' : `'${sanitizeDetail(printed)}'`} instead of a versioned store path)`, + notable: true, + } + } + runnable = packageManager.lockfileOnlyInstallCommand({ storeDir: path.dirname(printed) }) ?? runnable + } + debug(`Running ${runnable.unsafeDisplayCommand} in ${tempDir}`) - const result = await execa(runnable.executable, runnable.args, { - cwd: tempDir, - env: childEnv, - extendEnv: false, - timeout: installTimeoutMs, - reject: false, - }) + const { result, timeoutMs: installTimeoutMs } = await budget.spawn(runnable, { cwd: tempDir, env: childEnv }) if (result.timedOut) { - debugTimedOutChildOutput(runnable.executable, result) + debugChildOutput(runnable.executable, result, { timedOut: true }) return { status: 'failed', reason: `${runnable.executable} timed out after ${formatBudget(installTimeoutMs)}; ${TIMEOUT_HINT}` } } if ((result as any).code === 'ENOENT') { @@ -1313,22 +1488,11 @@ export async function pruneBundledLockfile ( return executableMissing(runnable.executable) } if (result.failed || result.exitCode !== 0) { - // On Windows a missing executable never surfaces as a spawn ENOENT: - // execa resolves the command itself (via which-command) and wraps an - // unresolvable command in cmd.exe, so the child "runs" and exits - // non-zero with cmd.exe's not-recognized message. Classify after the fact with a - // PATH probe — safe against probe/spawn resolution differences, - // because the command has already failed either way and only the - // reporting is at stake. Executables given as a path are left to the - // spawn's own error detail. - if (path.basename(runnable.executable) === runnable.executable) { - const executablePath = await new PathLookup().lookupPath(runnable.executable) - if (executablePath === undefined) { - return executableMissing(runnable.executable) - } + const missing = await missingFromPath(runnable.executable) + if (missing !== undefined) { + return missing } - const detail = [result.stderr, result.stdout, (result as any).shortMessage] - .find(value => typeof value === 'string' && value.trim() !== '') ?? 'unknown error' + const detail = firstChildOutput(result) // Yarn's blocked-request error is the network guard doing its job; // surfaced verbatim it reads like the user's own configuration is // broken. Name the two real causes instead — a stale lockfile, or a @@ -1354,7 +1518,7 @@ export async function pruneBundledLockfile ( } return { status: 'failed', - reason: `${runnable.unsafeDisplayCommand} failed: ${sanitizeDetail(String(detail))}`, + reason: `${runnable.unsafeDisplayCommand} failed: ${sanitizeDetail(detail)}`, } } diff --git a/packages/cli/src/services/check-parser/package-files/__tests__/package-manager.spec.ts b/packages/cli/src/services/check-parser/package-files/__tests__/package-manager.spec.ts index dbfdcf6a..ed6a055c 100644 --- a/packages/cli/src/services/check-parser/package-files/__tests__/package-manager.spec.ts +++ b/packages/cli/src/services/check-parser/package-files/__tests__/package-manager.spec.ts @@ -587,6 +587,12 @@ describe('lockfileOnlyInstallCommand', () => { ]) }) + it('pins the pnpm store when given one', () => { + const runnable = new PNpmDetector().lockfileOnlyInstallCommand({ storeDir: '/stores/pnpm' }) + expect(runnable.args.at(-1)).toEqual('--config.storeDir=/stores/pnpm') + expect(runnable.args).toHaveLength(new PNpmDetector().lockfileOnlyInstallCommand().args.length + 1) + }) + it('regenerates the lockfile without installing for npm, preferring cached metadata', () => { const runnable = new NpmDetector().lockfileOnlyInstallCommand() expect(runnable?.executable).toEqual('npm') @@ -613,6 +619,22 @@ describe('lockfileOnlyInstallCommand', () => { }) }) +describe('storeDirCommand', () => { + it('looks the store up with pnpm, whose store depends on the project location', () => { + const runnable = new PNpmDetector().storeDirCommand() + expect(runnable.executable).toEqual('pnpm') + expect(runnable.args).toEqual(['store', 'path']) + }) + + it('is absent for package managers with a single home-directory cache', () => { + for (const detector of [ + new NpmDetector(), new CNpmDetector(), new YarnDetector(), new BunDetector(), new DenoDetector(), + ]) { + expect(detector.storeDirCommand(), detector.name).toBeUndefined() + } + }) +}) + describe('PathLookup', () => { // The lookup must resolve like the spawn's own resolver (execa → // which-command) or the two disagree about whether an executable exists. diff --git a/packages/cli/src/services/check-parser/package-files/package-manager.ts b/packages/cli/src/services/check-parser/package-files/package-manager.ts index fe0a791d..9124a575 100644 --- a/packages/cli/src/services/check-parser/package-files/package-manager.ts +++ b/packages/cli/src/services/check-parser/package-files/package-manager.ts @@ -37,6 +37,16 @@ export interface AddCommandOptions { saveDev?: boolean } +export interface LockfileOnlyInstallOptions { + /** + * The content-addressable store the install must resolve from, as the + * value of the package manager's store setting (for pnpm the parent of + * what `storeDirCommand` prints, see `PNpmDetector`). Only meaningful for + * package managers with a `storeDirCommand`; others ignore it. + */ + storeDir?: string +} + export interface PackageManager { get name (): string get representativeLockfiles (): string[] @@ -44,6 +54,22 @@ export interface PackageManager { installCommand (): Runnable addCommand (options: AddCommandOptions): Runnable execCommand (args: string[]): Runnable + /** + * Command that prints the content-addressable store the project in the + * spawn working directory would install from, or undefined when the + * package manager's cache location does not depend on the project's + * location: npm and bun keep a single cache under the home directory, + * and yarn's lockfile-only install runs with the network blocked and + * never touches its cache. + * + * Output contract, which the lockfile pruner enforces: the last non-empty + * stdout line is an absolute path whose last segment is the store's + * version (`/v10`), and the PARENT of that path is what the + * pruner hands back to `lockfileOnlyInstallCommand` as `storeDir`. This + * is pnpm's `store path` shape; an implementation for another package + * manager must print the same shape or the pruner skips notably. + */ + storeDirCommand (): Runnable | undefined /** * Command that regenerates the lockfile from the manifests on disk without * installing anything (resolution only). Undefined when the package @@ -62,7 +88,7 @@ export interface PackageManager { * lockfile pruner refuses to run when its temp dir sits inside a * workspace. */ - lockfileOnlyInstallCommand (): Runnable | undefined + lockfileOnlyInstallCommand (options?: LockfileOnlyInstallOptions): Runnable | undefined lookupWorkspace (dir: string): Promise /** * Resolves the version of a single package as recorded in the package @@ -158,6 +184,15 @@ export abstract class PackageManagerDetector { return undefined } + /** + * Default: the cache location does not depend on the project's location, + * so the lockfile pruner has nothing to pin. See PNpmDetector for the one + * package manager where it does. + */ + storeDirCommand (): Runnable | undefined { + return undefined + } + /** * Default: lockfile parsing is unsupported, so callers fall back. Package * managers that can parse their lockfile override this. @@ -364,7 +399,16 @@ export class PNpmDetector extends PackageManagerDetector implements PackageManag return new Runnable('pnpm', ['install']) } - lockfileOnlyInstallCommand (): Runnable { + storeDirCommand (): Runnable { + // Prints the VERSIONED store (`/v10`) the project in the + // working directory would install from — the setting from + // pnpm-workspace.yaml, the npmrc layers or npm_config_store_dir, or + // pnpm's own per-mount default. See lockfileOnlyInstallCommand for why + // that matters and why the pruner pins the parent of the printed path. + return new Runnable('pnpm', ['store', 'path']) + } + + lockfileOnlyInstallCommand (options: LockfileOnlyInstallOptions = {}): Runnable { // --no-frozen-lockfile is load-bearing: pnpm auto-enables frozen mode // when CI=true, and a lockfile-only regeneration is by definition not a // frozen install. --lockfile-dir is equally load-bearing: as a CLI flag @@ -417,16 +461,43 @@ export class PNpmDetector extends PackageManagerDetector implements PackageManag // rejects any regenerated entry absent from the original lockfile, so // the worst case is a fallback, never a wrong lockfile. // + // --config.storeDir pins the store the temp-dir install resolves from. + // Left to pnpm, the choice depends on the temp dir's location: on a + // mount other than the home directory's (tmpfs /tmp, a TMPDIR on + // another volume, containers) pnpm picks an EMPTY store at the mount + // root, and then every package in the re-resolved trees needs registry + // metadata — thousands of requests behind a slow proxy, or, served by + // --prefer-offline from a metadata cache that predates the locked + // version, a different version that the subset verification rejects. + // With the workspace's own store pinned the re-resolution is answered + // from the store, as it is when the same command runs in the checkout. + // The value is the PARENT of what `pnpm store path` prints: pnpm + // appends its own version segment (`v10`, `v11`) to the setting unless + // the path already ends in exactly that segment, and the probe and the + // install can resolve to different pnpm majors (corepack, mise and + // volta pick the version per directory, and the temp dir's root + // manifest can be a faux one without a packageManager field). Handed + // `/v10`, a pnpm 11 install would use `/v10/v11` — + // an empty store again — whereas handed `` each pnpm lands in + // the workspace's store for its own major. Verified on pnpm 10 and 11: + // the flag is what `install --lockfile-only` resolves from (it reports + // the store in its ndjson context event), it outranks a storeDir in a + // materialized pnpm-workspace.yaml or .npmrc and npm_config_store_dir, + // and npm/bun keep a single home-directory cache regardless of the + // project's location, so they need no equivalent. + // // The pruner does not compare the lockfile's `settings` section, so any // setting pnpm records there ships to the runner unchecked and could // mismatch the runner's own install configuration. Verified on pnpm 10 - // and 11: neither --config. setting nor --prefer-offline is recorded in - // `settings` (only autoInstallPeers and excludeLinksFromLockfile are). + // and 11: neither --config. setting, --prefer-offline nor the storeDir + // is recorded in `settings` (only autoInstallPeers and + // excludeLinksFromLockfile are). return new Runnable('pnpm', [ 'install', '--lockfile-only', '--ignore-scripts', '--no-frozen-lockfile', '--lockfile-dir', '.', '--config.allowUnusedPatches=true', '--config.trustLockfile=true', '--prefer-offline', + ...options.storeDir !== undefined ? [`--config.storeDir=${options.storeDir}`] : [], ]) }