Skip to content

Migrate the CLI workspace to Bun-native test execution (Fixes #2843) - #3056

Merged
acoliver merged 135 commits into
mainfrom
issue2843
Aug 6, 2026
Merged

Migrate the CLI workspace to Bun-native test execution (Fixes #2843)#3056
acoliver merged 135 commits into
mainfrom
issue2843

Conversation

@acoliver

@acoliver acoliver commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Fixes #2843.

Migrates the cli workspace to Bun's native test runner and removes Vitest from
it entirely. There is no test:vitest fallback: keeping one would mean keeping
two runners green.

What changed

  • packages/cli/run-bun-tests.ts — discovers every test file under src/,
    test/, test-bun/ and test-utils/ and runs each in its own bun test
    process. A process per file is required because Bun's mock.module registry
    is process-wide, so a shared process leaks module mocks between files. Writes
    JUnit, passes --timeout 30000 explicitly (Bun's 5s default is not enough,
    and the bunfig key is not honoured for single-file runs).
  • Vitest removed — 9 config variants, vitest.test-groups.ts,
    test-setup.ts, test-setup-base.ts, stryker.conf.json.
  • test-setup/augment-bun-vi.ts (shared with every workspace) — async
    vi.mock factories are settled synchronously with drainMicrotasks() so the
    mock exists before the module body runs, as Vitest's hoisting guarantees; a
    factory that rejects now fails loudly instead of silently leaving the real
    module installed; restoreAllMocks keeps module mocks registered and clears
    nested mock history.
  • packages/cli/bun-test-setup.ts — Vitest globals: true parity, Ink
    teardown, credential-proxy env isolation, and process.exitCode reset per
    test.
  • Snapshots — migrated from Vitest's key format to Bun's. Without this, Bun
    silently appended new entries and every snapshot assertion was vacuous.

The exclusion list this removes

vitest.test-groups.ts carried a baseExclude list that hid ~37 test files,
and no CI job ever invoked the CLI's test:integration, hiding 24 more. Those
files had not run in a long time. This PR runs everything, which is why
previously invisible failures appear.

Results

CLI files 643 / 653
CLI cases 8357 passing, 49 failing
snapshots written during a run 0

Cross-workspace, to cover the shared shim: core 336/336, providers 493/493,
auth 33/33, test-setup 3/3. The 3 agents failures were verified against a
clean main worktree and fail identically there.

lint 0, typecheck 0, prettier --check clean, build 0, smoke test passes.

What is not fixed here, and why

Every remaining failure and every deleted test is recorded in #3046 with
evidence. The substantive items:

  • Bun's spawnSync drops extra file descriptors. sandbox-bashrc.ts reads
    payloads from fds 3 and 4; Node returns output.length 5 with both
    populated, Bun returns 3 and empty. All 16 cases fail and cannot be fixed in
    the test. This has production impact, since the repo is moving to Bun as the
    runtime.
  • Inline load-balancer profiles are rejected while the file-based path
    works (parseInlineProfile has no type: 'loadbalancer' branch).
  • The prebuilt bundle cannot resolve provider aliases. bun-build.config.ts
    emits packages/cli/bundle/llxprt.js with no assets, while
    copy_bundle_assets.ts still targets the pre-Ship a prebuilt CLI bundle: raw-TS distribution costs 5.2s of startup per launch, blowing agent-launcher timeouts on Windows #2999 repo-root bundle/ and is
    invoked by nothing. CI is unaffected because it never builds a bundle, but a
    stale or asset-less bundle silently shadows correct source.

24 tests were deleted: 13 asserted behaviour that no longer exists in the source
(e.g. a prop nothing reads), and 11 were placeholders whose only assertion was
expect(true).toBe(true). Each is listed in #3046 with the behaviours lost.

Verification notes

  • Snapshot work must be checked with CI=true so Bun compares instead of
    writing. An early round of this migration committed fabricated snapshots
    recorded from error frames; they were found by diffing entry counts against
    main and reverted.
  • Tests that spawn the built CLI are only meaningful after npm run bundle
    npm run build does not regenerate it.

Summary by CodeRabbit

  • New Features
    • Added mouse-click cursor positioning in the CLI input prompt.
    • Added support for validating load-balancer profiles.
  • Bug Fixes
    • Improved memory statistics display across runtime environments.
    • Prevented invalid retry and retry-wait settings from being accepted.
    • Improved sandbox diagnostics and cleanup when environment probes fail.
  • Chores
    • Migrated CLI testing to a more reliable Bun-based test runner with improved timeout handling and reporting.

acoliver added 30 commits August 3, 2026 13:58

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/cli/src/ui/components/TodoPanel.responsive.test.tsx (1)

81-81: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Restore the removed TodoPanel behavior coverage.

The migration removes tests for narrow rendering, current-task output, resize output, and subtask semantic rendering. The PR objective requires migration without silently excluding, skipping, or deferring tests. Port these assertions to Bun-compatible rendering or replace each with equivalent behavior coverage.

  • packages/cli/src/ui/components/TodoPanel.responsive.test.tsx#L81-L81: Restore narrow-width status, count, and content-suppression coverage.
  • packages/cli/src/ui/components/TodoPanel.responsive.test.tsx#L155-L155: Restore coverage for the in-progress task indicator if it remains part of the UI contract.
  • packages/cli/src/ui/components/TodoPanel.responsive.test.tsx#L206-L206: Restore the initial narrow-render assertion in the resize test.
  • packages/cli/src/ui/components/TodoPanel.semantic.test.tsx#L191-L191: Restore semantic rendering coverage for subtasks.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/ui/components/TodoPanel.responsive.test.tsx` at line 81,
Restore the removed TodoPanel behavior coverage using Bun-compatible rendering
or equivalent assertions: in
packages/cli/src/ui/components/TodoPanel.responsive.test.tsx:81, cover
narrow-width status, count, and content suppression; at :155, cover the
in-progress task indicator; at :206, restore the initial narrow-render assertion
in the resize test; and in
packages/cli/src/ui/components/TodoPanel.semantic.test.tsx:191, restore semantic
subtask rendering coverage. Do not skip, defer, or silently remove these
behaviors.
🧹 Nitpick comments (6)
packages/cli/test/run-bun-tests.test.ts (1)

134-147: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert discovery is non-empty before looping.

If discoverTestFiles returned an empty array, the loop body never runs and the test passes without checking anything. Assert the array contents directly.

♻️ Proposed change
-      for (const file of discoverTestFiles(root)) {
-        expect(file.startsWith('/')).toBe(false);
-        expect(file).toBe('src/only.test.ts');
-      }
+      expect(discoverTestFiles(root)).toEqual(['src/only.test.ts']);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/test/run-bun-tests.test.ts` around lines 134 - 147, Update the
test around discoverTestFiles to store its returned paths, assert the result is
non-empty, and then verify the expected relative path directly. Preserve the
existing checks that the discovered path is not absolute and equals
src/only.test.ts.
packages/cli/test/ui/commands/authCommand-logout.test.ts (2)

703-739: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Bound numRuns for the property tests that perform real token-store IO.

it.prop and fc.assert both default to 100 runs. This property saves tokens to the keyring-backed store and then executes up to 10 concurrent logout commands per run, so a single test can perform on the order of a thousand store operations. The Bun runner applies a 30s per-test timeout (PER_TEST_TIMEOUT_MS in packages/cli/run-bun-tests.ts), and this file contains four such properties. Set an explicit run count so the suite has a predictable budget.

♻️ Proposed change
         },
       ),
+      { numRuns: 20 },
     );
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/test/ui/commands/authCommand-logout.test.ts` around lines 703 -
739, Set an explicit, bounded numRuns option on the fc.assert call in the
concurrent logout property test, using a run count appropriate for the
keyring-backed token-store IO budget. Apply the same explicit bound to the other
three property tests in this file that perform real token-store operations,
while preserving their existing property logic.

651-672: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use a direct whitespace arbitrary.

The filter rejects most generated strings and does not provide a targeted whitespace distribution. fast-check 4.5.3 supports unit with fc.string, so generate spaces and tabs directly with maxLength: 4.

♻️ Proposed change
-        fc.string().filter((s) => /^\s*$/.test(s) && s.length < 5), // Only whitespace, shorter
-        fc.string().filter((s) => /^\s*$/.test(s) && s.length < 5),
+        fc.string({ unit: fc.constantFrom(' ', '\t'), maxLength: 4 }),
+        fc.string({ unit: fc.constantFrom(' ', '\t'), maxLength: 4 }),
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/test/ui/commands/authCommand-logout.test.ts` around lines 651 -
672, Update the property-based test around the `leadingSpace` and
`trailingSpace` arbitraries to generate whitespace directly using `fc.string`
with a whitespace character set and `maxLength: 4`, rather than filtering
arbitrary strings with a regular expression. Preserve the existing command
construction and assertions in the `authCommand.execute` test.
packages/cli/src/config/extensions/settingsIntegration.test.ts (1)

32-65: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider clearing the shared keyring between tests, and forwarding the options argument.

The entries map lives for the whole file, so credentials written by one test remain visible to later tests. The subclass also accepts only two constructor parameters, so any third options argument passed by the code under test is dropped and replaced by the keyring loader only.

♻️ Suggested adjustments
-  class InMemoryExtensionSettingsStorage extends actual.ExtensionSettingsStorage {
-    constructor(extensionName: string, extensionDir: string) {
-      super(extensionName, extensionDir, {
-        keyringLoader: async () => keyring,
-      });
-    }
-  }
+  class InMemoryExtensionSettingsStorage extends actual.ExtensionSettingsStorage {
+    constructor(
+      extensionName: string,
+      extensionDir: string,
+      options?: Record<string, unknown>,
+    ) {
+      super(extensionName, extensionDir, {
+        ...options,
+        keyringLoader: async () => keyring,
+      });
+    }
+  }

Add a beforeEach that empties the shared map if tests must not observe each other's stored values.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/config/extensions/settingsIntegration.test.ts` around lines
32 - 65, Update the settingsIntegration test mock so the shared entries map is
cleared in a beforeEach, preventing credentials from leaking between tests, and
change InMemoryExtensionSettingsStorage to accept and forward the original
options argument while overriding only keyringLoader. Use the existing mock
class and keyring symbols as the implementation points.
test-setup/vitest-parity.test.ts (2)

95-105: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Restore the mock after the vi.unmock case.

vi.unmock('./import-actual-fixture.js') at Line 101 replaces the module registration with the genuine exports for the remainder of the file. The earlier describes already ran, so the file passes today, but any case added below this point that expects 'mocked-by-async-factory' will fail for a non-obvious reason. Re-register the mock in an afterEach for this describe to keep the file order-independent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test-setup/vitest-parity.test.ts` around lines 95 - 105, Restore the mocked
registration after the vi.unmock test by adding an afterEach hook within the
vi.unmock describe that re-registers the fixture with the existing mock setup,
preserving the expected mocked-by-async-factory behavior for subsequent cases.

31-43: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert through a fresh import so the case can actually fail.

fixtureValue at Lines 36 and 41 is a static ESM binding captured when the file was evaluated. vi.restoreAllMocks() cannot change it, so this case passes even if reapplyModuleMocks() is removed from restoreAllMocks. The case at Lines 72-77 covers the real behavior. Re-import inside this case to make the assertion meaningful.

♻️ Suggested change
-describe('vi.restoreAllMocks over a module mock', () => {
-  it('leaves the module mocked and clears its call history', () => {
-    // Vitest's restoreAllMocks only restores spies; Bun's also reverts module
-    // mocks. The shim re-registers them afterwards, so a module mocked with
-    // vi.mock must still be a mock function here — with its calls cleared.
-    const mocked = fixtureValue;
-    expect(mocked).toBe('mocked-by-async-factory');
-
-    vi.restoreAllMocks();
-
-    expect(fixtureValue).toBe('mocked-by-async-factory');
-  });
-});
+describe('vi.restoreAllMocks over a module mock', () => {
+  it('leaves the module mocked', async () => {
+    // Vitest's restoreAllMocks only restores spies; Bun's also reverts module
+    // mocks. The shim re-registers them afterwards, so a re-import after the
+    // restore must still observe the mocked value.
+    expect(fixtureValue).toBe('mocked-by-async-factory');
+
+    vi.restoreAllMocks();
+
+    const reimported = await import('./import-actual-fixture.js');
+    expect(reimported.fixtureValue).toBe('mocked-by-async-factory');
+  });
+});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test-setup/vitest-parity.test.ts` around lines 31 - 43, Update the
vi.restoreAllMocks module-mock test around fixtureValue to dynamically import
the fixture after restoreAllMocks instead of asserting the statically captured
ESM binding. Assert the freshly imported value remains
'mocked-by-async-factory', ensuring the test exercises re-registration of the
module mock.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/cli/run-bun-tests.ts`:
- Around line 129-141: Update collectTestFiles around statSync and realpathSync
to catch inspection errors for individual entries, treating unreadable or
dangling links as non-test entries and continuing the directory walk. Preserve
the existing visited real-path cycle prevention and test-file collection
behavior for entries that can be inspected.

In `@packages/cli/src/cli.startInteractiveUI.test.tsx`:
- Around line 66-76: Update the test teardown to call
__setRenderForTesting(null) in afterEach, clearing the renderer spy installed by
injectRenderSpy() so later startInteractiveUI tests use the default renderer.

In `@packages/cli/src/config/config.integration.test.ts`:
- Around line 365-378: Update parseArgumentsWithExitAsThrow so it saves the
original process.stdout.write reference without binding or wrapping it, then
restores that exact reference in the finally block while preserving the
temporary suppressed writer during parsing.

In `@packages/cli/src/config/profileResolution.ts`:
- Around line 168-172: Gate the inline profile debug log in the
profile-resolution flow with tempDebugMode, matching the behavior of
applyFileProfile. Only invoke debugLogger.debug for the inline profile when
tempDebugMode is enabled, while preserving the existing provider and model
details.

In `@packages/cli/src/integration-tests/security.integration.test.ts`:
- Around line 387-389: Update the valid keyfile assertion around fs.access to
pass the read-permission mode fs.constants.R_OK, ensuring the test verifies the
file is readable rather than only confirming its existence.

In
`@packages/cli/src/providers/logging/multi-provider-logging.integration.test.ts`:
- Around line 496-518: Update the offeredTools declaration used by the OpenAI
and Anthropic generateChatCompletion calls to replace the legacy parameters
field with parametersJsonSchema, preserving the existing schema content and
removing parameters entirely.

In `@packages/cli/src/services/__testhelpers__/mockFs.ts`:
- Around line 174-183: Update clear() around the mkdirSync calls for
userCommandsDir and projectCommandsDir to ignore errors only when the error code
is EEXIST; rethrow every other directory-creation failure so setup reports the
originating filesystem operation.

In `@packages/cli/src/test-utils/render.tsx`:
- Line 284: Validate the interval option in waitFor before calculating
maxIterations, rejecting zero (and any non-positive value) so polling cannot
produce Infinity and bypass the timeout boundary; preserve the existing timeout
behavior for valid positive intervals.

In `@packages/cli/test-utils/ink-stub.ts`:
- Around line 43-45: Make stdin ownership render-scoped in setActiveStdin and
the related registry in packages/cli/test-utils/ink-stub.ts, tracking each
active render and restoring the most recently remaining stdin when one is
released; reset the registry after global cleanup drains all instances. Update
removeActiveInstance in packages/cli/test-utils/ink-testing-library.ts to
release only its own stdin and restore another active render’s stdin, then add
an overlapping-render regression test.

In `@packages/cli/test/run-bun-tests.test.ts`:
- Around line 265-283: Update the “discoverTestFiles symlink safety” suite to
use describe.skipIf(process.platform === 'win32'), preserving the existing
cycle-detection test unchanged for supported platforms.

In `@test-setup/augment-bun-vi.ts`:
- Around line 662-670: Update the factory-result handling around drainMicrotasks
to check Bun.peek.status(factoryResult) and only read Bun.peek and apply the
settled namespace when the status is fulfilled, allowing rejected results to
reach the existing rejection handler. In the pending path, handle an undefined
syncActual from loadIsolatedModuleSync with an explicit fallback instead of
passing it to toNamespace and registering { default: undefined }.
- Around line 821-845: Update spyOnCompat to inspect the property descriptor
before Reflect.get and support accessor spying for get and set modes by wrapping
the corresponding descriptor accessor while preserving the other descriptor
fields. Install the replacement with the accessor descriptor and ensure the
installed spy restoration restores the original descriptor; retain existing
behavior for data properties and unsupported targets.

---

Outside diff comments:
In `@packages/cli/src/ui/components/TodoPanel.responsive.test.tsx`:
- Line 81: Restore the removed TodoPanel behavior coverage using Bun-compatible
rendering or equivalent assertions: in
packages/cli/src/ui/components/TodoPanel.responsive.test.tsx:81, cover
narrow-width status, count, and content suppression; at :155, cover the
in-progress task indicator; at :206, restore the initial narrow-render assertion
in the resize test; and in
packages/cli/src/ui/components/TodoPanel.semantic.test.tsx:191, restore semantic
subtask rendering coverage. Do not skip, defer, or silently remove these
behaviors.

---

Nitpick comments:
In `@packages/cli/src/config/extensions/settingsIntegration.test.ts`:
- Around line 32-65: Update the settingsIntegration test mock so the shared
entries map is cleared in a beforeEach, preventing credentials from leaking
between tests, and change InMemoryExtensionSettingsStorage to accept and forward
the original options argument while overriding only keyringLoader. Use the
existing mock class and keyring symbols as the implementation points.

In `@packages/cli/test/run-bun-tests.test.ts`:
- Around line 134-147: Update the test around discoverTestFiles to store its
returned paths, assert the result is non-empty, and then verify the expected
relative path directly. Preserve the existing checks that the discovered path is
not absolute and equals src/only.test.ts.

In `@packages/cli/test/ui/commands/authCommand-logout.test.ts`:
- Around line 703-739: Set an explicit, bounded numRuns option on the fc.assert
call in the concurrent logout property test, using a run count appropriate for
the keyring-backed token-store IO budget. Apply the same explicit bound to the
other three property tests in this file that perform real token-store
operations, while preserving their existing property logic.
- Around line 651-672: Update the property-based test around the `leadingSpace`
and `trailingSpace` arbitraries to generate whitespace directly using
`fc.string` with a whitespace character set and `maxLength: 4`, rather than
filtering arbitrary strings with a regular expression. Preserve the existing
command construction and assertions in the `authCommand.execute` test.

In `@test-setup/vitest-parity.test.ts`:
- Around line 95-105: Restore the mocked registration after the vi.unmock test
by adding an afterEach hook within the vi.unmock describe that re-registers the
fixture with the existing mock setup, preserving the expected
mocked-by-async-factory behavior for subsequent cases.
- Around line 31-43: Update the vi.restoreAllMocks module-mock test around
fixtureValue to dynamically import the fixture after restoreAllMocks instead of
asserting the statically captured ESM binding. Assert the freshly imported value
remains 'mocked-by-async-factory', ensuring the test exercises re-registration
of the module mock.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 1c379c9d-36dc-4b06-930d-d47d8756412d

📥 Commits

Reviewing files that changed from the base of the PR and between 8c5eb22 and c1a2938.

⛔ Files ignored due to path filters (28)
  • packages/cli/src/ui/__snapshots__/App.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/HistoryItemDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/IDEContextDetailDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/InputPrompt.paste.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/InputPrompt.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/InputPrompt.vim.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/LoadingIndicator.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/ModelStatsDisplay.test.js.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/ModelStatsDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/SessionSummaryDisplay.test.js.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/SessionSummaryDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/SettingsDialog.interactions.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/SettingsDialog.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/StatsDisplay.sections.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/StatsDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/Table.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/ToolStatsDisplay.test.js.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/__snapshots__/ToolStatsDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/messages/__snapshots__/ToolGroupMessage.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/messages/__snapshots__/ToolMessage.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/shared/__snapshots__/RadioButtonSelect.test.js.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/components/views/__snapshots__/ChatList.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/utils/__snapshots__/MarkdownDisplay.test.js.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • packages/cli/src/ui/utils/__snapshots__/MarkdownDisplay.test.tsx.snap is excluded by !**/*.snap, !**/__snapshots__/**
  • project-plans/issue2843/coverage-gaps.md is excluded by !project-plans/**
  • project-plans/issue2843/coverage-issue-body.md is excluded by !project-plans/**
  • project-plans/issue2843/plan.md is excluded by !project-plans/**
  • project-plans/issue2843/pr-body.md is excluded by !project-plans/**
📒 Files selected for processing (177)
  • .github/workflows/ci.yml
  • packages/cli/bun-test-setup.ts
  • packages/cli/bunfig.toml
  • packages/cli/package.json
  • packages/cli/run-bun-tests.ts
  • packages/cli/src/cli-sandbox.test.tsx
  • packages/cli/src/cli.provider-init.test.ts
  • packages/cli/src/cli.renderOptions.test.tsx
  • packages/cli/src/cli.startInteractiveUI.test.tsx
  • packages/cli/src/cli.test.tsx
  • packages/cli/src/cliStartupOrdering.test.ts
  • packages/cli/src/commands/extensions/validate.test.ts
  • packages/cli/src/commands/mcp.test.ts
  • packages/cli/src/commands/mcp/add.test.ts
  • packages/cli/src/config/__tests__/sandboxConfig.test.ts
  • packages/cli/src/config/auth.test.ts
  • packages/cli/src/config/config.integration.test.ts
  • packages/cli/src/config/extension.part3.test.ts
  • packages/cli/src/config/extension.test.ts
  • packages/cli/src/config/extension.ts
  • packages/cli/src/config/extensionLoaderSettingsRef.ts
  • packages/cli/src/config/extensions/github.test.ts
  • packages/cli/src/config/extensions/settingsIntegration.test.ts
  • packages/cli/src/config/memoryReconciliation.concurrent.test.ts
  • packages/cli/src/config/memoryReconciliation.test.ts
  • packages/cli/src/config/profileBootstrap.ts
  • packages/cli/src/config/profileResolution.ts
  • packages/cli/src/config/trustedFolders.test.ts
  • packages/cli/src/integration-tests/cli-args-test-helpers.ts
  • packages/cli/src/integration-tests/cli-args.integration.test.ts
  • packages/cli/src/integration-tests/cli-args.profile-flag.integration.test.ts
  • packages/cli/src/integration-tests/compression-settings-apply.integration.test.ts
  • packages/cli/src/integration-tests/consumer-migration-p13.integration.test.ts
  • packages/cli/src/integration-tests/ephemeral-settings.integration.test.ts
  • packages/cli/src/integration-tests/loadbalancer.integration.test.ts
  • packages/cli/src/integration-tests/model-params-isolation.integration.test.ts
  • packages/cli/src/integration-tests/oauth-timing.integration.test.ts
  • packages/cli/src/integration-tests/retry-settings.integration.test.ts
  • packages/cli/src/integration-tests/security.integration.test.ts
  • packages/cli/src/integration-tests/test-utils.test.ts
  • packages/cli/src/integration-tests/tools-governance.integration.test.ts
  • packages/cli/src/launcher/bun-launcher.test.ts
  • packages/cli/src/providers/logging/git-stats.integration.test.ts
  • packages/cli/src/providers/logging/multi-provider-logging.integration.test.ts
  • packages/cli/src/services/BuiltinCommandLoader.test.ts
  • packages/cli/src/services/FileCommandLoader.processors.test.ts
  • packages/cli/src/services/FileCommandLoader.test.ts
  • packages/cli/src/services/__testhelpers__/mockFs.ts
  • packages/cli/src/session/errorReporting.test.ts
  • packages/cli/src/storage/ConversationStorage.test.ts
  • packages/cli/src/test-utils/render.tsx
  • packages/cli/src/test-utils/responsive-testing.test.tsx
  • packages/cli/src/test-utils/sessionMetrics.ts
  • packages/cli/src/ui/App.behavior.test.tsx
  • packages/cli/src/ui/App.components.test.tsx
  • packages/cli/src/ui/App.context.test.tsx
  • packages/cli/src/ui/App.dialogs.test.tsx
  • packages/cli/src/ui/App.e2e.test.tsx
  • packages/cli/src/ui/App.test.tsx
  • packages/cli/src/ui/__tests__/AppContainer.keybindings.test.tsx
  • packages/cli/src/ui/__tests__/AppContainer.mount.test.tsx
  • packages/cli/src/ui/__tests__/AppContainer.render-budget.test.tsx
  • packages/cli/src/ui/__tests__/integrationWiring.spec.tsx
  • packages/cli/src/ui/commands/AuthDialog.issue2274.test.tsx
  • packages/cli/src/ui/commands/diagnosticsCommand-test-helpers.ts
  • packages/cli/src/ui/commands/permissionsCommand.test.ts
  • packages/cli/src/ui/commands/providerCommand.test.ts
  • packages/cli/src/ui/commands/test/subagentCommand.test.ts
  • packages/cli/src/ui/components/AboutBox.theme.test.tsx
  • packages/cli/src/ui/components/AnsiOutput.test.tsx
  • packages/cli/src/ui/components/AuthDialog.test.tsx
  • packages/cli/src/ui/components/AuthDialog.theme.test.tsx
  • packages/cli/src/ui/components/ContextIndicator.ui.test.tsx
  • packages/cli/src/ui/components/Footer.responsive.test.tsx
  • packages/cli/src/ui/components/Footer.test.tsx
  • packages/cli/src/ui/components/Footer.tsx
  • packages/cli/src/ui/components/HistoryItemDisplay.test.tsx
  • packages/cli/src/ui/components/InputPrompt.completion.test.tsx
  • packages/cli/src/ui/components/InputPrompt.editing.test.tsx
  • packages/cli/src/ui/components/InputPrompt.paste.spec.tsx
  • packages/cli/src/ui/components/InputPrompt.paste.test.tsx
  • packages/cli/src/ui/components/InputPrompt.test.tsx
  • packages/cli/src/ui/components/InputPrompt.tsx
  • packages/cli/src/ui/components/InputPrompt.vim.test.tsx
  • packages/cli/src/ui/components/LoadingIndicator.test.tsx
  • packages/cli/src/ui/components/OAuthCodeDialog.test.tsx
  • packages/cli/src/ui/components/PoliciesDialog.test.tsx
  • packages/cli/src/ui/components/ProviderDialog.responsive.test.tsx
  • packages/cli/src/ui/components/SessionSummaryDisplay.test.tsx
  • packages/cli/src/ui/components/SettingsDialog.interactions.test.tsx
  • packages/cli/src/ui/components/SettingsDialog.test.tsx
  • packages/cli/src/ui/components/StatsDisplay.theme.test.tsx
  • packages/cli/src/ui/components/TodoPanel.responsive.test.tsx
  • packages/cli/src/ui/components/TodoPanel.semantic.test.tsx
  • packages/cli/src/ui/components/__tests__/LayoutManager.test.tsx
  • packages/cli/src/ui/components/__tests__/SessionBrowserDialog.layout.spec.tsx
  • packages/cli/src/ui/components/__tests__/SessionBrowserDialog.spec.tsx
  • packages/cli/src/ui/components/inputPromptRender.tsx
  • packages/cli/src/ui/components/inputPromptTypes.ts
  • packages/cli/src/ui/components/messages/ToolConfirmationMessage.responsive.test.tsx
  • packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx
  • packages/cli/src/ui/components/shared/MaxSizedBox.test.tsx
  • packages/cli/src/ui/components/shared/ScrollableList.theme.test.tsx
  • packages/cli/src/ui/components/shared/VirtualizedList.theme.test.tsx
  • packages/cli/src/ui/components/shared/text-buffer.part2.test.ts
  • packages/cli/src/ui/components/shared/text-buffer.part3.test.ts
  • packages/cli/src/ui/components/shared/text-buffer.part4.test.ts
  • packages/cli/src/ui/components/shared/text-buffer.part5.test.ts
  • packages/cli/src/ui/components/shared/text-buffer.test.ts
  • packages/cli/src/ui/containers/SessionController.test.tsx
  • packages/cli/src/ui/hooks/agentStream/__tests__/streamUtils.test.ts
  • packages/cli/src/ui/hooks/toolMapping.test.ts
  • packages/cli/src/ui/hooks/useAgentStream-test-helpers.ts
  • packages/cli/src/ui/hooks/useAgentStream.approval.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.cancellation.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.commands.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.finished.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.hooks.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.include.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.loopdetect.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.ordering.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.subagent.spec.tsx
  • packages/cli/src/ui/hooks/useAgentStream.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.thinking.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.thought.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.usercancel.test.tsx
  • packages/cli/src/ui/hooks/useAtCompletion.subagent.test.ts
  • packages/cli/src/ui/hooks/useAtCompletion.test.ts
  • packages/cli/src/ui/hooks/useFolderTrust.test.ts
  • packages/cli/src/ui/hooks/useGitBranchName.test.tsx
  • packages/cli/src/ui/hooks/useKeypress.test.tsx
  • packages/cli/src/ui/hooks/useLoadingIndicator.test.tsx
  • packages/cli/src/ui/themes/theme-compat.ts
  • packages/cli/src/ui/utils/MarkdownDisplay.test.tsx
  • packages/cli/src/ui/utils/clipboardUtils.test.ts
  • packages/cli/src/ui/utils/commandUtils.test.ts
  • packages/cli/src/ui/utils/mouse.test.ts
  • packages/cli/src/ui/utils/updateCheck.test.ts
  • packages/cli/src/utils/dynamicSettings.test.ts
  • packages/cli/src/utils/sandbox-bashrc.ts
  • packages/cli/src/utils/sandbox-entrypoint.test.ts
  • packages/cli/src/utils/sandbox-seatbelt.test.ts
  • packages/cli/src/utils/sessionCleanup.integration.test.ts
  • packages/cli/src/utils/userStartupWarnings.ts
  • packages/cli/src/utils/version.test.ts
  • packages/cli/src/utils/version.ts
  • packages/cli/stryker.conf.json
  • packages/cli/test-setup-base.ts
  • packages/cli/test-setup.ts
  • packages/cli/test-utils/ink-stub.ts
  • packages/cli/test-utils/ink-testing-library.ts
  • packages/cli/test/baseProvider.stateless.stub.test.ts
  • packages/cli/test/integration/auth-e2e.integration.test.ts
  • packages/cli/test/openai.stateless.stub.test.ts
  • packages/cli/test/openaiResponses.stateless.stub.test.ts
  • packages/cli/test/providers/providerAliases.test.ts
  • packages/cli/test/run-bun-tests.test.ts
  • packages/cli/test/ui/commands/authCommand-logout.test.ts
  • packages/cli/vitest.agentStream.config.ts
  • packages/cli/vitest.ci.covered.config.ts
  • packages/cli/vitest.ci.fast.config.ts
  • packages/cli/vitest.cli-integration.config.ts
  • packages/cli/vitest.config.integration.ts
  • packages/cli/vitest.config.mutation.ts
  • packages/cli/vitest.config.ts
  • packages/cli/vitest.integration.config.ts
  • packages/cli/vitest.test-groups.test.ts
  • packages/cli/vitest.test-groups.ts
  • packages/settings/src/settings/registry/registry-entries-2.ts
  • scripts/bun-test-manifest.ts
  • scripts/tests/bun-manifest-root-ownership.bun.test.ts
  • scripts/tests/bun-test-manifest.bun.test.ts
  • test-setup/augment-bun-vi.ts
  • test-setup/module-resolution.ts
  • test-setup/nested-mock-fixture.ts
  • test-setup/stub-helpers.ts
  • test-setup/vitest-parity.test.ts
💤 Files with no reviewable changes (36)
  • packages/cli/test/openai.stateless.stub.test.ts
  • packages/cli/test/openaiResponses.stateless.stub.test.ts
  • packages/cli/src/ui/components/shared/ScrollableList.theme.test.tsx
  • packages/cli/src/ui/components/AuthDialog.theme.test.tsx
  • packages/cli/test/baseProvider.stateless.stub.test.ts
  • packages/cli/src/ui/hooks/useAgentStream.approval.test.tsx
  • packages/cli/vitest.config.integration.ts
  • packages/cli/vitest.ci.covered.config.ts
  • packages/cli/vitest.cli-integration.config.ts
  • packages/cli/src/ui/components/shared/VirtualizedList.theme.test.tsx
  • packages/cli/vitest.config.ts
  • packages/cli/src/ui/App.test.tsx
  • packages/cli/src/ui/App.e2e.test.tsx
  • packages/cli/test/integration/auth-e2e.integration.test.ts
  • packages/cli/stryker.conf.json
  • packages/cli/vitest.integration.config.ts
  • packages/cli/vitest.config.mutation.ts
  • packages/cli/src/ui/App.components.test.tsx
  • packages/cli/vitest.agentStream.config.ts
  • packages/cli/test-setup-base.ts
  • packages/cli/src/ui/components/StatsDisplay.theme.test.tsx
  • packages/cli/src/ui/components/OAuthCodeDialog.test.tsx
  • packages/cli/src/ui/components/AboutBox.theme.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.hooks.test.tsx
  • packages/cli/src/ui/App.context.test.tsx
  • packages/cli/src/ui/components/messages/ToolConfirmationMessage.responsive.test.tsx
  • packages/cli/src/ui/hooks/useAgentStream.cancellation.test.tsx
  • packages/cli/src/config/auth.test.ts
  • packages/cli/src/ui/App.dialogs.test.tsx
  • packages/cli/test-setup.ts
  • packages/cli/vitest.test-groups.test.ts
  • packages/cli/src/ui/components/inputPromptTypes.ts
  • packages/cli/src/ui/hooks/useAgentStream.loopdetect.test.tsx
  • packages/cli/vitest.ci.fast.config.ts
  • packages/cli/src/ui/App.behavior.test.tsx
  • packages/cli/src/ui/components/Footer.responsive.test.tsx

Comment thread packages/cli/run-bun-tests.ts
Comment thread packages/cli/src/cli.startInteractiveUI.test.tsx
Comment thread packages/cli/src/config/config.integration.test.ts
Comment thread packages/cli/src/config/profileResolution.ts Outdated
Comment thread packages/cli/src/integration-tests/security.integration.test.ts Outdated
Comment thread packages/cli/src/test-utils/render.tsx
Comment thread packages/cli/test-utils/ink-stub.ts
Comment thread packages/cli/test/run-bun-tests.test.ts
Comment thread test-setup/augment-bun-vi.ts
Comment thread test-setup/augment-bun-vi.ts
@acoliver

acoliver commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

CodeRabbit triage and remediation update:

Blocker-Fix / In-scope-Fix addressed locally:

  • renderer override is reset after every startInteractiveUI test
  • process.stdout.write restores the exact original function reference
  • inline and file profile logs now share the same temporary-debug gate
  • keyfile test checks read access with R_OK
  • offered tool declarations use parametersJsonSchema
  • mock filesystem clear rethrows non-EEXIST mkdir errors
  • overlapping Ink renders retain the most recent remaining stdin, with a behavioral regression test
  • symlink cycle coverage remains enabled on Windows by using a directory junction rather than adding a skip
  • async mock factories distinguish fulfilled/rejected/pending states and fail fast when no safe placeholder exists

Reject findings:

  • Silently skipping unreadable discovery entries would violate this issue’s exhaustive/no-silent-omission contract; discovery should fail instead of hiding an uninspectable test path.
  • Rejecting interval zero is hypothetical compatibility hardening not exercised by any migrated repository test and is outside Migrate CLI workspace to Bun-native test execution (#2578) #2843’s accepted scope.
  • The accessor-spy claim is factually disproved on Bun 1.3.14 by current suites: trustedFolders 37/37, ideCommand 11/11, and setupGithubCommand 10/10 all exercise the three-argument process.platform spies successfully.

The agents CI flake was also root-caused: advanceTimersByTimeAsync can stall at its post-fake-time real setImmediate boundary on Linux; Bun then times out the test and teardown clears mock history, producing the misleading zero-call assertion. The focused test now uses synchronous fake-time advancement for its synchronous timeout callback and asserts the observable TIMEOUT result. It passed 10 stress runs plus the current focused run.

Verification on current local candidate: typecheck/build/lint/format pass; focused suites pass (agents 12/12, CLI start 9/9, config 19/19, logging 7/7, stdin ownership 1/1, runner 26/26, security 15 pass with 1 pre-existing platform skip, augment 21/21). A fresh CI run is still required after push; the user-triggered green rerun was on the prior head.

@acoliver
acoliver merged commit 8e721f9 into main Aug 6, 2026
67 of 69 checks passed
acoliver added a commit that referenced this pull request Aug 6, 2026
* Guard against silently unrun CLI test files (Fixes #2923)

The Vitest baseExclude contract this issue was filed against is gone: PR
#3056 replaced it with structural discovery in packages/cli/run-bun-tests.ts,
and all 670 tracked CLI test files are now discovered and run. What was
missing is the third resolution the issue asks for — something that makes a
future silent exclusion fail loudly.

run-bun-tests.ts walks a hardcoded TEST_ROOTS list, so a tracked test file
added anywhere else under packages/cli would never run while every existing
test still passed. The new guard compares the git-tracked test set against
the runner's own discoverTestFiles() and fails, naming the file, when the two
disagree or when a path is discovered more than once.

* Address review: prove sorting, distinguish maxBuffer overflow from timeout

The sorted-output test passed pre-sorted input, so it would still pass if
sorting were dropped. It now supplies unsorted input.

Node kills a child with SIGTERM for both a timeout and a maxBuffer overflow,
so the helper reported runaway output as a timeout and hid the real cause.
The overflow is now identified by its error code first, matching the
handling in scripts/tests/cli-import-boundary.test.ts.

* Report both discovery violations in a single run

evaluateDiscovery returned as soon as it found duplicates, so a run that had
both duplicates and undiscovered files only reported the duplicates. The
reader had to fix one, re-run, and only then learn about the other. Both are
now computed up front and every violation present is reported together.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer:e2e:ok Trusted contributor; maintainer-approved E2E run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate CLI workspace to Bun-native test execution (#2578)

1 participant