Skip to content

Fix settings sync coverage for releases (Fixes #3212) - #3214

Merged
acoliver merged 6 commits into
mainfrom
issue3212
Aug 13, 2026
Merged

Fix settings sync coverage for releases (Fixes #3212)#3214
acoliver merged 6 commits into
mainfrom
issue3212

Conversation

@acoliver

@acoliver acoliver commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

TLDR

Fixes the nightly release preflight failure by restoring the generated settings schema and CLI configuration documentation, then closes the PR CI gap that allowed stale settings artifacts to merge.

Settings-schema facade and modular-source changes now select the scripts test shard in addition to normal CLI coverage, ensuring the existing generated-settings synchronization test runs before merge.

Dive Deeper

The August 11 and August 12 scheduled releases failed during Run Preflight Checks, before publication. The scripts shard reported that schemas/settings.schema.json and docs/cli/configuration.md were stale relative to the canonical CLI settings schema.

The exact stale description originated in the modular settings-schema source merged by PR 3201. A later facade-level settings-schema change in PR 3207 also escaped the scripts shard, confirming that both schema source surfaces were missing from affected-test selection.

This PR:

  • regenerates the canonical JSON schema and CLI configuration reference;
  • adds generic checked-in path-observer rules for exact paths and directory prefixes;
  • selects scripts for packages/cli/src/config/settingsSchema.ts and packages/cli/src/config/settings-schema/ while preserving ordinary CLI owner and reverse-dependent coverage;
  • validates observer identities, target shards, path existence, and canonical directory-prefix syntax;
  • uses boundary-safe directory matching so sibling textual prefixes cannot overmatch;
  • adds behavioral tests for facade changes, modular changes, unrelated CLI changes, malformed prefixes, checker CLI errors, and prefix boundaries.

No release bypass or settings runtime behavior was added.

Reviewer Test Plan

  1. Run the focused behavioral checks:

    bun test scripts/tests/affected-test-shards.test.ts scripts/tests/affected-test-shards-prefix-boundary.test.ts scripts/tests/check-affected-test-shards.test.ts scripts/tests/generate-settings-doc.test.ts
    
  2. Validate the checked-in selector graph:

    bun scripts/check-affected-test-shards.ts
    
  3. Verify generated settings artifacts are synchronized:

    bun scripts/generate-settings-doc.ts --check
    
  4. Confirm selector behavior:

    • packages/cli/src/config/settingsSchema.ts selects cli and scripts.
    • packages/cli/src/config/settings-schema/schema-core.ts selects cli and scripts.
    • an unrelated CLI production source does not select scripts through the new rule.
  5. Run the standard lint, typecheck, format, build, and smoke gates.

Local focused tests, lint, typecheck, format, build, and the stepfun-37 smoke test pass. The full local macOS test command still reports existing PowerShell grammar and Homebrew ripgrep-path failures in unchanged packages/core tests; PR CI provides the authoritative Linux test signal.

Testing Matrix

🍏 🪟 🐧
npm run ⚠️ Existing macOS-only core test failures; issue-focused gates pass CI
npx
Docker
Podman - -
Seatbelt - -

Linked issues / bugs

Fixes #3212

Summary by CodeRabbit

  • Documentation

    • Clarified that the session retention size limit uses the configured default value across CLI documentation and settings references.
  • Improvements

    • Improved change detection for settings documentation and schema updates.
    • Added safer path matching and configuration validation to reduce missed or unnecessary checks.
    • Added an automated settings documentation/schema synchronization check to CI.
  • Tests

    • Expanded coverage for path matching, configuration validation, malformed inputs, and affected-check selection.

@github-actions github-actions Bot added the maintainer:e2e:ok Trusted contributor; maintainer-approved E2E run label Aug 12, 2026
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds path observers for exact files and directory descendants. Validates observer configuration and CLI arguments. Updates settings descriptions to reference the default property. Adds a CI settings-synchronization check and related tests.

Changes

Path observer shard selection

Layer / File(s) Summary
Observer configuration and selection
scripts/affected-test-shards.data.json, scripts/affected-test-shards.ts, scripts/tests/affected-test-shards.test.ts, scripts/tests/affected-test-shards-prefix-boundary.test.ts
Adds pathObservers rules. The selector matches exact paths and directory descendants, adds observer shards and reasons, and preserves full-run classifications.
Observer validation and CLI handling
scripts/check-affected-test-shards.ts, scripts/tests/check-affected-test-shards.test.ts
Validates observer targets, match criteria, canonical prefixes, filesystem paths, and CLI option values.
Settings documentation consistency
docs/cli/configuration.md, schemas/settings.schema.json
Replaces repeated 4096 MiB text with references to the default property while retaining the default metadata.
Settings synchronization CI gate
package.json, .github/workflows/ci.yml, scripts/tests/settings-sync-gate.test.ts
Adds the settings synchronization check to the JavaScript lint job and verifies its unconditional placement after build:types.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: 🔵 Low · up to 046d3

The PR restores generated settings checks and expands affected-test selection. It is mergeable with owner awareness of a bounded validation issue: non-canonical observer paths with duplicate separators are still accepted and should be corrected or explicitly accepted before merge.

Possibly related PRs

  • vybestack/llxprt-code#3103: Both PRs change scripts/check-affected-test-shards.ts path classification behavior and documentation.
  • vybestack/llxprt-code#3201: Both PRs update the sessionRetention.maxTotalSizeMB configuration documentation and schema descriptions.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: restoring settings synchronization coverage for releases and references issue #3212.
Description check ✅ Passed The description includes all required sections and provides clear scope, implementation details, testing steps, results, and linked issue information.
Linked Issues check ✅ Passed The changes address issue #3212 by ensuring settings schema changes select the scripts shard and run synchronization checks before release.
Out of Scope Changes check ✅ Passed The path-observer infrastructure, validation, tests, generated artifacts, and CI gate directly support the stated release-preflight objective.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue3212

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

Before this PR, release-time CI had no dedicated guard for settings-docs/schema drift, and the affected-test-shard selector lacked explicit boundary coverage for path-prefix rules, shard-selection behavior, and settings-sync correctness. That left a gap where a release could ship with mismatched settings documentation or schema without an unconditional CI failure, and where selector edge cases around prefix boundaries and affected shards were only implicitly covered. After this PR, CI always runs a settings-sync correctness gate, the selector’s path-observer rules are explicitly enforced so settings-schema source changes select the proper scripts test shard, and new behavioral tests lock in prefix-boundary safety, affected-shard selection, and the settings-sync gate behavior. In addition, the restore command now sorts checkpoint filenames deterministically, making restore suggestions and listings stable and predictable.

Release Notes

New Features

  • Added an unconditional lint:settings-sync CI guard and npm script so settings docs/schema drift fails releases consistently.
  • Introduced path-observer rules that ensure settings-schema source changes correctly select the scripts test shard.
  • Restore suggestions and checkpoint listings now sort filenames deterministically.

Bug Fixes

  • Closed the release-path gap where settings documentation and schema could drift without an unconditional CI failure.

Tests

  • Added behavioral tests for path-observer prefix-boundary safety.
  • Added behavioral tests for affected-shard selection correctness.
  • Added behavioral tests for the settings-sync correctness gate.
  • Added behavioral tests for the affected-test-shards drift-check path.

Documentation

  • Updated configuration docs to reflect release settings-sync behavior.
  • Updated issue planning notes for the settings-sync coverage work.
  • Updated the settings schema to match the documented release sync behavior.

Refactor

  • Hardened the affected-test-shard selector with explicit path-boundary and observer-rule behavior covered by tests.

Chore

  • Wired the new settings-sync guard into CI and package scripts.

Changes

Layer File(s) Summary
ci .github/workflows/ci.yml, package.json Adds an unconditional lint:settings-sync CI guard and its npm script to prevent settings docs/schema drift in releases
core scripts/affected-test-shards.ts, scripts/check-affected-test-shards.ts, scripts/affected-test-shards.data.json Introduces path-observer rules so settings-schema source changes correctly select the scripts test shard
tests scripts/tests/affected-test-shards-prefix-boundary.test.ts, scripts/tests/affected-test-shards.test.ts, scripts/tests/settings-sync-gate.test.ts, scripts/tests/check-affected-test-shards.test.ts Adds behavioral tests covering path-observer boundary safety, affected-shard selection, and the settings-sync correctness gate
docs docs/cli/configuration.md, project-plans/issue3212/PLAN.md, schemas/settings.schema.json Updates configuration docs, issue planning, and the settings schema to reflect release sync behavior
cli packages/cli/src/ui/commands/restoreCommand.ts Sorts checkpoint filenames deterministically in restore suggestions and listings

Magnitude

🎯 2 (M)
1176 additions, 41 deletions, 13 changed files across 1 package, 0 acceptance criteria

Related

No related items found.

Pre-merge Checks

Check Status Note
Title Clear and descriptive title that identifies the fix (settings sync coverage for releases) and references the linked issue #3212.
Description Includes all required template sections: TLDR, Dive Deeper, Reviewer Test Plan, Testing Matrix, and Linked issues / bugs. Provides specific test commands, validation steps, and CI context.
Linked Issues Addresses the root cause of issue #3212 (release failure due to stale settings artifacts) by regenerating artifacts, adding an unconditional CI guard (lint:settings-sync), and fixing affected-test selection via path observers so settings-schema changes trigger the scripts shard.
Out of Scope packages/cli/src/ui/commands/restoreCommand.ts sorts checkpoint JSON filenames for deterministic ordering; this change is unrelated to the settings synchronization issue (#3212) and release preflight failures.

Walkthrough generated by LLxprt PR Review. Planner issue: #2256

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/check-affected-test-shards.ts`:
- Around line 580-600: Update the argument parsing for --root and --data to
reject a following recognized option token as a missing value, rather than
resolving it as the path. Preserve existing missing-value errors and defaults,
and add regression coverage for both --root --data file.json and --data --root
repo.
- Around line 398-405: Update isValidCanonicalPrefix to reject Windows
drive-qualified prefixes matching a letter, colon, and slash before the existing
canonical-prefix checks; preserve validation for other prefixes. Add a
regression test asserting that values such as “C:/” are invalid.

In `@scripts/tests/affected-test-shards.test.ts`:
- Around line 100-109: Update expectPathReasonKeywords so each keyword is
lowercased before the case-insensitive containment check, matching the existing
lowercasing of pr!.reason and preserving the helper’s contract.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c4a43f59-f642-4e92-9b93-2352d81b1cb6

📥 Commits

Reviewing files that changed from the base of the PR and between 72f606a and 7f3da2d.

⛔ Files ignored due to path filters (1)
  • project-plans/issue3212/PLAN.md is excluded by !project-plans/**
📒 Files selected for processing (8)
  • docs/cli/configuration.md
  • schemas/settings.schema.json
  • scripts/affected-test-shards.data.json
  • scripts/affected-test-shards.ts
  • scripts/check-affected-test-shards.ts
  • scripts/tests/affected-test-shards-prefix-boundary.test.ts
  • scripts/tests/affected-test-shards.test.ts
  • scripts/tests/check-affected-test-shards.test.ts

Comment thread scripts/check-affected-test-shards.ts Outdated
Comment thread scripts/check-affected-test-shards.ts Outdated
Comment thread scripts/tests/affected-test-shards.test.ts
Comment thread scripts/check-affected-test-shards.ts Outdated
Comment thread scripts/check-affected-test-shards.ts Outdated
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

OpenCodeReview — automatic reviews suspended

Automatic OCR reviews are suspended for this PR after 2 of 2 automatic reviews.

To get more reviews you can:

  • Check the box below to re-enable automatic reviews (resets the counter), or

  • Comment /review, /ocr, or /open-code-review to request a single review on demand.

  • Re-enable automatic reviews

@acoliver

Copy link
Copy Markdown
Collaborator Author

Addressed the option-as-value finding in commit 7683ede. The checker now treats recognized option tokens as missing values for both --root and --data, with subprocess coverage for both reported forms.

@acoliver

Copy link
Copy Markdown
Collaborator Author

Addressed the Windows drive-prefix finding in commit 7683ede. Canonical prefixes now reject drive-qualified forward-slash paths such as C:/, with Bun regression coverage.

@acoliver

Copy link
Copy Markdown
Collaborator Author

Addressed the filesystem race finding in commit 7683ede. Prefix validation performs one narrowly scoped stat operation and reports missing, non-directory, or raced paths as drift. Direct tests cover nonexistent and existing-file prefixes.

@acoliver

Copy link
Copy Markdown
Collaborator Author

Addressed the case-insensitive helper finding in commit 7683ede. Both the path reason and expected keyword are normalized, and the selector test now uses mixed-case keywords.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
scripts/check-affected-test-shards.ts (1)

405-407: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject empty path segments in observer paths.

packages//cli/ and packages//cli/file.ts pass these validators. join() normalizes the duplicate separator before statSync, so the checker accepts a non-canonical observer rule. Reject empty segments so path observers use the same canonical path form as changed-file paths.

Proposed fix
 export function isValidDirectoryPrefix(prefix: string): boolean {
   if (!prefix.endsWith('/')) return false;
   if (prefix.startsWith('/')) return false;
   if (prefix.includes('\\')) return false;
   if (/^[a-z]:\//i.test(prefix)) return false;
-  return !prefix
+  return !prefix.slice(0, -1)
     .split('/')
-    .some((segment) => segment === '..' || segment === '.');
+    .some(
+      (segment) =>
+        segment.length === 0 || segment === '..' || segment === '.',
+    );
 }
 
 export function isValidExactPath(path: string): boolean {
   // ...
   return !path
     .split('/')
-    .some((segment) => segment === '..' || segment === '.');
+    .some(
+      (segment) =>
+        segment.length === 0 || segment === '..' || segment === '.',
+    );
 }

Also applies to: 428-430

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/check-affected-test-shards.ts` around lines 405 - 407, Update the
observer-path validation around the prefix segment check to reject empty path
segments in addition to "." and "..". Ensure paths such as packages//cli/ and
packages//cli/file.ts are rejected before join/stat normalization, while
preserving acceptance of canonical relative paths.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@scripts/check-affected-test-shards.ts`:
- Around line 405-407: Update the observer-path validation around the prefix
segment check to reject empty path segments in addition to "." and "..". Ensure
paths such as packages//cli/ and packages//cli/file.ts are rejected before
join/stat normalization, while preserving acceptance of canonical relative
paths.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a43ad9a-ac59-434c-a598-a03a97701e24

📥 Commits

Reviewing files that changed from the base of the PR and between 7683ede and 046d344.

⛔ Files ignored due to path filters (1)
  • project-plans/issue3212/PLAN.md is excluded by !project-plans/**
📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • package.json
  • scripts/check-affected-test-shards.ts
  • scripts/tests/check-affected-test-shards.test.ts
  • scripts/tests/settings-sync-gate.test.ts

@acoliver

Copy link
Copy Markdown
Collaborator Author

Addressed the duplicate-separator validation gap in commit ca262c2. Exact paths and directory prefixes now reject empty interior path segments, with Bun regression tests covering both forms.

@acoliver
acoliver merged commit a8b59e1 into main Aug 13, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer:e2e:ok Trusted contributor; maintainer-approved E2E run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release Failed for v0.11.0-nightly.260812.8f34d810f on 2026-08-12

1 participant