docs: sprint retrospective improvements (2026-07-27)

[ms2sato]

Sprint 2026-07-18b retrospective improvements. 19 PRs merged; the sprint's defining event was main sitting red for four days without anyone observing it.

Each change below is traceable to something that actually went wrong, or to something that was caught by luck and should not have to be next time.

1. CodeRabbit's fourth surface

.claude/skills/coderabbit-ops/SKILL.md, .claude/rules/workflow.md

The skill documents a 3-layer verdict (Pre-merge checks / reviewDecision / inline comments). None of the three is the CodeRabbit commit-status context — which nonetheless appears in gh pr view --json statusCheckRollup, the command everyone runs to confirm CI, rendered next to test and preflight as CodeRabbit=SUCCESS.

Its state is success in all three of these cases:

description	Reality
Review completed	reviewed
Review rate limited	never reviewed
Review skipped: ignored keyword in the PR title	skipped on purpose by .coderabbit.yaml

Three PRs this sprint (#1227, #1231, #1229) showed a green rollup with no review behind it. #1227 later produced a Major finding — a factually wrong claim about spyOn isolation that would have shipped in a standards document.

Adds the read command, the table, and a note that the status is per-head-SHA so updating a branch invalidates an earlier Review completed.

2. Re-running the bootstrap script against a live deployment

docs/multi-user-setup-guide.md

setup-multiuser-for-ubuntu.sh refuses to overwrite a differing systemd unit and tells you to re-run with --force. --force then replaces every parameter not passed explicitly with the script's defaults. The live service ran on a non-default port; a bare --force would have moved it and broken the URL in use.

Adds: read the live unit first, preview with --dry-run, carry over everything that differs. Plus the more general point — prefer the narrowest operation. The provisioning step actually needed was one install command; re-running the whole bootstrap was the wrong tool.

3. Consult permission is not a waiver of the report

.claude/skills/orchestrator/SKILL.md

A delegation prompt said "you may consult the architect directly — no need to route through me", meaning no pre-approval. The Architect then ruled on whether a PR required a production-side extraction — the thing that decides whether it is Orchestrator-mergeable or owner-gated — and asked for the outcome to reach the Orchestrator. It never did.

Asked directly, the delegate confirmed they knew who the Orchestrator was and had reported everything else correctly. The gap was the prompt. Adds a two-sentence template and names why the collapse happens: a worker who resolves a question has, from their side, handled it.

4-5. Two new Inference vs Verification sub-patterns

.claude/rules/workflow.md

Sub-pattern 5 — absence of a signal. The existing four sub-patterns all cover over-trusting a weak signal. These are the inverse: waiting on something with no path to arrive. ci.yml fires on pull_request only, so a push to a branch with no PR runs nothing — a delegate waited ~40 minutes after removing a temporary debug workflow that had a push trigger. And activityState: idle cannot distinguish an agent that replied from one that is stuck.

Sub-pattern 6 — hypothesis exhaustion. Four single-mechanism hypotheses for #1225 were disproven in sequence, each individually reasonable. What resolved it was re-running the last green CI run byte-identically (exonerating the repository in one step, isolating the runner image as the only changed variable) and then delta-debugging the real file. #1211 was settled the same day by the same shape: the identical config and binary run outside Docker. Codifies: pair every negative with a positive control, stop generating after ~3 disproven, prefer the control experiment, go wide when iterations cost a round trip.

6. Check main when the pipeline goes quiet

.claude/skills/orchestrator/sprint-lifecycle.md

main is only exercised when something merges. During a wait — an owner gate, a dogfood pass — nothing merges, so nothing runs, so a red main produces no signal. It is not that the failure is missed; no observation is being made.

It surfaced only because unrelated backfill work was dispatched and its CI failed. Four completed PRs had been silently unmergeable for days. Also notes the cause need not be in the repository.

7. Is the AC's own verification still satisfiable?

.claude/rules/pre-pr-completeness.md Q12

An AC mandating a real-environment check ("run the Docker smoke on a fresh checkout") asserts that check is currently runnable — an assumption nobody validates when writing it. On PR #1228, implementation finished and then the smoke could not run: a second, unrelated defect broke bun install on any fresh clone.

Q12 runs before implementation, unlike every other question here — run the verification once against unmodified main to see it execute, not to see it pass. Registered in "When to apply" including the not-required carve-out, since a single-file fix can still carry a real-environment AC.

Also

Architect-unresponsive escalation (orchestrator/SKILL.md): judge on whether the Architect produced output for anyone, not on session state; escalate to a fresh generation rather than re-sending.

Deferred to Issues

• TTM measurement — sprint-retro.js's time-to-mergeable is wall-clock and is dominated by owner gates, main outages, and bot rate limits. All five flagged outliers this sprint were waiting, and none matched the interpretation the script suggests. Needs a script change, not a doc change.
• Architect unresponsiveness detection — the procedure above is manual. Surfacing per-session last-output time is product work.

Test plan

• bun run check:lang — clean, 114 files
• node .claude/skills/orchestrator/rule-skill-duplication-check.js — no rule paragraphs duplicated into skills
• CI

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (1)

• docs:

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a691f5ed-1ff6-41a9-8656-318a479478b2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch worktree-docs+sprint-retro-2026-07-27

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Test Coverage Check

No production files matching coverage patterns were changed.

---

Rule/Skill Duplication Check

✅ No rule paragraphs found verbatim in any skill file.

---

Language Check (public artifacts)

✅ All public artifacts use Latin / Greek / Cyrillic scripts only.

---

Source-Comment Blame-Shift Check

✅ No new Issue / PR / dated CodeRabbit references in source comments.