Skip to content

fix(commands,workflows): make the planning step read issue comments as authoritative - #2404

Merged
Wirasm merged 1 commit into
devfrom
fix/planning-reads-comments
Aug 3, 2026
Merged

fix(commands,workflows): make the planning step read issue comments as authoritative#2404
Wirasm merged 1 commit into
devfrom
fix/planning-reads-comments

Conversation

@Wirasm

@Wirasm Wirasm commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Problem: three separate failures in the fix-github-issue pipeline that all shared one shape — the run reported success while doing materially less than intended.
  • Why it matters: each one degraded the pipeline silently. A discarded PR, a review phase reduced from five reviewers to one, and a review scope resolved against a PR that does not exist — all with a green run.
  • What changed: planning commands now treat issue comments as authoritative over the body and follow linked issues; review-gate fails safe toward more review; the PR number is captured deterministically instead of by the model.
  • What did NOT change (scope boundary): no engine changes. No new node types, no schema changes, no executor changes. Two bundled command prompts, one experimental workflow, and the regenerated bundle.

The three fixes

1. Planning ignored decisions recorded in comments (bundled commands — affects all users)

fetch-issue already pulls comments, and archon-investigate-issue already listed them under "Extract Context". The gap was authority, not availability.

On 2026-08-03, a run on #2359 implemented the issue body while a maintainer comment on that same issue specified a different design. The comment was posted at 07:54:13Z, the run started at 07:54:29Z, and the contract text (startedAt, --events, replaces, first appearance) was confirmed present in the run's fetch-issue output. The resulting PR (#2399) was closed.

Both planning entry points — archon-investigate-issue (bugs) and archon-create-plan (everything else) — now state:

  • read every comment before planning; a later maintainer comment supersedes the body
  • a comment stating a decision is the specification — do not re-derive it, do not compromise between it and the body
  • follow linked issues (#1234 or URL) one level and read their comments too
  • if body and decision conflict, record which was followed — a silent choice is the failure mode

The incident is written into both files. A rule with its origin attached survives; an abstract instruction gets skimmed.

Note: the workflow cannot inject this per-node. commandNodeSchema is { command: string } with no input channel, so a command: node renders the shared file and the only way to reach the planner is the prompt itself.

2. A failed gate silently reduced review coverage (experimental workflow)

review-gate read $smoke-validate.output.claims_accurate via strict field access. When smoke-validate timed out and emitted nothing, the strict ref failed the gate, every conditional reviewer skipped via trigger_rule, and the run completed successfully with one reviewer instead of the intended set. Observed on run 7f392748.

Now: reads the whole output and greps, runs under trigger_rule: all_done so a failed producer still reaches the branch, and suppresses extra reviewers only when scope is small AND the claims were positively confirmed. Missing output, timeout, or failure all bias toward more review.

3. The PR number was recorded by the model, and was usually wrong (experimental workflow)

create-pr asked the model to run gh pr view --json number and echo the result into $ARTIFACTS_DIR/.pr-number. Being a prompt instruction, it is advisory. Measured across four parallel runs:

run issue .pr-number written actual PR
3b03cb58 #2384 missing #2389
c77ed1bc #2323 2323 (issue no.) #2388
500fa086 #2279 2279 (issue no.) #2390
bd0db0cd #2385 2387 #2387

review-scope then resolved a nonexistent PR, correctly refused to fabricate a scope.md, and review-classify declined every specialist for lack of evidence. Three of four PRs received one generalist reviewer while the runs reported success.

Replaced with a deterministic capture-pr-number bash node that resolves the PR from the branch and exits non-zero if it cannot. Verified in the exact worktree that failed: the node returns 2388 where the model had written 2323.

Per the workflow constitution this is the reliability carve-out, not a style preference — a check with no judgment content whose silent failure guts the review phase belongs in a node that cannot decline to fire.

synthesize also moves one_successall_done. Defensive rather than causal, but one_success permits a synthesis to stand on a subset and report it as the whole review.

Validation Evidence

bun run check:bundled     # up to date (36 commands, 21 workflows)
bun run format:check      # clean
bun run cli validate workflows archon-fix-github-issue-experimental   # 1 valid, 0 errors
bun run cli validate commands archon-investigate-issue                # ok
bun run cli validate commands archon-create-plan                      # ok

The single remaining validator warning is pre-existing on fetch-issue and unrelated to this change.

End-to-end evidence for fix 3 — run abd3370c on #2344 with the deterministic node in place:

before (4 runs) after
.pr-number wrong ×2, missing ×1, right ×1 2391
scope.md missing in 3 of 4 3,524 b
classifier requested code_review only (3 of 4) code_review, error_handling, test_coverage

Fix 1 verification is in flight — a re-run of #2359 with the corrected planning prompts was still executing when this PR was opened. If it produces the contract shape (nodes replacing events, a --events flag, startedAt), the instruction works. If it reproduces the old shape, the prompt is insufficient and this PR should not merge on the strength of reasoning alone.

Security Impact

  • New permissions/capabilities? No
  • New external network calls? Nocapture-pr-number uses gh pr view, which create-pr already invoked at the same point
  • Secrets/tokens handling changed? No
  • File system access scope changed? No

Compatibility / Migration

  • Backward compatible? Yes. A repo-level .archon/commands/archon-*.md still overrides the bundled default by name.
  • Config/env changes? No
  • Database migration? No

Blast radius

The two command changes are bundled defaults, so they affect every workflow invoking those commands — the whole fix-github-issue family. They only add instruction; no behaviour is removed. The failure mode if the wording is wrong is a planner that over-weights a comment, which is far cheaper than the current failure mode of ignoring a decision entirely.

The workflow changes are confined to .archon/workflows/experimental/.

Rollback

git revert — prompts and one workflow file, no state, no migration. Operators can override either command locally without waiting for a release.

Risks and Mitigations

Summary by CodeRabbit

  • New Features

    • Improved GitHub issue investigations and planning by incorporating comments, maintainer decisions, and relevant linked issues.
    • Added conflict documentation when issue descriptions and later decisions differ.
    • Added automated validation of issue claims and pull request creation.
  • Bug Fixes

    • Workflows now stop early when no implementation changes are detected.
    • Improved review and reporting reliability with clearer validation gates and completion checks.

…s authoritative

Three reliability fixes to the fix-github-issue pipeline, all found by running it
against real issues and reading the resulting event streams.

**Planning ignored decisions recorded in comments.** `fetch-issue` already pulls
`comments`, and `archon-investigate-issue` already listed them as context — but
nothing said they outrank the issue body. On 2026-08-03 a run implemented an
issue's body while a maintainer comment on that same issue, present in the run's
fetched input and posted 16 seconds before it started, specified a different
design. The PR was discarded. Both planning entry points now state that a later
maintainer comment supersedes the body, that a comment stating a decision IS the
specification, and that linked issues are followed one level for their decisions.

**A failed gate silently reduced review coverage.** `review-gate` read
`$smoke-validate.output.claims_accurate` with strict field access. When
smoke-validate timed out and emitted nothing, the gate failed, every conditional
reviewer skipped via trigger_rule, and the run still reported success with one
reviewer instead of the intended set. The gate now reads the whole output
defensively, runs under `trigger_rule: all_done` so a failed producer still
reaches it, and biases toward MORE review whenever the claims are not positively
confirmed.

**The PR number was recorded by the model and was usually wrong.** `create-pr`
asked the model to echo `gh pr view --json number` into `$ARTIFACTS_DIR/.pr-number`.
Across four parallel runs it wrote the ISSUE number twice, skipped the file once,
and was correct once — so `review-scope` resolved a nonexistent PR, correctly
refused to fabricate a scope, and the classifier then declined every specialist
for lack of evidence. Three of four PRs got one generalist reviewer while the run
reported success. Replaced with a deterministic bash node that resolves the PR
from the branch and fails loudly. Verified in the exact worktree that failed: the
node returns 2388 where the model wrote 2323.

`synthesize` also moves from `one_success` to `all_done` — defensive rather than
causal, but `one_success` permits a synthesis to stand on a subset and present it
as the whole review.
@Wirasm Wirasm added bug Something is broken area: workflows Workflow engine labels Aug 3, 2026
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds issue-comment authority rules and one-level linked-issue review. It also updates the experimental workflow with dependency preparation, typed outputs, implementation and PR guards, and fail-safe conditional review orchestration.

Changes

Issue-aware workflow

Layer / File(s) Summary
Issue decision guidance
.archon/commands/defaults/archon-create-plan.md, .archon/commands/defaults/archon-investigate-issue.md
The commands read all issue comments, prioritize later maintainer decisions, review linked issues one level deep, and document conflicts with the issue body.
Workflow preparation and contracts
.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml
The workflow detects the package manager, installs locked dependencies, includes the operator request in classification, uses updated model tiers, and declares investigation and planning output types.
Implementation and PR capture
.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml
The workflow verifies implementation changes outside .archon/, then deterministically captures and validates the draft pull request number and URL.
Conditional review and reporting
.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml
A fail-safe review gate controls specialist reviewers. Review synthesis waits for all reviewer terminal states and declares typed synthesis and completion-report outputs.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant Workflow
  participant Worktree
  participant GitHub
  Operator->>Workflow: Submit issue request
  Workflow->>Worktree: Install locked dependencies
  Workflow->>Worktree: Implement issue
  Workflow->>Worktree: Verify changes or commits
  Workflow->>GitHub: Create draft PR
  Workflow->>GitHub: Capture PR number and URL
  Workflow->>Workflow: Run smoke validation and review gate
Loading

Possibly related PRs

  • coleam00/Archon#1263: Updates default command files that the generator scans and embeds.
  • coleam00/Archon#1430: Shares review gating, conditional reviewers, synthesis, and reporting workflow patterns.
  • coleam00/Archon#2147: Shares deterministic PR handling and conditional review orchestration changes.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes one major change and directly matches the authoritative issue-comment updates, although it does not mention the other reliability fixes.
Description check ✅ Passed The description is detailed and relevant, covering the problem, changes, scope, validation, security, compatibility, rollback, blast radius, and risks.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/planning-reads-comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml (1)

409-435: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Superseded advisory PR-number writing in create-pr should be simplified.

capture-pr-number now deterministically overwrites $ARTIFACTS_DIR/.pr-number and validates the result, which fully mitigates the failure modes this comment block documents. create-pr's own Step 7 (unchanged, outside this diff) still instructs the model to compute and write $ARTIFACTS_DIR/.pr-number itself — the exact advisory mechanism this new node was added to replace. Since capture-pr-number always runs afterward and unconditionally overwrites the file, Step 7 is now dead weight that only adds confusion about which node is authoritative for PR identity.

Consider removing or simplifying create-pr's Step 7 to avoid the redundant, now-superseded instruction.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml
around lines 409 - 435, Remove or simplify Step 7 in the create-pr workflow so
it no longer instructs the model to compute or write $ARTIFACTS_DIR/.pr-number.
Keep capture-pr-number as the sole authoritative PR-number capture and
validation step, preserving its existing behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.archon/commands/defaults/archon-create-plan.md:
- Around line 39-45: Update the input classification table in
archon-create-plan.md to add GitHub issue number and URL cases before the
Free-form text branch. Match the number and URL detection patterns and routing
used in archon-investigate-issue.md, ensuring these inputs reach the
comment-fetch path before Phase 0.3.
- Around line 47-52: The comment-authority rules in
.archon/commands/defaults/archon-create-plan.md lines 47-52 and
.archon/commands/defaults/archon-investigate-issue.md lines 54-62 must require
verified maintainer authorship before treating a comment as authoritative, and
must state that the latest verified maintainer decision takes precedence. Update
both command documents consistently while preserving the existing planning and
investigation guidance.

In @.archon/commands/defaults/archon-investigate-issue.md:
- Around line 63-68: Update the linked-issue guidance in
.archon/commands/defaults/archon-investigate-issue.md lines 63-68 and
.archon/commands/defaults/archon-create-plan.md lines 53-55 to preserve the
owner/repo parsed from full GitHub issue URLs and demonstrate gh issue view with
--repo owner/repo for URL references; explicitly distinguish this from bare
`#1234` references, which should use the current repository.

In @.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml:
- Around line 309-346: Update the rationale comment above assert-implemented to
accurately state that the guard checks both uncommitted working-tree changes
outside .archon/ and commits ahead of origin/$BASE_BRANCH via the commits count.
Remove or revise the claim that only the working tree is inspected, and explain
the commit-count fallback’s purpose without changing the bash logic.
- Around line 58-75: Update the prep-worktree prompt so a failed locked
dependency installation deterministically fails the node rather than merely
being reported in text. In the prep-worktree instructions, require the model to
stop and explicitly fail/exit when the install command returns non-zero, or add
a follow-up deterministic shell check that propagates the install exit status;
preserve the existing manager detection and successful-install reporting
behavior.

---

Nitpick comments:
In @.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml:
- Around line 409-435: Remove or simplify Step 7 in the create-pr workflow so it
no longer instructs the model to compute or write $ARTIFACTS_DIR/.pr-number.
Keep capture-pr-number as the sole authoritative PR-number capture and
validation step, preserving its existing behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 99f4bbfd-3015-4276-8cb2-0c6a67179c66

📥 Commits

Reviewing files that changed from the base of the PR and between 10f95ce and 2e9219a.

⛔ Files ignored due to path filters (1)
  • packages/workflows/src/defaults/bundled-defaults.generated.ts is excluded by !**/*.generated.*
📒 Files selected for processing (3)
  • .archon/commands/defaults/archon-create-plan.md
  • .archon/commands/defaults/archon-investigate-issue.md
  • .archon/workflows/experimental/archon-fix-github-issue-experimental.yaml

Comment on lines +39 to +45
### 0.1a If the input is a GitHub issue: comments outrank the body

Fetch the issue **with its comments**, and treat them as authoritative:

```bash
gh issue view {number} --json title,body,labels,comments,state,url,author
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Classify GitHub issue inputs before the free-form branch.

The input table above lists only PRD, document, and free-form inputs. It has no GitHub issue number or URL case. Therefore, 1234 or a GitHub issue URL can enter Phase 0.3 and skip this new comment-fetch path. Add number and URL cases before Free-form text, matching .archon/commands/defaults/archon-investigate-issue.md Lines 30-32.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.archon/commands/defaults/archon-create-plan.md around lines 39 - 45, Update
the input classification table in archon-create-plan.md to add GitHub issue
number and URL cases before the Free-form text branch. Match the number and URL
detection patterns and routing used in archon-investigate-issue.md, ensuring
these inputs reach the comment-fetch path before Phase 0.3.

Comment on lines +47 to +52
- **Read every comment before planning.** A later maintainer comment supersedes
the body wherever they disagree — the body is the opening position, not the
specification.
- **A comment stating a decision IS the specification.** If one settles a shape,
a field list, an interface, or an explicit "do X, not Y", plan that. Do not
re-derive the choice from the body and do not compromise between them.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Files:\n'
for f in \
  ".archon/commands/defaults/archon-create-plan.md" \
  ".archon/commands/defaults/archon-investigate-issue.md" \
  ".archon/commands/defaults/archon-triage-stale.md"
do
  if [ -f "$f" ]; then
    echo "== $f =="
    wc -l "$f"
    ast-grep outline "$f" || true
  fi
done

printf '\nRelevant excerpts:\n'
rg -n -C 3 "verified|maintainer|specification|decision|latest|supersedes|comment" .archon/commands/defaults/archon-create-plan.md .archon/commands/defaults.archon-investigate-issue.md 2>/dev/null || true

Repository: coleam00/Archon

Length of output: 4251


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '== .archon/commands/defaults/archon-investigate-issue.md excerpts ==\n'
sed -n '1,110p' .archon/commands/defaults/archon-investigate-issue.md
sed -n '210,360p' .archon/commands/defaults/archon-investigate-issue.md
sed -n '530,602p' .archon/commands/defaults/archon-investigate-issue.md || true

printf '\n= maintainers/verification checks =\n'
rg -n "verification|trusted|maintainer|team|collaborator|read access|write access|GH\\(|gh issue|issue view|GitHub\\(|GitHub" .archon/commands/defaults/archon-create-plan.md .archon/commands/defaults/archon-investigate-issue.md || true

Repository: coleam00/Archon

Length of output: 12406


Clarify comment-authority in both planning commands.

Both commands say any maintainer comment outranks the issue body, but they do not require verifying that the comment author is an authorized maintainer. Restrict the specification rule to verified maintainer decisions and state latest-decision precedence.

📍 Affects 2 files
  • .archon/commands/defaults/archon-create-plan.md#L47-L52 (this comment)
  • .archon/commands/defaults/archon-investigate-issue.md#L54-L62
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.archon/commands/defaults/archon-create-plan.md around lines 47 - 52, The
comment-authority rules in .archon/commands/defaults/archon-create-plan.md lines
47-52 and .archon/commands/defaults/archon-investigate-issue.md lines 54-62 must
require verified maintainer authorship before treating a comment as
authoritative, and must state that the latest verified maintainer decision takes
precedence. Update both command documents consistently while preserving the
existing planning and investigation guidance.

Comment on lines +63 to +68
- **Follow linked issues.** When the body or a comment points at another issue
(`#1234`, or a full URL) for a decision, design, or contract, fetch it and read
its comments too:
```bash
gh issue view 1234 --json title,body,comments,state,url
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== changed files/stat =="
git diff --stat || true

echo "== target snippets =="
for f in .archon/commands/defaults/archon-investigate-issue.md .archon/commands/defaults/archon-create-plan.md; do
  if [ -f "$f" ]; then
    echo "--- $f"
    nl -ba "$f" | sed -n '45,80p'
  else
    echo "missing $f"
  fi
done

echo "== gh issue view occurrences =="
rg -n "gh issue view|url parsing|repo " .archon/commands/defaults/archon-*.md || true

echo "== all linked-issue guidance occurrences =="
rg -n "Follow linked issues|follow the link|gh issue view" .archon/commands/defaults || true

Repository: coleam00/Archon

Length of output: 299


🌐 Web query:

GitHub CLI gh issue view syntax --repo owner/repo full URL support

💡 Result:

The GitHub CLI (gh) issue view command supports both issue numbers and full URLs as valid arguments [1]. Syntax: gh issue view { | } [flags] - You can pass an issue number (e.g., gh issue view 123) or a direct URL to the issue (e.g., gh issue view https://github.com/owner/repo/issues/123) [1]. - When working with a repository other than the current directory's local repo, use the --repo (or -R) flag with the [HOST/]OWNER/REPO format [1][2]. Note on syntax constraints: - The command expects the issue number as an integer. Using the # prefix (e.g., gh issue view #123``) is generally not supported, as many shells interpret the # character as the start of a comment and strip it before the command receives the argument [3]. - While there have been community requests to support `owner/repo#number` shorthand syntax, it is not currently integrated into the standard `gh issue view` command interface [4]. When viewing issues, you can use the `--web` flag to open the issue directly in your default web browser rather than displaying it in the terminal [1][2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== changed files/stat =="
git diff --stat || true

for f in .archon/commands/defaults/archon-investigate-issue.md .archon/commands/defaults/archon-create-plan.md; do
  if [ -f "$f" ]; then
    echo "--- $f"
    awk '{printf "%6d  %s\n", NR, $0}' "$f" | sed -n '45,80p'
  else
    echo "missing $f"
  fi
done

echo "== relevant occurrences in target files =="
grep -nE "Follow linked issues|gh issue view|owner/repo|`#1234`|issue URL|URL" .archon/commands/defaults/archon-investigate-issue.md .archon/commands/defaults/archon-create-plan.md || true

echo "== broader linked-issue guidance locations =="
grep -RInE "Follow linked issues|follow the link|gh issue view|owner/repo|`#1234`" .archon/commands/defaults || true

Repository: coleam00/Archon

Length of output: 6593


Preserve the linked issue repository when the reference is a full URL.

gh issue view accepts a full GitHub issue URL, so the #1234 example does not cover cross-repository linked issues. In both command docs, make URL parsing preserve owner/repo and show that URL references include --repo owner/repo, while bare #1234 uses the current repository.

📍 Affects 2 files
  • .archon/commands/defaults/archon-investigate-issue.md#L63-L68 (this comment)
  • .archon/commands/defaults/archon-create-plan.md#L53-L55
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.archon/commands/defaults/archon-investigate-issue.md around lines 63 - 68,
Update the linked-issue guidance in
.archon/commands/defaults/archon-investigate-issue.md lines 63-68 and
.archon/commands/defaults/archon-create-plan.md lines 53-55 to preserve the
owner/repo parsed from full GitHub issue URLs and demonstrate gh issue view with
--repo owner/repo for URL references; explicitly distinguish this from bare
`#1234` references, which should use the current repository.

Comment on lines +58 to +75
- id: prep-worktree
prompt: |
Install this project's dependencies so builds, type-checks and tests can run
in this fresh worktree.

Follow this order:

1. Locate the package-manager markers actually present in the repo — lockfiles
and manifests. Decide from evidence, never from convention.
2. Read the manifest head to confirm the manager and any pinned version.
3. Verify that manager responds to a version check before relying on it.
4. Install using the manager's locked mode — the variant that reproduces the
lockfile exactly and fails rather than updating it. The install must not
modify the lockfile.
5. Report one line: the manager you detected, and whether the install succeeded.

Bound every read; never print a whole file.
model: small

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Add a deterministic failure check to prep-worktree.

This node only asks the model to "Report one line: the manager you detected, and whether the install succeeded." A prompt-based report is advisory: if the model's turn completes normally (even after the install command exits non-zero), the node itself is not forced to fail. implement (Line 304) then proceeds against a broken or partially-installed dependency tree, and later build/type-check/test failures get misattributed to the implementation instead of the environment.

This is the same failure mode the PR explicitly calls out and fixes elsewhere in this file: capture-pr-number's rationale states create-pr's advisory reporting is unreliable because "Being a prompt instruction, it is advisory," and assert-implemented's rationale exists specifically because a node "can decline to change anything ... and still report success." Apply the same principle here: instruct the model to explicitly fail/exit the turn (or add a follow-up deterministic bash check of the install command's exit status) when the locked install fails, instead of relying on narrative text.

Proposed prompt addition
       4. Install using the manager's locked mode — the variant that reproduces the
          lockfile exactly and fails rather than updating it. The install must not
          modify the lockfile.
-      5. Report one line: the manager you detected, and whether the install succeeded.
+      5. If the install command exits non-zero, treat this as a hard failure of this
+         task — do not continue as if it succeeded.
+      6. Report one line: the manager you detected, and whether the install succeeded.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml
around lines 58 - 75, Update the prep-worktree prompt so a failed locked
dependency installation deterministically fails the node rather than merely
being reported in text. In the prep-worktree instructions, require the model to
stop and explicitly fail/exit when the install command returns non-zero, or add
a follow-up deterministic shell check that propagates the install exit status;
preserve the existing manager detection and successful-install reporting
behavior.

Comment on lines +309 to +346
#
# `implement` can decline to change anything — a blocked precondition, an unclear
# plan — and still report success, so the run continued through create-pr and the
# whole review fleet on an empty diff, then reported `completed`. Nine nodes ran
# after the work had already failed. This asks the only question that matters
# deterministically: is there a change to review?
#
# Exiting non-zero fails the node, and therefore the run, before any reviewer or
# PR node spends anything. A run with nothing to show should fail, not go green.
# `.archon/` is excluded deliberately. Archon copies the operator's workflow and
# command edits into every run worktree — that is the feature that lets a workflow
# be iterated on before it is committed — so those files are present BEFORE
# `implement` runs and are not its output. A first version of this guard counted
# them and passed three consecutive runs on an empty implementation, which is the
# exact failure it exists to catch.
#
# Only the working tree is inspected: `implement` leaves its changes uncommitted
# (committing is `create-pr`'s job). If that ever stops being true this guard fails
# a run that did have work — visible and annoying — rather than passing one that
# did not, which is silent. That is the safe direction for it to be wrong in.
- id: assert-implemented
bash: |
base=$BASE_BRANCH
tracked=$(git diff --name-only HEAD -- ':(exclude).archon' | head -1)
untracked=$(git ls-files --others --exclude-standard -- ':(exclude).archon' | head -1)
if commits=$(git rev-list --count "origin/$base..HEAD" 2>/dev/null); then :; else commits=unknown; fi

if [ -n "$tracked" ] || [ -n "$untracked" ]; then
git --no-pager diff --stat HEAD -- ':(exclude).archon' | tail -1
elif [ "$commits" = "unknown" ]; then
echo "WARNING: cannot compare against origin/$base — not failing on an unknown"
elif [ "$commits" != "0" ]; then
echo "$commits commit(s) ahead of origin/$base"
else
echo "implement produced neither a commit nor a working-tree change outside .archon/ — failing before the review phase" >&2
exit 1
fi
depends_on: [implement]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Update the stale rationale comment to match the guard's actual logic.

The comment block states: "Only the working tree is inspected: implement leaves its changes uncommitted (committing is create-pr's job). If that ever stops being true this guard fails a run that did have work — visible and annoying — rather than passing one that did not, which is silent."

The bash below it does not match this description: it also computes commits via git rev-list --count "origin/$base..HEAD" (Line 334) and treats a non-zero commit count as evidence of implementation (Lines 340-341). So the guard checks both the working tree and commits ahead of the base branch, not "only the working tree." A future maintainer reading only the comment could reasonably remove the commit-count branch, believing it is dead weight, and reintroduce the exact silent-pass failure this guard exists to prevent.

Update the comment to describe both checks, or explain why the commit-count branch exists despite the stated assumption that implement never commits.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.archon/workflows/experimental/archon-fix-github-issue-experimental.yaml
around lines 309 - 346, Update the rationale comment above assert-implemented to
accurately state that the guard checks both uncommitted working-tree changes
outside .archon/ and commits ahead of origin/$BASE_BRANCH via the commits count.
Remove or revise the claim that only the working tree is inspected, and explain
the commit-count fallback’s purpose without changing the bash logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: workflows Workflow engine bug Something is broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant