test(pr): keep GitHub helpers working when gh rejects --slurp - #3572
Closed
att430 wants to merge 1 commit into
Closed
test(pr): keep GitHub helpers working when gh rejects --slurp#3572att430 wants to merge 1 commit into
att430 wants to merge 1 commit into
Conversation
Ubuntu 24.04 ships gh 2.45.0, which has no gh api --slurp (added in 2.48.0) and no gh pr checks --json (2.50.0). Firstmate's own PR record, poll, and merge helpers already use older gh surfaces; the fakes now reject those later flags so a helper cannot grow the same CI-reader failure no-mistakes hits on this distro. The CI monitor that parks green PRs at ask-user lives in no-mistakes (kunchenguid/no-mistakes#942), not in this repo. Do not upgrade the system gh package as a firstmate-side workaround: Noble has no newer apt candidate, and apt reverse-depends lists only optional sugarjar.
Confidence Score: 5/5The PR appears safe to merge because it changes only test infrastructure and adds focused compatibility coverage without altering runtime behavior. The updated fakes exercise the intended Ubuntu gh 2.45 command surface, and the new regression path verifies that PR recording, polling, and merging continue to function through supported commands and existing fallbacks. Reviews (1): Last reviewed commit: "test(pr): keep GitHub helpers working wh..." | Re-trigger Greptile |
Contributor
Author
|
Withdrawing this - not pursuing upstream contributions. Closing so it does not sit in your review queue. Apologies for the noise. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Stop every PR run from parking at a decision gate because the CI check reader uses a flag the installed gh does not have.
THE GAP: the validation pipeline's CI monitor calls
gh api --slurp. The installed gh is 2.45.0 (Ubuntu package 2.45.0-1ubuntu0.3, released 2025-07-18), which predates that flag and rejects it withunknown flag: --slurp. The monitor therefore cannot read check results at all and parks the run at an ask-user gate claiming it cannot verify CI.WHY IT MATTERS: the runs are fine and CI is green. The failure is purely that the tool cannot read the answer, but it surfaces as a decision gate that stops the pipeline and asks a human. Every PR-producing run hits this, and each one costs a firstmate turn to resolve by hand. It has already done so on comment-relay PR #6 and PR #7.
VERIFIED WORKAROUND, for context:
gh-axi pr checks <n> -R <owner>/<repo>reads the same result without that flag and returned "1 passed, 0 failed, 1 total" immediately.THE FIX is one of: upgrade gh past the version that introduced --slurp, or point the CI check read at a call the installed gh supports. ESTABLISH WHICH BEFORE CHANGING ANYTHING. Do not upgrade a system package as a first move without checking what else on this host depends on it. State the evidence for the route you pick, and if the honest answer is that the fix belongs upstream rather than in this repo, say so and report it rather than working around it locally.
SCOPE: own the whole problem (duplicate backlog items describe the same issue). Do not conflate this with pipeline agent-selection work; it is unrelated to any codex switch.
ACCEPTANCE CRITERIA:
CONSTRAINTS:
DECISIONS MADE WHILE DOING THE WORK (reviewer context):
gh api --paginate --slurpis no-mistakesinternal/scm/github/github.gogetWorkflowRunChecks, still present on no-mistakes v1.62.0. Firstmate itself does not call that flag.gh api --slurpandgh pr checks --json(the other 2.45 gap), and an explicit test drives PR record, poll, and merge through that surface. That is intentional and must not be "fixed" by teaching firstmate to use --slurp.What Changed
gh api --slurpandgh pr checks --jsonusage.Risk Assessment
✅ Low: The change is limited to executable regression coverage for Firstmate’s GitHub helper paths, correctly rejects the unsupported gh 2.45.0 flags, and accompanies a precise upstream report for the actual no-mistakes defect.
Testing
No earlier baseline commands were supplied; both targeted PR-forge test scripts passed, including end-to-end PR registration, open/merged polling, and merge under a simulated gh 2.45 CLI that rejects the unsupported flags, with the reviewer-visible transcript captured as evidence.
Evidence: GitHub CLI 2.45 compatibility behavior transcript
Source: GitHub CLI 2.45 compatibility behavior transcript
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
Inspectedgit diff 5fb0ce7628f240f9844f8b4bcd32ecd6155c3778..b3e0827514a9b274be3d3d56fdb113b1f226a94ato identify the behavioral compatibility surface.Ranbin/fm-test-run.sh --per-script-timeout-secs 300 tests/fm-pr-check-security.test.sh tests/fm-pr-merge.test.sh.Verified the evidence transcript was created and the worktree remained clean withgit status --short.✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.