Build Failure Analysis: reuse Azure DevOps binlogs instead of rebuilding

.github/workflows/README.md

@@ -163,8 +163,8 @@ a workflow needs elevated access (then use the GitHub App above).
 
 | Workflow | Trigger | Description |
 | --- | --- | --- |
-| [`build-failure-analysis.md`](./build-failure-analysis.md) | PR opened/synchronize/reopened on `main` or `rel/*` | Runs `./build.sh --binaryLog`; on failure, the `build-failure-analyst` agent reads JSON dumps from the binlog, posts a summary comment, and attaches inline `suggestion` blocks. Advisory only — not a gating check. |
-| [`build-failure-analysis-command.md`](./build-failure-analysis-command.md) | `/analyze-build-failure` on a PR | Re-runs the build-failure analysis on demand (after force-pushes, dismissed comments, etc.). |
+| [`build-failure-analysis.md`](./build-failure-analysis.md) | Azure Pipelines `microsoft.testfx` check `completed` (failure) on a PR to `main` or `rel/*` | Downloads the binary logs the failed Azure DevOps build already produced (all build legs — it does **not** rebuild), and the `build-failure-analyst` agent queries them via `binlog-mcp`, posts a summary comment, and attaches inline `suggestion` blocks. Advisory only — not a gating check. |
+| [`build-failure-analysis-command.md`](./build-failure-analysis-command.md) | `/analyze-build-failure` on a PR | Re-runs the analysis on demand: inspects the PR's latest `microsoft.testfx` build and, only when it failed, downloads its binlogs and analyzes them (no rebuild). |
 | [`add-tests.md`](./add-tests.md) | `/add-tests` on a PR | Generates unit tests for code introduced in a pull request. |
 | [`grade-tests-on-pr.agent.md`](./grade-tests-on-pr.agent.md) | PR opened/reopened/synchronize/ready_for_review touching `test/**` | Automatically grades new and modified test methods and posts a single PR scorecard comment via the `grade-tests` skill. |
 | [`grade-tests.agent.md`](./grade-tests.agent.md) | `/grade-tests` on a PR | Re-runs the test-quality grading on demand. |

.github/workflows/shared/build-failure-analysis-shared.md

@@ -1,9 +1,9 @@
 ---
 # Shared body for the build-failure-analysis workflows.
 #
-# Imported by build-failure-analysis.md (pull_request trigger) and
-# build-failure-analysis-command.md (slash command). Keeps the prompt that
-# delegates to the build-failure-analyst agent in one place. Per-trigger
+# Imported by build-failure-analysis.md (check_run + workflow_dispatch
+# triggers) and build-failure-analysis-command.md (slash command). Keeps the
+# prompt that drives the build-failure analysis in one place. Per-trigger
 # wiring (steps, env, mcp-servers, permissions) lives in each caller because
 # gh-aw merges those fields from imports but each main workflow must still
 # re-declare its top-level permissions.
@@ -13,47 +13,50 @@ description: "Shared body for build-failure-analysis workflows"
 
 # Build Failure Analyst
 
-Delegate to the `build-failure-analyst` agent defined at
-`.github/agents/build-failure-analyst.agent.md` to analyze the binary log of
-the build that just ran and post a PR review when (and only when) the build
-failed.
+You are the **build-failure analyst**. Analyze the binary logs of the Azure
+DevOps build that just failed and produce a PR review using the safe-output
+tools (a later `safe_outputs` job performs the actual GitHub write).
+Do **not** try to spawn a sub-agent: the `task` tool is intentionally not
+available here. Work directly with the tools you do have: `binlog-mcp` to
+read the logs, the `github` tools to read PR/repo context (the GitHub MCP
+server is **read-only** here), the `safeoutputs` writers (including the
+`write` tool) to post results, and a small set of read-only `shell` commands
+(including `cat`).
 
 ## Instructions
 
 1. Read the agent-context environment variables: `GH_AW_BUILD_OUTCOME`,
-   `GH_AW_BINLOG_PATH`, `GH_AW_BINLOG_HOST_PATH`, `GH_AW_PR_NUMBER`,
-   `GH_AW_PR_HEAD_SHA`, `GH_AW_WORKSPACE`.
+   `GH_AW_BINLOG_LIST`, `GH_AW_BINLOG_DIR`, `GH_AW_BINLOG_PATH`,
+   `GH_AW_BINLOG_HOST_PATH`, `GH_AW_PR_NUMBER`, `GH_AW_PR_HEAD_SHA`,
+   `GH_AW_PR_MERGE_SHA`, `GH_AW_WORKSPACE`.
 
 2. If `GH_AW_BUILD_OUTCOME == 'success'`, the build did not actually fail —
-   there is nothing to analyse. Call `noop` with the message
+   there is nothing to analyze. Call `noop` with the message
    `"Build succeeded — no analysis required."` and stop.
 
-3. Otherwise, launch the `build-failure-analyst` agent as a **background**
-   task (`task` tool, `agent_type: "general-purpose"`,
-   `model: "claude-opus-4.6"`, `mode: "background"`). In the sub-agent prompt
-   include:
-   - All six `GH_AW_*` environment values verbatim so the sub-agent knows
-     which binlog to query (`GH_AW_BINLOG_PATH` is the in-container path
-     `/data/build.binlog` exposed by the `binlog-mcp` MCP server) and where
-     to post.
-   - A reminder that the binlog is live-queryable through the `binlog-mcp`
-     MCP server: the sub-agent should call MCP tools such as
-     `binlog_overview`, `binlog_errors`, `binlog_warnings` (and others as
-     needed) with `binlog_file: "$GH_AW_BINLOG_PATH"`, rather than reading
-     pre-dumped JSON files.
-   - A reminder that the parent workflow `noop`s immediately and that the
-     sub-agent itself is responsible for calling `add_comment` (summary) and
-     `create_pull_request_review_comment` (inline ```suggestion blocks).
-   - A reminder that `submit_pull_request_review` is **not** a safe output
-     for this workflow — inline comments stand alone.
-
-4. **Immediately after launching the background task** — do NOT wait for it
-   to finish and do NOT read its result — call `noop` with a brief status
-   message such as
-   `"Build-failure analyst launched in background for PR #N. It will post the analysis directly."`.
-   Then stop.
-
-> **Important**: Reading the background agent result would pull its entire
-> conversation (including every binlog query and every source file it
-> inspected) into your context. Do not call `read_agent` or any equivalent
-> after calling `noop`.
+3. Load your detailed playbook: `cat .github/agents/build-failure-analyst.agent.md`
+   (it is checked out with the repository config). Follow that methodology —
+   root-cause grouping, source-context reading via the GitHub API at
+   `GH_AW_PR_HEAD_SHA`, comment/suggestion formatting, and defensive behavior.
+   In summary:
+   - Iterate **every** path in `GH_AW_BINLOG_LIST` (newline-separated
+     in-container binlog paths, one per failed-build leg, under
+     `GH_AW_BINLOG_DIR` = `/data/binlogs`) and query the `binlog-mcp` MCP
+     server (`binlog_errors`, `binlog_overview`, `binlog_warnings`, …) with
+     `binlog_file` set to each leg's path — a failure usually surfaces in only
+     one leg, so do not analyse just the first. If no leg shows errors **and**
+     no failed-target/process evidence, the build compiled cleanly — the
+     pipeline failure is then a **non-build** (test/Helix/publishing) failure,
+     which is **out of scope**. This workflow analyses build failures only, so
+     **post nothing**: call `noop` with a short reason and stop. Do **not**
+     post a summary comment and do **not** invent fixes.
+   - Post exactly one summary via `add_comment` and any inline
+     `suggestion` blocks via `create_pull_request_review_comment`, **targeting
+     the pull request `GH_AW_PR_NUMBER` explicitly** (these workflows use
+     `target: "*"`, so there is no implicit "triggering PR" — pass the number
+     on every safe-output call).
+   - `submit_pull_request_review` is **not** a safe output for this workflow;
+     inline comments stand alone.
+
+4. When you have posted the analysis for a genuine build failure (or called
+   `noop` for a clean-compile / non-build failure), stop.