Gate Codex support on a real-host plugin and hook smoke test

[SUaDtL]

Gate Codex support on a real-host plugin and hook smoke test

Severity: high | Confidence: 0.98 | Effort: M

Where:

• docs/codex-parity-testing.md:3-18
• .github/workflows/ci.yml:442-453
• .github/workflows/ci.yml:745-775
• plugins/ca-codex/.codex-plugin/plugin.json:1-13
• .github/scripts/test_codex_adapter.py:549-590

Evidence: The live-parity runbook explicitly says the static adapter test cannot prove that Codex delivers payloads, runs trusted hooks, or honors a block, and records only a one-time Windows verification on Codex 0.144.1. Required CI invokes Python scripts directly for adapter parity and tolerates codex --version being absent before validating a locally maintained 0.144.1 schema baseline. The shipped manifest nevertheless advertises Codex >=0.143.0 and full blocking pre-exec/pre-write enforcement. No workflow installs or launches a real Codex runtime. The current workstation has codex-cli 0.144.4, confirming a runnable external host exists even though the gate never exercises it.

Impact: A Codex protocol, trust, hook-discovery, payload, structured-deny, or skill-loading regression can merge while every required parity check is green. Because the missing boundary is the host invoking the guard, static tests can prove the guard's answer yet miss the more dangerous outcome where Codex never calls it or ignores its block.

Recommendation: Add a bounded real-Codex fixture lane that installs exact supported runtime versions, installs/trusts the checked-out ca-codex package, starts in a disposable repository, and proves SessionStart injection plus one allow and representative EXEC/WRITE block through the actual host. Keep a nonblocking latest canary separate from the required supported baseline.

Acceptance criteria:

• Required CI launches the checked-out plugin through a real Codex runtime at the documented minimum or explicitly narrows the supported version claim to the versions continuously tested.
• The live fixture proves SessionStart context injection, skill discovery, an ordinary allowed write, an apply_patch H-18 or H-05 block, and a shell H-20 or H-03 block through Codex itself.
• A fixture with untrusted/unregistered hooks fails visibly rather than passing on static adapter results.
• A nonblocking latest-Codex canary reports upstream protocol drift independently of the pinned required lane.

[SUaDtL]

Feasibility spike, 2026-07-26 — run against real Codex 0.145.0

The open question gating this issue was whether a real-host lane needs OPENAI_API_KEY, because a required check that needs a secret cannot run on fork PRs. Answer: it splits cleanly in two, and only one half needs a credential.

Credential-free, and it works today

Installing the checked-out plugin through a real Codex host needs no credential and no network. Verified end to end against a disposable CODEX_HOME:

codex plugin marketplace add <path-to-checkout>     # -> Added marketplace `codearbiter`
codex plugin list                                   # -> ca-codex@codearbiter   not installed
codex plugin add ca-codex@codearbiter               # -> Added plugin `ca-codex`
codex plugin list                                   # -> ca-codex@codearbiter   installed, enabled   0.3.0

marketplace add accepts a local path (<SOURCE>: "a local path, owner/repo[@ref], HTTPS Git URL, or SSH Git URL"), so the marketplace source is the checkout itself — no publishing step, no network. The install is a faithful copy: .codex-plugin/plugin.json, hooks/ (including hooks/hooks.json), skills/, routines/, and includes/ all land under $CODEX_HOME/plugins/cache/codearbiter/ca-codex/0.3.0, and config.toml records [plugins."ca-codex@codearbiter"] enabled = true.

That covers a real slice of AC-1 (a real runtime installs the checked-out plugin at a pinned version) and gives AC-3 its negative case for free — a fixture with a broken or unregistered hooks/hooks.json can be asserted against codex plugin list output without a model.

Not credential-free: proving a hook actually fires

AC-2 cannot be reached without a provider. A hook only runs inside a turn, and a turn needs a model. Measured: codex exec pointed at an unreachable provider (http://127.0.0.1:9/v1, the discard port, with a DUMMY env key) initialises a complete session — workdir, model, provider, approval: never, sandbox: read-only, session id, the user message — and then dies at the network boundary:

ERROR: stream disconnected before completion: error sending request for url (http://127.0.0.1:9/v1/responses)

With ca-codex installed and enabled, hooks/hooks.json registering SessionStart, PreToolUse (2), PostToolUse, and UserPromptSubmit, and the fixture repo carrying arbiter: enabled, that run produced:

• no persona injection — zero matches for You are codeArbiter / orchestrator persona / /ca:commands in the transcript
• no hook artifacts — no .codearbiter/.markers, no gate-events.log

Reproduced with CODEX_HOME both inside and outside the system temp directory, so that is not the cause.

Two confounders remain, and they must be eliminated before this counts as a finding rather than an observation:

1. codex exec is non-interactive and may not emit SessionStart by design. Distinguishing "the host never called the guard" from "this entry point does not raise that event" needs a run that reaches a first assistant turn — i.e. a working provider.
2. This was 0.145.0. The manifest advertises >= 0.143.0 and records live verification on 0.144.1, so some of the gap may be version drift rather than a standing defect.

Either way it is exactly the boundary this issue names: static tests can prove the guard's answer yet miss the more dangerous outcome where Codex never calls it. I could not demonstrate the host calling it.

Concrete evidence that AC-4's canary is justified

wire_api = "chat" is removed in 0.145.0 — it is a hard config-load error, not a deprecation warning:

Error loading config.toml: `wire_api = "chat"` is no longer supported.
How to fix: set `wire_api = "responses"` in your provider config.

That is upstream protocol drift breaking a previously valid configuration between the version this repo live-verified (0.144.1) and current. A non-blocking latest canary would have reported it; nothing in required CI would.

Also worth pinning: the workstation is now on 0.145.0, while the manifest still claims live-verified on 0.144.1 and >= 0.143.0. The supported-version claim is drifting away from anything continuously tested, which is the other half of AC-1.

Two fixture-design gotchas, measured

• CODEX_HOME must not be under the system temp directory. Codex refuses to create its helper binaries there and warns on every invocation: Refusing to create helper binaries under temporary dir "C:\Users\brenn\AppData\Local\Temp\". Non-fatal, but a CI fixture should use a workspace-relative directory so the lane's output is not buried in warnings.
• codex doctor is not a gate surface. It reports no plugin or hook information at all, and it exits 1 purely because no credentials are present — so its exit status cannot express "the plugin is correctly wired" in a credential-free lane.

Recommendation: split this issue

• Required lane, no secret — install the checkout through a real pinned Codex runtime, assert codex plugin list reports installed, enabled, <expected version>, and assert a deliberately broken hooks/hooks.json fails visibly rather than silently. Runs on forks. Closes the "no workflow installs or launches a real Codex runtime" half of the finding.
• Separate lane, credentialed — the live enforcement proof (SessionStart injection, an allowed write, an apply_patch and a shell block through Codex itself). It needs a provider credential, so it cannot be a required check on fork PRs: pull_request_target, a self-hosted runner, or a scheduled run against main. That is a maintainer call about where a provider secret is allowed to exist, not something to decide inside this issue.
• Non-blocking canary — latest Codex, reporting drift like the wire_api removal above.

Splitting also narrows the version claim honestly: the required lane can pin what is continuously tested, which is the alternative AC-1 already permits.