fix(ai): record Codex compact usage diagnostics

[roboomp]

Repro

A synthetic Codex Responses usage payload matching the reporter's post-#4471 capture reproduces the remaining path: calling populateResponsesUsageFromResponse with {input_tokens:132278,input_tokens_details:{cached_tokens:124416},output_tokens:29,total_tokens:132307} and no orchestration_* fields emits usage.input=7862, usage.cacheRead=124416, and no orchestration bucket. The existing request stats only exposed request counts, so the raw uncached suffix could not be correlated with the WebSocket delta shape.

Cause

packages/ai/src/providers/openai-shared.ts correctly treats provider-reported input_tokens - cached_tokens as ordinary input when no explicit orchestration fields exist. The Codex transport in packages/ai/src/providers/openai-codex-responses.ts recorded only coarse OpenAICodexWebSocketDebugStats counters, so compact/resume WebSocket delta turns could prove previous_response_id chaining but could not show request input bytes/types, raw cached/uncached usage, or the normalized OMP buckets side by side.

Fix

• Extended OpenAICodexWebSocketDebugStats with lastTurn diagnostics covering transport, previous_response_id presence, input item count/types, serialized input bytes, prompt cache key, tools/options hashes, and append-chain state.
• Recorded raw Codex usage and normalized displayed buckets on terminal response events, including optional orchestration_* fields only when the provider reports them.
• Kept WebSocket and SSE request diagnostics aligned with the final payload after onPayload hooks, and added a regression test for a delta turn whose raw usage lacks orchestration fields.
• Added the package changelog entry for Codex compact/resume reports persistent uncached suffix on WebSocket deltas after #4471 #4707.

Verification

Reproduced the narrow raw-usage path with a JS eval against packages/ai/src/providers/openai-shared.ts: output was {"input":7862,"output":29,"cacheRead":124416,"cacheWrite":0,"totalTokens":132307,...}. Ran bun test packages/ai/test/openai-codex-stream.test.ts -t diagnostics (1 pass, 52 filtered). Ran bun test packages/ai/test/openai-codex-stream.test.ts (53 pass). Ran bun check (passed). Fixes #4707

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

[roboomp]

No code change needed for this directive. Codex review is blocked by the repo’s code-review quota; a repo admin needs to enable/add review credits in the linked Codex settings before that bot can review this PR.

[xuli500177]

Thanks — this looks like the right diagnostic hook, but I don't think this should auto-close #4707 yet.

This PR records the key evidence we need (previous_response_id, input item counts/types/bytes, prompt cache key, raw provider usage, displayed buckets). That is useful and should make the next capture much cleaner.

But it does not resolve the reported uncached suffix itself. The regression test intentionally preserves the problematic path:

rawInputTokens: 132278
rawCachedTokens: 124416
rawUncachedTokens: 7862
displayedInputTokens: 7862
displayedOrchestrationInputTokens: 0

So after this PR, a compact/resume delta turn with no orchestration_* fields is still reported as ordinary uncached input; it is just diagnosable now. That matches the issue evidence, but it does not answer where the persistent server-side uncached suffix comes from, nor whether it is expected Codex backend behavior.

Could you change the PR footer from:

Fixes #4707

to:

Refs #4707

or:

Part of #4707

I think this PR is good as a diagnostics improvement, but #4707 should stay open until either:

1. the suffix source is identified/fixed;
2. the suffix is documented as expected Codex backend behavior and labeled accordingly in UI/debug output; or
3. another PR changes the accounting/labeling semantics for this no-orchestration_* path.

[roboomp]

Amended and pushed the branch commit footer from Fixes #4707 to Refs #4707 (ea4e91330102). I also tried to update the PR body footer, but the PR host tool rejects bodies that do not end with an auto-close keyword for this issue, so the branch-side footer is now non-closing while the existing PR body still needs maintainer/tooling support to change.