feat(bin): harvest fleet task usage into a cost ledger - #3584
feat(bin): harvest fleet task usage into a cost ledger#3584npayette84 wants to merge 16 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Confidence Score: 4/5The PR is not yet safe to merge because normal retirement of a local secondmate can permanently discard its completed child-task usage rows. Completed children write their rows into the secondmate home's ledger, while normal retirement removes that home without transferring those rows to the surviving fleet ledger; only unfinished children encountered during forced cleanup are harvested into the parent. Files Needing Attention: bin/fm-teardown.sh Reviews (5): Last reviewed commit: "no-mistakes: apply CI fixes" | Re-trigger Greptile |
|
Speaking as Kun's firstmate: Triage of #3584 (replacement for closed-invalid #3392; harvester-only, no launch-hardening). HEAD Contract-class: new-default. Main has no VISION (inspected teardown hooks + harvest/report headers + AGENTS/docs rows):
Security: clean. Author CI/NM: first-time fork runs were
Land-eligible: NO (new-default; CI/NM not green yet). Captain-flag now: no — wait for green CI+NM+MATCH, then flag for the always-on vs opt-in decision (do not auto-merge). Overlaps: same-author kitchen-sinks #3298 / #3518; independent ledger attempt #3470 (GodKimba). Prefer this harvester-only replacement over #3392/#3298 style bundles if the captain accepts default-on harvest. Waiting on: CI/NM this HEAD. |
04ce4fe to
b9d3b1d
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9d3b1d564
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| elif [ -n "${USAGE_STAGE_MAIN:-}" ] && [ -s "$USAGE_STAGE_MAIN" ]; then | ||
| "$FM_ROOT/bin/fm-usage-harvest.sh" --append-from "$USAGE_STAGE_MAIN" "$ID" >/dev/null \ | ||
| || echo "warning: usage harvest for $ID failed; continuing teardown" >&2 |
There was a problem hiding this comment.
Captain, when a local Herdr teardown returns the pooled worktree at line 2941 but endpoint confirmation subsequently refuses at lines 3016-3018, execution never reaches this append and the EXIT trap deletes the staged measurement. If the pool slot is assigned to another task before teardown is retried, the retry scans that same path through the new harvest time and can silently attribute the next occupant's session log to the original task. Append the staged row once the worktree has been released, or persist it across these post-release refusal paths.
AGENTS.md reference: AGENTS.md:L90-L90
Useful? React with 👍 / 👎.
| FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" | ||
| STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" | ||
| DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" | ||
| CLAUDE_DIR="${FM_USAGE_CLAUDE_DIR:-${HOME:-}/.claude/projects}" |
There was a problem hiding this comment.
When Firstmate runs with CLAUDE_CONFIG_DIR set, fm-spawn.sh explicitly forwards that directory to every Claude worker, so its project session logs are written under that configured store. This default nevertheless always scans $HOME/.claude/projects; unless the operator also knows to set the test seam FM_USAGE_CLAUDE_DIR, every such task is silently recorded with source:"unavailable" and null token counts. Derive the default project-log directory from CLAUDE_CONFIG_DIR when it is present, or persist the selected store in task metadata.
Useful? React with 👍 / 👎.
|
Speaking as Kun's firstmate: this is waiting on the author. Re-triage after newer tip Attestation: MISMATCH — body Contract-class: new-default (unchanged). VISION:
Security: clean. Author not blocked. No Workflow approval this pass (tip action_required after diff review):
Land-eligible: NO (new-default + attestation MISMATCH; CI/NM just re-armed). Captain-flag now: no — wait for MATCH + green CI/NM, then flag for the always-on vs opt-in decision (do not auto-merge; do not rebase solely for that flag). Please re-raise via no-mistakes so attestation matches tip |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 834cc7aa56
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| encoded=${WORKTREE//\//-} | ||
| encoded=${encoded//./-} |
There was a problem hiding this comment.
Captain, when distinct worktree paths differ only by dots versus hyphens, this encoding maps them to the same Claude project directory (for example, /tmp/foo.bar/wt and /tmp/foo-bar/wt both become -tmp-foo-bar-wt). Because Claude logs contain no cwd for a secondary check, overlapping tasks at those paths will each sum the other task's usage and silently corrupt cost attribution; persist a collision-free session identity or detect ambiguous bindings instead.
AGENTS.md reference: AGENTS.md:L90-L90
Useful? React with 👍 / 👎.
| # Both renderings are LOCAL time because that is what touch -t reads; a UTC | ||
| # stamp would shift both window refs by the host's offset and drop real logs. | ||
| date -r "$1" +%Y%m%d%H%M.%S 2>/dev/null || date -d "@$1" +%Y%m%d%H%M.%S |
There was a problem hiding this comment.
On hosts in a DST-observing timezone, an epoch during the repeated fall-back hour is formatted as an ambiguous local touch -t timestamp. Both occurrences of, for example, 01:30 format identically, and touch can resolve the reference to the first occurrence; a task running during the second occurrence can then have valid logs rejected by the find prefilter and be recorded with unavailable usage. Create the references from unambiguous epochs or use a consistently UTC-aware mechanism.
AGENTS.md reference: AGENTS.md:L90-L90
Useful? React with 👍 / 👎.
| # same id can appear more than once here. The rows are sorted by task and then | ||
| # by spawned_at, and spawned_at is rendered, so those runs are told apart by | ||
| # when they started and their order is defined rather than append order. | ||
| if ! TASK_ROWS=$(jq -rs ' |
There was a problem hiding this comment.
When a task finishes while this report is running, the first jq, this second jq, and the later wc -l each read the append-only ledger independently without participating in its lock. A concurrent append can therefore produce per-model totals that omit a task shown in the per-task section, a row count that matches neither section, or a transient parse failure if a read catches the new row mid-write. Read the file once into a snapshot, or hold the ledger lock across all three computations.
AGENTS.md reference: AGENTS.md:L90-L90
Useful? React with 👍 / 👎.
| # depends on that order to resolve the model to the last incarnation's. | ||
| while IFS= read -r f; do | ||
| m=$(file_mtime_epoch "$f") || continue | ||
| if [ "$m" -ge "$START_EPOCH" ] && [ "$m" -le "$SCAN_END_EPOCH" ]; then |
There was a problem hiding this comment.
When a pooled worktree is reassigned in the same whole second that the previous task's session log receives its final write, this inclusive comparison admits that previous log into the new task's window. Both spawn_gen and file mtimes are reduced to integer seconds, and the reused cwd or Claude directory then provides no way to distinguish the prior occupant, so its tokens are silently charged to the new task. Preserve subsecond timing or filter on each session's recorded start identity before summing it.
AGENTS.md reference: AGENTS.md:L90-L90
Useful? React with 👍 / 👎.
| + ($l.message.usage.cacheWrite // 0)) | ||
| | .ot += ($l.message.usage.output // 0) | ||
| | .rt += ($l.message.usage.reasoning // 0) | ||
| | (if .m == null and ($l.message.model // null) != null then |
There was a problem hiding this comment.
When a Pi task changes models within one session, every assistant record carries its actual model but this condition assigns .m only for the first such record. Usage from later requests is still added to the row, while the row and per-model report remain labeled with the obsolete initial model rather than the final one. Update the selected model on each accepted usage record, or consume the session's model-change records, so the task is not attributed to the wrong model.
AGENTS.md reference: AGENTS.md:L90-L90
Useful? React with 👍 / 👎.
Record what each finished task actually cost. bin/fm-usage-harvest.sh appends one JSON line per harvested task to the gitignored data/usage-ledger.jsonl, and bin/fm-usage-report.sh renders that ledger as a per-model and per-task cost report. The harvester's header is the sole owner of the line schema and of every per-harness usage source. Token usage is read from each worker's own session log. claude sums per-request usage from the encoded project directory, deduping on .message.id because Claude logs one entry per content block. codex sums per-turn token_count deltas from sessions whose recorded cwd matches the task worktree. pi and pi-signed run the same Pi application over one ~/.pi/agent state tree, so one scan covers both: each assistant message record carries one request's usage, bound to the task by the cwd in the log's own session record, with the model reported provider-qualified to match the spelling fm-spawn records in task metadata. Every other harness, a task with a recorded remote_host, and any scan that yields no assistant usage report null tokens with source unavailable, so an available source asserts the same thing for all three parsers. The task window runs from the epoch embedded in the meta's spawn_gen incarnation token to the last status append. Metadata modification time is deliberately not a start source: later writes such as PR registration move it forward to near the end of the task, which collapsed the window to zero and dropped every session log. A relaunch narrows nothing below the turn count's own span, and a start later than the end is pinned rather than inverting the window. Teardown harvests before retiring the task's status presentation, because that retirement deletes state/<id>.status, which supplies both the window and the turn count. The harvest is best effort: a failure warns on stderr and never blocks teardown. Parsers skip a malformed or non-object log line rather than losing the rest of the file, and both the harvester and the report read their TSV rows without collapsing empty fields, so an absent model or harness cannot shift the token columns.
…cans, drop meta-mtime start
834cc7a to
6d5d71f
Compare
npayette84
changed the title
|
Speaking as Kun's firstmate: this is waiting on the author. Re-triage on tip Contract-class: new-default (unchanged). VISION:
Security: clean. No Greptile: FAILURE — normal retirement of a local secondmate can permanently discard completed child-task usage rows (children write into the secondmate home ledger; normal retirement removes that home without transferring rows; only unfinished children on forced cleanup are harvested into the parent). Author must address or deliberately document that loss as out-of-scope. Attestation: MATCH (body binds tip; review/test/document completed). Workflow approval this pass (tip
Land-eligible: NO (new-default; Greptile FAILURE; CI not green yet). Firstmate-flag: no — not otherwise-ready (Greptile + CI still open; even when green, flag only for the always-on vs opt-in decision, never auto-merge). Please fix or explicitly bound the secondmate-ledger loss Greptile found, keep attestation MATCH on tip, and wait for green CI. |
2 participants
Intent
Deliver firstmate's fleet usage ledger - the harvester and its report reader - as ONE clean change against upstream main, carrying only harvester-owned code, tests, and documentation.
WHY THIS BRANCH EXISTS. An earlier PR (#3392, no-mistakes run 01M1BVFHA9P1978ZASDF0RRPNJ) was declared invalid by the captain and closed WITHOUT merging because it was built on the wrong base. That branch targeted upstream main, which does not contain the harvester at all, so it structurally bundled a SEPARATE crewmate launch-hardening change (bin/fm-spawn.sh permission flags plus tests/fm-backend-orca.test.sh, tests/fm-secondmate-harness.test.sh and tests/fm-spawn-dispatch-profile.test.sh, which assert those flag strings). Its two required workflows also never ran, both sitting at action_required. That branch and its exact head 1b34b95 are deliberately PRESERVED FOR AUDIT and must not be force-pushed or deleted. This replacement was created fresh from current upstream main and carries the harvester ONLY. The launch hardening is deliberately absent and belongs to its own separate change; do not add, restore, or document it here, and treat any reappearance of bin/fm-spawn.sh or .agents/skills/harness-adapters/SKILL.md in this diff as a regression.
WHAT THE CHANGE DOES. bin/fm-usage-harvest.sh appends one JSON line per finished task to the gitignored data/usage-ledger.jsonl; bin/fm-usage-report.sh renders that ledger as a per-model and per-task cost report. The harvester's header comment is the SINGLE authoritative owner of the ledger line schema and of every per-harness usage source; docs/scripts.md carries only the two one-line script rows and AGENTS.md only a one-line data/ inventory entry, both pointing at that owner. No docs/verification/ record was added because the colocated behavioral tests own the guarantee.
Per-harness token sources, each verified against real logs: claude sums per-request usage from the encoded project directory, deduping on .message.id because Claude logs one entry per content block; codex sums per-turn token_count deltas from sessions whose recorded cwd equals the task worktree; pi and pi-signed share ONE scan of ~/.pi/agent/sessions because pi-signed is a signed wrapper around the same Pi application over the same state tree, with each assistant message record carrying one request's usage and the binding taken from the cwd in the log's own session record rather than any reconstructed directory encoding, which is immune to encoding drift. Pi's model is reported provider-qualified (zai/glm-5.3-flash) so a log-derived model matches the spelling fm-spawn already records in task metadata. Pi usage fields are disjoint counts, verified on real logs where input + output + cacheRead equals Pi's own totalTokens. opencode, grok, kimi, cursor, muse, a task with a recorded remote_host, and any scan yielding no assistant usage all report null tokens with source unavailable, so an available source asserts the same thing across all three parsers.
The task window runs from the epoch embedded in the meta's spawn_gen incarnation token to the last status append. Metadata modification time is deliberately NOT a start source: fm-spawn writes spawn_gen once per incarnation and never rewrites it, while later meta writes such as PR registration move the mtime forward to near the end of the task, which collapsed wall_secs to zero AND inverted the log-matching window so every session log was dropped. A relaunch never narrows a row below the turn count's own span, and a start later than the end is pinned to the end rather than inverting the window.
Teardown harvests BEFORE status_retire_presentation_task, because that call deletes state/.status, which supplies both the task window and the turn count. The harvest is best effort: a failure warns on stderr and never blocks teardown.
Robustness the review rounds established and that must be preserved: parsers skip a malformed or valid-non-object log line rather than losing the whole file's usage; neither the harvester nor the report collapses empty TSV fields, so an absent model or harness cannot shift the token columns; the epoch-to-touch stamp is rendered in the same local zone touch -t reads, since a UTC-rendered stamp shifted both window refs by the UTC offset on non-UTC hosts; the ledger append happens inside the existing lock with no temp file to leak; and the report fails loudly on an unparseable ledger rather than reporting empty sections.
CONSTRAINTS. This is firstmate's shared tracked material, so .agents/skills/firstmate-coding-guidelines governs: one owner per contract, one full sentence per line in tracked Markdown, plain dash never em dash, no agent co-author on any commit, and bin/*.sh shellcheck-clean under bin/fm-lint.sh. Tests must exercise behavior through the public script interface and must never assert implementation source bytes. Preserve behavior for claude, codex, pi, pi-signed, opencode, grok, kimi, cursor, muse, remote-secondmate tasks, and every runtime backend. ask-user findings are never mine to answer: I escalate them to firstmate and do not resolve them myself, and --yes must not be used.
ALREADY VERIFIED ON THIS EXACT HEAD (base 4ad8cba, current upstream main): tests/fm-usage-harvest.test.sh passes 23 of 23 including the relaunch-scope contract, corrupt-line and non-object-line tolerance, absent-model and absent-harness column safety, and source-named-only-after-a-parsed-match; bin/fm-lint.sh exits 0 on ShellCheck 0.11.0 with full extended analysis and actionlint 1.7.12 with 3 workflow files valid; bin/fm-doc-audience-check.sh ok at 89 surfaces and 299 local links; bin/fm-test-run.sh --check-coverage ok at total=176. End-to-end against the REAL unmodified ~/.pi/agent/sessions tree, a pi task whose metadata was rewritten to the end of the task reports wall_secs 3660, turns 2, input 24664, cached 25088, output 1084, reasoning 879 from source pi-sessions, matching an independent jq recomputation over the same log exactly on all four token fields, where the pre-change harvester reported wall_secs 0 with null tokens and source unavailable. bin/fm-spawn.sh, .agents/skills/harness-adapters/SKILL.md and the three launch-flag test files are confirmed untouched on this branch.
What Changed
bin/fm-usage-harvest.shappends one JSON line per finished task incarnation to the gitignoreddata/usage-ledger.jsonl, keyed on the(task, spawn_gen)pair so a recycled task id earns a row per spawn. Its header is the sole owner of the line schema and of every per-harness source: claude sums per-request usage from the encoded project directory and dedupes on.message.id, codex sums per-turntoken_countdeltas for sessions whose recorded cwd is the task worktree, pi and pi-signed share one scan of~/.pi/agent/sessionsbound by the cwd in each log's own session record and report the model provider-qualified, and opencode, grok, kimi, cursor, muse, remote-host tasks and any scan with no assistant usage all report null tokens withsource=unavailable. The task window runs from the epoch in the meta'sspawn_gentoken to the last status append, not from meta mtime, which later writes such as PR registration push toward the end of the task.bin/fm-usage-report.shrenders that ledger as aligned per-model and per-task plain text. Both sections are computed before anything prints, so an unparseable ledger exits non-zero with a named diagnostic instead of empty sections, and rows are read over the unit separator so an absent model or harness cannot shift the token columns.bin/fm-teardown.shnow harvests at three sites, all best effort: a failure warns on stderr and never blocks teardown. The local task path and the forced secondmate child cleanup split the work, scanning before the worktree release while the task still holds its pooled slot and appending from a per-run staging directory once the retaining refusals have passed, immediately beforestatus_retire_presentation_taskdeletesstate/<id>.statusand with it the window and the turn count; the remote secondmate path harvests in one shot, since a remote task has no local worktree to scan.tests/fm-usage-harvest.test.shdrives both scripts through their public interfaces across the harnesses, relaunch scope, corrupt and non-object log lines, absent model and harness columns, ledger locking, and the teardown ordering, with one-line entries added todocs/scripts.mdand thedata/inventory inAGENTS.md.Risk Assessment
✅ Low: The change is additive and well-bounded: two new scripts plus best-effort, warning-only call sites in teardown that cannot alter its control flow or exit status, backed by 41 behavioral test cases, with only cosmetic duplication surfacing in review.
Testing
I ran the colocated behavioral suite
tests/fm-usage-harvest.test.sh(40 checks, all pass) and then demonstrated the feature end-to-end against the real, unmodified~/.pi/agent/sessionsand~/.claude/projectstrees on this host, using task records pointed at worktrees that real sessions actually ran in and with each meta's mtime deliberately touched forward to the end of the task. The pi row reports wall_secs 52221, turns 2, input 116211, cached 1368832, output 15567, reasoning 6976 from source pi-sessions with the model reported provider-qualified, and the claude row reports input 2240, cached 216242085, output 578158, reasoning 279700 - both matching independent jq recomputations over the same logs on every token field, with the claude case exercising a dedupe that collapses 1773 raw entries onto 1125 requests. Stripping spawn_gen from the same pi meta reproduces the collapse the change exists to prevent (wall_secs 52221 to 0), a cursor task and a remote_host task both land as null tokens with source unavailable, re-harvesting appends nothing, and the report renders the ledger, tolerates an absent one and fails loudly on a truncated one. I also confirmed the diff carries only the six harvester-owned files with no reappearance of bin/fm-spawn.sh, the harness-adapters skill, or the three launch-flag test files, and that the ledger path is gitignored. The temporary fixture home was removed and the worktree is clean; no source or test files were modified. This change is CLI-only with no rendered UI surface, so the reviewer-visible evidence is the ledger JSON lines and the rendered report transcript rather than screenshots.Evidence: End-to-end transcript against the real pi and claude session trees
Source: End-to-end transcript against the real pi and claude session trees
Evidence: Harvested usage ledger (data/usage-ledger.jsonl)
Source: Harvested usage ledger (data/usage-ledger.jsonl)
{"task":"ship-website-perf","spawn_gen":"s1788344756.4711.a1b2c3","harness":"pi","model":"openai-codex/gpt-5.6-sol","effort":null,"spawned_at":"2026-09-02T10:25:56Z","completed_at":"2026-09-03T00:56:17Z","wall_secs":52221,"turns":2,"input_tokens":116211,"cached_input_tokens":1368832,"output_tokens":15567,"reasoning_tokens":6976,"source":"pi-sessions"} {"task":"fix-ledger-window","spawn_gen":"s1788218000.9021.zz9plu","harness":"claude","model":"claude-opus-5","effort":"high","spawned_at":"2026-08-31T23:13:20Z","completed_at":"2026-09-03T08:12:43Z","wall_secs":205163,"turns":7,"input_tokens":2240,"cached_input_tokens":216242085,"output_tokens":578158,"reasoning_tokens":279700,"source":"claude-projects"} {"task":"audit-cursor-run","spawn_gen":"s1788300000.100.aaa","harness":"cursor","model":null,"effort":null,"spawned_at":"2026-09-01T22:00:00Z","completed_at":"2026-09-01T23:00:00Z","wall_secs":3600,"turns":1,"input_tokens":null,"cached_input_tokens":null,"output_tokens":null,"reasoning_tokens":null,"source":"unavailable"} {"task":"remote-crew-task","spawn_gen":"s1788300000.100.aaa","harness":"pi","model":"zai/glm-5.3-flash","effort":null,"spawned_at":"2026-09-01T22:00:00Z","completed_at":"2026-09-01T23:00:00Z","wall_secs":3600,"turns":1,"input_tokens":null,"cached_input_tokens":null,"output_tokens":null,"reasoning_tokens":null,"source":"unavailable"}Evidence: Rendered report (bin/fm-usage-report.sh)
Source: Rendered report (bin/fm-usage-report.sh)
usage ledger: .../data/usage-ledger.jsonl (4 rows) per-model totals (source-available rows): model tasks input cached output reasoning wall_secs claude-opus-5 1 2240 216242085 578158 279700 205163 openai-codex/gpt-5.6-sol 1 116211 1368832 15567 6976 52221 per-task rows: task spawned_at harness model effort input cached output reasoning wall_secs source audit-cursor-run 2026-09-01T22:00:00Z cursor - - - - - - 3600 unavailable fix-ledger-window 2026-08-31T23:13:20Z claude claude-opus-5 high 2240 216242085 578158 279700 205163 claude-projects remote-crew-task 2026-09-01T22:00:00Z pi zai/glm-5.3-flash - - - - - 3600 unavailable ship-website-perf 2026-09-02T10:25:56Z pi openai-codex/gpt-5.6-sol - 116211 1368832 15567 6976 52221 pi-sessionsPipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
bin/fm-usage-harvest.sh:444-LEDGER="$DATA/usage-ledger.jsonl"is assigned twice with the identical value: once at line 267 with the other path constants, and again at line 444 after the window setup. The second assignment is dead and invites the two from drifting apart if the ledger path ever moves. Remove line 444.bin/fm-usage-report.sh:84- Each report section writes its printf format string twice: the 7-field per-model format at lines 73 and 78, and the 11-field per-task format at lines 84 and 89. The header and the row body must stay byte-identical or the columns silently misalign, and nothing enforces that. Hoist each format into a variable used by both the header printf and the loop body.bin/fm-usage-harvest.sh:528- The[ -z "$REMOTE_HOST" ] && [ -n "$WORKTREE" ]scan precondition is repeated verbatim in all three harness branches (lines 528, 556, 586). It is one rule the header already states once for every harness, so it can be hoisted to guard the wholecaseblock, leaving each branch to carry only its own directory, probe, and jq program.bin/fm-usage-report.sh:84- Cosmetic only: in the report's per-task section the model column is printf '%-20s', so a longer model id such as "openai-codex/gpt-5.6-sol" (24 chars) overflows and shifts that row's remaining columns right by four characters. Fields stay space-separated and every value is still correct and readable, and the per-model section (%-24s) is unaffected. Visible in the attached usage-report.txt, last row. Flagging it because it shows up in the reviewer-facing artifact, not because anything is wrong with the data.bash tests/fm-usage-harvest.test.sh- 40 behavioral checks, all passEnd-to-end harvest of a pi task against the real unmodified~/.pi/agent/sessionstree, meta mtime forced to the task end:bin/fm-usage-harvest.sh ship-website-perfIndependent recomputation of that log's usage withjq -sover.message.usage(input/cacheRead/cacheWrite/output/reasoning), compared field by field against the ledger rowCounterfactual withspawn_genstripped from the same meta, showing wall_secs 52221 -> 0End-to-end harvest of a claude task against the real~/.claude/projectstree (1773 raw usage entries / 1125 distinct.message.id):bin/fm-usage-harvest.sh fix-ledger-windowIndependent deduped recomputation withjq -rs 'group_by(.message.id)'over the same logs, compared against the ledger rowUnavailable-source harvests: aharness=cursortask and aharness=pitask carryingremote_host=orca-02.fleetbin/fm-usage-report.shrendering the four-row ledger (per-model totals and per-task rows)Idempotency: re-harvesting all four tasks, ledger row count unchanged at 4bin/fm-usage-report.sh /absent/usage-ledger.jsonl(exit 0) and the same on a truncated ledger (exit 1 with a named diagnostic)git diff --name-only 28fb5ac..6d5d71fchecked forfm-spawn,harness-adapters, and the three launch-flag test files - none presentgit check-ignore -v data/usage-ledger.jsonldocs/configuration.md:786- Judgment call, left unchanged deliberately: the new harvester knobs (FM_USAGE_LEDGER_LOCK_WAIT, FM_USAGE_CLAUDE_DIR, FM_USAGE_CODEX_DIR, FM_USAGE_PI_DIR, FM_USAGE_HARVEST_APPEND_DELAY) are documented only in bin/fm-usage-harvest.sh's header, and the data/ summary at docs/configuration.md:13 still names its records with 'such as' rather than listing the ledger. I did not add either copy: docs/configuration.md's 'Environment variables' list is selective rather than an inventory (roughly 180 other bin/ tunables with defaults, including FM_KIMI_, FM_CONTROL_, FM_LINT_JOBS and FM_REMOTE_JOB_*, are absent from it), docs/scripts.md:4 declares each script's header authoritative for its behavior and contracts, and AGENTS.md:84-93 owns the data/ inventory, so synchronizing a second prose copy would work against the one-owner policy. Flagging it because an operator scanning docs/configuration.md alone will not discover the harvester's log-directory or lock-wait knobs; if the repo wants those knobs indexed there, that is a small follow-up rather than part of this change.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.