fix(bin): compact unchanged tasks in the session-start fleet digest - #3571
Closed
Valentino-Sole wants to merge 8 commits into
Closed
fix(bin): compact unchanged tasks in the session-start fleet digest#3571Valentino-Sole wants to merge 8 commits into
Valentino-Sole wants to merge 8 commits into
Conversation
Posten 4 of the wachmeldung-dedup build plan: the session-start "Work under way" subsection printed every task's full .meta and status tail at every session start regardless of whether anything had changed since the last one, adding a steady context cost on every restart/compaction. Each task now gets a fingerprint marker (state/.session-start-seen-<task>, a digest of .meta content plus the last status line). An unchanged fingerprint prints one compact line (task id, endpoint alive/dead, last known verb, full status log path, and the exact marker to delete to force the full block back on) instead of the full block. A changed, missing, or unreadable marker - including a task's first session start - always falls back to the full block, so nothing is silently lost. fm-teardown.sh removes the marker with the rest of a retired task's per-id state. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TqTx6VbeR5eArzxxCcCcXy
… in state inventory
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Posten 4 des Wachmeldung-Dedup-Bauplans (data/wachmeldung-dedup/report.md): der Fleet-State-Digest in bin/fm-session-start.sh druckt pro Auftrag den vollen .meta+Status-Tail-Block nur noch, wenn sich seit dem letzten Sitzungsstart etwas geaendert hat. Ein Fingerabdruck pro Task (state/.session-start-seen-, sha256 aus .meta-Inhalt + letzter Statuszeile) entscheidet: unveraendert -> eine kompakte Zeile (Task-ID, Endpunkt lebt/tot, letztes bekanntes Verb, Pfad zum vollen Statuslog, exakter Marker zum Loeschen, um den vollen Block zu erzwingen); geaendert, fehlend, unlesbar oder allererster Sitzungsstart -> weiterhin der volle Block wie bisher. Rein additiv: keine bestehende Digest-Struktur umgebaut, nur die Ausgabeform dieser einen Subsektion (Work under way) betroffen. Wake-Queue, OPEN DECISIONS, UNREAD STATUS und der READ-ONCE-Vertrag bleiben unangetastet. bin/fm-teardown.sh raeumt den neuen Marker beim Teardown eines Auftrags mit auf. Dokumentiert im STATUS-TAILS-Kommentarblock von bin/fm-session-start.sh (dient zugleich als --help-Text), im Skript-Header selbst, und in einem neuen Absatz in docs/architecture.md nahe der fm-fleet-snapshot.sh-Beschreibung. Automatisierte Tests in tests/fm-session-start.test.sh (unveraendert -> Kompaktzeile; geaenderte .meta bzw. neue Statuszeile -> voller Block; erster Sitzungsstart -> voller Block; fehlender Marker -> voller Block) und ein neuer Test in tests/fm-teardown.test.sh, der die Marker-Entfernung beim Teardown belegt. shellcheck ueber bin/fm-lint.sh sauber. Ausdruecklich NICHT Teil dieses Auftrags: die Bauplan-Punkte 1-3 (Wedge-Eskalation in bin/fm-watch.sh, Fertig-Vokabular, OPEN-DECISIONS-Kompaktform in bin/fm-wake-drain.sh/bin/fm-classify-lib.sh) - diese Dateien wurden bewusst nicht angefasst; der Kapitaen hat die Reihenfolge der Punkte ausdruecklich festgelegt. Der Kapitaen merged selbst; PR melden, wenn CI gruen ist, nicht mergen.
What Changed
bin/fm-session-start.shnow fingerprints each task in the fleet-state digest intostate/.session-start-seen-<id>(sha256 over the.metacontent, last status line, status-log byte size and endpoint alive/dead/unknown verdict, keyed by theFM_SESSION_START_STATUS_TAILbound). A matching fingerprint collapses that task to one compact line — task id, endpoint, last known verb normalized viafm-classify-lib.sh, full status-log path, its.metapath and the exact marker to delete; a missing, unreadable, mismatched or absent marker, a--reemit, or a different tail bound still prints the full.metaplus status tail. A lock-refused read-only session compares an existing marker but writes none.AGENTS.md,docs/architecture.mdand theFM_SESSION_START_STATUS_TAILentry indocs/configuration.mdwere updated to describe the compact form and to explicitly permit reading a compacted task's named.metaand status log directly;.squish/was added to.gitignore.bin/fm-teardown.shdeletesstate/.session-start-seen-<id>alongside the rest of a retired task's per-id state, on both the local and remote-secondmate teardown paths.tests/fm-session-start.test.sh(first start full then compact,.metachange, new status line, repeated last line, blank-terminated append, dead endpoint, missing/unreadable marker,--reemit, read-only marker write, verb capping and normalization, absent status log) and one intests/fm-teardown.test.shcovering marker removal.Risk Assessment
✅ Low: The dedup is additive and confined to one digest subsection; the core invariant (a compact line can only follow a non-read-only session that actually printed the byte-identical full block) holds under every branch I traced, all four captain-approved fixes are genuinely present and covered by behavioral tests, and the only residual findings are a cosmetic stderr leak and two behavior-preserving fork removals.
Testing
I ran the two test files that own this change — tests/fm-session-start.test.sh (64 assertions, incl. all 14 new fingerprint cases) and tests/fm-teardown.test.sh (59 assertions, incl. the new marker-cleanup test) — and both pass. The first attempt failed in the unrelatedtest_output_ordering_diagnostics_lead, a pre-existing environment issue (the fixture deletesnodefrom its fakebin but this machine has /usr/bin/node in the base PATH, so the expectedMISSING: nodediagnostic never appears); I fixed it with the suite's intendedFM_TEST_BASE_PATHoverride and everything ran green. Beyond the unit level I drove the real bin/fm-session-start.sh manually and captured the actual "Work under way" output a captain sees across seven scenarios, which shows the full block on the first start, the compact line with endpoint/verb/status-log path/marker on the second, and the full block restored by a .meta change, a new status line, an alive->dead endpoint flip, deleting the named marker, and --reemit — plus the updated READ-ONCE CONTRACT text. Lint/shellcheck was deliberately not run (lint phase owns it). Worktree is clean; the transcript lives only in the evidence directory.Evidence: Session-start dedup CLI transcript (real fm-session-start.sh output, 7 scenarios)
Source: Session-start dedup CLI transcript (real fm-session-start.sh output, 7 scenarios)
Evidence: Excerpt: unchanged task collapses to one compact line
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
bin/fm-session-start.sh:922- The compact branch does not exclude--reemit, so a context re-emit collapses exactly the blocks it exists to restore. Concrete reachable sequence, already present in the existing fixture:tests/fm-session-start.test.sh:2405runs a full startup (which prints every task's full block and writes state/.session-start-seen-<task> at line 946), then runs--reemitwith sm-r.meta unchanged and no status file - meta content identical, last_line empty both times, STATUS_TAIL identical -> NEW_FP == OLD_FP -> the re-emit prints only--- sm-r --- unchanged since last session start; compact. A re-emit is by definition a session that LOST the context in which the full block was printed (its own banner at line 687 says "the durable records below are reprinted"), so it now receives no branch, worktree, project, kind, harness, or pr from .meta for any quiet task - while the read-once contract at line 855 forbids re-reading state/*.meta and the compact line offers no sanctioned alternative except deleting the marker. The existing re-emit test asserts only theFLEET STATEheading (line 2442), so it passes with every block gone. Suggested fix: require[ "$REEMIT" -eq 0 ]for the compact branch, so a re-emit behaves like a missing marker.bin/fm-session-start.sh:932- The header asserts "unchanged since last session start", but the fingerprint (lines 581-591) hashes only .meta content, the last status line, and the tail bound - not the endpoint verdict, which is recomputed fresh at lines 890-902 and printed on the next line. Reachable sequence: session 1 prints task-a's full block withendpoint: aliveand writes the marker; the crew's pane then dies without writing a final status line (the wedge/crash case); session 2 recomputes the same .meta, the same last status line, the same STATUS_TAIL -> fingerprint matches -> the digest prints--- task-a --- unchanged since last session start; compactimmediately followed byendpoint: dead (...). The label is wrong for exactly the transition AGENTS.md:549 makes actionable (load stuck-crewmate-recovery on a dead endpoint), and it is the header an agent skims to decide what changed. Two candidate fixes, both the captain's call because the intent fixes the fingerprint recipe verbatim ("sha256 aus .meta-Inhalt + letzter Statuszeile"): fold the alive/dead verdict into the hashed input so a flip restores the full block once, or narrow the wording to what is actually claimed (e.g. "meta and last status line unchanged").bin/fm-session-start.sh:918- The only status input to the fingerprint is the last non-blank line (line 914 -> hashed at 585/587), so an append sequence whose final line repeats the line recorded at the previous session start is reported as unchanged and its intervening lines are never shown. Concrete sequence: state/task-x.status holdsworking: running tests; session 1 prints the full block and records fp = sha256(<meta> + "\nstatus-tail=5\n" + "working: running tests\n"). The crewmate appendsfailed: tests redand, on its retry,working: running testsagain. Session 2: .meta unchanged, STATUS_TAIL unchanged, last_status_line() again returnsworking: running tests-> NEW_FP == OLD_FP -> compact line,last known verb: working, and the tail that would have shownfailed: tests redis suppressed with no error. The intent lists "neue Statuszeile -> voller Block" as acceptance behavior, and here a new status line does not restore it. Cheapest fix keeping the recipe's shape: add the status file's byte size (or line count) to the hashed input; alternatively hash the exact tail the full block would render. Captain's call because the intent specifies the recipe verbatim and it is documented in three places (script header FINGERPRINT DEDUP, docs/architecture.md:85, docs/configuration.md:802). Mitigating: the wake queue and OPEN DECISIONS still surface a captain-relevant verb, and the compact line still prints the full log path.bin/fm-session-start.sh:855- The read-once contract still states "Everything below is printed in full for this session start: every state/.meta, ... a bounded tail of every state/.status" and then forbids re-reading them; ORDERING item 6 in the header and AGENTS.md:182 ("everystate/<id>.meta; a bounded tail of each task'sstate/<id>.status") make the same claim. For a compacted task none of that holds, and the contract's "go to a source directly only when" list has no clause covering it - the compact line only offers "delete <marker> to force the full block", never "you may read this task's .meta directly". An agent obeying the contract therefore has no current identity data (branch, worktree, window, pr) for a quiet task and no sanctioned way to get it short of deleting the marker and rerunning the whole digest. The intent requires the READ-ONCE contract to stay untouched, so this is the captain's call: add one exception clause (a compacted task's .meta and status log may be read directly) plus an AGENTS.md:182 amendment, or accept the overclaim knowingly.bin/fm-session-start.sh:914-last_status_line(bin/fm-classify-lib.sh:110) isgrep -v '^[[:space:]]*$' "$f" | tail -1- a full read of the task's status log, which is an unbounded append-only lifetime log with no rotation anywhere in bin/. It now runs once per task on every session start, where the loop previously only rantail -n "$STATUS_TAIL"(O(1) on the tail). That lands on the path this file's own header calls the RUNTIME BOUND ("blocks either hook-driven session initialization or Pi's first provider preflight"), and fm-classify-lib.sh:575 built a byte-cursor fold specifically to avoid "re-reading each task's whole lifetime log every time". The round already collapsed two scans into one, so this is the irreducible remainder of the current design rather than a defect; if it ever measures, a boundedtail -n <k>prefilter with a full-read fallback when those k lines are all blank would keep the selected line identical. Noting the tradeoff, no action required..squish/squish.db:1- The stray local MCP database is correctly gone from the tree and now ignored (.gitignore:7, no .squish entry ingit ls-tree -r HEAD, clean worktree), but it was removed in a follow-up commit (eb884fc) rather than amended out of the commit that added it, so commit 74fab6c on this branch still carries a 724992-byte binary that the push will publish. Verified harmless as data:git cat-file blob 74fab6c:.squish/squish.dbcontains schema text only, with no email, token, or password strings. This is repository-history weight, not a leak; a squash merge - which the captain performs - drops it entirely, so no action is needed unless the branch is merged with history preserved.🔧 Fix: exclude reemit from compact, widen fleet-state fingerprint
3 infos still open:
bin/fm-session-start.sh:632-bytes=$(LC_ALL=C wc -c < "$1" 2>/dev/null | tr -d '[:space:]')applies its redirections left to right, so a failing< "$1"is reported before2>/dev/nulltakes effect - the exact defect round 4 already fixed at the marker read (line 974 is correctly writtenread -r OLD_FP 2>/dev/null < "$SEEN_MARKER"). Verified directly in this worktree:b=$(LC_ALL=C wc -c < f 2>/dev/null | tr -d '[:space:]')against a chmod 000 file emitsbash: line 1: f: Permission deniedon stderr, whilewc -c 2>/dev/null < femits nothing; both yield an emptybytes, so the${bytes:-0}fallback is already correct and the fingerprint still errs toward the full block. The only consequence is a raw shell error line naming the status path, injected into an agent-facing digest whose sibling read was deliberately silenced for the same case (and whose test at tests/fm-session-start.test.sh:1497 asserts exactly that silence for the marker). Fix: swap the two redirections.bin/fm-session-start.sh:973- Round 3 of this run removed$(printf ...)subshells from this loop precisely because the file tracks subprocess count on what its own header calls the RUNTIME BOUND; this round reintroduces some.SEEN_MARKER=$(fleet_state_seen_marker "$id")forks a subshell for a two-part string concat thatSEEN_MARKER="$STATE/.session-start-seen-$id"produces identically (the helper is still needed by write_fleet_state_seen_marker at line 644, so keeping it and inlining here is fine). Inside fleet_state_status_bytes (line 632) the| tr -d '[:space:]'fork exists only to strip the leading padding BSD/macOSwcemits, whichbytes=${bytes//[[:space:]]/}does without a process. Net per task: roughly two extra processes on a path whose whole purpose in this change is to make a repeated session start cheaper. Behavior-preserving cleanup only - the selected values are byte-identical either way.bin/fm-session-start.sh:897- Recorded for the audit trail, not as a blocker. The intent text statesWake-Queue, OPEN DECISIONS, UNREAD STATUS und der READ-ONCE-Vertrag bleiben unangetastetand fixes the fingerprint recipe verbatim assha256 aus .meta-Inhalt + letzter Statuszeile. The change rewrites the READ-ONCE CONTRACT heredoc (lines 897-921) and widens the key to also cover the status log's byte size, the endpoint verdict, and the FM_SESSION_START_STATUS_TAIL bound (line 588). Both deviations were explicitly directed by the captain in this run's recorded fix instructions (update the READ-ONCE CONTRACT text, header ORDERING item 6, and AGENTS.md:182 wording to acknowledge the compact form;fold endpoint liveness ... into the fingerprint definition itself;extend the fingerprint to also cover the status log's total content/line-count), so they are authorized amendments to the earlier text rather than contradictions. Verified the other required behaviors are intact: the wake queue, OPEN DECISIONS (bin/fm-wake-drain.sh) and UNREAD STATUS are untouched, the compact line carries every field the intent enumerates, the STATUS TAILS/FINGERPRINT DEDUP block is still emitted verbatim by--help(line 288 sed range covers it), teardown removes the marker on both the main path (line 2881) and the remote-secondmate path (line 687), and docs/architecture.md, docs/configuration.md and AGENTS.md all match the implemented behavior. No action needed.✅ **Test** - passed
✅ No issues found.
FM_TEST_BASE_PATH=<path-without-node> bash tests/fm-session-start.test.sh— 64 assertions pass, including all 14 newtest_fleet_state_fingerprint_*cases (first start full, unchanged compact, .meta change, new status line, missing marker, unreadable marker, read-only session writes no marker, verb cap, verb normalization, absent status log, blank trailing lines, blank-terminated append, --reemit never compacts, alive->dead endpoint flip, repeated last line)FM_TEST_BASE_PATH=<path-without-node> bash tests/fm-teardown.test.sh— 59 assertions pass, including the newtest_teardown_removes_the_session_start_fingerprint_markerManual end-to-end run of the realbin/fm-session-start.shagainst a throwaway FM_HOME built from the suite's own fixtures: 7 scenarios captured as a CLI transcript (first start full → unchanged compact → per-task change restores only that task → dead endpoint restores full → deleting the named marker restores full →--reemitreprints everything full → READ-ONCE CONTRACT wording)git diff --stat d22318e..0d4311e— confirmed bin/fm-watch.sh, bin/fm-wake-drain.sh and bin/fm-classify-lib.sh (build-plan points 1-3) are untouchedgit status --porcelain— worktree left clean, all transient test scratch removed✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.