Fix the macOS coreutils portability bugs in transcript streaming and gc

dev/changelog/mngr-timeout-tac.md

@@ -0,0 +1 @@
+Added a "Portable shell in host commands" section to the repo-root `style_guide.md`. It states the rule that commands passed to a host's `execute_*` methods run under the host's own userland -- BSD on a local macOS machine, GNU on Linux remotes -- and gives the validated portable forms for the cases mngr has hit: `du -sk` over `du -sb`, `stat -c … || stat -f …`, a forward scan over `tac`/`tail -r`, and a perl `alarm` form (with its two pitfalls) for bounding a sub-command in-shell where `timeout(1)` is unavailable.

libs/mngr/changelog/mngr-timeout-tac.md

@@ -0,0 +1,11 @@
+Fixed the remaining places where mngr shelled out to GNU-coreutils binaries and flags that stock macOS does not ship, each of which ran on the user's own machine whenever an agent used the `local` provider. All were verified failing on macOS 26.4.1 with a stock `PATH`, then verified fixed.
+
+Raw-transcript streaming re-emitted already-emitted lines on macOS. `mngr_transcript_reconcile_offset` reverse-scanned the session file with `tac` to find the last emitted line; with `tac` absent the scan read nothing and the offset reset to 0. It now scans forward and tracks the last match, which needs no reverse at all. This also fixes a latent off-by-one: the old code combined `wc -l` (which does not count an unterminated final line) with `tac` (which emits it), so a session file whose last line was still being appended -- the normal state of a live transcript -- reconciled one line too low.
+
+Transcript-streamer startup also got much faster. Both `mngr_transcript_build_id_set` and `mngr_transcript_reconcile_offset` used to extract each line's correlation field through a command substitution, which forks a subshell per line; they now match it with an inline bash regex and fork nothing. The dominant cost was `build_id_set`, which scans the whole already-emitted output: on a 50,000-line file it drops from about 32s to 0.7s on macOS, and about 14s to 0.6s on Linux (three trials each; forks are cheaper on Linux). `reconcile_offset` is subtler -- removing `tac` also removed its early exit on the first match from the end, so the new forward scan always reads the whole file (about 0.7s at 50,000 lines) instead of sometimes stopping early. But because it no longer forks, its worst case -- a match near the start, which made the old reverse scan fork through the entire file at about 12s -- is now that same bounded 0.7s. The inline match uses bash's built-in `[[ =~ ]]`, so unlike the `tac` it replaced it needs no external binary and behaves identically on macOS and Linux.
+
+`mngr gc` mis-measured orphaned work directories on macOS. `du -sb` has no BSD equivalent (`-b` is rejected), and because the command was piped into `cut`, the pipeline exit status was `cut`'s, so the failure looked like success with empty output and every orphan was reported as 0 bytes. `stat -c %Y` is likewise rejected by BSD `stat`, so each orphan's `created_at` fell back to `datetime.now()` and looked brand-new, meaning age-based collection never ran on macOS.
+
+Both `gc` call sites now go through the host interface instead of hand-rolled shell. `OuterHostInterface` gains a concrete `get_directory_size`, alongside the existing `path_exists` and `get_file_mtime`: it runs POSIX `test -d ... && du -sk` on the host -- locally via subprocess, remotely over SSH -- so callers never branch on `is_local`. Sizes are whole kibibytes, since `-k` is the only `du` block size POSIX defines, and a path that is not a directory reports 0. The trailing slash resolves a symlinked path to its target directory, and `du` exiting non-zero is tolerated because it still prints a correct total after skipping an unreadable subdirectory. The exact `du -sk <dir>/` invocation was verified on stock macOS (BSD `du`) and on Linux (GNU and busybox `du`) to agree on the load-bearing cases: a hard-linked inode counted once, a nested symlinked directory not followed, and a top-level symlink resolved.
+
+A further fix originally on this branch -- a perl `alarm` fallback for `timeout(1)` in the tmux message-submission path -- was dropped when #2475 replaced that path's confirmation engine wholesale (durable-evidence polling, no `timeout(1)`). The portable-shell technique it established, including the perl timeout form and its two pitfalls, is preserved in the repo-root `style_guide.md` instead of in code with no remaining caller.

libs/mngr/imbue/mngr/api/gc.py

@@ -809,24 +809,8 @@ def _get_orphaned_work_dirs(host: OnlineHostInterface, provider_name: ProviderIn
     work_dir_infos = []
     for work_dir_str in orphaned_work_dirs:
         work_dir_path = Path(work_dir_str)
-        # Get size if possible
-        size = SizeBytes(0)
-        try:
-            result = host.execute_idempotent_command(f"du -sb {shlex.quote(str(work_dir_path))} | cut -f1")
-            if result.success and result.stdout.strip():
-                size = SizeBytes(int(result.stdout.strip()))
-        except (ValueError, OSError):
-            # If we can't get the size, use 0
-            pass
-
-        # Get creation time from the directory
-        created_at = datetime.now(timezone.utc)
-        try:
-            stat_result = host.execute_idempotent_command(f"stat -c %Y {shlex.quote(str(work_dir_path))}")
-            if stat_result.success and stat_result.stdout.strip():
-                created_at = datetime.fromtimestamp(int(stat_result.stdout.strip()), tz=timezone.utc)
-        except (ValueError, OSError):
-            pass
+        size = host.get_directory_size(work_dir_path)
+        created_at = host.get_file_mtime(work_dir_path) or datetime.now(timezone.utc)
 
         work_dir_infos.append(
             WorkDirInfo(
@@ -1008,21 +992,8 @@ def _local_branches_not_on_any_remote_on_host(host: OnlineHostInterface, repo_pa
 def _build_source_dir_info(
     host: OnlineHostInterface, provider_name: ProviderInstanceName, source_path: Path
 ) -> WorkDirInfo:
-    size = SizeBytes(0)
-    try:
-        result = host.execute_idempotent_command(f"du -sb {shlex.quote(str(source_path))} | cut -f1")
-        if result.success and result.stdout.strip():
-            size = SizeBytes(int(result.stdout.strip()))
-    except (ValueError, OSError):
-        pass
-
-    created_at = datetime.now(timezone.utc)
-    try:
-        stat_result = host.execute_idempotent_command(f"stat -c %Y {shlex.quote(str(source_path))}")
-        if stat_result.success and stat_result.stdout.strip():
-            created_at = datetime.fromtimestamp(int(stat_result.stdout.strip()), tz=timezone.utc)
-    except (ValueError, OSError):
-        pass
+    size = host.get_directory_size(source_path)
+    created_at = host.get_file_mtime(source_path) or datetime.now(timezone.utc)
 
     return WorkDirInfo(
         path=source_path,

libs/mngr/imbue/mngr/conftest.py

@@ -44,6 +44,7 @@
 from imbue.mngr.utils.testing import init_git_repo
 from imbue.mngr.utils.testing import worker_docker_state_prefixes
 from imbue.mngr.utils.testing import worker_test_ids
+from imbue.mngr.utils.testing import write_executable_script
 
 # Resource guards (tmux, rsync, unison, modal, docker_cli, docker_sdk) are
 # registered automatically via the resource_guards entry point group.
@@ -800,3 +801,44 @@ def session_cleanup() -> Generator[None, None, None]:
             "These resources have been cleaned up, but tests should not leak!\n"
             "Please fix the test(s) that failed to clean up properly."
         )
+
+
+# Binaries that exist in GNU coreutils but not in the BSD userland macOS ships.
+_GNU_ONLY_BINARIES: Final[tuple[str, ...]] = ("tac", "timeout")
+
+# macOS ships bash 3.2, which has no associative arrays (`declare -A`).
+_BASH_CANDIDATES: Final[tuple[str, ...]] = ("bash", "/opt/homebrew/bin/bash", "/usr/local/bin/bash")
+_MINIMUM_BASH_MAJOR_VERSION: Final[int] = 4
+
+
+@pytest.fixture
+def posix_only_path(tmp_path_factory: pytest.TempPathFactory) -> dict[str, str]:
+    """An environment whose PATH shadows ``tac`` and ``timeout`` with shims that exit 127.
+
+    Stock macOS ships a BSD userland with neither, so poisoning them makes a
+    reintroduced dependency on either fail on Linux too, rather than passing in CI
+    and breaking only on a user's mac. It is a targeted guard for those two names,
+    not a general macOS-portability check: a GNU-only *flag* on a binary BSD does
+    ship (``du -b``, ``stat -c``) passes straight through it.
+    """
+    poison_dir = tmp_path_factory.mktemp("posix_only_bin")
+    for binary in _GNU_ONLY_BINARIES:
+        write_executable_script(poison_dir / binary, f'#!/bin/sh\necho "{binary}: command not found" >&2\nexit 127\n')
+    env = dict(os.environ)
+    env["PATH"] = f"{poison_dir}{os.pathsep}{env['PATH']}"
+    return env
+
+
+@pytest.fixture
+def bash_with_associative_arrays() -> str:
+    """Path to a bash new enough for ``declare -A``, which the shared shell libs require."""
+    for candidate in _BASH_CANDIDATES:
+        resolved = shutil.which(candidate)
+        if resolved is None:
+            continue
+        probe = subprocess.run(
+            [resolved, "-c", "echo ${BASH_VERSINFO[0]}"], capture_output=True, text=True, timeout=30
+        )
+        if probe.returncode == 0 and int(probe.stdout.strip()) >= _MINIMUM_BASH_MAJOR_VERSION:
+            return resolved
+    pytest.skip(f"no bash >= {_MINIMUM_BASH_MAJOR_VERSION} found (macOS ships 3.2; try `brew install bash`)")

libs/mngr/imbue/mngr/hosts/host_test.py

@@ -2,6 +2,7 @@
 
 import io
 import json
+import os
 import subprocess
 import threading
 from collections.abc import Callable
@@ -3972,3 +3973,40 @@ def test_merge_agent_type_provisioning_combines_provisioning_and_env() -> None:
     result = _merge_agent_type_provisioning(agent_config, options)
     assert result.provisioning.extra_provision_commands == ("echo setup",)
     assert result.environment.env_vars == (EnvVar(key="KEY", value="val"),)
+
+
+def test_get_directory_size_charges_hardlinks_once_and_ignores_nested_symlinks(
+    local_provider: LocalProviderInstance,
+    tmp_path: Path,
+) -> None:
+    """``du -sk`` sizing: a hard-linked inode counts once and a nested symlinked dir is not followed."""
+    host = local_provider.create_host(HostName(LOCAL_HOST_NAME))
+    tree = tmp_path / "tree"
+    (tree / "sub").mkdir(parents=True)
+    (tree / "a").write_bytes(b"\0" * 100 * 1024)
+    os.link(tree / "a", tree / "a_hardlink")
+    (tree / "sub" / "c").write_bytes(b"\0" * 50 * 1024)
+    outside = tmp_path / "outside"
+    outside.mkdir()
+    (outside / "big").write_bytes(b"\0" * 400 * 1024)
+    (tree / "link_to_outside").symlink_to(outside)
+
+    size = host.get_directory_size(tree)
+
+    assert size % 1024 == 0
+    # ``a`` (100 KiB) + ``sub/c`` (50 KiB) + a little directory overhead. Counting
+    # ``a_hardlink`` a second time would reach ~250 KiB; following the nested
+    # symlink would add the 400 KiB behind it.
+    assert 150 * 1024 <= size < 250 * 1024, f"hard-link double-count or followed a nested symlink: {size}"
+
+
+def test_get_directory_size_is_zero_for_a_non_directory(
+    local_provider: LocalProviderInstance,
+    tmp_path: Path,
+) -> None:
+    """A regular file or a missing path reports 0, not an error."""
+    host = local_provider.create_host(HostName(LOCAL_HOST_NAME))
+    a_file = tmp_path / "file"
+    a_file.write_text("x")
+    assert host.get_directory_size(a_file) == 0
+    assert host.get_directory_size(tmp_path / "does_not_exist") == 0

libs/mngr/imbue/mngr/interfaces/host.py

@@ -26,6 +26,7 @@
 from imbue.mngr.interfaces.data_types import HostLifecycleOptions
 from imbue.mngr.interfaces.data_types import HostResources
 from imbue.mngr.interfaces.data_types import PyinfraConnector
+from imbue.mngr.interfaces.data_types import SizeBytes
 from imbue.mngr.interfaces.data_types import SnapshotInfo
 from imbue.mngr.interfaces.data_types import VolumeFile
 from imbue.mngr.primitives import ActivitySource
@@ -427,6 +428,26 @@ def path_exists(self, path: Path) -> bool:
             return path.exists()
         return self.execute_idempotent_command(f"test -e {shlex.quote(str(path))}", timeout_seconds=5.0).success
 
+    def get_directory_size(self, path: Path) -> SizeBytes:
+        """Disk space used by ``path`` and its contents, or 0 if it is not a directory.
+
+        Measured with POSIX ``du -sk``: totals are whole kibibytes, since ``-k`` is
+        the only block size POSIX defines. The trailing slash resolves a symlinked
+        ``path`` to its target directory, and ``du``'s non-zero exit after skipping
+        an unreadable entry is tolerated, since it still prints a correct total.
+        """
+        quoted_path = shlex.quote(f"{path}/")
+        result = self.execute_idempotent_command(
+            f"test -d {quoted_path} && {{ du -sk {quoted_path} 2>/dev/null || true; }}",
+            timeout_seconds=30.0,
+        )
+        if not result.success or not result.stdout.strip():
+            return SizeBytes(0)
+        kibibytes = result.stdout.split()[0]
+        if not kibibytes.isdigit():
+            return SizeBytes(0)
+        return SizeBytes(int(kibibytes) * 1024)
+
 
 class OnlineHostInterface(HostInterface, OuterHostInterface, ABC):
     """Interface for hosts that are currently online and accessible for operations.

libs/mngr/imbue/mngr/resources/mngr_transcript_lib.sh

@@ -5,24 +5,17 @@
 # share the parts of raw-transcript capture that are structurally identical
 # regardless of the agent's native session schema:
 #
-#   - mngr_transcript_extract_field FIELD LINE
-#       Extract the first top-level "<FIELD>": "<value>" string from one JSONL
-#       line. Used to read uuid / id / similar correlation fields without
-#       requiring jq. Echoes the bare value (no quotes) on stdout, empty if no
-#       match. Always returns 0 (safe under `set -e` command substitution).
-#
 #   - mngr_transcript_build_id_set OUTPUT_FILE FIELD
 #       Populate the associative array _MNGR_TRANSCRIPT_ID_SET with every value
 #       of FIELD found in OUTPUT_FILE. Used at startup and when a late-
-#       appearing session file needs reconciliation. Iterates line-by-line via
-#       mngr_transcript_extract_field so the extraction is consistent with the
-#       per-line lookup in mngr_transcript_reconcile_offset.
+#       appearing session file needs reconciliation. Correlation fields (uuid,
+#       id) are matched with a bash regex, so no jq is required.
 #
 #   - mngr_transcript_reconcile_offset SESSION_FILE FIELD
-#       Echo the largest line offset N such that lines 1..N of SESSION_FILE
-#       have FIELD values already present in _MNGR_TRANSCRIPT_ID_SET. Used to
-#       recover after a crash that may have dropped the stored offset. Echoes
-#       0 if no match is found or the file is empty.
+#       Echo the highest 1-indexed line of SESSION_FILE whose FIELD value is
+#       already present in _MNGR_TRANSCRIPT_ID_SET, or 0 if there is none. Used
+#       to recover after a crash that may have dropped the stored offset. Scans
+#       forward, so it needs no `tac` (GNU coreutils, absent on macOS).
 #
 #   - mngr_transcript_emit_lines_range SESSION_FILE START END OUTPUT_FILE
 #       Append lines START..END (inclusive, 1-indexed) of SESSION_FILE to
@@ -40,22 +33,15 @@
 # (so it survives function-local scope) and for clearing it when no longer
 # needed.
 
-# Extract a top-level JSON string field from a single line.
+# The ERE matching a top-level "<FIELD>": "<value>" pair, capturing the value.
+# Matched inline in the loops below: a command substitution would fork a subshell
+# per line, and a session file runs to tens of thousands of lines.
 #
-# Limitations: matches the *first* "<FIELD>": "<value>" in the line. Nested
-# occurrences inside arrays / sub-objects with the same key are also matched
-# if they precede the top-level one, but in practice agent-emitted JSONL
-# events keep correlation fields (uuid, id) at the top so the first match
-# is the right one.
-mngr_transcript_extract_field() {
-    local field="$1"
-    local line="$2"
-    local pattern="\"${field}\"[[:space:]]*:[[:space:]]*\"([^\"]*)\""
-    if [[ "$line" =~ $pattern ]]; then
-        printf '%s\n' "${BASH_REMATCH[1]}"
-    fi
-    return 0
-}
+# Limitations: matches the *first* such pair in the line. Agent-emitted JSONL
+# events keep correlation fields (uuid, id) at the top, so the first match is the
+# right one. FIELD is interpolated into this ERE literally (as printf's %s), so
+# callers must pass a regex-safe field name; the uuid/id callers do.
+_MNGR_TRANSCRIPT_FIELD_ERE='"%s"[[:space:]]*:[[:space:]]*"([^"]*)"'
 
 # Build _MNGR_TRANSCRIPT_ID_SET from every FIELD value in OUTPUT_FILE.
 mngr_transcript_build_id_set() {
@@ -65,20 +51,20 @@ mngr_transcript_build_id_set() {
     if [ ! -s "$output_file" ]; then
         return 0
     fi
-    local line value
+    local pattern
+    printf -v pattern "$_MNGR_TRANSCRIPT_FIELD_ERE" "$field"
+    local line
     while IFS= read -r line; do
-        value=$(mngr_transcript_extract_field "$field" "$line")
-        if [ -n "$value" ]; then
-            _MNGR_TRANSCRIPT_ID_SET["$value"]=1
+        if [[ "$line" =~ $pattern ]] && [ -n "${BASH_REMATCH[1]}" ]; then
+            _MNGR_TRANSCRIPT_ID_SET["${BASH_REMATCH[1]}"]=1
         fi
     done < "$output_file"
 }
 
-# Reverse-scan SESSION_FILE to find the last emitted line.
+# Scan SESSION_FILE to find the last emitted line.
 #
-# Walks the file backwards (via `tac`) and returns the 1-indexed line number
-# whose FIELD value is already in _MNGR_TRANSCRIPT_ID_SET. Returns 0 if no
-# already-emitted line is found.
+# Returns the highest 1-indexed line number whose FIELD value is already in
+# _MNGR_TRANSCRIPT_ID_SET, or 0 if no already-emitted line is found.
 mngr_transcript_reconcile_offset() {
     local session_file="$1"
     local field="$2"
@@ -88,23 +74,24 @@ mngr_transcript_reconcile_offset() {
         return 0
     fi
 
-    local file_lines
-    file_lines=$(wc -l < "$session_file")
-    file_lines=${file_lines// /}
-
-    local reverse_idx=0
-    local line value found
+    local pattern
+    printf -v pattern "$_MNGR_TRANSCRIPT_FIELD_ERE" "$field"
+    local idx=0
+    local found=0
+    local line value
+    # read skips an unterminated final line, matching wc -l and the sed emit
+    # range, so a partially-written final record is deferred, not miscounted.
     while IFS= read -r line; do
-        reverse_idx=$((reverse_idx + 1))
-        value=$(mngr_transcript_extract_field "$field" "$line")
-        if [ -n "$value" ] && [ "${_MNGR_TRANSCRIPT_ID_SET[$value]+exists}" ]; then
-            found=$((file_lines - reverse_idx + 1))
-            echo "$found"
-            return 0
+        idx=$((idx + 1))
+        if [[ "$line" =~ $pattern ]]; then
+            value="${BASH_REMATCH[1]}"
+            if [ -n "$value" ] && [ "${_MNGR_TRANSCRIPT_ID_SET[$value]+exists}" ]; then
+                found=$idx
+            fi
         fi
-    done < <(tac "$session_file")
+    done < "$session_file"
 
-    echo 0
+    echo "$found"
 }
 
 # Append lines START..END of SESSION_FILE to OUTPUT_FILE.