fix(workflows): enforce hermetic include references

.archon/workflows/defaults/archon-workflow-builder.yaml

@@ -144,7 +144,7 @@ nodes:
           # Use for deterministic data transforms the shell would mangle (JSON parsing, etc.)
           script: |
             // JSON is valid JS expression syntax — assign directly (String.raw breaks on backticks)
-            const data = $other-node.output;
+            const data = $<other-node>.output;
             console.log(JSON.stringify({ count: data.items.length }));
           runtime: bun                       # required: 'bun' (.ts/.js) or 'uv' (.py)
           # deps: [requests]                 # uv only
@@ -199,10 +199,10 @@ nodes:
       5. Use `bash` nodes for deterministic shell operations (file checks, git commands, installs)
       6. Use `script` nodes for typed data transforms (TypeScript JSON parsing, Python with deps)
          — stdout is captured as output, stderr is forwarded as a warning.
-         `$nodeId.output` is NOT shell-quoted in script bodies.
-         - **TypeScript/bun**: assign directly — `const data = $nodeId.output;`
+         `$<node-id>.output` is NOT shell-quoted in script bodies.
+         - **TypeScript/bun**: assign directly — `const data = $<node-id>.output;`
            (JSON is valid JS expression syntax; avoid String.raw — it breaks on backticks)
-         - **Python/uv**: use json.loads — `import json; data = json.loads("""$nodeId.output""")`
+         - **Python/uv**: use json.loads — `import json; data = json.loads("""$<node-id>.output""")`
          Never interpolate into shell syntax.
       7. Use `prompt` nodes for AI reasoning tasks
       8. Use `approval` nodes to pause for human review at risky gates (plan→execute boundary, destructive actions)

packages/docs-web/src/content/docs/guides/authoring-workflows.md

@@ -948,13 +948,15 @@ written the nodes by hand. There is no separate child run.
   `id: review` yields `review__verify-pr-base`, `review__sync`, `review__implement-fixes`,
   and so on. These namespaced ids are what appear in the event stream and in
   `archon workflow get <id>`.
-- **Edges.** Internal `depends_on` edges and `$id.output` references in inline node text are
-  rewired to the namespaced ids automatically. Named `command:` and `loop.command` files
-  remain external and cannot be rewritten; when a readable command body references a
-  top-level block node whose id will be namespaced, workflow loading fails. This best-effort
-  scan includes nested `loop_group` bodies; unresolved files warn and are skipped. The include
-  node's own `depends_on` / `when` / `trigger_rule` attach to the block's **entry** nodes
-  (those with no upstream inside the block).
+- **Edges and command bodies.** Internal `depends_on` edges and `$id.output` references are
+  rewired to the namespaced ids automatically. This includes named `command:` and
+  `loop.command` files: discovery resolves their bodies and compiles them into the flat DAG
+  before namespacing. Compilation recurses through nested `loop_group` bodies. An unresolved
+  included command cannot start a fresh execution because its references cannot be proven safe.
+  Command nodes fail composition immediately; loop commands retain a private compilation error
+  so a paused loop can still resume from its persisted, validated prompt snapshot.
+  The include node's own `depends_on` / `when` / `trigger_rule` attach to the block's
+  **entry** nodes (those with no upstream inside the block).
 - **Sink asymmetry (a downstream node depending on the include).** A `depends_on:
   [<includeId>]` on a downstream node fans out to **all** of the block's sink nodes (every
   node with no dependents inside the block), so it waits for the whole block to finish.
@@ -1006,21 +1008,26 @@ Substitution applies everywhere the value could reach the model or the shell, in
 inside Markdown code fences and inline code spans — `$INPUTS.<name>` has no
 documentation-only meaning, so a fenced occurrence is still a live parameter.
 
-#### Command bodies cannot use include inputs
+#### Command bodies use the same explicit interface
 
-Phase 1 cannot parameterize a `command:` file or `loop.command` file used by an included
-block. Command bodies are read at execution time, after load-time include expansion has
-finished. When such a file can be read at load time and contains `$INPUTS.<name>` anywhere —
-including inside a code fence — workflow loading fails with a message directing you to inline
-the prompt text. Use an inline `prompt:` when the block needs include inputs.
+Named `command:` and `loop.command` files are the preferred home for substantial prompts and
+can use both workflow-local `$node.output` references and declared `$INPUTS.<name>` values.
+For an `include:`, Archon resolves and snapshots the command body during load-time composition,
+then applies the same input binding and node-id namespacing as an inline prompt. The authored
+workflow stays command-first; the executor receives a deterministic flat DAG.
 
-This check is best-effort, so a clean load is not a guarantee. It covers `command:` and
-`loop.command` files throughout the block, including nested `loop_group` bodies. A command
-file that cannot be resolved at load time is logged as a warning and skipped rather than
-failing the workflow. This restriction applies to `include:`
-only — a `workflow:` sub-run's named inputs **do** reach `command:` bodies, because they
-resolve at runtime rather than at load time (see the binding-time table in
-[Workflow Signature](#workflow-signature-inputs-returns-and-inputs)).
+Every live reference in the command body must belong to the included workflow's lexical node
+scope. A direct `$caller.output` reference is rejected whether or not the parent happens to
+have a node called `caller`; declare an input and pass it with `with:` instead. Failure to
+resolve or read an included command is never a warning or best-effort bypass: a fresh execution
+fails before an AI turn. A paused loop remains resumable from its saved prompt snapshot even if
+the command is later deleted or made invalid.
+Canonical references are live even inside Markdown code fences and inline code because runtime
+substitution is syntax-agnostic.
+
+Named `script:` files are different: they are opaque programs, not prompt templates. Archon
+does not scan or rewrite their source. They receive declared inputs through the documented
+`INPUTS_<UPPER_SNAKE>` environment variables.
 
 ### Non-goals (Phase 1)
 
@@ -1088,12 +1095,13 @@ which surfaces can read it:
 
 | Caller | When `$INPUTS` resolves | Reaches `prompt:`/`bash:`/`script:` | Reaches `command:` file bodies |
 |--------|-------------------------|-------------------------------------|--------------------------------|
-| `include:` | **Load time** (the block is inlined; values are spliced into node text) | Yes | **No** — a command file is read after expansion, so `$INPUTS` in one is a load error |
+| `include:` | **Load time** (the block and command bodies are compiled into the flat DAG) | Yes | **Yes** — resolved command bodies receive the same input binding before execution |
 | `workflow:` sub-run | **Runtime** (values become `$INPUTS` variables on the child run) | Yes | **Yes** — every child node flows through runtime substitution |
 
-That asymmetry is the point: a sub-run's named inputs reach `command:` bodies precisely because
-they resolve at runtime, where an include's load-time macro cannot. Sub-run inputs are also
-persisted to the child run's metadata at spawn, so `$INPUTS` reconstitutes on a cold resume.
+Both composition paths support command-backed prompts; their binding time differs. Includes
+snapshot and bind the resolved command body at load time, while sub-runs resolve inputs at
+runtime. Sub-run inputs are persisted to the child run's metadata at spawn, so `$INPUTS`
+reconstitutes on a cold resume.
 
 ### `$INPUTS` in `bash:`/`script:` nodes uses env vars
 

packages/workflows/src/compiled-command.ts

@@ -0,0 +1,11 @@
+/** Engine-private included-loop compilation metadata. Symbols survive object spreads
+ * but stay out of YAML, JSON, API payloads, and persisted workflow definitions. */
+export const COMPILED_LOOP_COMMAND = Symbol('archon.compiled-loop-command');
+
+export type CompiledLoopCommand =
+  | { prompt: string; error?: never }
+  | { prompt?: never; error: string };
+
+export interface LoopWithCompiledCommand {
+  [COMPILED_LOOP_COMMAND]?: CompiledLoopCommand;
+}

packages/workflows/src/dag-executor.test.ts

@@ -8469,6 +8469,145 @@ describe('executeDagWorkflow -- resume with priorCompletedNodes', () => {
       expect(failed).toHaveLength(0);
     });
 
+    it('reuses the pause-time prompt snapshot after a composed loop prompt is rediscovered', async () => {
+      mockSendQueryDag.mockImplementation(function* () {
+        yield { type: 'assistant', content: 'iteration output, no signal yet' };
+        yield { type: 'result', sessionId: 'sid-prompt-1' };
+      });
+
+      const platform = createMockPlatform();
+      const firstDeps = createMockDeps();
+      const blockWorkflow = {
+        name: 'materialized-loop-block',
+        description: 'Command-backed loop block',
+        nodes: [
+          {
+            id: 'gated-loop',
+            loop: {
+              command: 'materialized-loop-command',
+              until: 'COMPLETE',
+              max_iterations: 5,
+              interactive: true,
+              gate_message: 'Review materialized prompt.',
+            },
+          } as unknown as DagNode,
+        ],
+      } as WorkflowDefinition;
+      const parentWorkflow = {
+        name: 'materialized-loop-gated',
+        description: 'Includes the command-backed loop',
+        nodes: [{ id: 'included', include: 'materialized-loop-block' } as unknown as DagNode],
+      } as WorkflowDefinition;
+      const rawWorkflows = new Map([
+        [blockWorkflow.name, blockWorkflow],
+        [parentWorkflow.name, parentWorkflow],
+      ]);
+      const firstExpansion = expandWorkflowIncludes(
+        rawWorkflows,
+        new Map([
+          ['materialized-loop-command', 'ORIGINAL materialized command. USER=<<$LOOP_USER_INPUT>>'],
+        ])
+      );
+      const originalWorkflow = firstExpansion.workflows.get(parentWorkflow.name)!;
+      expect(firstExpansion.errors).toHaveLength(0);
+
+      await executeDagWorkflow(
+        firstDeps,
+        platform,
+        'conv-dag',
+        testDir,
+        originalWorkflow,
+        makeWorkflowRun(),
+        'claude',
+        undefined,
+        join(testDir, 'artifacts'),
+        join(testDir, 'state'),
+        join(testDir, 'logs'),
+        'main',
+        'docs/',
+        minimalConfig
+      );
+
+      const pauseCalls = (
+        firstDeps.store.pauseWorkflowRun as Mock<
+          (id: string, ctx: Record<string, unknown>) => Promise<void>
+        >
+      ).mock.calls;
+      const pausedContext = pauseCalls[0]?.[1] as Record<string, unknown>;
+      expect(pausedContext.commandSnapshot).toContain('ORIGINAL materialized command.');
+
+      mockSendQueryDag.mockClear();
+      mockSendQueryDag.mockImplementation(function* () {
+        yield { type: 'assistant', content: 'refined. <promise>COMPLETE</promise>' };
+        yield { type: 'result', sessionId: 'sid-prompt-2' };
+      });
+      // Cold rediscovery after the source command was deleted still returns the
+      // workflow. Its engine-private compilation error blocks a fresh run, while
+      // this resumed run can reach and reuse the persisted snapshot.
+      const rediscovery = expandWorkflowIncludes(
+        rawWorkflows,
+        new Map([['materialized-loop-command', null]])
+      );
+      expect(rediscovery.errors).toHaveLength(0);
+      const rediscoveredWorkflow = rediscovery.workflows.get(parentWorkflow.name)!;
+      mockSendQueryDag.mockClear();
+      const freshStore = createMockStore();
+      await executeDagWorkflow(
+        createMockDeps(freshStore),
+        platform,
+        'conv-dag',
+        testDir,
+        rediscoveredWorkflow,
+        makeWorkflowRun('fresh-invalid-command-run'),
+        'claude',
+        undefined,
+        join(testDir, 'artifacts'),
+        join(testDir, 'state'),
+        join(testDir, 'logs'),
+        'main',
+        'docs/',
+        minimalConfig
+      );
+      expect(mockSendQueryDag).not.toHaveBeenCalled();
+      const freshFailures = (
+        freshStore.createWorkflowEvent as ReturnType<typeof mock>
+      ).mock.calls.filter(
+        (call: unknown[]) => (call[0] as Record<string, unknown>).event_type === 'node_failed'
+      );
+      expect(freshFailures).toHaveLength(1);
+
+      const resumedRun = makeWorkflowRun('materialized-resume-run', {
+        metadata: {
+          approval: { ...pausedContext },
+          loop_user_input: 'tighten the summary',
+          loop_feedback_given: true,
+        },
+      });
+
+      mockSendQueryDag.mockClear();
+      await executeDagWorkflow(
+        createMockDeps(createMockStore()),
+        platform,
+        'conv-dag',
+        testDir,
+        rediscoveredWorkflow,
+        resumedRun,
+        'claude',
+        undefined,
+        join(testDir, 'artifacts'),
+        join(testDir, 'state'),
+        join(testDir, 'logs'),
+        'main',
+        'docs/',
+        minimalConfig
+      );
+
+      const resumedPrompt = mockSendQueryDag.mock.calls[0]?.[0] as string;
+      expect(resumedPrompt).toContain('ORIGINAL materialized command.');
+      expect(resumedPrompt).toContain('USER=<<tighten the summary>>');
+      expect(resumedPrompt).not.toContain('could not be resolved');
+    });
+
     it('closes the loop lifecycle with exactly one node_failed on max-iterations exhaustion', async () => {
       // Failure finalizer contract: every failed exit after node_started goes
       // through one finalizer — exactly one node_failed row per started loop

packages/workflows/src/dag-executor.ts

@@ -86,6 +86,7 @@ import {
 } from './output-ref';
 import { buildTruncationMarker } from './utils/output-truncation';
 import { writeNodeArtifact, readNodeArtifacts } from './artifacts-index';
+import { COMPILED_LOOP_COMMAND, type LoopWithCompiledCommand } from './compiled-command';
 import {
   logNodeStart,
   logNodeComplete,
@@ -760,10 +761,10 @@ function shellQuoteOrFile(
  * Substitute $node_id.output and $node_id.output.field references in a prompt.
  * Called AFTER the standard substituteWorkflowVariables pass.
  *
- * KEEP IN SYNC (three ref-surface enumerations must agree): the fields this is called on
- * (search call sites below), the loader's validateDagStructure scan (which validates the
- * same refs), and rewriteNodeOutputRefs in include-expander.ts (which renames them on
- * inline). Adding a substituted field to one means updating all three.
+ * KEEP IN SYNC (four ref-surface enumerations must agree): the fields this is called on
+ * (search call sites below), the loader's validateDagStructure scan, rewriteNodeOutputRefs,
+ * and applyInputsMacro in include-expander.ts. Adding a substituted field to one means
+ * checking all four.
  *
  * @param escapedForBash - When true, wraps substituted values in single quotes so
  *   they are safe to embed in bash scripts passed to `bash -c`. Set true only for
@@ -4190,23 +4191,27 @@ async function executeLoopNode(
     return { state: 'completed', output: finalizeOutput, sessionId: currentSessionId };
   }
 
-  // Resolve the iteration prompt source. `loop.prompt` is used directly;
-  // `loop.command` is read ONCE per run/node: the first invocation loads the
-  // command file, and the interactive gate persists the loaded text
-  // (`commandSnapshot` in the pause context) so a resumed invocation reuses the
-  // snapshot instead of re-reading — a command file edited or deleted while the
-  // run sat paused at a gate can neither change nor break the running loop's
-  // prompt. The schema guarantees exactly one of prompt/command is defined.
+  // Resolve the iteration prompt source once per run/node. The interactive gate
+  // persists the resolved template (`commandSnapshot` in the pause context) for
+  // both inline and command-backed loops. This also covers included commands,
+  // which composition materializes as inline prompts: rediscovery after a pause
+  // cannot change their running prompt if the source command is edited or deleted.
+  // The schema guarantees exactly one of prompt/command is defined.
   let loopPromptTemplate: string;
-  if (typeof loop.prompt === 'string') {
-    loopPromptTemplate = loop.prompt;
+  if (isLoopResume && typeof loopGateMeta?.commandSnapshot === 'string') {
+    loopPromptTemplate = loopGateMeta.commandSnapshot;
   } else if (typeof loop.command === 'string') {
-    if (isLoopResume && typeof loopGateMeta?.commandSnapshot === 'string') {
-      loopPromptTemplate = loopGateMeta.commandSnapshot;
+    const compiled = (loop as typeof loop & LoopWithCompiledCommand)[COMPILED_LOOP_COMMAND];
+    if (compiled?.error !== undefined) {
+      getLog().error(
+        { nodeId: node.id, command: loop.command, error: compiled.error },
+        'loop_node.command_compilation_failed'
+      );
+      return failLoopNode(compiled.error, { data: { command: loop.command } });
+    }
+    if (compiled?.prompt !== undefined) {
+      loopPromptTemplate = compiled.prompt;
     } else {
-      // Fresh execution — or a resume of a run paused under a build that
-      // predates commandSnapshot: fall back to a fresh read (documented,
-      // fail-safe) rather than failing an otherwise-valid resume.
       const promptResult = await loadCommandPrompt(
         deps,
         cwd,
@@ -4218,12 +4223,12 @@ async function executeLoopNode(
           { nodeId: node.id, command: loop.command, error: promptResult.message },
           'loop_node.command_load_failed'
         );
-        // The failing command name travels on the node_failed payload so the
-        // event stream carries the same context as the structured log.
         return failLoopNode(promptResult.message, { data: { command: loop.command } });
       }
       loopPromptTemplate = promptResult.content;
     }
+  } else if (typeof loop.prompt === 'string') {
+    loopPromptTemplate = loop.prompt;
   } else {
     // Unreachable: superRefine on loopNodeConfigSchema enforces exactly-one.
     throw new Error(
@@ -5083,10 +5088,10 @@ async function executeLoopNode(
         // Usage consumed up to this gate, so a bare approve (finalize, no re-run)
         // can persist it on node_completed instead of reporting nothing (#2333).
         signaledTokens: completionDetected ? (loopTotalTokens ?? null) : null,
-        // Read-once command body for command-backed loops: the resumed invocation
-        // reuses this snapshot instead of re-reading the file (explicit null for
-        // prompt-based loops — same json_patch convention as `sessionId`).
-        commandSnapshot: typeof loop.command === 'string' ? loopPromptTemplate : null,
+        // Read-once resolved template for both prompt- and command-backed loops.
+        // Included commands are materialized as prompts during composition, so
+        // snapshotting both forms preserves their resume determinism too.
+        commandSnapshot: loopPromptTemplate,
       });
       // Return completed — the between-layer status check sees 'paused' and halts cleanly.
       // This mirrors the approval-node pattern, preventing false "DAG nodes failed" warnings