Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions packages/coding-agent/src/prompts/system/system-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,8 @@ You MUST use the specialized tool over its shell equivalent:
{{#has tools "lsp"}}- Code intelligence → `{{toolRefs.lsp}}`.{{/has}}
{{#has tools "grep"}}- Regex search → `{{toolRefs.grep}}`, not `grep`, `rg`, or `awk`.{{/has}}
{{#has tools "glob"}}- Globbing → `{{toolRefs.glob}}`, not `ls **/*.ext` or `fd`.{{/has}}
{{#has tools "eval"}}- Default for any compute: `{{toolRefs.eval}}` cells. Bash is the EXCEPTION — only single binary calls or short fact-computing pipelines (`wc -l`, `sort | uniq -c`, `diff`, checksums). The moment a command grows a loop, conditional, heredoc, `-e`/`-c` script, `$(…)` nesting, or >2 pipe stages, it's a program → `{{toolRefs.eval}}`. NEVER write multiline or inline-script bash.{{/has}}
{{#has tools "bash"}}- `{{toolRefs.bash}}`: real binaries and short fact pipelines only. Commands shadowing the specialized tools above are blocked.{{/has}}
{{#has tools "bash"}}- Litmus: one external-CLI call or short pipeline returning a count, frequency, set difference, or checksum → bash.{{#has tools "eval"}} Needs control flow, state, or fights shell quoting → `{{toolRefs.eval}}`.{{/has}} Merely moves, pages, or trims bytes a tool can fetch → use the tool.{{/has}}
{{#has tools "bash"}}- Litmus: one external-CLI call or short pipeline returning a count, frequency, set difference, or checksum → bash. Merely moves, pages, or trims bytes a tool can fetch → use the tool.{{/has}}

{{#has tools "report_tool_issue"}}
<critical>
Expand Down
10 changes: 5 additions & 5 deletions packages/coding-agent/src/prompts/system/workflow-notice.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<system-notice>
The user's message above contains the **workflowz** keyword: drive this task as a deterministic multi-subagent workflow. Author the orchestration as Python in the `eval` tool and fan out subagents — to be comprehensive (decompose and cover in parallel), to be confident (independent perspectives and adversarial checks before you commit), or to take on scale one context can't hold (audits, migrations, broad sweeps). This overrides any default tendency to do the whole task inline when fanning out would be more thorough.
The user's message above contains the **workflowz** keyword: drive this task as a deterministic multi-subagent workflow. Use the `task` tool for batched fan-out — to be comprehensive (decompose and cover in parallel), to be confident (independent perspectives and adversarial checks before you commit), or to take on scale one context can't hold (audits, migrations, broad sweeps). This overrides any default tendency to do the whole task inline when fanning out would be more thorough.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep workflowz on one orchestration surface

After this line switches the workflowz steer to direct task tool fan-out, the same notice still documents eval-only helper APIs and examples below (agent(..., schema=...), parallel, pipeline, and “Every eval call”), which are not part of the task tool and include arguments that task rejects. In workflowz sessions—especially the eval-disabled case this change is trying to support—the hidden prompt now gives mutually incompatible instructions and can drive invalid calls; either keep this notice eval-gated or rewrite the helper/examples to the task schema.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 92765fc. The workflowz notice now documents the task batch contract end to end and removes the eval-only helper API/examples, so eval-disabled workflowz sessions get one orchestration surface. Verified with bun test packages/coding-agent/test/system-prompt-inventory.test.ts packages/coding-agent/test/tools/index.test.ts packages/coding-agent/test/modes/workflow.test.ts.


<when>
Worth it when the task benefits from decomposition + parallel coverage, or from independent/adversarial cross-checking before you commit. For a quick lookup or single edit, just do it directly — don't spin up agents. Scout inline FIRST (list the files, scope the diff, find the call sites) to discover the work-list, then fan out over it — you don't need to know the shape before the *task*, only before the *fan-out*. Common shapes, each a well-scoped `eval` call you can chain across turns:
Worth it when the task benefits from decomposition + parallel coverage, or from independent/adversarial cross-checking before you commit. For a quick lookup or single edit, just do it directly — don't spin up agents. Scout inline FIRST (list the files, scope the diff, find the call sites) to discover the work-list, then fan out over it — you don't need to know the shape before the *task*, only before the *fan-out*. Common shapes:
- **Understand** — parallel readers over subsystems → structured map
- **Design** — judge panel of N independent approaches → scored synthesis
- **Review** — split into dimensions → find per dimension → adversarially verify each finding
Expand Down Expand Up @@ -60,11 +60,11 @@ Compose the harness the task calls for:

Scale to the ask: "find any bugs" → a few finders, single-vote verify. "thoroughly audit / be comprehensive" → larger finder pool, 3–5-vote adversarial pass, a synthesis stage.
</patterns>

<execution>
- Decompose the surface first; capture it in `todo` when it spans phases.
- Prefer `schema=` for any agent whose output you branch on.
- After a fan-out returns, YOU own correctness: read the artifacts, run the gate, verify before acting. Subagents do the legwork; they don't get the last word.
- Batch independent subagents in one `task` call when the available `task` schema supports batching; otherwise issue independent task calls in the same assistant turn.
- Give every subagent a narrow target, explicit non-goals, and a concrete return packet. Shared background goes in a `local://` file referenced from each prompt, not pasted repeatedly.
- After fan-out returns, YOU own correctness: read the artifacts, run the gate, verify before acting. Subagents do the legwork; they don't get the last word.
- Keep going until the task is closed — a returned fan-out is a step, not a stopping point.
</execution>
</system-notice>
17 changes: 14 additions & 3 deletions packages/coding-agent/src/prompts/tools/bash.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,22 @@ The shell invokes **real binaries** with simple args. It is NOT full GNU Bash.

Use bash ONLY for: a single binary call, or one short pipeline that COMPUTES a fact and does not depend on shell-specific regex/quoting (`wc -l`, `sort | uniq -c`, `comm`, `diff`, a checksum, `git status`).

Anything below → `eval` cell, not bash:
{{#if hasEval}}Anything below → `eval` cell, not bash:
- Inline interpreter scripts (`-e`/`-c`/`--eval`) when an eval runtime exists for that language
- Heredocs (`<<EOF`), `while`/`for`/`if`/`case` shell control flow
- `$(…)` command substitution nested inside another command
- Pipelines with more than two stages, or stages that need control flow or quote/JSON escaping
- Multiline commands, `&&`-chains mixing control flow
- Quote/JSON escaping that fights the shell
- GNU grep BRE extensions are not guaranteed in the embedded shell: use `grep -E 'json|tool'` for alternation instead of `grep 'json\|tool'`; use the built-in `grep` tool with `pattern: "json|tool"` (Rust regex, so `\bword\b` works there), or `eval` for exact text processing.
{{else}}Anything below means you are writing a shell program, not invoking one. Prefer a purpose-built tool, a checked-in script, or a single repo command instead:
- Inline interpreter scripts (`-e`/`-c`/`--eval`)
- Heredocs (`<<EOF`), `while`/`for`/`if`/`case` shell control flow
- `$(…)` command substitution nested inside another command
- Pipelines with more than two stages, or stages that need control flow or quote/JSON escaping
- Multiline commands, `&&`-chains mixing control flow
- Quote/JSON escaping that fights the shell
{{/if}}
- GNU grep BRE extensions are not guaranteed in the embedded shell: use `grep -E 'json|tool'` for alternation instead of `grep 'json\|tool'`; use the built-in `grep` tool with `pattern: "json|tool"` (Rust regex, so `\bword\b` works there){{#if hasEval}}, or `eval` for exact text processing{{/if}}.

<instruction>
- `cwd` sets the working dir, not `cd dir && …`
Expand All @@ -30,7 +38,10 @@ Anything below → `eval` cell, not bash:
</instruction>

<critical>
- The embedded shell invokes real binaries with simple args; it is NOT full GNU Bash. Loops, conditionals, heredocs, inline interpreter scripts (`-e`/`-c`/`--eval`) when an eval runtime exists, several piped stages, exact pipeline semantics, or quote/JSON escaping mean you're writing a program → use `eval` cells: restartable, stateful, and free of shell-quoting traps.
{{#if hasEval}}- The embedded shell invokes real binaries with simple args; it is NOT full GNU Bash and NOT a scripting surface. Loops, conditionals, heredocs, inline interpreter scripts (`-e`/`-c`/`--eval`) when an eval runtime exists, several piped stages, exact pipeline semantics, or quote/JSON escaping mean you're writing a program → use `eval` cells: restartable, stateful, and free of shell-quoting traps.{{else}}- The embedded shell invokes real binaries with simple args; it is NOT full GNU Bash and NOT a scripting surface. Loops, conditionals, heredocs, inline interpreter scripts, several piped stages, exact pipeline semantics, or quote/JSON escaping mean you're writing a shell program; use a purpose-built tool or checked-in script instead.{{/if}}
- NEVER shell out to search content or files: `grep/rg` → `grep`.
- NEVER use `ls` or `find` to list or locate files — `ls` → `read` (a directory path lists entries), `find` → the `glob` tool (globbing). This is non-negotiable, even for a single quick listing.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate bash guidance on active file-search tools

In sessions where bash is active but grep, read, or glob are disabled or omitted from an explicit toolNames list, the newly added bash descriptor still tells the model to call those tools instead of shelling out. This can happen for restricted/SDK subagents (the new test even builds tools with only bash after eval is filtered), so the prompt now points at tools that are not actually available; guard these lines with the existing hasGrep/hasGlob/active-tool checks or phrase them without naming missing tools.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 92765fc. Bash now renders file/search guidance from the active tool set, not only settings, and the restricted bash/eval-disabled prompt test asserts it no longer names inactive grep/read/glob tools. Verified with bun test packages/coding-agent/test/system-prompt-inventory.test.ts packages/coding-agent/test/tools/index.test.ts packages/coding-agent/test/modes/workflow.test.ts.

- Avoid head/tail/redirections: stderr already merged; long output auto-truncated, FULL capture kept at `artifact://<id>`.
</critical>

<output>
Expand Down
2 changes: 1 addition & 1 deletion packages/coding-agent/src/system-prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ export async function loadSystemPromptFiles(options: LoadContextFilesOptions = {
return userLevel?.content ?? null;
}

export const DEFAULT_SYSTEM_PROMPT_TOOL_NAMES = ["read", "bash", "eval", "edit", "write"] as const;
export const DEFAULT_SYSTEM_PROMPT_TOOL_NAMES = ["read", "bash", "edit", "write"] as const;

export interface SystemPromptToolMetadata {
label: string;
Expand Down
3 changes: 3 additions & 0 deletions packages/coding-agent/src/tools/bash.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import { type BashInteractiveResult, runInteractiveBashPty } from "./bash-intera
import { checkBashInterception } from "./bash-interceptor";
import { canUseInteractiveBashPty } from "./bash-pty-selection";
import { expandInternalUrls, type InternalUrlExpansionOptions } from "./bash-skill-urls";
import { resolveEvalBackends } from "./eval-backends";
import { invalidateGithubCacheForBashCommand } from "./gh-cache-invalidation";
import {
formatStyledTruncationWarning,
Expand Down Expand Up @@ -397,6 +398,7 @@ export class BashTool implements AgentTool<typeof bashSchemaBase | typeof bashSc
),
);
this.parameters = this.#asyncEnabled ? bashSchemaWithAsync : bashSchemaBase;
const evalBackends = resolveEvalBackends(this.session);
this.description = prompt.render(bashDescription, {
asyncEnabled: this.#asyncEnabled,
autoBackgroundEnabled: this.#autoBackgroundEnabled,
Expand All @@ -405,6 +407,7 @@ export class BashTool implements AgentTool<typeof bashSchemaBase | typeof bashSc
hasAstEdit: this.session.settings.get("astEdit.enabled"),
hasGrep: this.session.settings.get("grep.enabled"),
hasGlob: this.session.settings.get("glob.enabled"),
hasEval: evalBackends.python || evalBackends.js || evalBackends.ruby || evalBackends.julia,
});
}

Expand Down
50 changes: 49 additions & 1 deletion packages/coding-agent/test/system-prompt-inventory.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@ import { afterEach, beforeEach, describe, expect, it } from "bun:test";
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { Settings } from "@oh-my-pi/pi-coding-agent/config/settings";
import { buildSystemPrompt as buildSdkSystemPrompt } from "@oh-my-pi/pi-coding-agent/sdk";
import {
buildSystemPrompt,
buildSystemPromptToolMetadata,
DEFAULT_SYSTEM_PROMPT_TOOL_NAMES,
type SystemPromptToolMetadata,
} from "@oh-my-pi/pi-coding-agent/system-prompt";
import type { Tool } from "@oh-my-pi/pi-coding-agent/tools";
import { createTools, type Tool, type ToolSession } from "@oh-my-pi/pi-coding-agent/tools";
import { cleanupTempHome } from "./helpers/temp-home-cleanup";

const EMPTY_TREE = {
Expand Down Expand Up @@ -92,6 +94,16 @@ describe("system prompt tool inventory", () => {
return text.slice(inventoryStart, inventoryEnd);
}

function makeToolSession(settings: Settings): ToolSession {
return {
cwd: tempDir,
hasUI: false,
getSessionFile: () => null,
getSessionSpawns: () => "*",
settings,
} as ToolSession;
}

it("renders a compact name list only when native tools are active and descriptors stay in schemas", async () => {
const text = await render({ nativeTools: true, inlineToolDescriptors: false });
expect(text).toContain("- Read: `read`");
Expand Down Expand Up @@ -132,6 +144,42 @@ describe("system prompt tool inventory", () => {
}
expect(inventory).not.toContain("- `browser`");
expect(inventory).not.toContain("- `task`");
expect(inventory).not.toContain("- `eval`");
});

it("omits eval prompt guidance when every eval backend is disabled", async () => {
const settings = Settings.isolated({
"eval.py": false,
"eval.js": false,
"eval.rb": false,
"eval.jl": false,
});
const session = makeToolSession(settings);
const tools = await createTools(session, ["bash", "eval"]);
const toolNames = tools.map(tool => tool.name);
const bash = tools.find(tool => tool.name === "bash");

expect(toolNames).toContain("bash");
expect(toolNames).not.toContain("eval");
expect(bash?.description).toContain("purpose-built tool");
expect(bash?.description).not.toContain("eval` cell");
expect(bash?.description).not.toContain("use `eval` cells");

const { systemPrompt } = await buildSystemPrompt({
cwd: tempDir,
contextFiles: [],
skills: [],
rules: [],
toolNames,
tools: buildSystemPromptToolMetadata(new Map(tools.map(tool => [tool.name, tool]))),
workspaceTree: { ...EMPTY_TREE, rootPath: tempDir },
nativeTools: true,
inlineToolDescriptors: true,
});
const text = systemPrompt.join("\n\n");

expect(text).not.toContain("Default for any compute");
expect(text).not.toContain("use `eval` cells");
});

it("SDK wrapper renders provided tools instead of the fallback inventory", async () => {
Expand Down