-
Notifications
You must be signed in to change notification settings - Fork 1.5k
fix(prompting): hide eval guidance when disabled #4644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
b765657
92765fc
25c94fa
96850a1
1936d4f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,14 +6,22 @@ The shell invokes **real binaries** with simple args. It is NOT full GNU Bash. | |
|
|
||
| Use bash ONLY for: a single binary call, or one short pipeline that COMPUTES a fact and does not depend on shell-specific regex/quoting (`wc -l`, `sort | uniq -c`, `comm`, `diff`, a checksum, `git status`). | ||
|
|
||
| Anything below → `eval` cell, not bash: | ||
| {{#if hasEval}}Anything below → `eval` cell, not bash: | ||
| - Inline interpreter scripts (`-e`/`-c`/`--eval`) when an eval runtime exists for that language | ||
| - Heredocs (`<<EOF`), `while`/`for`/`if`/`case` shell control flow | ||
| - `$(…)` command substitution nested inside another command | ||
| - Pipelines with more than two stages, or stages that need control flow or quote/JSON escaping | ||
| - Multiline commands, `&&`-chains mixing control flow | ||
| - Quote/JSON escaping that fights the shell | ||
| - GNU grep BRE extensions are not guaranteed in the embedded shell: use `grep -E 'json|tool'` for alternation instead of `grep 'json\|tool'`; use the built-in `grep` tool with `pattern: "json|tool"` (Rust regex, so `\bword\b` works there), or `eval` for exact text processing. | ||
| {{else}}Anything below means you are writing a shell program, not invoking one. Prefer a purpose-built tool, a checked-in script, or a single repo command instead: | ||
| - Inline interpreter scripts (`-e`/`-c`/`--eval`) | ||
| - Heredocs (`<<EOF`), `while`/`for`/`if`/`case` shell control flow | ||
| - `$(…)` command substitution nested inside another command | ||
| - Pipelines with more than two stages, or stages that need control flow or quote/JSON escaping | ||
| - Multiline commands, `&&`-chains mixing control flow | ||
| - Quote/JSON escaping that fights the shell | ||
| {{/if}} | ||
| - GNU grep BRE extensions are not guaranteed in the embedded shell: use `grep -E 'json|tool'` for alternation instead of `grep 'json\|tool'`; use the built-in `grep` tool with `pattern: "json|tool"` (Rust regex, so `\bword\b` works there){{#if hasEval}}, or `eval` for exact text processing{{/if}}. | ||
|
|
||
| <instruction> | ||
| - `cwd` sets the working dir, not `cd dir && …` | ||
|
|
@@ -30,7 +38,10 @@ Anything below → `eval` cell, not bash: | |
| </instruction> | ||
|
|
||
| <critical> | ||
| - The embedded shell invokes real binaries with simple args; it is NOT full GNU Bash. Loops, conditionals, heredocs, inline interpreter scripts (`-e`/`-c`/`--eval`) when an eval runtime exists, several piped stages, exact pipeline semantics, or quote/JSON escaping mean you're writing a program → use `eval` cells: restartable, stateful, and free of shell-quoting traps. | ||
| {{#if hasEval}}- The embedded shell invokes real binaries with simple args; it is NOT full GNU Bash and NOT a scripting surface. Loops, conditionals, heredocs, inline interpreter scripts (`-e`/`-c`/`--eval`) when an eval runtime exists, several piped stages, exact pipeline semantics, or quote/JSON escaping mean you're writing a program → use `eval` cells: restartable, stateful, and free of shell-quoting traps.{{else}}- The embedded shell invokes real binaries with simple args; it is NOT full GNU Bash and NOT a scripting surface. Loops, conditionals, heredocs, inline interpreter scripts, several piped stages, exact pipeline semantics, or quote/JSON escaping mean you're writing a shell program; use a purpose-built tool or checked-in script instead.{{/if}} | ||
| - NEVER shell out to search content or files: `grep/rg` → `grep`. | ||
| - NEVER use `ls` or `find` to list or locate files — `ls` → `read` (a directory path lists entries), `find` → the `glob` tool (globbing). This is non-negotiable, even for a single quick listing. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In sessions where Useful? React with 👍 / 👎.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 92765fc. Bash now renders file/search guidance from the active tool set, not only settings, and the restricted bash/eval-disabled prompt test asserts it no longer names inactive grep/read/glob tools. Verified with |
||
| - Avoid head/tail/redirections: stderr already merged; long output auto-truncated, FULL capture kept at `artifact://<id>`. | ||
| </critical> | ||
|
|
||
| <output> | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After this line switches the workflowz steer to direct
tasktool fan-out, the same notice still documents eval-only helper APIs and examples below (agent(..., schema=...),parallel,pipeline, and “Every eval call”), which are not part of thetasktool and include arguments thattaskrejects. In workflowz sessions—especially the eval-disabled case this change is trying to support—the hidden prompt now gives mutually incompatible instructions and can drive invalid calls; either keep this notice eval-gated or rewrite the helper/examples to the task schema.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 92765fc. The workflowz notice now documents the
taskbatch contract end to end and removes the eval-only helper API/examples, so eval-disabled workflowz sessions get one orchestration surface. Verified withbun test packages/coding-agent/test/system-prompt-inventory.test.ts packages/coding-agent/test/tools/index.test.ts packages/coding-agent/test/modes/workflow.test.ts.