fix(sdk): evict oversized text from multi-part tool results - #8
Merged
Conversation
The plugin bailed out on any result that was not a plain string:
if (typeof content !== 'string') return null
So a tool returning `[{type:'text'}, {type:'image_url'}]` — a page snapshot with
its JSON plus a screenshot — was never evictable at any threshold. Not a tuning
problem: no value of maxTokens could reach it.
Measured on a production session, one such snapshot adds a median 6.8k prompt
tokens and stays resident for the rest of the agent's life; four copies of the
same page occupied 27% of the largest request.
Now the text is measured and evicted while image parts pass through untouched.
Images are kept rather than dropped because there is no partial image — the only
alternative to keeping it is losing the observation. Text both dominates the
payload and survives being collapsed to a preview plus a spill-file path.
Behaviour for string results is unchanged.
Worth knowing when tuning maxTokens: this hook runs on afterToolCall and its
`modify` is applied before tool_completed is emitted, so it shapes what enters
history and never rewrites cached messages. Lowering the threshold therefore
costs nothing in prompt-cache terms, unlike retroactive eviction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yiScAyGwDx3A3DqaUasEA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The blind spot
A tool returning
[{type:'text'}, {type:'image_url'}]— a page snapshot with its JSON plus a screenshot — could never be evicted at any threshold. This is structural, not a tuning issue: no value ofmaxTokensreaches it.Why it matters
Measured on a production session: one
get_page_snapshotresult adds a median 6 802 prompt tokens (n=27 isolated measurements) and stays resident for the rest of that agent's life. In the largest request, four copies of the same page accounted for 27 % of a 150 818-token prompt — none of them reachable by eviction.The change
Measure the text, evict the text, pass image parts through untouched.
Images are kept rather than dropped because there is no partial image: the only alternative to keeping it is losing the observation entirely. Text both dominates the payload and survives being collapsed to a preview plus a spill-file path.
String results behave exactly as before.
Note for whoever tunes
maxTokensThis hook runs on
afterToolCall, and itsmodifyis applied beforetool_completedis emitted (agent.ts:982-986) — so it shapes what enters history and never rewrites cached messages. Lowering the threshold costs nothing in prompt-cache terms.That is worth stating because the opposite is true of retroactive eviction: rewriting history to drop X tokens from a P-token prefix costs a full-prefix rewrite and only pays back after
N > 11.5·P/Xfurther requests — at P=59 k, X=10.5 k that is 65 requests. Easy to conflate the two and reach the wrong conclusion about this plugin.Verification
bun run ts:buildclean,bun run lintclean (1 pre-existing unrelated warning)bun test— 1768 pass / 24 skip / 0 fail🤖 Generated with Claude Code
https://claude.ai/code/session_018yiScAyGwDx3A3DqaUasEA