Skip to content

fix(sdk): evict oversized text from multi-part tool results - #8

Merged
matej21 merged 1 commit into
mainfrom
fix/evict-multipart-results
Jul 30, 2026
Merged

fix(sdk): evict oversized text from multi-part tool results#8
matej21 merged 1 commit into
mainfrom
fix/evict-multipart-results

Conversation

@matej21

@matej21 matej21 commented Jul 30, 2026

Copy link
Copy Markdown
Member

The blind spot

const { content } = ctx.result
if (typeof content !== 'string') return null    // ← any multi-part result

A tool returning [{type:'text'}, {type:'image_url'}] — a page snapshot with its JSON plus a screenshot — could never be evicted at any threshold. This is structural, not a tuning issue: no value of maxTokens reaches it.

Why it matters

Measured on a production session: one get_page_snapshot result adds a median 6 802 prompt tokens (n=27 isolated measurements) and stays resident for the rest of that agent's life. In the largest request, four copies of the same page accounted for 27 % of a 150 818-token prompt — none of them reachable by eviction.

The change

Measure the text, evict the text, pass image parts through untouched.

Images are kept rather than dropped because there is no partial image: the only alternative to keeping it is losing the observation entirely. Text both dominates the payload and survives being collapsed to a preview plus a spill-file path.

String results behave exactly as before.

Note for whoever tunes maxTokens

This hook runs on afterToolCall, and its modify is applied before tool_completed is emitted (agent.ts:982-986) — so it shapes what enters history and never rewrites cached messages. Lowering the threshold costs nothing in prompt-cache terms.

That is worth stating because the opposite is true of retroactive eviction: rewriting history to drop X tokens from a P-token prefix costs a full-prefix rewrite and only pays back after N > 11.5·P/X further requests — at P=59 k, X=10.5 k that is 65 requests. Easy to conflate the two and reach the wrong conclusion about this plugin.

Verification

  • bun run ts:build clean, bun run lint clean (1 pre-existing unrelated warning)
  • bun test — 1768 pass / 24 skip / 0 fail
  • Two new integration tests: large multi-part (text collapsed, image survives) and small multi-part (untouched)

🤖 Generated with Claude Code

https://claude.ai/code/session_018yiScAyGwDx3A3DqaUasEA

The plugin bailed out on any result that was not a plain string:

    if (typeof content !== 'string') return null

So a tool returning `[{type:'text'}, {type:'image_url'}]` — a page snapshot with
its JSON plus a screenshot — was never evictable at any threshold. Not a tuning
problem: no value of maxTokens could reach it.

Measured on a production session, one such snapshot adds a median 6.8k prompt
tokens and stays resident for the rest of the agent's life; four copies of the
same page occupied 27% of the largest request.

Now the text is measured and evicted while image parts pass through untouched.
Images are kept rather than dropped because there is no partial image — the only
alternative to keeping it is losing the observation. Text both dominates the
payload and survives being collapsed to a preview plus a spill-file path.

Behaviour for string results is unchanged.

Worth knowing when tuning maxTokens: this hook runs on afterToolCall and its
`modify` is applied before tool_completed is emitted, so it shapes what enters
history and never rewrites cached messages. Lowering the threshold therefore
costs nothing in prompt-cache terms, unlike retroactive eviction.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yiScAyGwDx3A3DqaUasEA
@matej21
matej21 merged commit 7a42f87 into main Jul 30, 2026
1 check passed
@matej21
matej21 deleted the fix/evict-multipart-results branch July 30, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant