Skip to content

fix(openai): preserve DeepSeek cached token usage - #934

Open
m11y wants to merge 2 commits into
cloudwego:mainfrom
m11y:fix/openai-deepseek-cache-usage
Open

fix(openai): preserve DeepSeek cached token usage#934
m11y wants to merge 2 commits into
cloudwego:mainfrom
m11y:fix/openai-deepseek-cache-usage

Conversation

@m11y

@m11y m11y commented Jul 25, 2026

Copy link
Copy Markdown

Summary

  • map the DeepSeek OpenAI-compatible prompt_cache_hit_tokens usage extension into Eino PromptTokenDetails.CachedTokens
  • preserve a non-zero standard OpenAI prompt_tokens_details.cached_tokens value, while falling back to the DeepSeek extension when a gateway supplies an empty standard details object
  • fall back to zero for missing or malformed extensions

Problem

DeepSeek returns cache hits as a top-level usage field. The underlying go-openai client preserves unknown usage fields in Usage.ExtraFields, but toEinoTokenUsage only reads the standard nested OpenAI field. As a result, both Generate and Stream responses lose the cache-hit count while being converted to Eino metadata. Some compatible gateways may also synthesize prompt_tokens_details with a zero cached value, so zero must not mask a non-zero DeepSeek extension.

Tests

  • GOTOOLCHAIN=go1.24.6 go test -run TestToEinoTokenUsageCachedTokens .
  • GOTOOLCHAIN=go1.24.6 go test -gcflags="all=-N -l" ./...
  • GOTOOLCHAIN=go1.24.6 go vet ./...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant