Skip to content

fix(spend_tracking): store messages for all call types when enabled#34748

Open
baiyuxi930826 wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
baiyuxi930826:fix/store-prompts-spend-logs
Open

fix(spend_tracking): store messages for all call types when enabled#34748
baiyuxi930826 wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
baiyuxi930826:fix/store-prompts-spend-logs

Conversation

@baiyuxi930826

Copy link
Copy Markdown

Summary

Fixes #34747

With general_settings.store_prompts_in_spend_logs: true, spend log rows for acompletion / aresponses still persisted LiteLLM_SpendLogs.messages as {}.

Root cause: _get_messages_for_spend_logs_payload only returned messages when call_type == "_arealtime".

Change

When the store gate is on, dump standard_logging_payload["messages"] for all call types (still {} when disabled, missing, or dump fails). Response storage was already gated correctly.

Test plan

  • Logic coverage: acompletion / aresponses / realtime messages stored when gate on
  • Gate off still returns {}
  • Null-byte stripping still works via safe_dumps
  • Unit tests updated in tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py

_get_messages_for_spend_logs_payload only filled LiteLLM_SpendLogs.messages
for _arealtime. With store_prompts_in_spend_logs: true, acompletion and
aresponses rows still persisted messages as {}.

Persist standard_logging_payload.messages for every call type when the
gate is on (still {} when disabled or messages missing).

Fixes BerriAI#34747
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR broadens gated spend-log message persistence beyond realtime calls.

  • Serializes messages for every call type when prompt storage is enabled.
  • Preserves empty-object fallbacks when storage is disabled, messages are absent, or serialization fails.
  • Updates completion coverage and adds Responses API coverage.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code defect identified.

The implementation consistently applies the existing prompt-storage gate before serializing messages, retains safe empty fallbacks, and adds tests for the newly supported completion and Responses API paths.

Important Files Changed

Filename Overview
litellm/proxy/spend_tracking/spend_tracking_utils.py Removes the realtime-only restriction while preserving the existing storage gate and safe serialization fallbacks.
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py Updates completion expectations and adds Responses API coverage for the broadened message-storage behavior.

Reviews (1): Last reviewed commit: "fix(spend_tracking): store messages for ..." | Re-trigger Greptile

@codspeed-hq

codspeed-hq Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing baiyuxi930826:fix/store-prompts-spend-logs (1d28b0a) with litellm_internal_staging (2412326)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants