Skip to content

Commit 5bef93d

Browse files
MaximeRivestMaximeRivest
committed
Add cost tracking: estimate_cost, Result.cost, Model.total_cost
- Add input_audio_tokens and output_audio_tokens to Usage - Parse full token details in OpenAI (cache, reasoning, audio) - Parse cachedContentTokenCount and thoughtsTokenCount in Gemini - Add cost.py with estimate_cost() handling per-provider token semantics - Add configure(track_costs=True) for automatic pricing from models.dev - Add Result.cost property and Model.total_cost property - Fix fetch_models_dev: add User-Agent header, handle new API structure - Update serde for new Usage fields - Add cookbook 14-cost-tracking.md
1 parent cd9a190 commit 5bef93d

13 files changed

Lines changed: 886 additions & 15 deletions

File tree

‎docs/COOKBOOKS_V2/14-cost-tracking.md‎

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
# Cookbook 14 — Cost Tracking
2+
3+
Track per-call and cumulative costs across providers with automatic pricing from [models.dev](https://models.dev).
4+
5+
## Enable cost tracking
6+
7+
```python
8+
import lm15
9+
10+
lm15.configure(track_costs=True) # fetches pricing once (~300ms)
11+
```
12+
13+
This fetches the models.dev pricing catalog and enables `.cost` on every result.
14+
15+
## Per-call cost
16+
17+
```python
18+
import lm15
19+
20+
lm15.configure(track_costs=True)
21+
22+
result = lm15.call("gpt-4o", "Explain TCP in one paragraph")
23+
print(result.cost)
24+
# $0.000255 (input=$0.000045, output=$0.000210)
25+
26+
print(result.cost.total) # 0.000255
27+
print(result.cost.input) # 0.000045
28+
print(result.cost.output) # 0.000210
29+
print(result.cost.cache_read) # 0.0
30+
```
31+
32+
`.cost` returns a `CostBreakdown` with itemized fields, or `None` if tracking is not enabled.
33+
34+
## Cumulative cost on a model
35+
36+
```python
37+
import lm15
38+
39+
lm15.configure(track_costs=True)
40+
41+
m = lm15.model("claude-sonnet-4")
42+
m("What is TCP?")
43+
m("What about UDP?")
44+
m("Compare them")
45+
46+
print(m.total_cost)
47+
# $0.001830 (input=$0.000330, output=$0.001500)
48+
```
49+
50+
`model.total_cost` sums all calls in `model.history`. It resets when you call `model.history.clear()`.
51+
52+
## Cost with prompt caching
53+
54+
```python
55+
import lm15
56+
57+
lm15.configure(track_costs=True)
58+
59+
m = lm15.model("claude-sonnet-4", system="<long system prompt>", prompt_caching=True)
60+
r1 = m("First question") # cache write
61+
r2 = m("Second question") # cache hit
62+
63+
print(r1.cost) # higher — includes cache_write cost
64+
print(r2.cost) # lower — cache_read is cheaper than input
65+
print(m.total_cost)
66+
```
67+
68+
## Cost with reasoning models
69+
70+
```python
71+
import lm15
72+
73+
lm15.configure(track_costs=True)
74+
75+
result = lm15.call("o4-mini", "Solve this step by step: ...", reasoning=True)
76+
print(result.cost)
77+
# reasoning tokens are priced separately from output tokens
78+
print(result.cost.reasoning) # cost of thinking tokens
79+
print(result.cost.output) # cost of visible output only
80+
```
81+
82+
## Cost with audio
83+
84+
```python
85+
import lm15
86+
87+
lm15.configure(track_costs=True)
88+
89+
result = lm15.call("gpt-4o-audio-preview", [
90+
"Transcribe this:",
91+
lm15.Part.audio(data=open("meeting.wav", "rb").read(), media_type="audio/wav"),
92+
])
93+
print(result.cost)
94+
print(result.cost.input_audio) # audio input tokens priced separately
95+
print(result.cost.output_audio) # audio output tokens priced separately
96+
```
97+
98+
## Agent loop with cost budget
99+
100+
```python
101+
import lm15
102+
103+
lm15.configure(track_costs=True)
104+
105+
def search(query: str) -> str:
106+
return f"Results for: {query}"
107+
108+
agent = lm15.model("gpt-4o", tools=[search])
109+
MAX_BUDGET = 0.05 # $0.05
110+
111+
resp = agent("Research quantum computing advances in 2025")
112+
while resp.finish_reason == "tool_call":
113+
if agent.total_cost and agent.total_cost.total > MAX_BUDGET:
114+
print(f"Budget exceeded: {agent.total_cost}")
115+
break
116+
resp = agent("continue")
117+
118+
print(f"Final cost: {agent.total_cost}")
119+
```
120+
121+
## Manual cost estimation
122+
123+
For advanced use without `configure(track_costs=True)`:
124+
125+
```python
126+
from lm15 import estimate_cost, fetch_models_dev
127+
128+
# Build index once
129+
specs = fetch_models_dev()
130+
cost_index = {s.id: s for s in specs}
131+
132+
# Estimate from any Usage + ModelSpec
133+
result = lm15.call("gpt-4o", "hello")
134+
cost = estimate_cost(result.usage, cost_index[result.model])
135+
print(cost.total)
136+
```
137+
138+
You can also pass a raw cost dict:
139+
140+
```python
141+
from lm15 import estimate_cost
142+
143+
cost = estimate_cost(
144+
result.usage,
145+
{"input": 2.5, "output": 10.0, "cache_read": 1.25},
146+
provider="openai",
147+
)
148+
```
149+
150+
## CostBreakdown fields
151+
152+
| Field | Description |
153+
|---|---|
154+
| `total` | Sum of all cost components |
155+
| `input` | Non-cached, non-audio input tokens |
156+
| `output` | Non-reasoning, non-audio output tokens |
157+
| `cache_read` | Cached input tokens (cheaper rate) |
158+
| `cache_write` | Cache creation tokens (Anthropic) |
159+
| `reasoning` | Thinking/reasoning tokens |
160+
| `input_audio` | Audio input tokens (OpenAI) |
161+
| `output_audio` | Audio output tokens (OpenAI) |
162+
163+
All values are in US dollars.
164+
165+
## Provider token semantics
166+
167+
The cost calculation handles provider differences automatically:
168+
169+
| | OpenAI | Anthropic | Gemini |
170+
|---|---|---|---|
171+
| `input_tokens` | total (includes cached) | non-cached only | total (includes cached) |
172+
| `output_tokens` | total (includes reasoning) | total | total |
173+
| `cache_read` | subset of input | additive | subset of input |
174+
| `reasoning` | subset of output | n/a | separate count |
175+
176+
You don't need to worry about these — `estimate_cost` and `.cost` handle them correctly for each provider.

‎lm15/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77

88
from .api import acall, alive, call, configure, live, model, models, prepare, providers, providers_info, send, stream, upload
99
from .capabilities import hydrate_with_specs
10+
from .cost import CostBreakdown, estimate_cost, enable_cost_tracking, disable_cost_tracking
1011
from .client import UniversalLM
1112
from .conversation import Conversation
1213
from .live import AsyncLiveSession
@@ -117,6 +118,10 @@ def build_default(
117118
"with_history",
118119
"with_retries",
119120
"hydrate_with_specs",
121+
"estimate_cost",
122+
"CostBreakdown",
123+
"enable_cost_tracking",
124+
"disable_cost_tracking",
120125
"load_plugins",
121126
"discover_provider_entry_points",
122127
"fetch_models_dev",

‎lm15/api.py‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,15 +22,30 @@ def configure(
2222
*,
2323
env: str | None = None,
2424
api_key: str | dict[str, str] | None = None,
25+
track_costs: bool = False,
2526
) -> None:
26-
"""Set module-level defaults so you don't repeat them on every call."""
27+
"""Set module-level defaults so you don't repeat them on every call.
28+
29+
Parameters
30+
----------
31+
track_costs:
32+
If ``True``, fetch pricing data from models.dev so that
33+
``result.cost`` and ``model.total_cost`` work automatically.
34+
"""
2735
_defaults.clear()
2836
_client_cache.clear()
2937
if env is not None:
3038
_defaults["env"] = env
3139
if api_key is not None:
3240
_defaults["api_key"] = api_key
3341

42+
from .cost import disable_cost_tracking, enable_cost_tracking
43+
44+
if track_costs:
45+
enable_cost_tracking()
46+
else:
47+
disable_cost_tracking()
48+
3449

3550
def _resolve(key: str, explicit: Any) -> Any:
3651
if explicit is not None:

0 commit comments

Comments
 (0)