Skip to content

Commit c5bc34b

Browse files
MaximeRivestMaximeRivest
committed
polish error handling
1 parent 1ea7a9b commit c5bc34b

13 files changed

Lines changed: 750 additions & 23 deletions

‎docs/ADD_PROVIDER_GUIDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ Map provider stream frames to LM15 `StreamEvent`:
175175
### Rules
176176

177177
- Return `None` for ignorable events.
178-
- Emit `error` with normalized `{code, message}`.
178+
- Emit `error` with normalized `{code, provider_code, message}`.
179179
- Never leak provider event shapes directly.
180180

181181
---

‎docs/ERRORS.md‎

Lines changed: 190 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,195 @@
1-
# Errors
1+
# Error Handling
22

3-
## Normalized Error Classes
3+
## Error hierarchy
44

5-
- `AuthError`
6-
- `RateLimitError`
7-
- `TimeoutError`
8-
- `InvalidRequestError`
9-
- `ServerError`
10-
- `UnsupportedFeatureError`
11-
- `UnsupportedModelError`
12-
- `NotConfiguredError`
13-
- `ProviderError` (fallback)
5+
```
6+
ULMError
7+
├── TransportError # Network-level: DNS, connection refused, socket timeout
8+
└── ProviderError # API-level: the provider returned an error
9+
├── AuthError # 401, 403 — bad key, expired key, wrong permissions
10+
├── BillingError # 402 — payment or billing issue
11+
├── RateLimitError # 429 — too many requests or quota exceeded
12+
├── InvalidRequestError # 400, 404, 409, 413, 422 — bad model, bad params, not found
13+
│ └── ContextLengthError # input exceeds the model's context window
14+
├── TimeoutError # 408, 504 — request timed out
15+
├── ServerError # 5xx — provider is down or overloaded
16+
├── UnsupportedModelError
17+
├── UnsupportedFeatureError
18+
└── NotConfiguredError
19+
```
1420

15-
## Mapping
21+
`ContextLengthError` extends `InvalidRequestError` — existing `except InvalidRequestError` catches still work.
1622

17-
`map_http_error(status, body)` maps HTTP status to normalized class.
23+
## How errors flow
1824

19-
Adapters may override `normalize_error()` for provider-specific body parsing while preserving normalized class hierarchy.
25+
```
26+
Provider HTTP response (status >= 400)
27+
→ Adapter.normalize_error(status, body) # each adapter parses its own error format
28+
→ map_http_error(status, message) # maps HTTP status → error class
29+
→ AuthError / RateLimitError / etc.
30+
```
31+
32+
Each adapter owns its error parsing via `normalize_error`. The base `map_http_error` only maps status codes to error classes — adapters extract the human-readable message.
33+
34+
Streaming follows the same taxonomy. Provider stream errors are normalized to canonical stream error codes (`auth`, `billing`, `rate_limit`, `invalid_request`, `context_length`, `timeout`, `server`, `provider`) and `lm15.stream.Stream` raises the corresponding typed `ProviderError` subclass.
35+
36+
---
37+
38+
## Provider error mapping
39+
40+
### OpenAI
41+
42+
Source of truth: [platform.openai.com/docs/guides/error-codes](https://platform.openai.com/docs/guides/error-codes)
43+
44+
Error shape: `{"error": {"message": "...", "type": "...", "param": "...", "code": "..."}}`
45+
46+
| HTTP | `error.type` | `error.code` | lm15 error | Detection |
47+
|------|-------------|--------------|------------|-----------|
48+
| 400 | `invalid_request_error` | `context_length_exceeded` | `ContextLengthError` | ✅ structured `code` field |
49+
| 400 | `invalid_request_error` | `model_not_found` | `InvalidRequestError` | |
50+
| 400 | `invalid_request_error` | `missing_required_parameter` | `InvalidRequestError` | |
51+
| 400 | `invalid_request_error` | `invalid_value` | `InvalidRequestError` | |
52+
| 401 | `invalid_request_error` | `invalid_api_key` | `AuthError` | |
53+
| 403 | `invalid_request_error` | *(project access denied)* | `AuthError` | |
54+
| 429 | `rate_limit_error` | `rate_limit_exceeded` | `RateLimitError` | |
55+
| 429 | `insufficient_quota` | `insufficient_quota` | `BillingError` | ✅ structured `code`/`type` field |
56+
| 500 | `server_error` | *(varies)* | `ServerError` | |
57+
| 503 | `server_error` | `service_unavailable` | `ServerError` | |
58+
59+
The Responses API can also return **in-band errors** on HTTP 200 via `Response.error` (e.g. background task failures):
60+
61+
| `Response.error.code` | lm15 error |
62+
|-----------------------|------------|
63+
| `server_error` | `ServerError` |
64+
| `rate_limit_exceeded` | `RateLimitError` |
65+
| `invalid_prompt` | `InvalidRequestError` |
66+
| *(other)* | `ServerError` (fallback) |
67+
68+
### Anthropic
69+
70+
Source of truth: [docs.anthropic.com/en/api/errors](https://docs.anthropic.com/en/api/errors)
71+
72+
Error shape: `{"type": "error", "error": {"type": "...", "message": "..."}, "request_id": "..."}`
73+
74+
| HTTP | `error.type` | lm15 error | Notes |
75+
|------|-------------|------------|-------|
76+
| 400 | `invalid_request_error` | `InvalidRequestError` | |
77+
| 400 | `invalid_request_error` | `ContextLengthError` | ⚠️ message match: `"prompt is too long"`. No structured code exists |
78+
| 401 | `authentication_error` | `AuthError` | |
79+
| 402 | `billing_error` | `BillingError` | |
80+
| 403 | `permission_error` | `AuthError` | |
81+
| 404 | `not_found_error` | `InvalidRequestError` | |
82+
| 413 | `request_too_large` | `InvalidRequestError` | Payload > 32 MB (from Cloudflare, before API) |
83+
| 429 | `rate_limit_error` | `RateLimitError` | |
84+
| 500 | `api_error` | `ServerError` | |
85+
| 504 | `timeout_error` | `TimeoutError` | |
86+
| 529 | `overloaded_error` | `ServerError` | API-wide overload |
87+
88+
### Google Gemini
89+
90+
Source of truth: [ai.google.dev/gemini-api/docs/troubleshooting](https://ai.google.dev/gemini-api/docs/troubleshooting) and [Google API error model](https://cloud.google.com/apis/design/errors)
91+
92+
Error shape: `{"error": {"message": "...", "code": <int>, "status": "...", "details": [...]}}`
93+
94+
| HTTP | `error.status` | lm15 error | Notes |
95+
|------|---------------|------------|-------|
96+
| 400 | `INVALID_ARGUMENT` | `InvalidRequestError` | |
97+
| 400 | `FAILED_PRECONDITION` | `BillingError` | "Free tier not available in your country. Enable billing." |
98+
| 403 | `PERMISSION_DENIED` | `AuthError` | |
99+
| 404 | `NOT_FOUND` | `InvalidRequestError` | |
100+
| 429 | `RESOURCE_EXHAUSTED` | `RateLimitError` | |
101+
| 429 | `RESOURCE_EXHAUSTED` | `ContextLengthError` | ⚠️ message match: token + limit/exceed keywords. Gemini conflates context overflow with rate limiting under the same 429/RESOURCE_EXHAUSTED |
102+
| 500 | `INTERNAL` | `ServerError` | |
103+
| 500 | `INTERNAL` | `ContextLengthError` | ⚠️ message match: "input context is too long" — Gemini docs list this under 500/INTERNAL |
104+
| 503 | `UNAVAILABLE` | `ServerError` | |
105+
| 504 | `DEADLINE_EXCEEDED` | `TimeoutError` | Prompt/context too large to process in time |
106+
107+
Gemini `generateContent` can also return in-band generation failures on HTTP 200:
108+
109+
| Signal | lm15 error |
110+
|--------|------------|
111+
| `promptFeedback.blockReason` set | `InvalidRequestError` |
112+
| `candidate.finishReason` in safety/recitation/blocklist/prohibited/tool-malformed classes | `InvalidRequestError` |
113+
114+
---
115+
116+
## Context length detection reliability
117+
118+
| Provider | Detection method | Reliability |
119+
|----------|-----------------|-------------|
120+
| OpenAI | `error.code == "context_length_exceeded"` | ✅ Structured, reliable |
121+
| Anthropic | Message contains `"prompt is too long"` | ⚠️ String matching — could break if Anthropic changes wording |
122+
| Gemini | Message contains token + limit/exceed keywords | ⚠️ String matching — Gemini conflates context overflow with `RESOURCE_EXHAUSTED` (429), same as rate limiting |
123+
124+
Only OpenAI provides a machine-readable error code for context overflow. Anthropic and Gemini require message parsing. If detection fails, the error falls through to `InvalidRequestError` (Anthropic) or `RateLimitError` (Gemini), which are still catchable.
125+
126+
---
127+
128+
## Catching errors
129+
130+
```python
131+
import lm15
132+
from lm15.errors import AuthError, BillingError, RateLimitError, InvalidRequestError, ContextLengthError, ServerError
133+
134+
try:
135+
resp = lm15.complete("gpt-4.1-mini", "Hello.", env=".env")
136+
except ContextLengthError as e:
137+
print(f"Input too long: {e}")
138+
except AuthError as e:
139+
print(f"Bad credentials: {e}")
140+
except BillingError as e:
141+
print(f"Payment issue: {e}")
142+
except RateLimitError as e:
143+
print(f"Slow down or add credits: {e}")
144+
except InvalidRequestError as e:
145+
print(f"Fix your request: {e}")
146+
except ServerError as e:
147+
print(f"Provider is down, retry later: {e}")
148+
```
149+
150+
> **Order matters.** Catch `ContextLengthError` before `InvalidRequestError` — it's a subclass.
151+
152+
### Catch all provider errors
153+
154+
```python
155+
from lm15.errors import ProviderError
156+
157+
try:
158+
resp = lm15.complete("gpt-4.1-mini", "Hello.", env=".env")
159+
except ProviderError as e:
160+
print(f"Provider error: {e}")
161+
```
162+
163+
### Catch everything (including network issues)
164+
165+
```python
166+
from lm15.errors import ULMError
167+
168+
try:
169+
resp = lm15.complete("gpt-4.1-mini", "Hello.", env=".env")
170+
except ULMError as e:
171+
print(f"Something went wrong: {e}")
172+
```
173+
174+
---
175+
176+
## For plugin authors
177+
178+
Override `normalize_error` on your adapter to parse your provider's error format:
179+
180+
```python
181+
import json
182+
from lm15.errors import ProviderError, map_http_error
183+
from lm15.providers.base import BaseProviderAdapter
184+
185+
class MyAdapter(BaseProviderAdapter):
186+
def normalize_error(self, status: int, body: str) -> ProviderError:
187+
try:
188+
data = json.loads(body)
189+
msg = data.get("error_description", body[:200])
190+
except Exception:
191+
msg = body[:200] or f"HTTP {status}"
192+
return map_http_error(status, msg)
193+
```
194+
195+
`map_http_error` handles the status → error class mapping. You only need to extract the human-readable message. For provider-specific error types (like context length), detect them in your `normalize_error` and return the appropriate lm15 error class directly.

‎docs/STREAMING.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,11 @@ LM15 normalizes SSE into `StreamEvent`.
2121
- Convert provider-specific stream payloads into normalized deltas.
2222
- Emit `error` on provider stream error events.
2323
- Emit `end` on completion markers.
24+
25+
## Stream Error Shape
26+
27+
`StreamEvent(type="error")` uses normalized error metadata:
28+
29+
- `error.code`: canonical lm15 code (`auth`, `billing`, `rate_limit`, `invalid_request`, `context_length`, `timeout`, `server`, `provider`)
30+
- `error.provider_code`: provider-native code/status/type
31+
- `error.message`: provider message

‎lm15/errors.py‎

Lines changed: 56 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ class RateLimitError(ProviderError):
2121
pass
2222

2323

24+
class BillingError(ProviderError):
25+
"""402 — billing or payment issue."""
26+
27+
2428
class TimeoutError(ProviderError):
2529
pass
2630

@@ -29,6 +33,10 @@ class InvalidRequestError(ProviderError):
2933
pass
3034

3135

36+
class ContextLengthError(InvalidRequestError):
37+
"""The input exceeds the model's context window."""
38+
39+
3240
class ServerError(ProviderError):
3341
pass
3442

@@ -46,14 +54,60 @@ class NotConfiguredError(ProviderError):
4654

4755

4856
def map_http_error(status: int, message: str) -> ProviderError:
57+
"""Map an HTTP status + extracted message to a typed ProviderError.
58+
59+
Adapters are responsible for extracting the human-readable message
60+
from the provider's error body in their ``normalize_error`` override.
61+
This function only maps status codes to error classes.
62+
"""
4963
if status in (401, 403):
5064
return AuthError(message)
51-
if status == 408:
65+
if status == 402:
66+
return BillingError(message)
67+
if status in (408, 504):
5268
return TimeoutError(message)
5369
if status == 429:
5470
return RateLimitError(message)
55-
if status in (400, 404, 409, 422):
71+
if status in (400, 404, 409, 413, 422):
5672
return InvalidRequestError(message)
5773
if 500 <= status <= 599:
5874
return ServerError(message)
5975
return ProviderError(message)
76+
77+
78+
def canonical_error_code(error: type[ProviderError] | ProviderError) -> str:
79+
"""Return canonical lm15 error code for an error class/instance.
80+
81+
Canonical codes are provider-agnostic and stable across adapters.
82+
"""
83+
cls = error if isinstance(error, type) else type(error)
84+
85+
if issubclass(cls, ContextLengthError):
86+
return "context_length"
87+
if issubclass(cls, AuthError):
88+
return "auth"
89+
if issubclass(cls, BillingError):
90+
return "billing"
91+
if issubclass(cls, RateLimitError):
92+
return "rate_limit"
93+
if issubclass(cls, InvalidRequestError):
94+
return "invalid_request"
95+
if issubclass(cls, TimeoutError):
96+
return "timeout"
97+
if issubclass(cls, ServerError):
98+
return "server"
99+
return "provider"
100+
101+
102+
def error_class_for_canonical_code(code: str) -> type[ProviderError]:
103+
"""Return ProviderError subclass for canonical lm15 error code."""
104+
return {
105+
"auth": AuthError,
106+
"billing": BillingError,
107+
"rate_limit": RateLimitError,
108+
"invalid_request": InvalidRequestError,
109+
"context_length": ContextLengthError,
110+
"timeout": TimeoutError,
111+
"server": ServerError,
112+
"provider": ProviderError,
113+
}.get(code, ProviderError)

0 commit comments

Comments
 (0)