Support Pi in LLM gateway setup

[jerm-dro]

Summary

• Pi users need a supported way to route requests through the LLM gateway without editing provider files manually; register Pi with thv llm setup and provide safe setup and teardown of its custom providers.
• Models can use different wire protocols and report sizes in different fields; discover both OpenAI-compatible and Anthropic model lists, including Anthropic pagination, and configure each Pi provider with the sizes actually reported.

Type of change

• Bug fix
• New feature
• Refactoring (no behavior change)
• Dependency update
• Documentation
• Other (describe):

Test plan

• Unit tests (task test)
• E2E tests (task test-e2e)
• Linting (task lint-fix)
• Manual testing: Built with task build, configured Pi, and sent a minimal prompt through all 28 discovered models across both provider APIs. 27 returned a response; one returned an upstream 403 due to missing model invocation permissions. Compared every reported model size with the generated Pi configuration; all available metadata matched. One model supplies no size and therefore uses the Pi default. Also verified Pi low-thinking requests for adaptive and budget-based Anthropic models against staging.

Changes

File	Change
pkg/client/config.go, pkg/client/llm_gateway.go, pkg/client/llm_gateway_pi.go	Register Pi and manage both custom providers without overwriting unrelated settings.
pkg/llm/setup.go, pkg/llmgateway/config.go	Discover model IDs, context sizes, and Anthropic thinking capabilities; pass them to Pi setup.
pkg/client/llm_gateway_pi_test.go, pkg/llm/setup_test.go	Cover setup, teardown, model discovery, pagination, and protocol-specific sizes.
cmd/thv/app/llm.go, docs/cli/thv_llm_setup.md	Describe Pi setup and regenerate CLI documentation.

Does this introduce a user-facing change?

Yes. thv llm setup --client pi configures OpenAI-compatible and Anthropic models in ~/.pi/agent/models.json using the ToolHive token helper.

Special notes for reviewers

Discovery does report per-model context sizes (context_window on /v1/models, max_input_tokens on /anthropic/v1/models); ToolHive writes them as Pi contextWindow (e.g. gpt-5 is 400,000 and gpt-5.6-sol is 1,050,000). Separately, Pi defaults omitted input to text-only and omitted maxTokens to 16,384 output tokens. Neither discovery endpoint reports image-input or maximum-output metadata, so setup cannot override those Pi defaults from discovery; the 16,384 value is not the context size or an advertised model output limit. The OpenAI-compatible discovery response does not advertise reasoning support, so Pi also defaults those models to reasoning: false; this means Pi does not offer thinking-level controls for them. Anthropic discovery does advertise thinking modes for 11 of 12 models; those are configured with reasoning: true and adaptive compatibility where required. These limitations are also documented in thv llm setup --help.

The advertised us.anthropic.claude-fable-5-1 model currently returns 403 because its upstream role cannot invoke streaming inference; this is an external permission issue, not an integration failure. Models without reported context metadata keep the Pi default.

[codecov]

Codecov Report

❌ Patch coverage is 87.63251% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.66%. Comparing base (0dcd0ad) to head (e813675).

Files with missing lines	Patch %	Lines
pkg/client/llm_gateway_pi.go	84.93%	22 Missing ⚠️
pkg/llm/setup.go	90.24%	12 Missing ⚠️
pkg/client/llm_gateway.go	92.85%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6818      +/-   ##
==========================================
+ Coverage   79.64%   79.66%   +0.01%     
==========================================
  Files         816      817       +1     
  Lines       83168    83411     +243     
==========================================
+ Hits        66242    66446     +204     
- Misses      16921    16960      +39     
  Partials        5        5              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟡 Changes recommended

Lazy Anthropic models are misclassified, and valid Pi configurations or custom agent directories are not handled.

3 open findings

• Lazy setup assigns all models to the wrong protocol · New
• Hard-coded agent directory ignores PI_CODING_AGENT_DIR · New
• Strict JSON parsing rejects valid commented or BOM-prefixed files · New

What changed in this PR

Adds Pi support to LLM gateway setup, including dual-protocol model discovery and managed provider configuration.

Changes:

• Registers Pi and safely manages ToolHive providers.
• Discovers OpenAI and paginated Anthropic model metadata.
• Adds unit tests and CLI documentation.

File	Description
pkg/​llmgateway/​config.go	Adds Pi mode and model metadata fields.
pkg/​llm/​setup.go	Implements dual-protocol model discovery.
pkg/​llm/​setup_test.go	Tests discovery, pagination, and setup.
pkg/​client/​llm_gateway.go	Routes Pi configuration and teardown.
pkg/​client/​llm_gateway_pi.go	Manages Pi provider configuration.
pkg/​client/​llm_gateway_pi_test.go	Tests Pi setup and teardown.
pkg/​client/​config.go	Registers the Pi client integration.
cmd/​thv/​app/​llm.go	Documents Pi CLI behavior.
docs/​cli/​thv_llm_setup.md	Regenerates setup documentation.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[aponcedeleonch]

Nice work on this. I checked the setup against Pi's docs and source (pi.dev and earendil-works/pi). The !thv llm token apiKey, authHeader, the PI_CODING_AGENT_DIR handling and the JSONC-preserving edits all line up.

Before merge, I'd want the one-hour prompt cache and the hardcoded /anthropic prefix addressed. The rest are smaller and left inline.

[jerm-dro]

@aponcedeleonch Thanks for the detailed review. Both pre-merge concerns are fixed in df2918e: Pi setup resolves the Anthropic path prefix before discovery and uses the same base URL for Pi requests (explicit root/custom prefixes and auto-probe); setup gives a PI_CACHE_RETENTION=long hint unless short caching was selected, with CLI help explaining that Pi cannot persist this setting. Tests cover both paths, task test/lint-fix/build pass, and the dev gateway returns 401 to the prefix probe. I also addressed the smaller inline comments or documented the model-metadata limitation there.

[aponcedeleonch]

Thanks for working through all of the feedback. Everything I raised is addressed, and the reasoning discovery on top is a nice addition. LGTM.