Default Claude Code prompt cache to one hour

[Sanskarzz]

Summary

• Claude Code users should benefit from the lower cost of one-hour prompt caching without needing an extra opt-in. Configure the one-hour cache lifetime during normal thv llm setup runs.
• Add a persisted --short-prompt-cache opt-out to thv llm setup and thv llm config set, remove ToolHive-managed one-hour settings when enabled, and restore them with --short-prompt-cache=false.
• Warn when environment or managed Claude Code settings may override ToolHive's cache lifetime, while leaving those external settings unchanged.

Fixes #6453

Type of change

• Bug fix
• New feature
• Refactoring (no behavior change)
• Dependency update
• Documentation
• Other (describe):

Test plan

• Unit tests (task test)
• E2E tests (task test-e2e)
• Linting (task lint)
• Manual testing (describe below)

API Compatibility

• This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

Changes

Area	Change
LLM CLI and config	Add and persist the five-minute cache opt-out, including explicit restoration of the one-hour default.
Claude Code setup	Write or remove the main, subagent, and legacy one-hour cache settings.
Conflict reporting	Warn about environment, managed-settings, and Bedrock compatibility overrides without failing setup.
Tests and CLI docs	Cover configuration and setup behavior and document the new flags.

Does this introduce a user-facing change?

Yes. thv llm setup now configures Claude Code to use a one-hour prompt-cache lifetime by default. Users can persist the five-minute behavior with --short-prompt-cache and restore the one-hour default with --short-prompt-cache=false.

Implementation plan

Approved implementation plan

• Make one-hour prompt caching the default for Claude Code through its main, subagent, and legacy configuration keys.
• Persist a tri-state CLI opt-out so omission preserves the saved preference and explicit false restores one-hour caching.
• Detect higher-precedence environment and managed settings, emit warnings without mutating them, and cover the behavior with tests and generated CLI docs.

Special notes for reviewers

The cache setting applies only to Claude Code. Conflict detection is warning-only: ToolHive does not modify process environment variables or managed Claude Code settings.

[codecov]

Codecov Report

❌ Patch coverage is 85.50725% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.31%. Comparing base (326be93) to head (a38b032).

Files with missing lines	Patch %	Lines
pkg/client/llm_gateway.go	80.58%	20 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6515      +/-   ##
==========================================
- Coverage   79.35%   79.31%   -0.04%     
==========================================
  Files         802      802              
  Lines       81245    81372     +127     
==========================================
+ Hits        64470    64539      +69     
- Misses      16770    16828      +58     
  Partials        5        5              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[aponcedeleonch]

Thanks for this, it's a really thorough take on the issue, and the Claude Code wiring (both per-bucket settings plus the ENABLE_PROMPT_CACHING_1H fallback) matches the docs. We talked about it internally and we'd like to change direction a bit. The one-hour cache should be the default, and the flag should opt into the shorter one. On top of that, the global user setting should always win, with a warning when something else might override it. I know the issue says opt-in, that's on us, we'll update it. Comments below are roughly in order of importance. Also, the branch is quite a bit behind main, so it'll need a rebase. Please run task lint-fix and task test before the next push.

[Sanskarzz]

Addressed the review in a38b032 and rebased onto upstream/main. The implementation now defaults Claude Code to the one-hour cache, persists --short-prompt-cache as the five-minute opt-out, always writes user-level settings despite advisory overrides, and narrows detection to environment plus managed settings. I also simplified the interface/tests as requested and replied to each inline thread. Local task lint and task test both pass.