Skip to content

Make LLM teardown purge cached credentials reliably by default #6732

Description

@danbarr

Bug description

thv llm teardown --purge-tokens can report success without deleting cached LLM credentials, and a complete teardown retains those credentials by default. Together, these behaviors make it difficult to get the fresh authentication state that teardown appears to promise.

There are two confirmed cleanup gaps:

  1. pkg/llm/setup.go returns early when ConfiguredTools is empty, before the purge runs. This is reachable after an earlier teardown without --purge-tokens, a setup that authenticated but failed before recording a tool, or config drift.
  2. Secrets-provider initialization, listing, and deletion failures are downgraded to warnings. Teardown clears configuration metadata and exits successfully even when cached credentials remain.

An unexpired cached access token (*_AT) is trusted by its stored expiry before refresh or browser login. A subsequent setup can therefore report login success using the old token and fail later during authenticated discovery or gateway access.

The current default also seems backwards for a complete teardown. Once the last configured LLM tool is removed, retaining a long-lived refresh token is surprising and turns a fresh setup into an implicit session reuse. Retention does make sense for a targeted teardown that leaves another LLM tool configured, because the clients share the same cached session.

Steps to reproduce

  1. Configure LLM gateway authentication and at least one tool:

    thv llm setup --gateway-url <url> --issuer <issuer> --client-id <id>
  2. Tear down without purging:

    thv llm teardown

    This removes the last configured tool but deliberately retains cached token metadata and secrets.

  3. Try to purge afterward:

    thv llm teardown --purge-tokens
  4. The command prints No tools are currently configured. and exits successfully. The __thv_llm_* refresh/access-token entries remain in the secrets provider.

The same state can result when secrets-provider access or deletion fails during the first purge attempt: the command only prints a warning, loses the configured-tool record, and a retry then takes the no-tools early return.

Expected behavior

A teardown that removes the final configured LLM tool should delete all locally cached LLM credentials by default and fail clearly if cleanup cannot be completed.

Recommended command semantics:

  • Untargeted teardown, or targeted teardown of the last configured tool: purge cached LLM credentials by default.
  • --keep-tokens: explicitly retain the cached session during a complete teardown for users who intend to reconfigure immediately.
  • Targeted teardown that leaves other tools configured: retain the shared session by default.
  • --purge-tokens: remain available during a partial teardown as an explicit global-session purge, with clear messaging that the remaining tools will need to authenticate again.

Actual behavior

  • Complete teardown retains cached credentials unless --purge-tokens is supplied.
  • --purge-tokens is skipped entirely when no tools are recorded.
  • Provider and deletion errors are warnings; the command exits zero.
  • A running LLM proxy or helper can potentially persist rotated credentials again after purge.

Environment (if relevant)

  • OS/version: macOS (the control-flow issue is platform-independent)
  • ToolHive version: reproduced in v0.50.0 and present on current main

Additional context

With the encrypted secrets provider, LLM OAuth tokens are entries in the encrypted secrets_encrypted file. The OS keyring stores the password protecting that file. A remaining toolhive keyring item is expected and should not be removed by LLM teardown; the cleanup target is the __thv_llm_* secret scope.

pkg/registry/auth.Logout already provides a useful precedent: it deletes every secret in its OAuth scope so stale refresh and access-token entries cannot short-circuit the next login, and it returns deletion failures instead of reporting success.

Local deletion is not provider-side token revocation and does not clear the browser's SSO session. Help text should make that boundary explicit.

Suggested acceptance criteria:

  • Full/last-tool teardown deletes all LLM-scoped secrets and clears cached token metadata by default.
  • --keep-tokens explicitly preserves the cached session and is mutually exclusive with --purge-tokens.
  • Partial targeted teardown retains tokens by default; explicit --purge-tokens purges the shared session.
  • Purging still runs when zero tools are recorded, repairing partial or previous teardowns.
  • Secrets-provider initialization, listing, or deletion failure produces a nonzero exit and an actionable incomplete-cleanup error.
  • A running LLM proxy is detected/stopped, or the command tells the user to stop it before credentials can be considered purged.
  • Tests cover zero-tool cleanup, last-tool default purge, partial targeted teardown, both explicit flags, provider/delete failures, and credential re-creation by a live proxy.
  • Help text distinguishes local cache deletion from IdP revocation and browser-session logout.

Activity

  1. added
    bugSomething isn't working
    cliChanges that impact CLI functionality
    llm gatewayLLM gateway authentication feature
    needs-triageIssue needs initial triage by a maintainer
    on Oct 1, 2026
  2. self-assigned this
    on Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

authenticationbugSomething isn't workingcliChanges that impact CLI functionalityllm gatewayLLM gateway authentication featureneeds-triageIssue needs initial triage by a maintainer

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions