fix: repair LLM test-connection and make silent model/template fallbacks visible

[guofeng201507]

Summary

Two related problems, both of which made an LLM misconfiguration invisible to the client.

1. The admin "Test OpenRouter connection" button could never succeed. POST /settings/test-connection calls LLMService.test_connection(), a method that does not exist, so the handler raised AttributeError and always returned a generic failure:

ERROR app.routes.settings: Connection test failed: 'LLMService' object has no attribute 'test_connection'

Not a regression — git log -S'def test_connection' -- backend_api_python/app/services/llm.py returns no commits.

2. Model-level fallback was silent. call_llm_api tries [requested model] + [configured default] + [static fallback_model], so a bad model id quietly produced answers from a different model. The first symptom is only visible in the provider dashboard — e.g. an invalid deepseek/… id still "worked" because openai/gpt-4o-mini answered instead.

Refs #280.

Changes

app/services/llm.py

• Add LLMService.test_connection(provider=None) -> tuple[bool, str]: probes the provider's models endpoint with a 15s timeout. Spends no completion tokens and never falls back to another provider, so a failure identifies this provider / base URL / key combination.
• Credentials are sent in headers only, never in the URL, so provider error bodies and request exceptions cannot leak the key.
• OpenRouter uses GET /auth/key because its /models is public (returns 200 without a key). Provisioning/management keys — which pass /auth/key but are rejected by chat completions with 401 "User not found" — are reported explicitly.
• Google uses the x-goog-api-key header, matching _call_google_gemini.
• Record last_model_used / last_provider_used / model_fallback_used on the instance when a call succeeds.

app/routes/settings.py — return the probe's detail instead of a generic message.

app/services/indicator_ai_generation.py + app/routes/indicator.py — carry the model usage and the template-fallback reason into the streamed debug frame (generation_error, generation_model, generation_provider, generation_model_fallback). The built-in template passes validation, so previously a failed model call was indistinguishable from a successful response.

Verification

• python -m pytest tests/test_llm_litellm_provider.py -q → 41 passed (34 existing + 7 new: probe cases and model-fallback recording).
• Live against a local v5.5.4 deployment: with a valid key test_connection() returns (True, 'openrouter reachable · model=…'); with a provisioning key it returns an actionable message. A deliberately invalid model id now reports last_model_used = deepseek/deepseek-v4.1-flash, model_fallback_used = True.

[likangjie110]

Thanks — the endpoint really was dead before this (LLMService.test_connection() did not exist, so service=openrouter always answered code: 0 with an AttributeError), and surfacing which model actually answered is a good addition. Two things from applying it on current main:

1. The probe targets whichever provider is active, not OpenRouter

app/routes/settings.py branches on service == 'openrouter' but builds the service with no provider:

if service == 'openrouter':
    llm = LLMService()
    ok, detail = llm.test_connection()

LLMService.provider resolves from LLM_PROVIDER / the addon config, and otherwise auto-detects by key priority (DeepSeek > AtlasCloud > Grok > MiniMax > OpenAI > Google > OpenRouter). So on a deployment whose active provider is DeepSeek, clicking Test OpenRouter connection probes DeepSeek and reports that result under the OpenRouter button — an operator asking an OpenRouter question gets "deepseek reachable · model=…" or a DeepSeek failure. test_connection() already accepts a provider, so the fix is one argument:

ok, detail = llm.test_connection(LLMProvider.OPENROUTER)

2. Nothing covers the endpoint itself

The new tests exercise LLMService.test_connection() well, but POST /api/settings/test-connection — the route the report was actually about — had no coverage, which is why (1) passes CI. I added three route-level cases: a success path that asserts the probe hits /auth/key (not the public /models, which cannot validate a key), a failure path that asserts the response carries the provider's status rather than an AttributeError, and one that asserts the probe ignores the active provider.

Also worth noting: the new Google branch sends the credential in x-goog-api-key rather than the query string, which is the same direction as #279 (provider credentials out of URLs).

What I did with it

Applied on current main plus those two items: https://github.com/likangjie110/QuantDinger/tree/fix/llm-test-connection (4be97c3). Full suite on that branch: 3372 passed / 4 skipped; ruff check app scripts tests, compileall, scripts/backend_quality_check.py and scripts/check_requirements_lock.py pass. Two of the three route tests fail when that argument is reverted (verified by reverting just it) - the endpoint would otherwise probe whichever provider the deployment has active.