feat: route memory LLM completions through the host model runtime (opt-in transport) by gorkem2020 · Pull Request #947 · CortexReach/memory-lancedb-pro

gorkem2020 · 2026-07-13T22:01:31Z

Summary

Adds a host-managed completion transport alongside the existing direct OpenAI-compatible client, so the plugin's five completion lanes (extraction, admission-utility, dedup, merge, memory-upgrader) can route through OpenClaw's host-managed runtime LLM catalog instead of always talking to llm.baseURL directly. This is the structural fix requested in issue #901: today the plugin always uses the direct client, which means no provider routing, no app attribution, and ad hoc model-string handling.

What changed

Transport. llm.transport: "direct" | "host", default "direct" (zero behavior change for existing installs), new knob in openclaw.plugin.json's config schema. createHostClient() in src/llm-client.ts sits alongside the existing createApiKeyClient()/createOauthClient(), all three implementing the same LlmClient interface. createLlmClient() picks "host" only when a runtimeLlmComplete function is supplied (feature-detected via api.runtime?.llm?.complete); otherwise it falls back to the direct/oauth transport with one deduped warning per process, not one per client constructed, so a misconfigured or older host never crashes the lane and never spams the log. Model strings pass to the host transport unmodified; the direct client keeps its own openrouter/ prefix-strip normalization, now transport-aware so it never mangles a core-style catalog reference the host transport needs intact. runtime.llm.complete has no AbortSignal parameter, so timeoutMs is enforced with an application-level race instead of true cancellation, and it has no custom-header support, so the existing x-memory-call-label becomes a memory-lancedb-pro:<label> purpose string for equivalent call-site attribution. The host client reuses the same JSON-extraction-and-repair logic as the direct client, so malformed or fenced responses behave identically across transports.

Credential hygiene on fallback. When transport: "host" is configured but the runtime surface turns out to be unavailable, the fallback to the direct client no longer inherits embedding.apiKey/embedding.baseURL as a silent default. On a split-provider setup (embedding via one provider, the LLM lane via host-managed routing) this previously meant the fallback client silently talked to the embedding provider's endpoint with the embedding key instead of failing loudly. It now throws a clear error naming the missing llm.apiKey, caught by the existing surrounding try/catch so it degrades to regex extraction with a warning rather than crashing, and defaults baseURL to OpenRouter's API when llm.apiKey is set but llm.baseURL isn't. The fallback also normalizes a catalog-style model string to the bare provider/model form the direct client needs; an explicitly configured direct transport keeps sending whatever model string it was given, unchanged.

Reasoning effort. Both the host and direct transports now send an explicit reasoning effort on requests: the host transport as a reasoning parameter, the direct transport as reasoning: {effort: ...}. Renamed from llm.reasoningEffort to llm.thinkLevel for consistency with memoryReflection.thinkLevel; llm.reasoningEffort keeps working as a deprecated alias so existing configs don't break. resolveThinkLevel() centralizes the precedence: thinkLevel wins when both are configured (one deduped warning per process), reasoningEffort-only usage keeps working with its own deduped deprecation warning, and blank/whitespace-only values are treated as unset for both keys.

No schema defaults on the effort keys. Neither llm.thinkLevel nor the deprecated llm.reasoningEffort declares a JSON-schema default anymore. A live check found a config that set only llm.reasoningEffort: "low" on disk, but a schema default of "medium" on both keys got materialized into the config object upstream of this plugin on at least one host config-loading path, and once a default is materialized that way it is indistinguishable from a genuine user value by the time it reaches resolveThinkLevel, silently overriding the operator's explicit setting. The code-level "medium" default for the host transport (applied only when neither key is configured) is untouched and still the correct place for that default to live.

Composition with the rest of this wave

Built on plain master before several sibling branches merged, which needed reconciliation once everything came together in the deploy assembly: fix/prompt-architecture's {system, user} prompt split and three-argument completeJson() (the host transport was written with that signature from the start); feat/admission-batch-utility's batch scoring call, which still concatenated system and user into one string for the old two-argument completeJson(); feat/admission-lane-model-affinity's model resolver, which needed a transport-aware skip of the direct-only prefix normalization; and a grounding-v3 interaction where the constructed-grounding short-circuit needed to run before utility scoring on both the single-candidate and batch paths to keep its zero-call guarantee.

A related, pre-existing auto-capture watermark defect (a valid-empty extraction skips the watermark reset a successful extraction applies) was found via the same live probe that surfaced these issues, but is being fixed on its own branch, #948, since it doesn't touch the transport work.

Tests

Red-first throughout. test/llm-host-transport.test.mjs: transport selection, mocked runtimeLlmComplete capturing model and messages, fallback-to-direct with a deduped warning, malformed/throwing host calls, timeout bounding, catalog-model normalization on fallback, and reasoning-effort forwarding (default, explicit override, blank-value fallback). test/admission-control-host-transport.test.mjs and test/batch-admission-host-transport.test.mjs: AdmissionController degrades identically over both transports, including in batch mode. test/admission-model-resolution.test.mjs: transport-aware openrouter/ prefix handling. test/llm-thinklevel-alias.test.mjs: alias precedence and both deduped warnings. test/llm-transport-credential-hygiene.test.mjs: the fallback client never inherits embedding credentials on a split-provider config. test/plugin-manifest-regression.mjs: neither llm.thinkLevel nor llm.reasoningEffort declares a schema default. Existing llm-api-key-client/llm-oauth-client suites pass unchanged, confirming the direct path is unmodified by this PR.

Notes

authProfileId, agentId, and reasoning from the reference runtime-LLM contract aren't all wired through yet; there's no existing config concept in this plugin for auth-profile scoping today, and forwarding reasoning from the plugin is necessary but not sufficient until the host's own runtime-LLM layer also reads and forwards it, since at least one installed core version drops the field before any reasoning-effort default would apply. Can be extended in a follow-up once a host deployment needs it.

Update: remove the llm.reasoningEffort alias entirely (2026-07-17)

llm.reasoningEffort never shipped upstream (zero hits on synced master at the time this was checked), so there is no deprecation constituency to preserve for it. Removed it entirely rather than continuing to carry it as a deprecated alias:

resolveThinkLevel no longer reads reasoningEffort or emits either deprecation warning; it is now a plain presence check on llm.thinkLevel.
LlmClientConfig drops the reasoningEffort field.
The manifest drops the llm.reasoningEffort schema key.
llm.thinkLevel is simply the knob's name now.

Test changes: test/llm-thinklevel-alias.test.mjs is renamed to test/llm-thinklevel.test.mjs and stripped down to the single-key behavior (configured / unconfigured / blank). test/llm-host-transport.test.mjs's reasoning-effort tests now configure thinkLevel instead of reasoningEffort. test/plugin-manifest-regression.mjs's assertion that the deprecated key must stay in the schema is inverted to assert it is absent.

Update (2026-07-18)

Design change after operator review: llm.reasoningEffort never shipped in any release, so the deprecation alias is gone entirely. The knob is llm.thinkLevel only; the schema, client, and tests carry no legacy key. (The schema-default override fix from the previous revision is retained.)

…he direct client Adds llm.transport ("direct" | "host", default "direct") so the completion lane can route through OpenClaw's host-managed runtime LLM surface (runtimeLlmComplete, e.g. api.runtime.llm.complete) instead of always posting to llm.baseURL directly. All five completion lanes (extraction, admission-utility, dedup, merge, memory-upgrader) call the same LlmClient.completeJson() and are unchanged; the transport seam lives entirely under that interface. - src/llm-client.ts: createHostClient() mirrors the existing api-key/oauth clients (JSON extraction + repair reuse, its own timeout guard via Promise.race since the runtime surface has no AbortSignal param). createLlmClient() picks "host" only when runtimeLlmComplete is a function; otherwise it warns once and falls back to direct/oauth so a misconfigured or older host never crashes the lane. - index.ts: resolveRuntimeLlmComplete() feature-detects api.runtime?.llm?.complete (untyped in types/openclaw-plugin-sdk.d.ts, same probe-and-cast pattern already used for api.runtime?.agent?.runEmbeddedPiAgent). Wired into both createLlmClient call sites (smart-extraction, CLI). - openclaw.plugin.json: declares llm.transport in the schema (additionalProperties: false on this block) with a "direct" default. Tests (red-first): transport selection default/explicit, mocked runtimeLlmComplete capturing model+messages for the extract-candidates and admission-utility call shapes, fallback-on-missing-surface, malformed host response, host call throwing, timeout bounding, and an AdmissionController integration test proving the host transport degrades to the same fail-open "Utility scoring unavailable" result as the direct transport. Existing llm-api-key-client/llm-oauth-client suites pass untouched (direct-path regression). Registered in package.json's test chain and scripts/ci-test-manifest.mjs (llm-clients-and-auth group). Motivation: CortexReach#901

…ack path When transport: "host" is configured but the runtime.llm.complete surface is unavailable (older hosts, standalone CLI processes), createLlmClient falls back to the direct OpenAI-compatible client -- but the configured model may be a core-style catalog reference (e.g. "openrouter/openai/gpt-oss-120b") that only the host-managed runtime resolves. The direct client, talking straight to a provider's own API (e.g. OpenRouter), needs the bare "openai/gpt-oss-120b" form and rejects the prefixed one. Adds normalizeDirectModelRef(), the same provider-prefix strip admission-control.ts's normalizeAdmissionModelRef() already does for its own direct-path lane resolution (see feat/admission-lane-model-affinity), generalized into src/llm-client.ts so both the transport fallback and the admission lane resolver share one definition instead of duplicating it. The two will be reconciled at assembly time so admission-control.ts calls this shared helper rather than keeping its own copy. Applied ONLY on the fallback path: an explicitly configured direct transport keeps sending whatever model string it was given, unchanged -- today's behavior is untouched. Red-first: test/llm-host-transport.test.mjs gained two cases -- the fallback path normalizes a catalog-style model to the bare id (watched red: the direct server received the model with the openrouter/ prefix still attached), and an explicit direct transport's model string is byte-identical to before this change.

…nsport The host transport sent no reasoning parameter at all. Tracing a live Langfuse trace of an extraction call showed core's outgoing rawRequest carried reasoning:{effort:"none"} for a reasoning-capable model even though the plugin's own runtimeLlmComplete call never mentioned reasoning, which plausibly explains a run of valid-empty extractions against content with obvious durable facts. Reading the installed OpenClaw core (runtime-llm.runtime-Di6kMOWL.js, the implementation behind api.runtime.llm.complete): createRuntimeLlm's complete() only forwards maxTokens, temperature, and signal from the plugin's request into the underlying completion call. It never reads params.reasoning at all, so on this core version an omitted reasoning field isn't "defaulted to none" by this layer, it's dropped before it would even reach that decision. The disabled-reasoning default the trace showed must be injected further downstream (not in completeSimple, which has no reasoning/effort logic of its own either), most likely in the raw provider request builder for this model family. This means the fix here is necessary but not sufficient on the currently-installed core: forwarding an explicit reasoning value from the plugin is still correct, since it matches the reasoning field already declared on the reference RuntimeLlmCompleteFn contract and costs nothing when a given core version ignores it, but it cannot by itself unblock the host path until core's createRuntimeLlm is changed to also read and forward params.reasoning. - llm.reasoningEffort: new config knob (default "medium"), forwarded only on the host transport. "medium" is a level that never disables reasoning and is supported across the model families core's own reasoning-effort normalization already knows about. - Falls back to the default on a blank/empty-string config value so a minor misconfiguration can't silently reintroduce the incident. - Direct/oauth clients are untouched: they already send no reasoning parameter, letting the provider's own default apply, which is why the direct path never showed this symptom. Red-first: three new cases in test/llm-host-transport.test.mjs (default effort reaches the call, an explicit override reaches the call, a blank override falls back to the default); all prior host-transport cases stay green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…lback warn, and stop inheriting embedding credentials on host fallback Three fixes to the runtime-llm-completions transport layer, all live-motivated: - Direct client requests now send reasoning: {effort: ...} when llm.reasoningEffort is explicitly configured (both the plain direct path and the host->direct fallback, which share the same code path). Bare-slug requests without this get an OpenRouter-side effort:"none" injection that reasoning-mandatory endpoints hard-reject. Left omitted (unchanged behavior) when unconfigured. - The "runtime surface unavailable, falling back to direct" warning is now deduped to once per process instead of once per client constructed (two in the CLI alone). - When llm.transport is "host" and the runtime.llm.complete surface turns out to be unavailable, the direct fallback client no longer inherits embedding.apiKey/embedding.baseURL as a silent default. On a split- provider setup this previously meant the fallback talked to the embedding provider's endpoint with its key instead of failing loudly. It now throws a clear error naming the missing llm.apiKey (caught by the existing surrounding try/catch, so it degrades to regex extraction with a warning rather than crashing) and defaults baseURL to OpenRouter's API when llm.apiKey is set but llm.baseURL isn't.

…deprecated alias Renames the reasoning-effort config knob to llm.thinkLevel for consistency with memoryReflection.thinkLevel. llm.reasoningEffort keeps working as a deprecated alias so the operator's live config (which still sets reasoningEffort: low) does not break: - resolveThinkLevel centralizes the alias resolution inside createLlmClient: llm.thinkLevel wins when both are configured (a single once-per-process warning), and llm.reasoningEffort-only usage keeps working with its own once-per-process deprecation warning naming the new key. Blank/whitespace-only values are treated as unset for both keys, matching reasoningEffort's pre-existing semantics. - Both the host and direct transports now read the resolved config.thinkLevel value; llm.reasoningEffort is kept on LlmClientConfig only as the deprecated input field resolveThinkLevel reads. - openclaw.plugin.json adds llm.thinkLevel (same string/medium-default schema, same host/direct wire-behavior description) and keeps llm.reasoningEffort in the schema with a description marking it deprecated, so existing configs still validate. - index.ts's two createLlmClient call sites now pass both raw keys through unchanged, leaving the alias precedence and warning dedupe to resolveThinkLevel. Red-first: alias precedence, once-per-process dedupe for both the both-keys-configured and deprecated-alias-only warnings, and wire-level confirmation that the resolved value reaches the request in both the canonical and legacy-key cases.

…lts from silently overriding explicit config Both openclaw.plugin.json schema entries (llm.thinkLevel and the deprecated llm.reasoningEffort) declared a JSON-schema "default": "medium" left over from the D1 rename. A live check found an operator config setting only llm.reasoningEffort: "low" on disk, yet a CLI config load produced llm.thinkLevel=medium and resolveThinkLevel picked it, emitting the both-keys-configured warning and silently discarding the operator's explicit low. The gateway's own long-running config load never showed the warning against the same on-disk config and manifest, confirming the materialization happens upstream of this plugin's own code (index.ts's two createLlmClient call sites and resolveThinkLevel itself already read/prefer config.thinkLevel/reasoningEffort correctly given clean input -- all 4 precedence scenarios were already covered and passing) on at least one OpenClaw host config-loading path, not inside this repo. - Remove "default": "medium" from both llm.thinkLevel and llm.reasoningEffort in openclaw.plugin.json. Neither key should carry a schema default: once a default is materialized into the config object, it is indistinguishable from a genuine user value by the time it reaches resolveThinkLevel. The code-level medium default for the host transport (DEFAULT_HOST_REASONING_EFFORT in src/llm-client.ts) is unaffected and still applies when neither key is configured. - Document the presence-based contract and this invariant directly on resolveThinkLevel's docstring so a future edit doesn't reintroduce a schema default on either key. - test/plugin-manifest-regression.mjs: invert the assertion that used to pin thinkLevel's default to "medium" into an assertion that neither schema entry declares a default at all (red-before/green-after verified via git stash against the pre-fix manifest). - test/llm-thinklevel-alias.test.mjs: tighten the only-reasoningEffort- configured test to explicitly assert it never emits the both-keys-wins warning, naming this incident so the regression is traceable.

llm.reasoningEffort never shipped upstream (zero hits on synced master), so there was no deprecation constituency to preserve. llm.thinkLevel is simply the knob's name now: resolveThinkLevel drops the alias read and both warn paths, LlmClientConfig drops the deprecated field, and the manifest drops the legacy schema key. (cherry picked from commit 11a8391)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

gorkem2020 force-pushed the feat/runtime-llm-completions branch from 571abcd to d5dca72 Compare July 15, 2026 00:50

gorkem2020 mentioned this pull request Jul 15, 2026

Memory LLM completion lane bypasses the host model catalog (cannot honor OpenRouter provider routing or other model config) #901

Open

gorkem2020 force-pushed the feat/runtime-llm-completions branch 2 times, most recently from 11a8391 to 08b9083 Compare July 17, 2026 21:39

gorkem2020 and others added 12 commits July 18, 2026 18:53

build: recompile dist for the runtime-llm-completions transport layer

b7fe261

build: recompile dist for the reasoning-effort forwarding fix

61f5b9f

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

build: recompile dist for the thinkLevel schema-default fix

0fa6802

chore: drop the deleted alias test from the CI chain and rebuild dist

81e2dfb

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chore(dist): rebuild on current master

a31460f

gorkem2020 force-pushed the feat/runtime-llm-completions branch from 08b9083 to a31460f Compare July 18, 2026 15:53

gorkem2020 mentioned this pull request Jul 22, 2026

feat: per-lane admission model affinity with an explicit model override #942

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: route memory LLM completions through the host model runtime (opt-in transport)#947

feat: route memory LLM completions through the host model runtime (opt-in transport)#947
gorkem2020 wants to merge 12 commits into
CortexReach:masterfrom
gorkem2020:feat/runtime-llm-completions

gorkem2020 commented Jul 13, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gorkem2020 commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Composition with the rest of this wave

Tests

Notes

Update: remove the llm.reasoningEffort alias entirely (2026-07-17)

Update (2026-07-18)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gorkem2020 commented Jul 13, 2026 •

edited

Loading