Skip to content

fix(ollama): support gpt-oss thinking parameter - #3263

Open
goutamsidharth287 wants to merge 3 commits into
vectorize-io:mainfrom
goutamsidharth287:fix-ollama-gpt-oss-thinking
Open

fix(ollama): support gpt-oss thinking parameter#3263
goutamsidharth287 wants to merge 3 commits into
vectorize-io:mainfrom
goutamsidharth287:fix-ollama-gpt-oss-thinking

Conversation

@goutamsidharth287

Copy link
Copy Markdown

Summary

Fix Ollama native structured-output requests for gpt-oss models.

Previously, think was hardcoded to False, which causes structured
fact extraction to fail with gpt-oss models.

This change uses think="low" for gpt-oss models while preserving
the existing think=False behavior for other Ollama reasoning models.

Testing

  • Added regression coverage for the Ollama native API path.
  • Verifies the think parameter for both gpt-oss and non-gpt-oss models.
  • Verifies structured-output format is still included.

Fixes #3246

Use a low thinking level for gpt-oss models while preserving the existing behavior for other models.
Add regression coverage for model-specific Ollama think parameters and structured output.
Mock the async Ollama client context used by the native structured-output path.
@nicoloboschi

Copy link
Copy Markdown
Collaborator

Thanks for catching #3246! I opened #3344 as an alternative approach.

Instead of the model-name heuristic ("gpt-oss" in modelthink="low"), it makes think configurable through the existing extra_body surface — set HINDSIGHT_API_LLM_EXTRA_BODY='{"think": "low"}'. This avoids guessing the thinking level, gives an opt-out, and keeps model-specific behavior out of the transport layer (it also lets any future native field / options param be set without a code change).

Proposing #3344 supersede this one — but the fix is entirely thanks to your diagnosis here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

retain: every fact extraction fails on Ollama + gpt-oss — hardcoded think: False yields empty content under format

2 participants