[bong-water-water-bong]

Fixes engine#560. A request that asks for temperature: 0 leaves nothing to sample, but llama-server still ran the top-p/min-p passes over the whole vocabulary — on ZAYA1-8B's 262147 entries that is most of the decode step. 1bit serve now forwards such a request with top_k: 1, which takes the argmax and skips them.

The change is in forward(), the single choke point every proxied request passes through, so both OpenAI routes get it. A request that names its own top_k keeps it (only the absent case is filled in), a request with any other temperature is forwarded untouched, and a body with no temperature is not rewritten.

Measured (strixhalo, HRX, packed candidate llama-server at 851fa6c8d, 1bit serve proxying it, greedy 256-token answers, median of 2, --no-mtp)

model	after (top_k 1 via serve)	before (the full chain, top_k 40)
ZAYA1-8B Q4_K_M, code	129.3 tok/s	118.1	1.09x
ZAYA1-8B Q4_K_M, prose	128.9 tok/s	118.2	1.09x
Qwen3.8-27B UD-Q4_K_XL, code	12.0 tok/s	12.0	1.00x
Qwen3.8-27B UD-Q4_K_XL, prose	12.0 tok/s	12.0	1.00x

Two honest notes on those numbers:

• ZAYA gains 9%, not the 30% the issue projected (118 -> 153). The issue measured top_k: 1 on raw llama-server; this measurement goes through 1bit serve, whose own request path (JSON, chat-template application, backend hop) is part of the wall clock, so the share the sampler's skip can recover is smaller here. The direction and the mechanism match; the ceiling does not.
• The 27B is unchanged at 12.0 tok/s. Its decode step is ~83 ms, so a ~2 ms sampler pass is inside this measurement's noise, and nothing suggests the short-circuit hurts it.

One run of this measurement was voided by a GPU reset on the box (MES failed to respond to msg=REMOVE_QUEUE -> MODE2 reset -> reset(4) succeeded) during a plain ZAYA warm-up; the harness now restarts the backend once per arm, and the table above is from the clean re-run. Nothing in this change touches the GPU path.

Test

tests/fake_backend.py now echoes top_k and temperature back, and the new tests/greedy_shortcut.sh (ctest greedy_shortcut, no GPU) pins the four cases:

• temperature 0 on /v1/chat/completions gets top_k 1,
• temperature 0 on /v1/completions gets top_k 1,
• an explicit top_k 5 survives,
• temperature 0.7 adds no top_k,
• a body without temperature is not rewritten.

Run against a locally built 1bit: PASS (5 checks). tools/copyright.py --check → rc 0.

docs/serve.md gains a "Greedy requests (temperature 0)" section with the numbers above and the top_k escape hatch.

🤖 Generated with Claude Code

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 9d22ee5)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 560 - Partially compliant Compliant requirements: Implement greedy short-circuit for requests with temperature <= 0 Forward such requests with top_k: 1 to skip full-vocabulary sampler chain Preserve existing behavior for requests with explicit top_k or non-zero temperatures Update documentation to reflect the new behavior Add tests to verify the behavior Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Logic Condition The condition for applying the greedy shortcut only checks for temperature <= 0 but does not explicitly handle the case where temperature might be a string or other non-numeric type, which could lead to unexpected behavior or exceptions during JSON parsing. if (body.contains("temperature") && body["temperature"].is_number() && body["temperature"].get<double>() <= 0.0 && !body.contains("top_k")) body["top_k"] = 1; Test Coverage The test script does not verify that the temperature field is properly handled when it's not a number (e.g., a string or null), which could lead to edge cases where the shortcut is incorrectly applied or not applied. r=$(ask /v1/chat/completions '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":4,"temperature":0}') check "temperature 0 on /v1/chat/completions gets top_k 1" '[[ "$r" == *"\"fake_top_k\": 1"* || "$r" == *"\"fake_top_k\":1"* ]]' r=$(ask /v1/completions '{"model":"m","prompt":"hi","max_tokens":4,"temperature":0}') check "temperature 0 on /v1/completions gets top_k 1" '[[ "$r" == *"\"fake_top_k\": 1"* || "$r" == *"\"fake_top_k\":1"* ]]' r=$(ask /v1/chat/completions '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":4,"temperature":0,"top_k":5}') check "an explicit top_k is kept" '[[ "$r" == *"\"fake_top_k\": 5"* || "$r" == *"\"fake_top_k\":5"* ]]' r=$(ask /v1/chat/completions '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":4,"temperature":0.7}') check "a real temperature is forwarded untouched (no top_k added)" '[[ "$r" == *"\"fake_top_k\": null"* || "$r" == *"\"fake_top_k\":null"* ]] && [[ "$r" == *"0.7"* ]]' r=$(ask /v1/completions '{"model":"m","prompt":"hi","max_tokens":4}') check "a body without temperature is not rewritten" '[[ "$r" == *"\"fake_top_k\": null"* || "$r" == *"\"fake_top_k\":null"* ]]'

[github-actions]

Persistent review updated to latest commit b53ef31

[github-actions]

Persistent review updated to latest commit 87c8be7

[github-actions]

Persistent review updated to latest commit 9d22ee5