@@ -465,7 +465,7 @@ a warm rebuild was `cmp`-identical. Root cause and fix above.
465465 completes without core-body cooperation. Stage isolation by stream
466466 truncation does NOT work here.
467467
468- ### The lm-head container gap (found + gated 2026-10-22)
468+ ### The lm-head container gap (closed 2026-10-22)
469469
470470With the sampler fixed, decode ran — and produced ~ 4.8 tok/s of garbage: ids
471471248068/271/248069/271 (the think template, correct) then a fixed cycling
@@ -475,36 +475,42 @@ rows are NaN**, all within the first 191,736 ids, ~25% of them, no row
475475periodicity (mod 2/3/4/5/8 flat) — a garbage scale read, not a per-group
476476bug. Everything past file coverage recovers the memset zeros.
477477
478- Root cause: pool pack for the hybrid family. The pool's projection tensors
479- go through ` std_perm_gguf ` (which dequantizes/requantizes Q4_K/Q6_K/etc.
480- into the pool law), but the head op is ` lmhead_q8 ` (`src/open_qwen36/
481- pools.cpp` ~ 773), a ** plain memcpy in the kernel's supertile order** — it
482- assumes ` lm_head.weight ` (= gguf ` output.weight ` ) already holds the engine's
483- q8 chunk raster: 32-row × 256-column chunks, 256 bf16 scales then 8192 int8
484- codes, at pool chunk k ← file ` (4*(k/per_band) + k%4)*nk + k%per_band/4 ` ,
485- ` per_band = 4*nk ` (nk = in_dim/256). Only a q4nx-build conversion writes
486- that (Q8NX layout). A raw llama.cpp GGUF stores the head row-major in its
487- own quant; Ornith-1.5's ` output.weight ` is ** Q6_K** (4096 × 248320),
488- ` Qwen3.8-9B-Distill-GGUF ` 's is whole-file ** Q8_0** (byte count even lands
489- exactly on the chunk grid — 1,080,688,640 = 124,160 × 8,704 — but row-major
490- raster + fp16 scales, still wrong). The op's ` bounds() ` silently passed a
491- short count and filled the pool with misread bytes.
492-
493- Gate (now in ` lmhead_q8 ` ): a ` GgufFile ` source is refused by name + type
494- ("convert with q4nx-build"); also ` n % CH8 != 0 ` refuses non-chunk-aligned
495- survivors. Verified: ornith (Q6_K) and qwen3.8-distill (Q8_0) both fail
496- cleanly at load; the q4nx path (` qwen35-npu2:0.8b ` → full coherent
497- "** Paris** ." turn) is untouched. ` gguf_pack_test ` + the registered ctest
498- (GGUF-PACK, OPEN-MANIFEST) pass.
499-
500- ** Still missing** : the raw-GGUF q8 head requant — the analogue of
501- std_perm_gguf's kq path for the q8 law. Q8_0 needs no arithmetic at all (its
502- per-32-block fp16 d IS the group scale — remap into the bf16 scale plane
503- with RNE and apply the supertile permutation); Q6_K/Q4_K need
504- dequant-row → per-32-group requant of int8 code + bf16 scale, matching
505- ` bf16_rne ` semantics used elsewhere in this file. The head requant is our
506- open item on the hybrid family's GGUF path; while it is open, the packer
507- refuses and a hybrid GGUF needs a q4nx-build-converted container.
478+ Root cause: the head op ` lmhead_q8 ` (` src/open_qwen36/pools.cpp ` ~ 773) was a
479+ plain memcpy in the kernel's supertile order — it assumed ` lm_head.weight `
480+ (= gguf ` output.weight ` ) already held the engine's q8 chunk raster: 32-row ×
481+ 256-column chunks, 256 bf16 scales then 8192 int8 codes, at pool chunk
482+ k ← file ` (4*(k/per_band) + k%4)*nk + k%per_band/4 ` , ` per_band = 4*nk `
483+ (nk = in_dim/256). Only a q4nx-build conversion writes that (Q8NX layout).
484+ A raw llama.cpp GGUF stores the head row-major in its own quant; Ornith-1.5's
485+ ` output.weight ` is ** Q6_K** (4096 × 248320), ` Qwen3.8-9B-Distill-GGUF ` 's
486+ is whole-file ** Q8_0** (byte count even lands exactly on the chunk grid —
487+ 1,080,688,640 = 124,160 × 8,704 — but row-major raster + fp16 scales,
488+ still wrong). The op's ` bounds() ` silently passed a short count and filled
489+ the pool with misread bytes.
490+
491+ Implemented (` lmhead_q8 ` , GGUF branch): the same transcode the pool ops do.
492+ - Q8_0: pure remap — the source already carries per-32-block int8 codes
493+ and an fp16 scale, so widen fp16→bf16 (RNE, matching the converter's
494+ cast) and write into the chunk raster at the plain (row-major) index
495+ ` (row/32)*ncol + kt ` — a raw GGUF is row-major, so no supertile
496+ permutation (the memcpy path permutes because the q4nx container stores
497+ the head in supertile order). Bit-identical to a q4nx conversion of the
498+ same Q8_0 tensor.
499+ - anything else (Q6_K, Q4_K, Q4_0, ...): dequant each row to f32 (the
500+ same llama.cpp-faithful path ` std_perm_gguf ` uses for the pool tensors)
501+ and requant to int8 code + bf16 scale per 32-value group
502+ (` bf16_rne(mx/127) ` , clamp to [ -127,127] ), at the same plain index.
503+ Verified: a fresh CLI run on ornith-gguf:9b-i1 now reports ** 0 NaN logit
504+ rows** (was 47,695) with a real argmax; ` gguf_pack_test ` + the registered
505+ ctest (GGUF-PACK, OPEN-MANIFEST) pass; the q4nx path is untouched
506+ (` qwen35-npu2:0.8b ` still answers a full coherent "** Paris** ." turn).
507+
508+ Open: the 9B hybrid GGUF path ** hangs at prefill layer 30 (full_attention)**
509+ on every run — it reproduces with both the f32 and the q4_1 law, so it is
510+ not the head and not the law; it is a per-layer slice issue on the 9B hybrid
511+ path (the 0.8B native and the dense GGUFs run cleanly) that needs its own
512+ investigation. Until that is resolved, hybrid GGUF-direct does not complete
513+ a turn on this box.
508514
509515## Still open (Phase-MoE / later)
510516
0 commit comments