Skip to content

Preserve Qwen3.8-27B precision and validate full-depth decode(work in progress) - #213

Draft
PRIHLOP wants to merge 117 commits into
Atomic-Germ:mainfrom
PRIHLOP:qwen38-main-rebaseline
Draft

PRIHLOP wants to merge 117 commits into
Atomic-Germ:mainfrom
PRIHLOP:qwen38-main-rebaseline

Conversation

@PRIHLOP

@PRIHLOP PRIHLOP commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Preserve Qwen3.8-27B precision and validate full-depth decode(closes #148)

Preserve FP32 activation boundaries, integrate compensated DeltaNet readout,
and use consistent quantized activation sums in Q4 projections. The production
64-layer decode now passes the previously failing DeltaNet input gate, with
unchanged acceptance thresholds and weight packing.

Changes

  • Preserve FP32 normalization, DeltaNet convolution history and split-FFN
    boundaries; use paired-BF16 KV with FP32 attention output-to-projection.
  • Use triple-BF16 state updates and compensated factored DeltaNet readout.
    Share arithmetic helpers to fit full-layer program memory.
  • Use the same INT16-reconstructed activation in both affine Q4 terms of direct
    FP32 preparation. Select it in full lx/ax only for the validated wide dense
    Q4_1 geometry; other geometries and Q8/mixed-role recipes retain existing arithmetic.
  • Declare precision and arithmetic in the manifest, including
    q4_activation_sum: quantized_fp32; reject incompatible decode bundles.
    Preserve weight packing, buffer ABI and scheduling.
  • Add independent causal references, controlled replays, capture/hash checks,
    compiled-header/resource audits and reproducible build runbooks.
  • Add a production-arithmetic FFN probe and all-layer sweep with separate up/gate
    captures, byte-exact production norm/activation identity and streamed weights.
    Pin input/build/source provenance and reject partial fixtures or stale results.
  • Extend the independent prefill oracle with FP32 conv history and paired-BF16 KV.
    Add a native host-stage boundary checker with compiler/source provenance,
    byte-exact partition parity, independent numerical gates and padding checks.
  • Preserve FP32 history in rollback tapes, validate history format before state
    mutation and clear stale history when reusing tapes. Adapt manifest tests to
    move-only ownership by loading independent fixtures.

Validation

  • TDD: 1380 passed, 47 skipped; native CTest 4/4. The native manifest
    test also accepts the actual new 27B export.

  • Fresh export: 41 kernel sets / 82 verified binary hashes. lx/ax maximum
    text 15936 / 13760 bytes, L1 buffer ends 62336 / 59920, stacks 6144.
    All 66 model/pool hashes and 195 regenerated CPU references match provenance.

  • NPU 64 layers × 3 positions, 198 dispatches: 192/192 residuals pass,
    worst error 0.0030399006 <0.005. Logit correlations
    0.9999989597 / 0.9999982780 / 0.9999986060; argmax 8678 / 198 / 2 matches CPU.

  • DeltaNet input and composed state/output: 144/144 pass. Worst normalized
    K error is 0.0009785607 <0.005 at L34/T0. All 6912 strict head cases pass
    at maxrel <1e-4 and cosine >0.9999999; worst state/output errors
    1.2624e-6 / 2.3722e-6. History is byte-exact and padding zero.

  • Attention: 1152/1152 heads, 48/48 projection/sublayer cases; LM-head 3/3.
    FFN front and split-down: 192/192 each; worst front error 0.0002516594,
    down partials 0.0000958907 / 0.0000693562. All 7 boundary checkers pass.

  • Full-model resident runtime: all three logits are byte-identical to the
    harness after a repeated request. Full-buffer reset: 0/3 differing replays.

  • Separate FFN probe: 64 layers × 3 positions, 192/192 pass against unrounded
    FP64 norm/up/gate/activation references. Worst up/gate errors:
    0.0000919118 / 0.0001016790 <0.005. Norm and activation match production
    bytewise in 192/192 cases each. One pool BO uses 0.217 GiB logical NPU
    buffers; the production export and all 82 binary hashes remain unchanged.

  • Host prefill: 5/5 wide cases (1, 2, 255, 256, 257 tokens plus one continuation)
    and 5/5 small cases pass. Outputs, conv/state and KV are byte-identical
    between blocked and sequential native calls. Independent FP64 worst errors:
    DeltaNet 3.9750e-7, attention 8.8449e-7, state 2.5863e-7,
    reconstructed cache 6.1067e-6, all below 1e-3. Conv history is exact;
    cache prefix/tail and poisoned-padding checks pass.

Scope and remaining acceptance

This validates the tested three-position sequential decode and resident
lifecycle, supplemented by isolated per-layer FFN projection checks and synthetic
host prefill boundaries. Host checks condition on synthetic projections; actual
GEMM boundaries and complete model prefill remain unvalidated. Remaining
source/packing checks, full block-prefill parity, serving isolation and final resource/performance
acceptance remain open. Paired-KV block attention uses host attention; this PR
does not establish fully NPU prefill or long-context accuracy. No throughput
claim is made.

Existing model weights require no repacking. Weights, build/capture artifacts
and this description remain outside Git.

PRIHLOP and others added 30 commits October 3, 2026 01:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run Qwen3.8-27B (dense) on the open kernels

3 participants