Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preserve Qwen3.8-27B precision and validate full-depth decode(closes #148)
Preserve FP32 activation boundaries, integrate compensated DeltaNet readout,
and use consistent quantized activation sums in Q4 projections. The production
64-layer decode now passes the previously failing DeltaNet input gate, with
unchanged acceptance thresholds and weight packing.
Changes
boundaries; use paired-BF16 KV with FP32 attention output-to-projection.
Share arithmetic helpers to fit full-layer program memory.
FP32 preparation. Select it in full
lx/axonly for the validated wide denseQ4_1 geometry; other geometries and Q8/mixed-role recipes retain existing arithmetic.
q4_activation_sum: quantized_fp32; reject incompatible decode bundles.Preserve weight packing, buffer ABI and scheduling.
compiled-header/resource audits and reproducible build runbooks.
captures, byte-exact production norm/activation identity and streamed weights.
Pin input/build/source provenance and reject partial fixtures or stale results.
Add a native host-stage boundary checker with compiler/source provenance,
byte-exact partition parity, independent numerical gates and padding checks.
mutation and clear stale history when reusing tapes. Adapt manifest tests to
move-only ownership by loading independent fixtures.
Validation
TDD: 1380 passed, 47 skipped; native CTest 4/4. The native manifest
test also accepts the actual new 27B export.
Fresh export: 41 kernel sets / 82 verified binary hashes. lx/ax maximum
text 15936 / 13760 bytes, L1 buffer ends 62336 / 59920, stacks 6144.
All 66 model/pool hashes and 195 regenerated CPU references match provenance.
NPU 64 layers × 3 positions, 198 dispatches: 192/192 residuals pass,
worst error 0.0030399006 <0.005. Logit correlations
0.9999989597 / 0.9999982780 / 0.9999986060; argmax 8678 / 198 / 2 matches CPU.
DeltaNet input and composed state/output: 144/144 pass. Worst normalized
K error is 0.0009785607 <0.005 at L34/T0. All 6912 strict head cases pass
at maxrel <1e-4 and cosine >0.9999999; worst state/output errors
1.2624e-6 / 2.3722e-6. History is byte-exact and padding zero.
Attention: 1152/1152 heads, 48/48 projection/sublayer cases; LM-head 3/3.
FFN front and split-down: 192/192 each; worst front error 0.0002516594,
down partials 0.0000958907 / 0.0000693562. All 7 boundary checkers pass.
Full-model resident runtime: all three logits are byte-identical to the
harness after a repeated request. Full-buffer reset: 0/3 differing replays.
Separate FFN probe: 64 layers × 3 positions, 192/192 pass against unrounded
FP64 norm/up/gate/activation references. Worst up/gate errors:
0.0000919118 / 0.0001016790 <0.005. Norm and activation match production
bytewise in 192/192 cases each. One pool BO uses 0.217 GiB logical NPU
buffers; the production export and all 82 binary hashes remain unchanged.
Host prefill: 5/5 wide cases (1, 2, 255, 256, 257 tokens plus one continuation)
and 5/5 small cases pass. Outputs, conv/state and KV are byte-identical
between blocked and sequential native calls. Independent FP64 worst errors:
DeltaNet 3.9750e-7, attention 8.8449e-7, state 2.5863e-7,
reconstructed cache 6.1067e-6, all below 1e-3. Conv history is exact;
cache prefix/tail and poisoned-padding checks pass.
Scope and remaining acceptance
This validates the tested three-position sequential decode and resident
lifecycle, supplemented by isolated per-layer FFN projection checks and synthetic
host prefill boundaries. Host checks condition on synthetic projections; actual
GEMM boundaries and complete model prefill remain unvalidated. Remaining
source/packing checks, full block-prefill parity, serving isolation and final resource/performance
acceptance remain open. Paired-KV block attention uses host attention; this PR
does not establish fully NPU prefill or long-context accuracy. No throughput
claim is made.
Existing model weights require no repacking. Weights, build/capture artifacts
and this description remain outside Git.