Skip to content

Combine existing guarded common, Kimi and Muse source increments - #121

Merged
zozyo merged 50 commits into
codex/unified-serving-v0520from
codex/model-union-v0520-20260922
Sep 22, 2026
Merged

zozyo merged 50 commits into
codex/unified-serving-v0520from
codex/model-union-v0520-20260922

Conversation

@zozyo

@zozyo zozyo commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Unified source successor, 2026-09-22

This record does not update the frozen engine428 export, any Governor candidate,
image, CVM or production acceptance. No image build or registry write is needed.

The implementation is Phala-Network/sglang commit
6b3ca7eddd1f0a1eed2774bdfd461626c5e0e780, tree
b6730c9fe47e14319bb92dc61b8463ae7dcfedfd, on navigation branch
codex/model-union-v0520-20260922. It combines the three common increments,
six Kimi increments, and one Muse increment. No common patch bytes were copied.
Its complete tree equals the earlier combined candidate 83d8dc47ee; the new
linear history orders common first and shares the existing source commits.

unified-v0520-successor records the full source range, ten ordered patch
references, immutable commits and hashes. export_selectors.py independently
replays the 31-entry frozen series from official v0.5.20
94602c9c2b7cbdb8efd5c52802dac6a1c180089e to tree
83dcb00885129cc2afefdce2e169ebba697a9a67, then the successor to the tree above.
The source range is not a Governor v4 release input.

Current semantic scope

The original 110 per-source records, source hashes, ownership and old evidence
remain in COVERAGE.json. They describe frozen engine428 and
must not be silently relabeled as successor acceptance.

Model or family Present in this successor Still not established
Common llguidance mask callbacks/pinning 87ace709c88e, typed usage 89f583819b2f, opt-in mode defaults 1db913dd7b38 Linux package/import and combined runtime tests
Kimi K3 Owner identity 1e62ad753c42, absolute checkpoint grid 9bd58a168358, prefill sequence cap 6329c4dddd08, file ownership/prefix cd0cca6903e8, chunk guards 5a1799c43d72, virtual page continuation 02558c2aaab1 TP8/DCP1/DCP8 runtime and new Governor combination; old CPU/CUDA evidence is scoped to its original source
Muse Glimmer Required tool native channel delta 6b3ca7eddd1f; Gumbel/deferred-Mamba upstream coverage retains its original proof Historical template/cardinality/grammar/cancellation deltas are not all proven equivalent by this one parser patch
Nemotron 3.5 Lightning Frozen common behavior only Literal/control-token source 4dbfdf98a4d5 does not clean-apply: serving-chat call wiring and reasoning-parser shared methods changed. The helper, request/token ID propagation, model/template guard and regressions must migrate together
DeepSeek V4.1 Flash Frozen common behavior only Native closure selector still fails at dsv41-0011-chat-encoding.patch, after ten successful applications. Complete native ABI/recipe and model protocol guards remain mandatory
Gemma 4 26B A4B Common source; historical fork d0b3e70cbc5ed3cc757d22e79ffa1fff28f58571 remains identifiable Shared old image does not prove a Gemma-specific increment or qualify Qwen GGUF paths
Qwen3.8 27B / Qwen3.5 GGUF Current guarded Qwen/common implementation plus mode sampling defaults Historical GGUF load, vision and Q8_0 source records still require semantic and hardware review
Qwen3.6 27B Current common template, effort/history/schema/media paths; historical fork 711978779936d1918d038de8515e33f968cb5193 retained Native XGrammar version/required/XML equivalence is not established by source export
Qwen2.5 7B Common source only Do not infer Qwen3.5/GGUF applicability from shared historical image
GLM 5.3 Existing frozen nine logical changes and formatting correction unchanged No new GPU or Governor-on-GLM acceptance

Current source paths are the exact paths in each exported source delta.
Key boundaries are entrypoints/openai/serving_chat.py,
parser/reasoning_parser.py, function_call/muse_glimmer_detector.py,
managers/tokenizer_manager.py, managers/schedule_policy.py,
mem_cache/hicache_storage.py, and model_executor/runner/prefill_cuda_graph_runner.py
under python/sglang/srt.

Verification

  • Four exact selector replays passed, including the combined successor.
  • DeepSeek retained its expected failure; it did not become a pass.
  • Reference-only selectors verify immutable identity but report passed: null.
  • Seven real-Git fixture regressions passed: moved/deleted navigation refs,
    reference-only evidence, corrupt baseline bytes, wrong tree, unverified base,
    expected failure retention and invalid replay mode.
  • No Linux runtime imports, GPU, final-image or production tests were run here.
    The local Python has no torch, tokenizers or transformers. Source/tree
    equivalence cannot replace those gates.

Run python -m unittest discover -s tests -v and
python scripts/export_selectors.py --source /path/to/sglang --check.
The main frozen export remains independently checked by scripts/export.py.
No additional per-model workflow or long-lived profile was added.

zozyo and others added 30 commits September 21, 2026 19:04
Reconcile legacy 97882c5 and b431a69 with v0.5.20 batched-mask backend. Use one allocation helper for grammar, reasoning callbacks and cached buffers. CPU import regression pending.
Origin-Commit: 353a12f. Shared native schema/usage implementation replaces need for DS runtime model rebuild wrappers. CPU verification pending.
…licit input

Origin-Commit: 5a9b069. Preserve existing thinking budget and reasoning-end token handling; CPU verification pending.
Port the remaining common ownership delta from Kimi source 5256c31 while retaining existing GenerateReqInput parent registration. Candidate CPU verification pending.
Origin-Commit: bda19ad. Semantic source port; candidate CPU verification pending.
Origin-Commit: 37d88dc. Preserve default unrestricted behavior and shared rank-local voting. Candidate CPU verification pending.
Origin-Commits: cf08f42, 530bf4c. Rank/pool namespace and sparse endpoint intersection form one ordered storage correction. Candidate CPU verification pending.
Merge residual guards from Kimi b873d24 into existing GLM/common backup; no duplicated implementation. Candidate CPU verification pending.
DCP8 hybrid SSM exposes virtual512 pages over physical64 rows. Preserve compute chunk billing while capping and charging continuation allocation in allocator units. Includes independent paged-allocation oracle; CPU validation pending.
Origin-Commit: 287188b. Scoped Muse detector change with original controls; CPU verification pending.
Port source 4dbfdf9 without importing its unrelated shared-parser predecessors. Preserve non-Nemotron behavior, explicit template opt-in, continuation and multimodal guards. 9 default CPU tests pass; pinned Nemotron tokenizer cc84af2fe71647d87f4486c064f320e1e7535243 SHA256 623c34567aebb18582765289fbe23d901c62704d6518d71866e0e58db892b5b7 executes 19 tests with no skips, including real BPE Unicode and token-before-text boundaries. Not a Linux full-import, GPU or model qualification.
Source: 137f96a. Preserve current XGrammar guard and named/allowed tool filtering. Validate model dispatch, invalid efforts, token-first media, history and source methods with 10 CPU regressions. Native dependency closure and model runtime are not qualified.
Restore historical complete-call semantics without duplicating parameter conversion or stream state. Default unknown-name rejection retains the explicit SGLANG_FORWARD_UNKNOWN_TOOLS opt-in. New 16 CPU source-method tests, 16 existing Qwen source-method tests, 8 existing schema cases and all 4 boundary probes pass. Covers every truncation and split boundary, orphan parameters, multiple/repeated calls, schema refs/unions/consts and 4KiB-plus text. Removing name guard fails; removing finish discard fails 85 truncation cases. Eliminated old incremental cursor/parameter state and redundant text fallback. Not GPU, native grammar, SSE or model acceptance. No shared serving/reasoning edits.
Adapt historical 889dac0, c5052c1 and 7070885 contracts. Remove duplicate native-required overrides; keep shared schema roots. Eight CPU source-method/schema tests pass, including every two-chunk split, self-channel non-execution and non-Muse constructor controls. Chat and Responses route constrained framing explicitly. No native grammar or model acceptance.
Adapt c5af5ab to current multi-token terminators, request-specific terminator selection and min-think filtering. Snapshot channel state and count actual inner-grammar accepts for speculative rollback/copy. Muse-only marker opt-in; nine additional source-method lifecycle tests pass (17 total). No native grammar, real tokenizer or GPU evidence.
…te call chain

Adapt historical 889dac/7aad27/2224d5/69743b/ed4266 semantics on current shared schema and visibility code. Muse-only strength/default helpers; legacy protocol aliases preserve current exclusion precedence. Full self-recipient delimiter avoids prefix confusion. 27 CPU source/protocol tests pass with zero skips including three actual serving call-chain renders against pinned template900db3ef; DS10 and Qwen16 controls pass. External template bytes match immutable Compose2e16dfd8, not redistributed; packaging license/native XGrammar/GPU remain unqualified.
Port required correctness from original 5404f76, d67e821 and 2b25845 (extracted 81881bd, 39eedc1, 788a898). Official v0.5.20 94602c9 lacks these loader mappings/transforms, identity audit and projector path. Preserve supported qwen35 metadata, GGUF-only quantization, raw gated norms, non-GGUF and other-architecture defaults, and subclass embedding hooks. Include necessary packed weight/type, scalar slot, quantized head and embedding call sites.

20 CPU regressions pass after formatting: actual GGUF writer/reader and actual loader/iterator/Qwen body-head methods on a small tensor graph; multimodal loader/conditional-model chain; real Transformers meta mapping; Conv3d temporal equivalence; BF16 bytes; packed shards; missing/extra/duplicate/shape/type failures and negative guards. PyPI torch2.13.0 transformers5.12.1 gguf0.19.0 numpy2.5.3; offline HF. Initial harness sys.modules cleanup caused nine Torch re-registration errors, corrected by restoring only injected modules. Name-only audit ablation fails shared-alias fixture; removed redundant donor counters/factory registry. Ruff0.15.1 and compile/diff checks pass. Not full-model import/GPU/image/service acceptance. Q8_0 performance path is separate and unmodified.
28 CPU source/protocol/pinned-template cases pass. Keep existing nested exclusion precedence and named constraints despite native-required capability. Format changed files; two pre-existing protocol F811 redefinitions remain outside this migration. Full Responses unit import not run in the dependency-light harness.
Port c4afa98 (extracted be0e348), absent in fixed official v0.5.20 94602c9. Keep CUDA/BF16/Q8_0/rows>=128 and original MMVQ/MMQ/empty/unquantized fallbacks. Unlike the donor's global change, enable only through the supported Qwen GGUF loader; other models, Muse and GGUF MoE calls retain original dispatch. Pass the scope through actual linear merged/unmerged call sites. No user knob or cache. Depends on guarded GGUF loader migration 22eb063c56.

11 CPU source-call tests pass; 20 GGUF loader regressions re-pass. Device metadata and native kernels are simulated, CPU matmul validates operand routing only. Scope-flag removal fails the other-model default regression. No CUDA numerical, throughput or performance acceptance is claimed. Ruff0.15.1, compile and diff checks pass.

Dependency audit: current union and historical v0.5.19 pyproject pin torch2.13.0/transformers5.12.1/tokenizers0.22.2, matching this CPU environment. gguf and numpy are unpinned in both; tested gguf0.19.0/numpy2.5.3. Historical kernel pin0.4.6.post1 becomes current0.4.7; GGUF Python native wrapper source/signatures are identical between fixed bases, and actual wrapper call shape is tested. CUDA extension is not installed/executed here. Original r10 Dockerfile overlays source on immutable base without dependency installs; this is not an inventory/readback of that old image. No dependency manifest changes required by tested APIs.
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit 3401b75)
(cherry picked from commit 22d5407)
…r compress ratios (sgl-project#39921)

(cherry picked from commit 1f0c73e)
(cherry picked from commit 5648185)
…r MXFP8 GEMMs (sgl-project#40039)

(cherry picked from commit 1b200ff)
(cherry picked from commit e533126)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
(cherry picked from commit a6cf058)
(cherry picked from commit 9af234a)
(cherry picked from commit 7061256)
(cherry picked from commit 5645663)
harmya and others added 20 commits September 22, 2026 22:01
(cherry picked from commit 6cec021)
(cherry picked from commit 3c0eb7c)
(cherry picked from commit 1cfea70)
(cherry picked from commit f94b624)
(cherry picked from commit 7766836)
(cherry picked from commit 580fbdb87c120469ea58b8541a1ae4884f4c9296)
(cherry picked from commit ecd9f93)
…n boundaries

(cherry picked from commit 2d625cffb8c7ce1ee47cd5948589c0e0f4980798)
(cherry picked from commit 7a23a97)
(cherry picked from commit 70733f2ee7e565bed635cac4e8682a5fc75c9641)
(cherry picked from commit 81b67d8)
(cherry picked from commit 0ccc40dfd8ceb0fbe56cb68e1dce2890c21cc8bd)
(cherry picked from commit 6a899fd)
(cherry picked from commit 9acba212f3376c215a9b48a44dcbace15c0982b7)
(cherry picked from commit b6a5ec4)
Donors: 5ad27e1, 437b982, 127a364. Shared serving opt-in/media/integer-budget integration remains with integrator; no historical monkey-patch package.
@zozyo
zozyo merged commit c463a16 into codex/unified-serving-v0520 Sep 22, 2026
1 check passed
@zozyo
zozyo deleted the codex/model-union-v0520-20260922 branch September 22, 2026 15:09
@zozyo
zozyo restored the codex/model-union-v0520-20260922 branch September 22, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants