Combine existing guarded common, Kimi and Muse source increments - #121
Merged
zozyo merged 50 commits intoSep 22, 2026
Merged
Conversation
Origin-Commit: 353a12f. Shared native schema/usage implementation replaces need for DS runtime model rebuild wrappers. CPU verification pending.
…licit input Origin-Commit: 5a9b069. Preserve existing thinking budget and reasoning-end token handling; CPU verification pending.
Port the remaining common ownership delta from Kimi source 5256c31 while retaining existing GenerateReqInput parent registration. Candidate CPU verification pending.
Origin-Commit: bda19ad. Semantic source port; candidate CPU verification pending.
Origin-Commit: 37d88dc. Preserve default unrestricted behavior and shared rank-local voting. Candidate CPU verification pending.
Merge residual guards from Kimi b873d24 into existing GLM/common backup; no duplicated implementation. Candidate CPU verification pending.
DCP8 hybrid SSM exposes virtual512 pages over physical64 rows. Preserve compute chunk billing while capping and charging continuation allocation in allocator units. Includes independent paged-allocation oracle; CPU validation pending.
Origin-Commit: 287188b. Scoped Muse detector change with original controls; CPU verification pending.
Port source 4dbfdf9 without importing its unrelated shared-parser predecessors. Preserve non-Nemotron behavior, explicit template opt-in, continuation and multimodal guards. 9 default CPU tests pass; pinned Nemotron tokenizer cc84af2fe71647d87f4486c064f320e1e7535243 SHA256 623c34567aebb18582765289fbe23d901c62704d6518d71866e0e58db892b5b7 executes 19 tests with no skips, including real BPE Unicode and token-before-text boundaries. Not a Linux full-import, GPU or model qualification.
Source: 137f96a. Preserve current XGrammar guard and named/allowed tool filtering. Validate model dispatch, invalid efforts, token-first media, history and source methods with 10 CPU regressions. Native dependency closure and model runtime are not qualified.
Restore historical complete-call semantics without duplicating parameter conversion or stream state. Default unknown-name rejection retains the explicit SGLANG_FORWARD_UNKNOWN_TOOLS opt-in. New 16 CPU source-method tests, 16 existing Qwen source-method tests, 8 existing schema cases and all 4 boundary probes pass. Covers every truncation and split boundary, orphan parameters, multiple/repeated calls, schema refs/unions/consts and 4KiB-plus text. Removing name guard fails; removing finish discard fails 85 truncation cases. Eliminated old incremental cursor/parameter state and redundant text fallback. Not GPU, native grammar, SSE or model acceptance. No shared serving/reasoning edits.
Adapt historical 889dac0, c5052c1 and 7070885 contracts. Remove duplicate native-required overrides; keep shared schema roots. Eight CPU source-method/schema tests pass, including every two-chunk split, self-channel non-execution and non-Muse constructor controls. Chat and Responses route constrained framing explicitly. No native grammar or model acceptance.
Adapt c5af5ab to current multi-token terminators, request-specific terminator selection and min-think filtering. Snapshot channel state and count actual inner-grammar accepts for speculative rollback/copy. Muse-only marker opt-in; nine additional source-method lifecycle tests pass (17 total). No native grammar, real tokenizer or GPU evidence.
…te call chain Adapt historical 889dac/7aad27/2224d5/69743b/ed4266 semantics on current shared schema and visibility code. Muse-only strength/default helpers; legacy protocol aliases preserve current exclusion precedence. Full self-recipient delimiter avoids prefix confusion. 27 CPU source/protocol tests pass with zero skips including three actual serving call-chain renders against pinned template900db3ef; DS10 and Qwen16 controls pass. External template bytes match immutable Compose2e16dfd8, not redistributed; packaging license/native XGrammar/GPU remain unqualified.
Port required correctness from original 5404f76, d67e821 and 2b25845 (extracted 81881bd, 39eedc1, 788a898). Official v0.5.20 94602c9 lacks these loader mappings/transforms, identity audit and projector path. Preserve supported qwen35 metadata, GGUF-only quantization, raw gated norms, non-GGUF and other-architecture defaults, and subclass embedding hooks. Include necessary packed weight/type, scalar slot, quantized head and embedding call sites. 20 CPU regressions pass after formatting: actual GGUF writer/reader and actual loader/iterator/Qwen body-head methods on a small tensor graph; multimodal loader/conditional-model chain; real Transformers meta mapping; Conv3d temporal equivalence; BF16 bytes; packed shards; missing/extra/duplicate/shape/type failures and negative guards. PyPI torch2.13.0 transformers5.12.1 gguf0.19.0 numpy2.5.3; offline HF. Initial harness sys.modules cleanup caused nine Torch re-registration errors, corrected by restoring only injected modules. Name-only audit ablation fails shared-alias fixture; removed redundant donor counters/factory registry. Ruff0.15.1 and compile/diff checks pass. Not full-model import/GPU/image/service acceptance. Q8_0 performance path is separate and unmodified.
28 CPU source/protocol/pinned-template cases pass. Keep existing nested exclusion precedence and named constraints despite native-required capability. Format changed files; two pre-existing protocol F811 redefinitions remain outside this migration. Full Responses unit import not run in the dependency-light harness.
Port c4afa98 (extracted be0e348), absent in fixed official v0.5.20 94602c9. Keep CUDA/BF16/Q8_0/rows>=128 and original MMVQ/MMQ/empty/unquantized fallbacks. Unlike the donor's global change, enable only through the supported Qwen GGUF loader; other models, Muse and GGUF MoE calls retain original dispatch. Pass the scope through actual linear merged/unmerged call sites. No user knob or cache. Depends on guarded GGUF loader migration 22eb063c56. 11 CPU source-call tests pass; 20 GGUF loader regressions re-pass. Device metadata and native kernels are simulated, CPU matmul validates operand routing only. Scope-flag removal fails the other-model default regression. No CUDA numerical, throughput or performance acceptance is claimed. Ruff0.15.1, compile and diff checks pass. Dependency audit: current union and historical v0.5.19 pyproject pin torch2.13.0/transformers5.12.1/tokenizers0.22.2, matching this CPU environment. gguf and numpy are unpinned in both; tested gguf0.19.0/numpy2.5.3. Historical kernel pin0.4.6.post1 becomes current0.4.7; GGUF Python native wrapper source/signatures are identical between fixed bases, and actual wrapper call shape is tested. CUDA extension is not installed/executed here. Original r10 Dockerfile overlays source on immutable base without dependency installs; this is not an inventory/readback of that old image. No dependency manifest changes required by tested APIs.
…r compress ratios (sgl-project#39921) (cherry picked from commit 1f0c73e) (cherry picked from commit 5648185)
…r MXFP8 GEMMs (sgl-project#40039) (cherry picked from commit 1b200ff) (cherry picked from commit e533126)
Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai> Co-authored-by: Yuwei An <ayw.sirius19@gmail.com> Co-authored-by: Khoa Pham <khoa.pham@radixark.ai> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> Co-authored-by: Zhichen Zeng <zczeng@uw.edu> Co-authored-by: Ke Bao <ispobaoke@gmail.com> (cherry picked from commit a6cf058) (cherry picked from commit 9af234a)
…n boundaries (cherry picked from commit 2d625cffb8c7ce1ee47cd5948589c0e0f4980798) (cherry picked from commit 7a23a97)
(cherry picked from commit 70733f2ee7e565bed635cac4e8682a5fc75c9641) (cherry picked from commit 81b67d8)
(cherry picked from commit 0ccc40dfd8ceb0fbe56cb68e1dce2890c21cc8bd) (cherry picked from commit 6a899fd)
(cherry picked from commit 9acba212f3376c215a9b48a44dcbace15c0982b7) (cherry picked from commit b6a5ec4)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unified source successor, 2026-09-22
This record does not update the frozen engine428 export, any Governor candidate,
image, CVM or production acceptance. No image build or registry write is needed.
The implementation is Phala-Network/sglang commit
6b3ca7eddd1f0a1eed2774bdfd461626c5e0e780, treeb6730c9fe47e14319bb92dc61b8463ae7dcfedfd, on navigation branchcodex/model-union-v0520-20260922. It combines the three common increments,six Kimi increments, and one Muse increment. No common patch bytes were copied.
Its complete tree equals the earlier combined candidate
83d8dc47ee; the newlinear history orders common first and shares the existing source commits.
unified-v0520-successorrecords the full source range, ten ordered patchreferences, immutable commits and hashes.
export_selectors.pyindependentlyreplays the 31-entry frozen series from official v0.5.20
94602c9c2b7cbdb8efd5c52802dac6a1c180089eto tree83dcb00885129cc2afefdce2e169ebba697a9a67, then the successor to the tree above.The source range is not a Governor v4 release input.
Current semantic scope
The original 110 per-source records, source hashes, ownership and old evidence
remain in COVERAGE.json. They describe frozen engine428 and
must not be silently relabeled as successor acceptance.
87ace709c88e, typed usage89f583819b2f, opt-in mode defaults1db913dd7b381e62ad753c42, absolute checkpoint grid9bd58a168358, prefill sequence cap6329c4dddd08, file ownership/prefixcd0cca6903e8, chunk guards5a1799c43d72, virtual page continuation02558c2aaab16b3ca7eddd1f; Gumbel/deferred-Mamba upstream coverage retains its original proof4dbfdf98a4d5does not clean-apply: serving-chat call wiring and reasoning-parser shared methods changed. The helper, request/token ID propagation, model/template guard and regressions must migrate togetherdsv41-0011-chat-encoding.patch, after ten successful applications. Complete native ABI/recipe and model protocol guards remain mandatoryd0b3e70cbc5ed3cc757d22e79ffa1fff28f58571remains identifiable711978779936d1918d038de8515e33f968cb5193retainedCurrent source paths are the exact paths in each exported source delta.
Key boundaries are
entrypoints/openai/serving_chat.py,parser/reasoning_parser.py,function_call/muse_glimmer_detector.py,managers/tokenizer_manager.py,managers/schedule_policy.py,mem_cache/hicache_storage.py, andmodel_executor/runner/prefill_cuda_graph_runner.pyunder
python/sglang/srt.Verification
passed: null.reference-only evidence, corrupt baseline bytes, wrong tree, unverified base,
expected failure retention and invalid replay mode.
The local Python has no torch, tokenizers or transformers. Source/tree
equivalence cannot replace those gates.
Run
python -m unittest discover -s tests -vandpython scripts/export_selectors.py --source /path/to/sglang --check.The main frozen export remains independently checked by
scripts/export.py.No additional per-model workflow or long-lived profile was added.