Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
830 commits
Select commit Hold shift + click to select a range
5c5434e
Remove Quantization test parallelism (#49693)
khluu Jul 24, 2026
589a5b8
[PD][NixlPush][Bugfix] Fix blocking handshake call on writer thread (…
NickLucche Jul 24, 2026
163ecba
[Bugfix] Skip linear bias in layerwise reload to avoid corruption (#4…
li-jinpeng Jul 24, 2026
833483f
Encoder cache extension hooks (#48218)
hotTea123 Jul 24, 2026
a454a1d
[Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom datase…
mgazz Jul 24, 2026
8eac21a
[ROCM] Fix AITER Fused AllReduce RMSNorm for Transformers Backend (#4…
BadrBasowid Jul 24, 2026
7b40fb9
[UX] Reject incompatible nested runtime overrides (#49247)
taneem-ibrahim Jul 24, 2026
453f017
[UX] Improve data-parallel launch validation (#49124)
taneem-ibrahim Jul 24, 2026
d02df74
[Bugfix] Accept RFC 2397 parameters in base64 data URLs (#48973)
thomas-fahrner-parasail Jul 24, 2026
866fea2
[Kernel] ReplaySSM: cache SSM inputs for faster Mamba2 standard decod…
Johnny-Liou Jul 24, 2026
4179806
[CI][AMD] Deprecate DinD for MI355 tests (#49257)
AndreasKaratzas Jul 24, 2026
8c13ee5
Add `sm_107` for Rubin (#49387)
tlrmchlsmth Jul 24, 2026
9863102
[CI] Reuse loaded config for cached tokenizer (#49509)
AndreasKaratzas Jul 24, 2026
7e51939
[CI] Avoid unnecessary Hugging Face metadata requests (#49508)
AndreasKaratzas Jul 24, 2026
c064fa5
Fix GLM-4.1V video placeholder token ID handling. (#49484)
aarushjain29 Jul 24, 2026
e222c33
[Bugfix] Register axk1 config to fix A.X-K1 init (#49727)
djramic Jul 24, 2026
5d8e90a
[WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-op…
tlrmchlsmth Jul 24, 2026
2279575
[AMD][Bugfix][EPLB] Fix elastic EP scaling accuracy on ROCm (#47206)
okorzh-amd Jul 24, 2026
972848f
[Bugfix] Support non-uniform page sizes in KVBlockZeroer (#49704)
elvircrn Jul 24, 2026
9e6746b
[CI] Stabilize memory-sensitive compile and structured output tests (…
ZJY0516 Jul 24, 2026
84d26b9
[Model] Remove Plamo2 (#49729)
hmellor Jul 24, 2026
89f6aa3
[KV Offload][CI] Fall back to buffered I/O without O_DIRECT; fix flak…
hmellor Jul 24, 2026
7513d07
[ROCm][CI] Fix XPASS(strict) on mixed audio embeds test (#49733)
djramic Jul 24, 2026
caa9cad
[ROCm][Docker] Drop MORI_GPU_ARCHS so MoRI autodetects the device arc…
Rohan138 Jul 24, 2026
33c4f35
[ROCm][CI] Wait for ROCm VRAM to settle between compiled and eager LL…
aarushjain29 Jul 24, 2026
213f681
Revert "[Perf][GLM-5.2] Blackwell decode optimizations" (#49768)
WoosukKwon Jul 24, 2026
6a1acac
[BUGFIX] Fix log capture in KV test (#49655)
zhenwei-intel Jul 25, 2026
318b527
[XPU] add warning for xpu graph limitations (#49419)
zhenwei-intel Jul 25, 2026
70052fb
[Bugfix][KV Connector][Mooncake] Keep TP-sharded Mamba state out of t…
ivanium Jul 25, 2026
d9cd774
[ROCm][CI] Force native compile caches onto local disk (#49763)
aarushjain29 Jul 25, 2026
aaaeda9
[CI] fix compile test | refactor VLLM_DISABLE_COMPILE_CACHE for tests…
divakar-amd Jul 25, 2026
0ba2aa3
Stabilize GPU memory teardown between ROCm CI tests (#49242)
aarushjain29 Jul 25, 2026
94682b7
[multimodal] Make PyNvVideoCodec decoder concurrency configurable (#4…
brandonpelfrey Jul 25, 2026
190be7d
[Docs] Fix confusing docstring indentation in nemotron_h.py (#49781)
Johnny-Liou Jul 25, 2026
a82f1b3
[Perf][V1] Skip LRU hash-split in free_blocks when prefix caching is …
adobrzyn Jul 25, 2026
dbcc1cd
[Model] Remove Ouro (#49786)
hmellor Jul 25, 2026
fe51457
[Core] Keep attention backends eligible for text-only serving of pref…
qtris123 Jul 25, 2026
0b1a8bb
[Bugfix][CI] Fix stale Mooncake lookup expectation broken by a merge …
hmellor Jul 25, 2026
ca0defa
Make bare `hugging_face` imports forbidden (#49726)
hmellor Jul 25, 2026
b9b6306
feat[vLLM × v5]: Add audio support for the Transformers backend (#39330)
harshaljanjani Jul 25, 2026
9a50464
[CI] Stop flaky test from downloading model every time (#49800)
hmellor Jul 25, 2026
1423569
[Bugfix][Tool Parser] Fix dropped streaming arguments in Jamba and In…
mosya415 Jul 25, 2026
d1a8ba6
[Bugfix][MiniMax-M3] Fix token-major top-k buffer handling in Triton …
lengrongfu Jul 25, 2026
3e74c60
[Docs] Use `gen-files` for generated docs content (#49587)
hmellor Jul 25, 2026
33ef67e
[BugFix] Increase the max supported duration for MOSS-TD (#49403)
gcanlin Jul 25, 2026
0b0bd2b
[Feature] Add fault tolerance framework (simplified) for DP+EP extern…
fangyuchu Jul 25, 2026
2e0da24
Mergify message not on cancelled (#45117)
hmellor Jul 25, 2026
7fe6d3c
[Perf] Fix moe `reduce_scatter` perf regression by removing additiona…
yewentao256 Jul 25, 2026
26d725c
[Model] Add VaultGemma via Transformers modeling backend (#49803)
hmellor Jul 25, 2026
9321aff
[Bugfix] Wait for the linear bias before layerwise online processing …
hmellor Jul 25, 2026
6b0103d
[CI] Stabilize Pooling Rerank Equivalence Test (#49822)
taneem-ibrahim Jul 25, 2026
ee1d996
[Build] Fix for DeepEP manylinux pidfd sycall usage (#49814)
tlrmchlsmth Jul 25, 2026
70009fb
[MM][CG] Support ViT CUDA Graph for Gemma-4 (#46837)
anthonsu Jul 25, 2026
dbd80cc
[UX] DCP Topology Validation (#49777)
taneem-ibrahim Jul 25, 2026
d30b1ec
[Bugfix][KV Offloading] Defer request finalization until final store …
Palaiologos1453 Jul 25, 2026
0111002
[Kernel] TD operand loads for batched MoE GEMM (moe_mmk) on XPU (#46340)
oonyshch Jul 26, 2026
7a6a5b3
[CI] Compute speech WER directly with jiwer (#49773)
Change72 Jul 26, 2026
b153ae6
[XPU][CI] add heterogeneous TP UT (#49651)
zhenwei-intel Jul 26, 2026
48ebd6f
[Bugfix][KVConnector] Disable cross-layer KV blocks for per-token-hea…
Achyuthan-S Jul 26, 2026
1240c74
[Bugfix] Respect declared attention contract for ColQwen3.5 retriever…
athrael-soju Jul 26, 2026
7a29a3c
[Bugfix][KV Offload] Namespace persistent cache by model runner (#49440)
jongukc Jul 26, 2026
7eca0e1
[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (…
Change72 Jul 26, 2026
2e860de
[Doc] Add compile cache volume example to the Docker deployment page …
matteso1 Jul 26, 2026
30b0714
[Perf] DeepSeek-OCR-2 TTFT Optimize (#49531)
LiuLi1998 Jul 26, 2026
0164022
[CI] Fix speech correctness check rejecting improved WER (#49853)
taneem-ibrahim Jul 26, 2026
8d28b48
[Perf] Isolate MM preprocessing on its own executor (#49524)
guan404ming Jul 26, 2026
21fd9e8
[Model] Support top_k and top_p sampling for DiffusionGemma (#45429)
guan404ming Jul 26, 2026
da3a252
[KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving …
liranschour Jul 26, 2026
5559679
[Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks…
coltonottley Jul 26, 2026
0da6e7f
[Bugfix] Reject contradictory custom-op directives (#49134)
taneem-ibrahim Jul 26, 2026
3f1d409
[KV Offload] Fix num_tokens_after_batch for different termination typ…
Alex-ai-future Jul 26, 2026
7154856
[Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive…
dsocek Jul 26, 2026
b68d7ef
[Bugfix][KV Offload] Namespace auto cache dtype by effective dtype (#…
jongukc Jul 26, 2026
b5b61c6
[Core][Distributed] Add process-checkpoint lifecycle hooks for commun…
galletas1712 Jul 26, 2026
9e50e10
[Bugfix][CuMem] Make KV-cache wake cleanup tag-safe (#49857)
aoshen02 Jul 26, 2026
0934b26
[CI/Build] Refresh tags before building macOS wheel (#49901)
khluu Jul 26, 2026
fdaa0d9
[ModelRunner V2] Support encoder-only attention (#49331)
njhill Jul 27, 2026
f055388
[Bugfix] Prevent NaN poisoning in xpu_mla_sparse for fully-masked ind…
nickus Jul 27, 2026
50aa830
[BugFix][MRV2] Don't create dummy requests longer than `max_model_len…
njhill Jul 27, 2026
ffc4f08
[Core][KV-transfer] MoRIIO: heterogeneous TP<->DP prefill/decode read…
edwinlim0919 Jul 27, 2026
439f336
[Core] Fix gpu<->cpu syncs in MRV2 mamba_hybrid.py (#49736)
njhill Jul 27, 2026
ac87549
[CI][ROCm] Reduce V1 attention test runtime (#49916)
AndreasKaratzas Jul 27, 2026
854c33f
[CI][ROCm] Keep global GPU memory cleanup opt-in (#49911)
AndreasKaratzas Jul 27, 2026
bf4f633
[XPU] Enable QK Norm + RoPE fusion pass on XPU (#49394)
chaojun-zhang Jul 27, 2026
8040ef2
[Frontend] expose stream_interval as req sampling param (#49754)
walterbm Jul 27, 2026
da99ffc
[ROCm][CI] Keep native datasets cache off shared NFS (#49516)
AndreasKaratzas Jul 27, 2026
8de50e4
[Docs] Document NVFP4 GEMM kernel selection and Marlin weight-only fa…
harjothkhara Jul 27, 2026
ff61739
[CI] Add kimi and k3 auto-labeling rules (#49895)
jcotant-inferact Jul 27, 2026
29fdeab
[XPU][CI] Add more test cases in Intel GPU CI (#49422)
zxd1997066 Jul 27, 2026
74d3b79
[Bugfix] Fix mHC block-M prenorm GEMM cross-row reduction carry-over …
njhill Jul 27, 2026
49f31d7
[ROCm] Make vllm_c RMSNorm output contiguous (#49913)
AndreasKaratzas Jul 27, 2026
53397fb
[Bugfix][KV Offload][P2P] Fix EngineCore crash reconnecting to a reap…
thegoldenflow Jul 27, 2026
5f89a03
[CI] Explicitly tear down speculative decode runners (#49910)
AndreasKaratzas Jul 27, 2026
f19ee27
[Hardware][Power] Add FAST_EXP for Power (#49571)
Akashcodes732 Jul 27, 2026
c314af1
[CPU][Perf] INT8 Fused MoE Kernel for Arm CPUs (#48637)
fadara01 Jul 27, 2026
544cb72
[CPU][Spec Decode] Optimize GDN conv path for speculative decoding (#…
tianmu-li Jul 27, 2026
d742856
[3/N][Core][KV Connector] Support reliable partial-tail KV offload fo…
Dao007forever Jul 27, 2026
5d07e26
[Quantization][INC]Add MXFP8 Linear Support (#47514)
Zhenzhong1 Jul 27, 2026
e099004
[CI][ROCm] Reduce kernel test runtime (#49915)
AndreasKaratzas Jul 27, 2026
fd9d2ed
[Rust Frontend] Keep `--max-model-len` engine-owned (#49944)
BugenZhao Jul 27, 2026
8061dc2
[Bugfix] Normalize sparse MLA warmup compression ratios (#49392)
xwu-intel Jul 27, 2026
afc9452
[XPU][CI] Use platform device in InputBatch V2 test (#49939)
zhenwei-intel Jul 27, 2026
cbc3a87
[Tokenizer] Use HF config for HF tokenizers (#49907)
AndreasKaratzas Jul 27, 2026
eb290ab
[Bugfix][CPU] Zero-pad MoE intermediate size for grouped-gemm TP alig…
bigPYJ1151 Jul 27, 2026
7f599d7
[communication] [bugfix] fix quickreduce acc error in cudagraph mode …
haoyangli0109 Jul 27, 2026
394beb6
[Bugfix][ROCm] Use batch DMA for CPU KV cache loads (#49843)
AndreasKaratzas Jul 27, 2026
312ea82
[CI][ROCm] Make hf-xet reconstruction safe on shared NFS (#49837)
AndreasKaratzas Jul 27, 2026
bc3629b
[ROCm][CI] Skip three torchao tests of gfx950 until `torchao==0.18` i…
fxmarty Jul 27, 2026
0906123
[ROCm] [Model] Enable TML inkling (#48841)
tjtanaa Jul 27, 2026
30fbd05
[ROCm] Use backend-default dot precision for ReplaySSM (#49909)
AndreasKaratzas Jul 27, 2026
77cba02
[KV Offloading] Per-request tier filtering with TierFilter/TierMatche…
ronensc Jul 27, 2026
92e8518
Improve Transformers modelling backend `fx` tracer (#49957)
hmellor Jul 27, 2026
81962bb
[Bugfix]Reject invalid FlashInfer MNNVL workspaces (#49043)
lengrongfu Jul 27, 2026
96fa3f4
[Perf] Skip ll_bf16 router GEMM warmup for non-MoE models (#49659)
neweyes Jul 27, 2026
a89015c
[Perf] Make merge attention context count a runtime argument (#48739)
liminfei-amd Jul 27, 2026
dbccc5a
[Model] Enable EVS for Qwen3.5 (#48912)
garrygale Jul 27, 2026
59a6b04
[Core] Fix internal LB load-balancing (#49204)
njhill Jul 27, 2026
56c96b0
[Perf] Tune LL BF16 Router GEMM (#48774)
LopezCastroRoberto Jul 27, 2026
ef9975d
[Bugfix] Reject pipeline parallelism for DiffusionGemma (#45828)
guan404ming Jul 27, 2026
27d7061
[Bugfix] Restore truncate_prompt_tokens for Jina rerank/score online …
umut-polat Jul 27, 2026
d2ca300
[MRV2][Performance] Skip no-op FP32 logits materialization (#47711)
jesse996 Jul 27, 2026
04502de
[Perf] Hash videos by source bytes (#49607)
guan404ming Jul 27, 2026
3f47a83
[Bugfix] Fix VLLM_ENFORCE_STRICT_TOOL_CALLING mutation in tests (#49846)
yzong-rh Jul 27, 2026
2b465b2
[Misc][PD] Nixl cleanup `get_backend_aware_kv_block_len` and `virtual…
NickLucche Jul 27, 2026
e3c2fc3
[Rust Frontend][gRPC] Add server and model discovery (#49491)
Jul 27, 2026
15d65f8
[Bugfix] Changed speech to text chunk timestamp to cumulative approac…
TobyB1702 Jul 27, 2026
8112b6c
[MRV2] Always build attn metadata at capture time (#49364) (#49995)
njhill Jul 27, 2026
bf2b45b
[Attention] Integrate FlashAttention 4 SM100 headdim 256 support (#42…
MatthewBonanni Jul 27, 2026
99de48e
Fix MLA padding and grouped topk routing in the Transformers modellin…
hmellor Jul 27, 2026
ed13deb
[Bugfix][CPU] Fall back to torch for unaligned swigluoai on NEON/vec …
oops-oom Jul 27, 2026
fd10e89
[Test] Regression test for hybrid-Mamba eagle cache-peek in Mooncake …
puririshi98 Jul 27, 2026
831d384
[Core] Fail fast when /dev/shm is too small for the shm ring buffer (…
andreatassi Jul 27, 2026
b2f9e4c
[DSv4 Perf] Adaptive topk width, 1.0% E2E throughput improvement (#50…
yewentao256 Jul 27, 2026
b5bcb3c
[Refactor] Remove dead code in multiple files (#49745)
yewentao256 Jul 27, 2026
1053e24
[ROCm][Quantization][5/N] Refactor quark_moe w8a8-int8 w/ oracle (#46…
amd-sourjya Jul 27, 2026
99115fc
[CI] Initialize DeepEP FP8 test weights (#49912)
AndreasKaratzas Jul 27, 2026
53f6dd5
[CI][ROCm] Fix `test_ocp_mx_wikitext_correctness` reference value (#4…
fxmarty-amd Jul 27, 2026
28158b2
[ROCm] [BugFix] Fix Quark GLM-5.2 Checkpoint inference: indexer wk pe…
ColinZ22 Jul 27, 2026
ebcef33
Fix MQA with tensor parallelism on transformers modeling backend (#…
microslaw Jul 27, 2026
1e34a13
Fix Humming non-gated MoE (#49096)
netanel-haber Jul 27, 2026
272abd5
[Tests][Spec Decode] Add gemma4 MTP acceptance rates test (#47920)
TheEpicDolphin Jul 27, 2026
60417b4
[Core][PCP] Select MRV2 when PCP is enabled (#50034)
LucasWilkinson Jul 28, 2026
60b3d39
[Docs] Remove experimental warning for EP (#50057)
WoosukKwon Jul 28, 2026
1206891
[ROCm][KVConnector][MoRI-IO] Fix WRITE-mode remote-TP rank collapse (…
avininjamay8 Jul 28, 2026
02b6ecf
[ROCm][DSv3.2] Eliminate per-decode FillFunctor launches in sparse-ML…
frida-andersson Jul 28, 2026
e68bfc2
[CI][ROCm] Soft-fail Python-only installation mirror (#50041)
AndreasKaratzas Jul 28, 2026
73af7a3
[XPU] Add online fp8 quantization test (#44513)
yma11 Jul 28, 2026
7aea73d
[ROCm][Quark][6/N] Use MXFP4 linear kernel abstraction for `aiter` ba…
fxmarty-amd Jul 28, 2026
60915c9
[Feature] Add VidCom2 video token pruning (#47750)
nvbfalk Jul 28, 2026
d18ed23
[KV-offload][FS] : Batch store/load_block in C (#49152)
varun-sundar-rabindranath Jul 28, 2026
33fe71a
[AMD] Revert `Mxfp4MoeBackend.TRITON_UNFUSED` fallback (#46491)
fxmarty Jul 28, 2026
fbb1ef6
[Bugfix] Fix DeepseekV4FP8 Quark MXFP4 crash on list-valued weight (#…
ColinZ22 Jul 28, 2026
a8f2960
[KV Offload] Make compact secondary identity TP-independent (#49858)
Change72 Jul 28, 2026
52c3c4a
[Bugfix][KV Offload][OBJ] Preserve job completion during cleanup (#49…
mindungil Jul 28, 2026
d223c90
[Bugfix] Only pad transformers backend `value` when it is narrower (#…
njhill Jul 28, 2026
7458793
[Build] Fix CUDA arch detection producing kernel-less builds on SM121…
ayush1399 Jul 28, 2026
f472ab0
Remove triton per group quant [ROCm] [Bugfix] (#49621)
afriedri Jul 28, 2026
90245f4
[Bugfix] Fix multi-modal support on CPU MRV2 (#50073)
bigPYJ1151 Jul 28, 2026
9069a57
[Core][Frontend] Add weight version tagging for RL rollouts (#49040)
ShuoleiWang Jul 28, 2026
03a2d03
[Bugfix] Respect cgroup memory limits on all platforms (#49966)
chaunceyjiang Jul 28, 2026
b09688a
[Bugfix][Spec Decode] Preserve draft buffers across level-2 sleep (#4…
aoshen02 Jul 28, 2026
99b57a4
[CI][ROCm] Soft fail LoRA mirror (#50086)
AndreasKaratzas Jul 28, 2026
61ac368
[Kimi-K3] Add AttnRes kernels (#50090)
gau-nernst Jul 28, 2026
5ed3faa
[Rust Frontend] Add ordinary-text tokenizer encoding (#49992)
BugenZhao Jul 28, 2026
88402a4
[Test] Skip ROCm AITER MLA prefill tests on non-ROCm platforms (#49945)
Liangliang-Ma Jul 28, 2026
247470f
[CI] Add PyTorch stable ABI audit check (#48164)
cleonard530 Jul 28, 2026
25ace8f
[CI] Increase Qwen3.5 MTP GSM8K generation length (#49881)
ZJY0516 Jul 28, 2026
bf9f230
[Rust Frontend] Fix finish reason for named tool choices (#49496)
reidliu41 Jul 28, 2026
912d6b6
[Rust Frontend] Align sampling validation with Python (#47494)
reidliu41 Jul 28, 2026
d2bfc6f
[Build] Fix DeepEP CUDA driver stub linking (#50103)
khluu Jul 28, 2026
35efdf6
[Elastic EP] Async preparation (#47288)
itayalroy Jul 28, 2026
948107a
[Bugfix] Enhance extra_config handling for layer name suffix matching…
xin3he Jul 28, 2026
98e91a9
[PD][NixlPush] Skip extra `add_remote_agent` step in D->P handshake (…
NickLucche Jul 28, 2026
9b9fc40
add epilogue hook to flex attention (#45841)
liangel-02 Jul 28, 2026
601fa9a
[KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid …
njhill Jul 28, 2026
62d8db7
[Bugfix] Add missing `vllm/models/kimi_k3/__init__.py` (#50131)
hmellor Jul 28, 2026
94100b5
[CI] Wire untethered test files into CI jobs (#49340)
njhill Jul 28, 2026
b6cbba8
[Bugfix][Kernel] Fix batch invariance in RMSNorm kernels by pinning b…
oops-oom Jul 28, 2026
1e81853
[Bugfix][KV Offload] Keep Mamba block span unscaled under DCP (#49964)
jongukc Jul 28, 2026
0d0504b
[Core] Warm up runner-owned Triton kernels before the first request (…
njhill Jul 28, 2026
30217b0
[Bugfix][KV Offload][P2P] Scope serve state to fetch rounds (#49877)
Etelis Jul 28, 2026
4fb483c
[Docs] Expand llm-d integration page (#45432)
Ibrahim2595 Jul 28, 2026
6453fc0
[Bugfix] Don't reuse engine core payload buffer while zmq is sending …
njhill Jul 28, 2026
ba702e9
[Attention] Skip sparse indexer scoring for dense short prefills (#48…
qianlihuang Jul 28, 2026
01661cc
[Rust][Benchmark] Make `vllm bench serve` Rust delegation opt-in (#50…
BugenZhao Jul 28, 2026
4f56321
[ROCm] Cache fp32 upcast of static e8m0 weight scale in AITER scaled_…
jiacao-amd Jul 28, 2026
05a0814
[ROCm] Fix and optimize GPT-J-style MRoPE (#49906)
AndreasKaratzas Jul 28, 2026
8a7b3c2
[compressed-tensors] update `find_matched_target` order to prioritize…
brian-dellabetta Jul 28, 2026
1db989b
[Bugfix][Multimodal] Fix video temporal padding estimates (#49030)
labAxiaoming Jul 28, 2026
6c7e679
[ROCm][Bugfix] Sanitize AITER paged-MQA logits before sparse top-k fo…
shen-shanshan Jul 28, 2026
118bcde
[BugFix] Fix clang spinloop mwaitx include (#45532)
johnnyychiu Jul 28, 2026
d552a68
[Rust Frontend] Extract shared tracing setup logic into `vllm-tracing…
BugenZhao Jul 28, 2026
0b6aa3c
[Bugfix][Spec Decode] Size DFlash query buffers for cudagraph-padded …
siddhant-bharti Jul 28, 2026
2899dca
[Model] Add Kimi K3 support: Rust frontend [1/2] (#50104)
WoosukKwon Jul 28, 2026
bb3b61f
perf: dispatch non-grouped bias-less topk routing methods to fused pa…
jdebache Jul 28, 2026
a07fac7
[Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue …
mrn3088 Jul 28, 2026
e7f6a39
[Test] Make EPD correctness tests configurable for XPU (#50110)
zhenwei-intel Jul 28, 2026
5369f7b
[MXFP8][ROCm] Fix MXFP8 MoE backend selection (#49747)
fxmarty-amd Jul 28, 2026
176256b
[ROCm][CI] Stabilize ROCm audio streaming test (#50163)
AndreasKaratzas Jul 28, 2026
fe65aa6
[CI][NIXL] Fix flaky DP+EP test port conflict (#50171)
divakar-amd Jul 29, 2026
56f31af
[Bugfix] Fix /wake_up crash on hybrid models (Mamba/DeltaNet) (#41602)
kevglynn Jul 29, 2026
6fbbcf2
[BugFix] Stop dummy runs from writing mamba state through stale block…
njhill Jul 29, 2026
7f4c52f
[CI] Add comment-based Buildkite triggers (#50132)
khluu Jul 29, 2026
54ab69b
[CI] Allow comment-triggered builds past pipeline filters (#50197)
khluu Jul 29, 2026
17a74b7
[Model] Add Inkling compressed-tensors dynamic FP8 support (#48876)
krishnateja95 Jul 29, 2026
7398a30
[ROCm][CI] Stabilize ngram and suffix correctness test (#50190)
AndreasKaratzas Jul 29, 2026
30c2718
[CompressedTensors] FP4 Qutlass Integration (#43229)
kylesayrs Jul 29, 2026
32a423a
Integrate CuTeDSL MoE for ReLU2 NVFP4 (#49580)
danielafrimi Jul 29, 2026
f37f03d
[KV Connector] Support NIXL P/D for hybrid MLA+SSM models (#49762)
njhill Jul 29, 2026
58f9659
[Frontend][Core] Standardize request error handling with VLLMError hi…
zqzten Jul 29, 2026
0bb548b
[CI][ROCm] Stabilize Qwen2-VL LoRA test (#50161)
AndreasKaratzas Jul 29, 2026
dc1be79
Add CachePolicyFactory for pluggable/external eviction policies (#49114)
philippesic Jul 29, 2026
db7a79c
[CPU] Fix s390x builds and update torch version in dockerfile (#50144)
R3hankhan123 Jul 29, 2026
6f91edf
[Test] dynamic_shapes_compilation (#49974)
JaredforReal Jul 29, 2026
6f00a1a
fused_moe: add VLLM_TRITON_USE_TD tensor-descriptor path (#42436)
afierka-intel Jul 29, 2026
7c6729b
[Model] Add Kimi K3 support: model files and kernels [1/N] (#50089)
ZJY0516 Jul 29, 2026
7de49ba
[XPU][UT][CI] add xpu config to run gpt-oss accuracy in ut and ci (#4…
zufangzhu Jul 29, 2026
100d655
[CI] Allow PR comment acknowledgements (#50211)
khluu Jul 29, 2026
65a1a16
[CPU] Fix FP8 attention scratchpad sizing (#50194)
tianmu-li Jul 29, 2026
6370e53
[Frontend] Reuse prefill token ids on the decode chat path for disagg…
eicherseiji Jul 29, 2026
f5a7cce
[Model] Add Kimi K3 support: Python frontend [2/2] (#50093)
BugenZhao Jul 29, 2026
ad5d29d
[Model] Support Qwen3.5 text-only dense and MoE models (#50210)
PerkzZheng Jul 29, 2026
32e657e
[BugFix] eagle draft max position embeddings (#49343)
JaredforReal Jul 29, 2026
df2735e
[Misc][Minimax-M3]add default video_processor (#50092)
lengrongfu Jul 29, 2026
5b29c95
[XPU] upgrade to torch 2.13 (#48677)
yma11 Jul 29, 2026
dad7a63
[EC Connector] Add has_pending_push_work (#49582)
omerpaz95 Jul 29, 2026
5b14019
[CI] Fix MXFP8 MOE backend selection tests on gfx942 (#50222)
fxmarty-amd Jul 29, 2026
c44e191
[Rust Frontend] Add --limit-mm-per-prompt support (#49604)
cinnamonica02 Jul 29, 2026
9a4e5f9
[CI/Perf] Fix malformed serving benchmark config (#43538)
fallintoplace Jul 29, 2026
542a8fa
[KV Offload] Move CPUOffloadingSpec onto SharedOffloadRegion (#50094)
Change72 Jul 29, 2026
aeaa50a
[Bugfix][Multimodal] Include media IO config in MM cache hash (#49975)
guan404ming Jul 29, 2026
f51193b
[Kernel][Mamba] Fused-kernel support for align-mode DS-conv state mig…
sungsooha Jul 29, 2026
72297d8
[XPU] Route weightless RMSNorm to _C dispatch (#47121)
yintong-lu Jul 29, 2026
625871b
[CI][Test] Fix pooling truncation test after VLLMError hierarchy chan…
stefankoncarevic Jul 29, 2026
242c591
[Rust Frontend] Send multimodal tensors in auxiliary frames (#49341)
reidliu41 Jul 29, 2026
e0cfa52
[Bugfix][Frontend] Return transcription and translation verbose as fl…
wskr00 Jul 29, 2026
43eaefb
[ModelRunner V2] Enable sequence pooling for embedding and classifica…
taneem-ibrahim Jul 29, 2026
a0c092e
[BugFix] Fix `num_output_placeholders` preemption underflow (#48245)
njhill Jul 29, 2026
d6247d7
[Spec Decode][Perf] Replicate DSpark Markov head across TP ranks (#49…
mgoin Jul 29, 2026
381b691
[ROCm][CI] Fix Kimi K3 KDA on ROCm (#50262)
stefankoncarevic Jul 29, 2026
8255369
[KV Connector] Fix NIXL mamba state pairing for multi-slot block tabl…
njhill Jul 29, 2026
f98061c
fix(step3p5-mtp): honor exclude_modules for the MTP head via prefix (…
chanh Jul 29, 2026
5c7a7f9
[docs] Add documentation for pynvvideocodec video decoding backend (#…
brandonpelfrey Jul 29, 2026
9347745
[torch.compile] Compile `CustomOp.forward_native` for ReLU^2 to avoid…
roikoren755 Jul 29, 2026
82642d7
[Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance impr…
yewentao256 Jul 29, 2026
2ecd864
Revert "[Misc][Minimax-M3]add default video_processor (#50092)" (#50313)
njhill Jul 29, 2026
48fc2e2
feat(grpc): add KV event source discovery (#50033)
Jul 29, 2026
5fa0154
[Rubin] Enable NVLink all-reduce paths on SM107 (#49647)
zaristei Jul 29, 2026
68206c5
revert(model): remove flat SigLIP2 weight mapping
yuan-alex Jul 29, 2026
36b27a3
chore: sync upstream main
yuan-alex Jul 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
103 changes: 103 additions & 0 deletions .buildkite/check-torch-abi.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project

"""Audit vLLM compiled libraries for PyTorch stable ABI compliance."""

import fnmatch
import sys
from pathlib import Path

from torch_abi_audit import inspect_package
from torch_abi_audit.report import ExtensionReport, PackageReport

# Temporary allowlist of extensions not yet on the stable ABI.
# Shrink and remove over time.
ALLOWED_UNSTABLE_LIBRARIES: tuple[str, ...] = (
"_flashkda_C.abi3.so",
"vllm_flash_attn/_vllm_fa2_C.abi3.so",
"vllm_flash_attn/_vllm_fa3_C.abi3.so",
"third_party/deep_gemm/_C*.so",
)


def _relative_path(lib: ExtensionReport, package_root: Path) -> str:
try:
return lib.path.relative_to(package_root).as_posix()
except ValueError:
return lib.path.name


def _is_torch_unstable(lib: ExtensionReport) -> bool:
return lib.error is None and lib.torch.uses_torch and not lib.torch.stable


def _matches_allowlist(rel_path: str, patterns: tuple[str, ...]) -> bool:
return any(fnmatch.fnmatch(rel_path, pattern) for pattern in patterns)


def _iter_libs(report: PackageReport) -> tuple[ExtensionReport, ...]:
return (*report.extensions, *report.bundled_libs)


def _collect_unstable(report: PackageReport) -> list[str]:
return sorted(
_relative_path(lib, report.root)
for lib in _iter_libs(report)
if _is_torch_unstable(lib)
)


def _find_stale_allowlist_entries(
report: PackageReport, patterns: tuple[str, ...]
) -> list[str]:
"""Allowlist patterns that match a built library which is no longer unstable."""
stale: list[str] = []
for pattern in patterns:
for lib in _iter_libs(report):
if lib.error is not None:
continue
if not fnmatch.fnmatch(_relative_path(lib, report.root), pattern):
continue
if not _is_torch_unstable(lib):
stale.append(pattern)
break
return stale


def check_torch_abi(
package: str = "vllm",
patterns: tuple[str, ...] = ALLOWED_UNSTABLE_LIBRARIES,
) -> int:
report = inspect_package(package)
if report.error:
print(f"error: failed to inspect {package!r}: {report.error}", file=sys.stderr)
return 2

unstable = _collect_unstable(report)
unexpected = [
rel_path for rel_path in unstable if not _matches_allowlist(rel_path, patterns)
]
stale = _find_stale_allowlist_entries(report, patterns)

if unexpected or stale:
if unexpected:
print(
"Not allowed: torch-unstable libraries outside "
f"ALLOWED_UNSTABLE_LIBRARIES: {', '.join(unexpected)}",
file=sys.stderr,
)
if stale:
print(
"Not allowed: stale ALLOWED_UNSTABLE_LIBRARIES entries: "
f"{', '.join(stale)}",
file=sys.stderr,
)
return 1

print("Torch stable ABI check passed.")
return 0


if __name__ == "__main__":
print(">>> Auditing vLLM extension modules for PyTorch stable ABI compliance")
sys.exit(check_torch_abi())
1 change: 1 addition & 0 deletions .buildkite/ci_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ run_all_patterns:
- "setup.py"
- "csrc/"
- "cmake/"
- ".buildkite/check-torch-abi.py"
run_all_exclude_patterns:
- "docker/Dockerfile."
- "csrc/cpu/"
Expand Down
1 change: 1 addition & 0 deletions .buildkite/ci_config_rocm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ run_all_patterns:
- "docker/docker-bake-rocm.hcl"
- ".buildkite/hardware_tests/amd.yaml"
- ".buildkite/scripts/ci-bake-rocm.sh"
- ".buildkite/scripts/rocm/"
- ".buildkite/scripts/hardware_ci/run-amd-test.py"
- ".buildkite/scripts/hardware_ci/run-amd-test.sh"
- "CMakeLists.txt"
Expand Down
67 changes: 38 additions & 29 deletions .buildkite/hardware_tests/amd.yaml
Original file line number Diff line number Diff line change
@@ -1,25 +1,55 @@
group: Hardware - AMD Build

# ROCm image flow:
# 1. Refresh the long-lived ROCm base image only when Dockerfile.rocm_base changes.
# 2. Build ci_base from either the stable base or the freshly refreshed base.
# 3. Build the per-commit ROCm CI image and smoke-test it before GPU jobs run.
steps:
- label: "AMD: :docker: refresh ROCm base"
key: refresh-rocm-base-amd
depends_on: []
device: amd_cpu
no_plugin: true
commands:
- bash .buildkite/scripts/rocm/refresh-base-image.sh
env:
DOCKER_BUILDKIT: "1"
BUILDKIT_PROGRESS: "tty"
TERM: "xterm-256color"
retry:
automatic:
- exit_status: 1 # Transient Docker/BuildKit failure
limit: 1
- exit_status: -1 # Agent was lost
limit: 1
- exit_status: -10 # Agent was lost
limit: 1

# Ensure ci_base is up-to-date before building the test image.
# Compares a content hash of ci_base-affecting files against the remote
# image label. If hashes match the build is skipped (< 30 s); if they
# differ ci_base is rebuilt and pushed automatically.
- label: "AMD: :docker: ensure ci_base"
key: ensure-ci-base-amd
soft_fail: false
depends_on: []
depends_on:
- refresh-rocm-base-amd
device: amd_cpu
no_plugin: true
commands:
- bash .buildkite/scripts/ci-bake-rocm.sh ci-base-rocm-ci-with-deps
- bash .buildkite/scripts/rocm/build-ci-base.sh
env:
DOCKER_BUILDKIT: "1"
BUILDKIT_PROGRESS: "tty"
TERM: "xterm-256color"
VLLM_BAKE_FILE: "docker/docker-bake-rocm.hcl"
PYTORCH_ROCM_ARCH: "gfx90a;gfx942;gfx950"
REMOTE_VLLM: "1"
VLLM_BRANCH: "$BUILDKITE_COMMIT"
retry:
automatic:
- exit_status: 1 # Transient Docker/BuildKit failure
limit: 1
- exit_status: -1 # Agent was lost
limit: 1
- exit_status: -10 # Agent was lost
Expand All @@ -33,42 +63,21 @@ steps:
device: amd_cpu
no_plugin: true
commands:
- |
if [[ "${ROCM_CI_ARTIFACT_ONLY:-0}" == "1" ]]; then
echo "ROCM_CI_ARTIFACT_ONLY=1; building ROCm wheel artifact only"
IMAGE_TAG="" bash .buildkite/scripts/ci-bake-rocm.sh test-rocm-ci-with-artifacts
else
bash .buildkite/scripts/ci-bake-rocm.sh test-rocm-ci-with-wheel
fi
- |
docker run --rm --network=none --entrypoint /bin/bash "rocm/vllm-ci:${BUILDKITE_COMMIT}" -ec '
if [ ! -d /vllm-workspace ]; then echo Missing directory: /vllm-workspace >&2; exit 1; fi
if [ ! -d /vllm-workspace/tests ]; then echo Missing directory: /vllm-workspace/tests >&2; exit 1; fi
if [ ! -d /vllm-workspace/src/vllm ]; then echo Missing directory: /vllm-workspace/src/vllm >&2; exit 1; fi
if [ ! -x /vllm-workspace/src/vllm/vllm-rs ]; then echo Missing executable: /vllm-workspace/src/vllm/vllm-rs >&2; exit 1; fi
command -v python3
command -v uv
command -v pytest
if ! command -v amd-smi >/dev/null 2>&1 && ! command -v rocminfo >/dev/null 2>&1; then
echo No ROCm CLI found in image >&2
exit 1
fi
python3 - <<PY
import torch, vllm
print(torch.__version__)
print(vllm.__version__)
PY
echo AMD image smoke OK
'
- bash .buildkite/scripts/rocm/build-test-image.sh
- bash .buildkite/scripts/rocm/smoke-test-image.sh
env:
DOCKER_BUILDKIT: "1"
BUILDKIT_PROGRESS: "tty"
TERM: "xterm-256color"
VLLM_BAKE_FILE: "docker/docker-bake-rocm.hcl"
PYTORCH_ROCM_ARCH: "gfx90a;gfx942;gfx950"
IMAGE_TAG: "rocm/vllm-ci:$BUILDKITE_COMMIT"
REMOTE_VLLM: "1"
VLLM_BRANCH: "$BUILDKITE_COMMIT"
retry:
automatic:
- exit_status: 1 # Transient Docker/BuildKit failure
limit: 1
- exit_status: -1 # Agent was lost
limit: 1
- exit_status: -10 # Agent was lost
Expand Down
21 changes: 19 additions & 2 deletions .buildkite/hardware_tests/cpu.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ steps:
- tests/kernels/quantization/test_cpu_fp8_scaled_mm.py
- tests/kernels/mamba/cpu/test_cpu_gdn_ops.py
- tests/kernels/mamba/test_cpu_short_conv.py
- tests/kernels/mamba/test_causal_conv1d.py
- tests/kernels/mamba/test_mamba_ssm.py
commands:
- |
bash .buildkite/scripts/hardware_ci/run-cpu-test.sh 30m "
Expand All @@ -28,7 +30,9 @@ steps:
pytest -x -v -s tests/kernels/test_onednn.py
pytest -x -v -s tests/kernels/test_awq_int4_to_int8.py
pytest -x -v -s tests/kernels/quantization/test_cpu_fp8_scaled_mm.py
pytest -x -v -s tests/kernels/mamba/cpu/test_cpu_gdn_ops.py"
pytest -x -v -s tests/kernels/mamba/cpu/test_cpu_gdn_ops.py
pytest -x -v -s tests/kernels/mamba/test_causal_conv1d.py
pytest -x -v -s tests/kernels/mamba/test_mamba_ssm.py"

# Note: SDE can't be downloaded from CI host because of AWS WAF
# - label: CPU-Compatibility Tests
Expand Down Expand Up @@ -141,9 +145,22 @@ steps:
commands:
- |
bash .buildkite/scripts/hardware_ci/run-cpu-test.sh 45m "
pytest -x -v -s tests/models/multimodal/generation --ignore=tests/models/multimodal/generation/test_pixtral.py -m cpu_model --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
pytest -x -v -s tests/models/multimodal/generation --ignore=tests/models/multimodal/generation/test_pixtral.py --ignore=tests/models/multimodal/generation/test_qwen2_5_vl.py -m cpu_model --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
parallelism: 4

- label: CPU-Qwen2.5-VL Multimodal Tests
depends_on: []
device: intel_cpu
no_plugin: true
source_file_dependencies:
# - vllm/
- vllm/model_executor/layers/rotary_embedding
- tests/models/multimodal/generation/
commands:
- |
bash .buildkite/scripts/hardware_ci/run-cpu-test.sh 40m "
VLLM_CI_ENV=0 pytest -x -v -s tests/models/multimodal/generation/test_qwen2_5_vl.py"

- label: "Arm CPU Test"
depends_on: []
soft_fail: false
Expand Down
6 changes: 3 additions & 3 deletions .buildkite/hardware_tests/intel_xpu_ci/test-intel.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ steps:
- label: "XPU example Test"
depends_on:
- image-build-xpu
timeout_in_minutes: 30
timeout_in_minutes: 50
optional: true
device: intel_gpu
agent_tags:
Expand All @@ -39,7 +39,7 @@ steps:
- label: "XPU V1 test"
depends_on:
- image-build-xpu
timeout_in_minutes: 30
timeout_in_minutes: 70
optional: true
device: intel_gpu
agent_tags:
Expand All @@ -60,7 +60,7 @@ steps:
- label: "XPU server test"
depends_on:
- image-build-xpu
timeout_in_minutes: 30
timeout_in_minutes: 45
optional: true
device: intel_gpu
agent_tags:
Expand Down
20 changes: 20 additions & 0 deletions .buildkite/image_build/image_build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -79,12 +79,18 @@ setup_buildx_builder() {
docker buildx ls | grep -E '^\*|^NAME' || docker buildx ls
}

annotate_image_tags() {
.buildkite/scripts/annotate-image-build.sh \
"${IMAGE_TAG:-}" "${IMAGE_TAG_LATEST:-}"
}

check_and_skip_if_image_exists() {
if [[ -n "${IMAGE_TAG:-}" ]]; then
echo "--- :mag: Checking if image exists"
if docker manifest inspect "${IMAGE_TAG}" >/dev/null 2>&1; then
echo "Image already exists: ${IMAGE_TAG}"
echo "Skipping build"
annotate_image_tags
exit 0
fi
echo "Image not found, proceeding with build"
Expand Down Expand Up @@ -171,6 +177,18 @@ BRANCH=$4
IMAGE_TAG=$5
IMAGE_TAG_LATEST=${6:-} # only used for main branch, optional

# When TORCH_NIGHTLY=1, build the base CI image against PyTorch nightly so the
# entire existing pipeline runs on nightly torch (CUDA/GPU lane only). Delegate
# to the dedicated nightly build (PYTORCH_NIGHTLY=1, CUDA 13.0) and tag it at the
# normal IMAGE_TAG that every test step already pulls -- no separate image tag,
# no duplicate "vLLM Against PyTorch Nightly" pipeline section.
if [[ "${TORCH_NIGHTLY:-0}" == "1" ]]; then
echo "--- :warning: TORCH_NIGHTLY=1 -- building base image on PyTorch nightly"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "${SCRIPT_DIR}/image_build_torch_nightly.sh" \
"${REGISTRY}" "${REPO}" "${BUILDKITE_COMMIT}" "${BRANCH}" "${IMAGE_TAG}"
fi

# build config
TARGET="test-ci"
VLLM_BAKE_FILE_PATH="${VLLM_BAKE_FILE_PATH:-docker/docker-bake.hcl}"
Expand Down Expand Up @@ -254,3 +272,5 @@ echo "--- :docker: Building ${TARGET}"
docker --debug buildx bake -f "${VLLM_BAKE_FILE_PATH}" -f "${CI_HCL_PATH}" --progress plain "${TARGET}"

echo "--- :white_check_mark: Build complete"

annotate_image_tags
39 changes: 20 additions & 19 deletions .buildkite/image_build/image_build_arm64.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,31 @@ fi
REGISTRY=$1
REPO=$2
BUILDKITE_COMMIT=$3
IMAGE="$REGISTRY/$REPO:$BUILDKITE_COMMIT-arm64"

# authenticate with AWS ECR
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin "$REGISTRY" || true

# skip build if image already exists
if [[ -z $(docker manifest inspect "$REGISTRY"/"$REPO":"$BUILDKITE_COMMIT"-arm64) ]]; then
echo "Image not found, proceeding with build..."
else
if docker manifest inspect "$IMAGE" >/dev/null 2>&1; then
echo "Image found"
exit 0
else
echo "Image not found, proceeding with build..."
# build for arm64 GPU targets: Grace/GH200 (sm_90),
# Blackwell/Thor (sm_100/sm_103/sm_110), and DGX Spark/GB10
# (sm_121, family-covered by 12.0 under CUDA 13)
docker build --file docker/Dockerfile \
--platform linux/arm64 \
--build-arg max_jobs=16 \
--build-arg nvcc_threads=4 \
--build-arg torch_cuda_arch_list="9.0 10.0 11.0 12.0" \
--build-arg USE_SCCACHE=1 \
--build-arg buildkite_commit="$BUILDKITE_COMMIT" \
--tag "$IMAGE" \
--target test \
--progress plain .
# push
docker push "$IMAGE"
fi

# build for arm64 GPU targets: Grace/GH200 (sm_90) and DGX Spark/GB10
# (sm_121, family-covered by 12.0 under CUDA 13)
docker build --file docker/Dockerfile \
--platform linux/arm64 \
--build-arg max_jobs=16 \
--build-arg nvcc_threads=4 \
--build-arg torch_cuda_arch_list="9.0 12.0" \
--build-arg USE_SCCACHE=1 \
--build-arg buildkite_commit="$BUILDKITE_COMMIT" \
--tag "$REGISTRY"/"$REPO":"$BUILDKITE_COMMIT"-arm64 \
--target test \
--progress plain .

# push
docker push "$REGISTRY"/"$REPO":"$BUILDKITE_COMMIT"-arm64
.buildkite/scripts/annotate-image-build.sh "$IMAGE"
Loading
Loading