Skip to content

[module_moe_asm] refactor and rm torch#4368

Merged
amd-ruitang3 merged 1 commit into
ROCm:mainfrom
amd-ruitang3:module_moe_asm_refactor
Jul 27, 2026
Merged

[module_moe_asm] refactor and rm torch#4368
amd-ruitang3 merged 1 commit into
ROCm:mainfrom
amd-ruitang3:module_moe_asm_refactor

Conversation

@amd-ruitang3

@amd-ruitang3 amd-ruitang3 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Remove the torch dependency from module_moe_asm's C/HIP sources.
The [module_moe_asm] build time drop from 33.8s to 16.4s (-51.5%)

Compile-time improvement (single-file hipcc, gfx950)

source file before (torch) after (torch-free) saved
topk_softmax_kernels.cu 29.6 s 12.5 s −17.1 s (58%)
topk_softmax_kernels_group.cu 27.1 s 9.8 s −17.3 s (64%)
moe_fused_gate.cu 19.4 s 5.4 s −14.0 s (72%)
moe_align_block_size_kernels.cu 19.3 s 1.8 s −17.5 s (91%)
moe_op_pybind.cu 19.8 s 5.6 s −14.2 s (72%)
total (serial) 115.2 s 35.1 s −80.1 s (70%)

@amd-ruitang3
amd-ruitang3 requested a review from a team July 24, 2026 06:19
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4368 --add-label <label>

@amd-ruitang3
amd-ruitang3 requested a review from irvineoy July 24, 2026 06:36
@amd-ruitang3
amd-ruitang3 force-pushed the module_moe_asm_refactor branch from ff35740 to 15d4a83 Compare July 24, 2026 06:41
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@amd-ruitang3
amd-ruitang3 force-pushed the module_moe_asm_refactor branch from 15d4a83 to 0c332ad Compare July 24, 2026 06:43
@amd-ruitang3
amd-ruitang3 requested a review from zufayu July 24, 2026 06:47
@zufayu zufayu added the ci:all label Jul 24, 2026
@amd-ruitang3
amd-ruitang3 merged commit 0ba802e into ROCm:main Jul 27, 2026
64 of 66 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants