move all configs by Boss2002n · Pull Request #4379 · ROCm/aiter

Boss2002n · 2026-07-24T20:19:04Z

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

github-actions · 2026-07-24T20:25:46Z

🏷️ CI Guide

Runs automatically on every PR:

✅ Pre-checks (submodule verification, code formatting)
✅ Aiter op tests (gfx942 + gfx950)
✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
`ci:triton-300x`	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
`ci:sglang`	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
`ci:atom`	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
`ci:atom_full`	ATOM accuracy suite for PR and main models from ATOM `models_accuracy.json`
`ci:vllm`	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
`ci:all`	All standard extended tests (excludes `ci:atom_full`)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4379 --add-label <label>

Copilot

Pull request overview

This PR migrates Triton GEMM tuning configs to a new per-arch, per-backend directory layout under aiter/ops/triton/configs/<arch>/<backend>/gemm/, updates config resolution to prefer the new layout (with legacy fallback), and adds new AFP4WFP4 tuned configs for gfx950/gfx1250.

Changes:

Update get_gemm_config() resolution to search <arch>/<backend>/gemm/ (prefix-less) first, then fall back to legacy configs/gemm/ (arch-prefixed) files.
Switch Gluon AFP4WFP4 to read its config from the new directory layout, and enable the gfx1250 Gluon preshuffle path in the Triton AFP4WFP4 wrapper.
Add a large set of AFP4WFP4 tuning JSONs under the new gfx950/gfx1250 config directories (plus .gitkeep placeholders for MoE dirs).

Reviewed changes

Copilot reviewed 4 out of 47 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
aiter/ops/triton/utils/gemm_config_utils.py	Prefer new `<arch>/<backend>/gemm/` config layout with legacy fallback for unmigrated configs.
aiter/ops/triton/gluon/gemm_afp4wfp4.py	Load Gluon AFP4WFP4 config from new per-arch/per-backend path.
aiter/ops/triton/gemm/basic/gemm_afp4wfp4.py	Enable gfx1250 Gluon preshuffle path selection logic for AFP4WFP4 preshuffle wrapper.
aiter/ops/triton/configs/gfx950/triton/moe/.gitkeep	Create/keep gfx950 Triton MoE config directory in-tree.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4.json	New gfx950 Triton default AFP4WFP4 tuning config (new layout).
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=9216-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=8192-K=8192.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=8192-K=28672.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=7168-K=256.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=7168-K=2304.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=7168-K=2048.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=7168-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=57344-K=8192.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=53248-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=512-K=7168.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=512-K=128.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=4608-K=7168.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=4608-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=4096-K=7168.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=4096-K=14336.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=36864-K=7168.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=32768-K=512.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=3072-K=1536.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=28672-K=4096.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=26624-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=24576-K=1536.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=2304-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=2112-K=7168.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=18432-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=8192.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=6656.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=53248.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=4096.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=26624.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=2048.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=16384-K=13312.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=13312-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=1280-K=8192.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/triton/gemm/GEMM-AFP4WFP4-N=106496-K=16384.json	New gfx950 Triton N/K-specialized AFP4WFP4 config.
aiter/ops/triton/configs/gfx950/gluon/moe/.gitkeep	Create/keep gfx950 Gluon MoE config directory in-tree.
aiter/ops/triton/configs/gfx950/gluon/gemm/GEMM-AFP4WFP4.json	New gfx950 Gluon AFP4WFP4 tuning config (new layout).
aiter/ops/triton/configs/gfx1250/triton/moe/.gitkeep	Create/keep gfx1250 Triton MoE config directory in-tree.
aiter/ops/triton/configs/gfx1250/triton/gemm/.gitkeep	Create/keep gfx1250 Triton GEMM config directory in-tree.
aiter/ops/triton/configs/gfx1250/gluon/moe/.gitkeep	Create/keep gfx1250 Gluon MoE config directory in-tree.
aiter/ops/triton/configs/gfx1250/gluon/gemm/GEMM-AFP4WFP4.json	New gfx1250 Gluon AFP4WFP4 tuning config (new layout).
aiter/ops/triton/_triton_kernels/gemm/basic/gemm_afp4wfp4.py	Simplify `_get_config()` return paths (but currently drops required default meta-param filling).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

    assert arch_info.is_fp4_avail(), "MXFP4 is not available on your device"
-    use_gluon = False  # arch_info.get_arch() == "gfx1250" TODO: (Satya) revert after upstream triton is fixed
+    use_gluon = arch_info.get_arch() == "gfx1250"

    M, K_bytes = x_fp4.shape
    n16, _ = w_preshuf.shape


    # Note: Config files use K=2*K in their naming
    K = 2 * K
    if shuffle:
-        cfg, is_tuned = get_gemm_config(
+        return get_gemm_config(
            "GEMM-AFP4WFP4_PRESHUFFLED",


move all configs

44f0b76

format

bf85ab2

Boss2002n marked this pull request as ready for review July 24, 2026 20:31

Boss2002n requested review from a team and Copilot July 24, 2026 20:31

Copilot started reviewing on behalf of Boss2002n July 24, 2026 20:33 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

move all configs#4379

move all configs#4379
Boss2002n wants to merge 2 commits into
mainfrom
satya/move_fp4_configs

Boss2002n commented Jul 24, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Boss2002n commented Jul 24, 2026

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

github-actions Bot commented Jul 24, 2026

🏷️ CI Guide

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants