[qli88]

Fuse HY4's FP8 shared expert into the routed MXFP8 MoE kernels to reduce launch and scheduling overhead, with AOT coverage, correctness guards, and reproducible gfx950 tuning.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6050 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[Copilot]

Copilot review overview

🟡 Changes recommended

The tuner can modify mismatched configuration rows, accept non-finite baselines, and one HY4 reference path disagrees with runtime clamping.

Review effort: Balanced
Findings: 3

Open (3)

• Align unclamped reference selection with routed weight dtype · New
• Match all fixed HY4 contract fields when selecting merged rows · New
• Fail when baseline output is non-finite · New

What changed in this PR

Adds gfx950 HY4 MXFP8-routed/FP8-shared heterogeneous MoE fusion, tuning, AOT coverage, and validation.

Changes:

• Extends FlyDSL FHMoE kernels and dispatch to MXFP8 routed weights.
• Adds HY4 tuned configurations and an offline tuner.
• Expands correctness, AOT, pipeline, and CI coverage.

File	Description
.github/​scripts/​split_tests.sh	Adds tuner tests to CI splitting.
aiter/​aot/​flydsl/​fhmoe.py	Enables MXFP8 FHMoE AOT compilation.
aiter/​configs/​tuned_fhmoe.csv	Adds HY4 tuning rows.
aiter/​fhmoe.py	Adds HY4 validation and dispatch.
aiter/​ops/​flydsl/​fhmoe.py	Exposes MXFP8 and shared-clamp controls.
aiter/​ops/​flydsl/​kernels/​fhmoe.py	Forwards shared-clamp specialization.
aiter/​ops/​flydsl/​kernels/​mixed_moe_gemm_2stage.py	Updates module terminology.
aiter/​ops/​flydsl/​kernels/​mixed_moe_gemm_2stage_common.py	Implements MXFP8 heterogeneous kernel support.
op_tests/​test_fhmoe.py	Adds HY4 correctness and AOT tests.
op_tests/​tuners/​tune_fhmoe.py	Adds the FHMoE tuner.
op_tests/​tuning_tests/​README.md	Documents FHMoE tuning.
op_tests/​tuning_tests/​test_config_shape_collision.py	Validates merged HY4 rows.
op_tests/​tuning_tests/​test_fhmoe_tuner.py	Tests tuner utilities.
op_tests/​tuning_tests/​test_tune_pipeline.py	Adds FHMoE pipeline smoke coverage.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Production prefill and reduce-mode configurations lack oracle-backed numerical coverage.

Review effort: Balanced
Findings: 1

Open (1)

• Add oracle coverage for HY4 prefill and reduce-mode paths · New

Resolved since last review (3)

• Fail when baseline output is non-finite
• Match all fixed HY4 contract fields when selecting merged rows
• Align unclamped reference selection with routed weight dtype

[Copilot]

Copilot review overview

🟡 Changes recommended

The tuner crashes between token rows, and the AOT helper rejects its own default CU-count sentinel.

Review effort: Balanced
Findings: 1 · 2

Open (3)

• Use supported cache-clearing API for ROCm tuning loop · New
• Preserve cu_num=0 default sentinel in architecture mapping · New
• Add oracle coverage for HY4 prefill and reduce-mode paths

[Copilot]

Copilot review overview

🔵 Needs a closer look

Unresolved split-K clamping and architecture-specific numerical changes require human review with gfx950 correctness and matched performance validation.

Review effort: Balanced
Findings: 1

Open (1)

• Split-K stage-1 ignores clamp_shared=False for shared experts · New

Resolved since last review (3)

• Use supported cache-clearing API for ROCm tuning loop
• Preserve cu_num=0 default sentinel in architecture mapping
• Add oracle coverage for HY4 prefill and reduce-mode paths

[Copilot]

🔵 Needs a closer look

The extensive GPU kernel, dispatch, tuning, and AOT changes require final human review with matched gfx950 correctness and performance validation.

0 open findings

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

AOT generation drops the unclamped specialization and omits ordinary MoE debug variants.

2 open findings

• Forward clamp_shared when precompiling stage-1 kernels · New
• Generate AOT variants for ordinary FlyDSL MoE stage-2 kernels · New

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The pipeline architecture gate uses the build override rather than live hardware, making gfx950 smoke coverage unreliable.

1 open finding

• Use runtime GPU detection for hardware-dependent test gate · New

2 resolved since last review

• Generate AOT variants for ordinary FlyDSL MoE stage-2 kernels
• Forward clamp_shared when precompiling stage-1 kernels

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Architecture-detection failures can silently skip the new end-to-end FHMoE tuner coverage on gfx950 CI.

1 open finding

• Avoid masking architecture detection failures as skipped tests · New

1 resolved since last review

• Use runtime GPU detection for hardware-dependent test gate

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Forced-reduce handling can corrupt a16w output, and large HY4 reduce buffers exceed 32-bit kernel addressing.

2 open findings

• 32-bit reduce offsets overflow temporary buffers beyond 4 GiB · New
• Force-reduce override corrupts BF16 MXFP4/INT4 a16w output · New

1 resolved since last review

• Avoid masking architecture detection failures as skipped tests

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Unrelated Triton cleanup and DSV4 retuning violate the repository’s one-backend, one-concern PR policy.

3 open findings

• Move unrelated DSV4 retuning to a follow-up PR · New
• Remove or enforce inaccurate 2048 limit in docstring · New
• Move unrelated CPU-only cleanup to a follow-up PR · New

2 resolved since last review

• Force-reduce override corrupts BF16 MXFP4/INT4 a16w output
• 32-bit reduce offsets overflow temporary buffers beyond 4 GiB

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The unrelated DSV4 retuning and Triton test cleanup should be split into follow-up pull requests.

3 open findings

• Move unrelated CPU-only cleanup to a follow-up PR
• Remove or enforce inaccurate 2048 limit in docstring
• Move unrelated DSV4 retuning to a follow-up PR

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The unrelated Triton rope test change violates the repository’s one-backend and one-concern PR rule.

0 open findings

3 resolved since last review

• Move unrelated CPU-only cleanup to a follow-up PR
• Remove or enforce inaccurate 2048 limit in docstring
• Move unrelated DSV4 retuning to a follow-up PR

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Existing partial output files can silently truncate the tuned configuration, and an unrelated Triton test should be split out.

1 open finding

• Validate complete row set before merging tuned output · New

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The GPU kernel and dispatch changes require final gfx950 correctness and performance validation.

1 open finding

• Incorrectly claims output committed on baseline-only failure · New

1 resolved since last review

• Validate complete row set before merging tuned output

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The low-level GPU kernel, dispatch, tuning, and AOT changes require final human review with head-matched gfx950 correctness and performance validation.

0 open findings

1 resolved since last review

• Incorrectly claims output committed on baseline-only failure

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The route-buffer size guard rejects valid packed-FP8 reduction shapes using an uncompressed BF16/FP16 size calculation.

1 open finding

• Use packed FP8 byte count for stage2 buffer size validation · New

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The new workspace preflight incorrectly skips non-common reducers that are not subject to the FlyDSL buffer limit.

2 open findings

• Gate route output size preflight on FlyDSL stage classification · New
• Use packed FP8 byte count for stage2 buffer size validation

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Quick tuning does not honor its XCD exclusion, and workspace preflight miscalculates FP8-reduce storage.

2 open findings

• Preflight miscalculates temporary size for FP8 reduce mode · New
• Quick mode fails to filter XCD-swizzle candidates · New

2 resolved since last review

• Gate route output size preflight on FlyDSL stage classification
• Use packed FP8 byte count for stage2 buffer size validation

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The capability API can report support outside its gfx950-only runtime contract, leading callers into a path that later rejects execution.

1 open finding

• Gate FHMoE capability checks to gfx950 runtime · New

2 resolved since last review

• Quick mode fails to filter XCD-swizzle candidates
• Preflight miscalculates temporary size for FP8 reduce mode

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The broad GPU-kernel, AOT, dispatch, and tuning changes require head-matched gfx950 correctness and performance validation.

0 open findings

1 resolved since last review

• Gate FHMoE capability checks to gfx950 runtime

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

GPU kernel correctness, numerical behavior, AOT cache compatibility, and tuned performance require final gfx950 hardware validation.

0 open findings

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Shared dispatch, GPU kernel, and compilation changes need human confirmation backed by matched gfx950 correctness, cache-retrieval, and performance evidence.

0 open findings

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Two AOT coverage gaps leave supported FP8 stage-2 runtime variants without matching compiled artifacts.

2 open findings

• Stage-2 AOT manifest ignores FP8 accumulation and output variants · New
• Manifest omits environment-selected FP8 GEMM and reduction artifacts · New

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Shared kernel, dispatch, and compilation changes need human review backed by gfx950 correctness, cache-retrieval, and matched performance validation.

0 open findings

2 resolved since last review

• Manifest omits environment-selected FP8 GEMM and reduction artifacts
• Stage-2 AOT manifest ignores FP8 accumulation and output variants

🧠 Review effort: Balanced