[FlyDSL] [Feature] Precompile only the GEMM kernels the wheel is built for by RElbers · Pull Request #5342 · ROCm/aiter

RElbers · 2026-09-08T14:03:14Z

Stops a released wheel from precompiling FlyDSL GEMM kernels for architectures it is not built for. The packaging path compiled every job in the tuned CSVs regardless of the wheel's targets; it now filters by build target, the way the standalone entry point already did.

Stacked on #5340, which introduces AITER_GPU_TARGETS: build targets written as gfx:cu_num pairs. The filter here resolves through that machinery, so packaging selects AOT jobs the way the rest of the build selects tuned rows.

Motivation

setup.py → run_aot() → _collect_aot_jobs_for() called collect_aot_jobs() with no target filter, so approximately 220 gfx1250 kernels were built into every gfx942/gfx950 wheel, which never reaches them. The same gap shows up on MI350P: FlyDSL dispatch keys on (gfx, cu_num) from the live device, and every tuned FlyDSL row records 256 CUs, so on a 128-CU part every lookup misses, yet a gfx950:128 build was precompiling all of those kernels anyway. It now selects none of them, and once rows are tuned for 128 CUs it will select exactly those.

Scope

GEMM only. Each other OpKind reads its own CSVs and keeps only one of the two fields a target needs: MoE and chunk_gdn_h carry cu_num and recover an arch later through cu_num_to_arch, grouped MoE carries gfx alone, mxfp4 MoE hardcodes FLYDSL_GPU_ARCH to gfx950, and mega MoE builds its jobs without a CSV. Matching those jobs against a (gfx, cu_num) pair would drop every one. Filtering them means widening each parse_csv to carry both fields, plus a gfx column for the mxfp4 model CSVs that lack one, which is out of scope here.

Technical Details

1. Packaging now resolves targets the way CK does; the standalone entry point still does not. CK's codegen resolves GPU_ARCHS and CU_NUM into exact (gfx, cu_num) pairs and keeps only the tuned rows matching them. FlyDSL packaging did none of that, baking every row in the CSVs whatever the build targeted. It now resolves the same pairs, so both halves of a build select for the same SKU.

The standalone entry point is unchanged: it still matches on architecture alone and ignores CU_NUM. Only that divergence is now visible. Under GPU_ARCHS=gfx950 CU_NUM=128 packaging selects nothing, since every FlyDSL GEMM row records 256 CUs, while the CLI selects every gfx950 job. Neither result is wrong on its own, and an AOT miss only falls back to a runtime JIT compile, but one environment now means two different things depending on which entry point reads it.

2. An unknown arch now fails the build. GPU_ARCHS=gfx9999 raises, aborting the wheel build where packaging previously ignored the variable and baked everything. Intended, but it is a hard failure on a free-form workflow_dispatch input. Every in-tree caller passes a valid value and gpu_archs has a default, so no automated path can hit it.

3. The filter reads AITER_GPU_TARGETS and GPU_ARCHS only. ARCH stays on the standalone entry point: build images export it as a machine type such as x86_64, and an unrecognised value raises, so routing packaging through it would kill the wheel build on those hosts.

4. Uppercase now matches where it previously matched nothing. Case-folding means GFX950 selects every gfx950 job where it used to select none. On the packaging path that is a narrowing, since it baked every job in the CSVs before reading no target variable at all; on the standalone entry point it is the opposite, since an unmatched arch there selected nothing and fell back to runtime JIT.

Also in here

A release build gains one warning line naming gfx942, correct since no libtype=flydsl GEMM rows exist for it. parse_csv no longer raises on an empty cu_num cell; it parses as 0, which the filter treats as a wildcard. No shipped CSV trips this today.

Test Plan

op_tests/test_gemm_codegen.py, covering the packaging collector end to end, partial target coverage, legacy rows with no gfx, both separator forms, and each rejected GPU_ARCHS input.

Test Result

Passes; ruff and black clean. An AOT miss is not a failure, it falls back to a runtime JIT compile, so AITER_GPU_TARGETS=gfx950:128 selecting 0 jobs today is safe.

github-actions · 2026-09-08T14:04:05Z

🏷️ CI Guide

Runs automatically on every PR:

✅ Pre-checks (submodule verification, code formatting)
✅ Aiter op tests (gfx942 + gfx950)
✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
`ci:gfx1250-ffm-triton`	Run the five-shard gfx1250 FFM Triton test suite
`ci:triton-300x`	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
`multigpu`	Aiter multi-GPU tests on the 8-GPU runner
`ci:sglang`	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
`ci:atom`	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
`ci:atom_full`	ATOM accuracy suite for PR and main models from ATOM `models_accuracy.json`
`ci:vllm`	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
`ci:all`	All standard extended tests (excludes `ci:atom_full`)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5342 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

samremes · 2026-09-09T11:35:54Z

Review (FlyDSL GEMM AOT consumer / non-comm enablement)

This is the packaging consumer of #5340 for SILOTIGER-978. Runtime lookup already keys FlyDSL GEMM on (gfx, cu_num, …), so a 128-CU process does not pick a 256-CU tuned row; AOT miss falls back to JIT. GEMM-only is the right scope while other OpKinds record half a target. Keep this draft until #5340 lands; rebase first (docs/autotuning_pipeline.md already conflicts with the current #5340 head).

On the second-opinion item (GPU_ARCHS + CU_NUM vs AITER_GPU_TARGETS): use the same resolved (gfx, cu_num) pairs as the rest of the build for packaging. Arch-wide GPU_ARCHS=gfx950 CU_NUM=128 currently keeps all 3,135 gfx950 FlyDSL GEMM jobs (all 256-CU in this tree) and does not warn that exact gfx950:128 coverage is zero. Runtime is still safe; the cost is dead AOT data and a hidden coverage gap. CLI arch-wide filtering can stay a separate mode.

Also:

aiter/aot/flydsl/README.md still says ARCH / GPU_ARCHS are logging only. This PR makes them filter GEMM jobs. Update that README with GEMM-only, precedence, exact vs arch-wide, empty-cu_num wildcard, unknown-arch hard fail, and which op kinds stay unfiltered.
Tests construct {cu_num: 0} and only check GEMM. Add a real empty-cell CSV through parse_csv, and one job each for the unfiltered kinds through _filter_collected_aot_jobs.

I ran the codegen tests (82 function checks / 11 pytest cases). No full PREBUILD_KERNELS=1 wheel.

- Filters the packaging AOT job collector by build target, where previously only the CLI filtered and packaging baked every job in the CSVs. - Selects exact (gfx, cu_num) jobs for AITER_GPU_TARGETS, and keeps GPU_ARCHS and ARCH as arch-wide filters, warning for a requested target with no jobs and retaining its runtime fallback. - Reads both legacy variables through one parser, so each rejects an unknown arch, a separator-only value, and native combined with other entries, and both keep accepting ';' and ',' as they did before. - Keeps a row that records no CU count for every target of its arch, matching how a row that records no arch is inferred from its count. - Resolves the active target variable in one place, so the log line cannot name a variable the filter did not use. - Covers the packaging collector, partial coverage, legacy rows, the rejected inputs, and the separator contract in the codegen tests. The collector filters GEMM only: the other AOT kinds record half a target each, a CU count without an arch or an arch without a count, so a (gfx, cu_num) filter would drop all of them. main() catches the configuration errors, but the collector path raises them, so a caller there should expect a raise rather than a silent pass-through of a junk target env.

- Resolves GPU_ARCHS with CU_NUM into the same (gfx, cu_num) pairs the rest of the build compiles for, so a wheel carries only the jobs its targets name. - Keeps arch-wide selection for the CLI, where naming an arch to cross-compile all of it is the point, and for ARCH on both paths. - Corrects the AOT README, which said these variables were banner output and did not affect what gets built. - Moves the filter tests onto the resolved-pair contract and covers the CLI mode beside it. GPU_ARCHS=gfx950 CU_NUM=128 kept every gfx950 job and reported no gap in gfx950:128 coverage. Runtime dispatch was never at risk, since the lookup keys on (gfx, cu_num) regardless; the cost was dead kernels in the wheel and a coverage hole nothing named.

- Restricts the packaging path to AITER_GPU_TARGETS and GPU_ARCHS, the two variables this filter introduces, so a build host that exports ARCH for its own purposes cannot abort or narrow the wheel. - Leaves ARCH where python -m aiter.aot.flydsl.gemm already had it, selecting arch-wide and rejecting an unrecognised value exactly as before. - Drops the ARCH cases from the AOT filter test, which pinned behaviour this change has no reason to define. - Names main() and the full module command where the docs distinguish the two paths, neither of which said which entry point was meant. Build images and wrapper scripts commonly set ARCH to a machine type such as x86_64, and the filter raises on a value it does not recognise rather than ignoring it. Routing packaging through it would have made setup.py fail at the FlyDSL AOT stage on those hosts, and no in-tree workflow sets ARCH, so CI would not have caught it.

- Black rejects the trailing space.

…ctions added in earlier commits of this branch so CLI and packaging rules are easier to follow.

…cases.

…ydsl-aot # Conflicts: # aiter/aot/flydsl/README.md # aiter/aot/flydsl/gemm.py

Copilot

Copilot review overview

🔵 Needs a closer look

Unresolved critical cache-write and CLI failures, moderate configuration and MoE issues, and broad cross-backend changes require human review.

Review effort: Balanced
Findings: 1 · 1

Open (2)

Make QKVPACKED CLI defaults use equal sequence lengths · New
Move gfx950 config into the required nested path · New

What changed in this PR

This PR aims to precompile FlyDSL GEMM kernels only for the GPU targets of a wheel. The supplied changes also span other kernel backends, tuning data, tests, CI, and review tooling.

Changes:

Filters FlyDSL GEMM AOT jobs by build target.
Updates GEMM, attention, MoE, MLA, quantization, and distributed-communication code and tests.
Adds tuned configurations and revises development workflows.

File	Review summary
`README.md`	Updates guidance; nit (1 vote): split unrelated backend changes.
`op_tests/tuning_tests/test_tune_pipeline.py`	Adds mixed-MXFP tuning cases.
`op_tests/tuning_tests/test_run_config.py`	Adds mixed-MXFP dispatch checks.
`op_tests/tuning_tests/test_mp_tuner_logic.py`	Tests worker-exit tracking.
`op_tests/tuning_tests/test_csv_validation.py`	Covers new tuning CSVs.
`op_tests/tuning_tests/test_a6w6_tuning.py`	Tests compiled kernel selection.
`op_tests/triton_tests/moe/test_moe_gemm_a16w4.py`	Adds MoE test shapes.
`op_tests/triton_tests/attention/test_unified_attention.py`	Passes additional attention parameters.
`op_tests/test_rmsnorm2dFusedAddQuant.py`	Extends quantization shape coverage.
`op_tests/test_opus_a8w8_interface.py`	Updates scale-block fixtures.
`op_tests/test_mla_prefill_ps.py`	Supplies total query length.
`op_tests/test_mla_persistent.py`	Adds an MLA test case.
`op_tests/test_mha_varlen.py`	Adds a layout choice; critical (1 vote): `QKVPACKED` fails with default unequal sequence lengths.
`op_tests/test_gemm_a8w8_blockscale.py`	Strengthens split-K checks.
`op_tests/op_benchmarks/triton/bench_gemm_afp4wfp4.py`	Changes benchmark invocation.
`op_tests/multigpu_tests/test_reduce_scatter.py`	Adds teardown synchronization; nit (1 vote): share the copied barrier helper.
`op_tests/multigpu_tests/test_quick_all_reduce.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_parallel_groups.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_fused_qknorm_ar.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_fused_ar_rms.py`	Adds teardown barriers.
`op_tests/multigpu_tests/test_fused_ar_rms_per_group_quant.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_fused_ar_rms_mxfp4_quant.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_fused_ar_rms_memory_order.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_fused_ar_mhc_post_only.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_custom_group.py`	Adds teardown barriers.
`op_tests/multigpu_tests/test_custom_allreduce_fp8.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_collective_profile.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_car_rccl_latency.py`	Adds a teardown barrier.
`op_tests/multigpu_tests/test_allgather.py`	Adds teardown barriers.
`hsa/gfx950/fmha_v3_fwd/fmha_fwd.csv`	Registers forward attention kernels.
`hsa/gfx950/fmha_v3_bwd/fmha_bwd_odo.csv`	Registers a backward attention kernel.
`hsa/gfx950/fmha_v3_bwd/fmha_bwd_dqdkdv.csv`	Registers a backward gradient kernel.
`hsa/gfx950/f6f4gemm/f6f4gemm_bf16_per1x32Fp6Fp4.csv`	Adds a mixed-GEMM manifest.
`hsa/gfx950/f4f6gemm/f4f6gemm_bf16_per1x32Fp4Fp6.csv`	Adds a mixed-GEMM manifest.
`hsa/gfx1250/mxfp8fp4gemm/mxfp8fp4gemm.csv`	Registers GEMM variants.
`csrc/pybind/pa_mqa_logits_mxfp4_gfx1250_pybind.cu`	Removes the former binding location.
`csrc/pybind/mha_v4_quant_pybind.cu`	Extends quantization bindings.
`csrc/pybind/mha_v4_fwd_pybind.cu`	Adds optional forward arguments.
`csrc/py_itfs_cu/asm_mla.cu`	Adds an MLA dispatch case.
`csrc/opus_moe/README.md`	Documents the route-output switch.
`csrc/opus_moe/include/opus_moe_host_impl.cuh`	Selects the route-output format.
`csrc/kernels/rmsnorm_quant_kernels.cu`	Changes grouped-quantization dispatch.
`csrc/kernels/opus_mqa_logits/pa_mqa_logits_mxfp4/pa_mqa_logits_mxfp4_pybind.cu`	Adds the relocated binding.
`csrc/kernels/mxfp4_moe/moe_aux/codegen/gen_instances.py`	Adds MoE shapes.
`csrc/kernels/mla/reduce.cu`	Adds an MLA reduction case.
`csrc/kernels/mla/opus/mla_decode_traits.h`	Adds a wave-layout trait.
`csrc/kernels/mla/opus_mla_decode_fwd.cu`	Dispatches the wave-layout variant.
`csrc/kernels/mla/metadata/v1_comm.cuh`	Adds a planner parameter.
`csrc/kernels/activation_kernels.cu`	Implements ReLU².
`csrc/include/torch/mha_v4_fwd.h`	Declares optional forward arguments.
`csrc/include/quant.h`	Declares an output-buffer quantization API.
`csrc/include/mha_v4_quant.h`	Extends quantization declarations.
`csrc/include/inverse_rope_group_quant.h`	Clarifies input layout.
`csrc/include/activation.h`	Declares ReLU².
`csrc/gemm_mixed_mxfp/README.md`	Documents mixed-GEMM tuning.
`csrc/gemm_a6w4/gemm_a6w4_tune.py`	Adds an A6W4 tuner entry point.
`csrc/gemm_a4w6/gemm_a4w6_tune.py`	Adds an A4W6 tuner entry point.
`csrc/cpp_itfs/mha_fwd.cu`	Extends attention dispatch.
`csrc/cpp_itfs/mha_bwd.cu`	Extends backward attention dispatch.
`csrc/ck_gemm_a8w8_blockscale/include/gemm_a8w8_blockscale_cktile_common.cuh`	Uses a per-split loop count.
`CONTRIBUTE.md`	Documents backend-scoped PRs.
`aiter/ops/triton/utils/unified_attention_utils.py`	Adjusts attention segment selection.
`aiter/ops/triton/utils/mhc_config_utils.py`	Adds fused-MHC configuration lookup.
`aiter/ops/triton/utils/_triton/tunning/ut_template.py`	Removes the old tuning template.
`aiter/ops/triton/utils/_triton/tunning/ut_batched_gemm_a16w16.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_afp8wfp8_gemm_preshuffle.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_afp4wfp4_gemm.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_afp4wfp4_gemm_preshuffle.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_afp4wfp4_gemm_pre_quant_atomic.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a8wfp4_gemm.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a8w8_gemm.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a8w8_gemm_per_token_scale.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a8w8_gemm_blockscale.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a8w8_gemm_blockscale_preshuffle.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a16wfp4_gemm.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a16w8_gemm_blockscale.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a16w8_gemm_blockscale_preshuffle.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a16w16_gemm.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a16w16_gemm_gated.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tunning/ut_a16w16_gemm_atomic.py`	Removes an old tuning harness.
`aiter/ops/triton/utils/_triton/tuning/verify_configs.py`	Uses the unified tuning harness.
`aiter/ops/triton/utils/_triton/tuning/.gitignore`	Ignores tuning artifacts.
`aiter/ops/triton/README.md`	Updates tuning-directory references.
`aiter/ops/triton/quant/__init__.py`	Exports quantization helpers.
`aiter/ops/triton/moe/moe_utils.py`	Adds expert-block conversion; moderate (2 votes): avoid a per-call GPU readback that breaks graph capture.
`aiter/ops/triton/gemm/basic/gemm_afp4wfp4.py`	Allows the gfx1151 Triton path.
`aiter/ops/triton/gated_delta_net/__init__.py`	Exports a prefill operation.
`aiter/ops/triton/fusions/__init__.py`	Exports a fused MHC operation.
`aiter/ops/triton/configs/gfx950/triton/moe/mxfp8_fnuz/DEFAULT.json`	Adds MoE tuning; moderate (1 vote): select the fnuz branch only for fnuz inputs before enabling the gfx950 OCP path.
`aiter/ops/triton/configs/gfx950/triton/gemm/gemm_afp8wfp8/DEFAULT.json`	Extends GEMM defaults.
`aiter/ops/triton/configs/gfx950/triton/gemm/gemm_afp8wfp8_a32_w32x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx950/triton/gemm/gemm_afp8wfp8_a32_w1x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx950/triton/gemm/gemm_afp8wfp8_a32_w128x128/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx950/triton/gemm/gemm_afp8wfp8_a128_w32x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx950/triton/gemm/gemm_afp8wfp8_a128_w1x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx950-KDA_DECODE-DEFAULT.json`	Adds KDA tuning; moderate (3 votes): move it to the required nested layout and update its lookup.
`aiter/ops/triton/configs/gfx942/triton/moe/mxfp8_fnuz/DEFAULT.json`	Adds MoE tuning.
`aiter/ops/triton/configs/gfx942/triton/gemm/gemm_afp8wfp8/DEFAULT.json`	Extends GEMM defaults.
`aiter/ops/triton/configs/gfx942/triton/gemm/gemm_afp8wfp8_a32_w32x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx942/triton/gemm/gemm_afp8wfp8_a32_w1x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx942/triton/gemm/gemm_afp8wfp8_a32_w128x128/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx942/triton/gemm/gemm_afp8wfp8_a128_w32x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx942/triton/gemm/gemm_afp8wfp8_a128_w1x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx1250/triton/gemm/gemm_afp8wfp8/DEFAULT.json`	Extends GEMM defaults.
`aiter/ops/triton/configs/gfx1250/triton/gemm/gemm_afp8wfp8_a32_w32x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx1250/triton/gemm/gemm_afp8wfp8_a32_w1x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx1250/triton/gemm/gemm_afp8wfp8_a32_w128x128/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx1250/triton/gemm/gemm_afp8wfp8_a128_w32x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx1250/triton/gemm/gemm_afp8wfp8_a128_w1x32/DEFAULT.json`	Adds a GEMM variant default.
`aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a16w16/GEMM-A16W16-N=24576-K=1536.json`	Adds specialized GEMM tuning.
`aiter/ops/triton/configs/gfx1250-KDA_DECODE-DEFAULT.json`	Adds KDA tuning; moderate (3 votes): move it to the required nested layout and update its lookup.
`aiter/ops/triton/configs/gfx1151/triton/gemm/gemm_afp4wfp4/DEFAULT.json`	Adds gfx1151 GEMM tuning.
`aiter/ops/triton/configs/gfx1151/triton/attention/unified_attention/DEFAULT.json`	Retunes attention cases.
`aiter/ops/triton/configs/CLAUDE.md`	Documents configuration bounds.
`aiter/ops/triton/attention/pa_mqa_logits.py`	Adjusts strides and page checks.
`aiter/ops/triton/_triton_kernels/quant/sage_attention_quant.py`	Floors zero-channel scales.
`aiter/ops/triton/_triton_kernels/moe/moe_routing/routing.py`	Changes descriptor rank.
`aiter/ops/triton/_triton_kernels/gemm/batched/batched_gemm_afp4wfp4.py`	Removes grid-size specialization.
`aiter/ops/triton/_triton_kernels/gemm/batched/batched_gemm_a16wfp4.py`	Removes grid-size specialization.
`aiter/ops/triton/_triton_kernels/fusions/__init__.py`	Exports fused MHC kernels.
`aiter/ops/triton/_triton_kernels/chunk_delta_attn/chunk_fwd.py`	Forwards paged-state arguments.
`aiter/ops/triton/_triton_kernels/chunk_delta_attn/chunk_delta_attn_utils.py`	Extends the contiguity guard; critical (2 votes): positional skipped tensors can be copied, losing writes to the caller’s cache.
`aiter/ops/triton/_gluon_kernels/gfx1250/moe/moe_op_gemm_a4w4.py`	Changes a quantization import.
`aiter/ops/triton/_gluon_kernels/common/utils.py`	Adds device math helpers.
`aiter/ops/opus/policy.py`	Keys tuning by scale block.
`aiter/ops/opus/__init__.py`	Removes package-level exports.
`aiter/ops/moe_mxfp4_aux.py`	Adds MoE shapes.
`aiter/ops/mha.py`	Extends attention dispatch.
`aiter/ops/gemm_op_common.py`	Adds GEMM lookup and scale helpers.
`aiter/ops/flydsl/kernels/tensor_shim.py`	Refactors compilation and validation helpers.
`aiter/ops/flydsl/kernels/mxfp4_gemm_common.py`	Adjusts activation clamping.
`aiter/ops/flydsl/kernels/moe_2stage_a16wmix/gemm2.py`	Changes the atomic-add path.
`aiter/ops/flydsl/kernels/mega_moe_gfx1250/types.py`	Adds combine-quantization metadata.
`aiter/ops/flydsl/kernels/fused_compress_attn.py`	Passes the resolved quantization mode.
`aiter/ops/flydsl/kernels/flydsl_dispatch_combine_intranode_op.py`	Clarifies quantization naming.
`aiter/ops/flydsl/kernels/conv/__init__.py`	Adds a convolution package.
`aiter/ops/flydsl/grouped_gemm_mxfp4.py`	Passes combine-quantization settings.
`aiter/ops/flydsl/fp8_mqa_logits_kernels.py`	Retunes variant selection.
`aiter/ops/flydsl/batched_gemm_a8w8_gfx1250.py`	Reuses scale validation.
`aiter/ops/flydsl/__init__.py`	Exports a convolution entry point.
`aiter/ops/activation.py`	Exposes ReLU².
`aiter/mla.py`	Extends MLA dispatch.
`aiter/dist/parallel_state.py`	Closes a broadcaster during teardown.
`aiter/dist/device_communicators/custom_all_reduce.py`	Adds deterministic buffer cleanup.
`aiter/dist/device_communicators/communicator_cuda.py`	Restricts cleanup to owners.
`aiter/configs/mxfp8fp4_asm_untuned_gemm.csv`	Adds a tuning input schema.
`aiter/configs/mxfp8fp4_asm_tuned_gemm.csv`	Adds tuned GEMM rows.
`aiter/configs/model_configs/mimo_v2_5_pro_untuned_fmoe.csv`	Adds model MoE shapes.
`aiter/configs/bf16_untuned_conv3d.csv`	Adds a convolution input schema.
`aiter/configs/bf16_tuned_conv3d.csv`	Adds a convolution output schema.
`aiter/configs/a8w8_blockscale_mxscale_bpreshuffle_untuned_gemm.csv`	Adds a GEMM input schema.
`aiter/configs/a8w8_blockscale_mxscale_bpreshuffle_tuned_gemm.csv`	Adds a tuned-row schema.
`aiter/configs/a8w8_blockscale_group32_untuned_gemm.csv`	Adds a group-32 input schema.
`aiter/configs/a8w8_blockscale_group32_tuned_gemm.csv`	Adds a group-32 output schema.
`aiter/configs/a6w4_asm_untuned_gemm.csv`	Adds A6W4 tuning shapes.
`aiter/configs/a4w6_asm_untuned_gemm.csv`	Adds A4W6 tuning shapes.
`aiter/aot/flydsl/mxfp4_moe.py`	Adjusts a compiled activation limit; nit (1 vote): move this unrelated MoE change to a separate PR.
`aiter/__init__.py`	Exports mixed-GEMM operations.
`.github/workflows/tuning-tests.yaml`	Expands tuning CI.
`.github/workflows/pr-welcome-comment.yaml`	Updates PR guidance.
`.github/workflows/pr-title-tags.yaml`	Keeps labels synced on title opt-out.
`.github/scripts/split_tests.sh`	Adds tuning tests to shards.
`.github/scripts/aiter_test.sh`	Changes test invocations.
`.github/instructions/aiter-ops-triton.instructions.md`	Adds backend-scope guidance; nit (1 vote): split this PR’s unrelated backend changes.
`.github/actionlint.yaml`	Registers a runner label.
`.claude/skills/review-pr/RUNNER-SETUP.md`	Documents the review runner.
`.claude/skills/review-pr/render.sh`	Adds prompt rendering.
`.claude/skills/review-pr/prompts/README.md`	Documents prompt templates.
`.claude/skills/review-pr/fetch.sh`	Uses a configurable scratch path.
`.claude/skills/review-pr/authorized.txt`	Adds a review-trigger allowlist.
`.claude/skills/review-pr/_gates.py`	Adds a gate runner.
`.claude/skills/review-pr/_apply_refutation.py`	Adds a finding-filter utility.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…solution' into cu-aware-targets/flydsl-aot

RElbers · 2026-09-29T14:36:43Z

I merged main into this branch instead of merging the parent branch, so the stacked pr pulled commits from main into the diff. I'm fixing it now. The copilot comments were about changes made from some commits in main, not my branch.

Fixed :)

Copilot

Copilot review overview

🟡 Changes recommended

BMM jobs with recorded CU counts can still be precompiled for incompatible wheel targets.

Review effort: Balanced
Findings: 1

Open (1)

BMM jobs discard CSV CU counts during exact-CU filtering · New

Resolved since last review (2)

Make QKVPACKED CLI defaults use equal sequence lengths
Move gfx950 config into the required nested path

Copilot

Copilot review overview

🟡 Changes recommended

The new unconditional test calls abort the CPU-only test script when torch or FlyDSL is unavailable.

Review effort: Balanced
Findings: 1

Open (1)

Skip GEMM checks when torch or FlyDSL is unavailable · New

Resolved since last review (1)

BMM jobs discard CSV CU counts during exact-CU filtering

+    test_flydsl_aot_target_filter()
+    test_flydsl_aot_legacy_rows()
+    test_flydsl_aot_invalid_archs()


…argets/flydsl-aot

Copilot

🔵 Needs a closer look

Unconditional FlyDSL tests can abort the existing codegen suite when optional dependencies are unavailable.

1 open finding

Skip GEMM checks when torch or FlyDSL is unavailable

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

- Wraps the imports of the three FlyDSL AOT filter tests in a guard that reports SKIP and returns, so the file runs on CPU with only pandas installed, as its docstring states. - Re-raises any ModuleNotFoundError whose missing module is neither torch nor flydsl, so a broken FlyDSL submodule or other dependency still fails the run.

Copilot

🔵 Needs a closer look

End-to-end multi-architecture wheel validation remains unconfirmed and needs human review.

1 open finding

Skip GEMM checks when torch or FlyDSL is unavailable

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

RElbers · 2026-10-08T20:11:48Z

See:
#5340 (comment)

amd-bartgips mentioned this pull request Sep 8, 2026

[CK] [FlyDSL] [CI] [Bugfix] Explicit gfx in shipped fused-MoE tuning CSVs #5315

Open

This was referenced Sep 9, 2026

[JIT] Identify MI350P, and stop hardcoding the XCD count #5182

Open

[HIP] [CI] [JIT] [Feature] Add gfx:cu_num build targets so one build serves multiple SKUs of an arch #5340

Draft

samremes mentioned this pull request Sep 9, 2026

[HIP] [OPUS] [JIT] [Bugfix] Stop an unbaked tuned kernel id from killing the rank, and key the lookup on CU count #5341

Closed

RElbers force-pushed the cu-aware-targets/flydsl-aot branch from 5f9cd36 to 03d1d98 Compare September 9, 2026 15:13

RElbers changed the base branch from main to cu-aware-targets/gpu-target-resolution September 9, 2026 16:57

RElbers changed the title ~~[Feature] Bake only the FlyDSL GEMM kernels a builds own targets can run~~ [Feature] Precompile only the GEMM kernels the wheel is built for Sep 11, 2026

RElbers added the ci:all label Sep 11, 2026

RElbers marked this pull request as ready for review September 11, 2026 15:16

RElbers added this pull request to stack #5466 September 11, 2026 15:20

RElbers changed the title ~~[Feature] Precompile only the GEMM kernels the wheel is built for~~ [FlyDSL] [Feature] Precompile only the GEMM kernels the wheel is built for Sep 11, 2026

RElbers added the FlyDSL label Sep 11, 2026

RElbers requested a review from a team September 11, 2026 15:50

RElbers force-pushed the cu-aware-targets/flydsl-aot branch from 70038fe to bd52fd7 Compare September 11, 2026 15:50

zufayu requested a review from coderfeli September 14, 2026 01:44

RElbers added 5 commits September 22, 2026 15:19

Name parse_csv as the reason the AOT filter is GEMM only

a6f850f

Strip trailing whitespace from the AOT filter comment

4c3ad41

- Black rejects the trailing space.

RElbers force-pushed the cu-aware-targets/flydsl-aot branch from cb7cc4e to 4c3ad41 Compare September 22, 2026 13:19

RElbers added 3 commits September 22, 2026 17:24

Simplify FlyDSL GEMM AOT target selection. Consolidates the small fun…

75f76a2

…ctions added in earlier commits of this branch so CLI and packaging rules are easier to follow.

Simplify GEMM build-target tests by consolidating overlapping target …

ff78bae

…cases.

Merge remote-tracking branch 'upstream/main' into cu-aware-targets/fl…

2331cbf

…ydsl-aot # Conflicts: # aiter/aot/flydsl/README.md # aiter/aot/flydsl/gemm.py

github-actions Bot changed the title ~~[FlyDSL] [Feature] Precompile only the GEMM kernels the wheel is built for~~ [Triton/Gluon] [ASM] [HIP] [Feature] Precompile only the GEMM kernels the wheel is built for Sep 29, 2026

github-actions Bot added ASM CI labels Sep 29, 2026

Copilot started reviewing on behalf of Boss2002n September 29, 2026 13:34 View session

Copilot AI reviewed Sep 29, 2026

View reviewed changes

Comment thread op_tests/test_mha_varlen.py

Comment thread aiter/ops/triton/configs/gfx950-KDA_DECODE-DEFAULT.json

Merge remote-tracking branch 'upstream/cu-aware-targets/gpu-target-re…

5d9b9bd

…solution' into cu-aware-targets/flydsl-aot

github-actions Bot changed the title ~~[Triton/Gluon] [ASM] [HIP] [Feature] Precompile only the GEMM kernels the wheel is built for~~ [FlyDSL] [Feature] Precompile only the GEMM kernels the wheel is built for Sep 29, 2026

github-actions Bot removed CI Triton/Gluon HIP CK JIT OPUS ASM labels Sep 29, 2026

RElbers requested a balanced review from Copilot September 29, 2026 15:05

Copilot started reviewing on behalf of RElbers September 29, 2026 15:06 View session

Copilot AI reviewed Sep 29, 2026

View reviewed changes

Comment thread aiter/aot/flydsl/gemm.py

Keep the CSV CU count on FlyDSL BMM compile jobs

94bce7b

Copilot AI balanced review requested due to automatic review settings October 5, 2026 11:58

Copilot started reviewing on behalf of RElbers October 5, 2026 11:59 View session

Copilot AI reviewed Oct 5, 2026

View reviewed changes

Comment thread op_tests/test_gemm_codegen.py

Comment on lines +1073 to +1075

test_flydsl_aot_target_filter()

test_flydsl_aot_legacy_rows()

test_flydsl_aot_invalid_archs()

Merge branch 'cu-aware-targets/gpu-target-resolution' into cu-aware-t…

05b71c1

…argets/flydsl-aot

Copilot AI balanced review requested due to automatic review settings October 8, 2026 12:26

Copilot started reviewing on behalf of RElbers October 8, 2026 12:27 View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings October 8, 2026 12:52

Copilot started reviewing on behalf of RElbers October 8, 2026 12:53 View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

RElbers marked this pull request as draft October 8, 2026 20:09

zufayu removed the request for review from coderfeli October 9, 2026 00:29

Navigation Menu

Repository navigation

[FlyDSL] [Feature] Precompile only the GEMM kernels the wheel is built for - #5342

[FlyDSL] [Feature] Precompile only the GEMM kernels the wheel is built for#5342
RElbers wants to merge 12 commits into
cu-aware-targets/gpu-target-resolutionfrom
cu-aware-targets/flydsl-aot

RElbers commented Sep 8, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

samremes commented Sep 9, 2026 •

edited by atlassian Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

RElbers commented Sep 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

RElbers commented Oct 8, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

RElbers commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Scope

Technical Details

Also in here

Test Plan

Test Result

Uh oh!

github-actions Bot commented Sep 8, 2026

🏷️ CI Guide

Uh oh!

samremes commented Sep 9, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review (FlyDSL GEMM AOT consumer / non-comm enablement)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Uh oh!

Uh oh!

RElbers commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

RElbers commented Oct 8, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

RElbers commented Sep 8, 2026 •

edited

Loading

samremes commented Sep 9, 2026 •

edited by atlassian Bot

Loading

RElbers commented Sep 29, 2026 •

edited

Loading