Repository navigation
feat(dispatcher/codegen): fixed A/B vector widths for the default epilogue and comp_async - #13610
Merged
Merged
Conversation
This was referenced Oct 14, 2026
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
assistant-librarian Bot
pushed a commit
to ROCm/composable_kernel
that referenced
this pull request
Oct 14, 2026
feat(dispatcher/codegen): fixed A/B vector widths for the default epilogue and comp_async (#13610) JIRA ID : AICK-2558 ## Why Problems whose contiguous extents are not multiples of a kernel's native vector width are served by kernels compiled with narrower fixed A/B/C widths. Two pipelines' worth of those kernels were never generated: - **Default epilogue.** `resolve_gemm_vector_sizes` refused every fixed-width request unless the epilogue was cshuffle. The fixed A/B widths live in the pipeline problem, so they apply with any epilogue; only storing C narrower than its native width needs CShuffle, because the default epilogue's C width is fixed by the warp-tile layout. - **comp_async.** It was missing from `VECTOR_SIZE_PIPELINES`. Its pipeline problem honours fixed A/B widths like the other universal pipelines, with one extra limit: A and B are copied global to LDS by `amd_async_global_load_to_lds`, which moves only 4, 8, 12 or 16 bytes, or 1 byte for 8-bit types. ## What changes `dispatcher/codegen/codegen_common.py`: - Fixed widths are accepted with the default epilogue when C keeps its native width; a narrowed C is refused there, and the CShuffle split check is scoped to the cshuffle epilogue, the only one that shuffles C. - comp_async is added to `VECTOR_SIZE_PIPELINES`, and A/B widths whose copy size that instruction cannot move are rejected. comp_async's existing gfx1250 rules (row-major A, column-major B, full padding) are unchanged.
Contributor
TheRock Submodule Bump ActivityNewest first
|
shumway
pushed a commit
to ROCm/composable_kernel
that referenced
this pull request
Nov 4, 2026
feat(dispatcher/codegen): fixed A/B vector widths for the default epilogue and comp_async (#13610) JIRA ID : AICK-2558 ## Why Problems whose contiguous extents are not multiples of a kernel's native vector width are served by kernels compiled with narrower fixed A/B/C widths. Two pipelines' worth of those kernels were never generated: - **Default epilogue.** `resolve_gemm_vector_sizes` refused every fixed-width request unless the epilogue was cshuffle. The fixed A/B widths live in the pipeline problem, so they apply with any epilogue; only storing C narrower than its native width needs CShuffle, because the default epilogue's C width is fixed by the warp-tile layout. - **comp_async.** It was missing from `VECTOR_SIZE_PIPELINES`. Its pipeline problem honours fixed A/B widths like the other universal pipelines, with one extra limit: A and B are copied global to LDS by `amd_async_global_load_to_lds`, which moves only 4, 8, 12 or 16 bytes, or 1 byte for 8-bit types. ## What changes `dispatcher/codegen/codegen_common.py`: - Fixed widths are accepted with the default epilogue when C keeps its native width; a narrowed C is refused there, and the CShuffle split check is scoped to the cshuffle epilogue, the only one that shuffles C. - comp_async is added to `VECTOR_SIZE_PIPELINES`, and A/B widths whose copy size that instruction cannot move are rejected. comp_async's existing gfx1250 rules (row-major A, column-major B, full padding) are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA ID : AICK-2558
Why
Problems whose contiguous extents are not multiples of a kernel's native vector width are served by kernels compiled with narrower fixed A/B/C widths. Two pipelines' worth of those kernels were never generated:
resolve_gemm_vector_sizesrefused every fixed-width request unless the epilogue was cshuffle. The fixed A/B widths live in the pipeline problem, so they apply with any epilogue; only storing C narrower than its native width needs CShuffle, because the default epilogue's C width is fixed by the warp-tile layout.VECTOR_SIZE_PIPELINES. Its pipeline problem honours fixed A/B widths like the other universal pipelines, with one extra limit: A and B are copied global to LDS byamd_async_global_load_to_lds, which moves only 4, 8, 12 or 16 bytes, or 1 byte for 8-bit types.What changes
dispatcher/codegen/codegen_common.py:VECTOR_SIZE_PIPELINES, and A/B widths whose copy size that instruction cannot move are rejected. comp_async's existing gfx1250 rules (row-major A, column-major B, full padding) are unchanged.Testing
dispatcher/tests/test_gemm_vector_sizes.py. The driver and codegen suites (test_gemm_vector_sizes.py,test_gfx1250_tdm_pipelines.py,test_gemm_driver_verify.py,test_codegen_common.py, and tile_engine's gfx1250 pipeline tests) give 353 passed, 1 skipped on develop plus this PR (develop: 351 passed, the same skip).Merge order and related PRs
One of five independent PRs for gfx1250 GEMM support in the dispatcher and its heuristics. Suggested merge order:
--capture_hwon gfx950)This PR is number 3. Each of the five applies to develop on its own and merges cleanly with each of the others in either order (checked with three-way merges), so the order is a suggestion, not a requirement. This PR and #13609 both edit
codegen_common.pyandtest_gemm_vector_sizes.py, in different places.Related PRs from earlier work on the dispatcher heuristics and the GEMM sweep driver:
operationtocompute_tflops_efficiencyin evaluate.py #13332: their changes are already in develop, through feat(dispatcher): vector-width-aware feature engine and the universal-GEMM tile-divisibility rule #13272; no interaction.🤖 Generated with Claude Code