Skip to content

[FlyDSL] Support runtime softmax scale in gfx950 FP8 FMHA - #5405

Merged
coderfeli merged 5 commits into
mainfrom
gbyu/flydsl-fmha-softmax-scale
Sep 14, 2026
Merged

coderfeli merged 5 commits into
mainfrom
gbyu/flydsl-fmha-softmax-scale

Conversation

@gbyu-amd

@gbyu-amd gbyu-amd commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

The gfx950 FlyDSL FP8 FMHA kernel currently fixes the attention scale to 1 / sqrt(D). Consumers using a different layer scale, such as YaRN, must fold the correction into a quantization descale, and the AITER dispatch rejects nondefault scales.

Add softmax_scale=None to the direct FP8 API, using the BF16 API's parameter name and default. Pass it as a runtime FP32 scalar and apply softmax_scale * q_descale * k_descale to the logits inside FMHA. Dense and varlen dispatch and the large-tensor batch-slicing path forward the scale. This needs no descale mutation or additional GPU kernel launch, and scale values do not enter the build-cache key. Output and LSE use the same scale.

The existing online softmax requires a positive, finite scale: it takes maxima and applies masking before scaling. The direct API rejects zero, negative and non-finite scales; dispatch returns None for those configurations. Supporting them would require a separate change to the softmax algorithm.

This PR targets main and contains only FMHA scale support, dispatch changes and tests. The review follow-up removes the redundant head_dim_runtime launch argument and shares softmax-scale validation between the dispatch gate and direct FP8 API.

Validation:

  • Review follow-up (a859b8271): 24 CPU tests passed, 252 GPU tests skipped, with HIP/ROCR/CUDA device visibility disabled. Fake-tensor tests exercise the direct API and dense/varlen dispatch for default/custom/invalid scales; launch-boundary tests check default and custom scales in both run and compile argument preparation for D=128/192. No GPU execution or GPU compilation was performed. Black, Ruff and git diff --check pass.
  • Prior validation after the rebase onto main: Black, Ruff and git diff --check pass; all 252 FMHA tests collect successfully with GPUs hidden (AITER_AOT_IMPORT=1). Collection is not a correctness run.
  • At that rebase, git range-diff confirmed the feature patch was unchanged; all five changed files were byte-identical to the previously GPU-validated PR head. GPU tests were not repeated during the active TP8 workload.
  • Added coverage checks nondefault scales, default/explicit-scale equivalence, repeated scale changes on a cached launcher, 12-head K3 shapes, dense/varlen, GQA, split-K, fully masked rows, zero-length KV entries, graph replay, dispatch and batch slicing.
  • GPU validation before the main rebase, on a free gfx950 GPU with a fresh FlyDSL cache: 80 passed, 1 skipped (the cross-device test requires a second visible GPU). This includes the custom-scale correctness, LSE, dispatch and graph-replay tests above.
  • ATOM integration after replacing descale folding with direct softmax_scale forwarding: 112 tests passed.
  • Cached-chunk GPU traces for Q=70, H=12, D=192, Dv=128 and KV=16,384 / 11,008 confirm one FMHA kernel per call, with no added scale-preparation launch. Warmed CUDA-graph measurements were 278.99 / 190.98 microseconds per call (7 rounds of 100 replays); these are absolute measurements, not a speedup claim for this change.

@gbyu-amd
gbyu-amd marked this pull request as ready for review September 10, 2026 11:57
Base automatically changed from binding_fa_gfx950 to main September 10, 2026 12:42
@binding7012
binding7012 requested a review from a team September 10, 2026 12:42
@gbyu-amd
gbyu-amd force-pushed the gbyu/flydsl-fmha-softmax-scale branch from ade9a4a to 3407084 Compare September 10, 2026 14:50
Comment thread aiter/ops/flydsl/kernels/fmha_gfx950/flash_attn_fp8_gfx950.py Outdated
Comment thread aiter/ops/flydsl/fmha_kernels.py Outdated
Comment thread aiter/ops/flydsl/kernels/flash_attn_func_fp8_gfx950.py Outdated
@zufayu
zufayu requested a review from yadaish September 14, 2026 01:44
@binding7012

Copy link
Copy Markdown
Contributor

LGTM

@coderfeli
coderfeli merged commit c213dea into main Sep 14, 2026
56 checks passed
@coderfeli
coderfeli deleted the gbyu/flydsl-fmha-softmax-scale branch September 14, 2026 12:19
sammysun0711 pushed a commit to sammysun0711/aiter that referenced this pull request Sep 16, 2026
* [FlyDSL] Support runtime softmax scale in gfx950 FP8 FMHA

* [FlyDSL] Share FP8 FMHA scale validation and remove redundant launch argument

Signed-off-by: Xiake Sun <xiake.sun@amd.com>
RElbers pushed a commit that referenced this pull request Sep 25, 2026
* [FlyDSL] Support runtime softmax scale in gfx950 FP8 FMHA

* [FlyDSL] Share FP8 FMHA scale validation and remove redundant launch argument
RElbers pushed a commit that referenced this pull request Sep 25, 2026
* [FlyDSL] Support runtime softmax scale in gfx950 FP8 FMHA

* [FlyDSL] Share FP8 FMHA scale validation and remove redundant launch argument
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants