Repository navigation
feat(attention): share standalone FA3 varlen and LSE adapter - #7249
Closed
yeahdongcn wants to merge 3 commits into
Closed
yeahdongcn wants to merge 3 commits into
yeahdongcn wants to merge 3 commits into
Conversation
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
This was referenced Sep 8, 2026
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
This reverts commit c45d685.
Contributor
Author
|
Superseded by #8498, which carries a trimmed version of this adapter (no backend selector, no fa3_fwd_interface candidate). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extract the standalone FlashAttention-3 adapter from the MAGI-2/MUSA reference draft into the shared diffusion attention utilities.
(output, LSE)calls, includingreturn_attn_probs/return_softmax_lsealiases.vllm_flash_attn_varlen_with_lsewhen the bundled API cannot be imported. Preserve the existing bundled-backend priority and do not redirect explicit FA2/FA4 requests.No MAGI-2 algorithm, MUSA platform gating, kernel implementation, EP, sampler change or performance configuration is included. No environment-variable inventory changes.
Tests
Changed-file pre-commit: all pass (including Ruff and mypy).
55 CPU contract/attention tests pass in a CPU-only container, including existing version-selection coverage. The two MUSA numerical tests now pass; the two CUDA numerical tests remain not run. Five existing device-dependent attention tests were excluded from this CPU-only run; the initial unrestricted selection failed on accelerator allocation because the container had no GPU.
On the MUSA development image,
torchadawas imported before invokingpytest.mainwith the above arguments. The MUSA provider validation below covers this adapter only. CUDA runtime validation and model end-to-end/performance remain not run; this PR stays Draft.MUSA provider validation
Exact tested head:
612bdad0914c3c13214fd2d31d1f391a31eeeacf.rtol=2e-2, atol=2e-2; LSErtol=3e-3, atol=3e-3.5.2.0-server. Python 3.10.12, torch/torch_musa2.11.0.post1+musa5.2.0, torchada0.1.83, MATE0.2.6, flash_attn_30.2.6+musa, vLLM0.28.0, vllm-musa0.1.28.flash_attn_3.interfacethrough the installed torchada wrapper;return_attn_probsrequests LSE.registry.mthreads.com/mcconline/inference/vllm-omni@sha256:7f4f2cd83a88979025ef92968334f6e075eb24d45d02e0c30dbb222f7780d58a.SETUPTOOLS_SCM_PRETEND_VERSION=0.28.0 python -m pip install -e . --no-deps --no-build-isolation; import path verified. Install, tests and container exit codes were 0.After importing
torchada, the runner invokes:No native-sink, MAGI-2 consumer, full-model, compilation/capture or performance qualification is implied by these two adapter tests. The separate MAGI-2 FA3 consumer still has its own numerical gate; this result does not qualify that branch.
Split boundary
This is the generic adapter part of #7156, which remains unchanged as reference. The later MUSA wiring PR will consume the shared API; it is intentionally not bundled here. BF16 MoE remains in #7206.