Skip to content

feat(attention): share standalone FA3 varlen and LSE adapter - #7249

Closed
yeahdongcn wants to merge 3 commits into
vllm-project:mainfrom
yeahdongcn:xd/fa3-varlen-adapter
Closed

yeahdongcn wants to merge 3 commits into
vllm-project:mainfrom
yeahdongcn:xd/fa3-varlen-adapter

Conversation

@yeahdongcn

@yeahdongcn yeahdongcn commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Extract the standalone FlashAttention-3 adapter from the MAGI-2/MUSA reference draft into the shared diffusion attention utilities.

  • Support the standalone FA3 package layouts through one cached resolver.
  • Normalize output-only and (output, LSE) calls, including return_attn_probs / return_softmax_lse aliases.
  • Forward optional native sinks only when the provider explicitly supports them; do not silently emulate unsupported features.
  • Reuse this adapter from vllm_flash_attn_varlen_with_lse when the bundled API cannot be imported. Preserve the existing bundled-backend priority and do not redirect explicit FA2/FA4 requests.

No MAGI-2 algorithm, MUSA platform gating, kernel implementation, EP, sampler change or performance configuration is included. No environment-variable inventory changes.

Tests

Changed-file pre-commit: all pass (including Ruff and mypy).

55 CPU contract/attention tests pass in a CPU-only container, including existing version-selection coverage. The two MUSA numerical tests now pass; the two CUDA numerical tests remain not run. Five existing device-dependent attention tests were excluded from this CPU-only run; the initial unrestricted selection failed on accelerator allocation because the container had no GPU.

python -m pytest -q -o addopts='' -p no:cacheprovider \
  tests/diffusion/attention/test_fa3_varlen_adapter.py \
  tests/diffusion/attention/test_flash_attn.py -m cpu \
  -k 'not test_padding_equivalence and not test_fa_vs_sdpa and not test_flash_attn_func_preferred_over_varlen and not test_cross_attn_key_padding_vs_sdpa'

On the MUSA development image, torchada was imported before invoking pytest.main with the above arguments. The MUSA provider validation below covers this adapter only. CUDA runtime validation and model end-to-end/performance remain not run; this PR stays Draft.

MUSA provider validation

Exact tested head: 612bdad0914c3c13214fd2d31d1f391a31eeeacf.

  • 2 passed, 24 deselected, 0 skipped, including causal and non-causal packed BF16 GQA with unequal Q/K lengths.
  • Both output and packed LSE are checked against independent FP32 Torch references. Existing tolerances are unchanged: output rtol=2e-2, atol=2e-2; LSE rtol=3e-3, atol=3e-3.
  • MTT S5000, one 48-SM device, driver 5.2.0-server. Python 3.10.12, torch/torch_musa 2.11.0.post1+musa5.2.0, torchada 0.1.83, MATE 0.2.6, flash_attn_3 0.2.6+musa, vLLM 0.28.0, vllm-musa 0.1.28.
  • Resolved provider: flash_attn_3.interface through the installed torchada wrapper; return_attn_probs requests LSE.
  • Image: registry.mthreads.com/mcconline/inference/vllm-omni@sha256:7f4f2cd83a88979025ef92968334f6e075eb24d45d02e0c30dbb222f7780d58a.
  • Exact checkout installed with SETUPTOOLS_SCM_PRETEND_VERSION=0.28.0 python -m pip install -e . --no-deps --no-build-isolation; import path verified. Install, tests and container exit codes were 0.

After importing torchada, the runner invokes:

pytest.main([
    "-vv", "-s", "-o", "addopts=", "-p", "no:cacheprovider",
    "tests/diffusion/attention/test_fa3_varlen_adapter.py", "-m", "musa",
])

No native-sink, MAGI-2 consumer, full-model, compilation/capture or performance qualification is implied by these two adapter tests. The separate MAGI-2 FA3 consumer still has its own numerical gate; this result does not qualify that branch.

Split boundary

This is the generic adapter part of #7156, which remains unchanged as reference. The later MUSA wiring PR will consume the shared API; it is intentionally not bundled here. BF16 MoE remains in #7206.

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
@hsliuustc0106 hsliuustc0106 added the enhancement New feature or request label Sep 20, 2026
@yeahdongcn

Copy link
Copy Markdown
Contributor Author

Superseded by #8498, which carries a trimmed version of this adapter (no backend selector, no fa3_fwd_interface candidate).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion codes related to diffusion models enhancement New feature or request Kernel optimization Codes related to optimize kernel execution to improve hardware utilization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants