Skip to content

[Build] Update vllm to point to vllm-project/flash-attention commit that builds FA3 with torch stable API. - #46644

Merged
simon-mo merged 6 commits into
vllm-project:mainfrom
cleonard530:migrate_fa3_to_stable_abi
Jun 27, 2026
Merged

[Build] Update vllm to point to vllm-project/flash-attention commit that builds FA3 with torch stable API. #46644
simon-mo merged 6 commits into
vllm-project:mainfrom
cleonard530:migrate_fa3_to_stable_abi

Conversation

@cleonard530

@cleonard530 cleonard530 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Purpose

As a part of #26946. Updated vllm_flash_attn.cmake to use FA fork that builds a libtorch ABI stable FA3 library.

Currently, this is pointing towards my flash-attention fork but if this passes the CI, we will merge that fork with vllm-project/flash-attention, then we will point to the new tag.

see vllm-project/flash-attention#152 to see the commit it is pointing to.

cc @janeyx99 @Harry-Chen

Test Plan

build with new flash-attention commit
pytest tests/kernels/attention/test_flash_attn.py -k "3"

Test Result

Build succeded

image
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Migration progress of vLLM using the Audit Python extension torch-abi-audit:

-- extensions --
    [STABLE  ] [abi3-ok               ] _C_stable_libtorch.abi3.so  (stable_shim=87, unstable=0)
    [UNSTABLE] [abi3-ok               ] _flashmla_C.abi3.so  (stable_shim=0, unstable=72)
    [UNSTABLE] [abi3-ok               ] _flashmla_extension_C.abi3.so  (stable_shim=0, unstable=68)
    [STABLE  ] [abi3-ok               ] _moe_C_stable_libtorch.abi3.so  (stable_shim=72, unstable=0)
    [NO-TORCH] [abi3-ok               ] cumem_allocator.abi3.so
    [NO-TORCH] [abi3-ok               ] spinloop.abi3.so
    [UNSTABLE] [uses-private-api      ] third_party/deep_gemm/_C.cpython-312-x86_64-linux-gnu.so  (stable_shim=0, unstable=57)
    [UNSTABLE] [abi3-ok               ] vllm_flash_attn/_vllm_fa2_C.abi3.so  (stable_shim=0, unstable=84)
    [STABLE  ] [abi3-ok               ] vllm_flash_attn/_vllm_fa3_C.abi3.so  (stable_shim=65, unstable=0)

…abi stable FA3 api. If this passes the CI, we will merge that fork with vllm-project/flash-attention, then we will point to the new tag here

Signed-off-by: Chris Leonard <chleonar@redhat.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the ci/build label Jun 24, 2026
@Harry-Chen Harry-Chen added ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm labels Jun 25, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Jun 25, 2026
@Harry-Chen

Copy link
Copy Markdown
Member

Triggered a CI pipeline to test building

@cleonard530

Copy link
Copy Markdown
Contributor Author

Thanks @Harry-Chen. Non of these failures seem to be caused by the updates to FA3.

@Harry-Chen

Harry-Chen commented Jun 25, 2026

Copy link
Copy Markdown
Member

@cleonard530 It is merged! Could you please change the commit ref to the main repo? (and the commit title)

…t with torch stable abi fa3 build

Signed-off-by: Chris Leonard <chleonar@redhat.com>
@cleonard530

Copy link
Copy Markdown
Contributor Author

Thanks @Harry-Chen, just updated the GIT_TAG!

@Harry-Chen Harry-Chen changed the title [do not merge!] Update vllm to point to vllm-project/flash-attention commit that builds FA3 with torch stable API. [Build] Update vllm to point to vllm-project/flash-attention commit that builds FA3 with torch stable API. Jun 25, 2026

@Harry-Chen Harry-Chen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@Harry-Chen
Harry-Chen enabled auto-merge (squash) June 25, 2026 17:39
@cleonard530

Copy link
Copy Markdown
Contributor Author

@Harry-Chen, I checked each of these failures and I don't think they were coming from this PR. Also, since the test passed before when the tag pointed to my flash-attention fork, I don't think the last change would of broken anything. Can you please rerun the failed test and try to merge this?

@simon-mo
simon-mo merged commit d0f8008 into vllm-project:main Jun 27, 2026
217 of 229 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Jun 27, 2026
WindChimeRan pushed a commit to WindChimeRan/vllm that referenced this pull request Jun 27, 2026
wincent8 pushed a commit to wincent8/vllm that referenced this pull request Jun 29, 2026
rjrock pushed a commit to rjrock/vllm that referenced this pull request Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants