Disable abi3 for free-threaded python#2034
Conversation
rgommers
left a comment
rgommers
left a comment
There was a problem hiding this comment.
This is the right thing to do in order to allow flash-attention to be built from source under a free-threaded interpreter. I made a very similar change to the vLLM fork of this package at vllm-project#112.
It's not quite clear to me why this hopper/setup.py uses the stable ABI; the top-level setup.py produces cp3xx version-specific wheels anyway, so there doesn't seem to be a benefit from targeting the stable ABI.
| # Free-threaded Python does not support the limited API | ||
| # See: https://docs.python.org/3/howto/free-threading-extensions.html | ||
| FREE_THREADED = sysconfig.get_config_var("Py_GIL_DISABLED") | ||
| DISABLE_ABI3 = FREE_THREADED or os.getenv("FLASH_ATTENTION_DISABLE_ABI3", "FALSE") == "TRUE" |
There was a problem hiding this comment.
Adding the extra environment variable here is potentially useful, but not related to free-threading support. I'd personally not include it unless a maintainer asks for it.
|
@rgommers for context, xformers maintainers cc @danthe3rd wanted to be able to build a single wheel (since FA3 didn't publish wheels so the onus is on the users to produce the wheels) #1662. |
|
@janeyx99 sure that is perfectly fine. The point of this PR isn't to change what wheels are distributed nor what wheels are even built for a default (with-GIL) Python interpreter. However, free-threaded interpreters don't support the stable ABI at all yet, and once they do (expected for Python 3.15, see PEP 803) then that will be a new ABI rather than The goal of this PR is to fix that, while changing nothing for with-GIL interpreteres. This vLLM PR has a more extensive description to explain: vllm-project/vllm#29241. It'd have been useful for the PR description to articulate this. Note that I just happened to notice this PR and decided to add some context that I thought may be useful, I'm not otherwise involved. @kevmo314 may be able to add more context (maybe they want to experiment with free-threading and this package?). |
3 participants
No description provided.