[mgoin]

Overview

Pass --compress-mode=size to nvcc for CUDA >= 13. This mirrors the block in vLLM's CMakeLists.txt that already does this for vLLM's own extensions. VLLM_FA_GPU_FLAGS doesn't inherit VLLM_GPU_FLAGS, so FA was missing it. With default settings, fatbinary compresses only PTX and debug images, so all FA2/FA3 SASS has been stored uncompressed.

Impact (release arch list, CUDA 13.0)

Built at the commit vLLM pins (b273a02) plus this patch, with TORCH_CUDA_ARCH_LIST='7.5 8.0 8.6 8.9 9.0 10.0 11.0 12.0'. Compared against the .so files in the vLLM 0.33.0 x86_64 wheel on PyPI (same commit, nvcc 13.0.88):

	installed .so	.nv_fatbin	deflated (wheel cost)
_vllm_fa2_C before	308.0 MB	297.2 MB	129.6 MB
_vllm_fa2_C after	77.1 MB	63.3 MB	65.0 MB
_vllm_fa3_C before	172.6 MB	133.7 MB	42.6 MB
_vllm_fa3_C after	56.9 MB	17.2 MB	24.8 MB

That's about 82 MB off the vLLM wheel (335 MB → about 253 MB) and about 347 MB off the installed size.

Validation

• Kernel code is unchanged. I extracted every image with cuobjdump -xelf/-xptx and hashed each kernel's .text.* section:
  • FA2: 2,904/2,904 SASS kernels are byte-identical to the 0.33.0 release.
  • FA3: 1,333/1,333 are byte-identical.
  • The 76 FA2 PTX files differ only in nvcc's per-TU _INTERNAL_<hash> symbol names, which come from the build environment.
• Runtime on B300 (sm_103). FA2 runs there by JIT-compiling the compute_80 PTX, so this covers decompression plus JIT. A causal varlen call (bf16, hdim 128, GQA 32/8) matches fp32 scaled_dot_product_attention with a max abs error of 0.0118. The first call took 9.7 s with CUDA_CACHE_DISABLE=1.
• Not run: FA3 (sm_90a) and FA2's sm_80 SASS path, because I have no Hopper or Ampere hardware. Coverage there rests on the identical-SASS check above.

Choosing the flag

I compiled one FA2 TU (flash_fwd_split_align_hdim128_bf16_causal_sm80.cu, sm_80 SASS plus compute_80 PTX) with each option:

flags	.o	deflated
default / --compress-mode=balance	3397 KB	1380 KB
-Xfatbin=-compress-all	1304 KB	1088 KB
--compress-mode=size	900 KB	683 KB
--compress-mode=size -Xfatbin=-compress-all	900 KB	683 KB

--compress-mode=size already compresses every image, SASS included, so adding -Xfatbin=-compress-all changes nothing. -compress-all only helps for builds without --compress-mode, such as CUDA 12. I've kept this to the same CUDA >= 13 gate vLLM uses.

After this merges, vLLM needs a GIT_TAG bump in cmake/external_projects/vllm_flash_attn.cmake to pick it up.

AI assistance was used to write and validate this change.

🤖 Generated with Claude Code

[MatthewBonanni]

Nice find!