[BugFix] Fix whisper FA2 + full cudagraphs

[LucasWilkinson]

Fix: #33091

CrossAttentionBuilder.build() overrides max_seq_len with encoder_seq_lens (new_metadata.max_seq_len = max(encoder_seq_lens_cpu)) this leads to a CG capture with max_seq_len == 0 and an incorrect graph

[gemini-code-assist]

Code Review

This pull request fixes an issue with CUDA graph capture for encoder-decoder models using FlashAttention 2 by ensuring a non-zero max_seq_len is used during graph creation. The approach of setting a realistic encoder length during capture is correct. I've identified one area for improvement in the fallback logic for determining this length to make the fix more robust.

[NickLucche]

Thank you! If CI is green this should be good as we have FA2 machines :)

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mgoin]

Makes sense to me, thanks!