[Attention][4/n] Remove usage of deprecated seq_lens_cpu and num_computed_tokens_cpu CommonAttentionMetadata properties

[LucasWilkinson]

Continue deprecating seq_lens_cpu and num_computed_tokens_cpu

Don't fully deprecate _seq_lens_cpu due to FlashInfer perf regressions, see: #33771

[gemini-code-assist]

Code Review

This pull request is part of a series to remove deprecated properties from CommonAttentionMetadata, specifically seq_lens_cpu and num_computed_tokens_cpu. The changes correctly replace direct access to these properties by deriving the values from other available metadata, often directly on the GPU to avoid device-to-host synchronization. This refactoring is applied consistently across various components, including attention backends and speculative decoding logic. The changes are correct, improve code clarity, and should lead to better performance. I have not found any issues of high or critical severity.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[benchislett]

I don't understand why we're moving from .seq_lens_cpu to .seq_lens.cpu(). It feels like this would introduce more DtoH copies than would otherwise be necessary.

[benchislett]

move this so it only happens when we has num_prefills > 0

[benchislett]

Given that this impacts only rare cases and will eventually be phased out entirely, I'm happy with this change. Just needs one line moved to avoid synchronizing on the decode codepath when it's not needed

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint