[jayzuccarelli]

Purpose

Fixes #58597.

AttentionMetrics bills every layer of hybrid linear-attention models (Qwen3-Next, Qwen3.5) as full attention. SlidingWindowAttentionParser only reads layer_types when sliding_window is set and only counts "sliding" entries, so the "linear_attention" (Gated DeltaNet) layers are never subtracted. For a 32-layer Qwen3-Next config with full_attention_interval=4, layer_types has 8 full and 24 linear layers, but _layer_counts() returns (32, 32, 0). As the issue shows, the reported MBU then goes past the GPU's peak bandwidth.

As the reporter pointed out, removing those layers from attention alone under-bills (about 0.75x of the bytes actually moved), so this PR does both:

1. LinearAttentionLayerParser counts "linear_attention" entries in layer_types whether or not sliding_window is set. AttentionMetrics._layer_counts bills only num_hidden_layers - num_linear_attn_layers. The SWA split and the per-count pp_size division from [V1][Metrics] Support Sliding Window Attention (SWA) and hybrid layers in MFU/MBU estimation #55624 are unchanged.

2. New LinearAttentionMetrics component (linear_attn) for Gated DeltaNet layers, built from linear_num_{key,value}_heads, linear_{key,value}_head_dim and linear_conv_kernel_dim:

   • weights: in_proj_qkvz + in_proj_ba, conv1d, out_proj
   • per scheduled request, the conv state (conv_dim * (kernel - 1)) and recurrent state (Hv * dv * dk) are read and written once per step, with dtypes resolved from mamba_cache_dtype / mamba_ssm_cache_dtype the same way as MambaStateDtypeCalculator
   • no term that grows with context length
   • FLOPs: projections, the conv, and three dk x dv matrix-vector products per token and value head for the recurrence
   • heads split across TP (as in gated_delta_net_state_shape), layers divided across PP (as in _layer_counts)

   The component raises InvalidComponent when a model has no linear-attention layers, so dense, SWA and MLA models are unaffected.

Not duplicate work: gh pr list --state open --search "58597 in:body" returns nothing, and a search for MFU / linear-attention / hybrid PRs finds only #55624 (merged, SWA only). The two open PRs touching perf.py (#55474 logging interval, #54228 MoE expert activation) do not touch layer counting.

Left for follow-ups:

• Mamba / Mamba2 hybrids that use layers_block_type (Jamba, Falcon-H1, NemotronH) are still billed as full attention.
• The Qwen3.5 / Qwen3-Next attention output gate (attn_output_gate: true doubles the q_proj width) is not in qkv_weight.
• Small GDN parameters (A_log, dt_bias, gated norm weight) are left out.
• Models that label non-GDN layers "linear_attention" in layer_types without the linear_num_* fields (for example glm5_next, and bailing_moe_v3_vl, which maps mamba/conv to that label) now drop those layers from attention but get no linear_attn component. They need their own component.

Test Plan

Six new tests in tests/v1/metrics/test_perf_metrics.py, using create_mock_vllm_config with Qwen3NextConfig(num_hidden_layers=32, full_attention_interval=4):

• test_attention_metrics_excludes_linear_attention_layers: _layer_counts(per_gpu=False) == (8, 8, 0)
• test_model_metrics_includes_linear_attention_for_hybrid: ModelMetrics has a linear_attn component with 24 layers
• test_model_metrics_excludes_linear_attention_for_dense: a dense Qwen3Config gets no linear_attn component and its attention counts are unchanged
• test_linear_attention_decode_read_bytes_independent_of_context: 8 decode requests at 1k and 32k context give the same linear-attention reads, writes and FLOPs
• test_linear_attention_state_bytes_per_request: state read/write bytes match R * (conv_state + recurrent_state) * L by hand
• test_linear_attention_per_gpu_layers_sum_to_whole_model: per-GPU bytes times pp_size equal the whole model

create_mock_vllm_config now sets mamba_cache_dtype and mamba_ssm_cache_dtype to "auto" on the mock cache_config, matching the CacheConfig defaults. The existing SWA, dense, MoE and MLA tests are unchanged.

pytest tests/v1/metrics/test_perf_metrics.py -q
ruff check vllm/v1/metrics/perf.py tests/v1/metrics/test_perf_metrics.py      # ruff 0.14.0, as pinned in pre-commit
ruff format --check vllm/v1/metrics/perf.py tests/v1/metrics/test_perf_metrics.py

The before run used the same test file against the unmodified perf.py from main (import of LinearAttentionMetrics stubbed so the file collects).

Test Result

With this PR (CPU):

78 passed in 0.73s

On main:

E       assert (32.0, 32.0, 0.0) == (8, 8, 0)
E       AssertionError: assert 'linear_attn' in {'attn': AttentionMetrics(num_hidden_layers=32, ...), 'ffn': ..., 'unembed': ...}
6 failed, 72 passed

The other four failures on main are the new component not existing.

Config-only check against the model from the issue (get_config("Qwen/Qwen3.5-4B") through create_mock_vllm_config, no GPU):

components: ['attn', 'linear_attn', 'ffn', 'unembed']
attn _layer_counts: (8.0, 8.0, 0.0)
linear_attn weight params: 1011.35M

The 1,011M matches the linear-attention parameter count the reporter took from the safetensors headers. ruff check: All checks passed!; ruff format: 2 files already formatted.

AI assistance (Claude Code) was used to write the code and tests and to draft this description. The tests above were run locally.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[jayzuccarelli]

@markmc follow-up to the hybrid MFU work you reviewed in #55624/#57070: this bills Gated DeltaNet layers separately so Qwen3-Next/Qwen3.5 MBU stops going past peak bandwidth (#58597). Still merges cleanly on current main, and the CPU-runnable tests in tests/v1/metrics/test_perf_metrics.py pass on the merged tree (the rest need a GPU). CI hasn't started since pre-run-check wants a ready label. Could you take a look?

[markmc]

@thillai-c please take a look 👍