Revert "Skip materialised causal attn_bias on FSDPA for non-GDN hybri…#1481
Merged
Conversation
…d models (vllm-project#1413)" This reverts commit 808dbfa. Signed-off-by: Radoslaw Smyrek <radoslawx.smyrek@intel.com>
rsmyrek
had a problem deploying
to
pre-merge-approval
GitHub Actions
Error
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR removes the “non-GDN hybrid” topology detection and the associated early-return optimization that skipped materializing attention bias for certain hybrid models when using FusedSDPA.
Changes:
- Removed
is_non_gdn_hybridcomputation in runner and related component init paths. - Removed “FSDPA-native causal short-circuit” branches in
set_attn_biasand_set_attn_biasthat previously returned early for non-GDN hybrid models.
Comment on lines
3890
to
3894
| or not attn_metadata.is_prompt): | ||
| return attn_metadata | ||
|
|
||
| # Extended FSDPA-native causal short-circuit for non-GDN hybrid models | ||
| # (e.g. Granite-4 Mamba2+Transformer). FusedSDPA can encode a purely | ||
| # causal mask natively via is_causal=True + valid_seq_lengths, including | ||
| # chunked prefill where block_list is non-None. Skipping the | ||
| # materialised [bs, 1, q_len, total_kv_len] attn_bias avoids a large | ||
| # add_bf16 on the attention critical path (significant at long | ||
| # context). Conservative scope: only non-GDN hybrid models; GDN / | ||
| # pure-transformer / other topologies keep the materialised bias path | ||
| # until validated. | ||
| if (self.prefill_use_fusedsdpa and self.is_causal and not self.is_pooling_model | ||
| and not getattr(self, 'sliding_window', None) | ||
| and not getattr(self, 'model_has_chunked_attention', False) | ||
| and getattr(self, 'alibi_slopes', None) is None and self.is_non_gdn_hybrid): | ||
| return attn_metadata | ||
|
|
||
| if attn_metadata.attn_bias is not None: | ||
| return attn_metadata |
Comment on lines
6764
to
6768
| or not attn_metadata.is_prompt): | ||
| return attn_metadata | ||
|
|
||
| # Extended FSDPA-native causal short-circuit for non-GDN hybrid models | ||
| # (e.g. Granite-4 Mamba2+Transformer). FusedSDPA handles a purely | ||
| # causal mask natively (is_causal=True + valid_seq_lengths). Skip | ||
| # materialising a [bs, 1, q_len, total_kv_len] attn_bias even during | ||
| # chunked prefill (block_list is non-None) for these topologies; this | ||
| # removes a sizable add_bf16 from the attention critical path during | ||
| # long-context chunked prefill. interleaved_sliding_window and | ||
| # chunked-attention bias paths (window_attn_bias / chunked_attn_bias) | ||
| # are populated later in process_metadata and used by hpu_attn | ||
| # instead. Conservative scope: only non-GDN hybrid models; all other | ||
| # topologies retain the original behaviour. | ||
| if (self.prefill_use_fusedsdpa and not self.interleaved_sliding_window and self.is_non_gdn_hybrid): | ||
| return attn_metadata | ||
|
|
||
| if attn_metadata.attn_bias is not None: | ||
| return attn_metadata |
rsmyrek
requested review from
PatrykWo,
mgawarkiewicz-intel and
wpyszka
as code owners
rsmyrek
temporarily deployed
to
pre-merge-approval
GitHub Actions
Inactive
jbyczkow
approved these changes
May 25, 2026
…ausal_attn_bias_revert_0.21.0
jbyczkow
temporarily deployed
to
pre-merge-approval
GitHub Actions
Inactive
jbyczkow
approved these changes
May 25, 2026
🚧 CI BlockedThe main CI workflow was not started for the following reason:
|
…ausal_attn_bias_revert_0.21.0
jbyczkow
temporarily deployed
to
pre-merge-approval
GitHub Actions
Inactive
jbyczkow
approved these changes
May 25, 2026
✅ CI PassedAll checks passed successfully against the following vllm commit: |
mgawarkiewicz-intel
merged commit 5121be2
into
vllm-project:releases/v0.21.0
3 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…d models (#1413)"
This reverts commit 808dbfa.