[do420]

What this PR does / why we need it?

AscendAttentionBackendImpl.forward chose the bidirectional pooling path from attn_metadata.causal alone. ENCODER_ONLY layers of hybrid models (Gated DeltaNet + full attention, e.g. the Qwen3.5-based Decision 3.0 / D3 checkpoints) reach the backend with causal=True, because they do not go through the EncoderOnlyAttention wrapper that clears the flag. These layers therefore silently ran causal attention even though the model sets config.is_causal = False.

This PR also routes layers whose attn_type is ENCODER_ONLY to _forward_encoder_attention, which is what upstream backends do for such layers regardless of metadata.causal.

• DECODER layers are unchanged.
• Models whose encoder-only layers already receive causal=False (e.g. BERT-style models through EncoderOnlyAttention) take the same branch as before.
• Generation models are unchanged (model_runner_type is not "pooling").

Prefill context parallelism (PCP) is not supported for ENCODER_ONLY attention. The encoder attention path attends over rank-local K/V only. Unlike the KV-cache write, it never gathers the other PCP ranks' tokens (encoder-only layers have no KV cache). Pooling models default to the V1 model runner, which already rejects PCP, but VLLM_USE_V2_MODEL_RUNNER=1 can force V2 and bypass that check, which would silently give wrong results. This PR therefore fails at model construction with NotImplementedError for ENCODER_ONLY attention when prefill_context_parallel_size > 1, and documents the limitation in the context parallel guide.

Fixes #19918

Does this PR introduce any user-facing change?

Yes. Pooling models with ENCODER_ONLY full-attention layers in hybrid architectures now produce bidirectional (correct) results instead of causal ones, so their outputs change. No API, parameter or configuration change. Pooling models with ENCODER_ONLY attention now also fail at startup under PCP (reachable only with the V2 runner forced), instead of silently computing with rank-local K/V. A release note fragment is included.

How was this patch tested?

Unit test (tests/ut/attention/test_attention_v1.py)

test_forward_pooling_encoder_only_is_bidirectional_despite_causal_metadata checks that a pooling layer with attn_type=ENCODER_ONLY uses the bidirectional attention kernel and returns its output even when the metadata says causal=True.

The isolated CPU check passed with the fix and failed on the original backend because it never called the encoder kernel. After syncing with current main (b93e8f711), the repository tests passed against vLLM 24420b7a: python -m pytest -q tests/ut/attention/test_attention_v1.py tests/ut/test_platform.py — 203 passed, 1 skipped, in a CPU-only container with no NPU devices (2 CPUs, 12 GiB RAM).

All manual-stage pre-commit hooks passed on the four PR-changed files. bash format.sh ci was also run across the whole repository; it reported two staged secret-scan false positives and two long-function warnings in files inherited unchanged from main. The actual PR diff passed secret scanning. The earlier CI Gitleaks failure was caused by a missing origin/main ref in the runner checkout; the branch is now synced with current main's CI setup.

Ascend hardware (Ascend950DT, CANN 9.2.0.b080, torch_npu 2.10.0.post4, vLLM c0b6f74, vLLM Ascend with the D3 adapter from #19804): vllm-sr/d3-edge and vllm-sr/d3-nano with runner="pooling", max_model_len=4096. Eager runs had multimodal inputs enabled; graph runs had them limited to zero (graph-mode startup can also crash when a stale torch.compile cache written with a different limit_mm_per_prompt setting is reused; that is a known upstream vLLM issue, vllm-project/vllm#50891, and unrelated to this change). Compared against the checkpoint's reference runtime on the same prompts, softmax over each question's options (3 questions: choice, noul, score):

model	mode	max softmax diff before	max softmax diff after
d3-edge	eager and graph	0.33	0.0065
d3-nano	graph	not run	0.0193

Before the change the outputs matched a causal reference (0.0035) instead. POST /v1/systemone on d3-edge returns the same probabilities as the offline run after the change.

Not covered

• Other pooling models (BERT-style, non-hybrid Qwen3 bidirectional embedding) were not run on hardware.
• No performance benchmark. Affected layers now use the unmasked npu_fusion_attention path, which bidirectional attention requires.
• Multi-NPU validation of the PCP startup guard and requests with image or video inputs remain untested.

---

Checklist

Testing

• Boundary / edge-case coverage

• Regression test: the ENCODER_ONLY unit test fails without the fix.

• Error / failure path coverage: the retained test focuses on the attention dispatch.

• Meaningful & reliable tests: the regression test checks that the encoder kernel runs and its output is returned.

• Performance impact: no benchmark attached, see "Not covered" above.

• vLLM main: vllm-project/vllm@24420b7

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where ENCODER_ONLY layers in hybrid models were incorrectly executing causal attention due to misleading metadata. By explicitly routing these layers to the encoder attention path, the implementation now correctly handles bidirectional attention for these components, improving output accuracy for pooling models.

Highlights

• Logic correction for ENCODER_ONLY layers: Updated AscendAttentionBackendImpl.forward to ensure ENCODER_ONLY layers are processed using bidirectional attention, overriding incorrect causal metadata in hybrid models.
• Regression testing: Added unit tests to verify that ENCODER_ONLY layers correctly trigger the encoder attention path while maintaining causal behavior for DECODER layers.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

---

Tip

💡 Consider Linking a Related Issue or RFC

Your PR title contains the [BugFix] tag, indicating a bug fix or new feature.

Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:

• Fixes #<issue_number>
• Closes #<issue_number>
• Resolves #<issue_number>
• Refs #<rfc_or_issue_number> (for RFCs)

🙏 Thanks for helping us keep the project well-organized!

[gemini-code-assist]

Code Review

Suggested PR Title:

[Attention][BugFix] Ensure ENCODER_ONLY layers in pooling models bypass causal path

Suggested PR Summary:

### What this PR does / why we need it?
This PR updates the attention forward logic for pooling models in `vllm_ascend`. Specifically, it ensures that `ENCODER_ONLY` layers (which are bidirectional) bypass the causal path and correctly use the encoder attention path, even when the metadata of a hybrid model indicates it is causal. This is necessary because `ENCODER_ONLY` layers never go through the `EncoderOnlyAttention` wrapper.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Added unit tests in `tests/ut/attention/test_attention_v1.py` to verify that ENCODER_ONLY layers use the encoder path and DECODER layers keep the causal path.

No review comments were provided, so there is no feedback to evaluate.

[QwertyJack]

The implementation matches upstream and covers the reported D3 regression. Before merge, can you confirm whether pooling + ENCODER_ONLY + PCP is supported? This path does not gather PCP K/V; if supported, please add a test or fail-fast, otherwise document the limitation.

[do420]

The implementation matches upstream and covers the reported D3 regression. Before merge, can you confirm whether pooling + ENCODER_ONLY + PCP is supported? This path does not gather PCP K/V; if supported, please add a test or fail-fast, otherwise document the limitation.

Thanks for the input. pooling + ENCODER_ONLY + PCP is not supported: the encoder attention path only sees rank-local K/V and never gathers the other PCP ranks' tokens.
By default it can't be reached, since pooling models run on the V1 runner, which already rejects PCP. VLLM_USE_V2_MODEL_RUNNER=1 bypasses that check.As discussed, I pushed a new commit a fail-fast NotImplementedError for ENCODER_ONLY attention when PCP > 1, unit tests, and the limitation documented.

[QwertyJack]

LGTM

[do420]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[yiz-liu]

Please remove unnecessary test cases.

[do420]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[do420]

Please remove unnecessary test cases.

The test cases are removed, only one kept.

[do420]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E