Skip to content

fix(attention): unwrap hybrid backend for FP8 DSA MHA - #38508

Open
ActiveSky wants to merge 1 commit into
sgl-project:mainfrom
ActiveSky:codex/fix-38477-hybrid-fp8-mha
Open

ActiveSky wants to merge 1 commit into
sgl-project:mainfrom
ActiveSky:codex/fix-38477-hybrid-fp8-mha

Conversation

@ActiveSky

@ActiveSky ActiveSky commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Motivation

Fixes #38477. FP8 DSA MHA reads forward_metadata from the outer
HybridLinearAttnBackend during cached-prefix prefill, although the metadata
is owned by its full-attention backend.

Modifications

  • Reuse the existing attention-backend resolver when reading FP8 DSA metadata.
  • Add a CPU-only regression covering TBO-wrapped hybrid backends.

Validation

  • pre-commit run --files python/sglang/srt/models/deepseek_common/attention_forward_methods/forward_mha.py test/registered/unit/models/test_deepseek_fp8_dsa_backend.py passed.
  • python -m py_compile passed for the changed Python files.

Checklist

  • Format code with the repository pre-commit configuration.
  • Add focused regression tests.
  • No documentation update is required.
  • No speed benchmark is required.

CI States

Latest PR Test (Base): ❌ Run #34226402992
Latest PR Test (Extra): ❌ Run #34226402882
Latest PR Test (AMD ROCm 7.2): ❌ Run #34226402994

…ting backend resolver when reading FP8 DSA metadata so hybrid full-attention backends provide the page table. Add a CPU regression covering TBO-wrapped hybrid backends.\n\nFixes sgl-project#38477.
@ActiveSky

ActiveSky commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Hi maintainers, thank you for taking a look at this PR.

The implementation and focused checks for #38477 are complete. The current CI gate stops before scheduling the test jobs because this PR does not yet have the required run-ci label. If appropriate, could someone with the necessary permission please add run-ci and rerun the CI?

Thank you for your guidance.

@ActiveSky

Copy link
Copy Markdown
Contributor Author

Hi @fzyzcjy @ispobock @ch-wan @Fridge003,

There has been no formal review activity on this PR for some time, so we wanted to check whether this change is still relevant and whether we should continue working on it. We understand priorities can change; if the issue is already covered elsewhere or the scope should be adjusted, please let us know and we will follow your guidance.

If the PR is still useful, could you please take a look when you have time? Feedback on the implementation, tests, or whether the branch should be updated would be very helpful. We hope to contribute in a way that helps move the SGLang project forward.

Thank you very much for your time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] FP8 MHA hybrid model crashes: HybridLinearAttnBackend has no forward_metadata

1 participant