Skip to content

[NPU] [bugfix] Fix import of ggml_moe_a8_vec and Fix NPU MLA HiCache backup accessing missing data_ptrs - #36640

Merged
Fridge003 merged 2 commits into
sgl-project:mainfrom
chx96642264:fix_bug
Aug 27, 2026
Merged

Fridge003 merged 2 commits into
sgl-project:mainfrom
chx96642264:fix_bug

Conversation

@chx96642264

@chx96642264 chx96642264 commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

PR #30393 generalized the MLA HiCache backup path to support packed target and draft KV buffers. However,  NPUMLATokenToKVPool intentionally does not create data_ptrs (it uses contiguous multi-layer K/V/index-K tensors for the NPU transfer kernel). 
And, Ascend NPU does not support ggml_moe_a8_vec which in sgl_kernel.

Errors as follows:
error-1
error-2

Modifications

Accuracy Tests

use DeepSeek-V2-Lite-W8A8 test gsm8k

deepseek-v2-w8a8-gsm8k

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #33071938518
Latest PR Test (Extra): ❌ Run #33071938243
Latest PR Test (AMD ROCm 7.2): ❌ Run #33071938450

@github-actions github-actions Bot added the hicache Hierarchical Caching for SGLang label Aug 27, 2026
@chx96642264 chx96642264 changed the title [NPU] [bugfix] Fix import of ggml_moe_a8_vec and Fix NPU MLA HiCache [NPU] [bugfix] Fix import of ggml_moe_a8_vec and Fix NPU MLA HiCache backup accessing missing data_ptrs Aug 27, 2026
@sglang-npu-bot

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Aug 27, 2026
@chx96642264
chx96642264 force-pushed the fix_bug branch 3 times, most recently from cf72d22 to 1ed8542 Compare August 27, 2026 12:14
@Fridge003
Fridge003 merged commit 5640e53 into sgl-project:main Aug 27, 2026
135 of 175 checks passed
@Fridge003

Copy link
Copy Markdown
Collaborator

The root fix is in #36529

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hicache Hierarchical Caching for SGLang run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants