Skip to content

[Fix] hisparse: Release HiSparse rows on the KV checksum mismatch and chunked abort exits - #43911

Merged
hnyls2002 merged 2 commits into
mainfrom
lsyin/sched-pr17
Nov 4, 2026
Merged

hnyls2002 merged 2 commits into
mainfrom
lsyin/sched-pr17

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Two exits free a request's KV with release_kv_cache without releasing its HiSparse state first, so the rows leak. Both now call hisparse_coordinator.request_finished before the KV free, under the same enable_hisparse guard as the other exits.

  • PD decode KV checksum mismatch (_handle_kv_checksum_mismatch): the transfer has already admitted the request into HiSparse, so its host pages and device buffer were never returned.
  • Chunked prefill abort (process_pending_chunked_abort): a request aborted between chunks holds only the spec scratch row its extend allocated; request_finished frees exactly that (its no-device-buffer branch), and the prefill KV is still freed by release_kv_cache.

No change without --enable-hisparse. No stream wait is added or moved; request_finished keeps its own wait on the forward stream, as at the other exits.

Test plan

  • One unit test per exit checks that the HiSparse release runs, and runs before the KV free; both fail without the fix.
  • /rerun-test on this head:
    • test_hisparse_unit.py (1-gpu-5090): passed, run
    • test_disaggregation_hisparse.py (8-gpu-h200, PD + HiSparse, DeepSeek-V4-Flash): passed, run
    • test_dsa_glm52_hisparse.py (8-gpu-b200, HiSparse + speculative decoding): passed, run
    • test_kv_checksum.py (1-gpu-h100): queued, run
  • The leaks are found by reading the exits and confirmed by the unit tests; an end-to-end reproduction on a server is in progress.

CI States

Latest PR Test (Base): 🚫 Run #41746867200
Latest PR Test (Extra): ❌ Run #41746867142
Latest PR Test (AMD ROCm 10): 🚫 Run #41746867236

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@hnyls2002 hnyls2002 changed the title [PP] Run the pipeline loops through one ring driver [Fix] hisparse: Release HiSparse rows on the KV checksum mismatch and chunked abort exits Nov 4, 2026
@hnyls2002
hnyls2002 marked this pull request as ready for review November 4, 2026 00:20

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_kv_checksum.py test_hisparse_unit.py test_disaggregation_hisparse.py test_dsa_glm52_hisparse.py

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🚀 1-gpu-h100 (1 test): ❌ View workflow run

cd test/ && python3 registered/unit/disaggregation/test_kv_checksum.py

🚀 1-gpu-5090 (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/managers/test_hisparse_unit.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_hisparse.py

🚀 8-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_dsa_glm52_hisparse.py

@hnyls2002 hnyls2002 added the run-ci CI: run the baseline test suite on this PR label Nov 4, 2026
@hnyls2002
hnyls2002 merged commit d091f25 into main Nov 4, 2026
125 of 158 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/sched-pr17 branch November 4, 2026 00:54
Yuxingwang-intel pushed a commit to Yuxingwang-intel/sglang that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant