Skip to content

Declare separate draft sidecars for HiCache - #40914

Open
ispobock wants to merge 2 commits into
mainfrom
hicache-decl-b
Open

ispobock wants to merge 2 commits into
mainfrom
hicache-decl-b

Conversation

@ispobock

@ispobock ispobock commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Part 2 of 5 of #40608. The separate (non packed) draft path in build_full_draft_pools still picks the DSA indexer by isinstance. A draft pool's declarations are now renamed into the target's sidecar namespace (KV to DRAFT, INDEXER to DRAFT_INDEXER, indices from target KV, layout on DRAFT) and assembled by the same builder as the target. Hybrid Mamba, QSA and plain KV keep their existing paths.

Stack: #40913 (declarations and DSA target), #40914 (separate draft), #40915 (hybrid Mamba), #40916 (QSA), #40917 (plain KV and cleanup). Each PR is based on the previous one.

Accuracy

Structure tests pin the DRAFT and DRAFT_INDEXER entries, their layer mapping and geometry, and that a draft without index buffers gets no indexer sidecar. No DSA family standalone draft model is available, so this path is CPU tested only, as it was before.


CI States

Latest PR Test (Base): ❌ Run #37405381419
Latest PR Test (Extra): 🚫 Run #37405381225
Latest PR Test (AMD ROCm 10): ❌ Run #37405381351

@github-actions github-actions Bot added the hicache Hierarchical Caching for SGLang label Sep 23, 2026
Base automatically changed from hicache-decl-a to main October 1, 2026 05:08
@ispobock ispobock added run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) labels Oct 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) hicache Hierarchical Caching for SGLang memory-pool run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant