[ziqifan617]

Stack and review base

Depends on #45646. Logical base: ziqifan617/kvcr-shared-foundation at e17419bd4a3e8d6783202359fe6eab1ad41d2183.

GitHub base is main, so Files changed is cumulative until ancestors land. Upstream cannot target a base branch in the contributor fork. Review this slice using the exact incremental comparison: ziqifan617/sglang@e17419b...a2f9da8

Do not merge before the parent. Rebase/revalidate after it lands. HiCache and linker are independent sibling modes.

Motivation

Provide generic request-lifetime and inventory hooks for an optional KVCR direct linker while preserving current SGLang request-private load safety.

Modifications

• Snapshot attempt-scoped router hints; release contexts without revoking accepted native transfers.
• Confirm sparse shared-prefix residency and preserve cancellation ownership across request-ID reuse.
• Publish lower-tier inventory only after rank-agreed successful offload; retain removal identity after GPU eviction/reset.
• Mark only opted-in CPU inventory as KVCR-owned. GPU inventory and engine clears remain framework-owned; never coalesce across owners.
• Preserve LoRA namespace identity and optional ownership fields in Python and Rust.
• Existing backends retain no-op/default hooks. No KVCR runtime implementation is introduced here.

Includes attributed Moein/Lin work and its adaptation to newer request-private loading. The old early-tree-publication behavior is not restored.

Integration

The direct-device backend is a child. Dynamo ai-dynamo/dynamo#16827 provides the external state-agent bridge for opt-in TP1 aggregated linker workers. It does not enable the same bridge for HiCache or qualify TP>1/PD.

Accuracy Tests

Recorded CPU checks: 90 wrapper cases and 24 event cases passed; the initial CPU run skipped 20 CUDA-dependent cases. A subsequent matching Linux production/inspection native build ran all 110 wrapper cases with zero skips, including the 20 GPU cases. Exact binary and source identities were verified. An initial test collection failure caused by UUID-vs-numeric CUDA visibility was preserved; only test launch configuration changed for the successful retry.

Locked Rust core/test/binding type checks and formatting also passed. Type checking alone is not a Cargo runtime-test pass.

Speed Tests and Profiling

No isolated speed claim for generic hooks. The direct-linker leaf reports its exact model acceptance separately; Guard replacement, independent-inventory eviction/recovery and broader configurations remain separate gates.

Known merge gate

Fresh comparison against upstream a3d00eff58b6c22ea7539f882cc10c537634200b conflicts in rust/sglang-radix-tree/src/python_bindings.rs. This is the tested prepared source, not a newly validated rebase.

Checklist

• Focused lifecycle, ownership and namespace tests.
• Recorded CPU and native/GPU wrapper validation.
• Reconcile upstream conflict, then revalidate.
• Upstream CI/review and broader inventory acceptance.

Full draft stack

Shared foundation #45646
├── HiCache storage #45647
│   └── HiCache performance #45649
└── Linker contracts #45648
    └── Direct linker #45650

• Foundation: [KVCR 1/5] Share rank-local endpoint contracts #45646
• HiCache storage: [Feat] hicache: Add KVCR storage backend #45647
• HiCache performance: [KVCR 3/5] Add opt-in HiCache delivery and placement controls #45649
• Linker contracts: [KVCR 4/5] Add request-scoped linker hooks and owned inventory #45648
• Direct linker: [Feat] cache: Add KVCR direct-device linker #45650
• KVCR optional SDK tuning: perf: add opt-in DRAM descriptor coalescing and UCX batching ai-dynamo/kvcr#113
• KVCR SGLang quick-start preview: docs: add SGLang HiCache quick start alongside vLLM ai-dynamo/kvcr#114

These are upstream draft review slices with logical dependencies, not branches based on upstream staging refs. Dependent Files changed views are cumulative; the incremental comparison above isolates this slice. Do not merge dependent drafts before their ancestors. The original #43635 and existing Dynamo PRs remain unchanged.