[callanjfox]

Summary

Draft for review. Adds a KV Cache Management section to the Developer Guide with the KV Cache Runner (KVCR) pages for vLLM and SGLang. Please review the KVCR pages; the overview page is included so the section links work.

New pages in docs/fern/pages/developer-guide/knowledge-base/kv-offload/ (flat navigation, engine in each title):

Page	Type
KV Cache Management Overview	Concept
KV Cache Runner (KVCR)	Concept, with configuration diagrams
How KV Hints Work	Concept
Configure KVCR for vLLM	Task, one GPU server
Run KVCR on Kubernetes with vLLM	Task
Run KVCR Guard on Kubernetes with vLLM	Task
Verify and Monitor KVCR with vLLM	Task
Configure KVCR for SGLang	Task, preview
Configure KVCR Linker Mode for SGLang	Task, preview
Run KVCR on Kubernetes with SGLang	Task, preview
Run KVCR Guard on Kubernetes with SGLang	Task, preview
Tune Routing for KVCR	Task

Also:

• docs/fern/index.yml: adds the section to the Developer Guide tab.
• Links into the section from the router pages (Configuration and Tuning, Topology-Aware KV Transfer, Offloading Support Matrix).
• examples/backends/sglang/deploy/kvcr/: draft SGLang manifests used by the two SGLang Kubernetes pages.

Validation

• vLLM pages: commands run as written on 2x H200 (peer-only, KVCR pool, Guard with an engine crash and reattach); both vLLM Kubernetes examples deployed on two nodes (transfer, crash recovery, reattach).
• SGLang HiCache and Linker (with Guard) pages: run as written on 2x H200 with the unmerged SGLang KVCR series.
• SGLang Kubernetes: the HiCache example ran on two H200 nodes (the second worker reused 2,112 of 2,161 prompt tokens through KVCR). The Linker + Guard example passes a server-side dry run only; it has not been deployed.
• Routing: frontend starts with the recommended policy and flags; routing under load not benchmarked. Recommended values come from the simulations in feat(router): credit KV blocks a worker can fetch, choosing the fetch source during selection #16701.
• docs_lint.py: 0 errors. Every relative link and anchor in the new pages resolves.

Not in this PR

• Hybrid models, disaggregated serving with KVCR, local disk tier (g3), TensorRT-LLM.
• Open dependencies: SGLang #45646 to #45650, KVCR feat: LLMAPI PoC with dynamo-run launcher #114, Dynamo fix(sglang): attach the KVCR linker to the external state agent #16827 (closed unmerged; the SGLang Linker pages need it reopened), vLLM #53073 (/dev/shm leak on crash). KVCR v0.2.0 is tagged but not yet on PyPI.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[callanjfox]

/ok to test 8957248

[copy-pr-bot]

/ok to test 8957248

@callanjfox, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a184c6-87a5-760d-be63-9a5733b077c9.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

✅ Dynamo PR CI passed — run 40784644315 (attempt 1) on b8c8d06f53

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

_{Posted automatically by Devin for run 40784644315. Updated on every full-CI run of this PR.}