Skip to content

docs: add SGLang HiCache quick start alongside vLLM - #114

Merged
ziqifan617 merged 17 commits into
mainfrom
ziqifan617/sglang-quick-start
Oct 30, 2026
Merged

ziqifan617 merged 17 commits into
mainfrom
ziqifan617/sglang-quick-start

Conversation

@ziqifan617

@ziqifan617 ziqifan617 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add a public-source SGLang HiCache/KVCR quick start with a separate Dockerfile, alongside the vLLM guide and image.

  • Build KVCR from the local Docker build-context checkout, including local source changes, rather than fetching a separate KVCR revision.
  • Pin SGLang #45647 at b0506add3eac49bbb2c303e83169ea19c907fb98 and Dynamo at 83ea37e849e883c27d7ed848e0f329961e1fb173.
  • Run the state agent and publish KVCR-owned inventory for router-guided peer delivery; verify completed delivery, destination reuse, and matching output tokens. No KVCR production-code changes.
  • After this PR merges, the guide clones the default branch, so the KVCR build context includes changes already merged to main, including Perf: serve ready local DRAM sources without the core state lock #148.

Validation

  • Built the SGLang quick-start Dockerfile from KVCR PR checkout d186752ce78092aa2bd0cc4c1d48d24a1c247354, SGLang b0506add3eac49bbb2c303e83169ea19c907fb98, and Dynamo 83ea37e849e883c27d7ed848e0f329961e1fb173. KVCR 0.2.0 was built from the checkout and the image's import/version checks passed. The unchanged Dockerfile was built with BuildKit; this was not a GPU inference test.
  • With the earlier SGLang 0d37a2bf4db3521a876d1b450a884b91c0e334c9 and KVCR 7b7743fe0104393ac56c6fe83ae58387268dc72b source pair, a custom two-worker Qwen3-32B GPU test verified cross-node peer delivery, 115,968 restored tokens, and matching output IDs. This was not the guide-as-written Qwen3-0.6B test.
  • The earlier SGLang 4a083003f9393b758077afe9bbce3f6b1c64f790 and KVCR 7186e7c20ba6e4d71ac3aff507fac9bfd83e8a07 pair passed the public-source Qwen3-0.6B two-worker check: 1,984/2,048 peer-reused tokens, matching 16 output token IDs, and 31 blocks / 62 spans with no failed keys.
  • GPU end-to-end validation of the current b0506add SGLang pin together with the post-merge KVCR main containing Perf: serve ready local DRAM sources without the core state lock #148 remains pending. Do not infer that validation from the successful image build or earlier source-pair runs.

@copy-pr-bot

copy-pr-bot Bot commented Oct 21, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ziqifan617 ziqifan617 changed the title docs: add SGLang HiCache and linker quick-start preview docs: add public SGLang HiCache build and acceptance quick start Oct 21, 2026
@ziqifan617 ziqifan617 changed the title docs: add public SGLang HiCache build and acceptance quick start docs: add SGLang HiCache quick start alongside vLLM Oct 21, 2026
@ziqifan617
ziqifan617 force-pushed the ziqifan617/sglang-quick-start branch from 40e3013 to 5e1bebc Compare October 21, 2026 22:17
@ziqifan617
ziqifan617 marked this pull request as ready for review October 21, 2026 22:18
@ziqifan617
ziqifan617 force-pushed the ziqifan617/sglang-quick-start branch from 4c436b6 to 3d68be1 Compare October 26, 2026 19:10
Keep the vLLM Dockerfile byte-for-byte unchanged under an engine-specific name. Preserve the walkthrough in its own guide, make the original path a landing page, and update vLLM references.

Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Build the pinned SGLang, Dynamo, and KVCR source tuple in its own image with import/version checks. Include only the required documentation in its build context; leave the vLLM Dockerfile unchanged.

Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Document the public Qwen3-0.6B build, launch, and cold-source/peer-destination checks. Link the new guide from the quick-start index, README, and developer guide, while keeping the two engine paths distinct.

Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
@ziqifan617
ziqifan617 force-pushed the ziqifan617/sglang-quick-start branch from 95bc6df to dc57894 Compare October 27, 2026 04:51
Update the separate SGLang image recipe and guide to the latest #16117 head for fresh public Qwen3-0.6B validation.

Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Use the Dynamo main merge commit containing owned KV inventory (#17531) for the SGLang image and document the exact source tuple.

Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
@ziqifan617
ziqifan617 merged commit 7ce17c3 into main Oct 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants