Repository navigation
docs: add SGLang HiCache quick start alongside vLLM - #114
Merged
Merged
Conversation
This was referenced Oct 21, 2026
ziqifan617
force-pushed
the
ziqifan617/sglang-quick-start
branch
from
October 21, 2026 22:17
40e3013 to
5e1bebc
Compare
ziqifan617
marked this pull request as ready for review
October 21, 2026 22:18
ziqifan617
force-pushed
the
ziqifan617/sglang-quick-start
branch
from
October 26, 2026 19:10
4c436b6 to
3d68be1
Compare
Keep the vLLM Dockerfile byte-for-byte unchanged under an engine-specific name. Preserve the walkthrough in its own guide, make the original path a landing page, and update vLLM references. Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Build the pinned SGLang, Dynamo, and KVCR source tuple in its own image with import/version checks. Include only the required documentation in its build context; leave the vLLM Dockerfile unchanged. Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Document the public Qwen3-0.6B build, launch, and cold-source/peer-destination checks. Link the new guide from the quick-start index, README, and developer guide, while keeping the two engine paths distinct. Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
ziqifan617
force-pushed
the
ziqifan617/sglang-quick-start
branch
from
October 27, 2026 04:51
95bc6df to
dc57894
Compare
Update the separate SGLang image recipe and guide to the latest #16117 head for fresh public Qwen3-0.6B validation. Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Use the Dynamo main merge commit containing owned KV inventory (#17531) for the SGLang image and document the exact source tuple. Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
mkhazraee
approved these changes
Oct 30, 2026
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a public-source SGLang HiCache/KVCR quick start with a separate Dockerfile, alongside the vLLM guide and image.
b0506add3eac49bbb2c303e83169ea19c907fb98and Dynamo at83ea37e849e883c27d7ed848e0f329961e1fb173.Validation
d186752ce78092aa2bd0cc4c1d48d24a1c247354, SGLangb0506add3eac49bbb2c303e83169ea19c907fb98, and Dynamo83ea37e849e883c27d7ed848e0f329961e1fb173. KVCR 0.2.0 was built from the checkout and the image's import/version checks passed. The unchanged Dockerfile was built with BuildKit; this was not a GPU inference test.0d37a2bf4db3521a876d1b450a884b91c0e334c9and KVCR7b7743fe0104393ac56c6fe83ae58387268dc72bsource pair, a custom two-worker Qwen3-32B GPU test verified cross-node peer delivery, 115,968 restored tokens, and matching output IDs. This was not the guide-as-written Qwen3-0.6B test.4a083003f9393b758077afe9bbce3f6b1c64f790and KVCR7186e7c20ba6e4d71ac3aff507fac9bfd83e8a07pair passed the public-source Qwen3-0.6B two-worker check: 1,984/2,048 peer-reused tokens, matching 16 output token IDs, and 31 blocks / 62 spans with no failed keys.b0506addSGLang pin together with the post-merge KVCR main containing Perf: serve ready local DRAM sources without the core state lock #148 remains pending. Do not infer that validation from the successful image build or earlier source-pair runs.