Repository navigation
Conversation
…, and PD bootstrap (sgl-project#39677) (cherry picked from commit 2f7d04d)
(cherry picked from commit 91f691c)
Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: DarkSharpness <2040703891@qq.com> Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com> Co-authored-by: Ke Bao <ispobaoke@gmail.com> Co-authored-by: Khoa Pham <khoa.pham@radixark.ai> Co-authored-by: Xiaoyu Zhang <1182563586@qq.com> Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> Co-authored-by: Yuwei An <ayw.sirius19@gmail.com> Co-authored-by: Zhichen Zeng <zczeng@uw.edu> Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai> (cherry picked from commit faaff1e)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: DarkSharpness <2040703891@qq.com> Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com> Co-authored-by: Ke Bao <ispobaoke@gmail.com> Co-authored-by: Khoa Pham <khoa.pham@radixark.ai> Co-authored-by: Xiaoyu Zhang <1182563586@qq.com> Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> Co-authored-by: Yuwei An <ayw.sirius19@gmail.com> Co-authored-by: Zhichen Zeng <zczeng@uw.edu> Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai> (cherry picked from commit 7d5696b)
Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com> (cherry picked from commit 13d593b)
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com> Co-authored-by: Xiaoyu Zhang <1182563586@qq.com> (cherry picked from commit 35b7589)
Co-authored-by: DarkSharpness <2040703891@qq.com> Co-authored-by: Xiaoyu Zhang <1182563586@qq.com> Co-authored-by: BBuf <1182563586@qq.com> (cherry picked from commit c244345)
Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> (cherry picked from commit 46ae84d)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com> Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com> (cherry picked from commit 408d233)
Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> (cherry picked from commit c89c63f)
Co-authored-by: BBuf <1182563586@qq.com> (cherry picked from commit 464fffb)
Co-authored-by: BBuf <1182563586@qq.com> (cherry picked from commit 3401b75)
…r compress ratios (sgl-project#39921) (cherry picked from commit 1f0c73e)
…r MXFP8 GEMMs (sgl-project#40039) (cherry picked from commit 1b200ff)
Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai> Co-authored-by: Yuwei An <ayw.sirius19@gmail.com> Co-authored-by: Khoa Pham <khoa.pham@radixark.ai> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> Co-authored-by: Zhichen Zeng <zczeng@uw.edu> Co-authored-by: Ke Bao <ispobaoke@gmail.com> (cherry picked from commit a6cf058)
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add the required V4.1 model, vision, Engram, compression-ratio and candidate-indexer support using 15 attributed upstream donor commits. Preserve stable scheduler, cache and graph-pool behavior; conflict adaptations are documented in the subsequent integration audit.
Part 1/11 of the DeepSeek V4.1 stable-source review stack. Base
codex/serving-v0520-upstream-r1at94602c9c2b7cbdb8efd5c52802dac6a1c180089e; head9af234a4891a8facbe92626a2c54fffce5f04637. Review this PR against that base to see only this problem. No merge or deployment is requested by this PR.The complete classified stack has tree
1fa53db30f547b951bb50119a88f2d4cb25c306f, byte-identical to frozen candidate9acba212f3376c215a9b48a44dcbace15c0982b7. Original donor authors and source trailers are retained. The standalone intermediate commit is not claimed to be an accepted deployment.Validation on the identical final source tree: final-image packaging/native/CPU contracts, 82 protocol selftest checks and 5 routed-ABI tests passed. The bitmap correction passed 24 native eager cases, 20 CUDA graph replays, and known-input sorting at 1M, scratch-tail and 2M addressable extents. Earlier prefill, padding, four-card finalize and HiCache evidence remains bound to its original candidate. Full model qualification of the latest image is in progress; known strict-JSON array-length and vision semantic failures are retained, not waived. No throughput gain or production acceptance is claimed.
Source commits in this PR:
5392a7324f23e0f1d960edb2c60e4fd9020978e9: dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap (dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap sgl-project/sglang#39677)6842c9135a8a2af8458f836bb81ac6ff83e33c32: dsv4.1: standalone kernels and Python wrappers (dsv4.1: standalone kernels and Python wrappers sgl-project/sglang#39646)ae829a492f29d50286ca62e524f66bdab7a314b3: dsv4.1: Top-k kernels and candidate selection (dsv4.1: Top-k kernels and candidate selection sgl-project/sglang#39648)2058fb0a106ef3e9a1431f6400d512333643f618: dsv4.1: communication kernels and wrappers (dsv4.1: communication kernels and wrappers sgl-project/sglang#39653)d25d72a68e5b240851a0b645b98582d7924b540f: dsv4.1: compression, KV I/O, and metadata kernels (dsv4.1: compression, KV I/O, and metadata kernels sgl-project/sglang#39652)df74a3ed2401723ff865b514f5bba23a5a0164e4: dsv4.1: candidate indexer library (dsv4.1: candidate indexer library sgl-project/sglang#39671)79a3ac5416bfa512b95d7e615c901eaf9dd63482: dsv4.1: RoPE and FP4 packing kernels (dsv4.1: RoPE and FP4 packing kernels sgl-project/sglang#39656)5919a29f559e8a8a83eab1ca8e2e858ff4a07f9e: dsv4.1: Hopper FP8 matmul kernels and tuning (dsv4.1: Hopper FP8 matmul kernels and tuning sgl-project/sglang#39657)a55765773e792cb3a7dbbf152a0968eb31c74354: dsv4.1: mHC computation and compensated projections (dsv4.1: mHC computation and compensated projections sgl-project/sglang#39664)d31c3fa098ffd37ff1b3bbe27219eb5f990e030e: dsv4.1: vision tower and image preprocessing (dsv4.1: vision tower and image preprocessing sgl-project/sglang#39668)137f96a5cc08d3fd74c5c6cc8055aec73c59af07: dsv4.1: chat encoding and tool parsing (dsv4.1: chat encoding and tool parsing sgl-project/sglang#39665)22d54075f900b0c78d677d8a28d4db855e274c4a: dsv4.1: Engram module and request history support (dsv4.1: Engram module and request history support sgl-project/sglang#39666)5648185cee4c1d0dc49525dc50d4bfab0fb9bc95: [DSV4] Generalize attention metadata, sparse prefill, and KV pool over compress ratios ([DSV4] Generalize attention metadata, sparse prefill, and KV pool over compress ratios sgl-project/sglang#39921)e53312611f87fe7fffa895c1e0259f3f7e665e07: [Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs ([Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs sgl-project/sglang#40039)9af234a4891a8facbe92626a2c54fffce5f04637: dsv4.1: remaining model and runtime integration (dsv4.1: remaining model and runtime integration sgl-project/sglang#38798)CI States
Latest PR Test (Base): ❌ Run #35490992289
Latest PR Test (Extra): ❌ Run #35490991873