Repository navigation
[review] [Perf][GDN] Mixed-step host diet (mirror of vllm-project/vllm#61034) - #4
Open
kimseunghyun-kr wants to merge 341 commits into
Open
kimseunghyun-kr wants to merge 341 commits into
kimseunghyun-kr wants to merge 341 commits into
Conversation
Owner
Author
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
vllm-project#61253) Signed-off-by: Maroon Ayoub <mayoub@redhat.com> Co-authored-by: Flora Feng <4florafeng@gmail.com>
…TING_FOR_REMOTE_KVS (vllm-project#57705) Signed-off-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com> Co-authored-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
vllm-project#59314) Signed-off-by: wxin <wxin@nvidia.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…61213) Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com> Signed-off-by: Misha Goin <mgoin64@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…erify, with empty-segment/padding hardening (vllm-project#56148) Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com> Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com> Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…roject#59070) Signed-off-by: whx-sjtu <xiaowang990929@gmail.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…t#61082) Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…59039) Signed-off-by: Frida Andersson <fanderss@amd.com>
…vllm-project#61183) Signed-off-by: Dakai An <dakaian108@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…hat message parsing (vllm-project#56129) Signed-off-by: pctablet505 <55033230+pctablet505@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…vllm-project#61299) Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>
…olution (vllm-project#61272) Signed-off-by: Yuan Li <dskwelmcy@163.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
… response parser (vllm-project#61297) Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>
…ed pixels (vllm-project#61104) Signed-off-by: godmook <cmoh4135@naver.com> Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
…-project#57911) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: xiangdong <40376367+zxd1997066@users.noreply.github.com>
…ct#60322) Signed-off-by: Divakar Verma <divakar.verma@amd.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…llm-project#51309) Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Signed-off-by: Tianyu Guo <guoty@inferact.ai> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com>
…lm-project#61285) Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…parse indexer (vllm-project#61149) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…TP (vllm-project#61235) Signed-off-by: aoshen524 <aoshen524@gmail.com> Co-authored-by: aoshen524 <aoshen524@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
…atically (vllm-project#61202) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…m-project#60602) Signed-off-by: kimseunghyun <hellonypcteam@gmail.com> Signed-off-by: kimseunghyun <incongnito12@gmail.com> Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: kimseunghyun <hellonypcteam@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
…env var (vllm-project#60630) Signed-off-by: talora <talora@nvidia.com>
…ardown (vllm-project#60279) Signed-off-by: nej1gotnochill <divyanshlalotra2@gmail.com> Co-authored-by: Codebuff <noreply@codebuff.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
…project#61262) Signed-off-by: Micah Williamson <micah.williamson@amd.com>
…der output (vllm-project#61241) Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
… requests (vllm-project#61427) Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
…llm-project#61044) Signed-off-by: Laith Sakka <lsakka@meta.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
…d3-lite, d3-edge) (vllm-project#61778) Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
…ie sessions (vllm-project#58942) Signed-off-by: David Cheung <d7cheung@gmail.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Benedikt Falk <bfalk@nvidia.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
…58689) Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com> Co-authored-by: Shanshan Shen <467638484@qq.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
…6005) Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com> Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com> Co-authored-by: Shanshan Shen <467638484@qq.com>
…xt (vllm-project#61757) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
…ry reservation (vllm-project#59472) Signed-off-by: Mo <mosafariuk@gmail.com>
…ows (vllm-project#61647) Signed-off-by: Zheng Cai <caizheng1993@gmail.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
…llm-project#61643) Signed-off-by: Ronald1995 <ronaldautomobile@163.com> Co-authored-by: aoshen02 <aoshen@inferact.ai>
…project#55090) Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
…roject#59518) Signed-off-by: Yin Li <kevinli.ai.work@gmail.com> Co-authored-by: Yin Li <kevinli.ai.work@gmail.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
…ecycle (vllm-project#61789) Signed-off-by: CharlesWu <Charles.Wu@amd.com> Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
…, 7 kernels -> 1, 6x speedup on all shapes for this kernel (vllm-project#58737) Signed-off-by: lifulu <fululi12@amd.com> Signed-off-by: lifulu <lifulu@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: lifulu <lifulu@users.noreply.github.com>
…-project#54627) Signed-off-by: lifulu <fululi12@amd.com> Signed-off-by: lifulu <lifulu@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: lifulu <lifulu@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…st (vllm-project#61648) Signed-off-by: Oxana Korzh <okorzh@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com>
…llm-project#61825) Signed-off-by: Kevin <jia__kuan@163.com>
…utCollector (vllm-project#58941) Signed-off-by: David Cheung <d7cheung@gmail.com> Co-authored-by: Benedikt Falk <bfalk@nvidia.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io> Co-authored-by: Flora Feng <4florafeng@gmail.com>
…r token budget (vllm-project#59468) Signed-off-by: zixi-qi <zixi@inferact.ai> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
… and non-gated) (vllm-project#61293) Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
…igmoid, through AITER topk_gating (vllm-project#56226) Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Review-only PR inside the fork (not for merge). It mirrors upstream vllm-project#61034 so that Codex can audit the diff against upstream main 276fbcf. GitHub Actions is disabled on this fork.
What to check: the GDN prefill state index_select / masked_fill_ / index_copy_ path with int64 indices built once per step; the spec_tokens_first slices; strided q/k/v views to the recurrent update. The claim is bit-identical output (A100 parity tests pass). Please look for shape, dtype, aliasing or ordering cases where the output could differ, and for any missed hybrid-model caller (Qwen3-Next, Qwen3.5, Qwen4Exp).
Measured: C20 strict pass 92.75 % vs 91.70 % mean (3 x 60 min), C18 qualifies (97.81 / 96.44 / 96.55 %), py-spy GDN host share 24.4 -> 19.1 %, nsys mixed step 103.7 -> 92.6 ms.
AI assistance: prepared with Claude Code.
🤖 Generated with Claude Code