Skip to content

[review] [Perf][GDN] Mixed-step host diet (mirror of vllm-project/vllm#61034) - #4

Open
kimseunghyun-kr wants to merge 341 commits into
review-base-276fbcff2from
gdn-mixed-step-host-diet
Open

kimseunghyun-kr wants to merge 341 commits into
review-base-276fbcff2from
gdn-mixed-step-host-diet

Conversation

@kimseunghyun-kr

Copy link
Copy Markdown
Owner

Review-only PR inside the fork (not for merge). It mirrors upstream vllm-project#61034 so that Codex can audit the diff against upstream main 276fbcf. GitHub Actions is disabled on this fork.

What to check: the GDN prefill state index_select / masked_fill_ / index_copy_ path with int64 indices built once per step; the spec_tokens_first slices; strided q/k/v views to the recurrent update. The claim is bit-identical output (A100 parity tests pass). Please look for shape, dtype, aliasing or ordering cases where the output could differ, and for any missed hybrid-model caller (Qwen3-Next, Qwen3.5, Qwen4Exp).

Measured: C20 strict pass 92.75 % vs 91.70 % mean (3 x 60 min), C18 qualifies (97.81 / 96.44 / 96.55 %), py-spy GDN host share 24.4 -> 19.1 %, nsys mixed step 103.7 -> 92.6 ms.

AI assistance: prepared with Claude Code.

🤖 Generated with Claude Code

@kimseunghyun-kr

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T16:52:25.509575Z 9c4f4fa Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 9c4f4fac3f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

vMaroon and others added 27 commits October 12, 2026 21:28
vllm-project#61253)

Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
…TING_FOR_REMOTE_KVS (vllm-project#57705)

Signed-off-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com>
Co-authored-by: Akshay Viswanathan <4953343+akshayv@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
vllm-project#59314)

Signed-off-by: wxin <wxin@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…61213)

Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…erify, with empty-segment/padding hardening (vllm-project#56148)

Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com>
Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…roject#59070)

Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…t#61082)

Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
)

Signed-off-by: zengxian <xiangdong.zeng@intel.com>
Signed-off-by: xiangdong <40376367+zxd1997066@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
…vllm-project#61183)

Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…hat message parsing (vllm-project#56129)

Signed-off-by: pctablet505 <55033230+pctablet505@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…vllm-project#61299)

Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
…olution (vllm-project#61272)

Signed-off-by: Yuan Li <dskwelmcy@163.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
… response parser (vllm-project#61297)

Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
…ed pixels (vllm-project#61104)

Signed-off-by: godmook <cmoh4135@naver.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
…-project#57911)

Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: xiangdong <40376367+zxd1997066@users.noreply.github.com>
…ct#60322)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
)

Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
…llm-project#51309)

Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
…lm-project#61285)

Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…parse indexer (vllm-project#61149)

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…TP (vllm-project#61235)

Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: linitra24 <renshuang.zhou@daocloud.io>
…atically (vllm-project#61202)

Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…m-project#60602)

Signed-off-by: kimseunghyun <hellonypcteam@gmail.com>
Signed-off-by: kimseunghyun <incongnito12@gmail.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
talorabr and others added 30 commits October 15, 2026 11:07
…ardown (vllm-project#60279)

Signed-off-by: nej1gotnochill <divyanshlalotra2@gmail.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
…project#61262)

Signed-off-by: Micah Williamson <micah.williamson@amd.com>
…der output (vllm-project#61241)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
… requests (vllm-project#61427)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
…llm-project#61044)

Signed-off-by: Laith Sakka <lsakka@meta.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
…d3-lite, d3-edge) (vllm-project#61778)

Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com>
…ie sessions (vllm-project#58942)

Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Benedikt Falk <bfalk@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
…58689)

Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
…6005)

Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
…xt (vllm-project#61757)

Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
…ry reservation (vllm-project#59472)

Signed-off-by: Mo <mosafariuk@gmail.com>
…ows (vllm-project#61647)

Signed-off-by: Zheng Cai <caizheng1993@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
…llm-project#61643)

Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
…roject#59518)

Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
…ecycle (vllm-project#61789)

Signed-off-by: CharlesWu <Charles.Wu@amd.com>
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
…, 7 kernels -> 1, 6x speedup on all shapes for this kernel (vllm-project#58737)

Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
…-project#54627)

Signed-off-by: lifulu <fululi12@amd.com>
Signed-off-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: lifulu <lifulu@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…st (vllm-project#61648)

Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
…utCollector (vllm-project#58941)

Signed-off-by: David Cheung <d7cheung@gmail.com>
Co-authored-by: Benedikt Falk <bfalk@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
…r token budget (vllm-project#59468)

Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
… and non-gated) (vllm-project#61293)

Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
…igmoid, through AITER topk_gating (vllm-project#56226)

Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.