Skip to content

Support packed sequences in Gated Delta Net - #85

Draft
yapdianang wants to merge 1 commit into
radixark:miles-mainfrom
yapdianang:dydx/gdn-packed-sequences
Draft

Support packed sequences in Gated Delta Net#85
yapdianang wants to merge 1 commit into
radixark:miles-mainfrom
yapdianang:dydx/gdn-packed-sequences

Conversation

@yapdianang

Copy link
Copy Markdown

Summary

Motivation

Qwen3.8-27B varlen training currently reaches the explicit GDN does not support packed sequence for now guard. Dense Qwen jobs do not exercise this path, which is why the same B300 stack worked for those models.

Testing

  • ruff check on all changed files
  • Black 24.4.2 check on all changed files
  • Python compile check on all changed files
  • focused pytest collection attempted; this development host does not have Transformer Engine installed, so GPU test collection stops at that dependency
  • B300 end-to-end validation pending a Miles image built from this commit

Upstream implementation: NVIDIA#2645

…VIDIA#2645)

Signed-off-by: yuzhongw <yuzhongw@nvidia.com>
Co-authored-by: kunlunl <kunlunl@nvidia.com>
Co-authored-by: Xuesong Ye <xuesongyey@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Daisy Gao <daisyg@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants