Skip to content

feat(hybrid): pass token ids to hash-routed layers - #5945

Closed
FDecaYed wants to merge 1 commit into
NVIDIA:pull-request/5938from
FDecaYed:codex/hybrid-hash-routing-adapter
Closed

feat(hybrid): pass token ids to hash-routed layers#5945
FDecaYed wants to merge 1 commit into
NVIDIA:pull-request/5938from
FDecaYed:codex/hybrid-hash-routing-adapter

Conversation

@FDecaYed

Copy link
Copy Markdown
Contributor

What

Teach HybridModel/HybridStack to supply token IDs only to TransformerLayer consumers that use Hash MoE routing.

  • pass input_ids through the HybridModel forward path;
  • preserve the strict call signature for non-transformer layers;
  • keep selective/full recompute and local/TE CUDA-graph execution covered;
  • add focused HybridModel coverage without introducing GPTModel support.

This is a child of #5938. Its GitHub Files changed view contains only the HybridModel adapter and its focused test.

Provenance

Testing

  • isort --check-only, ruff, Black --check, Python compilation, and git diff --check passed for the slice.
  • Focused CUDA/Torch execution is delegated to PR CI because this local macOS host has no CUDA devices.

Thread input_ids through HybridModel and HybridStack only to TransformerLayer instances, including recompute and CUDA-graph paths, while preserving non-transformer signatures.

Signed-off-by: Deyu Fu <Deyu.Foo@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 21, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@FDecaYed

Copy link
Copy Markdown
Contributor Author

/ok to test 6f8a121

Copy link
Copy Markdown
Contributor Author

Closing this stacked adapter because it is absorbed into P4: one complete HybridModel-only Hash MoE feature rebuilt through #5938 after P2 merges. Tracking PR: #5795.

@copy-pr-bot
copy-pr-bot Bot deleted the branch NVIDIA:pull-request/5938 July 23, 2026 13:49
@copy-pr-bot copy-pr-bot Bot closed this Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant