Skip to content

Support Hunyuan Dense V1 export - #2189

Merged
kunal-vaishnavi merged 4 commits into
mainfrom
sunghcho/hunyuan
Jun 13, 2026
Merged

Support Hunyuan Dense V1 export#2189
kunal-vaishnavi merged 4 commits into
mainfrom
sunghcho/hunyuan

Conversation

@hanbitmyths

Copy link
Copy Markdown
Collaborator

Summary

  • Update the Hunyuan Dense V1 builder to bake Hunyuan dynamic NTK-alpha RoPE into rope_theta.
  • Clear generic RoPE scaling metadata after baking the effective theta so the base builder does not apply another scaling path.
  • Alias Hunyuan query/key layernorm modules to the existing q_norm/k_norm export path when needed.

Validation

  • Ran py_compile for src/python/py/models/builders/hunyuan.py.
  • Ran git diff --check for src/python/py/models/builders/hunyuan.py.

Copilot AI review requested due to automatic review settings May 31, 2026 06:33
@hanbitmyths
hanbitmyths requested a review from a team as a code owner May 31, 2026 06:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the HunYuan Dense V1 model builder to better support export across Transformers config variants by baking Hunyuan dynamic NTK-alpha RoPE scaling into rope_theta and adapting Q/K norm module aliases.

Changes:

  • Reads RoPE metadata from either rope_scaling or rope_parameters.
  • Bakes Hunyuan alpha scaling into config.rope_theta and clears generic scaling metadata.
  • Handles q_norm/k_norm aliases when actual modules are exposed as query_layernorm/key_layernorm.

Comment thread src/python/py/models/builders/hunyuan.py
@kunal-vaishnavi

Copy link
Copy Markdown
Contributor

/azp run Integration Tests

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@kunal-vaishnavi
kunal-vaishnavi enabled auto-merge (squash) June 13, 2026 22:50
@kunal-vaishnavi
kunal-vaishnavi merged commit ba22948 into main Jun 13, 2026
63 of 65 checks passed
@kunal-vaishnavi
kunal-vaishnavi deleted the sunghcho/hunyuan branch June 13, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants