[Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection - #4531

Merged

Connor-XY merged 22 commits into

NVIDIA:mainfrom

Connor-XY:mhc-pr1-core

Aug 18, 2026

Commits on Aug 17, 2026

[Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection

committed
feba22c
test(mHC): make test_transformer_layer self-contained for split #1
Connor-XY
committed
2dd457d
test(mHC): update test_hybrid_moe_model GOLDEN_CONFIG for new mHC fields
Connor-XY
committed
1a31f14
test(mHC): add direct coverage for BDA recompute path and block-level forward
Connor-XY
committed
f1d51a5
style: collapse short attention_mask to one line for black
Connor-XY
committed
3acfdb0
refactor(mHC): address mathemakitten review on PR #4531

Connor-XY
and
claude
committed
c453dc7
refactor(mHC): drop dead is_last_layer_in_recompute_block kwarg pop

Connor-XY
and
claude
committed
210f189
refactor(mHC): extract attention hooks to dedup _forward_attention

Connor-XY
and
claude
committed
aa12d2d
refactor(mHC): thread h_res/h_post via attn_state slot, not instance attrs

Connor-XY
and
claude
committed
cdb576c
style(mHC): restore _set_proj_residual context comment

Connor-XY
and
claude
committed
07365cb
refactor(mHC): extract MLP hooks to dedup _forward_mlp

Connor-XY
and
claude
committed
41e0e17
style(mHC): shorten docstring line to fit 100-char pylint limit

Connor-XY
and
claude
committed
143b9cf
fix(mHC): only thread mhc_recompute_manager when manager exists

Connor-XY
and
claude
committed
481138b
refactor(mHC): add back-compat shim for renamed _forward_post_mlp hook

Connor-XY
and
claude
committed
009de20
test(mHC): pass pipeline rank to offload manager
Connor-XY
committed
096064e
fix(mhc): update CUDA graph module API

FDecaYed
authored and
Connor-XY
committed
f51948a
refactor(mHC): address review feedback
Connor-XY
committed
c8cda51
fix(mHC): drop a bad rebase hunk and guard unsupported mHC configs
Connor-XY
committed
dd54c69
refactor(mHC): drop dead code and move remaining guards to config/init
Connor-XY
committed
2e5bf78
test(mHC): make the full-recompute guard case reach the mHC check
Connor-XY
committed
b57eba5
fix(mHC): do not reject MoE at config level
Connor-XY
committed
700a9ef
test(mHC): give the PP-guard case a pipeline_dtype
Connor-XY
committed
cc80c69

Navigation Menu

Uh oh!

[Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection - #4531

[Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection#4531
Connor-XY merged 22 commits into
NVIDIA:mainfrom
Connor-XY:mhc-pr1-core

Commits on Aug 17, 2026

[Split 1/N of #3430] feat(mHC): basic pytorch implementation of manifold hyper connection

test(mHC): make test_transformer_layer self-contained for split #1

test(mHC): update test_hybrid_moe_model GOLDEN_CONFIG for new mHC fields

test(mHC): add direct coverage for BDA recompute path and block-level forward

style: collapse short attention_mask to one line for black

refactor(mHC): address mathemakitten review on PR #4531

refactor(mHC): drop dead is_last_layer_in_recompute_block kwarg pop

refactor(mHC): extract attention hooks to dedup _forward_attention

refactor(mHC): thread h_res/h_post via attn_state slot, not instance attrs

style(mHC): restore _set_proj_residual context comment

refactor(mHC): extract MLP hooks to dedup _forward_mlp

style(mHC): shorten docstring line to fit 100-char pylint limit

fix(mHC): only thread mhc_recompute_manager when manager exists

refactor(mHC): add back-compat shim for renamed _forward_post_mlp hook

test(mHC): pass pipeline rank to offload manager

fix(mhc): update CUDA graph module API

refactor(mHC): address review feedback

fix(mHC): drop a bad rebase hunk and guard unsupported mHC configs

refactor(mHC): drop dead code and move remaining guards to config/init

test(mHC): make the full-recompute guard case reach the mHC check

fix(mHC): do not reject MoE at config level

test(mHC): give the PP-guard case a pipeline_dtype