[dev] feat(mHC): Add basic pytorch implementation of manifold hyper connection(mHC). - #2943

Merged

Victarry merged 85 commits into

jingqiny-99:jingqiny/feature-mHC

Mar 6, 2026

Commits on Feb 2, 2026

upd: resolve conflicts
jingqiny-99
committed
720f39f
upd: resolve conflicts
jingqiny-99
committed
daaefa1
upd: resolve conflicts
jingqiny-99
committed
191112b
Update: remove useless codes
jingqiny-99
committed
60fe478
Fix: remove hyper connections in mtp layers
jingqiny-99
committed
5313101
Fix: bugs in hyper_connection.py

Jingqin Yang
authored and
jingqiny-99
committed
8753197
Feature: Block level checkpoint manager for mHC recomputations
jingqiny-99
committed
8a6f06c
Fix: bugs in test_random.py
jingqiny-99
committed
b5832df
Fix: bugs in test_random.py

Jingqin Yang
authored and
jingqiny-99
committed
0e28db1
update: Refine MHC Recompute Design Doc
jingqiny-99
committed
c0157c9
update: refine design doc for MHC recomputation
jingqiny-99
committed
004bfb5
upd: resolve conflicts
jingqiny-99
committed
3e37187
upd: resolve conflicts
jingqiny-99
committed
bc8bf78
update: Local merge

root
authored and
jingqiny-99
committed
1d60007
Fix: bugs in hyper connection module UTs

Jingqin Yang
authored and
jingqiny-99
committed
eff5d8d
resolve conflicts
jingqiny-99
committed
560ea12
Update: Unit test for transformer layer
jingqiny-99
committed
e7e8520
Fix: Non tensor variables to checkpoint
jingqiny-99
committed
15b46d3
upd: resolve conflicts
jingqiny-99
committed
09e722a
Fix: bugs in transformers

Jingqin Yang
authored and
jingqiny-99
committed
4f4110d
Fix: arguments for hyper connection recompute
jingqiny-99
committed
30fa8dd
upd: fix bugs
jingqiny-99
committed
37c81b4
Fix: should no grad in SinkhornKnopp forward
jingqiny-99
committed
7aa5740
Fix: Bugs in tblock

Jingqin Yang
authored and
jingqiny-99
committed
9e240d4
Fix: Hyper Connection + Sequence Parallel
jingqiny-99
committed
878be0e
Update: Move mhc block manager tests to stand-alone file
jingqiny-99
committed
407b807
rebase
jingqiny-99
committed
063505a
Fix conflicts
jingqiny-99
committed
c1d750f
add latency tests
jingqiny-99
committed
ef3ce5d
Fix bugs
jingqiny-99
committed
4b2ab6f
Fix: bugs in hyper connection _projection_and_rms
jingqiny-99
committed
797dd71
upd: remove recompute mappings, skip dropout when set to be 0.0
jingqiny-99
committed
84e156d
update test
jingqiny-99
committed
1f486ac
remove: norm layer in hyper connection
jingqiny-99
committed
43ad9ab
upd: fix bugs
jingqiny-99
committed
bc93585
upd: resolve rebase conflicts
jingqiny-99
committed
024d2cc
update: fix bugs in layer specs
jingqiny-99
committed
2bc7acb

Commits on Feb 4, 2026

upd: local merge
jingqiny-99
committed
926617e
Fix: pass pp_group instead of pp_rank and pp_size
jingqiny-99
committed
3e9187a
upd: resolve conflicts
jingqiny-99
committed
22bc89e
Fix: useless patterns
jingqiny-99
committed
6666653

Commits on Feb 6, 2026

Commits on Feb 9, 2026

Refactor: random.py and hyper connection
jingqiny-99
committed
4cd2b08

Commits on Feb 12, 2026

Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
2beaadd

Commits on Feb 13, 2026

Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
8be1626
upd: fix lint and copyright errors
Jingqin Yang
committed
e617dea
upd: FT for mhc
Jingqin Yang
committed
d2a59d8
upd: remove temp files
Jingqin Yang
committed
040ec3c
Fix: restore copyright.sh
Jingqin Yang
committed
627f03d
Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
00fd68e
upd: fix bugs in schedules.py
jingqiny-99
committed
5698c78
upd: try to fix ut error
jingqiny-99
committed
6b71925
upd: fix cuda graph bugs
jingqiny-99
committed
c5dc878
upd: autoformat
jingqiny-99
committed
6e615e4
resolve conflicts
jingqiny-99
committed
c75980d
upd: resolve conflicts
Jingqin Yang
committed
bd5d093
Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
e95fae3

Commits on Feb 14, 2026

upd: fix bugs

Jingqin Yang
authored and
jingqiny-99
committed
cc146fd
upd: fix mamba moe model
jingqiny-99
committed
1e66788
Update functional test cases and add golden values.
Dennis Liu
committed
cd68755

Commits on Feb 15, 2026

Minor revision of hyper connection.
Dennis Liu
committed
2d72aa4

Commits on Feb 26, 2026

Refactor hyper connection implementation.
Dennis Liu
committed
0b5bd26
update: new UTs for cuda graph / offloading / PP / VPP for mHC
jingqiny-99
committed
28bec70

Commits on Feb 28, 2026

update: Refactor and fix bugs
jingqiny-99
committed
fc27b34
Merge Dev into feature-mHC
jingqiny-99
committed
02516b2
update: lint
jingqiny-99
committed
d12c37e
Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
4fe98a2
upd: fix UT bugs
jingqiny-99
committed
cf2d676
Update: golden value for mHC FT
jingqiny-99
committed
701fb36

Commits on Mar 2, 2026

upd: remove sequence parallel check and readd torch.compile
jingqiny-99
committed
7ea5f6f
update FT for mHC
jingqiny-99
committed
4ff8abe
upd: fix bugs
jingqiny-99
committed
f7c63e2

Commits on Mar 3, 2026

Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
d86d741
Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
5522ebf

Commits on Mar 4, 2026

remove useless codes, update layernorm offload checks
jingqiny-99
committed
3b44787
fix: transformer config
jingqiny-99
committed
21eb1f3
Fix: transformer config
jingqiny-99
committed
18f66d1
upd: lint
jingqiny-99
committed
90f26ec

Commits on Mar 6, 2026

upd: fix claude review bugs
jingqiny-99
committed
fc861ab
Merge branch 'dev' into jingqiny/feature-mHC
jingqiny-99
authored
dc917a6
Fix: lint
jingqiny-99
committed
e7e1a13