Add flexible virtual pipeline parallel (fVPP) to hybrid model - #3377

Merged

duncanriach merged 44 commits into

NVIDIA:mainfrom

duncanriach:add-hybrid-fvpp-to-main-v1

Feb 27, 2026

Commits on Feb 26, 2026

Add flexible virtual pipeline parallel (fVPP)
duncanriach
committed
73e8701
Fix bug for PP=1
duncanriach
committed
85b953c
Fix bug related to MTP being on final PP stage
duncanriach
committed
15ef275
Improve comment
duncanriach
committed
a5f163c
Fixed linting errors
duncanriach
committed
ead6aee
Fix isort error
duncanriach
committed
ed207bd
Fix test
duncanriach
committed
e1c9d2e
Fix tests
duncanriach
committed
96acc0c
Fix progress_groups test
duncanriach
committed
37eba8c
Still able to use --num-layers with --hybrid-override-pattern
duncanriach
committed
9b920d3
Add better deprecation support to MambaModel
duncanriach
committed
4191701
Make new style consistent with old
duncanriach
committed
3446808
Fix linting
duncanriach
committed
c1ff791
Fix ModelOpt builder
duncanriach
committed
80f5ed3
More fixes for ModelOpt
duncanriach
committed
de4c555
Re-enable hybrid pp4 test
duncanriach
committed
9fbfaa5
Added backwards compatibility support for attention and mlp ratios
duncanriach
committed
53be6ef
Fix MambaModel parameter deprecation code
duncanriach
committed
a22f2f4
Fix functional test loader to handle pipe symbol
duncanriach
committed
d790107
Fix test
duncanriach
committed
3866244
Update golden values for hybrid pp=4 test
duncanriach
committed
8116705
Improve functional test handling of special symbols in layer patterns
duncanriach
committed
301ddbf
Add hybrid model functional test for pp2vpp2
duncanriach
committed
15f2854
Add golden values for hybrid model pp2vpp2 functional test
duncanriach
committed
ebd5ed4
Fix linting
duncanriach
committed
16cc06c
Fix loading of legacy hybriid model checkpoints
duncanriach
committed
f301b34
Cleanup previous commit
duncanriach
committed
6a104f4
Improve comment
duncanriach
committed
370f580
Enable different inference pipelining of pre-trained model
duncanriach
committed
7462f5d
Improve comment
duncanriach
committed
3bbd0be
Replace warnings.warn with log_single_rank
duncanriach
committed
11e36a4
Fix type hints
duncanriach
committed
9c522d4
Make variable access consistent
duncanriach
committed
08390f7
Improve type hint
duncanriach
committed
cb71210
Add layer pattern validation
duncanriach
committed
97447bd
Fix type hint
duncanriach
committed
444587a
Improved self-documenting of code
duncanriach
committed
bc56e01
Update examples/mamba
duncanriach
committed
f7b755e
(Re-)added backwards compatibility mode for hybrid model pipeline parallel
duncanriach
committed
6036cd6
Fix tests based on previous commit
duncanriach
committed
5d1cd21
Fix linting
duncanriach
committed
5981e2f
Add error check
duncanriach
committed
dd15b09
Merge branch 'main' into add-hybrid-fvpp-to-main-v1
duncanriach
authored
f1b62b0
Merge branch 'main' into add-hybrid-fvpp-to-main-v1
duncanriach
authored
aca6ca3