Add DP-balanced sequence-packing scheduler and integrate it into GPT pretraining - #6679

Closed

ilml wants to merge 4 commits into

NVIDIA:mainfrom

ilml:split/3386-dcp-scheduler

Commits on Aug 19, 2026

Add sequence_packing_scheduler config field, CLI arg, and validation

ilml
and
claude
committed
2d7566d
Add DP-balanced sequence-packing scheduler and wrap_data_iterator entry point

ilml
and
claude
committed
4d19446
Add mock SFT dataset and generalize SFT padding divisor

ilml
and
claude
committed
bb0e877
Integrate sequence-packing scheduler into training loop and GPT pretraining

ilml
and
claude
committed
65727ed