Add VarlenDataset family and wire it into GPT pretraining - #6680

Closed

ilml wants to merge 6 commits into

NVIDIA:mainfrom

ilml:split/3386-dataset-varlen

Commits on Aug 19, 2026

Add sequence_packing_scheduler config field, CLI arg, and validation

ilml
and
claude
committed
2d7566d
Add DP-balanced sequence-packing scheduler and wrap_data_iterator entry point

ilml
and
claude
committed
4d19446
Add mock SFT dataset and generalize SFT padding divisor

ilml
and
claude
committed
bb0e877
Integrate sequence-packing scheduler into training loop and GPT pretraining

ilml
and
claude
committed
65727ed
Add VarlenLowLevelDataset, VarlenDataset, and MockVarlenDataset

ilml
and
claude
committed
b6acb40
Wire varlen dataset into pretraining CLI, dataloader, and dataset provider

ilml
and
claude
committed
cd0a019