fix(train): skip optimizer and scheduler for eval-only

[EazyReal]

Replaces #1494. Twin of THUDM/slime#2296.

Desired vs bug

Eval-only is intended. train.py already has:

# special case for eval-only
if args.num_rollout == 0 and args.eval_interval is not None:
    await rollout_manager.eval.remote(rollout_id=0)

The train loop is range(..., num_rollout), so this correctly does zero training steps. Eval still needs the actor loaded and update_weights() into the rollout engines, so create_training_models runs first.

The bug is that bring-up always constructed the optimizer and LR scheduler. With --num-rollout 0, train_iters is 0 → lr_decay_steps is 0, and both Megatron and FSDPLRScheduler assert lr_decay_steps > 0. You never reach the eval special case.

Why not train_iters = 1

That was #1494. It lies to the scheduler, then still runs zero train steps. Eval-only does not train, so it should not build a train stack.

Change

• Megatron setup_model_and_optimizer: after get_model, if num_rollout == 0, set no_load_optim and return (model, None, None). Same pattern as --debug-disable-optimizer.
• FSDP actor: skip AdamW / LR scheduler; offload and checkpoint load ignore a null optimizer.
• create_training_models: do not allocate a critic when num_rollout == 0.

num_rollout > 0 is unchanged.

Test plan

• --num-rollout 0 --eval-interval 1 loads the actor, skips optimizer/scheduler, hits rollout_manager.eval
• A normal Megatron and FSDP training run still builds optimizer + scheduler

Made with Cursor

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.