[zupengwang]

What does this PR do?

V1 batch balancing can construct synthetic samples with an all-zero response_mask. When a top-k distillation microbatch contains only these samples, the current metric code calls min()/max() on empty tensors and aborts training before backward. Guarding only those reductions is insufficient: omitting padding observations also gives different metric counts across DP ranks, which Metric.aggregate_dp currently rejects.

Represent absent observations using empty lists and empty Metric instances, and aggregate MIN/MAX over the observations that exist across DP ranks. If every rank has no observation, the DP extremum is undefined (NaN). SUM/MEAN retain their existing aligned-count requirement. The loss, normalization, backward participation, and valid-token metric calculations are preserved. The license checker also accepts the 2026 individual-contributor header used by the new tests (two lines).

Duplicate-work check

Searched open and closed PRs for distillation + empty, student_mass, all-masked, and empty metrics, and inspected relevant patches. Rechecked immediately before publication; the related patches below are unchanged.

• [misc] fix: reduce_metrics crash on empty lists and tighten max/min k… #7361 changes reduce_metrics and Metric._aggregate; it does not guard distillation tensor reductions or support unequal observation counts in Metric.aggregate_dp. This patch leaves both of those functions unchanged.
• [megatron] fix: compute distillation mass metrics before clamping #7723 changes the order of clamping and mass measurement in the Megatron kernel.
• [algo, fsdp, megatron, veomni] feat: add tail-aware top-k KL for on-policy distillation #7855 adds tail-aware top-k KL; its common loss changes do not handle empty response masks.
• test(distillation): add value-level CPU tests for forward_kl_topk OPD loss #6822 adds value-level kernel tests. These new regressions cover the production loss callback, synthetic V1 padding, and worker aggregation behavior.

Test

Validated commit: 37836a40bdc265e5f4bff615023946b90f744bb4, based on c8faa0870a115b7bc73ee76545b804f5ffb15c5b.

Remote Linux host with RTX 4090; Python 3.12.14, PyTorch 2.13.0+cu130, Transformers 5.15.1, TensorDict 0.10.0.

CUDA_VISIBLE_DEVICES= python -m pytest -q \
  tests/workers/test_distillation_empty_response_on_cpu.py \
  tests/utils/test_metric_empty_extrema_on_cpu.py \
  tests/trainer/ppo/test_metric_utils_on_cpu.py \
  tests/trainer/ppo/test_value_loss_normalization_on_cpu.py \
  tests/workers/test_distillation_topk_symmetry_on_cpu.py

• 90 passed. New tests alone against unmodified base: 14 failed, 6 passed. Failures are empty tensor/array extrema and mismatched DP observation counts; controls on real samples pass.
• A tiny random Qwen2 model on RTX 4090, with the production FSDP top-k kernel and distillation loss callback: FP32/BF16 × ordinary/chunked top-k, three SGD updates each. Adding a synthetic padding microbatch preserves gradients and updated parameters exactly, while the padding loss remains differentiable zero.
• Two real CPU/Gloo processes, one entirely padded: the production allgather_dict_into_dict plus worker reductions preserve all real-sample metrics.
• All 14 configured hook commands passed using an equivalent local-hook configuration with pinned Ruff 0.12.2 and mypy 1.17.0. The original isolated hook bootstrap timed out while fetching GitHub; the repository configuration is unchanged. The local configuration preserves the configured hook commands, arguments, and file filters, while using installed tools instead of bootstrapping isolated hook environments.

GPU evidence is for the model/loss path on one GPU; it does not establish multi-GPU FSDP/NCCL behavior, end-to-end rollout training, convergence, performance, or other engine support.

AI assistance

Prepared and tested with OpenAI Codex assistance; submitted at the author's request.

API and usage

No user-facing API or configuration changes. Existing top-k distillation training automatically handles padding-only microbatches.

Submission checklist

• Read the contribution guide and checked for duplicate work.
• Added regression tests discovered by the existing *_on_cpu.py CPU workflow.
• No user-facing documentation or recipe-submodule update is needed.
• Original isolated pre-commit bootstrap: blocked by GitHub fetch timeout; all 14 equivalent local hook commands passed as described above.
• Maintainer CI authorization, if required.