Skip to content

Eval metrics: a group with an aborted episode no longer crashes the zero-std count - #4518

Draft
dzorlu wants to merge 1 commit into
radixark:mainfrom
dzorlu:pr10-zero-std-skips-aborted-groups-v2
Draft

dzorlu wants to merge 1 commit into
radixark:mainfrom
dzorlu:pr10-zero-std-skips-aborted-groups-v2

Conversation

@dzorlu

@dzorlu dzorlu commented Oct 29, 2026 •

Copy link
Copy Markdown

Problem. An agentic episode whose environment never opened comes back ABORTED with reward None. _compute_zero_std_metrics rounds each group's reward to build its count key, so one such group raises TypeError and the whole eval pass fails.

groups: A = [None, None] (both aborted), B = [1.0, 1.0]
now:   round(None) -> TypeError, no eval metrics written
fixed: A is skipped: zero_std/count_1.0 = 1, zero_std/all_one_percentage = 0.5, no zero_std/count_None

Fix. _is_zero_std returns False for a group that has a None reward, so the group is left out of the count.

     def _is_zero_std(samples: list[Sample]):
         rewards = [sample.get_reward_value(args) for sample in samples]
+        if any(reward is None for reward in rewards):
+            return False
         return len(rewards) == 0 or all(rewards[0] == r for r in rewards)

Tests. tests/fast/ray/rollout/test_metrics.py::test_zero_std_metrics_skip_a_group_whose_episode_aborted.

Without the fix, the new test fails with TypeError: type NoneType doesn't define __round__ method at miles/ray/rollout/metrics.py:324. On CPU, tests/fast/ray/rollout has 0 new failures against main (the same 5 tests fail on both: 2 in test_engine_addressing.py, 3 in test_inference_runtime_topology.py).

Not yet proven in a run: no run so far had an aborted group reach eval scoring with this change.

🤖 Generated with Claude Code

Verified on GPU (2026-10-31)

Qwen3.8-27B VL on Megatron, 4 nodes of 8x B300, computer-use episodes through the TITO session server (screenshots, context folding on), 3 train steps with an eval pass between steps: all 3 eval passes completed (32 episodes each). Group cancel fired twice during training ("cancelled 4 running sibling episode(s)" and 7), but no eval group held a cancelled episode, so the case this PR fixes was not exercised. Not yet verified on GPU; the CPU tests cover it.

An agentic episode whose hosted environment never opened comes back ABORTED
with reward None (generate_hub.agentic_tool_call); _compute_zero_std_metrics
rounded that None and the whole eval raised. A group with a None reward is
neither all-easy nor all-hard, so it is left out of the count.

(cherry picked from commit bd9ad97)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant