[fix] fix sample type non numeric by guapisolo · Pull Request #966 · radixark/miles

guapisolo · 2026-04-10T04:21:25Z

The default value of sglang weight version is "default" rather than a numeric value. See https://github.com/sgl-project/sglang/blob/e77bfba24d892563fb2d91192e8841b0c59c7828/python/sglang/srt/server_args.py#L435.

When enabled debug rollout only, there is no explicit weight version so it will fail sglang e2e tests.

Fix this CI error. https://github.com/radixark/miles/actions/runs/24220053896/job/70709321587

Traceback (most recent call last):
  File "/__w/miles/miles/train.py", line 109, in <module>
    asyncio.run(train(args))
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/__w/miles/miles/train.py", line 73, in train
    rollout_data_ref = await rollout_manager.generate.remote(rollout_id)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ray.exceptions.RayTaskError(ValueError): ray::RolloutManager.generate() (pid=9882, ip=172.18.0.2, actor_id=995f53213b22eb599ff8dc8f02000000, repr=<miles.ray.rollout.RolloutManager object at 0x77e5b77202f0>)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/miles/miles/miles/ray/rollout.py", line 453, in generate
    _log_rollout_data(rollout_id, self.args, data, metrics, time.time() - start_time)
  File "/__w/miles/miles/miles/ray/rollout.py", line 1189, in _log_rollout_data
    log_dict |= dict_add_prefix(compute_metrics_from_samples(args, samples), "rollout/")
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/miles/miles/miles/ray/rollout.py", line 1209, in compute_metrics_from_samples
    oldest_versions = [s.oldest_weight_version for s in samples if s.oldest_weight_version is not None]
                                                                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/miles/miles/miles/utils/types.py", line 219, in oldest_weight_version
    return min(int(v) for v in self.weight_versions)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/miles/miles/miles/utils/types.py", line 219, in <genexpr>
    return min(int(v) for v in self.weight_versions)
               ^^^^^^
ValueError: invalid literal for int() with base 10: 'default'

gemini-code-assist

Code Review

This pull request updates the oldest_weight_version property to handle non-numeric versions by ignoring them and returning None when no valid integers are found. Unit tests were added to verify this behavior. Feedback was provided to simplify the implementation using a generator expression and the default parameter in the min function to avoid creating intermediate lists.

gemini-code-assist · 2026-04-10T04:27:43Z

+        versions = []
+        for version in self.weight_versions:
+            try:
+                versions.append(int(version))
+            except (TypeError, ValueError):
+                continue
+        return min(versions) if versions else None


The current implementation of oldest_weight_version is quite verbose and creates an intermediate list. It can be simplified using a generator expression and the default parameter of the min function for better readability and efficiency.

numeric_versions = (int(v) for v in self.weight_versions if str(v).isdigit()) return min(numeric_versions, default=None)

Add a second execution backend to the OpenEnv TB2 adapter: instead of a shared env server on a Docker host, every episode can run in its OWN Daytona cloud sandbox, built from the task's official image (task.toml docker_image) plus a tbench2_env server layer, and deleted when the episode ends. Same per-task image fidelity as docker mode with zero resident infrastructure (no Docker socket, no server to size or babysit) and zero cross-episode state leakage. Backend selection is episode-scoped in _multi_turn: set OPENENV_TB2_TASKS_DIR to a terminal-bench-2 checkout and each episode declaratively builds its sandbox from the task's Image definition (layer-cached by definition hash: first episode of a task ~10 min, repeats ~1 min; no named snapshots, so no org snapshot quota; sandboxes carry an openenv-tbench2-task=<task_id> label for safe sweeping in a shared org). Unset -> the existing OPENENV_ENV_URL shared-server path, behaviorally unchanged (the shared leg now reads OPENENV_ENV_URL at its own use site instead of threading it through _multi_turn's signature). The two backends deliberately score differently, each matching what its server provides: the shared-server leg keeps the adapter-driven canonical exec + reward-marker parse (compensates for an UNMODIFIED upstream server); the per-task leg uses the standard evaluate action, because the sandbox recipe (tbench2_env.task_snapshots -- tbench2_env is OpenEnv's Terminal-Bench-2 env package, envs/tbench2_env in huggingface/openenv; fixes proposed upstream in openenv#965 + #966) bakes a patched server that runs the same canonical tests/test.sh natively and resolves the task WORKDIR server-side (so no _apply_workdir prefix on this leg either -- task WORKDIRs are not uniformly /app: fix-git, prove-plus-comm). Sandbox creation is throttled process-wide with jittered backoff (Daytona rate-limits creates). Ships with the two tools that produced the validation evidence, both driving the adapter's own code paths so what they check is what training runs: scan_golden.py -- infra baseline without any LLM: replay each task's OFFICIAL solution/solve.sh (oracle-faithful: staged at /solution, DEBIAN_FRONTEND=noninteractive, task.toml [solution].env exported, task workdir cwd) through the same sandbox + evaluate scoring, expecting 1.0; --logs captures solve.log/test-log tails below 1.0. eval_tbench2_via_api.py -- the EXACT training agent-env loop (_multi_turn) with any OpenAI-compatible API standing in for the policy: no GPU, no Ray/Megatron. For end-to-end smokes and for measuring base-model solve rates when picking a variance-band training subset. plus offline unit tests (tests/, run manually: pytest examples/experimental/openenv/tests/ -q -- not collected by the repo-level suite) for the two things a live episode cannot cheaply prove: backend dispatch on both legs, and the create-throttling retry/backoff/give-up behavior that only triggers under production rate limits. Validation: - per-task leg: golden sweep passes 82/89 of the TB2 suite (0 infra errors; the 7 remaining have upstream-broken solutions); golden regression on this exact merge (fix-git, chess-best-move, circuit-fibsqrt all 1.0 -- covering per-task WORKDIR dispatch, apt paths, long canonical evals); DeepSeek end-to-end episodes through _multi_turn solve openssl-selfsigned-cert with reward 1.0. - shared-server leg: dispatch unit test asserts behavior identical to before this change (exec prefixed with cd /app, canonical-exec scoring, rm-hack present). - unit tests: 6 passed (dispatch both legs, throttle retry/give-up/ passthrough, throttle-error classification). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add a second execution backend to the OpenEnv TB2 adapter: instead of a shared env server on a Docker host, every episode can run in its OWN Daytona cloud sandbox, built from the task's official image (task.toml docker_image) plus a tbench2_env server layer, and deleted when the episode ends. Same per-task image fidelity as docker mode with zero resident infrastructure (no Docker socket, no server to size or babysit) and zero cross-episode state leakage. Backend selection is episode-scoped in _multi_turn: set OPENENV_TB2_TASKS_DIR to a terminal-bench-2 checkout and each episode declaratively builds its sandbox from the task's Image definition (layer-cached by definition hash: first episode of a task ~10 min, repeats ~1 min; no named snapshots, so no org snapshot quota; sandboxes carry an openenv-tbench2-task=<task_id> label for safe sweeping in a shared org). Unset -> the existing OPENENV_ENV_URL shared-server path, behaviorally unchanged (the shared leg now reads OPENENV_ENV_URL at its own use site instead of threading it through _multi_turn's signature). The two backends deliberately score differently, each matching what its server provides: the shared-server leg keeps the adapter-driven canonical exec + reward-marker parse (compensates for an UNMODIFIED upstream server); the per-task leg uses the standard evaluate action, because the sandbox recipe (tbench2_env.task_snapshots -- tbench2_env is OpenEnv's Terminal-Bench-2 env package, envs/tbench2_env in huggingface/openenv; fixes proposed upstream in openenv#965 + #966) bakes a patched server that runs the same canonical tests/test.sh natively and resolves the task WORKDIR server-side (so no _apply_workdir prefix on this leg either -- task WORKDIRs are not uniformly /app: fix-git, prove-plus-comm). Sandbox creation is throttled process-wide with jittered backoff (Daytona rate-limits creates). Ships with the two tools that produced the validation evidence, both driving the adapter's own code paths so what they check is what training runs: scan_golden.py -- infra baseline without any LLM: replay each task's OFFICIAL solution/solve.sh (oracle-faithful: staged at /solution, DEBIAN_FRONTEND=noninteractive, task.toml [solution].env exported, task workdir cwd) through the same sandbox + evaluate scoring, expecting 1.0; --logs captures solve.log/test-log tails below 1.0. eval_tbench2_via_api.py -- the EXACT training agent-env loop (_multi_turn) with any OpenAI-compatible API standing in for the policy: no GPU, no Ray/Megatron. For end-to-end smokes and for measuring base-model solve rates when picking a variance-band training subset. plus offline unit tests (tests/, run manually: pytest examples/experimental/openenv/tests/ -q -- not collected by the repo-level suite) for the two things a live episode cannot cheaply prove: backend dispatch on both legs, and the create-throttling retry/backoff/give-up behavior that only triggers under production rate limits. Validation: - per-task leg, no LLM (scan_golden.py): golden sweep passes 82/89 of the TB2 suite (0 infra errors; the 7 remaining have upstream-broken solutions); golden regression on this exact merge (fix-git, chess-best-move, circuit-fibsqrt all 1.0 -- covering per-task WORKDIR dispatch, apt paths, long canonical evals). - per-task leg, end-to-end with a live API policy (eval_tbench2_via_api.py): DeepSeek drives the full training loop through _multi_turn (sandbox create -> reset -> multi-turn exec -> evaluate -> delete) and solves openssl-selfsigned-cert with reward 1.0 -- run twice, including once after the final signature refactor; this also exercises the eval tool itself. - shared-server leg: dispatch unit test asserts behavior identical to before this change (exec prefixed with cd /app, canonical-exec scoring, rm-hack present). - unit tests: 7 passed (dispatch both legs; throttle retry/give-up/passthrough; throttle-error classification incl. the typed DaytonaRateLimitError path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add a second execution backend to the OpenEnv TB2 adapter: instead of a shared env server on a Docker host, every episode can run in its OWN Daytona cloud sandbox, built from the task's official image (task.toml docker_image) plus a tbench2_env server layer, and deleted when the episode ends. Same per-task image fidelity as docker mode with zero resident infrastructure (no Docker socket, no server to size or babysit) and zero cross-episode state leakage. Backend selection is episode-scoped in _multi_turn: set OPENENV_TB2_TASKS_DIR to a terminal-bench-2 checkout and each episode declaratively builds its sandbox from the task's Image definition (layer-cached by definition hash: first episode of a task ~10 min, repeats ~1 min; no named snapshots, so no org snapshot quota; sandboxes carry an openenv-tbench2-task=<task_id> label for safe sweeping in a shared org). Unset -> the existing OPENENV_ENV_URL shared-server path, behaviorally unchanged (the shared leg now reads OPENENV_ENV_URL at its own use site instead of threading it through _multi_turn's signature). The two backends deliberately score differently, each matching what its server provides: the shared-server leg keeps the adapter-driven canonical exec + reward-marker parse (compensates for an UNMODIFIED upstream server); the per-task leg uses the standard evaluate action, because the sandbox recipe (tbench2_env.task_snapshots -- tbench2_env is OpenEnv's Terminal-Bench-2 env package, envs/tbench2_env in huggingface/openenv; fixes proposed upstream in openenv#965 + #966) bakes a patched server that runs the same canonical tests/test.sh natively and resolves the task WORKDIR server-side (so no _apply_workdir prefix on this leg either -- task WORKDIRs are not uniformly /app: fix-git, prove-plus-comm). Sandbox creation is throttled process-wide with jittered backoff (Daytona rate-limits creates). Ships with the two tools that produced the validation evidence, both driving the adapter's own code paths so what they check is what training runs: scan_golden.py -- infra baseline without any LLM: replay each task's OFFICIAL solution/solve.sh (oracle-faithful: staged at /solution, DEBIAN_FRONTEND=noninteractive, task.toml [solution].env exported, task workdir cwd) through the same sandbox + evaluate scoring, expecting 1.0; --logs captures solve.log/test-log tails below 1.0. eval_tbench2_via_api.py -- the EXACT training agent-env loop (_multi_turn) with any OpenAI-compatible API standing in for the policy: no GPU, no Ray/Megatron. For end-to-end smokes and for measuring base-model solve rates when picking a variance-band training subset. plus offline unit tests (tests/, run manually: pytest examples/experimental/openenv/tests/ -q -- not collected by the repo-level suite) for the two things a live episode cannot cheaply prove: backend dispatch on both legs, and the create-throttling retry/backoff/give-up behavior that only triggers under production rate limits. Validation: - per-task leg, no LLM (scan_golden.py): golden sweep passes 82/89 of the TB2 suite (0 infra errors; the 7 remaining have upstream-broken solutions); golden regression on this exact merge (fix-git, chess-best-move, circuit-fibsqrt all 1.0 -- covering per-task WORKDIR dispatch, apt paths, long canonical evals). - per-task leg, end-to-end with a live API policy (eval_tbench2_via_api.py): full 89-task sweep with DeepSeek as the policy (single sample per task, 30-turn cap, up to 38 concurrent episodes) -- 33/89 solved, 0 systematic infra failures (10 non-scoring episodes: 8 heavyweight-task timeouts = reward 0 under training semantics, 1 transient registry network error, 1 sandbox-resource overrun). - shared-server leg: dispatch unit test asserts behavior identical to before this change (exec prefixed with cd /app, canonical-exec scoring, rm-hack present). - unit tests: 7 passed (dispatch both legs; throttle retry/give-up/passthrough; throttle-error classification incl. the typed DaytonaRateLimitError path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The per-task leg reached into the recipe module's private _make_daytona; openenv PR #966 promotes it to public API (same env-var contract: DAYTONA_API_KEY, optional DAYTONA_API_URL), so use the public name and stop depending on an underscore-private symbol that upstream review could rename without notice. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The daytona-SDK preflight closed one launch-time gap but not the other half of the same failure mode: the per-task leg also hard-depends on tbench2_env.task_snapshots.make_daytona / create_task_sandbox, which only exist on the openenv #965/#972/#966 branch. An upstream-main tbench2_env install passes the daytona check, then every episode's sandbox start fails, the sample aborts, the group drops, and the rollout loop refills forever — exactly the silent GPU-burning churn the first preflight was written to prevent. Check the recipe symbols at launch and fail with the install instruction instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>