Skip to content

feat: name runtime resources after the rollout id - #1596

Merged
mikasenghaas merged 1 commit into
feat/nano-as-v1from
feat/runtime-name-rollout-id
Jun 9, 2026
Merged

feat: name runtime resources after the rollout id#1596
mikasenghaas merged 1 commit into
feat/nano-as-v1from
feat/runtime-name-rollout-id

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

  • The subprocess workdir (/tmp/<id>), docker --name, and prime sandbox name now equal the rollout's auto-generated trace id, so every provisioned resource is greppable back to the rollout it serves.
  • make_runtime(config, name=...) threads the name through to each runtime; Rollout.run() passes name=trace.id.
  • The Runtime base holds self.name, falling back to a unique vf-<uuid> when no name is given (standalone / tool runtimes, where there's no single owning rollout).

Previously the three were independent and un-correlatable: a random mkdtemp /tmp/vf-* workdir, a vf-<uuid> container, and a static vf-program sandbox name (every prime sandbox shared one name).

Verification

Debug rollout (eval gsm8k-v1 -n 1 --harness.runtime.type <rt>) against all three runtimes, capturing the live resource and matching it to the trace id, with a no-leak check before/after:

runtime resource trace id == resource name cleaned up
subprocess /tmp/<id> workdir ✓ (full 32-hex match, captured live) ✓ workdir removed, 0 leftover
docker container --name ✓ (full 32-hex match, captured live) ✓ container removed (count 1→1)
prime sandbox name ✓ (seen live in sandbox list) ✓ sandbox TERMINATED after rollout

All three rollouts completed with exit=0.

Note

Name runtime resources after the rollout trace ID

  • Runtimes now accept an optional name parameter, defaulting to a generated vf-<12 hex> identifier in the base Runtime class.
  • Docker containers, Prime sandboxes, and subprocess workspaces all use this name for their underlying resources instead of randomly generated values.
  • Rollout.run passes the rollout trace ID as the runtime name, making provisioned resource names deterministic and traceable.
  • Behavioral Change: SubprocessRuntime now creates a fixed /tmp/<name> directory via mkdir() instead of using tempfile.mkdtemp(); the directory is no longer guaranteed unique if a run is retried with the same trace ID.

Macroscope summarized 231513d.


Note

Medium Risk
Subprocess isolation now uses a fixed /tmp/<name> path with plain mkdir(), which can collide on retry with the same trace id; Docker/Prime naming changes are mostly operational traceability.

Overview
Runtime resources are now named from the rollout trace id so subprocess workdirs, Docker containers, and Prime sandboxes can be correlated with a specific rollout in logs and on the host.

make_runtime accepts an optional name, stored on the base Runtime (default vf-<12 hex> when omitted, e.g. tool servers). Rollout.run passes trace.id into that parameter. Docker uses it for --name; Prime for sandbox name (replacing the shared vf-program label); Subprocess provisions /tmp/<name> via mkdir() instead of tempfile.mkdtemp() with a random vf-* prefix.

Note: subprocess workspaces are no longer guaranteed unique if provisioning runs again with the same name before cleanup.

Reviewed by Cursor Bugbot for commit 231513d. Bugbot is set up for automated code reviews on this repo. Configure here.

The subprocess workdir, docker `--name`, and prime sandbox name now equal the
rollout's trace id (passed via `make_runtime(config, name=trace.id)`), so each
provisioned resource is greppable back to the rollout it serves. Previously they
were independent: a random `/tmp/vf-*` workdir, a `vf-<uuid>` container, and a
static `vf-program` sandbox name.

Standalone / tool runtimes (no single owning rollout) fall back to a unique
`vf-<uuid>` name on the `Runtime` base.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mikasenghaas
mikasenghaas marked this pull request as ready for review June 9, 2026 22:53
@mikasenghaas
mikasenghaas merged commit 4535551 into feat/nano-as-v1 Jun 9, 2026
4 of 5 checks passed
@mikasenghaas
mikasenghaas deleted the feat/runtime-name-rollout-id branch June 9, 2026 22:56
@macroscopeapp

macroscopeapp Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This feature changes runtime resource creation across Docker, Prime, and Subprocess runtimes. The subprocess change from tempfile.mkdtemp() to explicit Path.mkdir() has different error handling semantics. Runtime behavior changes of this scope warrant human review.

You can customize Macroscope's approvability policy. Learn more.

pull Bot pushed a commit to Stars1233/verifiers that referenced this pull request Jun 23, 2026
…1596)

The subprocess workdir, docker `--name`, and prime sandbox name now equal the
rollout's trace id (passed via `make_runtime(config, name=trace.id)`), so each
provisioned resource is greppable back to the rollout it serves. Previously they
were independent: a random `/tmp/vf-*` workdir, a `vf-<uuid>` container, and a
static `vf-program` sandbox name.

Standalone / tool runtimes (no single owning rollout) fall back to a unique
`vf-<uuid>` name on the `Runtime` base.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant