feat: name runtime resources after the rollout id - #1596
Merged
Conversation
The subprocess workdir, docker `--name`, and prime sandbox name now equal the rollout's trace id (passed via `make_runtime(config, name=trace.id)`), so each provisioned resource is greppable back to the rollout it serves. Previously they were independent: a random `/tmp/vf-*` workdir, a `vf-<uuid>` container, and a static `vf-program` sandbox name. Standalone / tool runtimes (no single owning rollout) fall back to a unique `vf-<uuid>` name on the `Runtime` base. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
ApprovabilityVerdict: Needs human review This feature changes runtime resource creation across Docker, Prime, and Subprocess runtimes. The subprocess change from You can customize Macroscope's approvability policy. Learn more. |
pull Bot
pushed a commit
to Stars1233/verifiers
that referenced
this pull request
Jun 23, 2026
…1596) The subprocess workdir, docker `--name`, and prime sandbox name now equal the rollout's trace id (passed via `make_runtime(config, name=trace.id)`), so each provisioned resource is greppable back to the rollout it serves. Previously they were independent: a random `/tmp/vf-*` workdir, a `vf-<uuid>` container, and a static `vf-program` sandbox name. Standalone / tool runtimes (no single owning rollout) fall back to a unique `vf-<uuid>` name on the `Runtime` base. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/tmp/<id>), docker--name, and prime sandbox name now equal the rollout's auto-generated trace id, so every provisioned resource is greppable back to the rollout it serves.make_runtime(config, name=...)threads the name through to each runtime;Rollout.run()passesname=trace.id.Runtimebase holdsself.name, falling back to a uniquevf-<uuid>when no name is given (standalone / tool runtimes, where there's no single owning rollout).Previously the three were independent and un-correlatable: a random
mkdtemp/tmp/vf-*workdir, avf-<uuid>container, and a staticvf-programsandbox name (every prime sandbox shared one name).Verification
Debug rollout (
eval gsm8k-v1 -n 1 --harness.runtime.type <rt>) against all three runtimes, capturing the live resource and matching it to the trace id, with a no-leak check before/after:/tmp/<id>workdir--nameTERMINATEDafter rolloutAll three rollouts completed with
exit=0.Note
Name runtime resources after the rollout trace ID
nameparameter, defaulting to a generatedvf-<12 hex>identifier in the base Runtime class.SubprocessRuntimenow creates a fixed/tmp/<name>directory viamkdir()instead of usingtempfile.mkdtemp(); the directory is no longer guaranteed unique if a run is retried with the same trace ID.Macroscope summarized 231513d.
Note
Medium Risk
Subprocess isolation now uses a fixed
/tmp/<name>path with plainmkdir(), which can collide on retry with the same trace id; Docker/Prime naming changes are mostly operational traceability.Overview
Runtime resources are now named from the rollout trace id so subprocess workdirs, Docker containers, and Prime sandboxes can be correlated with a specific rollout in logs and on the host.
make_runtimeaccepts an optionalname, stored on the baseRuntime(defaultvf-<12 hex>when omitted, e.g. tool servers).Rollout.runpassestrace.idinto that parameter. Docker uses it for--name; Prime for sandboxname(replacing the sharedvf-programlabel); Subprocess provisions/tmp/<name>viamkdir()instead oftempfile.mkdtemp()with a randomvf-*prefix.Note: subprocess workspaces are no longer guaranteed unique if provisioning runs again with the same name before cleanup.
Reviewed by Cursor Bugbot for commit 231513d. Bugbot is set up for automated code reviews on this repo. Configure here.