feat: enable labels for prime sandboxes + tunnels - #1604
Merged
Conversation
Add `PrimeConfig.labels` (settable via `--harness.runtime.labels`), passed to both `CreateSandboxRequest` and every `Tunnel`. `run_eval` defaults the labels to the eval run's uuid when unset — after `save_config`, so re-running `@ config.toml` gets a fresh uuid rather than reusing the saved one. Every sandbox and tunnel a run creates then shares the run's uuid label, so they can be found and cleaned up together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mikasenghaas
changed the title
Leave `PrimeConfig.labels` settable (sandbox + tunnels), but don't auto-default them to the eval run uuid in run_eval for now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mikasenghaas
marked this pull request as ready for review
June 10, 2026 18:10
pull Bot
pushed a commit
to Stars1233/verifiers
that referenced
this pull request
Jun 23, 2026
…1604) * feat(v1): label prime sandboxes + tunnels with the eval run uuid Add `PrimeConfig.labels` (settable via `--harness.runtime.labels`), passed to both `CreateSandboxRequest` and every `Tunnel`. `run_eval` defaults the labels to the eval run's uuid when unset — after `save_config`, so re-running `@ config.toml` gets a fresh uuid rather than reusing the saved one. Every sandbox and tunnel a run creates then shares the run's uuid label, so they can be found and cleaned up together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(v1): drop the run-uuid default for prime labels for now Leave `PrimeConfig.labels` settable (sandbox + tunnels), but don't auto-default them to the eval run uuid in run_eval for now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PrimeConfig.labels: list[str](settable via--harness.runtime.labels), passed toCreateSandboxRequestand to everyTunnel. A run's prime sandboxes and tunnels can then share a label so they're findable / cleanable together.Verification (e2e against prime)
uv run eval gsm8k-v1 -n 2 -r 2 --harness.id default --harness.runtime.type prime:SandboxClient.list(labels=[label])returned the run's sandboxes, each carrying the label (withnamestill the per-rollout trace id).list(exclude_terminated=True)→ 0).Tunnel(labels=[…])→TunnelClient.list_tunnels(labels=[…])round-trip (status=connected, labels=[…]).Note: auto-defaulting the labels to the eval run uuid was dropped for now — set them explicitly via
--harness.runtime.labels.Note
Low Risk
Additive config wiring to Prime sandbox/tunnel APIs with no changes to auth, exec, or teardown logic.
Overview
Adds
PrimeConfig.labelsso Prime eval runs can tag every sandbox and tunnel with the same strings (via--harness.runtime.labels), making resources listable and cleanable as a group.On sandbox create, labels are forwarded to
CreateSandboxRequest. Onexpose, the same list is passed intoTunnel(empty config becomesNonefor the tunnel API). Labels are opt-in; there is no automatic run-uuid default in this change.Reviewed by Cursor Bugbot for commit 7748616. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add labels support to prime sandbox and tunnel creation in
PrimeRuntimeAdds a
labels: list[str]field toPrimeConfigin prime.py. Labels are forwarded toCreateSandboxRequeston sandbox start and to tunnel construction in theexposemethod (passed asNonewhen the list is empty).Macroscope summarized 7748616.