Skip to content

[TLT-5991][Bugfix] NVBug 6682592: resolve the NemoClaw sandbox container by label - #221

Merged
helenzhou0523 merged 2 commits into
mainfrom
bugfix/nemoclaw-resolve-sandbox-container-by-label
Aug 31, 2026
Merged

helenzhou0523 merged 2 commits into
mainfrom
bugfix/nemoclaw-resolve-sandbox-container-by-label

Conversation

@helenzhou0523

Copy link
Copy Markdown
Contributor

Problem

setup-tao-nemoclaw.sh aborts at step 0 on a healthy, onboarded sandbox:

$ ./setup-tao-nemoclaw.sh taotesting ~/workspace --restart-server
[tao-nemoclaw] ERROR: no running container for sandbox 'taotesting' (nemoclaw list?)

Both scripts located the sandbox container by matching an anchored name prefix,
openshell-<sandbox>-. That was correct when this integration was written
(2026-07-09): OpenShell's docker driver named sandbox containers
openshell-<name>-<id>.

NVIDIA/OpenShell#2243 (5952a5a2)
introduced the workspace resource model and changed container_name_for_sandbox:

-    format!("{CONTAINER_NAME_PREFIX}{truncated_name}-{id_suffix}")
+    format!("{CONTAINER_NAME_PREFIX}{workspace}--{truncated_name}-{id_suffix}")

It first shipped in OpenShell v0.0.88 (v0.0.87 still emits the old form). The
anchored prefix cannot match openshell-<workspace>--<name>-<id>, so on any host
whose gateway has been redeployed onto v0.0.88+:

  • setup aborts in step 0, which runs ahead of every branch — so even the
    narrow --restart-server path cannot run. Nothing downstream is reachable:
    no MCP server, no skills in the sandbox, no openclaw.json entry, no egress
    policy, so the agent never gets tao_ls / tao_run.
  • uninstall (not in the bug report) hits the same line but fails soft —
    it warns and skips, so it silently leaves the skills, the openclaw.json
    entry and the egress policy in place while reporting success.

Fix

Resolve by label, not by name. OpenShell stamps openshell.ai/sandbox-name on
every sandbox container and has since well before this integration existed —
present in v0.0.71 through main — so it is exact, version-portable across the
rename, and not an internal convention we have to keep chasing.

_sandbox_cid() {
  docker ps -q --filter "label=openshell.ai/sandbox-name=$1" | head -1
}

A lookup miss now prints the sandbox containers docker ps did return, so the
next mismatch diagnoses itself instead of sending the operator back to re-check
onboarding.

openshell.ai/managed-by is deliberately not part of the filter: the docker
driver only began applying it after v0.0.71, so requiring it would break the
older gateways this change is meant to keep working.

Validation

Run on a NemoClaw host (openshell 0.0.71, real Docker) against containers
carrying the exact names and labels the driver emits on each side of the rename:

post-v0.0.88 name, old pattern finds nothing — reproduces the reported abort
post-v0.0.88 name, new lookup resolves the correct container
post-v0.0.88 name + an openshell-<sb>-local neighbour, old pattern resolves the wrong container — a latent mis-resolution the abort was masking
same, new lookup still resolves the correct container
pre-v0.0.88 name, new lookup resolves — no regression on older gateways
prefix bleed (taotest vs taotesting) no match, as intended
unknown sandbox resolves to nothing, diagnostic lists real candidates

Both scripts also run end to end past step 0 on that host:

[tao-nemoclaw] sandbox=taotesting container=00fe5f00b589 bridge-gateway=172.17.0.1 workspace=/home/ubuntu/tao-workspace

setup then proceeds to the credentials check and uninstall to the confirmation
prompt — both unrelated to this change and expected on a bare host. With an
unknown sandbox:

[tao-nemoclaw] ERROR: no running container for sandbox 'nosuchsb' (nemoclaw list?)
  running sandbox containers: openshell-default--taotesting-8d3c1f92-...

Notes

  • NVBug 6682592 (P0-Stopper), Jira TLT-5991.
  • This is a re-file of soft-deleted bug ed5913e0, which reported the same
    defect against a different sandbox on a different host. Consistent with the
    root cause: hosts break one at a time, as each gateway gets redeployed onto
    a v0.0.88+ cluster image.

…ner by label

setup-tao-nemoclaw.sh and uninstall-tao-nemoclaw.sh located the sandbox
container by matching an anchored name prefix, 'openshell-<sandbox>-'.
That was correct when the integration was written: OpenShell's docker
driver named sandbox containers 'openshell-<name>-<id>'.

OpenShell v0.0.88 introduced the workspace resource model and changed
container_name_for_sandbox to 'openshell-<workspace>--<name>-<id>'. The
anchored prefix cannot match the new form, so on any host whose gateway
has been redeployed onto v0.0.88 or later:

  - setup aborts in step 0, ahead of every branch, so even
    --restart-server cannot run; and
  - uninstall silently skips all sandbox-side cleanup and reports
    success, leaving the skills, the openclaw.json entry and the egress
    policy behind.

The name is an internal convention; the label is the supported lookup.
OpenShell has stamped openshell.ai/sandbox-name on every sandbox
container since well before this integration existed (present in v0.0.71
through main), so filtering on it is exact and works on both sides of the
rename. A lookup miss now prints the sandbox containers docker did
return, so the next mismatch diagnoses itself instead of pointing the
operator back at onboarding.

Validated on a NemoClaw host (openshell 0.0.71, real Docker) against
containers carrying the exact names and labels the driver emits:

  - post-v0.0.88 name: old pattern finds nothing, new lookup resolves it
  - with an 'openshell-<sb>-local' neighbour present, the old pattern
    resolves the WRONG container -- a latent mis-resolution the reported
    abort was masking
  - pre-v0.0.88 name still resolves through the same lookup
  - no prefix bleed ('taotest' does not match 'taotesting')
  - both scripts run end to end past step 0; setup reaches the
    credentials check and uninstall reaches the confirmation prompt

Ref: NVIDIA/OpenShell#2243 (5952a5a2, first released in v0.0.88)
Signed-off-by: Bingjie Zhou <bingjiez@nvidia.com>
@helenzhou0523
helenzhou0523 requested a review from a team as a code owner August 31, 2026 22:44
@tao-pr-bot

tao-pr-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown

For security reasons, CI does not run automatically on NVIDIA's runners — it must be triggered per commit.

  • Comment /build to run CI on your latest commit, and make sure blossom-ci is green before merging.
  • Re-run /build after each new push — CI is pinned to the head commit, so a stale run won't count.

Tip

Should this fix also ship in a release? Add a release/X.Y.Z label (matching the target release branch) and tao-cherry-pick-bot will backport it after merge.

Important

Before merging: every file needs a license header and every commit must be DCO signed-off — see the 📖 Contribution guide for details.

@vpraveen-nv vpraveen-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@helenzhou0523

Copy link
Copy Markdown
Contributor Author

/build

…moClaw's and moves

The version matrix read as though OpenShell 0.0.85 were pinned on our side.
It is not: the pin lives in NemoClaw's nemoclaw-blueprint/blueprint.yaml as
min == max, and it moved 0.0.72 -> 0.0.85 -> 0.0.99 -> 0.0.101 -> 0.0.106
between 2026-07-03 and 2026-08-20. The 0.0.85 -> 0.0.99 bump on 2026-08-07
(NemoClaw #8523) jumped straight over the v0.0.88 container rename that broke
container resolution, so a reader checking this table would have concluded we
were insulated when we were not.

Record the pin history, state plainly that the table is what was verified
rather than what is enforced, and generalise the lesson: do not re-derive
OpenShell's internal conventions.

Signed-off-by: Bingjie Zhou <bingjiez@nvidia.com>
@helenzhou0523
helenzhou0523 merged commit bad500f into main Aug 31, 2026
5 checks passed
@helenzhou0523
helenzhou0523 deleted the bugfix/nemoclaw-resolve-sandbox-container-by-label branch August 31, 2026 23:46
@tao-cherry-pick-bot

Copy link
Copy Markdown

🍒 Cherry-pick summary

  • release/7.2.0: ✅ cherry-picked cleanly (c3d5ff2)

tao-cherry-pick-bot Bot pushed a commit that referenced this pull request Aug 31, 2026
…ner by label (#221)

* [TLT-5991][Bugfix] NVBug 6682592: resolve the NemoClaw sandbox container by label

setup-tao-nemoclaw.sh and uninstall-tao-nemoclaw.sh located the sandbox
container by matching an anchored name prefix, 'openshell-<sandbox>-'.
That was correct when the integration was written: OpenShell's docker
driver named sandbox containers 'openshell-<name>-<id>'.

OpenShell v0.0.88 introduced the workspace resource model and changed
container_name_for_sandbox to 'openshell-<workspace>--<name>-<id>'. The
anchored prefix cannot match the new form, so on any host whose gateway
has been redeployed onto v0.0.88 or later:

  - setup aborts in step 0, ahead of every branch, so even
    --restart-server cannot run; and
  - uninstall silently skips all sandbox-side cleanup and reports
    success, leaving the skills, the openclaw.json entry and the egress
    policy behind.

The name is an internal convention; the label is the supported lookup.
OpenShell has stamped openshell.ai/sandbox-name on every sandbox
container since well before this integration existed (present in v0.0.71
through main), so filtering on it is exact and works on both sides of the
rename. A lookup miss now prints the sandbox containers docker did
return, so the next mismatch diagnoses itself instead of pointing the
operator back at onboarding.

Validated on a NemoClaw host (openshell 0.0.71, real Docker) against
containers carrying the exact names and labels the driver emits:

  - post-v0.0.88 name: old pattern finds nothing, new lookup resolves it
  - with an 'openshell-<sb>-local' neighbour present, the old pattern
    resolves the WRONG container -- a latent mis-resolution the reported
    abort was masking
  - pre-v0.0.88 name still resolves through the same lookup
  - no prefix bleed ('taotest' does not match 'taotesting')
  - both scripts run end to end past step 0; setup reaches the
    credentials check and uninstall reaches the confirmation prompt

Ref: NVIDIA/OpenShell#2243 (5952a5a2, first released in v0.0.88)
Signed-off-by: Bingjie Zhou <bingjiez@nvidia.com>

* [TLT-5991][Bugfix] NVBug 6682592: record that the OpenShell pin is NemoClaw's and moves

The version matrix read as though OpenShell 0.0.85 were pinned on our side.
It is not: the pin lives in NemoClaw's nemoclaw-blueprint/blueprint.yaml as
min == max, and it moved 0.0.72 -> 0.0.85 -> 0.0.99 -> 0.0.101 -> 0.0.106
between 2026-07-03 and 2026-08-20. The 0.0.85 -> 0.0.99 bump on 2026-08-07
(NemoClaw #8523) jumped straight over the v0.0.88 container rename that broke
container resolution, so a reader checking this table would have concluded we
were insulated when we were not.

Record the pin history, state plainly that the table is what was verified
rather than what is enforced, and generalise the lesson: do not re-derive
OpenShell's internal conventions.

Signed-off-by: Bingjie Zhou <bingjiez@nvidia.com>

---------

Signed-off-by: Bingjie Zhou <bingjiez@nvidia.com>
(cherry picked from commit bad500f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants