Skip to content

feat(state): segregate host state root per gateway port - #6711

Merged
cv merged 44 commits into
mainfrom
feat/3053-instance-state-root
Jul 15, 2026
Merged

feat(state): segregate host state root per gateway port#6711
cv merged 44 commits into
mainfrom
feat/3053-instance-state-root

Conversation

@laitingsheng

@laitingsheng laitingsheng commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

A non-default NEMOCLAW_GATEWAY_PORT previously shared the single ~/.nemoclaw/ host state root, so its sandbox registry, legacy credential-migration files, snapshots, and onboarding state collided with the default gateway's. This routes the host state root through one nemoclawStateRoot resolver keyed off the existing gateway port, so a non-default port now owns a separate ~/.nemoclaw/gateways/<port>/ root while the default port stays byte-identical at ~/.nemoclaw/.

Related Issue

Fixes #3053

Changes

  • Add src/lib/state/state-root.tsnemoclawStateRoot(home, gatewayPort) returns ~/.nemoclaw for the default gateway port and ~/.nemoclaw/gateways/<port> otherwise. It is keyed off the existing GATEWAY_PORT constant so the state root and the gateway name (nemoclaw / nemoclaw-<port>) always agree.
  • Route the host state root through the resolver at every writer: the sandbox registry (sandboxes.json), legacy credential-migration path, config-io permission-heal and remediation, onboard session, rebuild backups, blueprint cache, model router, local-inference adapter state and Ollama proxy token, share mounts, usage notice, and onboard-failure diagnostics.
  • Keep the CLI source clone and the managed_swap marker at the shared ~/.nemoclaw/ root — one CLI binary and one host swap file exist per machine, not per gateway.
  • Uninstall now targets the port-scoped state root, always preserves the gateways/ subtree so other environments survive even --destroy-user-data, prints the resolved state directory in its plan output, and skips host-swap teardown while other gateway-port environments remain.
  • Document the per-port state root in docs/manage-sandboxes/lifecycle.mdx and docs/reference/architecture.mdx.

This adds one resolver abstraction because the ~/.nemoclaw root was inlined at every state-path site with no single lever; segregation needs a single keyed source of truth. Its current consumer is #3053 (two segregated NemoClaw environments on one host). The default-port byte-identical contract is protected by src/lib/state/state-root.test.ts; the end-to-end per-port fork by src/lib/state/registry-port-segregation.test.ts; and the uninstall preservation plus host-swap guard by src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification:
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: mechanical state-root path routing that preserves byte-identical default-port behavior (test-covered); no change to credential handling, auth, or policy semantics. Requesting maintainer sensitive-path review.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result: npm run test:changed — 154 files and 1,541 tests passed; npm run checks — passed; npm run typecheck:cli — 0 errors
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Tinson Lai tinsonl@nvidia.com

Summary by CodeRabbit

  • New Features
    • Non-default gateway ports now keep port-scoped host state under ~/.nemoclaw/gateways/<port>/ (separate registries/snapshots/state, mounts, diagnostics, model-router state, and adapter/Ollama token paths).
    • Uninstall is now strictly scoped to the selected gateway; dashboard-port selection accounts for host-wide gateway scopes.
    • Legacy port state is migrated only when the recorded gateway identity matches the selected port.
  • Bug Fixes
    • Improved uninstall cleanup behavior (including swap handling) when other gateway-port environments exist, plus clearer failure when selection can’t be verified.
    • Stronger protections against symlinked state paths when reading/writing state.
  • Documentation
    • Updated architecture and command/uninstall/sandbox/reference docs and expanded the NEMOCLAW_GATEWAY_PORT description.
  • Tests
    • Added/expanded coverage for port segregation, scoped uninstall, legacy migration, dashboard-port isolation, and symlink safety.

A non-default NEMOCLAW_GATEWAY_PORT now resolves its own host state
root at ~/.nemoclaw/gateways/<port>/, covering the sandbox registry,
credentials, snapshots, onboard session, and inference adapter state,
so two gateways on one host stay fully segregated. The default port
keeps the shared ~/.nemoclaw/ location, so existing single-gateway
hosts are byte-identical.

Route the registry, credentials, config-io, onboard, inference,
share, and diagnostics paths through one nemoclawStateRoot resolver
keyed off GATEWAY_PORT. Keep the CLI source clone and the managed_swap
marker host-shared. Uninstall targets the port-scoped state root,
always preserves the gateways/ subtree so other environments survive
even --destroy-user-data, and no longer tears down the host-shared
swap file while other gateway-port environments remain.

Fixes #3053

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7660676f-02c4-416a-acf3-db532c77a666

📥 Commits

Reviewing files that changed from the base of the PR and between 0cba390 and 71dab25.

📒 Files selected for processing (5)
  • docs/reference/commands.mdx
  • scripts/install.sh
  • src/lib/inference/local.test.ts
  • src/lib/inference/local.ts
  • src/lib/state/onboard-session.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • src/lib/inference/local.ts
  • src/lib/state/onboard-session.ts
  • src/lib/inference/local.test.ts
  • docs/reference/commands.mdx
  • scripts/install.sh

📝 Walkthrough

Walkthrough

This change introduces port-scoped host state, legacy-state migration, host-wide registry validation, gateway-scoped installer and uninstall behavior, symlink-safe state writers, CLI migration gating, and corresponding tests and documentation.

Changes

Gateway-port state segregation

Layer / File(s) Summary
State-root and filesystem safety
src/lib/state/*, src/lib/credentials/store.ts, src/lib/inference/*, src/lib/onboard/*, src/lib/share-command.ts
Centralizes gateway state roots and applies them to registries, sessions, credentials, backups, mounts, diagnostics, adapters, and model-router state with symlink-safe writes.
Registry and migration
src/lib/state/gateway-registry.ts, src/lib/state/legacy-port-migration.ts
Validates host registries, resolves gateway identities, and migrates eligible legacy state with locks, atomic moves, and recovery.
Installer, dispatch, and uninstall
scripts/install.sh, src/lib/cli/public-dispatch.ts, src/lib/actions/uninstall/*
Scopes installer, CLI migration, and uninstall behavior to the selected gateway while preserving sibling resources.
Validation and documentation
test/*, docs/reference/*, docs/manage-sandboxes/*
Adds coverage and documents port isolation, migration, symlink rejection, dashboard allocation, installer recovery, and uninstall semantics.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI as public-dispatch
  participant Migration as migrateLegacyPortState
  participant SharedRoot as Shared NemoClaw state
  participant GatewayRoot as Port-scoped state

  CLI->>Migration: migrate stateful invocation
  Migration->>SharedRoot: read and classify legacy state
  Migration->>GatewayRoot: move selected registry and files
  Migration->>SharedRoot: rewrite remaining registry
  Migration-->>CLI: migration result
Loading
sequenceDiagram
  participant User
  participant Uninstall as runUninstallPlan
  participant Registry as gateway registries
  participant OpenShell
  participant Docker

  User->>Uninstall: uninstall selected gateway
  Uninstall->>Registry: inspect sibling environments
  Registry-->>Uninstall: scope and sandbox names
  Uninstall->>OpenShell: remove selected gateway resources
  Uninstall->>Docker: remove selected gateway containers
  Uninstall-->>User: cleanup result
Loading

Possibly related PRs

Suggested labels: area: install, area: onboarding, area: security, refactor

Suggested reviewers: cjagwani

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 13.10% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: host state is now segregated by gateway port.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/3053-instance-state-root

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the feat/3053-instance-s... branch remains at 80%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.
File main 627154d feat/3053-instance-s... eeeb935 +/-
src/lib/state/registry.ts 91% 85% -6%
src/lib/onboard...shboard-port.ts 94% 88% -6%
src/lib/state/o...oard-session.ts 85% 84% -1%
src/lib/actions...all/run-plan.ts 81% 83% +2%
src/lib/onboard...usage-notice.ts 22% 30% +8%
src/lib/onboard...eate-failure.ts 79% 90% +11%
src/lib/inferen...er-lifecycle.ts 57% 70% +13%
src/lib/state/l...rt-migration.ts 0% 90% +90%
src/lib/state/g...way-registry.ts 0% 95% +95%
src/lib/state/state-root.ts 0% 100% +100%

Updated July 15, 2026 17:32 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

Copy link
Copy Markdown
Contributor

@laitingsheng laitingsheng added area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery feature PR adds or expands user-visible functionality labels Jul 12, 2026
@github-actions

github-actions Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Blocking findings reported

Advisor assessment: Blockers require maintainer review
Next action: Review the blockers below.
Findings: 2 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 2 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings differ; normalized E2E selections differ; Nemotron reported 2 fewer blockers, the same number of warnings, the same number of suggestions.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, concurrent-gateway-ports, inference-routing, network-policy, onboard-repair, onboard-resume

1 optional E2E recommendation
  • model-router-provider-routed-inference

Blockers

PRA-1 Blocker — Migrate legacy state before the internal uninstall route

  • Location: src/lib/cli/public-dispatch.ts:112
  • Category: correctness
  • Problem: The dispatcher excludes `internal uninstall plan` and `internal uninstall run-plan` from `migrateLegacyPortState()`. Both the installed CLI's uninstall wrapper and `uninstall.sh` invoke `internal uninstall run-plan`, while `runUninstallPlan` does not migrate state itself. Consequently, a non-default-port uninstall on a pre-segregation host reads an empty port-scoped registry rather than partitioning matching rows from the shared registry before selecting and deleting the gateway.
  • Impact: The uninstall can skip cleanup of the selected gateway's legacy sandboxes and state, then make final-gateway/shared-resource decisions from incomplete state. This violates the selected-gateway lifecycle isolation the change introduces and can leave orphaned resources or preserve/remove the wrong shared resources.
  • Fix: Invoke the same fail-closed legacy-port migration before building or executing the internal uninstall plan (or place it inside the uninstall entrypoint), before any registry inspection or cleanup, and abort on migration failure.
  • Verification: Read `uninstall.sh` and `src/commands/uninstall.ts` to confirm they route to `internal uninstall run-plan`, then inspect `dispatchCli`'s `isMigrationRecoveryInvocation` exemption and `runUninstallPlan` for the absent migration call.
  • Test coverage: Add an end-to-end unit test through the internal uninstall command or wrapper path with `NEMOCLAW_GATEWAY_PORT=9123` and a shared legacy registry containing 9123 and sibling rows; assert the 9123 row is migrated and only its sandbox/gateway is cleaned while the sibling row and resources remain.
  • Evidence: `src/lib/cli/public-dispatch.ts:112-120` classifies `internal uninstall plan/run-plan` as migration-recovery invocations and skips the migration at lines 480-500. `uninstall.sh` executes `internal uninstall run-plan`, and `src/commands/uninstall.ts` delegates to that local script. `src/lib/actions/uninstall/run-plan.ts` has no `migrateLegacyPortState` call; its selected registry lookup reads `paths.nemoclawStateDir/sandboxes.json`. `src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts:361` manually calls `migrateLegacyPortState` before testing legacy sibling cleanup, demonstrating the missing production precondition.

PRA-2 Blocker — Segregate the credential-bearing Model Router per gateway

  • Location: src/lib/onboard/model-router.ts:232
  • Category: security
  • Problem: The new per-gateway state roots do not isolate the Model Router: every routed instance still obtains port 4000 from the shared blueprint and starts its proxy on `0.0.0.0:4000`. When a second gateway configures routed inference, it encounters the first router as a healthy listener and reconciliation can stop it as an orphan before starting a router with the second gateway's credential. Both sandboxes route to the same host port.
  • Impact: A sandbox on the first gateway can have inference served by the second gateway's Model Router and credential context, violating the PR's credential and inference isolation boundary between gateway instances. The globally bound listener also exposes this cross-instance service beyond loopback.
  • Fix: Make Model Router port allocation and process ownership gateway-scoped (or explicitly reject concurrent routed gateways); derive the routed endpoint and health/cleanup checks from that scoped identity, and avoid binding the credential-bearing proxy to all interfaces unless required and policy-protected.
  • Verification: Inspect `nemoclaw-blueprint/blueprint.yaml:88-99` and `startModelRouter` in `src/lib/onboard/model-router.ts`: both use static port 4000, and the proxy is spawned with `--host 0.0.0.0`; trace `reconcileModelRouter` when a healthy router belongs to a different instance.
  • Test coverage: Add a two-gateway routed-inference test using distinct credentials that proves each sandbox reaches only its own router, that starting the second gateway does not stop or replace the first router, and that the configured listener exposure matches the intended policy.
  • Evidence: `nemoclaw-blueprint/blueprint.yaml:88-99` sets the routed endpoint to `http://localhost:4000/v1\` and router port to 4000 for every instance. `src/lib/onboard/model-router.ts:232-305` computes `routerCfg.port || 4000` and spawns the proxy with `--host 0.0.0.0` without incorporating `GATEWAY_PORT`. `src/lib/onboard/model-router.ts:~390-430` treats a healthy router whose recorded PID does not own port 4000 as an orphan and can stop it before starting a replacement. The changed multi-gateway E2E configures a custom provider rather than two routed providers with distinct credentials (`test/e2e/live/concurrent-gateway-ports.test.ts:55-75`).

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/manage-sandboxes/lifecycle.mdx`:
- Around line 154-157: Update the agent-specific lifecycle documentation block
to replace the invalid $$nemoclaw uninstall command with the literal Deep Agents
CLI command nemo-deepagents uninstall, while preserving the surrounding
gateway-state behavior and wording.
- Line 150: Update the lifecycle documentation near the NEMOCLAW_GATEWAY_PORT
state-root description to replace “fully segregated” with a narrower claim about
gateway-scoped state. Mention that the CLI source clone and managed_swap marker
remain shared under ~/.nemoclaw/ if describing shared host-level artifacts,
while preserving the separate registry, credentials, snapshots, and uninstall
behavior.

In `@docs/reference/architecture.mdx`:
- Line 311: Update the preceding ~/.nemoclaw/sandboxes.json documentation row to
state explicitly that it is the registry for the default gateway port only;
retain the separate non-default registry location described by the
gateways/<port> entry.

In `@src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts`:
- Around line 70-73: Remove the conditional inside the mock run handler and
record every invoked command unconditionally in swapCommands. Update the related
assertions to verify unwanted commands with not.toContain, preserving the test’s
intent without adding an if statement.

In `@src/lib/onboard/usage-notice.ts`:
- Line 56: Update the usage-notice path construction to pass the active gateway
port to nemoclawStateRoot instead of relying on its default-port behavior.
Preserve the existing HOME fallback and usage-notice.json filename while
ensuring non-default gateways resolve their port-scoped state root.

In `@src/lib/share-command.ts`:
- Line 64: The defaultShareMountDir resolver currently ignores gateway identity,
causing all gateways to use the same mount root. Update defaultShareMountDir to
accept the active gateway port and include that port when deriving the mount
directory, then update every caller to pass the corresponding gateway port so
non-default gateways use isolated mount state.

In `@src/lib/state/onboard-session.ts`:
- Around line 48-50: Update the onboarding session path definitions around
SESSION_DIR, SESSION_FILE, and LOCK_FILE to derive the directory using the
active gateway port, rather than a single module-global nemoclawStateRoot value.
Ensure each non-default gateway receives distinct session and lock paths while
preserving the existing filenames.

In `@src/lib/state/state-root.test.ts`:
- Around line 37-41: Update the getNemoclawStateRoot default-port test to
explicitly control NEMOCLAW_GATEWAY_PORT, clearing it before module
initialization or using the explicit-port helper to assert default behavior.
Ensure the test remains isolated from the host environment and preserves the
expected shared root assertion.

In `@src/lib/state/state-root.ts`:
- Around line 19-24: Update resolveHome to use an empty-value fallback, matching
the credentials store behavior, so empty HOME values fall back to os.homedir()
before getNemoclawStateRoot constructs the state path. Keep the existing
getNemoclawStateRoot and nemoclawStateRoot flow unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3fa4a17d-c76f-46ab-82a5-a14de9de2fd0

📥 Commits

Reviewing files that changed from the base of the PR and between e04a344 and 8d028b8.

📒 Files selected for processing (21)
  • docs/manage-sandboxes/lifecycle.mdx
  • docs/reference/architecture.mdx
  • src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts
  • src/lib/actions/uninstall/run-plan.ts
  • src/lib/credentials/store.ts
  • src/lib/domain/uninstall/paths.ts
  • src/lib/inference/local-adapter-lifecycle.ts
  • src/lib/inference/local.ts
  • src/lib/onboard/docker-gpu-patch.ts
  • src/lib/onboard/model-router.ts
  • src/lib/onboard/sandbox-create-failure.ts
  • src/lib/onboard/usage-notice.ts
  • src/lib/share-command.ts
  • src/lib/state/config-io.ts
  • src/lib/state/onboard-session.ts
  • src/lib/state/paths.ts
  • src/lib/state/registry-port-segregation.test.ts
  • src/lib/state/registry.ts
  • src/lib/state/sandbox.ts
  • src/lib/state/state-root.test.ts
  • src/lib/state/state-root.ts

Comment thread docs/manage-sandboxes/lifecycle.mdx Outdated
Comment thread docs/manage-sandboxes/lifecycle.mdx Outdated
Comment thread docs/reference/architecture.mdx Outdated
Comment thread src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts
Comment thread src/lib/onboard/usage-notice.ts Outdated
Comment thread src/lib/share-command.ts Outdated
Comment thread src/lib/state/onboard-session.ts Outdated
Comment thread src/lib/state/state-root.test.ts
Comment thread src/lib/state/state-root.ts
Fixes a regression the port-keyed state root introduced in a pre-existing
CLI status test that temporarily overrides NEMOCLAW_GATEWAY_PORT to probe
for a gateway-port conflict, closes the empty-HOME relative-path edge
case in the state-root resolver, makes the default-port unit test
control the environment variable it claims to test, corrects two doc
wording issues (an overstated segregation claim and a Deep Agents
command name), and drops a conditional from the new uninstall test to
satisfy the codebase-growth guardrail.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/repro-2666-silent-list-status.test.ts`:
- Line 210: Strengthen the port-specific scenario setup around seedRegistry so
the default ~/.nemoclaw registry cannot satisfy the test: clear it before the
scenario or seed it with distinct sentinel data, then assert the selected result
comes only from nemoclawStateRoot(home, port). Apply the same correction to the
corresponding occurrence.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 54a56463-3080-4275-8500-99fb477832c9

📥 Commits

Reviewing files that changed from the base of the PR and between 8d028b8 and d560c34.

📒 Files selected for processing (6)
  • docs/manage-sandboxes/lifecycle.mdx
  • docs/reference/architecture.mdx
  • src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts
  • src/lib/state/state-root.test.ts
  • src/lib/state/state-root.ts
  • test/repro-2666-silent-list-status.test.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • src/lib/state/state-root.ts
  • src/lib/state/state-root.test.ts
  • docs/manage-sandboxes/lifecycle.mdx
  • src/lib/actions/uninstall/run-plan-gateway-segregation.test.ts
  • docs/reference/architecture.mdx

Comment thread test/repro-2666-silent-list-status.test.ts
The port-specific #3271 scenario seeded identical my-assist data under
both the default and port-scoped registries, so a regression that kept
reading the default root under a non-default NEMOCLAW_GATEWAY_PORT
would still pass. Clear the default registry before that scenario so
the test only succeeds when the CLI actually reads the port-scoped
root.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@cv cv added v0.0.82 and removed v0.0.82 labels Jul 12, 2026
…ate-root

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

# Conflicts:
#	src/lib/onboard/docker-gpu-patch.ts
@cv cv added the v0.0.82 label Jul 13, 2026
Every production call site relied on nemoclawStateRoot's default
gatewayPort parameter, which resolves to the correct value only
because GATEWAY_PORT happens to be a process-wide constant computed
once at module load. That's easy to misread as always resolving the
default port. Thread GATEWAY_PORT through all 18 call sites explicitly
so each one is self-evidently correct without tracing into
core/ports.ts. No behavioural change — GATEWAY_PORT is the exact value
the default parameter already supplied.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@prekshivyas prekshivyas self-assigned this Jul 13, 2026
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Some tests cancelled — partial pass

Run: 29265242441
Workflow ref: feat/3053-instance-state-root
Requested targets: (default — all supported)
Requested test IDs: credential-sanitization,security-posture,inference-routing,network-policy,onboard-repair,onboard-resume,cloud-onboard,concurrent-gateway-ports
Summary: 1 passed, 0 failed, 7 cancelled, 0 skipped, 0 unknown

Test Result
cloud-onboard ⚠️ cancelled
concurrent-gateway-ports ⚠️ cancelled
credential-sanitization ⚠️ cancelled
inference-routing ✅ success
network-policy ⚠️ cancelled
onboard-repair ⚠️ cancelled
onboard-resume ⚠️ cancelled
security-posture ⚠️ cancelled

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ❌ Some tests failed

Run: 29265796896
Workflow ref: feat/3053-instance-state-root
Requested targets: (default — all supported)
Requested test IDs: credential-sanitization,security-posture,inference-routing,network-policy,onboard-repair,onboard-resume,concurrent-gateway-ports,cloud-onboard,model-router-provider-routed-inference
Summary: 4 passed, 1 failed, 4 cancelled, 0 skipped, 0 unknown

Test Result
cloud-onboard ⚠️ cancelled
concurrent-gateway-ports ❌ failure
credential-sanitization ✅ success
inference-routing ✅ success
model-router-provider-routed-inference ⚠️ cancelled
network-policy ⚠️ cancelled
onboard-repair ✅ success
onboard-resume ✅ success
security-posture ⚠️ cancelled

Failed tests: concurrent-gateway-ports. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ❌ Some tests failed

Run: 29266737670
Workflow ref: feat/3053-instance-state-root
Requested targets: (default — all supported)
Requested test IDs: credential-sanitization,security-posture,inference-routing,network-policy,onboard-repair,onboard-resume,concurrent-gateway-ports,cloud-onboard,model-router-provider-routed-inference
Summary: 8 passed, 1 failed, 0 cancelled, 0 skipped, 0 unknown

Test Result
cloud-onboard ✅ success
concurrent-gateway-ports ❌ failure
credential-sanitization ✅ success
inference-routing ✅ success
model-router-provider-routed-inference ✅ success
network-policy ✅ success
onboard-repair ✅ success
onboard-resume ✅ success
security-posture ✅ success

Failed tests: concurrent-gateway-ports. Check run artifacts for logs.

@cjagwani

Copy link
Copy Markdown
Collaborator

Exact-head live-E2E diagnosis for the failing concurrent-gateway-ports target: the failure at test/e2e/live/concurrent-gateway-ports.test.ts:424 is consistent with this PR's intended state-root segregation, not a transient runner error. After sandbox B is onboarded under NEMOCLAW_GATEWAY_PORT=, the test invokes a single unscoped nemoclaw list (default gateway A) and still expects both A and B. With per-gateway sandboxes.json, that command correctly sees only A. Please update the live proof to list each instance with its own NEMOCLAW_GATEWAY_PORT, assert A is present only in A's registry and B only in B's, and derive each dashboard port from the corresponding scoped output. Then rerun the exact-head required live targets. This closes the detection gap while matching #3053's stated requirement that each instance own its own registry/state root.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/manage-sandboxes/run-sandboxes.mdx`:
- Line 61: Reformat the affected documentation paragraphs so every sentence
begins on its own source line, including the sentences around the gateway
state-root and uninstall behavior. Preserve all wording and paragraph structure;
only add line breaks at sentence boundaries.

In `@docs/manage-sandboxes/uninstall-nemoclaw.mdx`:
- Line 35: Revise the uninstall behavior description so it no longer promises
deleting every entry under ~/.nemoclaw/ when no sibling gateways remain.
Document only the specific shared resources eligible for final-environment
cleanup, including managed_swap as established by the gateway segregation
behavior, while preserving the selected gateway’s resource and port-scoped
cleanup details.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 476cb23d-5784-43a7-aae9-d5fa12da9015

📥 Commits

Reviewing files that changed from the base of the PR and between 290da9b and 144ffa5.

📒 Files selected for processing (3)
  • docs/manage-sandboxes/run-sandboxes.mdx
  • docs/manage-sandboxes/uninstall-nemoclaw.mdx
  • docs/reference/commands.mdx
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/reference/commands.mdx

Comment thread docs/manage-sandboxes/run-sandboxes.mdx Outdated
Comment thread docs/manage-sandboxes/uninstall-nemoclaw.mdx Outdated
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@cv cv added v0.0.84 and removed v0.0.83 labels Jul 14, 2026
@cv cv assigned jyaunches and unassigned prekshivyas and cjagwani Jul 14, 2026
@cv
cv marked this pull request as draft July 15, 2026 16:44
@cv
cv marked this pull request as ready for review July 15, 2026 16:44

@cv cv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer deep review complete for commit 0cba390. Gateway-scoped state migration, registry validation, collision handling, uninstall preservation, security boundaries, DCO, and the focused/live test coverage are sound. The remaining blockers are the requested re-review and refreshed E2E gate.

@cv
cv dismissed stale reviews from cjagwani and cjagwani July 15, 2026 17:44

Superseded by later verified commits that address gateway-scoped state pairing, migration validation and collision handling, fail-closed registry inspection, and uninstall preservation, with focused regressions. All 54 current checks are green.

@cv
cv merged commit d92642e into main Jul 15, 2026
54 checks passed
@cv
cv deleted the feat/3053-instance-state-root branch July 15, 2026 17:44
cv pushed a commit that referenced this pull request Jul 16, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Adds the canonical `docs/changelog/2026-07-15.mdx` entry with the exact
`## v0.0.84` heading for the release candidate range from `v0.0.83`
through `710d2b36b9eebcb6bca3c2b2f796a1bdb69c3a31`.
Fills two owner-page gaps for model-aware local inference health and
pre-write OpenClaw candidate validation.

## Changes

- Add the complete shared Fern changelog entry for `v0.0.84`, with
literal CLI names and root-absolute OpenClaw and Hermes routes.
- Document that sandbox status and doctor compare the configured Ollama
or vLLM model with provider inventory without issuing a completion.
- Document that host-side OpenClaw `config set` validates the complete
candidate before replacing live config or reaching gateway restart.
- Reconcile the `v0.0.84` release label with the commit range. PR #6773
is already contained in `v0.0.83` and remains documented there; CI,
test-harness, docs-infrastructure, and `.js` to `.mts` migration-only
changes require no additional user guidance.

### Source summary

- [#6882](#6882) ->
`docs/manage-sandboxes/backup-restore.mdx`,
`docs/changelog/2026-07-15.mdx`: Explain that OpenClaw runtime identity
and pairing state are excluded from snapshots and ignored during
restore.
- [#6873](#6873) ->
`docs/inference/set-up-ollama.mdx`, `docs/changelog/2026-07-15.mdx`:
Record the Ollama requested-model environment fallback and interactive
default.
- [#6835](#6835) ->
`docs/changelog/2026-07-15.mdx`: Include the sandbox name in the
documented rebuild resume-recovery behavior.
- [#6886](#6886) ->
`docs/inference/custom-endpoint-security.mdx`,
`docs/inference/set-up-openai-compatible-endpoint.mdx`,
`docs/changelog/2026-07-15.mdx`: Explain the exact-host trusted-private
endpoint opt-in and retained SSRF boundaries.
- [#6887](#6887) ->
`docs/reference/commands.mdx`, `docs/changelog/2026-07-15.mdx`: Document
Telegram channel health verdicts, summary behavior, and exit status.
- [#6863](#6863) ->
`docs/manage-sandboxes/lifecycle.mdx`, `docs/changelog/2026-07-15.mdx`:
Add the missing model-inventory behavior for local status and doctor
checks.
- [#6902](#6902) ->
`docs/manage-sandboxes/runtime-controls.mdx`,
`docs/changelog/2026-07-15.mdx`: Add the missing pre-write OpenClaw
candidate-validation contract.
- [#6916](#6916) ->
`docs/changelog/2026-07-15.mdx`: Preserve the failed-session
fresh-install recovery correction in the release entry.
- [#6934](#6934) ->
`docs/reference/commands.mdx`, `docs/reference/troubleshooting.mdx`,
`docs/security/credential-storage.mdx`, `docs/changelog/2026-07-15.mdx`:
Summarize completed-prompt checkpointing and validated credential reuse
during OpenClaw resume.
- [#6898](#6898) ->
`docs/inference/switch-models.mdx`,
`docs/inference/switch-providers.mdx`,
`docs/reference/troubleshooting.mdx`, `docs/changelog/2026-07-15.mdx`:
Explain Hermes dashboard convergence after in-place inference changes.
- [#6711](#6711) ->
`docs/manage-sandboxes/run-sandboxes.mdx`,
`docs/manage-sandboxes/uninstall-nemoclaw.mdx`,
`docs/reference/architecture.mdx`, `docs/reference/commands.mdx`,
`docs/changelog/2026-07-15.mdx`: Summarize port-scoped host state and
uninstall preservation.
- [#6767](#6767) ->
`docs/inference/configure-model-limits.mdx`,
`docs/inference/set-up-ollama.mdx`,
`docs/reference/troubleshooting.mdx`, `docs/changelog/2026-07-15.mdx`:
Record the Hermes `64000`-token Ollama floor and unchanged OpenClaw
floor.
- [#6862](#6862) ->
`docs/get-started/quickstart.mdx`,
`docs/inference/verify-inference-route.mdx`,
`docs/changelog/2026-07-15.mdx`: Explain retryable not-ready
finalization for unhealthy inference routes.
- [#6766](#6766) ->
`docs/security/tcb-boundary.mdx`, `docs/changelog/2026-07-15.mdx`:
Document definitive stale transition-lock recovery and fail-closed
ambiguous cases.
- [#6948](#6948) ->
`docs/manage-sandboxes/manage-mcp-servers.mdx`,
`docs/changelog/2026-07-15.mdx`: Include Hermes MCP apply-state race
recovery in the release entry without changing the established user
workflow.
- [#6964](#6964) ->
`docs/reference/troubleshooting.mdx`, `docs/changelog/2026-07-15.mdx`:
Record complete agent-specific fresh-install and resume recovery
commands.
- [#6883](#6883) ->
`docs/get-started/quickstart.mdx`, `docs/inference/set-up-vllm.mdx`,
`docs/reference/platform-support.mdx`, `docs/changelog/2026-07-15.mdx`:
Summarize the DGX Station Nemotron Ultra express path and pinned
managed-vLLM recipe.
- [#6985](#6985) ->
`docs/inference/set-up-vllm.mdx`, `docs/reference/commands.mdx`,
`docs/changelog/2026-07-15.mdx`: Capture the final automated and
interactive storage-warning behavior.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior —
`test/changelog-docs.test.ts` validates the dated-entry structure, exact
version heading, and preserved history.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — `npx vitest run
test/changelog-docs.test.ts` (6 passed)
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — not run for this doc-only change.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
completed with 0 errors; Fern reported the unchanged unauthenticated
redirect-check and light-theme contrast warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— the native changelog entry uses the required parser-safe MDX SPDX
comment and intentionally has no frontmatter.

---
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added the v0.0.84 changelog entry covering setup, endpoint onboarding,
model handling, sandbox readiness, recovery, channel status, and
configuration safeguards.
* Clarified that sandbox health checks validate configured models
against local Ollama and vLLM provider inventories without generating
completions or consuming tokens.
* Documented that invalid runtime configuration changes are rejected
while preserving the existing working configuration.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery feature PR adds or expands user-visible functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support multiple NemoClaw-managed instances on a single host

5 participants