fix(inference): re-seed Hermes dashboard config on in-place inference set

[yanyunl1991]

Summary

nemoclaw inference set on a running Hermes sandbox updated the OpenShell route, the registry, and the main Hermes gateway config, then reported Inference route synced — but the separately isolated Hermes Dashboard profile (/sandbox/.hermes/dashboard-home/config.yaml) stayed on the previous model, so Dashboard Chat and /api/model/info kept using the old one. This PR re-runs the dashboard config seeder after the switch so the dashboard converges.

Closes #6893.

Reproduction

On our DGX Spark aarch64 test host (GB10 GPU) — matching the reporter's DGX Station aarch64 — a real Hermes + ollama-local sandbox with two models (llama3.1:8b, qwen2.5:3b).

Environment

• Test machine: our DGX Spark aarch64 test host (GB10 GPU), Ubuntu 24.04
• NemoClaw main (v0.0.82); agent Hermes; provider ollama-local

Observed on main (before fix) — inference set --provider ollama-local --model qwen2.5:3b --sandbox hermes prints Inference route synced, then:

gateway   /sandbox/.hermes/config.yaml                 default: qwen2.5:3b
dashboard /sandbox/.hermes/dashboard-home/config.yaml  default: llama3.1:8b   <- stale
/api/model/info                                        {"model":"llama3.1:8b",...}   <- stale

Observed on fix/... (after fix) — the same switch (and a switch back) converge every surface:

switch -> qwen2.5:3b   gateway: qwen2.5:3b   dashboard: qwen2.5:3b   /api/model/info: {"model":"qwen2.5:3b",...}
switch -> llama3.1:8b   gateway: llama3.1:8b   dashboard: llama3.1:8b   /api/model/info: {"model":"llama3.1:8b",...}

Analysis

Hermes keeps an isolated dashboard profile under HERMES_DASHBOARD_HOME (/sandbox/.hermes/dashboard-home) for privilege separation. Its config only receives the gateway's model routing (model / custom_providers / _nemoclaw_upstream) via seed-hermes-dashboard-config.py, which agents/hermes/start.sh runs at sandbox startup. runInferenceSet (src/lib/actions/inference-set.ts) patches the gateway config (patchHermesInferenceConfig) and writes it with writeSandboxConfig, but never re-runs the seeder, so the dashboard config is left on the prior model. A hermes gateway restart does not converge it either, because it restarts the gateway, not the long-running dashboard, so the startup seeder never re-runs. Related to #4765 (which added the dashboard-home mirror for cold startup) but a distinct in-place-switch gap. The dashboard reads its config live (/api/model/info reflects the file immediately after a re-seed), so re-seeding the file is sufficient — no dashboard restart is needed.

Fix

• Add seedHermesDashboardConfig(sandboxName, target) in src/lib/sandbox/config.ts that runs the same seeder inside the sandbox via a non-privileged sandbox exec (matching start.sh's step-down before touching sandbox-owned dashboard state; the seeder itself does no-follow atomic writes and refuses symlinks), mirroring the gateway config into the dashboard-home config.
• Call it from runInferenceSet's Hermes branch after the gateway config write succeeds. On failure, the route and main config remain committed, but the command exits nonzero and tells the user to restart the sandbox. OpenClaw has no isolated dashboard config and is unaffected.

Changes

• src/lib/sandbox/config.ts: add + export seedHermesDashboardConfig.
• src/lib/actions/inference-set.ts: dep + call in the Hermes branch after the config write.
• src/lib/actions/inference-set.test-support.ts: mock the new dep.
• src/lib/actions/inference-set-hermes-run.test.ts: re-seed happens after the write; skipped when the write fails.
• src/lib/actions/inference-set-openclaw-run.test.ts: boundary — OpenClaw never re-seeds.

Type of Change

• Code change (feature, bug fix, or refactor)

Verification

• npm test passes (touched files); broader inference-set + sandbox/config suites pass (147 tests)
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed

AI Disclosure

• AI-assisted — tool: Claude Code

Update — converge-or-don't-claim-synced + tri-state

The re-seed now reports a tri-state (converged / absent / failed) and, when the dashboard did not converge, inference set withholds the Inference route synced line (the route and main config remain committed, but the command exits nonzero and warns), matching the reporter's "converge on the selected model, or fail without printing Inference route synced". An absent Dashboard profile (Dashboard disabled) is a clean no-op that still reports synced.

• src/lib/actions/inference-set-gateway-restart.ts: withhold the success line for a Hermes switch whose dashboard did not converge (dashboardConverged === false).

Note: this fully re-mirrors the dashboard routing (verified: model.default, custom_providers, _nemoclaw_upstream, and providers.<provider>.default_model all converge — matching cold startup), rather than setting a subset of keys.

Signed-off-by: Yanyun Liao yanyunl@nvidia.com

Summary by CodeRabbit

• New Features
  • Added normalization for installer-style --provider inputs to match supported provider identifiers.
  • Hermes sandboxes now re-seed the dashboard “home” configuration after model routing changes, with converge tracking and updated behavior based on the result.
• Bug Fixes
  • If the Hermes dashboard doesn’t converge, success messaging is suppressed and a warning is logged; when the dashboard is disabled, syncing is still reported.
  • OpenClaw routing changes no longer trigger Hermes dashboard reseeding.
• Tests
  • Expanded Vitest coverage for Hermes in-place switching, reseed outcomes (converged/absent/failed), call ordering, and safe reseed execution.
  • Enhanced Hermes end-to-end switch to verify dashboard model info via the internal port.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Hermes inference-route updates now reseed isolated dashboard configuration after successful in-place sandbox writes. Dashboard convergence is propagated to final status handling, with coverage for failure and absent-dashboard cases, argv-safe seeding, OpenClaw exclusion, and live dashboard state.

Changes

Hermes dashboard routing

Layer / File(s)	Summary
Hermes dashboard seeder and outcome handling src/lib/sandbox/config.ts, src/lib/sandbox/hermes-dashboard-reseed.test.ts	Inspects dashboard-home, runs the Hermes seeder through trusted Python binaries, classifies outcomes, and validates argv-safe execution.
Inference-set integration and routing status src/lib/actions/inference-set.ts, src/lib/actions/inference-set-gateway-restart.ts, src/lib/actions/inference-set.test-support.ts	Wires Hermes dashboard reseeding after successful updates, propagates dashboardConverged, and suppresses the sync message when convergence fails.
Routing behavior coverage src/lib/actions/inference-set-hermes-run.test.ts, src/lib/actions/inference-set-openclaw-run.test.ts	Verifies ordering, write and hash failure suppression, convergence logging, absent-dashboard handling, and OpenClaw exclusion.
Live dashboard validation test/e2e/live/hermes-inference-switch.test.ts	Enables the dashboard and verifies its switched config and model-info API response.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant InferenceSet
  participant SandboxConfig
  participant OpenShell
  participant GatewayRestart
  InferenceSet->>SandboxConfig: write in-sandbox gateway configuration
  SandboxConfig-->>InferenceSet: configuration and hash update succeeds
  InferenceSet->>SandboxConfig: seed Hermes dashboard configuration
  SandboxConfig->>OpenShell: inspect dashboard-home and run seeder
  OpenShell-->>SandboxConfig: return convergence status
  SandboxConfig-->>InferenceSet: return converged, absent, or failed
  InferenceSet->>GatewayRestart: provide dashboardConverged
  GatewayRestart-->>InferenceSet: emit or suppress sync message

Loading

Possibly related issues

• [DGX Station][Hermes][Inference] nemoclaw inference set leaves dashboard chat on previous model #6893 — Covers reseeding Hermes’s isolated dashboard configuration after inference route changes.

Possibly related PRs

• NVIDIA/NemoClaw#6897 — Overlaps Hermes dashboard convergence propagation and sync-log handling.
• NVIDIA/NemoClaw#3319 — Also updates model/provider consistency and config-hash handling in inference routing.

Suggested labels: integration: hermes, bug-fix, area: sandbox

Suggested reviewers: ericksoa

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: re-seeding Hermes dashboard config during in-place inference set for Hermes sandboxes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/hermes-dashboard-reseed-inference-set-6893

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the fix/hermes-dashboard... branch remains at 80%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.

File	main 627154d	fix/hermes-dashboard... 45d6db0	+/-
src/lib/agent/dashboard-ui.ts	91%	85%	-6%
src/lib/onboard...shboard-port.ts	94%	89%	-5%
src/lib/messagi...nes/template.ts	100%	95%	-5%
src/lib/state/config-io.ts	95%	92%	-3%
src/lib/messagi.../persistence.ts	86%	89%	+3%
src/lib/messagi...n-validation.ts	96%	100%	+4%
src/lib/policy/tiers.ts	92%	96%	+4%
src/lib/sandbox/config.ts	68%	73%	+5%
src/lib/adapter...shell/client.ts	83%	88%	+5%
src/lib/core/pr...mpt-activity.ts	67%	92%	+25%

---

_{Updated July 15, 2026 17:13 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

[github-actions]

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Model comparison: normalized findings match; normalized E2E selections differ; severity counts match.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, hermes-inference-switch, inference-routing, network-policy

1 optional E2E recommendation

• hermes-dashboard

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

src/lib/sandbox/config.ts (1)

615-632: 🔒 Security & Privacy | 🔵 Trivial | ⚖️ Poor tradeoff

Shell-script construction instead of structured argv for a security-boundary file.

seedHermesDashboardConfig builds a for/&&/exec control-flow string and runs it via sh -c rather than invoking the seeder through argv arrays. All interpolated values are shellQuote-escaped and derived from trusted internal state (constants + resolved AgentConfigTarget), so exploitability here is low, but this still departs from the stated preference for this file.

Since the goal is only "pick the first executable python3 candidate," this could alternatively be resolved without a shell wrapper (e.g., have the caller/adapter probe candidates or pass the candidate list as discrete argv items to a small trusted resolver), keeping sh -c string construction out of this security boundary.

As per path instructions, "src/lib/sandbox/{config,privileged-exec}.ts: Treat this as a security boundary. ... Prefer argv arrays and structured APIs over shell command construction."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/sandbox/config.ts` around lines 615 - 632, Update
seedHermesDashboardConfig to avoid constructing a shell control-flow string and
invoking sh -c. Resolve the first executable Python candidate through structured
argv/API handling, then invoke HERMES_DASHBOARD_SEEDER_PATH with discrete
arguments via captureOpenshellCommand, preserving the existing failure result
when no candidate is available.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/sandbox/config.ts`:
- Around line 615-632: Add direct unit tests for seedHermesDashboardConfig using
adversarial configPath and configDir values, including spaces and shell
metacharacters, and mock captureOpenshellCommand/getOpenshellBinary to inspect
the generated command safely. Cover a failing seed execution with stderr/stdout
handling and assert the helper returns false while preserving safe shell
quoting.

---

Nitpick comments:
In `@src/lib/sandbox/config.ts`:
- Around line 615-632: Update seedHermesDashboardConfig to avoid constructing a
shell control-flow string and invoking sh -c. Resolve the first executable
Python candidate through structured argv/API handling, then invoke
HERMES_DASHBOARD_SEEDER_PATH with discrete arguments via
captureOpenshellCommand, preserving the existing failure result when no
candidate is available.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ace5f16d-1a1b-4abc-9963-54b85538ac3d

📥 Commits

Reviewing files that changed from the base of the PR and between 8599fc7 and 82e07e1.

📒 Files selected for processing (5)

• src/lib/actions/inference-set-hermes-run.test.ts
• src/lib/actions/inference-set-openclaw-run.test.ts
• src/lib/actions/inference-set.test-support.ts
• src/lib/actions/inference-set.ts
• src/lib/sandbox/config.ts

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/sandbox/config.ts`:
- Line 632: Update the dashboard path validation command near the dashboard-home
check so only a genuinely nonexistent path emits HERMES_DASHBOARD_ABSENT_MARKER
and returns "absent"; treat files, broken symlinks, and inspection errors as
failures instead. Ensure the caller cannot report a synced route for
present-but-invalid or uninspectable dashboard paths.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 82724a68-fbb1-4edc-9733-80f6f4736b9f

📥 Commits

Reviewing files that changed from the base of the PR and between 82e07e1 and 6e12a12.

📒 Files selected for processing (5)

• src/lib/actions/inference-set-gateway-restart.ts
• src/lib/actions/inference-set-hermes-run.test.ts
• src/lib/actions/inference-set.test-support.ts
• src/lib/actions/inference-set.ts
• src/lib/sandbox/config.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/lib/actions/inference-set.test-support.ts

[github-actions]

E2E Target Results — ✅ All requested tests passed

Run: 29389872917
Workflow ref: fix/hermes-dashboard-reseed-inference-set-6893
Requested targets: (default — all supported)
Requested test IDs: hermes-inference-switch
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped, 0 unknown

Test	Result	Total wall clock time
hermes-inference-switch	✅ success	—

[github-actions]

E2E Target Results — ✅ All requested tests passed

Run: 29390643378
Workflow ref: fix/hermes-dashboard-reseed-inference-set-6893
Requested targets: (default — all supported)
Requested test IDs: inference-routing
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped, 0 unknown

Test	Result	Total wall clock time
inference-routing	✅ success	2m 12s

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/sandbox/config.ts`:
- Around line 645-696: Add a finite timeout option to the shared capture wrapper
inside seedHermesDashboardConfig, passing it to every captureOpenshellCommand
invocation used for Python detection, dashboard inspection, and seeding.
Preserve the existing failed-result handling so timed-out calls return the
best-effort "failed" result without blocking indefinitely.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c36de39d-ea23-467e-b55b-41a3b6ed4f0f

📥 Commits

Reviewing files that changed from the base of the PR and between 6e12a12 and 437757e.

📒 Files selected for processing (3)

• src/lib/actions/inference-set-hermes-run.test.ts
• src/lib/sandbox/config.ts
• src/lib/sandbox/hermes-dashboard-reseed.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/lib/actions/inference-set-hermes-run.test.ts

[ericksoa]

PRA-1 addressed in 1b5ba245: successful dashboard convergence now requires the trusted seeder’s exact model-routing write marker for the requested config path. Zero-status skip/no-route outcomes now fail post-commit convergence, while an absent dashboard remains a clean no-op. Added regressions for PyYAML unavailable, missing or unreadable gateway config, no model routing, and a mismatched destination marker.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-6898.docs.buildwithfern.com/nemoclaw

[cv]

Maintainer review complete for commit e1e00a9. The seeder now requires write evidence before declaring dashboard convergence, zero-exit no-ops fail the post-commit command path, argv handling remains injection-safe, and user recovery behavior is documented. Focused tests, typecheck, hooks, and Fern docs validation pass.