From 5958362da3767b49f3dcce30ac5915e64428d0e2 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 11:28:01 -0700 Subject: [PATCH 01/31] fix(inference): contain shared gateway routes Signed-off-by: Aaron Erickson --- docs/inference/switch-inference-providers.mdx | 10 + docs/reference/commands-nemohermes.mdx | 21 +- docs/reference/commands.mdx | 21 +- docs/reference/troubleshooting.mdx | 28 ++ src/commands/inference/set.ts | 11 +- .../inference/oclif-command-adapters.test.ts | 3 +- src/commands/sandbox/inference/set.ts | 9 +- .../inference-set-compatible-provider.test.ts | 2 + ...ence-set-gateway-route-containment.test.ts | 147 +++++++++++ .../actions/inference-set-hermes-run.test.ts | 3 + .../inference-set-openclaw-run.test.ts | 3 + src/lib/actions/inference-set.ts | 136 +++++++--- .../sandbox/connect-route-lifecycle.test.ts | 79 ++++++ .../sandbox/connect-route-repair.test.ts | 20 ++ src/lib/actions/sandbox/connect.ts | 22 ++ .../gateway-route-compatibility.test.ts | 240 ++++++++++++++++++ .../inference/gateway-route-compatibility.ts | 231 +++++++++++++++++ src/lib/onboard.ts | 14 + .../onboard/machine/core-flow-phases.test.ts | 1 + .../handlers/provider-inference.test.ts | 37 +++ .../machine/handlers/provider-inference.ts | 13 + .../setup-inference-route-containment.test.ts | 65 +++++ src/lib/onboard/setup-inference.ts | 18 ++ test/cli/list-inference.test.ts | 16 +- test/onboard-inference-smoke.test.ts | 3 + test/onboard-selection.test.ts | 11 +- test/support/connect-flow-test-harness.ts | 24 +- test/support/setup-inference-test-harness.ts | 1 + 28 files changed, 1118 insertions(+), 71 deletions(-) create mode 100644 src/lib/actions/inference-set-gateway-route-containment.test.ts create mode 100644 src/lib/inference/gateway-route-compatibility.test.ts create mode 100644 src/lib/inference/gateway-route-compatibility.ts create mode 100644 src/lib/onboard/setup-inference-route-containment.test.ts diff --git a/docs/inference/switch-inference-providers.mdx b/docs/inference/switch-inference-providers.mdx index 4019fcb7313..9c7e8934277 100644 --- a/docs/inference/switch-inference-providers.mdx +++ b/docs/inference/switch-inference-providers.mdx @@ -27,6 +27,15 @@ If you run `$$nemoclaw inference set` before NemoClaw has a selected sandbox, th Run `$$nemoclaw onboard` first, then retry the switch. + +OpenShell currently exposes one inference route per gateway, not one route per sandbox. +All registered sandboxes on the same gateway must use the same provider and model. +Custom compatible routes must also use the same endpoint and API family. +NemoClaw checks stopped sandboxes because they depend on that route when restarted. +When a route conflicts, onboarding, runtime switching, and connect-time repair exit non-zero before changing the gateway and name the affected sandboxes. +Align the routes, remove the conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`. + + ## Find the Provider Name The `--provider` value must match a provider already configured on the sandbox. @@ -170,6 +179,7 @@ The error includes a redacted OpenShell detail, lists the registered providers w Run `$$nemoclaw onboard` to register the provider, then retry `$$nemoclaw inference set`. Use `--no-verify` only when OpenShell cannot verify the provider at switch time but you have already confirmed the provider and credential. +`--no-verify` does not bypass the one-route-per-gateway compatibility check. ## Tune Model Metadata diff --git a/docs/reference/commands-nemohermes.mdx b/docs/reference/commands-nemohermes.mdx index a7a0f303380..71f945757df 100644 --- a/docs/reference/commands-nemohermes.mdx +++ b/docs/reference/commands-nemohermes.mdx @@ -536,8 +536,10 @@ If another terminal is already connected to the sandbox, `connect` prints a note An unknown model slug, malformed extra-args JSON, or a gated model (for example `deepseek-r1-distill-70b`) with no `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN` exits non-zero with the same error the installer would emit, before any sandbox readiness probe or SSH attach. Unset the managed-vLLM variable, or fix the value, before retrying. -When the live OpenShell gateway inference route differs from the route recorded in the NemoClaw registry, `connect` prints an explicit warning and realigns the shared gateway to the recorded route. -Use `nemohermes inference set --provider --model ` to make an intentional route change. +When the live OpenShell gateway inference route differs from the route recorded in the NemoClaw registry, `connect` checks every registered sandbox on that gateway before attempting a repair. +It realigns the route only when those registry entries are compatible with the requested provider and model. +If another sandbox records a conflicting route, `connect` exits non-zero without changing the gateway and names the affected sandboxes. +Use `nemohermes inference set --provider --model ` to make an intentional compatible route change. If the sandbox is registered locally but missing from a healthy gateway, `connect` preserves the registry entry and points you to `rebuild --yes`, `onboard`, or `destroy` instead of deleting the metadata needed for recovery. After a host reboot, the OpenShell gateway rotates its SSH host keys. @@ -1885,6 +1887,12 @@ By default, the command syncs the default registered sandbox. The command refuses before changing the OpenShell route when the selected sandbox has shields up. Run `nemohermes shields down`, apply the inference change, then run `nemohermes shields up` again. +Each OpenShell gateway exposes one inference route to every sandbox registered on that gateway. +Before changing the route, NemoClaw compares the requested provider and model with every same-gateway registry entry, including stopped sandboxes. +Custom compatible routes must also have matching normalized endpoint URLs and API families. +If a route conflicts or a legacy custom route lacks enough endpoint or API-family metadata to prove compatibility, the command exits non-zero before changing the OpenShell route, agent config, or host registry and names the conflicting sandboxes. +Align those sandboxes to the same route, remove the conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`. + ```bash nemohermes inference set --provider --model [--sandbox ] [--no-verify] [--endpoint-url ] [--credential-env ] [--inference-api ] ``` @@ -1897,19 +1905,20 @@ nemohermes my-assistant inference set --provider nvidia-prod --model nvidia/nemo ``` Pass both `--provider` and `--model` when you want NemoClaw to update the OpenShell inference route and sync the selected sandbox's agent config. -If you only want the lower-level OpenShell route operation, run `openshell inference set -g nemoclaw --model --provider ` directly. -When either flag is missing, `nemohermes inference set` prints that OpenShell command instead of an oclif flag-validation error. +NemoClaw resolves the OpenShell gateway from the target sandbox's recorded gateway binding, including non-default `NEMOCLAW_GATEWAY_PORT` deployments. +Do not run `openshell inference set` directly on a shared NemoClaw gateway because that bypasses registry compatibility checks and can break other sandboxes. +When either flag is missing, `nemohermes inference set` reports both required flags without suggesting a raw OpenShell command. The command updates the host registry immediately after the gateway route changes. If the in-sandbox config sync fails, NemoClaw keeps the gateway and registry aligned, warns that the running image may still need a rebuild, and points you to `nemohermes rebuild`. Supported provider names are `nvidia-prod`, `nvidia-nim`, `nvidia-router`, `openai-api`, `anthropic-prod`, `compatible-anthropic-endpoint`, `gemini-api`, `compatible-endpoint`, `hermes-provider`, `ollama-local`, and `vllm-local`. Use `--no-verify` only when OpenShell cannot verify the provider at switch time but you have already confirmed the provider and credential. -When switching to `compatible-endpoint` or `compatible-anthropic-endpoint` from a different provider family, pass `--endpoint-url` with the trusted custom provider URL so NemoClaw can persist durable rebuild metadata. +When switching to `compatible-endpoint` or `compatible-anthropic-endpoint` from a different provider family, pass `--endpoint-url` with the trusted custom provider URL and `--inference-api` with its API family so NemoClaw can persist a complete route identity for rebuild and shared-gateway checks. NemoClaw rejects loopback, link-local, private, and internal endpoint addresses, including public hostnames that resolve to a private address. For public HTTP URLs, NemoClaw stores the validated IP address to prevent DNS rebinding. DNS-backed HTTPS URLs are rejected because NemoClaw cannot pin the downstream peer address while preserving TLS SNI and host validation across the OpenShell runtime boundary; HTTPS IP-literal URLs remain supported. NemoClaw accepts `http://host.openshell.internal:` only with an explicit port from `1024` through `65535`; this narrow exception supports NemoClaw's sandbox-to-host inference routes and is not a general private-endpoint bypass. -`--credential-env` and `--inference-api` may also be supplied for the compatible provider metadata; supported API values are `openai-completions`, `anthropic-messages`, and `openai-responses`. +`--credential-env` may also be supplied for compatible provider metadata; supported `--inference-api` values are `openai-completions`, `anthropic-messages`, and `openai-responses`. ### `nemohermes setup` diff --git a/docs/reference/commands.mdx b/docs/reference/commands.mdx index be6cc145ff6..407f9a2429e 100644 --- a/docs/reference/commands.mdx +++ b/docs/reference/commands.mdx @@ -640,8 +640,10 @@ If another terminal is already connected to the sandbox, `connect` prints a note An unknown model slug, malformed extra-args JSON, or a gated model (for example `deepseek-r1-distill-70b`) with no `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN` exits non-zero with the same error the installer would emit, before any sandbox readiness probe or SSH attach. Unset the managed-vLLM variable, or fix the value, before retrying. -When the live OpenShell gateway inference route differs from the route recorded in the NemoClaw registry, `connect` prints an explicit warning and realigns the shared gateway to the recorded route. -Use `$$nemoclaw inference set --provider --model ` to make an intentional route change. +When the live OpenShell gateway inference route differs from the route recorded in the NemoClaw registry, `connect` checks every registered sandbox on that gateway before attempting a repair. +It realigns the route only when those registry entries are compatible with the requested provider and model. +If another sandbox records a conflicting route, `connect` exits non-zero without changing the gateway and names the affected sandboxes. +Use `$$nemoclaw inference set --provider --model ` to make an intentional compatible route change. If the sandbox is registered locally but missing from a healthy gateway, `connect` preserves the registry entry and points you to `rebuild --yes`, `onboard`, or `destroy` instead of deleting the metadata needed for recovery. After a host reboot, the OpenShell gateway rotates its SSH host keys. @@ -2309,6 +2311,12 @@ By default, the command syncs the default registered sandbox. The command refuses before changing the OpenShell route when the selected sandbox has shields up. Run `$$nemoclaw shields down`, apply the inference change, then run `$$nemoclaw shields up` again. +Each OpenShell gateway exposes one inference route to every sandbox registered on that gateway. +Before changing the route, NemoClaw compares the requested provider and model with every same-gateway registry entry, including stopped sandboxes. +Custom compatible routes must also have matching normalized endpoint URLs and API families. +If a route conflicts or a legacy custom route lacks enough endpoint or API-family metadata to prove compatibility, the command exits non-zero before changing the OpenShell route, agent config, or host registry and names the conflicting sandboxes. +Align those sandboxes to the same route, remove the conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`. + ```bash $$nemoclaw inference set --provider --model [--sandbox ] [--no-verify] [--endpoint-url ] [--credential-env ] [--inference-api ] ``` @@ -2321,19 +2329,20 @@ $$nemoclaw my-assistant inference set --provider nvidia-prod --model nvidia/nemo ``` Pass both `--provider` and `--model` when you want NemoClaw to update the OpenShell inference route and sync the selected sandbox's agent config. -If you only want the lower-level OpenShell route operation, run `openshell inference set -g nemoclaw --model --provider ` directly. -When either flag is missing, `$$nemoclaw inference set` prints that OpenShell command instead of an oclif flag-validation error. +NemoClaw resolves the OpenShell gateway from the target sandbox's recorded gateway binding, including non-default `NEMOCLAW_GATEWAY_PORT` deployments. +Do not run `openshell inference set` directly on a shared NemoClaw gateway because that bypasses registry compatibility checks and can break other sandboxes. +When either flag is missing, `$$nemoclaw inference set` reports both required flags without suggesting a raw OpenShell command. The command updates the host registry immediately after the gateway route changes. If the in-sandbox config sync fails, NemoClaw keeps the gateway and registry aligned, warns that the running image may still need a rebuild, and points you to `$$nemoclaw rebuild`. Supported provider names are `nvidia-prod`, `nvidia-nim`, `nvidia-router`, `openai-api`, `anthropic-prod`, `compatible-anthropic-endpoint`, `gemini-api`, `compatible-endpoint`, `hermes-provider`, `ollama-local`, and `vllm-local`. Use `--no-verify` only when OpenShell cannot verify the provider at switch time but you have already confirmed the provider and credential. -When switching to `compatible-endpoint` or `compatible-anthropic-endpoint` from a different provider family, pass `--endpoint-url` with the trusted custom provider URL so NemoClaw can persist durable rebuild metadata. +When switching to `compatible-endpoint` or `compatible-anthropic-endpoint` from a different provider family, pass `--endpoint-url` with the trusted custom provider URL and `--inference-api` with its API family so NemoClaw can persist a complete route identity for rebuild and shared-gateway checks. NemoClaw rejects loopback, link-local, private, and internal endpoint addresses, including public hostnames that resolve to a private address. For public HTTP URLs, NemoClaw stores the validated IP address to prevent DNS rebinding. DNS-backed HTTPS URLs are rejected because NemoClaw cannot pin the downstream peer address while preserving TLS SNI and host validation across the OpenShell runtime boundary; HTTPS IP-literal URLs remain supported. NemoClaw accepts `http://host.openshell.internal:` only with an explicit port from `1024` through `65535`; this narrow exception supports NemoClaw's sandbox-to-host inference routes and is not a general private-endpoint bypass. -`--credential-env` and `--inference-api` may also be supplied for the compatible provider metadata; supported API values are `openai-completions`, `anthropic-messages`, and `openai-responses`. +`--credential-env` may also be supplied for compatible provider metadata; supported `--inference-api` values are `openai-completions`, `anthropic-messages`, and `openai-responses`. ### `$$nemoclaw setup` diff --git a/docs/reference/troubleshooting.mdx b/docs/reference/troubleshooting.mdx index 861ffc077ee..3b2ed10a585 100644 --- a/docs/reference/troubleshooting.mdx +++ b/docs/reference/troubleshooting.mdx @@ -323,6 +323,34 @@ $$nemoclaw list `$$nemoclaw list` prints the recorded dashboard URL for each sandbox. +OpenShell currently exposes one inference route per gateway. +Sandboxes that share a gateway must record the same provider and model. +Custom compatible routes must also record the same normalized endpoint and API family. +NemoClaw includes stopped sandboxes in this check because they use the gateway route when restarted. + +If onboarding, `inference set`, or connect-time route repair reports a shared-gateway conflict, choose one of these recovery paths: + +- Align every named sandbox to the same route. +- Remove a conflicting sandbox with `$$nemoclaw destroy` when you no longer need it. +- Onboard the sandbox against another gateway port when it needs a different route. + +An aligned group cannot move to a wholly new route one sandbox at a time because the first change would conflict with the remaining registry rows. +Back up each workspace, remove or migrate the peer sandboxes, switch the last sandbox on the original gateway, then re-onboard the peers with that same route or with separate gateway ports. + +If the error names incomplete legacy custom-route metadata, back up and remove the affected sandbox, then re-onboard it with an explicit custom endpoint and API family. +If the error names an invalid gateway binding, restore the affected row's known-good `gatewayName` and `gatewayPort` metadata from a trusted backup; otherwise back up and remove the sandbox, then re-onboard it. +Do not guess or copy a binding from another sandbox because lifecycle commands use it to select the gateway. + +For example, keep the first sandbox on the default gateway and assign a separate gateway to the second sandbox: + +```bash +$$nemoclaw onboard --name first-sandbox +NEMOCLAW_GATEWAY_PORT=8990 $$nemoclaw onboard --name second-sandbox +``` + +NemoClaw derives a separate gateway name and state directory from the second port. +Using a different dashboard port alone does not create a separate inference route. + ## Onboarding ### Cgroup v2 errors during onboard diff --git a/src/commands/inference/set.ts b/src/commands/inference/set.ts index 6d5398b7c44..a8c85b0ea19 100644 --- a/src/commands/inference/set.ts +++ b/src/commands/inference/set.ts @@ -49,7 +49,7 @@ export default class InferenceSetCommand extends NemoClawCommand { public async run(): Promise { const { flags } = await this.parse(InferenceSetCommand); if (!flags.provider || !flags.model) { - this.printOpenShellRedirect(); + this.printRequiredFlags(); return; } try { @@ -71,14 +71,13 @@ export default class InferenceSetCommand extends NemoClawCommand { } } - private printOpenShellRedirect(): void { + private printRequiredFlags(): void { this.failWithLines( [ - ` Unknown ${CLI_NAME} command: inference set`, + ` ${CLI_NAME} inference set requires --provider and --model.`, "", - " This operation belongs to OpenShell.", - " Run: openshell inference set -g nemoclaw --model --provider ", - ` To also sync the running sandbox config, pass --provider and --model to ${CLI_NAME} inference set.`, + ` Run: ${CLI_NAME} inference set --provider --model [--sandbox ]`, + " NemoClaw must perform this operation so it can protect every sandbox sharing the target gateway.", "", ` Run '${CLI_NAME} help' for NemoClaw commands.`, ], diff --git a/src/commands/sandbox/inference/oclif-command-adapters.test.ts b/src/commands/sandbox/inference/oclif-command-adapters.test.ts index c3d96e6ee9c..457ea8c9d03 100644 --- a/src/commands/sandbox/inference/oclif-command-adapters.test.ts +++ b/src/commands/sandbox/inference/oclif-command-adapters.test.ts @@ -89,7 +89,7 @@ describe("sandbox inference oclif command adapters (#5977)", () => { }); }); - it("prints the missing-flags redirect without calling runInferenceSet", async () => { + it("prints guarded missing-flags guidance without calling runInferenceSet", async () => { const error = vi.spyOn(console, "error").mockImplementation(() => undefined); const previousExitCode = process.exitCode; process.exitCode = undefined; @@ -101,6 +101,7 @@ describe("sandbox inference oclif command adapters (#5977)", () => { expect(error).toHaveBeenCalledWith( expect.stringContaining("inference set requires --provider and --model"), ); + expect(error).not.toHaveBeenCalledWith(expect.stringContaining("openshell inference set")); } finally { process.exitCode = previousExitCode; error.mockRestore(); diff --git a/src/commands/sandbox/inference/set.ts b/src/commands/sandbox/inference/set.ts index 3631c0fa87a..aa337d8f88a 100644 --- a/src/commands/sandbox/inference/set.ts +++ b/src/commands/sandbox/inference/set.ts @@ -53,7 +53,7 @@ export default class SandboxInferenceSetCommand extends NemoClawCommand { public async run(): Promise { const { args, flags } = await this.parse(SandboxInferenceSetCommand); if (!flags.provider || !flags.model) { - this.printOpenShellRedirect(); + this.printRequiredFlags(); return; } try { @@ -75,14 +75,13 @@ export default class SandboxInferenceSetCommand extends NemoClawCommand { } } - private printOpenShellRedirect(): void { + private printRequiredFlags(): void { this.failWithLines( [ ` ${CLI_NAME} inference set requires --provider and --model.`, "", - " To change only the OpenShell route, run:", - " openshell inference set -g nemoclaw --model --provider ", - ` To also sync the sandbox config, pass --provider and --model to ${CLI_NAME} inference set.`, + ` Run: ${CLI_NAME} inference set --provider --model `, + " NemoClaw must perform this operation so it can protect every sandbox sharing the target gateway.", "", ` Run '${CLI_NAME} help' for NemoClaw commands.`, ], diff --git a/src/lib/actions/inference-set-compatible-provider.test.ts b/src/lib/actions/inference-set-compatible-provider.test.ts index fb8d0ecf59f..9bfc05c7d2c 100644 --- a/src/lib/actions/inference-set-compatible-provider.test.ts +++ b/src/lib/actions/inference-set-compatible-provider.test.ts @@ -329,6 +329,8 @@ describe("runInferenceSet compatible providers", () => { provider === "compatible-endpoint" ? "COMPATIBLE_API_KEY" : "COMPATIBLE_ANTHROPIC_API_KEY", + inferenceApi: + provider === "compatible-endpoint" ? "openai-completions" : "anthropic-messages", }, deps, ), diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts new file mode 100644 index 00000000000..cd3396826f4 --- /dev/null +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -0,0 +1,147 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it } from "vitest"; +import type { SandboxEntry } from "../state/registry"; +import { runInferenceSet } from "./inference-set"; +import { createDeps } from "./inference-set.test-support"; + +const entry = (name: string, overrides: Partial = {}): SandboxEntry => ({ + name, + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/model-a", + ...overrides, +}); + +describe("runtime shared gateway route containment", () => { + it("rejects a same-gateway conflict before OpenShell, config, or registry mutation (#6315)", async () => { + const deps = createDeps({ + config: {}, + entries: [entry("alpha"), entry("stopped-peer")], + defaultSandbox: "alpha", + }); + + await expect( + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-b", sandboxName: "alpha" }, + deps, + ), + ).rejects.toThrow("stopped-peer"); + + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.writeSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + expect(deps.calls.updateSession).not.toHaveBeenCalled(); + expect(deps.calls.appendAuditEntry).not.toHaveBeenCalled(); + }); + + it("targets the selected sandbox gateway and allows a conflicting route elsewhere (#6315)", async () => { + const deps = createDeps({ + config: {}, + entries: [ + entry("alpha", { gatewayName: "nemoclaw-9090", gatewayPort: 9090 }), + entry("default-gateway-peer"), + ], + defaultSandbox: "alpha", + contextWindow: 32_768, + }); + + await expect( + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-b", sandboxName: "alpha" }, + deps, + ), + ).resolves.toMatchObject({ sandboxName: "alpha", model: "nvidia/model-b" }); + + expect(deps.calls.captureOpenshell).toHaveBeenCalledWith( + [ + "inference", + "set", + "-g", + "nemoclaw-9090", + "--provider", + "nvidia-prod", + "--model", + "nvidia/model-b", + ], + expect.objectContaining({ ignoreError: true }), + ); + }); + + it("blocks a custom endpoint conflict before DNS validation or mutation (#6315)", async () => { + const deps = createDeps({ + config: {}, + entries: [ + entry("alpha", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://alpha.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + }), + entry("custom-peer", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://peer.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + }), + ], + defaultSandbox: "alpha", + }); + + await expect( + runInferenceSet( + { + provider: "compatible-endpoint", + model: "custom/model", + sandboxName: "alpha", + endpointUrl: "https://alpha.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + inferenceApi: "openai-completions", + }, + deps, + ), + ).rejects.toThrow("custom-peer"); + + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).not.toHaveBeenCalled(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + }); + + it("blocks an incomplete legacy custom target even without a peer (#6315)", async () => { + const deps = createDeps({ + config: {}, + entries: [ + entry("alpha", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: null, + preferredInferenceApi: null, + }), + ], + defaultSandbox: "alpha", + }); + + await expect( + runInferenceSet( + { + provider: "compatible-endpoint", + model: "custom/model", + sandboxName: "alpha", + }, + deps, + ), + ).rejects.toThrow("requested custom route lacks durable endpoint or API-family metadata"); + + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).not.toHaveBeenCalled(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + }); +}); diff --git a/src/lib/actions/inference-set-hermes-run.test.ts b/src/lib/actions/inference-set-hermes-run.test.ts index 8afd64e028b..d8818030675 100644 --- a/src/lib/actions/inference-set-hermes-run.test.ts +++ b/src/lib/actions/inference-set-hermes-run.test.ts @@ -192,6 +192,9 @@ describe("runInferenceSet Hermes routing", () => { agent: "hermes", provider: "compatible-anthropic-endpoint", model: "anthropic.claude-3-5-sonnet-20240620-v1:0", + endpointUrl: "https://inference.local/v1", + credentialEnv: "COMPATIBLE_ANTHROPIC_API_KEY", + preferredInferenceApi: "openai-completions", }, defaultSandbox: "hermes", target: HERMES_TARGET, diff --git a/src/lib/actions/inference-set-openclaw-run.test.ts b/src/lib/actions/inference-set-openclaw-run.test.ts index 8b2a6974a0d..785647d4b99 100644 --- a/src/lib/actions/inference-set-openclaw-run.test.ts +++ b/src/lib/actions/inference-set-openclaw-run.test.ts @@ -192,6 +192,9 @@ describe("runInferenceSet OpenClaw routing", () => { agent: "openclaw", provider: "compatible-anthropic-endpoint", model: "anthropic.claude-3-5-sonnet-20240620-v1:0", + endpointUrl: "https://inference.local/v1", + credentialEnv: "COMPATIBLE_ANTHROPIC_API_KEY", + preferredInferenceApi: "openai-completions", }, session: baseSession({ provider: "compatible-anthropic-endpoint", diff --git a/src/lib/actions/inference-set.ts b/src/lib/actions/inference-set.ts index 4af3566584b..1bde2348311 100644 --- a/src/lib/actions/inference-set.ts +++ b/src/lib/actions/inference-set.ts @@ -11,8 +11,13 @@ import { type SandboxInferenceConfig, } from "../inference/config"; import { resolveContextWindowForModel } from "../inference/context-window"; +import { + checkGatewayRouteCompatibility, + formatGatewayRouteConflict, +} from "../inference/gateway-route-compatibility"; import { type ValidationResult, validateLocalProvider } from "../inference/local"; import { inferenceSelectionRegistryFields } from "../inference/selection"; +import { resolveSandboxGatewayName } from "../onboard/gateway-binding"; import { ensureLocalProviderReachable } from "../onboard/local-inference-topology"; import { type AgentConfigTarget, @@ -351,6 +356,7 @@ function updateMatchingOnboardSession( } function openshellInferenceSetArgs(options: { + gatewayName: string; provider: string; model: string; noVerify?: boolean; @@ -359,7 +365,7 @@ function openshellInferenceSetArgs(options: { "inference", "set", "-g", - "nemoclaw", + options.gatewayName, "--provider", options.provider, "--model", @@ -430,19 +436,8 @@ export async function normalizeCustomEndpointUrl( value: string | null | undefined, rewriteUrlWithDnsPinning: InferenceSetDeps["rewriteConfigUrlsWithDnsPinning"], ): Promise { - const raw = typeof value === "string" ? value.trim() : ""; - if (!raw) - throw new InferenceSetError("endpoint-url is required for custom-compatible metadata.", 2); - let shaped: { url: URL; normalized: string }; - try { - shaped = normalizeEndpointUrlShape(raw); - } catch { - throw new InferenceSetError( - "endpoint-url must be a valid http(s) URL without embedded credentials.", - 2, - ); - } - + const normalized = normalizeCustomEndpointUrlWithoutDns(value); + const shaped = normalizeEndpointUrlShape(normalized); const hostname = shaped.url.hostname.replace(/\.$/, "").toLowerCase(); const port = Number(shaped.url.port); if ( @@ -456,11 +451,11 @@ export async function normalizeCustomEndpointUrl( // unprivileged HTTP listeners; do not generalize this exemption to HTTPS, // default/privileged ports, localhost, RFC1918 addresses, or arbitrary // internal DNS names. - return shaped.normalized; + return normalized; } try { - const validated = await rewriteUrlWithDnsPinning(shaped.normalized); + const validated = await rewriteUrlWithDnsPinning(normalized); if (typeof validated !== "string") throw new Error("URL validator returned a non-string value"); return normalizeEndpointUrlShape(validated).normalized; } catch (error) { @@ -469,6 +464,22 @@ export async function normalizeCustomEndpointUrl( } } +function normalizeCustomEndpointUrlWithoutDns(value: string | null | undefined): string { + const raw = typeof value === "string" ? value.trim() : ""; + if (!raw) + throw new InferenceSetError("endpoint-url is required for custom-compatible metadata.", 2); + let shaped: { url: URL; normalized: string }; + try { + shaped = normalizeEndpointUrlShape(raw); + } catch { + throw new InferenceSetError( + "endpoint-url must be a valid http(s) URL without embedded credentials.", + 2, + ); + } + return shaped.normalized; +} + function normalizeExplicitCredentialEnv( provider: string, value: string | null | undefined, @@ -490,12 +501,14 @@ function allowedExplicitInferenceApis(provider: string): string[] { : Array.from(INFERENCE_SET_APIS); } -function normalizeExplicitInferenceApi( - provider: string, - value: string | null | undefined, -): string | null { +function normalizeExplicitInferenceApi(provider: string, value: string | null | undefined): string { const normalized = typeof value === "string" ? value.trim() : ""; - if (!normalized) return null; + if (!normalized) { + throw new InferenceSetError( + `inference-api is required for '${provider}' so the shared gateway route can be identified safely.`, + 2, + ); + } const allowed = allowedExplicitInferenceApis(provider); if (!allowed.includes(normalized)) { throw new InferenceSetError( @@ -506,11 +519,10 @@ function normalizeExplicitInferenceApi( return normalized; } -async function explicitCustomProviderMetadata( +function explicitCustomProviderMetadataWithoutDns( provider: string, options: InferenceSetOptions, - rewriteUrlWithDnsPinning: InferenceSetDeps["rewriteConfigUrlsWithDnsPinning"], -): Promise { +): RegistryInferenceMetadata | null { if (!hasExplicitCustomMetadata(options)) return null; if (!isCustomCompatibleProvider(provider)) { throw new InferenceSetError( @@ -525,13 +537,24 @@ async function explicitCustomProviderMetadata( // for this switch, after URL and credential-env validation, instead of // borrowing from an unrelated onboard session or global OpenShell provider. return { - endpointUrl: await normalizeCustomEndpointUrl(options.endpointUrl, rewriteUrlWithDnsPinning), + endpointUrl: normalizeCustomEndpointUrlWithoutDns(options.endpointUrl), credentialEnv: normalizeExplicitCredentialEnv(provider, options.credentialEnv), preferredInferenceApi: normalizeExplicitInferenceApi(provider, options.inferenceApi), nimContainer: null, }; } +async function materializeExplicitCustomProviderMetadata( + metadata: RegistryInferenceMetadata | null, + rewriteUrlWithDnsPinning: InferenceSetDeps["rewriteConfigUrlsWithDnsPinning"], +): Promise { + if (!metadata) return null; + return { + ...metadata, + endpointUrl: await normalizeCustomEndpointUrl(metadata.endpointUrl, rewriteUrlWithDnsPinning), + }; +} + function matchingSessionMetadata(options: { session: onboardSession.Session | null; sandboxName: string; @@ -611,6 +634,36 @@ async function runInferenceSetWithoutHostLock( 2, ); } + let gatewayName: string; + try { + gatewayName = resolveSandboxGatewayName(entry); + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + throw new InferenceSetError( + `Cannot resolve the OpenShell gateway for sandbox '${sandboxName}': ${detail}`, + 2, + ); + } + const session = deps.loadSession(); + const preliminaryExplicitMetadata = explicitCustomProviderMetadataWithoutDns(provider, options); + const preliminaryRegistryMetadata = registryMetadataForProviderSwitch({ + entry, + provider, + model, + sandboxName, + session, + explicitMetadata: preliminaryExplicitMetadata, + }); + const compatibility = checkGatewayRouteCompatibility({ + gatewayName, + sandboxName, + route: { provider, model, ...preliminaryRegistryMetadata }, + sandboxes: deps.listSandboxes().sandboxes, + }); + if (!compatibility.ok) { + throw new InferenceSetError(formatGatewayRouteConflict(compatibility), 2); + } + const target = deps.resolveAgentConfig(sandboxName); const targetAgent = normalizeSandboxAgent(target.agentName); if (targetAgent !== agentName) { @@ -625,21 +678,23 @@ async function runInferenceSetWithoutHostLock( 2, ); } - const session = deps.loadSession(); - const explicitMetadata = await explicitCustomProviderMetadata( - provider, - options, + const explicitMetadata = await materializeExplicitCustomProviderMetadata( + preliminaryExplicitMetadata, deps.rewriteConfigUrlsWithDnsPinning, ); const explicitPreferredInferenceApi = explicitMetadata?.preferredInferenceApi ?? null; - const registryMetadata = registryMetadataForProviderSwitch({ - entry, - provider, - model, - sandboxName, - session, - explicitMetadata, - }); + const registryMetadata = explicitMetadata ?? preliminaryRegistryMetadata; + if (explicitMetadata) { + const finalizedCompatibility = checkGatewayRouteCompatibility({ + gatewayName, + sandboxName, + route: { provider, model, ...registryMetadata }, + sandboxes: deps.listSandboxes().sandboxes, + }); + if (!finalizedCompatibility.ok) { + throw new InferenceSetError(formatGatewayRouteConflict(finalizedCompatibility), 2); + } + } // Local providers (ollama-local, vllm-local) route through the sandbox-facing // host.openshell.internal hostname, which the host-side `openshell inference set` @@ -672,7 +727,12 @@ async function runInferenceSetWithoutHostLock( deps.log(` Setting OpenShell inference route: ${provider} / ${model}`); const setResult = deps.captureOpenshell( - openshellInferenceSetArgs({ provider, model, noVerify: effectiveNoVerify }), + openshellInferenceSetArgs({ + gatewayName, + provider, + model, + noVerify: effectiveNoVerify, + }), { ignoreError: true, includeStreams: true, diff --git a/src/lib/actions/sandbox/connect-route-lifecycle.test.ts b/src/lib/actions/sandbox/connect-route-lifecycle.test.ts index e04d2ca93ae..0dec005661c 100644 --- a/src/lib/actions/sandbox/connect-route-lifecycle.test.ts +++ b/src/lib/actions/sandbox/connect-route-lifecycle.test.ts @@ -79,6 +79,85 @@ describe("connectSandbox route lifecycle", () => { ); }); + it("exits before connect-time route writes when another sandbox conflicts (#6315)", async () => { + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "anthropic-prod", + model: "claude-sonnet-4-20250514", + } as const; + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + registryEntry: alpha, + registryEntries: [ + alpha, + { + name: "stopped-peer", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + }, + ], + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( + "openshell", + ["sandbox", "connect", "alpha"], + expect.any(Object), + ); + const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); + expect(errorOutput).toContain("stopped-peer"); + expect(errorOutput).toContain("NEMOCLAW_GATEWAY_PORT"); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + + it("exits before repairing a lone incomplete legacy custom route (#6315)", async () => { + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + registryEntry: { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: null, + preferredInferenceApi: null, + }, + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( + "openshell", + ["sandbox", "connect", "alpha"], + expect.any(Object), + ); + const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); + expect(errorOutput).toContain( + "requested custom route lacks durable endpoint or API-family metadata", + ); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + it("wires the forced VM DNS monkeypatch into connect route repair", async () => { vi.stubEnv("NEMOCLAW_FORCE_VM_DNS_MONKEYPATCH", "1"); try { diff --git a/src/lib/actions/sandbox/connect-route-repair.test.ts b/src/lib/actions/sandbox/connect-route-repair.test.ts index c06d83a5f30..032de156f41 100644 --- a/src/lib/actions/sandbox/connect-route-repair.test.ts +++ b/src/lib/actions/sandbox/connect-route-repair.test.ts @@ -144,6 +144,26 @@ describe("sandbox connect route repair unit flow", () => { } }); + it("stops after a broken probe but before any repair mutation when routes conflict (#6315)", () => { + const conflict = new Error("shared gateway route conflict"); + const assertRouteCompatible = vi.fn(() => { + throw conflict; + }); + const { calls, deps } = makeRepairDeps([broken()], { assertRouteCompatible }); + + expect(() => + repairSandboxInferenceRouteWithDeps("vm-box", sandbox({ openshellDriver: "vm" }), {}, deps), + ).toThrow(conflict); + + expect(assertRouteCompatible).toHaveBeenCalledWith( + "vm-box", + expect.objectContaining({ name: "demo" }), + ); + expect(calls.monkeypatches).toEqual([]); + expect(calls.reapplications).toEqual([]); + expect(calls.legacyRepairs).toEqual([]); + }); + it("repairs legacy kubernetes routes through the DNS proxy path", () => { const { calls, deps } = makeRepairDeps([broken(), healthy()]); diff --git a/src/lib/actions/sandbox/connect.ts b/src/lib/actions/sandbox/connect.ts index 901ecb9efe8..3e1d6e812d7 100644 --- a/src/lib/actions/sandbox/connect.ts +++ b/src/lib/actions/sandbox/connect.ts @@ -23,6 +23,10 @@ import { planInferenceRouteReconcile, sanitizeRouteValueForDisplay, } from "../../inference/config"; +import { + assertGatewayRouteCompatibility, + GatewayRouteConflictError, +} from "../../inference/gateway-route-compatibility"; import { findReachableOllamaHost, probeLocalProviderHealth } from "../../inference/local"; import { ensureOllamaAuthProxy, probeOllamaAuthProxyHealth } from "../../inference/ollama/proxy"; import { LOCAL_INFERENCE_TIMEOUT_SECS } from "../../onboard/env"; @@ -124,6 +128,7 @@ export type SandboxInferenceRouteRepairDeps = { sandboxName: string, quiet: boolean, ) => { exitCode: number; message?: string | null }; + assertRouteCompatible?: (sandboxName: string, sb: SandboxEntry | null) => void; log?: (message: string) => void; error?: (message: string) => void; }; @@ -405,6 +410,16 @@ function buildInferenceSetArgs(provider: string, model: string): string[] { return args; } +function assertSandboxGatewayRouteCompatible(sandboxName: string, sb: SandboxEntry | null): void { + if (!sb) return; + assertGatewayRouteCompatibility({ + gatewayName: resolveSandboxGatewayName(sb), + sandboxName, + route: sb, + sandboxes: registry.listSandboxes().sandboxes, + }); +} + function reapplyVmInferenceRoute( sandboxName: string, sb: SandboxEntry | null, @@ -437,6 +452,7 @@ export function repairSandboxInferenceRouteWithDeps( if (!initialProbe.broken) { return { healthy: true, repairAttempted: false, detail: initialProbe.detail }; } + deps.assertRouteCompatible?.(sandboxName, sb); if (!shouldUseLegacyDnsProxyRepair(sb)) { if (deps.shouldApplyVmDnsMonkeypatch(sb)) { @@ -574,6 +590,7 @@ function repairSandboxInferenceRouteIfNeeded( { gatewayName: resolveSandboxGatewayName(sb), sandboxName: name }, { log: isQuiet ? () => undefined : console.log }, ), + assertRouteCompatible: assertSandboxGatewayRouteCompatible, }, ); } @@ -717,6 +734,7 @@ function ensureSandboxInferenceRoute( ); const plan = planInferenceRouteReconcile(live, { provider, model }); if (plan.kind !== "aligned") { + assertSandboxGatewayRouteCompatible(sandboxName, sb); const recordedRoute = `${sanitizeRouteValueForDisplay(provider)}/${sanitizeRouteValueForDisplay(model)}`; if (plan.kind === "diverged") { // Shared gateway: re-point loudly (even when quiet) — silent revert was @@ -759,6 +777,10 @@ function ensureSandboxInferenceRoute( return { sandbox: sb, routeHealthy: repairResult.healthy }; } catch (error) { if (!sb || inference?.kind !== "configured") return { sandbox: sb, routeHealthy: null }; + if (error instanceof GatewayRouteConflictError) { + console.error(` Error: ${error.message}`); + process.exit(1); + } const detail = error instanceof Error && error.message ? error.message : String(error); if (!quiet) { console.error(` Error: failed to verify or repair inference route: ${detail}`); diff --git a/src/lib/inference/gateway-route-compatibility.test.ts b/src/lib/inference/gateway-route-compatibility.test.ts new file mode 100644 index 00000000000..4fdc81c9988 --- /dev/null +++ b/src/lib/inference/gateway-route-compatibility.test.ts @@ -0,0 +1,240 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it } from "vitest"; +import type { SandboxEntry } from "../state/registry"; +import { + checkGatewayRouteCompatibility, + formatGatewayRouteConflict, + type GatewayInferenceRoute, +} from "./gateway-route-compatibility"; + +const route = ( + provider: string, + model: string, + overrides: Partial = {}, +): GatewayInferenceRoute => ({ + provider, + model, + endpointUrl: null, + preferredInferenceApi: null, + credentialEnv: null, + ...overrides, +}); + +const sandbox = (name: string, overrides: Partial = {}): SandboxEntry => ({ + name, + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/model-a", + ...overrides, +}); + +function check(requested: GatewayInferenceRoute, sandboxes: SandboxEntry[]) { + return checkGatewayRouteCompatibility({ + gatewayName: "nemoclaw", + sandboxName: "target", + route: requested, + sandboxes, + }); +} + +describe("shared gateway inference route compatibility", () => { + it("allows identical routes and ignores the target sandbox itself (#6315)", () => { + expect( + check(route("nvidia-prod", "nvidia/model-a"), [ + sandbox("target", { provider: "anthropic-prod", model: "claude-old" }), + sandbox("stopped-peer"), + ]), + ).toEqual({ ok: true }); + }); + + it("blocks provider or model conflicts from every same-gateway registry row (#6315)", () => { + const result = check(route("anthropic-prod", "claude-new"), [sandbox("stopped-peer")]); + + expect(result).toMatchObject({ + ok: false, + conflicts: [{ sandboxName: "stopped-peer", reason: "provider-model" }], + }); + if (result.ok) throw new Error("expected a route conflict"); + expect(formatGatewayRouteConflict(result)).toContain("Stopped sandboxes are included"); + }); + + it("allows different routes on different gateways (#6315)", () => { + expect( + check(route("anthropic-prod", "claude-new"), [ + sandbox("other-gateway", { + gatewayName: "nemoclaw-9090", + gatewayPort: 9090, + }), + ]), + ).toEqual({ ok: true }); + }); + + it("normalizes equivalent custom endpoint URLs before comparison (#6315)", () => { + expect( + check( + route("compatible-endpoint", "custom/model", { + endpointUrl: "https://EXAMPLE.test/v1/?token=ignored", + preferredInferenceApi: "openai-completions", + }), + [ + sandbox("custom-peer", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + }), + ], + ), + ).toEqual({ ok: true }); + }); + + it("normalizes Anthropic endpoint suffixes for custom route identity (#6315)", () => { + expect( + check( + route("compatible-anthropic-endpoint", "anthropic/model", { + endpointUrl: "https://example.test/v1/messages", + preferredInferenceApi: "anthropic-messages", + }), + [ + sandbox("anthropic-peer", { + provider: "compatible-anthropic-endpoint", + model: "anthropic/model", + endpointUrl: "https://example.test", + preferredInferenceApi: "anthropic-messages", + }), + ], + ), + ).toEqual({ ok: true }); + }); + + it("ignores credential environment differences in route identity (#6315)", () => { + expect( + check( + route("compatible-endpoint", "custom/model", { + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + credentialEnv: "REQUESTED_KEY", + }), + [ + sandbox("custom-peer", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + credentialEnv: "RECORDED_KEY", + }), + ], + ), + ).toEqual({ ok: true }); + }); + + it.each([ + [ + "endpoint", + { endpointUrl: "https://other.test/v1", preferredInferenceApi: "openai-completions" }, + "custom-endpoint", + ], + [ + "API family", + { endpointUrl: "https://example.test/v1", preferredInferenceApi: "openai-responses" }, + "custom-api", + ], + ] as const)("blocks custom %s conflicts (#6315)", (_label, recordedMetadata, reason) => { + const result = check( + route("compatible-endpoint", "custom/model", { + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + }), + [ + sandbox("custom-peer", { + provider: "compatible-endpoint", + model: "custom/model", + ...recordedMetadata, + }), + ], + ); + + expect(result).toMatchObject({ ok: false, conflicts: [{ reason }] }); + }); + + it.each([ + ["endpoint", null, "openai-completions"], + ["API family", "https://example.test/v1", null], + ] as const)("fails closed when legacy custom route %s metadata is missing (#6315)", (_label, endpointUrl, preferredInferenceApi) => { + const result = check( + route("compatible-endpoint", "custom/model", { + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + }), + [ + sandbox("legacy-custom", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl, + preferredInferenceApi, + }), + ], + ); + + expect(result).toMatchObject({ + ok: false, + conflicts: [{ sandboxName: "legacy-custom", reason: "incomplete-custom-route" }], + }); + if (result.ok) throw new Error("expected a route conflict"); + expect(formatGatewayRouteConflict(result)).toContain( + "remove and re-onboard that sandbox with complete custom-route metadata", + ); + }); + + it("fails closed when a requested custom route has no API metadata or peers (#6315)", () => { + const result = check( + route("compatible-endpoint", "custom/model", { + endpointUrl: "https://example.test/v1", + preferredInferenceApi: null, + }), + [], + ); + + expect(result).toMatchObject({ + ok: false, + conflicts: [ + { + sandboxName: "target", + reason: "incomplete-custom-route", + scope: "requested", + }, + ], + }); + if (result.ok) throw new Error("expected a route conflict"); + expect(formatGatewayRouteConflict(result)).toContain( + "requested custom route lacks durable endpoint or API-family metadata", + ); + }); + + it("skips registry rows without a complete provider and model (#6315)", () => { + expect( + check(route("anthropic-prod", "claude-new"), [ + sandbox("empty", { provider: null, model: null }), + sandbox("provider-only", { provider: "nvidia-prod", model: null }), + ]), + ).toEqual({ ok: true }); + }); + + it("fails closed when a registry row has an invalid gateway binding (#6315)", () => { + const result = check(route("nvidia-prod", "nvidia/model-a"), [ + sandbox("unknown-gateway", { gatewayName: "not-a-nemoclaw-gateway", gatewayPort: null }), + ]); + + expect(result).toMatchObject({ + ok: false, + conflicts: [{ sandboxName: "unknown-gateway", reason: "invalid-gateway-binding" }], + }); + if (result.ok) throw new Error("expected a route conflict"); + expect(formatGatewayRouteConflict(result)).toContain( + "restore its known-good gateway binding or remove and re-onboard that sandbox", + ); + }); +}); diff --git a/src/lib/inference/gateway-route-compatibility.ts b/src/lib/inference/gateway-route-compatibility.ts new file mode 100644 index 00000000000..2e134f3b594 --- /dev/null +++ b/src/lib/inference/gateway-route-compatibility.ts @@ -0,0 +1,231 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { canonicalEndpoint, type EndpointFlavor } from "../core/url-utils"; +import { resolveSandboxGatewayName } from "../onboard/gateway-binding"; +import type { SandboxEntry } from "../state/registry"; + +export type GatewayInferenceRoute = Pick< + SandboxEntry, + "provider" | "model" | "endpointUrl" | "preferredInferenceApi" | "credentialEnv" +>; + +export interface GatewayRouteCompatibilityRequest { + gatewayName: string; + sandboxName: string | null; + route: GatewayInferenceRoute; + sandboxes: readonly SandboxEntry[]; +} + +export type CurrentGatewayRouteCompatibilityRequest = Pick< + GatewayRouteCompatibilityRequest, + "sandboxName" | "route" +>; + +export type CurrentGatewayRouteCompatibilityCheck = ( + request: CurrentGatewayRouteCompatibilityRequest, +) => GatewayRouteCompatibilityResult; + +export type GatewayRouteConflictReason = + | "provider-model" + | "custom-endpoint" + | "custom-api" + | "incomplete-custom-route" + | "invalid-gateway-binding"; + +export interface GatewayRouteConflict { + sandboxName: string; + reason: GatewayRouteConflictReason; + scope?: "requested" | "registered"; +} + +export type GatewayRouteCompatibilityResult = + | { ok: true } + | { + ok: false; + gatewayName: string; + sandboxName: string | null; + route: { provider: string; model: string }; + conflicts: GatewayRouteConflict[]; + }; + +const CUSTOM_ROUTE_PROVIDERS = new Set(["compatible-endpoint", "compatible-anthropic-endpoint"]); + +const SUPPORTED_INFERENCE_APIS = new Set([ + "openai-completions", + "anthropic-messages", + "openai-responses", +]); + +function nonEmptyString(value: unknown): string | null { + if (typeof value !== "string") return null; + const trimmed = value.trim(); + return trimmed || null; +} + +function configuredRoute(route: GatewayInferenceRoute): { provider: string; model: string } | null { + const provider = nonEmptyString(route.provider); + const model = nonEmptyString(route.model); + return provider && model ? { provider, model } : null; +} + +function endpointFlavor(provider: string): EndpointFlavor { + return provider === "compatible-anthropic-endpoint" ? "anthropic" : "openai"; +} + +function normalizedInferenceApi(value: unknown): string | null { + const api = nonEmptyString(value); + return api && SUPPORTED_INFERENCE_APIS.has(api) ? api : null; +} + +function customRouteConflict( + provider: string, + requested: GatewayInferenceRoute, + recorded: GatewayInferenceRoute, +): GatewayRouteConflictReason | null { + const flavor = endpointFlavor(provider); + const requestedEndpoint = canonicalEndpoint(requested.endpointUrl, flavor); + const recordedEndpoint = canonicalEndpoint(recorded.endpointUrl, flavor); + const requestedApi = normalizedInferenceApi(requested.preferredInferenceApi); + const recordedApi = normalizedInferenceApi(recorded.preferredInferenceApi); + if (!requestedEndpoint || !recordedEndpoint || !requestedApi || !recordedApi) { + return "incomplete-custom-route"; + } + if (requestedEndpoint !== recordedEndpoint) return "custom-endpoint"; + if (requestedApi !== recordedApi) return "custom-api"; + return null; +} + +/** + * Compare a requested route with every configured registry row on the same + * OpenShell gateway. Registry rows are intentionally used without a live-state + * filter because stopped sandboxes still depend on the gateway route when they + * restart. + */ +export function checkGatewayRouteCompatibility( + request: GatewayRouteCompatibilityRequest, +): GatewayRouteCompatibilityResult { + const requested = configuredRoute(request.route); + if (!requested) { + throw new Error("Requested gateway inference route requires a provider and model"); + } + if ( + CUSTOM_ROUTE_PROVIDERS.has(requested.provider) && + (!canonicalEndpoint(request.route.endpointUrl, endpointFlavor(requested.provider)) || + !normalizedInferenceApi(request.route.preferredInferenceApi)) + ) { + return { + ok: false, + gatewayName: request.gatewayName, + sandboxName: request.sandboxName, + route: requested, + conflicts: [ + { + sandboxName: request.sandboxName ?? "requested route", + reason: "incomplete-custom-route", + scope: "requested", + }, + ], + }; + } + + const conflicts: GatewayRouteConflict[] = []; + for (const sandbox of request.sandboxes) { + if (sandbox.name === request.sandboxName) continue; + let recordedGatewayName: string; + try { + recordedGatewayName = resolveSandboxGatewayName(sandbox); + } catch { + conflicts.push({ sandboxName: sandbox.name, reason: "invalid-gateway-binding" }); + continue; + } + if (recordedGatewayName !== request.gatewayName) continue; + const recorded = configuredRoute(sandbox); + if (!recorded) continue; + + if (recorded.provider !== requested.provider || recorded.model !== requested.model) { + conflicts.push({ sandboxName: sandbox.name, reason: "provider-model" }); + continue; + } + if (CUSTOM_ROUTE_PROVIDERS.has(requested.provider)) { + const reason = customRouteConflict(requested.provider, request.route, sandbox); + if (reason) conflicts.push({ sandboxName: sandbox.name, reason }); + } + } + + return conflicts.length === 0 + ? { ok: true } + : { + ok: false, + gatewayName: request.gatewayName, + sandboxName: request.sandboxName, + route: requested, + conflicts, + }; +} + +function safeDisplay(value: string): string { + return value.replace(/[\u0000-\u001f\u007f]/g, "?"); +} + +export function formatGatewayRouteConflict( + result: Exclude, +): string { + const requestedRouteIncomplete = result.conflicts.some( + (conflict) => conflict.reason === "incomplete-custom-route" && conflict.scope === "requested", + ); + const names = [ + ...new Set( + result.conflicts + .filter((conflict) => conflict.scope !== "requested") + .map((conflict) => safeDisplay(conflict.sandboxName)), + ), + ] + .sort() + .map((name) => `'${name}'`) + .join(", "); + const target = result.sandboxName ? ` for sandbox '${safeDisplay(result.sandboxName)}'` : ""; + const hasIncompleteCustomRoute = result.conflicts.some( + (conflict) => conflict.reason === "incomplete-custom-route", + ); + const hasInvalidGatewayBinding = result.conflicts.some( + (conflict) => conflict.reason === "invalid-gateway-binding", + ); + const detail = [ + hasIncompleteCustomRoute + ? "At least one custom route lacks durable endpoint or API-family metadata, so compatibility cannot be proven; remove and re-onboard that sandbox with complete custom-route metadata." + : null, + hasInvalidGatewayBinding + ? "At least one registry row has an invalid gateway binding, so gateway separation cannot be proven; restore its known-good gateway binding or remove and re-onboard that sandbox." + : null, + ] + .filter(Boolean) + .join(" "); + return ( + `OpenShell gateway '${safeDisplay(result.gatewayName)}' has one inference route shared by every registered sandbox. ` + + `Cannot set ${safeDisplay(result.route.provider)} / ${safeDisplay(result.route.model)}${target}${ + requestedRouteIncomplete + ? " because the requested custom route lacks durable endpoint or API-family metadata." + : ` because it conflicts with ${names}.` + }${detail && !requestedRouteIncomplete ? ` ${detail}` : ""}\n` + + "Stopped sandboxes are included because they use the same gateway route when restarted. " + + (requestedRouteIncomplete + ? "Remove and re-onboard the sandbox with complete custom-route metadata." + : "Align the routes, remove the conflicting sandbox, or use another NEMOCLAW_GATEWAY_PORT.") + ); +} + +export class GatewayRouteConflictError extends Error { + readonly result: Exclude; + + constructor(result: Exclude) { + super(formatGatewayRouteConflict(result)); + this.name = "GatewayRouteConflictError"; + this.result = result; + } +} + +export function assertGatewayRouteCompatibility(request: GatewayRouteCompatibilityRequest): void { + const result = checkGatewayRouteCompatibility(request); + if (!result.ok) throw new GatewayRouteConflictError(result); +} diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 0faf51df834..b66a9579a6a 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -353,6 +353,8 @@ const { cleanupStaleHostFiles, }: typeof import("./host-artifact-cleanup") = require("./host-artifact-cleanup"); const registry: typeof import("./state/registry") = require("./state/registry"); +const gatewayRouteCompatibility: typeof import("./inference/gateway-route-compatibility") = + require("./inference/gateway-route-compatibility"); const sandboxMutationLock: typeof import("./state/mcp-lifecycle-lock") = require("./state/mcp-lifecycle-lock"); const { resolveSandboxImageTagFromCreateOutput } = @@ -637,6 +639,16 @@ const RESET = USE_COLOR ? "\x1b[0m" : ""; let OPENSHELL_BIN: string | null = null; let GATEWAY_PORT = DEFAULT_GATEWAY_PORT; let GATEWAY_NAME = gatewayBinding.resolveGatewayName(GATEWAY_PORT); + +function checkCurrentGatewayRouteCompatibility( + request: import("./inference/gateway-route-compatibility").CurrentGatewayRouteCompatibilityRequest, +): import("./inference/gateway-route-compatibility").GatewayRouteCompatibilityResult { + return gatewayRouteCompatibility.checkGatewayRouteCompatibility({ + ...request, + gatewayName: GATEWAY_NAME, + sandboxes: registry.listSandboxes().sandboxes, + }); +} const { clearDockerDriverGatewayRuntimeFiles, getDockerDriverGatewayEnv, @@ -4158,6 +4170,7 @@ async function setupNim(gpu: ReturnType, sandboxName: stri function getSetupInferenceDeps(): SetupInferenceDeps { return { + checkGatewayRouteCompatibility: checkCurrentGatewayRouteCompatibility, step, getGatewayName: () => GATEWAY_NAME, runOpenshell, @@ -4853,6 +4866,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { hermesApiKeyCredentialEnv: HERMES_NOUS_API_KEY_CREDENTIAL_ENV, }, providerDeps: { + checkGatewayRouteCompatibility: checkCurrentGatewayRouteCompatibility, normalizeHermesAuthMethod, setupNim: (gpu, sandboxName, agent, recoverProvider) => setupNim(gpu, sandboxName, agent, recoverProvider, opts.rebuildRegistryInferenceRoute), diff --git a/src/lib/onboard/machine/core-flow-phases.test.ts b/src/lib/onboard/machine/core-flow-phases.test.ts index 691c68e4664..7f27a684428 100644 --- a/src/lib/onboard/machine/core-flow-phases.test.ts +++ b/src/lib/onboard/machine/core-flow-phases.test.ts @@ -83,6 +83,7 @@ function createPhases( hermesApiKeyCredentialEnv: "HERMES_API_KEY", }, providerDeps: { + checkGatewayRouteCompatibility: () => ({ ok: true }), normalizeHermesAuthMethod: (value) => value === "oauth" || value === "api_key" ? value : null, setupNim: vi.fn(async () => ({ diff --git a/src/lib/onboard/machine/handlers/provider-inference.test.ts b/src/lib/onboard/machine/handlers/provider-inference.test.ts index ce09dc30c88..e98a21cafa6 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.test.ts @@ -7,6 +7,7 @@ import path from "node:path"; import { describe, expect, it, vi } from "vitest"; +import type { CurrentGatewayRouteCompatibilityCheck } from "../../../inference/gateway-route-compatibility"; import { createSession, type Session, type SessionUpdates } from "../../../state/onboard-session"; import { patchStagedDockerfile } from "../../dockerfile-patch"; import { clearCompatibleEndpointReasoning } from "../../reasoning-mode"; @@ -36,6 +37,9 @@ function createDeps( overrides: Partial["deps"]> = {}, ) { const calls = { + checkGatewayRouteCompatibility: vi.fn(() => ({ + ok: true, + })), setupNim: vi.fn(async () => ({ ...baseSelection })), setupInference: vi.fn(async () => ({ ok: true as const })), startStep: vi.fn(async () => undefined), @@ -72,6 +76,7 @@ function createDeps( return { calls, deps: { + checkGatewayRouteCompatibility: calls.checkGatewayRouteCompatibility, normalizeHermesAuthMethod: (value: string | null | undefined) => value === "oauth" || value === "api_key" ? value : null, setupNim: calls.setupNim, @@ -825,6 +830,38 @@ describe("handleProviderInferenceState", () => { expect(calls.reconcileRouter).toHaveBeenCalledOnce(); }); + it("blocks a conflicting resumed routed provider before gateway or registry mutation (#6315)", async () => { + const session = createSession({ + provider: "nvidia-router", + model: "router/model", + endpointUrl: "http://host.openshell.internal:4000/v1", + preferredInferenceApi: "openai-completions", + }); + session.steps.provider_selection.status = "complete"; + const { deps, calls } = createDeps({ isInferenceRouteReady: vi.fn(() => true) }); + calls.checkGatewayRouteCompatibility.mockReturnValue({ + ok: false, + gatewayName: "nemoclaw", + sandboxName: "router-sandbox", + route: { provider: "nvidia-router", model: "router/model" }, + conflicts: [{ sandboxName: "existing-sandbox", reason: "provider-model" }], + }); + + await expect( + handleProviderInferenceState({ + ...baseOptions(deps, session), + resume: true, + sandboxName: "router-sandbox", + }), + ).rejects.toThrow("exit 1"); + + expect(calls.reconcileRouter).not.toHaveBeenCalled(); + expect(calls.reupsertRoutedProvider).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + expect(calls.setupInference).not.toHaveBeenCalled(); + expect(calls.error).toHaveBeenCalledWith(expect.stringContaining("existing-sandbox")); + }); + // #5974 instance 5: the Model Router Python preflight (`prepareModelRouterVenv`) // throws a plain Error (e.g. "above supported ceiling", with no `oclif.exit`) // out of `reconcileModelRouter`. The routed branch must catch that throw and diff --git a/src/lib/onboard/machine/handlers/provider-inference.ts b/src/lib/onboard/machine/handlers/provider-inference.ts index 4cfcd0ddf3c..5c0886fea1a 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.ts @@ -2,6 +2,10 @@ // SPDX-License-Identifier: Apache-2.0 import { coerceAgentInferenceApi } from "../../../inference/config"; +import { + type CurrentGatewayRouteCompatibilityCheck, + formatGatewayRouteConflict, +} from "../../../inference/gateway-route-compatibility"; import type { WebSearchConfig } from "../../../inference/web-search"; import type { HermesAuthMethod, Session, SessionUpdates } from "../../../state/onboard-session"; import { withInferenceTrace, withProviderSelectionTrace } from "../../tracing"; @@ -67,6 +71,7 @@ export interface ProviderInferenceStateOptions { hermesApiKeyCredentialEnv: string; }; deps: { + checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; normalizeHermesAuthMethod(value: string | null | undefined): HermesAuthMethod | null; setupNim( gpu: Gpu, @@ -420,6 +425,14 @@ export async function handleProviderInferenceState({ effectiveResume && deps.isInferenceRouteReady(provider, model); if (resumeInference) { + const compatibility = deps.checkGatewayRouteCompatibility({ + sandboxName, + route: { provider, model, endpointUrl, preferredInferenceApi }, + }); + if (!compatibility.ok) { + deps.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); + deps.exitProcess(1); + } if (provider === constants.hermesProviderName) { let inferenceResult: ProviderInferenceRetry; try { diff --git a/src/lib/onboard/setup-inference-route-containment.test.ts b/src/lib/onboard/setup-inference-route-containment.test.ts new file mode 100644 index 00000000000..7914e88a4a6 --- /dev/null +++ b/src/lib/onboard/setup-inference-route-containment.test.ts @@ -0,0 +1,65 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it, vi } from "vitest"; +import { createSetupInference, type SetupInferenceDeps } from "./setup-inference"; + +describe("onboard shared gateway route containment", () => { + it("rejects a conflict before selecting the gateway or mutating provider state (#6315)", async () => { + const events: string[] = []; + const runOpenshell = vi.fn(() => { + events.push("openshell"); + return { status: 0 }; + }); + const updateSandbox = vi.fn(() => true); + const upsertProvider = vi.fn(() => ({ ok: true })); + const verifyInferenceRoute = vi.fn(); + const verifyOnboardInferenceSmoke = vi.fn(); + const error = vi.fn((message: string) => events.push(`error:${message}`)); + const exitProcess = vi.fn((code: number): never => { + events.push(`exit:${code}`); + throw new Error(`exit ${code}`); + }); + const checkGatewayRouteCompatibility = vi.fn(() => { + events.push("guard"); + return { + ok: false as const, + gatewayName: "nemoclaw", + sandboxName: "new-sandbox", + route: { provider: "anthropic-prod", model: "claude-new" }, + conflicts: [{ sandboxName: "stopped-sandbox", reason: "provider-model" as const }], + }; + }); + const setupInference = createSetupInference({ + checkGatewayRouteCompatibility, + step: () => events.push("step"), + getGatewayName: () => "nemoclaw", + runOpenshell, + updateSandbox, + upsertProvider, + verifyInferenceRoute, + verifyOnboardInferenceSmoke, + error, + exitProcess, + } as unknown as SetupInferenceDeps); + + await expect( + setupInference( + "new-sandbox", + "claude-new", + "anthropic-prod", + "https://api.anthropic.com", + "ANTHROPIC_API_KEY", + ), + ).rejects.toThrow("exit 1"); + + expect(events[0]).toBe("guard"); + expect(runOpenshell).not.toHaveBeenCalled(); + expect(upsertProvider).not.toHaveBeenCalled(); + expect(verifyInferenceRoute).not.toHaveBeenCalled(); + expect(verifyOnboardInferenceSmoke).not.toHaveBeenCalled(); + expect(updateSandbox).not.toHaveBeenCalled(); + expect(error).toHaveBeenCalledWith(expect.stringContaining("stopped-sandbox")); + expect(exitProcess).toHaveBeenCalledWith(1); + }); +}); diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index 0b00e46aa40..077762d07b7 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -1,6 +1,10 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +import { + type CurrentGatewayRouteCompatibilityCheck, + formatGatewayRouteConflict, +} from "../inference/gateway-route-compatibility"; import type { HermesAuthMethod } from "./hermes-auth"; import type { CommonDeps, @@ -65,6 +69,7 @@ type ProviderBranchDeps = Pick< Pick; export type SetupInferenceDeps = ProviderBranchDeps & { + checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; step: (current: number, total: number, label: string) => void; getGatewayName: () => string; runOpenshell: import("./openshell-cli").OpenshellCliHelpers["runOpenshell"]; @@ -129,6 +134,19 @@ export function createSetupInference( hermesToolGateways: string[] = [], options: ProviderInferenceSetupOptions = {}, ): Promise { + const compatibility = deps.checkGatewayRouteCompatibility({ + sandboxName, + route: { + provider, + model, + endpointUrl, + preferredInferenceApi: options.preferredInferenceApi ?? null, + }, + }); + if (!compatibility.ok) { + deps.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); + return deps.exitProcess(1); + } deps.step(4, 8, "Setting up inference provider"); deps.runOpenshell(["gateway", "select", deps.getGatewayName()], { ignoreError: true }); diff --git a/test/cli/list-inference.test.ts b/test/cli/list-inference.test.ts index 79bfaff3865..f9271ee89ec 100644 --- a/test/cli/list-inference.test.ts +++ b/test/cli/list-inference.test.ts @@ -20,7 +20,7 @@ import { describe("CLI dispatch", () => { it( - "redirects `inference set` to openshell when provider or model is missing", + "keeps `inference set` inside NemoClaw when provider or model is missing", () => { for (const argv of [ "inference set 2>&1", @@ -29,11 +29,13 @@ describe("CLI dispatch", () => { ]) { const r = run(argv); expect(r.code, `nemoclaw ${argv}`).toBe(1); - expect(r.out, `nemoclaw ${argv}`).toContain("Unknown nemoclaw command: inference set"); - expect(r.out, `nemoclaw ${argv}`).toContain("This operation belongs to OpenShell."); expect(r.out, `nemoclaw ${argv}`).toContain( - "Run: openshell inference set -g nemoclaw --model --provider ", + "nemoclaw inference set requires --provider and --model", ); + expect(r.out, `nemoclaw ${argv}`).toContain( + "Run: nemoclaw inference set --provider --model [--sandbox ]", + ); + expect(r.out, `nemoclaw ${argv}`).not.toContain("openshell inference set"); expect(r.out, `nemoclaw ${argv}`).not.toContain("Missing required flag"); expect(r.out, `nemoclaw ${argv}`).not.toContain("FailedFlagValidationError"); expect(r.out, `nemoclaw ${argv}`).not.toContain("node_modules/@oclif/core"); @@ -65,11 +67,11 @@ describe("CLI dispatch", () => { hermesCode = result.code; } expect(hermesCode).toBe(1); - expect(hermesOut).toContain("Unknown nemohermes command: inference set"); - expect(hermesOut).toContain("This operation belongs to OpenShell."); + expect(hermesOut).toContain("nemohermes inference set requires --provider and --model"); expect(hermesOut).toContain( - "Run: openshell inference set -g nemoclaw --model --provider ", + "Run: nemohermes inference set --provider --model [--sandbox ]", ); + expect(hermesOut).not.toContain("openshell inference set"); }, testTimeout(15_000), ); diff --git a/test/onboard-inference-smoke.test.ts b/test/onboard-inference-smoke.test.ts index 25af5044bf2..41d2a04eb03 100644 --- a/test/onboard-inference-smoke.test.ts +++ b/test/onboard-inference-smoke.test.ts @@ -114,6 +114,9 @@ const { setupInference } = require(${onboardPath}); "compatible-endpoint", "https://broken.example.invalid/v1", "BROKEN_API_KEY", + null, + [], + { preferredInferenceApi: "openai-completions" }, ); console.log(JSON.stringify({ outcome: "resolved", calls })); })().catch((error) => { diff --git a/test/onboard-selection.test.ts b/test/onboard-selection.test.ts index 7a9bdca0c7e..f3b202f0d2e 100644 --- a/test/onboard-selection.test.ts +++ b/test/onboard-selection.test.ts @@ -6113,7 +6113,16 @@ runner.runCapture = (cmd) => { process.env.COMPATIBLE_API_KEY = "test-key"; const { setupInference } = require(${onboardPath}); (async () => { - await setupInference(null, "qwen3.6:35b", "compatible-endpoint", "http://lan-server:11434/v1", "COMPATIBLE_API_KEY"); + await setupInference( + null, + "qwen3.6:35b", + "compatible-endpoint", + "http://lan-server:11434/v1", + "COMPATIBLE_API_KEY", + null, + [], + { preferredInferenceApi: "openai-completions" }, + ); process.exit(0); })().catch((err) => { console.error(err); process.exit(1); }); `; diff --git a/test/support/connect-flow-test-harness.ts b/test/support/connect-flow-test-harness.ts index 21fe98c98db..4e169b3185d 100644 --- a/test/support/connect-flow-test-harness.ts +++ b/test/support/connect-flow-test-harness.ts @@ -39,6 +39,7 @@ export type ConnectHarnessOptions = { inferenceGetOutput?: string; inferenceProbeResponses?: string[]; registryEntry?: Partial; + registryEntries?: Array & Pick>; sessionAgent?: unknown; listOutput?: string; processCheck?: { @@ -152,7 +153,7 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne const ensureOllamaAuthProxySpy = vi .spyOn(ollamaProxy, "ensureOllamaAuthProxy") .mockImplementation(() => undefined); - vi.spyOn(registry, "getSandbox").mockReturnValue({ + const primaryRegistryEntry: SandboxEntry = { name: "alpha", agent: options.agentName ?? "openclaw", provider: null, @@ -160,6 +161,27 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne gpuEnabled: false, policies: [], ...options.registryEntry, + }; + const registryEntries: SandboxEntry[] = options.registryEntries + ? options.registryEntries.map((candidate) => + candidate.name === primaryRegistryEntry.name + ? { ...primaryRegistryEntry, ...candidate } + : { + agent: "openclaw", + provider: null, + model: null, + gpuEnabled: false, + policies: [], + ...candidate, + }, + ) + : [primaryRegistryEntry]; + vi.spyOn(registry, "getSandbox").mockImplementation( + (name: unknown) => registryEntries.find((candidate) => candidate.name === String(name)) ?? null, + ); + vi.spyOn(registry, "listSandboxes").mockReturnValue({ + sandboxes: registryEntries, + defaultSandbox: primaryRegistryEntry.name, }); vi.spyOn(agentRuntime, "getSessionAgent").mockReturnValue( (options.sessionAgent ?? { name: "openclaw" }) as never, diff --git a/test/support/setup-inference-test-harness.ts b/test/support/setup-inference-test-harness.ts index 77bd0ccdf26..2b4030a9a06 100644 --- a/test/support/setup-inference-test-harness.ts +++ b/test/support/setup-inference-test-harness.ts @@ -269,6 +269,7 @@ export function createDirectSetupInferenceHarnessFactory( return directRunResult(options.runOpenshell?.(args, runOptions, commands)); }; const setupInference = createSetupInference({ + checkGatewayRouteCompatibility: () => ({ ok: true }), step: () => {}, getGatewayName: () => "nemoclaw", runOpenshell, From 2f578e986ad5162776e9fcc473adf55470b9b963 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 11:31:46 -0700 Subject: [PATCH 02/31] test(onboard): keep route fixture within budget Signed-off-by: Aaron Erickson --- test/onboard-selection.test.ts | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/test/onboard-selection.test.ts b/test/onboard-selection.test.ts index f3b202f0d2e..8009e94536e 100644 --- a/test/onboard-selection.test.ts +++ b/test/onboard-selection.test.ts @@ -6113,16 +6113,7 @@ runner.runCapture = (cmd) => { process.env.COMPATIBLE_API_KEY = "test-key"; const { setupInference } = require(${onboardPath}); (async () => { - await setupInference( - null, - "qwen3.6:35b", - "compatible-endpoint", - "http://lan-server:11434/v1", - "COMPATIBLE_API_KEY", - null, - [], - { preferredInferenceApi: "openai-completions" }, - ); + await setupInference(null, "qwen3.6:35b", "compatible-endpoint", "http://lan-server:11434/v1", "COMPATIBLE_API_KEY", null, [], { preferredInferenceApi: "openai-completions" }); process.exit(0); })().catch((err) => { console.error(err); process.exit(1); }); `; From fa6991dc9c7de7774ab6e7f4cb4f80889d629cfc Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 11:33:00 -0700 Subject: [PATCH 03/31] fix(connect): guard routes before endpoint probes Signed-off-by: Aaron Erickson --- .../sandbox/connect-route-lifecycle.test.ts | 38 +++++++++++++++++++ .../sandbox/connect-route-repair.test.ts | 3 +- src/lib/actions/sandbox/connect.ts | 3 +- 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/src/lib/actions/sandbox/connect-route-lifecycle.test.ts b/src/lib/actions/sandbox/connect-route-lifecycle.test.ts index 0dec005661c..a7ed7ff1bac 100644 --- a/src/lib/actions/sandbox/connect-route-lifecycle.test.ts +++ b/src/lib/actions/sandbox/connect-route-lifecycle.test.ts @@ -158,6 +158,44 @@ describe("connectSandbox route lifecycle", () => { expect(exitSpy).toHaveBeenCalledWith(1); }); + it("exits before an endpoint probe when an aligned route conflicts with a stopped sandbox (#6315)", async () => { + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + } as const; + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + registryEntry: alpha, + registryEntries: [ + alpha, + { + name: "stopped-peer", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "anthropic-prod", + model: "claude-sonnet-4-20250514", + }, + ], + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + const routeProbeCalls = harness.captureOpenshellSpy.mock.calls.filter((call) => + JSON.stringify(call[0]).includes("inference.local/v1/models"), + ); + expect(routeProbeCalls).toHaveLength(0); + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + it("wires the forced VM DNS monkeypatch into connect route repair", async () => { vi.stubEnv("NEMOCLAW_FORCE_VM_DNS_MONKEYPATCH", "1"); try { diff --git a/src/lib/actions/sandbox/connect-route-repair.test.ts b/src/lib/actions/sandbox/connect-route-repair.test.ts index 032de156f41..3b6ae0b1be0 100644 --- a/src/lib/actions/sandbox/connect-route-repair.test.ts +++ b/src/lib/actions/sandbox/connect-route-repair.test.ts @@ -144,7 +144,7 @@ describe("sandbox connect route repair unit flow", () => { } }); - it("stops after a broken probe but before any repair mutation when routes conflict (#6315)", () => { + it("stops before the initial endpoint probe or repair mutation when routes conflict (#6315)", () => { const conflict = new Error("shared gateway route conflict"); const assertRouteCompatible = vi.fn(() => { throw conflict; @@ -159,6 +159,7 @@ describe("sandbox connect route repair unit flow", () => { "vm-box", expect.objectContaining({ name: "demo" }), ); + expect(deps.probe).not.toHaveBeenCalled(); expect(calls.monkeypatches).toEqual([]); expect(calls.reapplications).toEqual([]); expect(calls.legacyRepairs).toEqual([]); diff --git a/src/lib/actions/sandbox/connect.ts b/src/lib/actions/sandbox/connect.ts index 3e1d6e812d7..3ba4f748bf2 100644 --- a/src/lib/actions/sandbox/connect.ts +++ b/src/lib/actions/sandbox/connect.ts @@ -445,6 +445,7 @@ export function repairSandboxInferenceRouteWithDeps( if (deps.isRepairDisabled?.()) { return { healthy: true, repairAttempted: false, detail: "route repair disabled" }; } + deps.assertRouteCompatible?.(sandboxName, sb); const initialProbe = deps.probe(sandboxName); if (initialProbe.healthy) { return { healthy: true, repairAttempted: false, detail: initialProbe.detail }; @@ -452,8 +453,6 @@ export function repairSandboxInferenceRouteWithDeps( if (!initialProbe.broken) { return { healthy: true, repairAttempted: false, detail: initialProbe.detail }; } - deps.assertRouteCompatible?.(sandboxName, sb); - if (!shouldUseLegacyDnsProxyRepair(sb)) { if (deps.shouldApplyVmDnsMonkeypatch(sb)) { if (!quiet) { From ecd6b6394c31b525361d3f3c32911f8fc788e8b9 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 11:41:24 -0700 Subject: [PATCH 04/31] refactor(onboard): keep route guard wiring focused Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 22 ++++------------------ src/lib/onboard/inference-route.ts | 19 +++++++++++++++++-- 2 files changed, 21 insertions(+), 20 deletions(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index b66a9579a6a..730f5f42c2a 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -353,8 +353,6 @@ const { cleanupStaleHostFiles, }: typeof import("./host-artifact-cleanup") = require("./host-artifact-cleanup"); const registry: typeof import("./state/registry") = require("./state/registry"); -const gatewayRouteCompatibility: typeof import("./inference/gateway-route-compatibility") = - require("./inference/gateway-route-compatibility"); const sandboxMutationLock: typeof import("./state/mcp-lifecycle-lock") = require("./state/mcp-lifecycle-lock"); const { resolveSandboxImageTagFromCreateOutput } = @@ -639,16 +637,6 @@ const RESET = USE_COLOR ? "\x1b[0m" : ""; let OPENSHELL_BIN: string | null = null; let GATEWAY_PORT = DEFAULT_GATEWAY_PORT; let GATEWAY_NAME = gatewayBinding.resolveGatewayName(GATEWAY_PORT); - -function checkCurrentGatewayRouteCompatibility( - request: import("./inference/gateway-route-compatibility").CurrentGatewayRouteCompatibilityRequest, -): import("./inference/gateway-route-compatibility").GatewayRouteCompatibilityResult { - return gatewayRouteCompatibility.checkGatewayRouteCompatibility({ - ...request, - gatewayName: GATEWAY_NAME, - sandboxes: registry.listSandboxes().sandboxes, - }); -} const { clearDockerDriverGatewayRuntimeFiles, getDockerDriverGatewayEnv, @@ -1020,9 +1008,8 @@ function upsertMessagingProviders( const providerExistsInGateway = (name: string) => onboardProviders.providerExistsInGateway(name, runOpenshell); -const { verifyInferenceRoute, isInferenceRouteReady } = - inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); - +const { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility } = + inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell, () => GATEWAY_NAME); const { inspectSandboxForCreate, pruneStaleSandboxEntry, @@ -4165,12 +4152,11 @@ async function setupNim(gpu: ReturnType, sandboxName: stri reuseGatewayCredentialWithoutLocalKey: reuseGatewayCredential, }; } - // ── Step 4: Inference provider ─────────────────────────────────── function getSetupInferenceDeps(): SetupInferenceDeps { return { - checkGatewayRouteCompatibility: checkCurrentGatewayRouteCompatibility, + checkGatewayRouteCompatibility, step, getGatewayName: () => GATEWAY_NAME, runOpenshell, @@ -4866,7 +4852,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { hermesApiKeyCredentialEnv: HERMES_NOUS_API_KEY_CREDENTIAL_ENV, }, providerDeps: { - checkGatewayRouteCompatibility: checkCurrentGatewayRouteCompatibility, + checkGatewayRouteCompatibility, normalizeHermesAuthMethod, setupNim: (gpu, sandboxName, agent, recoverProvider) => setupNim(gpu, sandboxName, agent, recoverProvider, opts.rebuildRegistryInferenceRoute), diff --git a/src/lib/onboard/inference-route.ts b/src/lib/onboard/inference-route.ts index 397e50a255a..e7eb2325ff6 100644 --- a/src/lib/onboard/inference-route.ts +++ b/src/lib/onboard/inference-route.ts @@ -2,10 +2,18 @@ // SPDX-License-Identifier: Apache-2.0 import { parseGatewayInference } from "../inference/config"; +import { + type CurrentGatewayRouteCompatibilityCheck, + checkGatewayRouteCompatibility as checkGatewayRouteCompatibilityForRegistry, +} from "../inference/gateway-route-compatibility"; +import { listSandboxes } from "../state/registry"; type RunCaptureOpenshell = (args: string[], options?: { ignoreError?: boolean }) => string | null; -export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpenshell) { +export function createInferenceRouteHelpers( + runCaptureOpenshell: RunCaptureOpenshell, + getGatewayName: () => string = () => "nemoclaw", +) { function verifyInferenceRoute(_provider: string, _model: string): void { const output = runCaptureOpenshell(["inference", "get"], { ignoreError: true }); if (!output || /Gateway inference:\s*[\r\n]+\s*Not configured/i.test(output)) { @@ -21,5 +29,12 @@ export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpens return Boolean(live && live.provider === provider && live.model === model); } - return { verifyInferenceRoute, isInferenceRouteReady }; + const checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck = (request) => + checkGatewayRouteCompatibilityForRegistry({ + ...request, + gatewayName: getGatewayName(), + sandboxes: listSandboxes().sandboxes, + }); + + return { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility }; } From e9b96585535f6453bf08ca45c9e5049e4b9f2bb0 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 11:47:04 -0700 Subject: [PATCH 05/31] test(inference): keep route matrix linear Signed-off-by: Aaron Erickson --- .../inference/gateway-route-compatibility.test.ts | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/src/lib/inference/gateway-route-compatibility.test.ts b/src/lib/inference/gateway-route-compatibility.test.ts index 4fdc81c9988..b7cf611f5c5 100644 --- a/src/lib/inference/gateway-route-compatibility.test.ts +++ b/src/lib/inference/gateway-route-compatibility.test.ts @@ -57,8 +57,9 @@ describe("shared gateway inference route compatibility", () => { ok: false, conflicts: [{ sandboxName: "stopped-peer", reason: "provider-model" }], }); - if (result.ok) throw new Error("expected a route conflict"); - expect(formatGatewayRouteConflict(result)).toContain("Stopped sandboxes are included"); + expect(formatGatewayRouteConflict(result as Exclude)).toContain( + "Stopped sandboxes are included", + ); }); it("allows different routes on different gateways (#6315)", () => { @@ -183,8 +184,7 @@ describe("shared gateway inference route compatibility", () => { ok: false, conflicts: [{ sandboxName: "legacy-custom", reason: "incomplete-custom-route" }], }); - if (result.ok) throw new Error("expected a route conflict"); - expect(formatGatewayRouteConflict(result)).toContain( + expect(formatGatewayRouteConflict(result as Exclude)).toContain( "remove and re-onboard that sandbox with complete custom-route metadata", ); }); @@ -208,8 +208,7 @@ describe("shared gateway inference route compatibility", () => { }, ], }); - if (result.ok) throw new Error("expected a route conflict"); - expect(formatGatewayRouteConflict(result)).toContain( + expect(formatGatewayRouteConflict(result as Exclude)).toContain( "requested custom route lacks durable endpoint or API-family metadata", ); }); @@ -232,8 +231,7 @@ describe("shared gateway inference route compatibility", () => { ok: false, conflicts: [{ sandboxName: "unknown-gateway", reason: "invalid-gateway-binding" }], }); - if (result.ok) throw new Error("expected a route conflict"); - expect(formatGatewayRouteConflict(result)).toContain( + expect(formatGatewayRouteConflict(result as Exclude)).toContain( "restore its known-good gateway binding or remove and re-onboard that sandbox", ); }); From bdc0c6eb0905cf71adf60955f68a1c385a893fb4 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 11:58:22 -0700 Subject: [PATCH 06/31] test(onboard): preserve recovered route API metadata Signed-off-by: Aaron Erickson --- test/onboard-remote-recreate-credential-reuse.test.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/test/onboard-remote-recreate-credential-reuse.test.ts b/test/onboard-remote-recreate-credential-reuse.test.ts index 09d10ae7c1b..dc1db52ae11 100644 --- a/test/onboard-remote-recreate-credential-reuse.test.ts +++ b/test/onboard-remote-recreate-credential-reuse.test.ts @@ -120,6 +120,7 @@ const { setupNim, setupInference } = require(${onboardPath}); selected.hermesAuthMethod, selected.hermesToolGateways, { + preferredInferenceApi: selected.preferredInferenceApi, skipHostInferenceSmoke: selected.skipHostInferenceSmoke, reuseGatewayCredentialWithoutLocalKey: process.env.NEMOCLAW_TEST_OMIT_REUSE_AUTHORIZATION === "1" From de3b246aa227633ca6b3d8fb31de8f6625bbbf3e Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 12:20:13 -0700 Subject: [PATCH 07/31] fix(inference): close shared gateway route guard gaps Signed-off-by: Aaron Erickson --- docs/about/release-notes.mdx | 5 + .../inference-set-route-containment.ts | 342 ++++++++++++++++++ src/lib/actions/inference-set.ts | 291 +-------------- .../sandbox/connect-inference-gateway.ts | 46 +++ .../sandbox/connect-route-containment.test.ts | 251 +++++++++++++ .../sandbox/connect-route-lifecycle.test.ts | 125 +------ .../sandbox/connect-route-repair.test.ts | 21 -- src/lib/actions/sandbox/connect.ts | 58 ++- .../inference/gateway-route-compatibility.ts | 2 +- src/lib/onboard.ts | 3 +- src/lib/onboard/inference-route.ts | 6 +- .../onboard/machine/core-flow-phases.test.ts | 1 + src/lib/onboard/machine/core-flow-phases.ts | 2 + ...ovider-inference-route-containment.test.ts | 224 ++++++++++++ .../provider-inference-route-containment.ts | 27 ++ .../handlers/provider-inference.test.ts | 33 +- .../machine/handlers/provider-inference.ts | 30 +- src/lib/onboard/setup-inference.ts | 1 + 18 files changed, 971 insertions(+), 497 deletions(-) create mode 100644 src/lib/actions/inference-set-route-containment.ts create mode 100644 src/lib/actions/sandbox/connect-inference-gateway.ts create mode 100644 src/lib/actions/sandbox/connect-route-containment.test.ts create mode 100644 src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts create mode 100644 src/lib/onboard/machine/handlers/provider-inference-route-containment.ts diff --git a/docs/about/release-notes.mdx b/docs/about/release-notes.mdx index d45d6e9cabc..4efdea3c0bf 100644 --- a/docs/about/release-notes.mdx +++ b/docs/about/release-notes.mdx @@ -30,6 +30,11 @@ NemoClaw v0.0.74 upgrades the OpenShell policy boundary, adds managed MCP and pr The selected mode persists through resume and transactional rebuilds, and model-specific compatibility safeguards can keep an incompatible model on direct disclosure. Sandbox-first `inference get` and `inference set` commands now provide the same route controls as their global forms. For more information, refer to [Tool Calling Reliability](../inference/tool-calling-reliability), [Model Capability Audit](../inference/model-capability-audit), and [NemoClaw CLI Commands Reference](../reference/commands). +- Shared OpenShell gateways now enforce a single compatible inference route across every registered sandbox, including stopped sandboxes. + Onboarding, connect-time repair, and `inference set` reject provider/model conflicts; custom routes must also match the normalized endpoint and API family. + As a migration requirement, switching to a custom compatible provider now requires both `--endpoint-url` and `--inference-api`, and incomplete legacy custom-route metadata fails closed until the sandbox is removed and re-onboarded with complete metadata. + Use a different `NEMOCLAW_GATEWAY_PORT` when sandboxes need independent routes. + For more information, refer to [Switch Inference Providers](../inference/switch-inference-providers), [NemoClaw CLI Commands Reference](../reference/commands), and [Troubleshooting](../reference/troubleshooting). - LangChain Deep Agents Code now provides managed `status`, `whoami`, and `identity` commands without launching the interactive UI, validates the installed agent version during onboarding, and keeps credential-shaped or tracing configuration out of persisted runtime metadata. Its rebuild path validates recreation before destructive handoff and preserves the managed proxy, tool-disclosure, and MCP boundaries. For more information, refer to [Quickstart with LangChain Deep Agents Code](../../openclaw/get-started/quickstart-langchain-deepagents-code), [NemoClaw CLI Commands Reference](../reference/commands), and [Security Best Practices](../security/best-practices). diff --git a/src/lib/actions/inference-set-route-containment.ts b/src/lib/actions/inference-set-route-containment.ts new file mode 100644 index 00000000000..4989bebac19 --- /dev/null +++ b/src/lib/actions/inference-set-route-containment.ts @@ -0,0 +1,342 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { + checkGatewayRouteCompatibility, + formatGatewayRouteConflict, +} from "../inference/gateway-route-compatibility"; +import { resolveSandboxGatewayName } from "../onboard/gateway-binding"; +import type { ConfigValue } from "../security/credential-filter"; +import type { Session } from "../state/onboard-session"; +import type { SandboxEntry } from "../state/registry"; +import { InferenceSetError } from "./inference-set-error"; + +export type RegistryInferenceMetadata = Pick< + SandboxEntry, + "endpointUrl" | "credentialEnv" | "preferredInferenceApi" | "nimContainer" +>; + +export interface ExplicitCustomRouteOptions { + endpointUrl?: string | null; + credentialEnv?: string | null; + inferenceApi?: string | null; +} + +type RewriteConfigUrlsWithDnsPinning = (value: ConfigValue) => Promise; + +export interface PreparedInferenceSetRoute { + gatewayName: string; + preliminaryExplicitMetadata: RegistryInferenceMetadata | null; + preliminaryRegistryMetadata: RegistryInferenceMetadata; +} + +const CUSTOM_COMPATIBLE_CREDENTIAL_ENV: Record = { + "compatible-endpoint": "COMPATIBLE_API_KEY", + "compatible-anthropic-endpoint": "COMPATIBLE_ANTHROPIC_API_KEY", +}; + +const INFERENCE_SET_APIS = new Set([ + "openai-completions", + "anthropic-messages", + "openai-responses", +]); + +function isCustomCompatibleProvider(provider: string): boolean { + return provider === "compatible-endpoint" || provider === "compatible-anthropic-endpoint"; +} + +function hasExplicitCustomMetadata(options: ExplicitCustomRouteOptions): boolean { + return Boolean(options.endpointUrl || options.credentialEnv || options.inferenceApi); +} + +// TRUST BOUNDARY: host.openshell.internal is the single sandbox-to-host bridge +// hostname provisioned by OpenShell. It resolves to the Docker host gateway +// only inside the sandbox network namespace. This exemption is intentionally +// limited below to HTTP, an explicit unprivileged port, and the exact hostname; +// do not extend it to HTTPS, wildcard subdomains, localhost, RFC1918 literals, +// or other internal DNS names. +const ALLOWED_PRIVATE_CUSTOM_ENDPOINT_HOSTS = new Set(["host.openshell.internal"]); + +function normalizeEndpointUrlShape(value: string): { url: URL; normalized: string } { + const url = new URL(value); + if ((url.protocol !== "http:" && url.protocol !== "https:") || url.username || url.password) { + throw new Error("unsupported URL shape"); + } + url.search = ""; + url.hash = ""; + const pathname = url.pathname.replace(/\/+$/, ""); + url.pathname = pathname || "/"; + return { + url, + normalized: url.pathname === "/" ? url.origin : `${url.origin}${url.pathname}`, + }; +} + +function normalizeCustomEndpointUrlWithoutDns(value: string | null | undefined): string { + const raw = typeof value === "string" ? value.trim() : ""; + if (!raw) + throw new InferenceSetError("endpoint-url is required for custom-compatible metadata.", 2); + try { + return normalizeEndpointUrlShape(raw).normalized; + } catch { + throw new InferenceSetError( + "endpoint-url must be a valid http(s) URL without embedded credentials.", + 2, + ); + } +} + +export async function normalizeCustomEndpointUrl( + value: string | null | undefined, + rewriteUrlWithDnsPinning: RewriteConfigUrlsWithDnsPinning, +): Promise { + const normalized = normalizeCustomEndpointUrlWithoutDns(value); + const shaped = normalizeEndpointUrlShape(normalized); + const hostname = shaped.url.hostname.replace(/\.$/, "").toLowerCase(); + const port = Number(shaped.url.port); + if ( + ALLOWED_PRIVATE_CUSTOM_ENDPOINT_HOSTS.has(hostname) && + shaped.url.protocol === "http:" && + Number.isInteger(port) && + port >= 1024 + ) { + // This is the single sandbox-to-host bridge name that NemoClaw itself + // provisions for local inference. Its supported routes are explicit + // unprivileged HTTP listeners; do not generalize this exemption to HTTPS, + // default/privileged ports, localhost, RFC1918 addresses, or arbitrary + // internal DNS names. + return normalized; + } + + try { + const validated = await rewriteUrlWithDnsPinning(normalized); + if (typeof validated !== "string") throw new Error("URL validator returned a non-string value"); + return normalizeEndpointUrlShape(validated).normalized; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + throw new InferenceSetError(`endpoint-url is not allowed: ${message}`, 2); + } +} + +function normalizeExplicitCredentialEnv( + provider: string, + value: string | null | undefined, +): string { + const expected = CUSTOM_COMPATIBLE_CREDENTIAL_ENV[provider]; + const normalized = typeof value === "string" && value.trim() ? value.trim() : expected; + if (normalized !== expected) { + throw new InferenceSetError( + `credential-env for '${provider}' must be '${expected}' so rebuild can safely reuse it.`, + 2, + ); + } + return normalized; +} + +function allowedExplicitInferenceApis(provider: string): string[] { + return provider === "compatible-endpoint" + ? ["openai-completions", "openai-responses"] + : Array.from(INFERENCE_SET_APIS); +} + +function normalizeExplicitInferenceApi(provider: string, value: string | null | undefined): string { + const normalized = typeof value === "string" ? value.trim() : ""; + if (!normalized) { + throw new InferenceSetError( + `inference-api is required for '${provider}' so the shared gateway route can be identified safely.`, + 2, + ); + } + const allowed = allowedExplicitInferenceApis(provider); + if (!allowed.includes(normalized)) { + throw new InferenceSetError( + `inference-api for '${provider}' must be one of: ${allowed.join(", ")}.`, + 2, + ); + } + return normalized; +} + +function explicitCustomProviderMetadataWithoutDns( + provider: string, + options: ExplicitCustomRouteOptions, +): RegistryInferenceMetadata | null { + if (!hasExplicitCustomMetadata(options)) return null; + if (!isCustomCompatibleProvider(provider)) { + throw new InferenceSetError( + "endpoint-url, credential-env, and inference-api are only supported for compatible-endpoint and compatible-anthropic-endpoint.", + 2, + ); + } + + // Source boundary: custom-compatible endpoint URLs are operator-supplied and + // not discoverable from the gateway provider registry with a sandbox-scoped + // trust guarantee. Treat these explicit flags as the durable metadata source + // for this switch, after URL and credential-env validation, instead of + // borrowing from an unrelated onboard session or global OpenShell provider. + return { + endpointUrl: normalizeCustomEndpointUrlWithoutDns(options.endpointUrl), + credentialEnv: normalizeExplicitCredentialEnv(provider, options.credentialEnv), + preferredInferenceApi: normalizeExplicitInferenceApi(provider, options.inferenceApi), + nimContainer: null, + }; +} + +function matchingSessionMetadata(options: { + session: Session | null; + sandboxName: string; + provider: string; + model: string; +}): RegistryInferenceMetadata | null { + const { session, sandboxName, provider, model } = options; + if ( + session?.sandboxName !== sandboxName || + session.provider !== provider || + session.model !== model || + !session.endpointUrl + ) { + return null; + } + return { + endpointUrl: session.endpointUrl, + credentialEnv: session.credentialEnv ?? null, + preferredInferenceApi: session.preferredInferenceApi ?? null, + nimContainer: session.nimContainer ?? null, + }; +} + +function registryMetadataForProviderSwitch(options: { + entry: SandboxEntry; + provider: string; + model: string; + sandboxName: string; + session: Session | null; + explicitMetadata: RegistryInferenceMetadata | null; +}): RegistryInferenceMetadata { + const { entry, provider, model, sandboxName, session, explicitMetadata } = options; + if (explicitMetadata) return explicitMetadata; + if (entry.provider === provider) { + return { + endpointUrl: entry.endpointUrl ?? null, + credentialEnv: entry.credentialEnv ?? null, + preferredInferenceApi: entry.preferredInferenceApi ?? null, + nimContainer: entry.nimContainer ?? null, + }; + } + const sessionMetadata = matchingSessionMetadata({ session, sandboxName, provider, model }); + if (sessionMetadata) return sessionMetadata; + if (isCustomCompatibleProvider(provider)) { + throw new InferenceSetError( + `Cannot switch sandbox '${sandboxName}' to '${provider}' without trusted durable endpoint metadata. ` + + `Re-run onboarding for this custom endpoint or restore a matching onboard session before using inference set.`, + 2, + ); + } + return { + endpointUrl: null, + credentialEnv: null, + preferredInferenceApi: null, + nimContainer: null, + }; +} + +function assertGatewayRouteCompatibility(options: { + gatewayName: string; + sandboxName: string; + provider: string; + model: string; + metadata: RegistryInferenceMetadata; + sandboxes: SandboxEntry[]; +}): void { + const compatibility = checkGatewayRouteCompatibility({ + gatewayName: options.gatewayName, + sandboxName: options.sandboxName, + route: { provider: options.provider, model: options.model, ...options.metadata }, + sandboxes: options.sandboxes, + }); + if (!compatibility.ok) { + throw new InferenceSetError(formatGatewayRouteConflict(compatibility), 2); + } +} + +export function prepareInferenceSetRoute(options: { + entry: SandboxEntry; + sandboxName: string; + provider: string; + model: string; + customRoute: ExplicitCustomRouteOptions; + session: Session | null; + sandboxes: SandboxEntry[]; +}): PreparedInferenceSetRoute { + let gatewayName: string; + try { + gatewayName = resolveSandboxGatewayName(options.entry); + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + throw new InferenceSetError( + `Cannot resolve the OpenShell gateway for sandbox '${options.sandboxName}': ${detail}`, + 2, + ); + } + + const preliminaryExplicitMetadata = explicitCustomProviderMetadataWithoutDns( + options.provider, + options.customRoute, + ); + const preliminaryRegistryMetadata = registryMetadataForProviderSwitch({ + entry: options.entry, + provider: options.provider, + model: options.model, + sandboxName: options.sandboxName, + session: options.session, + explicitMetadata: preliminaryExplicitMetadata, + }); + assertGatewayRouteCompatibility({ + gatewayName, + sandboxName: options.sandboxName, + provider: options.provider, + model: options.model, + metadata: preliminaryRegistryMetadata, + sandboxes: options.sandboxes, + }); + return { gatewayName, preliminaryExplicitMetadata, preliminaryRegistryMetadata }; +} + +export async function finalizeInferenceSetRoute(options: { + prepared: PreparedInferenceSetRoute; + sandboxName: string; + provider: string; + model: string; + sandboxes: SandboxEntry[]; + rewriteUrlWithDnsPinning: RewriteConfigUrlsWithDnsPinning; +}): Promise<{ + registryMetadata: RegistryInferenceMetadata; + explicitPreferredInferenceApi: string | null; +}> { + const { prepared } = options; + if (!prepared.preliminaryExplicitMetadata) { + return { + registryMetadata: prepared.preliminaryRegistryMetadata, + explicitPreferredInferenceApi: null, + }; + } + const registryMetadata: RegistryInferenceMetadata = { + ...prepared.preliminaryExplicitMetadata, + endpointUrl: await normalizeCustomEndpointUrl( + prepared.preliminaryExplicitMetadata.endpointUrl, + options.rewriteUrlWithDnsPinning, + ), + }; + assertGatewayRouteCompatibility({ + gatewayName: prepared.gatewayName, + sandboxName: options.sandboxName, + provider: options.provider, + model: options.model, + metadata: registryMetadata, + sandboxes: options.sandboxes, + }); + return { + registryMetadata, + explicitPreferredInferenceApi: registryMetadata.preferredInferenceApi ?? null, + }; +} diff --git a/src/lib/actions/inference-set.ts b/src/lib/actions/inference-set.ts index 1bde2348311..6041894f9d4 100644 --- a/src/lib/actions/inference-set.ts +++ b/src/lib/actions/inference-set.ts @@ -11,13 +11,8 @@ import { type SandboxInferenceConfig, } from "../inference/config"; import { resolveContextWindowForModel } from "../inference/context-window"; -import { - checkGatewayRouteCompatibility, - formatGatewayRouteConflict, -} from "../inference/gateway-route-compatibility"; import { type ValidationResult, validateLocalProvider } from "../inference/local"; import { inferenceSelectionRegistryFields } from "../inference/selection"; -import { resolveSandboxGatewayName } from "../onboard/gateway-binding"; import { ensureLocalProviderReachable } from "../onboard/local-inference-topology"; import { type AgentConfigTarget, @@ -39,7 +34,13 @@ import { isSafeModelId } from "../validation"; import { hermesApiMode, resolveRuntimeInferenceApi } from "./inference-route-api"; import { InferenceSetError, OPEN_SHELL_FAILURE_CAPTURE_MAX_BUFFER } from "./inference-set-error"; import { buildInferenceSetFailure } from "./inference-set-provider-diagnostics"; +import { + finalizeInferenceSetRoute, + prepareInferenceSetRoute, + type RegistryInferenceMetadata, +} from "./inference-set-route-containment"; +export { normalizeCustomEndpointUrl } from "./inference-set-route-containment"; export { InferenceSetError }; export interface InferenceSetOptions { @@ -385,234 +386,6 @@ function getPreferredInferenceApi(config: ConfigObject): string | null { return typeof inferenceProvider.api === "string" ? inferenceProvider.api : null; } -type RegistryInferenceMetadata = Pick< - SandboxEntry, - "endpointUrl" | "credentialEnv" | "preferredInferenceApi" | "nimContainer" ->; - -const CUSTOM_COMPATIBLE_CREDENTIAL_ENV: Record = { - "compatible-endpoint": "COMPATIBLE_API_KEY", - "compatible-anthropic-endpoint": "COMPATIBLE_ANTHROPIC_API_KEY", -}; - -const INFERENCE_SET_APIS = new Set([ - "openai-completions", - "anthropic-messages", - "openai-responses", -]); - -function isCustomCompatibleProvider(provider: string): boolean { - return provider === "compatible-endpoint" || provider === "compatible-anthropic-endpoint"; -} - -function hasExplicitCustomMetadata(options: InferenceSetOptions): boolean { - return Boolean(options.endpointUrl || options.credentialEnv || options.inferenceApi); -} - -// TRUST BOUNDARY: host.openshell.internal is the single sandbox-to-host bridge -// hostname provisioned by OpenShell. It resolves to the Docker host gateway -// only inside the sandbox network namespace. This exemption is intentionally -// limited below to HTTP, an explicit unprivileged port, and the exact hostname; -// do not extend it to HTTPS, wildcard subdomains, localhost, RFC1918 literals, -// or other internal DNS names. -const ALLOWED_PRIVATE_CUSTOM_ENDPOINT_HOSTS = new Set(["host.openshell.internal"]); - -function normalizeEndpointUrlShape(value: string): { url: URL; normalized: string } { - const url = new URL(value); - if ((url.protocol !== "http:" && url.protocol !== "https:") || url.username || url.password) { - throw new Error("unsupported URL shape"); - } - url.search = ""; - url.hash = ""; - const pathname = url.pathname.replace(/\/+$/, ""); - url.pathname = pathname || "/"; - return { - url, - normalized: url.pathname === "/" ? url.origin : `${url.origin}${url.pathname}`, - }; -} - -export async function normalizeCustomEndpointUrl( - value: string | null | undefined, - rewriteUrlWithDnsPinning: InferenceSetDeps["rewriteConfigUrlsWithDnsPinning"], -): Promise { - const normalized = normalizeCustomEndpointUrlWithoutDns(value); - const shaped = normalizeEndpointUrlShape(normalized); - const hostname = shaped.url.hostname.replace(/\.$/, "").toLowerCase(); - const port = Number(shaped.url.port); - if ( - ALLOWED_PRIVATE_CUSTOM_ENDPOINT_HOSTS.has(hostname) && - shaped.url.protocol === "http:" && - Number.isInteger(port) && - port >= 1024 - ) { - // This is the single sandbox-to-host bridge name that NemoClaw itself - // provisions for local inference. Its supported routes are explicit - // unprivileged HTTP listeners; do not generalize this exemption to HTTPS, - // default/privileged ports, localhost, RFC1918 addresses, or arbitrary - // internal DNS names. - return normalized; - } - - try { - const validated = await rewriteUrlWithDnsPinning(normalized); - if (typeof validated !== "string") throw new Error("URL validator returned a non-string value"); - return normalizeEndpointUrlShape(validated).normalized; - } catch (error) { - const message = error instanceof Error ? error.message : String(error); - throw new InferenceSetError(`endpoint-url is not allowed: ${message}`, 2); - } -} - -function normalizeCustomEndpointUrlWithoutDns(value: string | null | undefined): string { - const raw = typeof value === "string" ? value.trim() : ""; - if (!raw) - throw new InferenceSetError("endpoint-url is required for custom-compatible metadata.", 2); - let shaped: { url: URL; normalized: string }; - try { - shaped = normalizeEndpointUrlShape(raw); - } catch { - throw new InferenceSetError( - "endpoint-url must be a valid http(s) URL without embedded credentials.", - 2, - ); - } - return shaped.normalized; -} - -function normalizeExplicitCredentialEnv( - provider: string, - value: string | null | undefined, -): string { - const expected = CUSTOM_COMPATIBLE_CREDENTIAL_ENV[provider]; - const normalized = typeof value === "string" && value.trim() ? value.trim() : expected; - if (normalized !== expected) { - throw new InferenceSetError( - `credential-env for '${provider}' must be '${expected}' so rebuild can safely reuse it.`, - 2, - ); - } - return normalized; -} - -function allowedExplicitInferenceApis(provider: string): string[] { - return provider === "compatible-endpoint" - ? ["openai-completions", "openai-responses"] - : Array.from(INFERENCE_SET_APIS); -} - -function normalizeExplicitInferenceApi(provider: string, value: string | null | undefined): string { - const normalized = typeof value === "string" ? value.trim() : ""; - if (!normalized) { - throw new InferenceSetError( - `inference-api is required for '${provider}' so the shared gateway route can be identified safely.`, - 2, - ); - } - const allowed = allowedExplicitInferenceApis(provider); - if (!allowed.includes(normalized)) { - throw new InferenceSetError( - `inference-api for '${provider}' must be one of: ${allowed.join(", ")}.`, - 2, - ); - } - return normalized; -} - -function explicitCustomProviderMetadataWithoutDns( - provider: string, - options: InferenceSetOptions, -): RegistryInferenceMetadata | null { - if (!hasExplicitCustomMetadata(options)) return null; - if (!isCustomCompatibleProvider(provider)) { - throw new InferenceSetError( - "endpoint-url, credential-env, and inference-api are only supported for compatible-endpoint and compatible-anthropic-endpoint.", - 2, - ); - } - - // Source boundary: custom-compatible endpoint URLs are operator-supplied and - // not discoverable from the gateway provider registry with a sandbox-scoped - // trust guarantee. Treat these explicit flags as the durable metadata source - // for this switch, after URL and credential-env validation, instead of - // borrowing from an unrelated onboard session or global OpenShell provider. - return { - endpointUrl: normalizeCustomEndpointUrlWithoutDns(options.endpointUrl), - credentialEnv: normalizeExplicitCredentialEnv(provider, options.credentialEnv), - preferredInferenceApi: normalizeExplicitInferenceApi(provider, options.inferenceApi), - nimContainer: null, - }; -} - -async function materializeExplicitCustomProviderMetadata( - metadata: RegistryInferenceMetadata | null, - rewriteUrlWithDnsPinning: InferenceSetDeps["rewriteConfigUrlsWithDnsPinning"], -): Promise { - if (!metadata) return null; - return { - ...metadata, - endpointUrl: await normalizeCustomEndpointUrl(metadata.endpointUrl, rewriteUrlWithDnsPinning), - }; -} - -function matchingSessionMetadata(options: { - session: onboardSession.Session | null; - sandboxName: string; - provider: string; - model: string; -}): RegistryInferenceMetadata | null { - const { session, sandboxName, provider, model } = options; - if ( - session?.sandboxName !== sandboxName || - session.provider !== provider || - session.model !== model || - !session.endpointUrl - ) { - return null; - } - return { - endpointUrl: session.endpointUrl, - credentialEnv: session.credentialEnv ?? null, - preferredInferenceApi: session.preferredInferenceApi ?? null, - nimContainer: session.nimContainer ?? null, - }; -} - -function registryMetadataForProviderSwitch(options: { - entry: SandboxEntry; - provider: string; - model: string; - sandboxName: string; - session: onboardSession.Session | null; - explicitMetadata: RegistryInferenceMetadata | null; -}): RegistryInferenceMetadata { - const { entry, provider, model, sandboxName, session, explicitMetadata } = options; - if (explicitMetadata) return explicitMetadata; - if (entry.provider === provider) { - return { - endpointUrl: entry.endpointUrl ?? null, - credentialEnv: entry.credentialEnv ?? null, - preferredInferenceApi: entry.preferredInferenceApi ?? null, - nimContainer: entry.nimContainer ?? null, - }; - } - const sessionMetadata = matchingSessionMetadata({ session, sandboxName, provider, model }); - if (sessionMetadata) return sessionMetadata; - if (isCustomCompatibleProvider(provider)) { - throw new InferenceSetError( - `Cannot switch sandbox '${sandboxName}' to '${provider}' without trusted durable endpoint metadata. ` + - `Re-run onboarding for this custom endpoint or restore a matching onboard session before using inference set.`, - 2, - ); - } - return { - endpointUrl: null, - credentialEnv: null, - preferredInferenceApi: null, - nimContainer: null, - }; -} - async function runInferenceSetWithoutHostLock( options: InferenceSetOptions, deps: InferenceSetDeps = defaultDeps(), @@ -634,35 +407,16 @@ async function runInferenceSetWithoutHostLock( 2, ); } - let gatewayName: string; - try { - gatewayName = resolveSandboxGatewayName(entry); - } catch (error) { - const detail = error instanceof Error ? error.message : String(error); - throw new InferenceSetError( - `Cannot resolve the OpenShell gateway for sandbox '${sandboxName}': ${detail}`, - 2, - ); - } const session = deps.loadSession(); - const preliminaryExplicitMetadata = explicitCustomProviderMetadataWithoutDns(provider, options); - const preliminaryRegistryMetadata = registryMetadataForProviderSwitch({ + const preparedRoute = prepareInferenceSetRoute({ entry, + sandboxName, provider, model, - sandboxName, + customRoute: options, session, - explicitMetadata: preliminaryExplicitMetadata, - }); - const compatibility = checkGatewayRouteCompatibility({ - gatewayName, - sandboxName, - route: { provider, model, ...preliminaryRegistryMetadata }, sandboxes: deps.listSandboxes().sandboxes, }); - if (!compatibility.ok) { - throw new InferenceSetError(formatGatewayRouteConflict(compatibility), 2); - } const target = deps.resolveAgentConfig(sandboxName); const targetAgent = normalizeSandboxAgent(target.agentName); @@ -678,23 +432,14 @@ async function runInferenceSetWithoutHostLock( 2, ); } - const explicitMetadata = await materializeExplicitCustomProviderMetadata( - preliminaryExplicitMetadata, - deps.rewriteConfigUrlsWithDnsPinning, - ); - const explicitPreferredInferenceApi = explicitMetadata?.preferredInferenceApi ?? null; - const registryMetadata = explicitMetadata ?? preliminaryRegistryMetadata; - if (explicitMetadata) { - const finalizedCompatibility = checkGatewayRouteCompatibility({ - gatewayName, - sandboxName, - route: { provider, model, ...registryMetadata }, - sandboxes: deps.listSandboxes().sandboxes, - }); - if (!finalizedCompatibility.ok) { - throw new InferenceSetError(formatGatewayRouteConflict(finalizedCompatibility), 2); - } - } + const { registryMetadata, explicitPreferredInferenceApi } = await finalizeInferenceSetRoute({ + prepared: preparedRoute, + sandboxName, + provider, + model, + sandboxes: deps.listSandboxes().sandboxes, + rewriteUrlWithDnsPinning: deps.rewriteConfigUrlsWithDnsPinning, + }); // Local providers (ollama-local, vllm-local) route through the sandbox-facing // host.openshell.internal hostname, which the host-side `openshell inference set` @@ -728,7 +473,7 @@ async function runInferenceSetWithoutHostLock( deps.log(` Setting OpenShell inference route: ${provider} / ${model}`); const setResult = deps.captureOpenshell( openshellInferenceSetArgs({ - gatewayName, + gatewayName: preparedRoute.gatewayName, provider, model, noVerify: effectiveNoVerify, diff --git a/src/lib/actions/sandbox/connect-inference-gateway.ts b/src/lib/actions/sandbox/connect-inference-gateway.ts new file mode 100644 index 00000000000..ad5c259faa6 --- /dev/null +++ b/src/lib/actions/sandbox/connect-inference-gateway.ts @@ -0,0 +1,46 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { assertGatewayRouteCompatibility } from "../../inference/gateway-route-compatibility"; +import { LOCAL_INFERENCE_TIMEOUT_SECS } from "../../onboard/env"; +import type { SandboxEntry } from "../../state/registry"; +import * as registry from "../../state/registry"; + +export function buildGatewayInferenceGetArgs(gatewayName: string): string[] { + return ["inference", "get", "-g", gatewayName]; +} + +export function buildGatewayInferenceSetArgs( + gatewayName: string, + provider: string, + model: string, +): string[] { + const args = [ + "inference", + "set", + "-g", + gatewayName, + "--provider", + provider, + "--model", + model, + "--no-verify", + ]; + if (["compatible-endpoint", "ollama-local", "vllm-local"].includes(provider)) { + args.push("--timeout", String(LOCAL_INFERENCE_TIMEOUT_SECS)); + } + return args; +} + +export function assertSandboxGatewayRouteCompatible( + sandboxName: string, + sb: SandboxEntry, + gatewayName: string, +): void { + assertGatewayRouteCompatibility({ + gatewayName, + sandboxName, + route: sb, + sandboxes: registry.listSandboxes().sandboxes, + }); +} diff --git a/src/lib/actions/sandbox/connect-route-containment.test.ts b/src/lib/actions/sandbox/connect-route-containment.test.ts new file mode 100644 index 00000000000..1ea5da324f2 --- /dev/null +++ b/src/lib/actions/sandbox/connect-route-containment.test.ts @@ -0,0 +1,251 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { afterEach, beforeEach, describe, expect, it, type MockInstance, vi } from "vitest"; + +import { + connectModulePath, + createConnectHarness, + requireDist, +} from "../../../../test/support/connect-flow-test-harness"; +import type { SandboxEntry } from "../../state/registry"; +import { + repairSandboxInferenceRouteWithDeps, + type SandboxInferenceRouteRepairDeps, +} from "./connect"; + +describe("connect route containment", () => { + let exitSpy: MockInstance; + const originalStdoutIsTty = process.stdout.isTTY; + + beforeEach(() => { + process.env.NEMOCLAW_TEST_NO_SLEEP = "1"; + Object.defineProperty(process.stdout, "isTTY", { configurable: true, value: true }); + exitSpy = vi.spyOn(process, "exit").mockImplementation(((code?: number | string | null) => { + throw new Error(`process.exit(${code ?? 0})`); + }) as never); + }); + + afterEach(() => { + vi.restoreAllMocks(); + vi.unstubAllEnvs(); + Object.defineProperty(process.stdout, "isTTY", { + configurable: true, + value: originalStdoutIsTty, + }); + delete process.env.NEMOCLAW_TEST_NO_SLEEP; + delete require.cache[requireDist.resolve(connectModulePath)]; + }); + + it("stops before the initial endpoint probe or repair mutation when routes conflict (#6315)", () => { + const conflict = new Error("shared gateway route conflict"); + const assertRouteCompatible = vi.fn(() => { + throw conflict; + }); + const probe = vi.fn(() => ({ healthy: false, broken: true, detail: "BROKEN 503" })); + const applyVmDnsMonkeypatch = vi.fn(() => ({ ok: false })); + const reapplyVmInferenceRoute = vi.fn(() => null); + const repairLegacyDnsProxy = vi.fn(() => ({ exitCode: 0 })); + const deps: SandboxInferenceRouteRepairDeps = { + probe, + shouldApplyVmDnsMonkeypatch: vi.fn(() => false), + applyVmDnsMonkeypatch, + reapplyVmInferenceRoute, + repairLegacyDnsProxy, + assertRouteCompatible, + }; + const sandbox: SandboxEntry = { + name: "demo", + model: "nvidia/nemotron-3-super-120b-a12b", + provider: "nvidia-prod", + openshellDriver: "vm", + gpuEnabled: false, + policies: [], + }; + + expect(() => repairSandboxInferenceRouteWithDeps("vm-box", sandbox, {}, deps)).toThrow( + conflict, + ); + + expect(assertRouteCompatible).toHaveBeenCalledWith("vm-box", sandbox); + expect(probe).not.toHaveBeenCalled(); + expect(applyVmDnsMonkeypatch).not.toHaveBeenCalled(); + expect(reapplyVmInferenceRoute).not.toHaveBeenCalled(); + expect(repairLegacyDnsProxy).not.toHaveBeenCalled(); + }); + + it("exits before connect-time route writes when another sandbox conflicts (#6315)", async () => { + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "anthropic-prod", + model: "claude-sonnet-4-20250514", + } as const; + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + registryEntry: alpha, + registryEntries: [ + alpha, + { + name: "stopped-peer", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + }, + ], + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( + "openshell", + ["sandbox", "connect", "alpha"], + expect.any(Object), + ); + const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); + expect(errorOutput).toContain("stopped-peer"); + expect(errorOutput).toContain("NEMOCLAW_GATEWAY_PORT"); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + + it("exits before repairing a lone incomplete legacy custom route (#6315)", async () => { + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + registryEntry: { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: null, + preferredInferenceApi: null, + }, + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( + "openshell", + ["sandbox", "connect", "alpha"], + expect.any(Object), + ); + const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); + expect(errorOutput).toContain( + "requested custom route lacks durable endpoint or API-family metadata", + ); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + + it("exits before an endpoint probe when an aligned route conflicts with a stopped sandbox (#6315)", async () => { + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + } as const; + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + registryEntry: alpha, + registryEntries: [ + alpha, + { + name: "stopped-peer", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "anthropic-prod", + model: "claude-sonnet-4-20250514", + }, + ], + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + const routeProbeCalls = harness.captureOpenshellSpy.mock.calls.filter((call) => + JSON.stringify(call[0]).includes("inference.local/v1/models"), + ); + expect(routeProbeCalls).toHaveLength(0); + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + + it("scopes every inference read and repair write to the target non-default gateway", async () => { + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw-9090", + gatewayPort: 9090, + openshellDriver: "docker", + provider: "anthropic-prod", + model: "claude-sonnet-4-20250514", + } as const; + const harness = createConnectHarness({ + inferenceGetOutput: + "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", + inferenceProbeResponses: ["BROKEN 503", "BROKEN 503", "OK 200"], + registryEntry: alpha, + registryEntries: [ + alpha, + { + name: "default-gateway-peer", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + }, + ], + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).resolves.toBeUndefined(); + + const inferenceReads = harness.captureOpenshellSpy.mock.calls + .map((call) => call[0]) + .filter((args) => Array.isArray(args) && args[0] === "inference" && args[1] === "get"); + expect(inferenceReads).toEqual([["inference", "get", "-g", "nemoclaw-9090"]]); + + const inferenceWrites = harness.runOpenshellSpy.mock.calls + .map((call) => call[0]) + .filter((args) => Array.isArray(args) && args[0] === "inference" && args[1] === "set"); + expect(inferenceWrites).toHaveLength(3); + for (const args of inferenceWrites) { + expect(args).toEqual([ + "inference", + "set", + "-g", + "nemoclaw-9090", + "--provider", + "anthropic-prod", + "--model", + "claude-sonnet-4-20250514", + "--no-verify", + ]); + } + expect([...inferenceReads, ...inferenceWrites]).not.toContainEqual( + expect.arrayContaining(["-g", "nemoclaw"]), + ); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + }); +}); diff --git a/src/lib/actions/sandbox/connect-route-lifecycle.test.ts b/src/lib/actions/sandbox/connect-route-lifecycle.test.ts index a7ed7ff1bac..b6fe791d921 100644 --- a/src/lib/actions/sandbox/connect-route-lifecycle.test.ts +++ b/src/lib/actions/sandbox/connect-route-lifecycle.test.ts @@ -64,6 +64,8 @@ describe("connectSandbox route lifecycle", () => { [ "inference", "set", + "-g", + "nemoclaw", "--provider", "anthropic-prod", "--model", @@ -79,123 +81,6 @@ describe("connectSandbox route lifecycle", () => { ); }); - it("exits before connect-time route writes when another sandbox conflicts (#6315)", async () => { - const alpha = { - name: "alpha", - agent: "openclaw", - gatewayName: "nemoclaw", - gatewayPort: 8080, - provider: "anthropic-prod", - model: "claude-sonnet-4-20250514", - } as const; - const harness = createConnectHarness({ - inferenceGetOutput: - "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", - registryEntry: alpha, - registryEntries: [ - alpha, - { - name: "stopped-peer", - agent: "openclaw", - gatewayName: "nemoclaw", - gatewayPort: 8080, - provider: "nvidia-prod", - model: "nvidia/nemotron-3-super-120b-a12b", - }, - ], - }); - - await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( - "process.exit(1)", - ); - - expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); - expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); - expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); - expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( - "openshell", - ["sandbox", "connect", "alpha"], - expect.any(Object), - ); - const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); - expect(errorOutput).toContain("stopped-peer"); - expect(errorOutput).toContain("NEMOCLAW_GATEWAY_PORT"); - expect(exitSpy).toHaveBeenCalledWith(1); - }); - - it("exits before repairing a lone incomplete legacy custom route (#6315)", async () => { - const harness = createConnectHarness({ - inferenceGetOutput: - "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", - registryEntry: { - name: "alpha", - agent: "openclaw", - gatewayName: "nemoclaw", - gatewayPort: 8080, - provider: "compatible-endpoint", - model: "custom/model", - endpointUrl: null, - preferredInferenceApi: null, - }, - }); - - await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( - "process.exit(1)", - ); - - expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); - expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); - expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); - expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( - "openshell", - ["sandbox", "connect", "alpha"], - expect.any(Object), - ); - const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); - expect(errorOutput).toContain( - "requested custom route lacks durable endpoint or API-family metadata", - ); - expect(exitSpy).toHaveBeenCalledWith(1); - }); - - it("exits before an endpoint probe when an aligned route conflicts with a stopped sandbox (#6315)", async () => { - const alpha = { - name: "alpha", - agent: "openclaw", - gatewayName: "nemoclaw", - gatewayPort: 8080, - provider: "nvidia-prod", - model: "nvidia/nemotron-3-super-120b-a12b", - } as const; - const harness = createConnectHarness({ - inferenceGetOutput: - "Gateway inference:\n Provider: nvidia-prod\n Model: nvidia/nemotron-3-super-120b-a12b\n", - registryEntry: alpha, - registryEntries: [ - alpha, - { - name: "stopped-peer", - agent: "openclaw", - gatewayName: "nemoclaw", - gatewayPort: 8080, - provider: "anthropic-prod", - model: "claude-sonnet-4-20250514", - }, - ], - }); - - await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( - "process.exit(1)", - ); - - const routeProbeCalls = harness.captureOpenshellSpy.mock.calls.filter((call) => - JSON.stringify(call[0]).includes("inference.local/v1/models"), - ); - expect(routeProbeCalls).toHaveLength(0); - expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); - expect(exitSpy).toHaveBeenCalledWith(1); - }); - it("wires the forced VM DNS monkeypatch into connect route repair", async () => { vi.stubEnv("NEMOCLAW_FORCE_VM_DNS_MONKEYPATCH", "1"); try { @@ -239,7 +124,7 @@ describe("connectSandbox route lifecycle", () => { await expect(harness.connectSandbox("alpha", { probeOnly: true })).resolves.toBeUndefined(); expect(harness.captureOpenshellSpy).not.toHaveBeenCalledWith( - ["inference", "get"], + ["inference", "get", "-g", "nemoclaw"], expect.any(Object), ); expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); @@ -258,7 +143,7 @@ describe("connectSandbox route lifecycle", () => { await expect(harness.connectSandbox("alpha", { probeOnly: true })).resolves.toBeUndefined(); expect(harness.captureOpenshellSpy).toHaveBeenCalledWith( - ["inference", "get"], + ["inference", "get", "-g", "nemoclaw"], expect.objectContaining({ ignoreError: true }), ); expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); @@ -284,6 +169,8 @@ describe("connectSandbox route lifecycle", () => { [ "inference", "set", + "-g", + "nemoclaw", "--provider", "nvidia-prod", "--model", diff --git a/src/lib/actions/sandbox/connect-route-repair.test.ts b/src/lib/actions/sandbox/connect-route-repair.test.ts index 3b6ae0b1be0..c06d83a5f30 100644 --- a/src/lib/actions/sandbox/connect-route-repair.test.ts +++ b/src/lib/actions/sandbox/connect-route-repair.test.ts @@ -144,27 +144,6 @@ describe("sandbox connect route repair unit flow", () => { } }); - it("stops before the initial endpoint probe or repair mutation when routes conflict (#6315)", () => { - const conflict = new Error("shared gateway route conflict"); - const assertRouteCompatible = vi.fn(() => { - throw conflict; - }); - const { calls, deps } = makeRepairDeps([broken()], { assertRouteCompatible }); - - expect(() => - repairSandboxInferenceRouteWithDeps("vm-box", sandbox({ openshellDriver: "vm" }), {}, deps), - ).toThrow(conflict); - - expect(assertRouteCompatible).toHaveBeenCalledWith( - "vm-box", - expect.objectContaining({ name: "demo" }), - ); - expect(deps.probe).not.toHaveBeenCalled(); - expect(calls.monkeypatches).toEqual([]); - expect(calls.reapplications).toEqual([]); - expect(calls.legacyRepairs).toEqual([]); - }); - it("repairs legacy kubernetes routes through the DNS proxy path", () => { const { calls, deps } = makeRepairDeps([broken(), healthy()]); diff --git a/src/lib/actions/sandbox/connect.ts b/src/lib/actions/sandbox/connect.ts index 3ba4f748bf2..0878065bf2a 100644 --- a/src/lib/actions/sandbox/connect.ts +++ b/src/lib/actions/sandbox/connect.ts @@ -23,13 +23,9 @@ import { planInferenceRouteReconcile, sanitizeRouteValueForDisplay, } from "../../inference/config"; -import { - assertGatewayRouteCompatibility, - GatewayRouteConflictError, -} from "../../inference/gateway-route-compatibility"; +import { GatewayRouteConflictError } from "../../inference/gateway-route-compatibility"; import { findReachableOllamaHost, probeLocalProviderHealth } from "../../inference/local"; import { ensureOllamaAuthProxy, probeOllamaAuthProxyHealth } from "../../inference/ollama/proxy"; -import { LOCAL_INFERENCE_TIMEOUT_SECS } from "../../onboard/env"; import { resolveSandboxGatewayName } from "../../onboard/gateway-binding"; import { isWsl } from "../../platform"; import { ROOT } from "../../runner"; @@ -59,6 +55,11 @@ import { exitOnMcpReconciliationRefusal, exitOnSecretBoundaryRefusal, } from "./connect-boundary-refusal"; +import { + assertSandboxGatewayRouteCompatible, + buildGatewayInferenceGetArgs, + buildGatewayInferenceSetArgs, +} from "./connect-inference-gateway"; import { buildSandboxInferenceRouteProbeArgs, type InferenceRouteProbeAgent, @@ -402,32 +403,15 @@ function shouldUseLegacyDnsProxyRepair(sb: SandboxEntry | null): boolean { return driver !== "vm" && driver !== "docker"; } -function buildInferenceSetArgs(provider: string, model: string): string[] { - const args = ["inference", "set", "--provider", provider, "--model", model, "--no-verify"]; - if (["compatible-endpoint", "ollama-local", "vllm-local"].includes(provider)) { - args.push("--timeout", String(LOCAL_INFERENCE_TIMEOUT_SECS)); - } - return args; -} - -function assertSandboxGatewayRouteCompatible(sandboxName: string, sb: SandboxEntry | null): void { - if (!sb) return; - assertGatewayRouteCompatibility({ - gatewayName: resolveSandboxGatewayName(sb), - sandboxName, - route: sb, - sandboxes: registry.listSandboxes().sandboxes, - }); -} - function reapplyVmInferenceRoute( sandboxName: string, sb: SandboxEntry | null, agent: InferenceRouteProbeAgent, + gatewayName: string, ): SandboxInferenceRouteProbe | null { const inference = sb ? registry.getSandboxEntryInference(sb) : null; if (inference?.kind !== "configured") return null; - runOpenshell(buildInferenceSetArgs(inference.provider, inference.model), { + runOpenshell(buildGatewayInferenceSetArgs(gatewayName, inference.provider, inference.model), { ignoreError: true, timeout: OPENSHELL_OPERATION_TIMEOUT_MS, }); @@ -572,6 +556,7 @@ function repairSandboxInferenceRouteIfNeeded( sandboxName: string, sb: SandboxEntry | null, agent: InferenceRouteProbeAgent, + gatewayName: string, { quiet = false }: { quiet?: boolean } = {}, ): SandboxInferenceRouteRepairResult { return repairSandboxInferenceRouteWithDeps( @@ -583,13 +568,16 @@ function repairSandboxInferenceRouteIfNeeded( probe: (name, options) => probeSandboxInferenceRoute(name, agent, options), shouldApplyVmDnsMonkeypatch, applyVmDnsMonkeypatch: applyOpenShellVmDnsMonkeypatch, - reapplyVmInferenceRoute: (name, sandbox) => reapplyVmInferenceRoute(name, sandbox, agent), + reapplyVmInferenceRoute: (name, sandbox) => + reapplyVmInferenceRoute(name, sandbox, agent, gatewayName), repairLegacyDnsProxy: (name, isQuiet) => runSetupDnsProxy( - { gatewayName: resolveSandboxGatewayName(sb), sandboxName: name }, + { gatewayName, sandboxName: name }, { log: isQuiet ? () => undefined : console.log }, ), - assertRouteCompatible: assertSandboxGatewayRouteCompatible, + assertRouteCompatible: (name, sandbox) => { + if (sandbox) assertSandboxGatewayRouteCompatible(name, sandbox, gatewayName); + }, }, ); } @@ -692,6 +680,7 @@ function resetManagedInferenceRoute( sandboxName: string, sb: SandboxEntry, agent: InferenceRouteProbeAgent, + gatewayName: string, { detail, quiet = false }: { detail: string; quiet?: boolean }, ): boolean { return resetManagedInferenceRouteWithDeps( @@ -701,7 +690,7 @@ function resetManagedInferenceRoute( { verifyLocalInferenceRouteDependencies, runInferenceSet: (provider, model) => - runOpenshell(buildInferenceSetArgs(provider, model), { + runOpenshell(buildGatewayInferenceSetArgs(gatewayName, provider, model), { ignoreError: true, timeout: OPENSHELL_OPERATION_TIMEOUT_MS, }), @@ -725,15 +714,16 @@ function ensureSandboxInferenceRoute( inference = registry.getSandboxEntryInference(sb); if (inference.kind !== "configured") return { sandbox: sb, routeHealthy: null }; const { provider, model } = inference; + const gatewayName = resolveSandboxGatewayName(sb); const live = parseGatewayInference( - captureOpenshell(["inference", "get"], { + captureOpenshell(buildGatewayInferenceGetArgs(gatewayName), { ignoreError: true, timeout: OPENSHELL_PROBE_TIMEOUT_MS, }).output, ); const plan = planInferenceRouteReconcile(live, { provider, model }); if (plan.kind !== "aligned") { - assertSandboxGatewayRouteCompatible(sandboxName, sb); + assertSandboxGatewayRouteCompatible(sandboxName, sb, gatewayName); const recordedRoute = `${sanitizeRouteValueForDisplay(provider)}/${sanitizeRouteValueForDisplay(model)}`; if (plan.kind === "diverged") { // Shared gateway: re-point loudly (even when quiet) — silent revert was @@ -755,7 +745,7 @@ function ensureSandboxInferenceRoute( // plan.kind === "repair": empty gateway, genuine repair — quiet-aware. console.log(` Setting inference route to ${recordedRoute} for sandbox '${sandboxName}'`); } - const swapResult = runOpenshell(buildInferenceSetArgs(provider, model), { + const swapResult = runOpenshell(buildGatewayInferenceSetArgs(gatewayName, provider, model), { ignoreError: true, timeout: OPENSHELL_OPERATION_TIMEOUT_MS, }); @@ -765,9 +755,11 @@ function ensureSandboxInferenceRoute( ); } } - const repairResult = repairSandboxInferenceRouteIfNeeded(sandboxName, sb, agent, { quiet }); + const repairResult = repairSandboxInferenceRouteIfNeeded(sandboxName, sb, agent, gatewayName, { + quiet, + }); if (!repairResult.healthy && repairResult.repairAttempted) { - const resetResult = resetManagedInferenceRoute(sandboxName, sb, agent, { + const resetResult = resetManagedInferenceRoute(sandboxName, sb, agent, gatewayName, { detail: repairResult.detail, quiet, }); diff --git a/src/lib/inference/gateway-route-compatibility.ts b/src/lib/inference/gateway-route-compatibility.ts index 2e134f3b594..a1c92912991 100644 --- a/src/lib/inference/gateway-route-compatibility.ts +++ b/src/lib/inference/gateway-route-compatibility.ts @@ -19,7 +19,7 @@ export interface GatewayRouteCompatibilityRequest { export type CurrentGatewayRouteCompatibilityRequest = Pick< GatewayRouteCompatibilityRequest, - "sandboxName" | "route" + "gatewayName" | "sandboxName" | "route" >; export type CurrentGatewayRouteCompatibilityCheck = ( diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 730f5f42c2a..01deb1cf744 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -1009,7 +1009,7 @@ const providerExistsInGateway = (name: string) => onboardProviders.providerExistsInGateway(name, runOpenshell); const { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility } = - inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell, () => GATEWAY_NAME); + inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); const { inspectSandboxForCreate, pruneStaleSandboxEntry, @@ -4843,6 +4843,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { const [providerInferencePhase, sandboxPhase] = createCoreOnboardFlowPhases({ + gatewayName: GATEWAY_NAME, forceProviderSelection: forceProviderSelectionForAgentChange, authoritativeResumeConfig: opts.authoritativeResumeConfig === true, env: process.env, diff --git a/src/lib/onboard/inference-route.ts b/src/lib/onboard/inference-route.ts index e7eb2325ff6..14535c3a449 100644 --- a/src/lib/onboard/inference-route.ts +++ b/src/lib/onboard/inference-route.ts @@ -10,10 +10,7 @@ import { listSandboxes } from "../state/registry"; type RunCaptureOpenshell = (args: string[], options?: { ignoreError?: boolean }) => string | null; -export function createInferenceRouteHelpers( - runCaptureOpenshell: RunCaptureOpenshell, - getGatewayName: () => string = () => "nemoclaw", -) { +export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpenshell) { function verifyInferenceRoute(_provider: string, _model: string): void { const output = runCaptureOpenshell(["inference", "get"], { ignoreError: true }); if (!output || /Gateway inference:\s*[\r\n]+\s*Not configured/i.test(output)) { @@ -32,7 +29,6 @@ export function createInferenceRouteHelpers( const checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck = (request) => checkGatewayRouteCompatibilityForRegistry({ ...request, - gatewayName: getGatewayName(), sandboxes: listSandboxes().sandboxes, }); diff --git a/src/lib/onboard/machine/core-flow-phases.test.ts b/src/lib/onboard/machine/core-flow-phases.test.ts index 7f27a684428..b2c719cb3a8 100644 --- a/src/lib/onboard/machine/core-flow-phases.test.ts +++ b/src/lib/onboard/machine/core-flow-phases.test.ts @@ -75,6 +75,7 @@ function createPhases( } = {}, ) { return createCoreOnboardFlowPhases({ + gatewayName: "nemoclaw", forceProviderSelection: false, env: {}, constants: { diff --git a/src/lib/onboard/machine/core-flow-phases.ts b/src/lib/onboard/machine/core-flow-phases.ts index 876d4ea151a..d734f177c51 100644 --- a/src/lib/onboard/machine/core-flow-phases.ts +++ b/src/lib/onboard/machine/core-flow-phases.ts @@ -25,6 +25,7 @@ export interface CoreOnboardFlowPhaseOptions< MessagingChannelConfig = unknown, ResourceProfile = unknown, > { + gatewayName: string; forceProviderSelection: boolean; authoritativeResumeConfig?: boolean; env: NodeJS.ProcessEnv; @@ -55,6 +56,7 @@ export function createCoreOnboardFlowPhases< ): [OnboardSequencePhase, OnboardSequencePhase] { const providerInferencePhase = createProviderInferencePhase(async (context) => { const providerInferenceResult = await handleProviderInferenceState({ + gatewayName: options.gatewayName, resume: context.resume, fresh: context.fresh, session: context.session, diff --git a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts new file mode 100644 index 00000000000..fe5c55f9a4a --- /dev/null +++ b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts @@ -0,0 +1,224 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it, vi } from "vitest"; + +import type { CurrentGatewayRouteCompatibilityCheck } from "../../../inference/gateway-route-compatibility"; +import { createSession, type Session, type SessionUpdates } from "../../../state/onboard-session"; +import { + handleProviderInferenceState, + type ProviderInferenceStateOptions, + type ProviderSelectionResult, +} from "./provider-inference"; + +type Options = ProviderInferenceStateOptions; + +const fallbackSelection: ProviderSelectionResult = { + model: "nvidia/test", + provider: "nvidia-prod", + endpointUrl: "https://integrate.api.nvidia.com/v1", + credentialEnv: "NVIDIA_INFERENCE_API_KEY", + hermesAuthMethod: null, + hermesToolGateways: [], + preferredInferenceApi: "openai-responses", + compatibleEndpointReasoning: null, + nimContainer: null, +}; + +function createDeps() { + const calls = { + checkGatewayRouteCompatibility: vi.fn(() => ({ + ok: true, + })), + setupNim: vi.fn(async () => ({ ...fallbackSelection })), + setupInference: vi.fn(async () => ({ ok: true as const })), + recordStepComplete: vi.fn(async () => createSession()), + reconcileRouter: vi.fn(async () => undefined), + reupsertRoutedProvider: vi.fn( + (_provider: string, endpointUrl: string | null, _credentialEnv: string | null) => ({ + ok: true as const, + endpointUrl: endpointUrl ?? "http://host.openshell.internal:4000/v1", + }), + ), + updateSandbox: vi.fn(), + log: vi.fn(), + error: vi.fn(), + exit: vi.fn((code: number): never => { + throw new Error(`exit ${code}`); + }), + }; + const deps: Options["deps"] = { + checkGatewayRouteCompatibility: calls.checkGatewayRouteCompatibility, + normalizeHermesAuthMethod: () => null, + setupNim: calls.setupNim, + setupInference: calls.setupInference, + startRecordedStep: vi.fn(async () => undefined), + recordStepComplete: calls.recordStepComplete, + toSessionUpdates: (updates: Record) => updates as SessionUpdates, + skippedStepMessage: vi.fn(), + ensureResumeProviderReady: vi.fn(async (_provider, credentialEnv) => ({ + forceInferenceSetup: false, + credentialEnv: credentialEnv ?? null, + })), + recordStateSkipped: vi.fn(async () => createSession()), + recordRepairEvent: vi.fn(async () => createSession()), + hydrateCredentialEnv: vi.fn(() => "test-key"), + configureCompatibleEndpointReasoning: vi.fn(async () => "false" as const), + clearCompatibleEndpointReasoning: () => null, + repairLocalInferenceSystemdOverrideOrExit: vi.fn(), + isNonInteractive: () => true, + getOpenshellBinary: () => "/usr/bin/openshell", + needsBedrockRuntimeAdapter: () => false, + isInferenceRouteReady: () => true, + isRoutedInferenceProvider: (provider) => provider === "nvidia-router", + reconcileModelRouter: calls.reconcileRouter, + reupsertRoutedProvider: calls.reupsertRoutedProvider, + registryUpdateSandbox: calls.updateSandbox, + promptValidatedSandboxName: vi.fn(async () => "target-sandbox"), + assessHost: () => ({ cpus: 8 }), + formatSandboxBuildEstimateNote: () => "estimate", + formatOnboardConfigSummary: ({ provider, model, sandboxName }) => + `summary:${provider}/${model}/${sandboxName}`, + promptYesNoOrDefault: vi.fn(async () => true), + cliName: () => "nemoclaw", + log: calls.log, + error: calls.error, + exitProcess: calls.exit, + deleteEnv: vi.fn(), + }; + return { calls, deps }; +} + +function resumeOptions( + deps: Options["deps"], + session: Session, + selectedMessagingChannels: string[] = [], +): Options { + return { + gatewayName: "nemoclaw-9090", + resume: true, + fresh: false, + session, + gpu: null, + sandboxName: "target-sandbox", + agent: null, + initial: { + model: session.model, + provider: session.provider, + endpointUrl: session.endpointUrl, + credentialEnv: session.credentialEnv, + hermesAuthMethod: session.hermesAuthMethod, + hermesToolGateways: session.hermesToolGateways ?? [], + preferredInferenceApi: session.preferredInferenceApi, + compatibleEndpointReasoning: session.compatibleEndpointReasoning, + nimContainer: session.nimContainer, + webSearchConfig: session.webSearchConfig, + }, + selectedMessagingChannels, + env: {}, + constants: { + hermesProviderName: "hermes-provider", + hermesApiKeyAuthMethod: "api_key", + hermesApiKeyCredentialEnv: "NOUS_API_KEY", + }, + deps, + }; +} + +function rejectRoute( + calls: ReturnType["calls"], + provider: string, + model: string, +) { + calls.checkGatewayRouteCompatibility.mockReturnValue({ + ok: false, + gatewayName: "nemoclaw-9090", + sandboxName: "target-sandbox", + route: { provider, model }, + conflicts: [{ sandboxName: "existing-sandbox", reason: "provider-model" }], + }); +} + +describe("provider route containment", () => { + it("rejects a fresh selection before completing its session step or starting inference", async () => { + const { calls, deps } = createDeps(); + rejectRoute(calls, "nvidia-prod", "nvidia/test"); + const options = resumeOptions(deps, createSession()); + + await expect( + handleProviderInferenceState({ ...options, resume: false, sandboxName: null }), + ).rejects.toThrow("exit 1"); + + expect(calls.setupNim).toHaveBeenCalledOnce(); + expect(calls.checkGatewayRouteCompatibility).toHaveBeenCalledWith({ + gatewayName: "nemoclaw-9090", + sandboxName: null, + route: { + provider: "nvidia-prod", + model: "nvidia/test", + endpointUrl: "https://integrate.api.nvidia.com/v1", + preferredInferenceApi: "openai-responses", + }, + }); + expect(calls.recordStepComplete).not.toHaveBeenCalled(); + expect(calls.setupInference).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + }); + + it("blocks routed-provider repair before gateway or registry mutation", async () => { + const session = createSession({ provider: "nvidia-router", model: "router/model" }); + session.steps.provider_selection.status = "complete"; + const { calls, deps } = createDeps(); + rejectRoute(calls, "nvidia-router", "router/model"); + + await expect(handleProviderInferenceState(resumeOptions(deps, session))).rejects.toThrow( + "exit 1", + ); + + expect(calls.checkGatewayRouteCompatibility).toHaveBeenCalledWith({ + gatewayName: "nemoclaw-9090", + sandboxName: "target-sandbox", + route: { + provider: "nvidia-router", + model: "router/model", + endpointUrl: null, + preferredInferenceApi: null, + }, + }); + expect(calls.reconcileRouter).not.toHaveBeenCalled(); + expect(calls.reupsertRoutedProvider).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + expect(calls.setupInference).not.toHaveBeenCalled(); + }); + + it("blocks compatible-endpoint messaging refresh before endpoint or gateway work", async () => { + const session = createSession({ + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + }); + session.steps.provider_selection.status = "complete"; + const { calls, deps } = createDeps(); + rejectRoute(calls, "compatible-endpoint", "custom/model"); + + await expect( + handleProviderInferenceState(resumeOptions(deps, session, ["telegram"])), + ).rejects.toThrow("exit 1"); + + expect(calls.checkGatewayRouteCompatibility).toHaveBeenCalledWith({ + gatewayName: "nemoclaw-9090", + sandboxName: "target-sandbox", + route: { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + }, + }); + expect(calls.setupInference).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + expect(calls.error).toHaveBeenCalledWith(expect.stringContaining("existing-sandbox")); + }); +}); diff --git a/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts b/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts new file mode 100644 index 00000000000..f22285b1f94 --- /dev/null +++ b/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts @@ -0,0 +1,27 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { + type CurrentGatewayRouteCompatibilityCheck, + formatGatewayRouteConflict, + type GatewayInferenceRoute, +} from "../../../inference/gateway-route-compatibility"; + +export interface ProviderInferenceRouteContainmentDeps { + checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; + error(message: string): void; + exitProcess(code: number): never; +} + +export function assertProviderInferenceRouteCompatible( + deps: ProviderInferenceRouteContainmentDeps, + gatewayName: string, + sandboxName: string | null, + route: GatewayInferenceRoute, +): void { + const compatibility = deps.checkGatewayRouteCompatibility({ gatewayName, sandboxName, route }); + if (!compatibility.ok) { + deps.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); + deps.exitProcess(1); + } +} diff --git a/src/lib/onboard/machine/handlers/provider-inference.test.ts b/src/lib/onboard/machine/handlers/provider-inference.test.ts index e98a21cafa6..07315d14e85 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.test.ts @@ -125,6 +125,7 @@ function baseOptions( session: Session | null = createSession(), ): ProviderInferenceStateOptions { return { + gatewayName: "nemoclaw", resume: false, fresh: false, session, @@ -830,38 +831,6 @@ describe("handleProviderInferenceState", () => { expect(calls.reconcileRouter).toHaveBeenCalledOnce(); }); - it("blocks a conflicting resumed routed provider before gateway or registry mutation (#6315)", async () => { - const session = createSession({ - provider: "nvidia-router", - model: "router/model", - endpointUrl: "http://host.openshell.internal:4000/v1", - preferredInferenceApi: "openai-completions", - }); - session.steps.provider_selection.status = "complete"; - const { deps, calls } = createDeps({ isInferenceRouteReady: vi.fn(() => true) }); - calls.checkGatewayRouteCompatibility.mockReturnValue({ - ok: false, - gatewayName: "nemoclaw", - sandboxName: "router-sandbox", - route: { provider: "nvidia-router", model: "router/model" }, - conflicts: [{ sandboxName: "existing-sandbox", reason: "provider-model" }], - }); - - await expect( - handleProviderInferenceState({ - ...baseOptions(deps, session), - resume: true, - sandboxName: "router-sandbox", - }), - ).rejects.toThrow("exit 1"); - - expect(calls.reconcileRouter).not.toHaveBeenCalled(); - expect(calls.reupsertRoutedProvider).not.toHaveBeenCalled(); - expect(calls.updateSandbox).not.toHaveBeenCalled(); - expect(calls.setupInference).not.toHaveBeenCalled(); - expect(calls.error).toHaveBeenCalledWith(expect.stringContaining("existing-sandbox")); - }); - // #5974 instance 5: the Model Router Python preflight (`prepareModelRouterVenv`) // throws a plain Error (e.g. "above supported ceiling", with no `oclif.exit`) // out of `reconcileModelRouter`. The routed branch must catch that throw and diff --git a/src/lib/onboard/machine/handlers/provider-inference.ts b/src/lib/onboard/machine/handlers/provider-inference.ts index 5c0886fea1a..14746a7860e 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.ts @@ -2,14 +2,12 @@ // SPDX-License-Identifier: Apache-2.0 import { coerceAgentInferenceApi } from "../../../inference/config"; -import { - type CurrentGatewayRouteCompatibilityCheck, - formatGatewayRouteConflict, -} from "../../../inference/gateway-route-compatibility"; +import type { CurrentGatewayRouteCompatibilityCheck } from "../../../inference/gateway-route-compatibility"; import type { WebSearchConfig } from "../../../inference/web-search"; import type { HermesAuthMethod, Session, SessionUpdates } from "../../../state/onboard-session"; import { withInferenceTrace, withProviderSelectionTrace } from "../../tracing"; import { advanceTo, type OnboardStateTransitionResult, retryTo } from "../result"; +import { assertProviderInferenceRouteCompatible } from "./provider-inference-route-containment"; export type ProviderInferenceRetry = { retry: "selection" } | { ok: true; retry?: undefined }; @@ -42,6 +40,7 @@ export interface ProviderSelectionResult { } export interface ProviderInferenceStateOptions { + gatewayName: string; resume: boolean; fresh: boolean; session: Session | null; @@ -229,6 +228,7 @@ function shouldRefreshCompatibleEndpointRouteForMessaging( } export async function handleProviderInferenceState({ + gatewayName, resume, fresh, session, @@ -281,6 +281,12 @@ export async function handleProviderInferenceState({ typeof model === "string"; let shouldRecordProviderSelection = false; if (resumeProviderSelection) { + assertProviderInferenceRouteCompatible(deps, gatewayName, sandboxName, { + provider, + model, + endpointUrl, + preferredInferenceApi, + }); const recovery = await deps.ensureResumeProviderReady(provider, credentialEnv); forceInferenceSetup = recovery.forceInferenceSetup; credentialEnv = recovery.credentialEnv; @@ -395,6 +401,14 @@ export async function handleProviderInferenceState({ const selectedModel = selected.model; provider = selectedProvider; model = selectedModel; + if (!resumeProviderSelection) { + assertProviderInferenceRouteCompatible(deps, gatewayName, sandboxName, { + provider, + model, + endpointUrl, + preferredInferenceApi, + }); + } if (shouldRecordProviderSelection) { session = await deps.recordStepComplete( "provider_selection", @@ -425,14 +439,6 @@ export async function handleProviderInferenceState({ effectiveResume && deps.isInferenceRouteReady(provider, model); if (resumeInference) { - const compatibility = deps.checkGatewayRouteCompatibility({ - sandboxName, - route: { provider, model, endpointUrl, preferredInferenceApi }, - }); - if (!compatibility.ok) { - deps.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); - deps.exitProcess(1); - } if (provider === constants.hermesProviderName) { let inferenceResult: ProviderInferenceRetry; try { diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index 077762d07b7..9a708ade329 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -135,6 +135,7 @@ export function createSetupInference( options: ProviderInferenceSetupOptions = {}, ): Promise { const compatibility = deps.checkGatewayRouteCompatibility({ + gatewayName: deps.getGatewayName(), sandboxName, route: { provider, From 35cf64f32147dbedd1518625202b44d2d97a122d Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 12:39:51 -0700 Subject: [PATCH 08/31] refactor(cli): share inference set flag guidance Signed-off-by: Aaron Erickson --- src/commands/inference/set.ts | 12 ++---------- src/commands/sandbox/inference/set.ts | 14 ++------------ src/lib/cli/inference-set-help.ts | 16 ++++++++++++++++ 3 files changed, 20 insertions(+), 22 deletions(-) create mode 100644 src/lib/cli/inference-set-help.ts diff --git a/src/commands/inference/set.ts b/src/commands/inference/set.ts index a8c85b0ea19..cbf1329f3ad 100644 --- a/src/commands/inference/set.ts +++ b/src/commands/inference/set.ts @@ -4,8 +4,8 @@ import { Flags } from "@oclif/core"; import { InferenceSetError, runInferenceSet } from "../../lib/actions/inference-set"; -import { CLI_NAME } from "../../lib/cli/branding"; import { nonEmptyFlag } from "../../lib/cli/flag-helpers"; +import { inferenceSetRequiredFlagsFailureLines } from "../../lib/cli/inference-set-help"; import { NemoClawCommand } from "../../lib/cli/nemoclaw-oclif-command"; // Global inference:set is paired with the sandbox-first sandbox:inference:set @@ -73,15 +73,7 @@ export default class InferenceSetCommand extends NemoClawCommand { private printRequiredFlags(): void { this.failWithLines( - [ - ` ${CLI_NAME} inference set requires --provider and --model.`, - "", - ` Run: ${CLI_NAME} inference set --provider --model [--sandbox ]`, - " NemoClaw must perform this operation so it can protect every sandbox sharing the target gateway.", - "", - ` Run '${CLI_NAME} help' for NemoClaw commands.`, - ], - 1, + inferenceSetRequiredFlagsFailureLines("inference set", " [--sandbox ]"), ); } } diff --git a/src/commands/sandbox/inference/set.ts b/src/commands/sandbox/inference/set.ts index aa337d8f88a..2af01c9c3ba 100644 --- a/src/commands/sandbox/inference/set.ts +++ b/src/commands/sandbox/inference/set.ts @@ -4,8 +4,8 @@ import { Flags } from "@oclif/core"; import { InferenceSetError, runInferenceSet } from "../../../lib/actions/inference-set"; -import { CLI_NAME } from "../../../lib/cli/branding"; import { nonEmptyFlag } from "../../../lib/cli/flag-helpers"; +import { inferenceSetRequiredFlagsFailureLines } from "../../../lib/cli/inference-set-help"; import { NemoClawCommand } from "../../../lib/cli/nemoclaw-oclif-command"; import { sandboxNameArg } from "../../../lib/sandbox/command-support"; @@ -76,16 +76,6 @@ export default class SandboxInferenceSetCommand extends NemoClawCommand { } private printRequiredFlags(): void { - this.failWithLines( - [ - ` ${CLI_NAME} inference set requires --provider and --model.`, - "", - ` Run: ${CLI_NAME} inference set --provider --model `, - " NemoClaw must perform this operation so it can protect every sandbox sharing the target gateway.", - "", - ` Run '${CLI_NAME} help' for NemoClaw commands.`, - ], - 1, - ); + this.failWithLines(inferenceSetRequiredFlagsFailureLines(" inference set")); } } diff --git a/src/lib/cli/inference-set-help.ts b/src/lib/cli/inference-set-help.ts new file mode 100644 index 00000000000..80c95ffc520 --- /dev/null +++ b/src/lib/cli/inference-set-help.ts @@ -0,0 +1,16 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { CLI_NAME } from "./branding"; + +export function inferenceSetRequiredFlagsFailureLines(command: string, usageSuffix = ""): string[] { + const displayCommand = `${CLI_NAME} ${command}`; + return [ + ` ${displayCommand} requires --provider and --model.`, + "", + ` Run: ${displayCommand} --provider --model ${usageSuffix}`, + " NemoClaw must perform this operation so it can protect every sandbox sharing the target gateway.", + "", + ` Run '${CLI_NAME} help' for NemoClaw commands.`, + ]; +} From 6f5f2b53b81d751e4b7d33982de16c01412014f2 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 12:39:52 -0700 Subject: [PATCH 09/31] test(inference): exercise incomplete registry route Signed-off-by: Aaron Erickson --- src/lib/inference/gateway-route-compatibility.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/inference/gateway-route-compatibility.test.ts b/src/lib/inference/gateway-route-compatibility.test.ts index 76d667fb6f3..14354869b22 100644 --- a/src/lib/inference/gateway-route-compatibility.test.ts +++ b/src/lib/inference/gateway-route-compatibility.test.ts @@ -237,9 +237,9 @@ describe("shared gateway inference route compatibility", () => { it("skips registry rows without a complete provider and model (#6315)", () => { expect( - check(route("anthropic-prod", "claude-new"), [ + check(route("nvidia-prod", "nvidia/model-a"), [ sandbox("empty", { provider: null, model: null }), - sandbox("provider-only", { provider: "nvidia-prod", model: null }), + sandbox("provider-only", { model: null }), ]), ).toEqual({ ok: true }); }); From 4b882a32bd8ff51e6f2fd017490b531ed7defdce Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 12:42:11 -0700 Subject: [PATCH 10/31] refactor(onboard): keep entrypoint net neutral Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index c8df5cac2b1..8c4d59cabc3 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -3799,7 +3799,6 @@ function getSetupNimDeps(): SetupNimDeps { } const setupNim = setupNimFlow.createSetupNim(getSetupNimDeps()); - // ── Step 4: Inference provider ─────────────────────────────────── function getSetupInferenceDeps(): SetupInferenceDeps { @@ -4488,7 +4487,6 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { sandboxGpuConfig, gpuPassthrough, }; - const [providerInferencePhase, sandboxPhase] = createCoreOnboardFlowPhases({ gatewayName: GATEWAY_NAME, From 2a32eb7a22b0472c0daf2747cb2861165bcb2beb Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 12:57:25 -0700 Subject: [PATCH 11/31] test(connect): expect gateway-scoped route repair Signed-off-by: Aaron Erickson --- .../route-swap-repair.test.ts | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/test/sandbox-connect-inference/route-swap-repair.test.ts b/test/sandbox-connect-inference/route-swap-repair.test.ts index f874ad0900b..e38dc87eaea 100644 --- a/test/sandbox-connect-inference/route-swap-repair.test.ts +++ b/test/sandbox-connect-inference/route-swap-repair.test.ts @@ -29,9 +29,11 @@ describe("sandbox connect inference route swap (#1248)", () => { expect(result.status).toBe(0); const state = JSON.parse(fs.readFileSync(stateFile, "utf-8")); - expect(state.inferenceGetCalls).toEqual([[]]); + expect(state.inferenceGetCalls).toEqual([["-g", "nemoclaw"]]); expect(state.inferenceSetCalls.length).toBe(1); expect(state.inferenceSetCalls[0]).toEqual([ + "-g", + "nemoclaw", "--provider", "anthropic-prod", "--model", @@ -84,7 +86,17 @@ describe("sandbox connect inference route swap (#1248)", () => { const curlCalls = state.curlCalls as string[][]; const curlEnvs = state.curlEnvs as Record[]; expect(state.inferenceSetCalls).toEqual([ - ["--provider", "ollama-local", "--model", "qwen3:0.6b", "--no-verify", "--timeout", "321"], + [ + "-g", + "nemoclaw", + "--provider", + "ollama-local", + "--model", + "qwen3:0.6b", + "--no-verify", + "--timeout", + "321", + ], ]); if (!isHostWsl()) { expect(curlCalls.some((call) => call.join(" ").includes("127.0.0.1:11435/v1/models"))).toBe( @@ -155,7 +167,17 @@ describe("sandbox connect inference route swap (#1248)", () => { .map((call, index) => (call.join(" ").includes("host.docker.internal:11434") ? index : -1)) .filter((index) => index >= 0); expect(state.inferenceSetCalls).toEqual([ - ["--provider", "ollama-local", "--model", "qwen3:0.6b", "--no-verify", "--timeout", "180"], + [ + "-g", + "nemoclaw", + "--provider", + "ollama-local", + "--model", + "qwen3:0.6b", + "--no-verify", + "--timeout", + "180", + ], ]); expect(windowsHostIndexes.length).toBeGreaterThan(0); for (const index of windowsHostIndexes) { From 20114a850bb42e2450f99aab80cfdea8e02dba1d Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 13:31:37 -0700 Subject: [PATCH 12/31] fix(onboard): scope inference setup to target gateway Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 42 +++--- src/lib/onboard/gateway-binding.test.ts | 56 ++++++++ src/lib/onboard/gateway-binding.ts | 12 ++ src/lib/onboard/inference-route.ts | 10 +- .../onboard/machine/core-flow-phases.test.ts | 28 ++-- .../handlers/provider-inference.test.ts | 84 +++++++++--- .../machine/handlers/provider-inference.ts | 21 ++- src/lib/onboard/resume-provider-shim.ts | 5 +- .../setup-inference-gateway-scope.test.ts | 107 +++++++++++++++ src/lib/onboard/setup-inference.ts | 91 ++++++++++--- src/lib/onboard/setup-nim-flow.test.ts | 3 +- src/lib/onboard/setup-nim-flow.ts | 12 +- ...-anthropic-compatible-openai-agent.test.ts | 8 +- test/onboard-fsm-live-slices.test.ts | 96 +++++++++++-- test/onboard-inference-failure-paths.test.ts | 116 +++++++++++----- test/onboard-inference-gateway-scope.test.ts | 127 ++++++++++++++++++ ...d-remote-recreate-credential-reuse.test.ts | 18 ++- test/onboard.test.ts | 84 ++++++------ test/support/setup-inference-test-harness.ts | 33 +++-- 19 files changed, 776 insertions(+), 177 deletions(-) create mode 100644 src/lib/onboard/setup-inference-gateway-scope.test.ts create mode 100644 test/onboard-inference-gateway-scope.test.ts diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index f9872a3f471..61688f5bced 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -909,20 +909,15 @@ function persistMigratedLegacyKeys(): void { } } -function upsertProvider( - name: string, - type: string, - credentialEnv: string, - baseUrl: string | null, - env: NodeJS.ProcessEnv = {}, -) { +// biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. +function upsertProvider(name: string, type: string, credentialEnv: string, baseUrl: string | null, env: NodeJS.ProcessEnv = {}, gatewayName: string = GATEWAY_NAME) { const result = onboardProviders.upsertProvider( name, type, credentialEnv, baseUrl, env, - runOpenshell, + setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, gatewayName), ); if (result.ok && credentialEnv) { const stagedValue = stagedLegacyValues.get(credentialEnv); @@ -996,8 +991,8 @@ function upsertMessagingProviders( if (mutated) persistMigratedLegacyKeys(); return upserted; } -const providerExistsInGateway = (name: string) => - onboardProviders.providerExistsInGateway(name, runOpenshell); +// biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. +const providerExistsInGateway = (name: string, gatewayName: string = GATEWAY_NAME) => onboardProviders.providerExistsInGateway(name, setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, gatewayName)); const { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility } = inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); @@ -3170,7 +3165,7 @@ type SetupNimSelectionState = type SetupNimSelectionResult = "selected" | "retry-selection"; // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. -type RemoteProviderSelectionArgs = { selected: ProviderChoice; requestedModel: string | null; recoveredFromSandbox: boolean; recoveredModel: string | null; sandboxName: string | null }; +type RemoteProviderSelectionArgs = { selected: ProviderChoice; requestedModel: string | null; recoveredFromSandbox: boolean; recoveredModel: string | null; sandboxName: string | null; gatewayName: string | null }; async function handleVllmSelection( state: SetupNimSelectionState, @@ -3560,7 +3555,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, provider: state.provider, helpUrl: REMOTE_PROVIDER_CONFIG.build.helpUrl, recoveredFromSandbox, - providerExistsInGateway, + providerExistsInGateway: (name) => providerExistsInGateway(name, args.gatewayName ?? GATEWAY_NAME), }); state.skipHostInferenceSmoke = reuseGatewayCredential; state.reuseGatewayCredentialWithoutLocalKey = reuseGatewayCredential; @@ -3620,7 +3615,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. recoveredProviderReuse.resolveRecoveredProviderCredentialReuse( { selected, remoteConfig, state, selectedCredentialEnv, recoveredFromSandbox, selectedModel: defaultModel, sandboxName, recoveredRegistryRoute }, - { resolveProviderCredential, readRecordedInferenceRoute, readRecordedProviderEndpoints, readGatewayProviderMetadata: (provider) => onboardProviders.readGatewayProviderMetadata(provider, runOpenshell), note }, + { resolveProviderCredential, readRecordedInferenceRoute, readRecordedProviderEndpoints, readGatewayProviderMetadata: (provider) => onboardProviders.readGatewayProviderMetadata(provider, runOpenshell, args.gatewayName ?? GATEWAY_NAME), note }, ); } else { const credentialResult = await ensureNamedCredential( @@ -4067,7 +4062,7 @@ async function preflightAuthoritativeRebuildTarget( ensureOpenshellForOnboard((code) => fail(`OpenShell component preflight exited with code ${String(code)}`), ), - inferenceRouteReady: isInferenceRouteReady, + inferenceRouteReady: (p, m) => isInferenceRouteReady(authoritativeGateway.name, p, m), captureForwardList: () => runCaptureOpenshell(["forward", "list"], { ignoreError: true }), checkPort: (port) => checkPortAvailable(port), }, @@ -4456,9 +4451,13 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { sandboxGpuConfig, gpuPassthrough, }; + // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. + const coreGatewayName = gatewayBinding.resolveCoreOnboardGatewayName({ authoritativeGatewayName: authoritativeGateway?.name, currentGatewayName: GATEWAY_NAME, resume, sandbox: sandboxName ? registry.getSandbox(sandboxName) : null }); + // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. + const runCoreGatewayOpenshell = setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, coreGatewayName); const [providerInferencePhase, sandboxPhase] = createCoreOnboardFlowPhases({ - gatewayName: GATEWAY_NAME, + gatewayName: coreGatewayName, forceProviderSelection: forceProviderSelectionForAgentChange, authoritativeResumeConfig: opts.authoritativeResumeConfig === true, env: process.env, @@ -4470,8 +4469,8 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { providerDeps: { checkGatewayRouteCompatibility, normalizeHermesAuthMethod, - setupNim: (gpu, sandboxName, agent, recoverProvider) => - setupNim(gpu, sandboxName, agent, recoverProvider, opts.rebuildRegistryInferenceRoute), + setupNim: (g, s, a, recover, gateway) => + setupNim(g, s, a, recover, opts.rebuildRegistryInferenceRoute, gateway), setupInference, startRecordedStep, recordStepComplete, @@ -4493,9 +4492,10 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { isInferenceRouteReady, isRoutedInferenceProvider, reconcileModelRouter, - reupsertRoutedProvider: (p, url, ce) => { + reupsertRoutedProvider: (gatewayName, p, url, ce) => { const r = routedInference.upsertRoutedProvider(p, url, ce, { - upsertProvider, + // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. + upsertProvider: setupInferenceFactory.bindGatewayUpsertProvider(upsertProvider, gatewayName), hydrateCredentialEnv, }); return { @@ -4654,8 +4654,8 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { loadSession: onboardSession.loadSession, getActiveSandbox: (name) => registry.getSandbox(name), mergePolicyMessagingChannels, - verifyCompatibleEndpointSandboxSmoke: (options) => - verifyCompatibleEndpointSandboxSmoke({ ...options, runOpenshell, redact }), + // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. + verifyCompatibleEndpointSandboxSmoke: (options) => verifyCompatibleEndpointSandboxSmoke({ ...options, runOpenshell: runCoreGatewayOpenshell, redact }), preparePolicyPresetResumeSelection: (name, options) => preparePolicyPresetResumeSelection({ policies }, name, options), arePolicyPresetsApplied, diff --git a/src/lib/onboard/gateway-binding.test.ts b/src/lib/onboard/gateway-binding.test.ts index 97aae82bb80..606cedf7dd5 100644 --- a/src/lib/onboard/gateway-binding.test.ts +++ b/src/lib/onboard/gateway-binding.test.ts @@ -19,6 +19,7 @@ import { BASE_GATEWAY_NAME, BASE_GATEWAY_STATE_DIR_NAME, createDynamicGatewayRuntimeHelpers, + resolveCoreOnboardGatewayName, resolveGatewayCompatContainerName, resolveGatewayName, resolveGatewayPortFromName, @@ -238,6 +239,61 @@ describe("resolveSandboxGatewayName", () => { }); }); +describe("resolveCoreOnboardGatewayName", () => { + const currentGatewayName = "nemoclaw"; + + it("prefers the authoritative rebuild handoff when the registry row is gone", () => { + expect( + resolveCoreOnboardGatewayName({ + authoritativeGatewayName: "nemoclaw-9090", + currentGatewayName, + resume: true, + sandbox: null, + }), + ).toBe("nemoclaw-9090"); + }); + + it("uses the registered sandbox binding for an ordinary resume", () => { + expect( + resolveCoreOnboardGatewayName({ + currentGatewayName, + resume: true, + sandbox: { gatewayName: "nemoclaw-9090", gatewayPort: 9090 }, + }), + ).toBe("nemoclaw-9090"); + }); + + it("keeps the requested gateway for fresh or pre-registration flows", () => { + expect( + resolveCoreOnboardGatewayName({ + currentGatewayName: "nemoclaw-9191", + resume: false, + sandbox: { gatewayPort: 9090 }, + }), + ).toBe("nemoclaw-9191"); + expect( + resolveCoreOnboardGatewayName({ + currentGatewayName: "nemoclaw-9191", + resume: true, + sandbox: null, + }), + ).toBe("nemoclaw-9191"); + }); + + it("uses the default for legacy rows and rejects invalid persisted bindings", () => { + expect(resolveCoreOnboardGatewayName({ currentGatewayName, resume: true, sandbox: {} })).toBe( + BASE_GATEWAY_NAME, + ); + expect(() => + resolveCoreOnboardGatewayName({ + currentGatewayName, + resume: true, + sandbox: { gatewayName: "../other" }, + }), + ).toThrow(/Invalid persisted sandbox gateway binding/); + }); +}); + describe("docker-driver compat container is gateway-port scoped (#4422)", () => { function withTempState( fn: (paths: { gatewayBin: string; sandboxBin: string; stateDir: string }) => T, diff --git a/src/lib/onboard/gateway-binding.ts b/src/lib/onboard/gateway-binding.ts index 9d247985bff..9e43c352f1b 100644 --- a/src/lib/onboard/gateway-binding.ts +++ b/src/lib/onboard/gateway-binding.ts @@ -141,6 +141,18 @@ export function resolveSandboxGatewayName( throw new Error(`Invalid persisted sandbox gateway binding (${detail.join(", ")})`); } +/** Resolve the core onboarding target without overriding an authoritative rebuild handoff. */ +export function resolveCoreOnboardGatewayName(options: { + authoritativeGatewayName?: string | null; + currentGatewayName: string; + resume: boolean; + sandbox: SandboxGatewayBinding | null | undefined; +}): string { + if (options.authoritativeGatewayName) return options.authoritativeGatewayName; + if (!options.resume || !options.sandbox) return options.currentGatewayName; + return resolveSandboxGatewayName(options.sandbox); +} + /** * Resolve the Docker-driver gateway state directory leaf name for a gateway * port. The state dir holds the gateway pid file and runtime marker, so a diff --git a/src/lib/onboard/inference-route.ts b/src/lib/onboard/inference-route.ts index 14535c3a449..3445b9dcf54 100644 --- a/src/lib/onboard/inference-route.ts +++ b/src/lib/onboard/inference-route.ts @@ -11,17 +11,19 @@ import { listSandboxes } from "../state/registry"; type RunCaptureOpenshell = (args: string[], options?: { ignoreError?: boolean }) => string | null; export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpenshell) { - function verifyInferenceRoute(_provider: string, _model: string): void { - const output = runCaptureOpenshell(["inference", "get"], { ignoreError: true }); + function verifyInferenceRoute(gatewayName: string, _provider: string, _model: string): void { + const output = runCaptureOpenshell(["inference", "get", "-g", gatewayName], { + ignoreError: true, + }); if (!output || /Gateway inference:\s*[\r\n]+\s*Not configured/i.test(output)) { console.error(" OpenShell inference route was not configured."); process.exit(1); } } - function isInferenceRouteReady(provider: string, model: string): boolean { + function isInferenceRouteReady(gatewayName: string, provider: string, model: string): boolean { const live = parseGatewayInference( - runCaptureOpenshell(["inference", "get"], { ignoreError: true }), + runCaptureOpenshell(["inference", "get", "-g", gatewayName], { ignoreError: true }), ); return Boolean(live && live.provider === provider && live.model === model); } diff --git a/src/lib/onboard/machine/core-flow-phases.test.ts b/src/lib/onboard/machine/core-flow-phases.test.ts index 10ec6316a52..ca570471037 100644 --- a/src/lib/onboard/machine/core-flow-phases.test.ts +++ b/src/lib/onboard/machine/core-flow-phases.test.ts @@ -105,10 +105,16 @@ function createPhases( ), toSessionUpdates: (updates) => updates as SessionUpdates, skippedStepMessage: vi.fn(), - ensureResumeProviderReady: vi.fn(async () => ({ - forceInferenceSetup: false, - credentialEnv: null, - })), + ensureResumeProviderReady: vi.fn( + async ( + _gatewayName: string, + _provider: string | null | undefined, + _credentialEnv: string | null | undefined, + ) => ({ + forceInferenceSetup: false, + credentialEnv: null, + }), + ), isResumeProviderSurfaceReady: vi.fn(() => true), recordStateSkipped: vi.fn(async () => createSession()), recordRepairEvent: vi.fn(async () => createSession()), @@ -119,10 +125,13 @@ function createPhases( isNonInteractive: () => true, getOpenshellBinary: () => "openshell", needsBedrockRuntimeAdapter: () => false, - isInferenceRouteReady: () => false, + isInferenceRouteReady: (_gatewayName, _provider, _model) => false, isRoutedInferenceProvider: () => false, reconcileModelRouter: vi.fn(async () => undefined), - reupsertRoutedProvider: () => ({ ok: true, endpointUrl: "https://example.test/v1" }), + reupsertRoutedProvider: (_gatewayName, _provider, _endpointUrl, _credentialEnv) => ({ + ok: true, + endpointUrl: "https://example.test/v1", + }), registryUpdateSandbox: vi.fn(), promptValidatedSandboxName: vi.fn(async () => "my-sandbox"), assessHost: () => ({ memoryGb: 64 }), @@ -263,6 +272,7 @@ describe("core onboard flow phases", () => { "my-sandbox", { name: "openclaw" }, false, + "nemoclaw", ); }); @@ -270,11 +280,11 @@ describe("core onboard flow phases", () => { const setupInference = vi.fn(async () => ({ ok: true as const })); const [providerPhase, sandboxPhase] = createPhases({ providerDeps: { - ensureResumeProviderReady: vi.fn(async () => ({ + ensureResumeProviderReady: vi.fn(async (_gatewayName, _provider, _credentialEnv) => ({ forceInferenceSetup: false, credentialEnv: "HERMES_API_KEY", })), - isInferenceRouteReady: () => true, + isInferenceRouteReady: (_gatewayName, _provider, _model) => true, setupInference, }, }); @@ -309,7 +319,7 @@ describe("core onboard flow phases", () => { "HERMES_API_KEY", "api_key", ["nous-web"], - { allowToolsIncompatible: false }, + { gatewayName: "nemoclaw", allowToolsIncompatible: false }, ); expect(result.context.hermesToolGateways).toEqual(["nous-web"]); diff --git a/src/lib/onboard/machine/handlers/provider-inference.test.ts b/src/lib/onboard/machine/handlers/provider-inference.test.ts index 320477e5767..d9b9d36356b 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.test.ts @@ -46,7 +46,11 @@ function createDeps( complete: vi.fn(async () => createSession()), skipped: vi.fn(), recoverProvider: vi.fn( - async (_provider: string | null | undefined, credentialEnv: string | null | undefined) => ({ + async ( + _gatewayName: string, + _provider: string | null | undefined, + credentialEnv: string | null | undefined, + ) => ({ forceInferenceSetup: false, credentialEnv: credentialEnv ?? null, }), @@ -56,10 +60,15 @@ function createDeps( repairEvent: vi.fn(async () => createSession()), hydrate: vi.fn(), repair: vi.fn(), - routeReady: vi.fn(() => false), + routeReady: vi.fn((_gatewayName: string, _provider: string, _model: string) => false), reconcileRouter: vi.fn(async () => undefined), reupsertRoutedProvider: vi.fn( - (_provider: string, endpointUrl: string | null, _credentialEnv: string | null) => ({ + ( + _gatewayName: string, + _provider: string, + endpointUrl: string | null, + _credentialEnv: string | null, + ) => ({ ok: true as const, endpointUrl: "http://host.openshell.internal:4000/v1", }), @@ -164,7 +173,7 @@ describe("handleProviderInferenceState", () => { const result = await handleProviderInferenceState(baseOptions(deps)); expect(calls.startStep).toHaveBeenNthCalledWith(1, "provider_selection"); - expect(calls.setupNim).toHaveBeenCalledWith({ type: "nvidia" }, null, null, true); + expect(calls.setupNim).toHaveBeenCalledWith({ type: "nvidia" }, null, null, true, "nemoclaw"); expect(calls.promptName).toHaveBeenCalledWith(null); expect(calls.log).toHaveBeenCalledWith("summary:nvidia-prod/nvidia/test/my-assistant"); expect(calls.startStep).toHaveBeenNthCalledWith(2, "inference", { @@ -179,7 +188,11 @@ describe("handleProviderInferenceState", () => { "NVIDIA_INFERENCE_API_KEY", null, [], - { allowToolsIncompatible: false, preferredInferenceApi: "openai-responses" }, + { + gatewayName: "nemoclaw", + allowToolsIncompatible: false, + preferredInferenceApi: "openai-responses", + }, ); expect(calls.deleteEnv).toHaveBeenCalledWith("NVIDIA_INFERENCE_API_KEY"); expect(result).toMatchObject({ @@ -243,7 +256,11 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_ANTHROPIC_API_KEY", null, [], - { allowToolsIncompatible: false, preferredInferenceApi: "openai-completions" }, + { + gatewayName: "nemoclaw", + allowToolsIncompatible: false, + preferredInferenceApi: "openai-completions", + }, ); expect(result.preferredInferenceApi).toBe("openai-completions"); }); @@ -281,7 +298,11 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_ANTHROPIC_API_KEY", null, [], - { allowToolsIncompatible: false, preferredInferenceApi: "openai-completions" }, + { + gatewayName: "nemoclaw", + allowToolsIncompatible: false, + preferredInferenceApi: "openai-completions", + }, ); expect(calls.complete).toHaveBeenCalledWith( "inference", @@ -333,7 +354,11 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_ANTHROPIC_API_KEY", null, [], - { allowToolsIncompatible: false, preferredInferenceApi: "openai-completions" }, + { + gatewayName: "nemoclaw-9090", + allowToolsIncompatible: false, + preferredInferenceApi: "openai-completions", + }, ); }); @@ -412,7 +437,13 @@ describe("handleProviderInferenceState", () => { sandboxName: "dcode-station", }); - expect(calls.setupNim).toHaveBeenCalledWith({ type: "nvidia" }, "dcode-station", null, false); + expect(calls.setupNim).toHaveBeenCalledWith( + { type: "nvidia" }, + "dcode-station", + null, + false, + "nemoclaw", + ); }); it("does not use resume shortcuts when fresh is also set", async () => { @@ -429,7 +460,13 @@ describe("handleProviderInferenceState", () => { expect(calls.recoverProvider).not.toHaveBeenCalled(); expect(calls.skipped).not.toHaveBeenCalledWith("provider_selection", expect.anything()); - expect(calls.setupNim).toHaveBeenCalledWith({ type: "nvidia" }, "dcode-station", null, false); + expect(calls.setupNim).toHaveBeenCalledWith( + { type: "nvidia" }, + "dcode-station", + null, + false, + "nemoclaw", + ); expect(calls.setupInference).toHaveBeenCalled(); }); @@ -451,7 +488,11 @@ describe("handleProviderInferenceState", () => { }); expect(calls.setupNim).not.toHaveBeenCalled(); - expect(calls.recoverProvider).toHaveBeenCalledWith("compatible-endpoint", "COMPATIBLE_API_KEY"); + expect(calls.recoverProvider).toHaveBeenCalledWith( + "nemoclaw", + "compatible-endpoint", + "COMPATIBLE_API_KEY", + ); expect(calls.complete).toHaveBeenCalledWith( "provider_selection", expect.objectContaining({ @@ -534,7 +575,7 @@ describe("handleProviderInferenceState", () => { expect(calls.setupNim).not.toHaveBeenCalled(); expect(calls.setupInference).not.toHaveBeenCalled(); - expect(calls.recoverProvider).toHaveBeenCalledWith("ollama-local", null); + expect(calls.recoverProvider).toHaveBeenCalledWith("nemoclaw", "ollama-local", null); expect(calls.skipped).toHaveBeenCalledWith("provider_selection", "ollama-local / llama3.1"); expect(calls.recordSkip).toHaveBeenCalledWith("provider_selection", { reason: "resume", @@ -595,7 +636,10 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_ANTHROPIC_API_KEY", null, [], - expect.objectContaining({ preferredInferenceApi: "openai-completions" }), + expect.objectContaining({ + gatewayName: "nemoclaw", + preferredInferenceApi: "openai-completions", + }), ); // The coerced value is persisted only after the setup succeeded, with the // inference step record — never with a pre-setup provider_selection write @@ -737,7 +781,7 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_API_KEY", null, [], - { allowToolsIncompatible: false }, + { gatewayName: "nemoclaw", allowToolsIncompatible: false }, ); }); @@ -774,7 +818,7 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_API_KEY", null, [], - { allowToolsIncompatible: false }, + { gatewayName: "nemoclaw", allowToolsIncompatible: false }, ); expect(calls.log).toHaveBeenCalledWith( " [resume] Refreshing compatible-endpoint inference route for messaging.", @@ -847,6 +891,7 @@ describe("handleProviderInferenceState", () => { null, [], { + gatewayName: "nemoclaw", allowToolsIncompatible: false, skipHostInferenceSmoke: true, reuseGatewayCredentialWithoutLocalKey: true, @@ -940,7 +985,7 @@ describe("handleProviderInferenceState", () => { "COMPATIBLE_API_KEY", null, [], - { allowToolsIncompatible: false }, + { gatewayName: "nemoclaw", allowToolsIncompatible: false }, ); expect(calls.log).toHaveBeenCalledWith( " [resume] Refreshing compatible-endpoint inference route for messaging.", @@ -1012,6 +1057,7 @@ describe("handleProviderInferenceState", () => { expect(calls.reconcileRouter).toHaveBeenCalledOnce(); expect(calls.reupsertRoutedProvider).toHaveBeenCalledWith( + "nemoclaw", "nvidia-router", "http://localhost:4000/v1", "NVIDIA_INFERENCE_API_KEY", @@ -1129,7 +1175,11 @@ describe("handleProviderInferenceState", () => { null, null, [], - { allowToolsIncompatible: true, preferredInferenceApi: "openai-responses" }, + { + gatewayName: "nemoclaw", + allowToolsIncompatible: true, + preferredInferenceApi: "openai-responses", + }, ); }); }); diff --git a/src/lib/onboard/machine/handlers/provider-inference.ts b/src/lib/onboard/machine/handlers/provider-inference.ts index f56d4908326..c2aed2c440d 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.ts @@ -12,6 +12,7 @@ import { assertProviderInferenceRouteCompatible } from "./provider-inference-rou export type ProviderInferenceRetry = { retry: "selection" } | { ok: true; retry?: undefined }; export interface ProviderInferenceSetupOptions { + gatewayName?: string; allowToolsIncompatible?: boolean; skipHostInferenceSmoke?: boolean; reuseGatewayCredentialWithoutLocalKey?: boolean; @@ -77,6 +78,7 @@ export interface ProviderInferenceStateOptions { sandboxName: string | null, agent: Agent, allowRecordedProviderRecovery?: boolean, + gatewayName?: string, ): Promise; setupInference( sandboxName: string | null, @@ -96,6 +98,7 @@ export interface ProviderInferenceStateOptions { toSessionUpdates(updates: Record): SessionUpdates; skippedStepMessage(stepName: string, detail?: string | null): void; ensureResumeProviderReady( + gatewayName: string, provider: string | null | undefined, credentialEnv: string | null | undefined, ): Promise<{ forceInferenceSetup: boolean; credentialEnv: string | null }>; @@ -128,10 +131,11 @@ export interface ProviderInferenceStateOptions { isNonInteractive(): boolean; getOpenshellBinary(): string; needsBedrockRuntimeAdapter(provider: string, endpointUrl: string | null): boolean; - isInferenceRouteReady(provider: string, model: string): boolean; + isInferenceRouteReady(gatewayName: string, provider: string, model: string): boolean; isRoutedInferenceProvider(provider: string): boolean; reconcileModelRouter(): Promise; reupsertRoutedProvider( + gatewayName: string, provider: string, endpointUrl: string | null, credentialEnv: string | null, @@ -296,7 +300,7 @@ export async function handleProviderInferenceState({ endpointUrl, preferredInferenceApi, }); - const recovery = await deps.ensureResumeProviderReady(provider, credentialEnv); + const recovery = await deps.ensureResumeProviderReady(gatewayName, provider, credentialEnv); forceInferenceSetup = recovery.forceInferenceSetup; credentialEnv = recovery.credentialEnv; // Rebuild may be resuming a legacy session whose step marker was never @@ -393,7 +397,7 @@ export async function handleProviderInferenceState({ const selection = await withProviderSelectionTrace( sandboxName, (agent as { name?: string } | null)?.name, - () => deps.setupNim(gpu, sandboxName, agent, !fresh), + () => deps.setupNim(gpu, sandboxName, agent, !fresh, gatewayName), ); model = selection.model; provider = selection.provider; @@ -466,7 +470,7 @@ export async function handleProviderInferenceState({ !forceProviderSelection && !forceInferenceSetup && effectiveResume && - deps.isInferenceRouteReady(provider, model); + deps.isInferenceRouteReady(gatewayName, provider, model); if (resumeInference) { if (provider === constants.hermesProviderName) { let inferenceResult: ProviderInferenceRetry; @@ -474,6 +478,7 @@ export async function handleProviderInferenceState({ if (!sandboxName) sandboxName = await deps.promptValidatedSandboxName(agent); const confirmedSandboxName = sandboxName; const inferenceOptions = { + gatewayName, allowToolsIncompatible, ...(skipHostInferenceSmoke ? { skipHostInferenceSmoke } : {}), ...(reuseGatewayCredentialWithoutLocalKey @@ -536,7 +541,12 @@ export async function handleProviderInferenceState({ // #4564: re-upsert the gateway provider with the sandbox-facing // endpoint so a stale localhost base URL recorded by an earlier run is // repaired on resume instead of surviving and breaking inference.local. - const reupserted = deps.reupsertRoutedProvider(provider, endpointUrl, credentialEnv); + const reupserted = deps.reupsertRoutedProvider( + gatewayName, + provider, + endpointUrl, + credentialEnv, + ); if (!reupserted.ok) { deps.error( ` ${reupserted.message ?? "Failed to update the routed inference provider."}`, @@ -598,6 +608,7 @@ export async function handleProviderInferenceState({ } const inferenceOptions = { + gatewayName, allowToolsIncompatible, ...(skipHostInferenceSmoke ? { skipHostInferenceSmoke } : {}), ...(reuseGatewayCredentialWithoutLocalKey ? { reuseGatewayCredentialWithoutLocalKey } : {}), diff --git a/src/lib/onboard/resume-provider-shim.ts b/src/lib/onboard/resume-provider-shim.ts index 1dd7f02c356..c3ef1337dd1 100644 --- a/src/lib/onboard/resume-provider-shim.ts +++ b/src/lib/onboard/resume-provider-shim.ts @@ -31,7 +31,7 @@ const onboardProviders = require("./providers") as { // and its `module.exports.resumeProviderShimDeps` is populated. type OnboardLazy = { isNonInteractive: ResumeProviderRecoveryDeps["isNonInteractive"]; - providerExistsInGateway: ResumeProviderRecoveryDeps["providerExistsInGateway"]; + providerExistsInGateway(name: string, gatewayName: string): boolean; resumeProviderShimDeps: { isRoutedInferenceProvider: ResumeProviderRecoveryDeps["isRoutedInferenceProvider"]; replaceNamedCredential: ResumeProviderRecoveryDeps["replaceNamedCredential"]; @@ -39,6 +39,7 @@ type OnboardLazy = { }; export async function ensureResumeProviderReady( + gatewayName: string, provider: string | null | undefined, credentialEnv: string | null | undefined, ): Promise { @@ -47,7 +48,7 @@ export async function ensureResumeProviderReady( remoteProviderConfig: onboardProviders.REMOTE_PROVIDER_CONFIG, defaultRouteCredentialEnv: DEFAULT_ROUTE_CREDENTIAL_ENV, isRoutedInferenceProvider: o.resumeProviderShimDeps.isRoutedInferenceProvider, - providerExistsInGateway: o.providerExistsInGateway, + providerExistsInGateway: (name) => o.providerExistsInGateway(name, gatewayName), hydrateCredentialEnv, getProviderLabel: onboardProviders.getProviderLabel, isNonInteractive: o.isNonInteractive, diff --git a/src/lib/onboard/setup-inference-gateway-scope.test.ts b/src/lib/onboard/setup-inference-gateway-scope.test.ts new file mode 100644 index 00000000000..982eb48863e --- /dev/null +++ b/src/lib/onboard/setup-inference-gateway-scope.test.ts @@ -0,0 +1,107 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it, vi } from "vitest"; +import { createInferenceRouteHelpers } from "./inference-route"; +import { + bindGatewayUpsertProvider, + createGatewayScopedOpenshellRunner, + scopeGatewayOpenshellArgs, +} from "./setup-inference"; + +const GATEWAY = "nemoclaw-9090"; + +describe("gateway-scoped onboarding OpenShell commands", () => { + it.each([ + [ + ["provider", "get", "openai-api"], + ["provider", "get", "-g", GATEWAY, "openai-api"], + ], + [ + ["inference", "set", "--provider", "openai-api", "--model", "gpt-test"], + ["inference", "set", "-g", GATEWAY, "--provider", "openai-api", "--model", "gpt-test"], + ], + [ + ["sandbox", "provider", "detach", "alpha", "openai-api"], + ["sandbox", "provider", "detach", "-g", GATEWAY, "alpha", "openai-api"], + ], + ])("adds the target gateway to %j", (input, expected) => { + expect(scopeGatewayOpenshellArgs(input, GATEWAY)).toEqual(expected); + }); + + it("targets sandbox execution at the same gateway", () => { + expect( + scopeGatewayOpenshellArgs(["sandbox", "exec", "-n", "alpha", "--", "true"], GATEWAY), + ).toEqual(["sandbox", "exec", "-g", GATEWAY, "-n", "alpha", "--", "true"]); + }); + + it.each([ + ["-g", GATEWAY], + ["--gateway", GATEWAY], + [`--gateway=${GATEWAY}`], + ])("accepts an identical existing target: %j", (...gatewayArgs) => { + const command = ["provider", "list", ...gatewayArgs]; + expect(scopeGatewayOpenshellArgs(command, GATEWAY)).toEqual(command); + }); + + it("rejects a conflicting, duplicate, missing, or selection-based target", () => { + expect(() => + scopeGatewayOpenshellArgs(["provider", "get", "-g", "nemoclaw", "openai-api"], GATEWAY), + ).toThrow(/instead of 'nemoclaw-9090'/); + expect(() => + scopeGatewayOpenshellArgs(["inference", "get", "-g", GATEWAY, "--gateway", GATEWAY], GATEWAY), + ).toThrow(/multiple gateway targets/); + expect(() => scopeGatewayOpenshellArgs(["provider", "list", "-g"], GATEWAY)).toThrow( + /instead of 'nemoclaw-9090'/, + ); + expect(() => scopeGatewayOpenshellArgs(["gateway", "select", GATEWAY], GATEWAY)).toThrow( + /must not change the selected gateway/, + ); + }); + + it("scopes every command sent through the runner without mutating the caller argv", () => { + const run = vi.fn((_args: string[], _options?: { ignoreError?: boolean }) => ({ status: 0 })); + const scoped = createGatewayScopedOpenshellRunner(run, GATEWAY); + const command = ["provider", "delete", "openai-api"]; + scoped(command, { ignoreError: true }); + expect(command).toEqual(["provider", "delete", "openai-api"]); + expect(run).toHaveBeenCalledWith(["provider", "delete", "-g", GATEWAY, "openai-api"], { + ignoreError: true, + }); + }); + + it("keeps an omitted provider env separate from the bound gateway", () => { + const upsert = vi.fn(() => ({ ok: true })); + bindGatewayUpsertProvider(upsert, GATEWAY)("openai-api", "openai", "OPENAI_API_KEY", null); + expect(upsert).toHaveBeenCalledWith( + "openai-api", + "openai", + "OPENAI_API_KEY", + null, + undefined, + GATEWAY, + ); + }); +}); + +describe("gateway-scoped inference route readers", () => { + const output = [ + "Gateway inference:", + " Provider: openai-api", + " Model: gpt-test", + " Version: 1", + ].join("\n"); + + it("uses the explicit gateway for verification and readiness", () => { + const capture = vi.fn(() => output); + const route = createInferenceRouteHelpers(capture); + + route.verifyInferenceRoute(GATEWAY, "openai-api", "gpt-test"); + expect(route.isInferenceRouteReady(GATEWAY, "openai-api", "gpt-test")).toBe(true); + expect(route.isInferenceRouteReady(GATEWAY, "openai-api", "other")).toBe(false); + expect(capture).toHaveBeenCalledTimes(3); + for (const call of capture.mock.calls) { + expect(call).toEqual([["inference", "get", "-g", GATEWAY], { ignoreError: true }]); + } + }); +}); diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index 5ee5c464f14..8d43cff4327 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -21,20 +21,13 @@ import type { ProviderInferenceSetupOptions } from "./machine/handlers/provider- type ProviderBranchDeps = Pick< CommonDeps, - | "upsertProvider" - | "verifyInferenceRoute" - | "verifyOnboardInferenceSmoke" - | "isNonInteractive" - | "exitProcess" - | "error" - | "log" + "verifyOnboardInferenceSmoke" | "isNonInteractive" | "exitProcess" | "error" | "log" > & Pick< HermesDeps, | "lookup" | "hermesProviderAuth" | "getHermesToolGatewayBroker" - | "providerExistsInGateway" | "normalizeHermesAuthMethod" | "resolveHermesNousApiKey" | "checkHermesProviderStoreReachable" @@ -73,6 +66,16 @@ export type SetupInferenceDeps = ProviderBranchDeps & { step: (current: number, total: number, label: string) => void; getGatewayName: () => string; runOpenshell: import("./openshell-cli").OpenshellCliHelpers["runOpenshell"]; + upsertProvider: ( + name: string, + type: string, + credentialEnv: string, + baseUrl: string | null, + env: NodeJS.ProcessEnv | undefined, + gatewayName: string, + ) => ReturnType; + verifyInferenceRoute: (gatewayName: string, provider: string, model: string) => void; + providerExistsInGateway: (name: string, gatewayName: string) => boolean; run: typeof import("../runner").run; updateSandbox: CommonDeps["registry"]["updateSandbox"]; localInferenceTimeoutSecs: number; @@ -90,11 +93,59 @@ export type SetupInferenceDeps = ProviderBranchDeps & { exitProcess: (code: number) => never; }; -function resolveLocalInferenceRouteApplier(deps: SetupInferenceDeps) { +export function scopeGatewayOpenshellArgs(args: string[], gatewayName: string): string[] { + if (!gatewayName) throw new Error("OpenShell gateway name is required."); + if (args[0] === "gateway" && args[1] === "select") { + throw new Error("Gateway-scoped OpenShell operations must not change the selected gateway."); + } + const providerCommand = args[0] === "inference" || args[0] === "provider"; + const sandboxCommand = args[0] === "sandbox" && typeof args[1] === "string"; + const sandboxProviderCommand = sandboxCommand && args[1] === "provider"; + if (!providerCommand && !sandboxCommand) return [...args]; + const gatewayFlagIndex = sandboxProviderCommand ? 3 : 2; + const gatewayTargets = args.flatMap((value, index) => { + if (index < gatewayFlagIndex) return []; + if (value === "-g" || value === "--gateway") return [args[index + 1] ?? ""]; + return value.startsWith("--gateway=") ? [value.slice("--gateway=".length)] : []; + }); + if (gatewayTargets.length > 1) { + throw new Error("OpenShell command contains multiple gateway targets."); + } + const existingGatewayName = gatewayTargets[0]; + if (existingGatewayName !== undefined) { + if (existingGatewayName !== gatewayName) { + throw new Error( + `OpenShell command targets gateway '${existingGatewayName}' instead of '${gatewayName}'.`, + ); + } + return [...args]; + } + return [...args.slice(0, gatewayFlagIndex), "-g", gatewayName, ...args.slice(gatewayFlagIndex)]; +} + +export function createGatewayScopedOpenshellRunner( + runOpenshell: (args: string[], ...rest: Rest) => Result, + gatewayName: string, +): (args: string[], ...rest: Rest) => Result { + return (args, ...rest) => runOpenshell(scopeGatewayOpenshellArgs(args, gatewayName), ...rest); +} + +export function bindGatewayUpsertProvider( + upsertProvider: SetupInferenceDeps["upsertProvider"], + gatewayName: string, +): CommonDeps["upsertProvider"] { + return (name, type, credentialEnv, baseUrl, env) => + upsertProvider(name, type, credentialEnv, baseUrl, env, gatewayName); +} + +function resolveLocalInferenceRouteApplier( + deps: SetupInferenceDeps, + runOpenshell: SetupInferenceDeps["runOpenshell"], +) { return ( deps.applyLocalInferenceRoute ?? createLocalInferenceRouteApplier({ - runOpenshell: deps.runOpenshell, + runOpenshell, isNonInteractive: deps.isNonInteractive, promptValidationRecovery: deps.promptValidationRecovery, classifyApplyFailure: deps.classifyApplyFailure, @@ -134,7 +185,7 @@ export function createSetupInference( hermesToolGateways: string[] = [], options: ProviderInferenceSetupOptions = {}, ): Promise { - const gatewayName = deps.getGatewayName(); + const gatewayName = options.gatewayName ?? deps.getGatewayName(); const compatibility = deps.checkGatewayRouteCompatibility({ gatewayName, sandboxName, @@ -150,12 +201,13 @@ export function createSetupInference( return deps.exitProcess(1); } deps.step(4, 8, "Setting up inference provider"); - deps.runOpenshell(["gateway", "select", gatewayName], { ignoreError: true }); + const runGatewayOpenshell = createGatewayScopedOpenshellRunner(deps.runOpenshell, gatewayName); const commonDeps = { - runOpenshell: deps.runOpenshell, - upsertProvider: deps.upsertProvider, - verifyInferenceRoute: deps.verifyInferenceRoute, + runOpenshell: runGatewayOpenshell, + upsertProvider: bindGatewayUpsertProvider(deps.upsertProvider, gatewayName), + verifyInferenceRoute: (selectedProvider: string, selectedModel: string) => + deps.verifyInferenceRoute(gatewayName, selectedProvider, selectedModel), verifyOnboardInferenceSmoke: deps.verifyOnboardInferenceSmoke, isNonInteractive: deps.isNonInteractive, registry: { updateSandbox: deps.updateSandbox }, @@ -179,7 +231,8 @@ export function createSetupInference( ...commonDeps, hermesProviderAuth: deps.hermesProviderAuth, getHermesToolGatewayBroker: deps.getHermesToolGatewayBroker, - providerExistsInGateway: deps.providerExistsInGateway, + providerExistsInGateway: (name: string) => + deps.providerExistsInGateway(name, gatewayName), normalizeHermesAuthMethod: deps.normalizeHermesAuthMethod, resolveHermesNousApiKey: deps.resolveHermesNousApiKey, checkHermesProviderStoreReachable: deps.checkHermesProviderStoreReachable, @@ -229,7 +282,7 @@ export function createSetupInference( validateLocalProvider: deps.validateLocalProvider, getLocalProviderHealthCheck: deps.getLocalProviderHealthCheck, getLocalProviderBaseUrl: deps.getLocalProviderBaseUrl, - applyLocalInferenceRoute: resolveLocalInferenceRouteApplier(deps), + applyLocalInferenceRoute: resolveLocalInferenceRouteApplier(deps, runGatewayOpenshell), run: deps.run, VLLM_LOCAL_CREDENTIAL_ENV: deps.vllmLocalCredentialEnv, }, @@ -242,7 +295,7 @@ export function createSetupInference( ...commonDeps, validateLocalProvider: deps.validateLocalProvider, getLocalProviderBaseUrl: deps.getLocalProviderBaseUrl, - applyLocalInferenceRoute: resolveLocalInferenceRouteApplier(deps), + applyLocalInferenceRoute: resolveLocalInferenceRouteApplier(deps, runGatewayOpenshell), getOllamaWarmupCommand: deps.getOllamaWarmupCommand, run: deps.run, shouldFrontOllamaWithProxy: deps.shouldFrontOllamaWithProxy, @@ -272,7 +325,7 @@ export function createSetupInference( deps.exitProcess(1); } - deps.verifyInferenceRoute(provider, model); + commonDeps.verifyInferenceRoute(provider, model); if (options.skipHostInferenceSmoke === true) deps.log(" Reusing existing gateway credential; skipping host inference smoke."); else deps.verifyOnboardInferenceSmoke({ provider, model, endpointUrl, credentialEnv }); diff --git a/src/lib/onboard/setup-nim-flow.test.ts b/src/lib/onboard/setup-nim-flow.test.ts index 31693ad5a61..2d305f98d3d 100644 --- a/src/lib/onboard/setup-nim-flow.test.ts +++ b/src/lib/onboard/setup-nim-flow.test.ts @@ -167,13 +167,14 @@ describe("createSetupNim", () => { }), ); - const result = await setupNim(null); + const result = await setupNim(null, null, null, true, null, "nemoclaw-9090"); expect(step).toHaveBeenCalledWith(3, 8, "Configuring inference provider"); expect(log).toHaveBeenCalledWith(" Detected local inference option: Ollama"); expect(prompt).toHaveBeenCalledOnce(); expect(prompt).toHaveBeenCalledWith(" Choose [1]: "); expect(handleRemoteProviderSelection).toHaveBeenCalledOnce(); + expect(handleRemoteProviderSelection.mock.calls[0]?.[0].gatewayName).toBe("nemoclaw-9090"); expect(maybePromptForInferenceInputCapability).toHaveBeenCalledWith( "nvidia/nemotron-3-super-120b-a12b", ); diff --git a/src/lib/onboard/setup-nim-flow.ts b/src/lib/onboard/setup-nim-flow.ts index 490ef8a9922..dd06fd2875f 100644 --- a/src/lib/onboard/setup-nim-flow.ts +++ b/src/lib/onboard/setup-nim-flow.ts @@ -27,6 +27,7 @@ export interface SetupNimRemoteProviderConfigEntry { } export interface SetupNimRemoteSelectionArgs { + gatewayName: string | null; selected: ProviderMenuChoice; requestedModel: string | null; recoveredFromSandbox: boolean; @@ -40,6 +41,7 @@ export type SetupNim = ( agent?: AgentDefinition | null, recoverProvider?: boolean, rebuildRegistryInferenceRoute?: RebuildRouteHandoff | null, + gatewayName?: string | null, ) => Promise; export interface SetupNimFlowDeps { @@ -163,6 +165,7 @@ export function createSetupNim( agent: AgentDefinition | null = null, recoverProvider = true, rebuildRegistryInferenceRoute: RebuildRouteHandoff | null = null, + gatewayName: string | null = null, ): Promise { deps.step(3, 8, "Configuring inference provider"); @@ -319,7 +322,14 @@ export function createSetupNim( nvidiaFeaturedModels, }; const result = await deps.handleRemoteProviderSelection( - { selected, requestedModel, recoveredFromSandbox, recoveredModel, sandboxName }, + { + selected, + requestedModel, + recoveredFromSandbox, + recoveredModel, + sandboxName, + gatewayName, + }, state, recoveredRegistryRoute, ); diff --git a/test/onboard-anthropic-compatible-openai-agent.test.ts b/test/onboard-anthropic-compatible-openai-agent.test.ts index 2380e5d6c33..1867c735252 100644 --- a/test/onboard-anthropic-compatible-openai-agent.test.ts +++ b/test/onboard-anthropic-compatible-openai-agent.test.ts @@ -81,7 +81,7 @@ describe("compatible-anthropic-endpoint registration for OpenAI-only agents (#62 expect(createCommand?.command).toContain(`--credential ${CREDENTIAL_ENV}`); expect( harness.commands.some(({ command }) => - command.includes(`inference set --provider ${PROVIDER} --model ${MODEL}`), + command.includes(`inference set -g nemoclaw --provider ${PROVIDER} --model ${MODEL}`), ), ).toBe(true); }); @@ -101,9 +101,9 @@ describe("compatible-anthropic-endpoint registration for OpenAI-only agents (#62 }); // Plain delete succeeded (default status 0) — no force-detach recovery. - expect(harness.commands.some(({ command }) => command === `provider delete ${PROVIDER}`)).toBe( - true, - ); + expect( + harness.commands.some(({ command }) => command === `provider delete -g nemoclaw ${PROVIDER}`), + ).toBe(true); expect(deleteGatewayProvider).not.toHaveBeenCalled(); const createCommand = harness.commands.find(({ command }) => command.startsWith("provider create"), diff --git a/test/onboard-fsm-live-slices.test.ts b/test/onboard-fsm-live-slices.test.ts index 8ed4fd1a17b..47f34a59b7a 100644 --- a/test/onboard-fsm-live-slices.test.ts +++ b/test/onboard-fsm-live-slices.test.ts @@ -12,7 +12,12 @@ const repoRoot = path.join(import.meta.dirname, ".."); const probeTimeoutMs = 10_000; type SliceName = "initial" | "core" | "final"; -type ProbeMode = "fresh" | "resume-initial" | "ahead-core"; +type ProbeMode = + | "fresh" + | "resume-initial" + | "resume-core-gateway" + | "authoritative-core-gateway" + | "ahead-core"; interface ProbeOptions { slice: SliceName; @@ -72,6 +77,11 @@ const requiredDistArtifacts: readonly DistArtifact[] = [ "provider-inference.ts", ), }, + { + label: "gateway handler", + sourcePath: path.join(repoRoot, "src", "lib", "onboard", "machine", "handlers", "gateway.ts"), + distPath: path.join(repoRoot, "src", "lib", "onboard", "machine", "handlers", "gateway.ts"), + }, ]; function distArtifactStatus(): { ok: true } | { ok: false; reason: string } { @@ -161,6 +171,10 @@ function runSliceProbe(options: ProbeOptions) { const providerHandlerPath = JSON.stringify( path.join(repoRoot, "src", "lib", "onboard", "machine", "handlers", "provider-inference.ts"), ); + const gatewayHandlerPath = JSON.stringify( + path.join(repoRoot, "src", "lib", "onboard", "machine", "handlers", "gateway.ts"), + ); + const registryPath = JSON.stringify(path.join(repoRoot, "src", "lib", "state", "registry.ts")); fs.writeFileSync( scriptPath, @@ -171,6 +185,8 @@ const { advanceTo, branchTo } = require(${resultPath}); const onboardSession = require(${sessionPath}); const preflightHandlers = require(${preflightHandlerPath}); const providerHandlers = require(${providerHandlerPath}); +const gatewayHandlers = require(${gatewayHandlerPath}); +const registry = require(${registryPath}); const called = []; const sentinel = new Error("slice-called"); @@ -179,12 +195,18 @@ function machine(state, revision = 1) { } function seedResumeSession(state) { - onboardSession.saveSession(onboardSession.createSession({ + const session = onboardSession.createSession({ mode: "non-interactive", sandboxName: "fsm-sandbox", + provider: "openai-api", + model: "gpt-test", machine: machine(state), metadata: { gatewayName: "nemoclaw", fromDockerfile: null }, - })); + }); + for (const step of ["preflight", "gateway", "provider_selection", "sandbox"]) { + session.steps[step].status = "complete"; + } + onboardSession.saveSession(session); } function baseContext(context, overrides = {}) { @@ -211,7 +233,21 @@ function baseContext(context, overrides = {}) { }; } -preflightHandlers.handlePreflightState = async () => { +preflightHandlers.handlePreflightState = async (options) => { + if (scenario.mode === "resume-core-gateway" || scenario.mode === "authoritative-core-gateway") { + return { + gpu: null, + sandboxGpuConfig: { sandboxGpuEnabled: false, mode: "0" }, + resumePreflight: true, + resumeHasResolvedGpuIntent: true, + requestedGpuPassthrough: false, + gpuPassthrough: false, + effectiveSandboxGpuFlag: "disable", + effectiveSandboxGpuDevice: null, + session: options.session, + stateResult: advanceTo("gateway", { metadata: { state: "preflight" } }), + }; + } if (scenario.mode !== "resume-initial") { throw new Error("unexpected preflight compatibility handler"); } @@ -219,11 +255,24 @@ preflightHandlers.handlePreflightState = async () => { throw sentinel; }; -providerHandlers.handleProviderInferenceState = async () => { - if (scenario.mode !== "ahead-core") { +gatewayHandlers.handleGatewayState = async (options) => { + if (scenario.mode !== "resume-core-gateway" && scenario.mode !== "authoritative-core-gateway") { + throw new Error("unexpected gateway compatibility handler"); + } + return { + gatewayReuseState: "healthy", + session: options.session, + stateResult: advanceTo("provider_selection", { metadata: { state: "gateway" } }), + }; +}; + +providerHandlers.handleProviderInferenceState = async (options) => { + if (!["ahead-core", "resume-core-gateway", "authoritative-core-gateway"].includes(scenario.mode)) { throw new Error("unexpected provider compatibility handler"); } - called.push("provider-compat"); + called.push( + scenario.mode === "ahead-core" ? "provider-compat" : "provider-compat:" + options.gatewayName, + ); throw sentinel; }; @@ -268,6 +317,18 @@ flowSlices.runFinalOnboardFlowSequence = async ({ context }) => { if (scenario.mode === "resume-initial") { seedResumeSession("preflight"); } +if (scenario.mode === "resume-core-gateway" || scenario.mode === "authoritative-core-gateway") { + seedResumeSession("inference"); +} +if (scenario.mode === "resume-core-gateway") { + registry.registerSandbox({ + name: "fsm-sandbox", + provider: "openai-api", + model: "gpt-test", + gatewayName: "nemoclaw-9090", + gatewayPort: 9090, + }); +} const { onboard } = require(${onboardPath}); @@ -279,7 +340,14 @@ const { onboard } = require(${onboardPath}); acceptThirdPartySoftware: true, noGpu: true, sandboxName: "fsm-sandbox", - resume: scenario.mode === "resume-initial", + resume: scenario.mode === "resume-initial" || scenario.mode.includes("core-gateway"), + ...(scenario.mode === "authoritative-core-gateway" + ? { + authoritativeResumeConfig: true, + targetGatewayName: "nemoclaw-9090", + targetGatewayPort: 9090, + } + : {}), }); throw new Error("expected slice sentinel"); } catch (error) { @@ -355,4 +423,16 @@ describe("live onboard FSM slice boundaries", () => { "provider-compat", ]); }); + + it("routes ordinary resume through the sandbox's recorded gateway", () => { + assert.deepEqual(runSliceProbe({ slice: "core", mode: "resume-core-gateway" }), [ + "provider-compat:nemoclaw-9090", + ]); + }); + + it("keeps an authoritative rebuild gateway after the registry row is removed", () => { + assert.deepEqual(runSliceProbe({ slice: "core", mode: "authoritative-core-gateway" }), [ + "provider-compat:nemoclaw-9090", + ]); + }); }); diff --git a/test/onboard-inference-failure-paths.test.ts b/test/onboard-inference-failure-paths.test.ts index 6dba8bedce6..ed2c7ef631e 100644 --- a/test/onboard-inference-failure-paths.test.ts +++ b/test/onboard-inference-failure-paths.test.ts @@ -71,7 +71,7 @@ function stubMissingBedrockAuth(): void { function expectNoPostFailureSideEffects( harness: DirectSetupInferenceHarness, - expectedCommands = ["gateway select nemoclaw"], + expectedCommands: string[] = [], ): void { expect(harness.commands.map(({ command }) => command)).toEqual(expectedCommands); expect(harness.verifyInferenceRoute).not.toHaveBeenCalled(); @@ -79,6 +79,21 @@ function expectNoPostFailureSideEffects( expect(harness.updateSandbox).not.toHaveBeenCalled(); } +function expectNemoclawScopedRunner( + harness: DirectSetupInferenceHarness, + runOpenshell: SetupInferenceDeps["runOpenshell"], +): void { + expect(runOpenshell).not.toBe(harness.runOpenshell); + const commandCount = harness.commands.length; + runOpenshell(["provider", "list"], { ignoreError: true }); + expect(harness.commands.at(-1)).toEqual({ + command: "provider list -g nemoclaw", + env: undefined, + ignoreError: true, + }); + harness.commands.splice(commandCount); +} + describe("setupInference dependency failures", () => { afterEach(() => { vi.unstubAllEnvs(); @@ -177,6 +192,7 @@ describe("setupInference dependency failures", () => { "OPENAI_API_KEY", expect.any(String), { OPENAI_API_KEY: "openai-secret" }, + "nemoclaw", ); expect(promptValidationRecovery).not.toHaveBeenCalled(); expect(exitProcess).toHaveBeenCalledOnce(); @@ -223,8 +239,7 @@ describe("setupInference dependency failures", () => { expect(harness.errors.join("\n")).toContain("route failed"); expect(harness.errors.join("\n")).not.toContain(NVIDIA_REDACTION_CANARY); expectNoPostFailureSideEffects(harness, [ - "gateway select nemoclaw", - "inference set --no-verify --provider openai-api --model gpt-test", + "inference set -g nemoclaw --no-verify --provider openai-api --model gpt-test", ]); }); @@ -348,6 +363,9 @@ describe("setupInference dependency failures", () => { expect(harness.errors).toEqual([ " container cannot reach Ollama", " Diagnostic: proxy probe failed", + ...(process.platform === "darwin" + ? [" On macOS, local inference also depends on OpenShell host routing support."] + : []), ]); expectNoPostFailureSideEffects(harness); }); @@ -384,13 +402,17 @@ describe("setupInference dependency failures", () => { it("exits through injected Hermes boundaries when provider storage is unavailable", async () => { const exitProcess = createInjectedExit(); - const isHermesProviderRegistered = vi.fn(() => true); + const isHermesProviderRegistered = vi.fn( + (_runOpenshell: SetupInferenceDeps["runOpenshell"]) => true, + ); const ensureHermesProviderApiKeyCredentials = vi.fn(async () => ({})); const ensureHermesProviderOAuthCredentials = vi.fn(async () => ({})); - const checkHermesProviderStoreReachable = vi.fn(() => ({ - ok: false, - message: "provider store unavailable", - })); + const checkHermesProviderStoreReachable = vi.fn( + (_runOpenshell: SetupInferenceDeps["runOpenshell"]) => ({ + ok: false, + message: "provider store unavailable", + }), + ); const harness = createDirectSetupInferenceHarness({ overrides: { isNonInteractive: () => true, @@ -409,7 +431,8 @@ describe("setupInference dependency failures", () => { harness.setupInference("test-box", "moonshotai/kimi-k2.6", "hermes-provider"), ).rejects.toThrow("EXIT_CALLED:1"); - expect(checkHermesProviderStoreReachable).toHaveBeenCalledWith(harness.runOpenshell); + const runGatewayOpenshell = checkHermesProviderStoreReachable.mock.calls[0][0]; + expectNemoclawScopedRunner(harness, runGatewayOpenshell); expect(isHermesProviderRegistered).not.toHaveBeenCalled(); expect(ensureHermesProviderApiKeyCredentials).not.toHaveBeenCalled(); expect(ensureHermesProviderOAuthCredentials).not.toHaveBeenCalled(); @@ -425,14 +448,18 @@ describe("setupInference dependency failures", () => { it("exits through injected boundaries when Hermes API-key preparation throws", async () => { const exitProcess = createInjectedExit(); - const isHermesProviderRegistered = vi.fn(() => false); + const isHermesProviderRegistered = vi.fn( + (_runOpenshell: SetupInferenceDeps["runOpenshell"]) => false, + ); const ensureHermesProviderApiKeyCredentials = vi.fn(async () => { throw new Error("API-key preparation failed"); }); const ensureHermesProviderOAuthCredentials = vi.fn(async () => ({})); const providerExistsInGateway = vi.fn(() => true); const resolveHermesNousApiKey = vi.fn(() => "nous-secret"); - const checkHermesProviderStoreReachable = vi.fn(() => ({ ok: true })); + const checkHermesProviderStoreReachable = vi.fn( + (_runOpenshell: SetupInferenceDeps["runOpenshell"]) => ({ ok: true }), + ); const harness = createDirectSetupInferenceHarness({ overrides: { isNonInteractive: () => true, @@ -461,13 +488,14 @@ describe("setupInference dependency failures", () => { ), ).rejects.toThrow("EXIT_CALLED:1"); - expect(checkHermesProviderStoreReachable).toHaveBeenCalledWith(harness.runOpenshell); - expect(isHermesProviderRegistered).toHaveBeenCalledWith(harness.runOpenshell); + const runGatewayOpenshell = checkHermesProviderStoreReachable.mock.calls[0][0]; + expectNemoclawScopedRunner(harness, runGatewayOpenshell); + expect(isHermesProviderRegistered).toHaveBeenCalledWith(runGatewayOpenshell); expect(providerExistsInGateway).not.toHaveBeenCalled(); expect(ensureHermesProviderApiKeyCredentials).toHaveBeenCalledOnce(); expect(ensureHermesProviderApiKeyCredentials).toHaveBeenCalledWith("test-box", { apiKey: "nous-secret", - runOpenshell: harness.runOpenshell, + runOpenshell: runGatewayOpenshell, baseUrl: undefined, }); expect(ensureHermesProviderOAuthCredentials).not.toHaveBeenCalled(); @@ -481,14 +509,18 @@ describe("setupInference dependency failures", () => { it("exits through injected boundaries when Hermes OAuth preparation throws", async () => { const exitProcess = createInjectedExit(); - const isHermesProviderRegistered = vi.fn(() => false); + const isHermesProviderRegistered = vi.fn( + (_runOpenshell: SetupInferenceDeps["runOpenshell"]) => false, + ); const ensureHermesProviderApiKeyCredentials = vi.fn(async () => ({})); const ensureHermesProviderOAuthCredentials = vi.fn(async () => { throw new Error("OAuth preparation failed"); }); const providerExistsInGateway = vi.fn(() => true); const resolveHermesNousApiKey = vi.fn(() => "unused-key"); - const checkHermesProviderStoreReachable = vi.fn(() => ({ ok: true })); + const checkHermesProviderStoreReachable = vi.fn( + (_runOpenshell: SetupInferenceDeps["runOpenshell"]) => ({ ok: true }), + ); const harness = createDirectSetupInferenceHarness({ overrides: { isNonInteractive: () => true, @@ -517,15 +549,16 @@ describe("setupInference dependency failures", () => { ), ).rejects.toThrow("EXIT_CALLED:1"); - expect(checkHermesProviderStoreReachable).toHaveBeenCalledWith(harness.runOpenshell); - expect(isHermesProviderRegistered).toHaveBeenCalledWith(harness.runOpenshell); + const runGatewayOpenshell = checkHermesProviderStoreReachable.mock.calls[0][0]; + expectNemoclawScopedRunner(harness, runGatewayOpenshell); + expect(isHermesProviderRegistered).toHaveBeenCalledWith(runGatewayOpenshell); expect(providerExistsInGateway).not.toHaveBeenCalled(); expect(resolveHermesNousApiKey).not.toHaveBeenCalled(); expect(ensureHermesProviderApiKeyCredentials).not.toHaveBeenCalled(); expect(ensureHermesProviderOAuthCredentials).toHaveBeenCalledOnce(); expect(ensureHermesProviderOAuthCredentials).toHaveBeenCalledWith("test-box", { allowInteractiveLogin: false, - runOpenshell: harness.runOpenshell, + runOpenshell: runGatewayOpenshell, baseUrl: undefined, toolGatewayPresets: [], }); @@ -779,8 +812,7 @@ describe("setupInference dependency failures", () => { " Bedrock Runtime adapter ready: region us-east-1, sandbox route http://host.openshell.internal:11436/v1, host log /tmp/bedrock-adapter.log", ]); expectNoPostFailureSideEffects(harness, [ - "gateway select nemoclaw", - `inference set --no-verify --provider compatible-anthropic-endpoint --model ${BEDROCK_MODEL} --timeout 180`, + `inference set -g nemoclaw --no-verify --provider compatible-anthropic-endpoint --model ${BEDROCK_MODEL} --timeout 180`, ]); }); @@ -823,8 +855,7 @@ describe("setupInference dependency failures", () => { " Bedrock Runtime adapter ready: region us-east-1, sandbox route http://host.openshell.internal:11436/v1, host log /tmp/bedrock-adapter.log", ]); expectNoPostFailureSideEffects(harness, [ - "gateway select nemoclaw", - `inference set --no-verify --provider compatible-anthropic-endpoint --model ${BEDROCK_MODEL} --timeout 180`, + `inference set -g nemoclaw --no-verify --provider compatible-anthropic-endpoint --model ${BEDROCK_MODEL} --timeout 180`, ]); }); @@ -887,7 +918,9 @@ describe("setupInference dependency failures", () => { const reconcileModelRouter = vi.fn(async () => {}); const upsertProvider = vi.fn(() => ({ ok: true })); const hydrateCredentialEnv = vi.fn(() => "unused-secret"); - const upsertRoutedProvider = vi.fn(() => ({ + const upsertRoutedProvider = vi.fn< + SetupInferenceDeps["routedInference"]["upsertRoutedProvider"] + >(() => ({ ok: false, result: { status: 29, message: "routed provider registration rejected" }, })); @@ -918,10 +951,30 @@ describe("setupInference dependency failures", () => { "nvidia-router", "http://host.openshell.internal:4000/v1", "NVIDIA_INFERENCE_API_KEY", - { upsertProvider, hydrateCredentialEnv }, + { + upsertProvider: expect.any(Function), + hydrateCredentialEnv, + }, ); + const routedUpsertProvider = upsertRoutedProvider.mock.calls[0][3].upsertProvider; + expect(routedUpsertProvider).not.toBe(upsertProvider); expect(upsertProvider).not.toHaveBeenCalled(); expect(hydrateCredentialEnv).not.toHaveBeenCalled(); + routedUpsertProvider( + "nvidia-router", + "openai", + "NVIDIA_INFERENCE_API_KEY", + "http://host.openshell.internal:4000/v1", + { NVIDIA_INFERENCE_API_KEY: "test-secret" }, + ); + expect(upsertProvider).toHaveBeenCalledWith( + "nvidia-router", + "openai", + "NVIDIA_INFERENCE_API_KEY", + "http://host.openshell.internal:4000/v1", + { NVIDIA_INFERENCE_API_KEY: "test-secret" }, + "nemoclaw", + ); expect(exitProcess).toHaveBeenCalledOnce(); expect(exitProcess).toHaveBeenCalledWith(29); expect(harness.errors).toEqual([" routed provider registration rejected"]); @@ -970,8 +1023,7 @@ describe("setupInference dependency failures", () => { expect(harness.errors.join("\n")).toContain("routed apply failed"); expect(harness.errors.join("\n")).not.toContain(NVIDIA_REDACTION_CANARY); expectNoPostFailureSideEffects(harness, [ - "gateway select nemoclaw", - "inference set --no-verify --provider nvidia-router --model router/model", + "inference set -g nemoclaw --no-verify --provider nvidia-router --model router/model", ]); }); @@ -1001,15 +1053,19 @@ describe("setupInference dependency failures", () => { expect(reconcileModelRouter).toHaveBeenCalledOnce(); expect(upsertRoutedProvider).toHaveBeenCalledOnce(); expect(harness.commands).toEqual([ - { command: "gateway select nemoclaw", ignoreError: true, env: undefined }, { - command: "inference set --no-verify --provider nvidia-router --model router/model", + command: + "inference set -g nemoclaw --no-verify --provider nvidia-router --model router/model", ignoreError: true, env: undefined, }, ]); expect(harness.verifyInferenceRoute).toHaveBeenCalledOnce(); - expect(harness.verifyInferenceRoute).toHaveBeenCalledWith("nvidia-router", "router/model"); + expect(harness.verifyInferenceRoute).toHaveBeenCalledWith( + "nemoclaw", + "nvidia-router", + "router/model", + ); expect(harness.verifyOnboardInferenceSmoke).toHaveBeenCalledOnce(); expect(harness.verifyOnboardInferenceSmoke).toHaveBeenCalledWith({ provider: "nvidia-router", diff --git a/test/onboard-inference-gateway-scope.test.ts b/test/onboard-inference-gateway-scope.test.ts new file mode 100644 index 00000000000..d45ddbff111 --- /dev/null +++ b/test/onboard-inference-gateway-scope.test.ts @@ -0,0 +1,127 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it, vi } from "vitest"; +import type { SetupInference, SetupInferenceDeps } from "../src/lib/onboard/setup-inference.js"; +import { + createDirectCommandRouter, + createDirectSetupInferenceHarnessFactory, + withProcessEnv, +} from "./support/setup-inference-test-harness.js"; + +const onboard = require("../src/lib/onboard") as { + createSetupInference: (overrides?: Partial) => SetupInference; +}; + +const createHarness = createDirectSetupInferenceHarnessFactory(onboard.createSetupInference); +const GATEWAY = "nemoclaw-9090"; + +function expectCommandsTargetOnly(commands: Array<{ command: string }>): void { + expect(commands.some(({ command }) => command.startsWith("gateway select"))).toBe(false); + const gatewayStateCommands = commands.filter( + ({ command }) => /^(provider|inference) /.test(command) || /^sandbox provider /.test(command), + ); + for (const { command } of gatewayStateCommands) { + expect(command.match(/(?:^| )-g nemoclaw-9090(?: |$)/g)).toHaveLength(1); + } +} + +describe("onboarding inference gateway scope", () => { + it("targets a non-default gateway for provider creation, route apply, and verification", async () => { + await withProcessEnv({ OPENAI_API_KEY: "sk-TEST-NOT-A-REAL-VALUE" }, async () => { + const harness = createHarness({ + runOpenshell: (args) => + args.slice(0, 2).join(" ") === "provider get" ? { status: 1 } : undefined, + }); + + await expect( + harness.setupInference( + "test-box", + "gpt-test", + "openai-api", + "https://api.openai.com/v1", + "OPENAI_API_KEY", + null, + [], + { gatewayName: GATEWAY }, + ), + ).resolves.toEqual({ ok: true }); + + expect(harness.commands.map(({ command }) => command)).toEqual([ + `provider get -g ${GATEWAY} openai-api`, + `provider create -g ${GATEWAY} --name openai-api --type openai --credential OPENAI_API_KEY --config OPENAI_BASE_URL=https://api.openai.com/v1`, + `inference set -g ${GATEWAY} --no-verify --provider openai-api --model gpt-test`, + ]); + expect(harness.verifyInferenceRoute).toHaveBeenCalledWith(GATEWAY, "openai-api", "gpt-test"); + expectCommandsTargetOnly(harness.commands); + }); + }); + + it("keeps compatible-endpoint replacement and detach recovery on the target gateway", async () => { + await withProcessEnv( + { COMPATIBLE_ANTHROPIC_API_KEY: "sk-ant-TEST-NOT-A-REAL-VALUE" }, + async () => { + const commandRouter = createDirectCommandRouter([ + { + name: "provider-get", + matches: (command) => command.startsWith(`provider get -g ${GATEWAY}`), + results: [ + { + status: 0, + stdout: [ + "Name: compatible-anthropic-endpoint", + "Type: anthropic", + "Credential keys: COMPATIBLE_ANTHROPIC_API_KEY", + "Config keys: ANTHROPIC_BASE_URL", + ].join("\n"), + }, + { status: 1 }, + ], + }, + { + name: "provider-delete", + matches: (command) => command.startsWith(`provider delete -g ${GATEWAY}`), + results: [ + { + status: 1, + stderr: + "provider 'compatible-anthropic-endpoint' is attached to sandbox(es): test-box", + }, + { + status: 1, + stderr: + "provider 'compatible-anthropic-endpoint' is attached to sandbox(es): test-box", + }, + { status: 0 }, + ], + }, + ]); + const harness = createHarness({ + runOpenshell: commandRouter.runOpenshell, + overrides: { + probeOpenAiLikeEndpoint: vi.fn(() => ({ ok: true })), + }, + }); + + await expect( + harness.setupInference( + "test-box", + "claude-test", + "compatible-anthropic-endpoint", + "https://example.test", + "COMPATIBLE_ANTHROPIC_API_KEY", + null, + [], + { gatewayName: GATEWAY, preferredInferenceApi: "openai-completions" }, + ), + ).resolves.toEqual({ ok: true }); + + expect(commandRouter.callCount("provider-delete")).toBe(3); + expect(harness.commands.map(({ command }) => command)).toContain( + `sandbox provider detach -g ${GATEWAY} test-box compatible-anthropic-endpoint`, + ); + expectCommandsTargetOnly(harness.commands); + }, + ); + }); +}); diff --git a/test/onboard-remote-recreate-credential-reuse.test.ts b/test/onboard-remote-recreate-credential-reuse.test.ts index dc1db52ae11..50c4b25a1d8 100644 --- a/test/onboard-remote-recreate-credential-reuse.test.ts +++ b/test/onboard-remote-recreate-credential-reuse.test.ts @@ -45,7 +45,7 @@ Gateway inference: Version: 1 EOF fi -if [ "$1" = "provider" ] && [ "$2" = "get" ] && [ "$3" = "compatible-endpoint" ]; then +if [ "$1" = "provider" ] && [ "$2" = "get" ] && [ "$3" = "-g" ] && [ "$5" = "compatible-endpoint" ]; then cat <<'EOF' Provider: @@ -176,9 +176,15 @@ const { setupNim, setupInference } = require(${onboardPath}); `only exact loopback discovery probes may run without a local credential: ${curlLog}`, ); const openshellLog = fs.readFileSync(openshellLogPath, "utf8"); - assert.match(openshellLog, /provider get compatible-endpoint/); - assert.match(openshellLog, /inference set --no-verify --provider compatible-endpoint/); - assert.ok(!openshellLog.includes("provider update compatible-endpoint"), openshellLog); + assert.match(openshellLog, /provider get -g nemoclaw compatible-endpoint/); + assert.match( + openshellLog, + /inference set -g nemoclaw --no-verify --provider compatible-endpoint/, + ); + assert.ok( + !openshellLog.includes("provider update -g nemoclaw compatible-endpoint"), + openshellLog, + ); assert.ok(!openshellLog.includes("OPENAI_BASE_URL="), openshellLog); assert.ok(!openshellLog.includes("--credential"), openshellLog); @@ -237,7 +243,7 @@ const { setupNim, setupInference } = require(${onboardPath}); assert.match(unauthorizedOutput, /A host credential is required to configure provider/); const unauthorizedOpenshellLog = fs.readFileSync(openshellLogPath, "utf8"); assert.ok( - !unauthorizedOpenshellLog.includes("provider update compatible-endpoint") && + !unauthorizedOpenshellLog.includes("provider update") && !unauthorizedOpenshellLog.includes("inference set"), `smoke suppression alone must not authorize gateway credential reuse: ${unauthorizedOpenshellLog}`, ); @@ -275,7 +281,7 @@ const { setupNim, setupInference } = require(${onboardPath}); ); const conflictingEndpointOpenshellLog = fs.readFileSync(openshellLogPath, "utf8"); assert.ok( - !conflictingEndpointOpenshellLog.includes("provider update compatible-endpoint") && + !conflictingEndpointOpenshellLog.includes("provider update") && !conflictingEndpointOpenshellLog.includes("inference set"), `endpoint drift must fail before provider or route mutation: ${conflictingEndpointOpenshellLog}`, ); diff --git a/test/onboard.test.ts b/test/onboard.test.ts index 51d91ca5c85..0731ed67ed8 100644 --- a/test/onboard.test.ts +++ b/test/onboard.test.ts @@ -736,11 +736,8 @@ startGateway(null).catch(() => {}); assert.deepEqual(evidence.argvContainingSecret, []); assert.deepEqual(evidence.secretBearingCommands, ["provider update"]); assert.equal(evidence.providerCommand.env.NVIDIA_INFERENCE_API_KEY, credentialValue); - assert.equal( - evidence.unscopedCommandKinds.join(","), - "gateway select,provider get,inference set", - ); - assert.deepEqual(evidence.unscopedCredentialValues, [null, null, null]); + assert.deepEqual(evidence.unscopedCommandKinds, []); + assert.deepEqual(evidence.unscopedCredentialValues, []); assert.deepEqual(evidence.unscopedCommandsContainingSecret, []); assert.deepEqual(evidence.setupCredentialValues, [credentialValue, credentialValue]); assert.equal(evidence.parentCredentialUnchanged, true); @@ -754,7 +751,7 @@ startGateway(null).catch(() => {}); async () => { const harness = createDirectSetupInferenceHarness({ runOpenshell: (args) => - args.join(" ") === "provider get hermes-provider" + args.join(" ") === "provider get -g nemoclaw hermes-provider" ? { status: 0, stdout: "Provider: hermes-provider", stderr: "" } : undefined, overrides: { isNonInteractive: () => true }, @@ -770,11 +767,14 @@ startGateway(null).catch(() => {}); ); const commands = harness.commands; - assert.equal(commands.length, 4); - assert.match(commands[0].command, /gateway select nemoclaw/); - assert.match(commands[1].command, /provider list/); - assert.match(commands[2].command, /provider get hermes-provider/); - assert.match(commands[3].command, /inference set --no-verify --provider hermes-provider/); + assert.equal(commands.length, 3); + assert.equal(commands[0].command, "provider list -g nemoclaw"); + assert.equal(commands[1].command, "provider get -g nemoclaw hermes-provider"); + assert.match( + commands[2].command, + /inference set -g nemoclaw --no-verify --provider hermes-provider/, + ); + assert.ok(!commands.some((entry) => entry.command.startsWith("gateway select"))); assert.ok(!commands.some((entry) => /provider (create|update)/.test(entry.command))); assert.ok(!commands.some((entry) => entry.env?.NOUS_API_KEY || entry.env?.OPENAI_API_KEY)); assert.ok( @@ -798,7 +798,7 @@ startGateway(null).catch(() => {}); const setupBedrockRuntimeInference = bedrockRuntimeOnboard.setupBedrockRuntimeInference; const harness = createDirectSetupInferenceHarness({ runOpenshell: (args) => - args.join(" ") === "provider get compatible-anthropic-endpoint" + args.join(" ") === "provider get -g nemoclaw compatible-anthropic-endpoint" ? { status: 1, stdout: "", stderr: "" } : undefined, overrides: { @@ -865,7 +865,7 @@ startGateway(null).catch(() => {}); ); assert.match( commands.at(-1)?.command || "", - /inference set --no-verify --provider compatible-anthropic-endpoint --model anthropic\.claude-3-5-sonnet-20240620-v1:0/, + /inference set -g nemoclaw --no-verify --provider compatible-anthropic-endpoint --model anthropic\.claude-3-5-sonnet-20240620-v1:0/, ); expect(updateSandbox).toHaveBeenCalledWith("test-box", { model: "anthropic.claude-3-5-sonnet-20240620-v1:0", @@ -1091,7 +1091,7 @@ const { onboard } = require(${onboardPath}); ); assert.ok( payload.commands.some((entry) => - /inference set --no-verify --provider hermes-provider/.test(entry.command), + /inference set -g nemoclaw --no-verify --provider hermes-provider/.test(entry.command), ), "resume should reach openshell inference set", ); @@ -1121,7 +1121,7 @@ const { onboard } = require(${onboardPath}); async () => { const harness = createDirectSetupInferenceHarness({ runOpenshell: (args) => - args.join(" ") === "provider get hermes-provider" + args.join(" ") === "provider get -g nemoclaw hermes-provider" ? { status: 0, stdout: "Provider: hermes-provider", stderr: "" } : undefined, overrides: { isNonInteractive: () => true }, @@ -1137,7 +1137,7 @@ const { onboard } = require(${onboardPath}); ); const update = harness.commands.find((entry) => - /provider update hermes-provider/.test(entry.command), + /provider update -g nemoclaw hermes-provider/.test(entry.command), ); assert.ok(update); assert.match(update.command, /--credential NOUS_API_KEY/); @@ -1148,7 +1148,7 @@ const { onboard } = require(${onboardPath}); ); assert.match( harness.commands.at(-1)?.command || "", - /inference set --no-verify --provider hermes-provider/, + /inference set -g nemoclaw --no-verify --provider hermes-provider/, ); }, ); @@ -1250,7 +1250,7 @@ const { onboard } = require(${onboardPath}); assert.doesNotMatch(providerCommand.command, /proxy-token/); assert.ok( harness.commands.some((entry) => - entry.command.includes("inference set --no-verify --provider ollama-local"), + entry.command.includes("inference set -g nemoclaw --no-verify --provider ollama-local"), ), "expected ollama-local inference route to be selected", ); @@ -1301,7 +1301,7 @@ const { onboard } = require(${onboardPath}); warn.mockRestore(); } const setCmd = harness.commands.find((entry) => - entry.command.includes("inference set --no-verify --provider ollama-local"), + entry.command.includes("inference set -g nemoclaw --no-verify --provider ollama-local"), ); assert.ok(setCmd, "expected ollama-local inference set command to be issued"); assert.equal( @@ -1346,7 +1346,7 @@ const { onboard } = require(${onboardPath}); ); const setCmd = harness.commands.find((entry) => - entry.command.includes("inference set --no-verify --provider vllm-local"), + entry.command.includes("inference set -g nemoclaw --no-verify --provider vllm-local"), ); assert.ok(setCmd, "expected vllm-local inference set command to be issued"); assert.equal( @@ -1370,7 +1370,7 @@ const { onboard } = require(${onboardPath}); fs.writeFileSync( fakeOpenshell, `#!/usr/bin/env bash -if [ "$1" = "inference" ] && [ "$2" = "get" ]; then +if [ "$1" = "inference" ] && [ "$2" = "get" ] && [ "$3" = "-g" ] && [ "$4" = "team-gateway" ]; then cat <<'EOF' Gateway inference: @@ -1391,9 +1391,9 @@ exit 1 ` const { isInferenceRouteReady } = require(${onboardPath}); console.log(JSON.stringify({ - same: isInferenceRouteReady("nvidia-prod", "nvidia/nemotron-3-super-120b-a12b"), - otherModel: isInferenceRouteReady("nvidia-prod", "nvidia/other-model"), - otherProvider: isInferenceRouteReady("openai-api", "nvidia/nemotron-3-super-120b-a12b"), + same: isInferenceRouteReady("team-gateway", "nvidia-prod", "nvidia/nemotron-3-super-120b-a12b"), + otherModel: isInferenceRouteReady("team-gateway", "nvidia-prod", "nvidia/other-model"), + otherProvider: isInferenceRouteReady("team-gateway", "openai-api", "nvidia/nemotron-3-super-120b-a12b"), })); `, ); @@ -1547,13 +1547,14 @@ console.log(JSON.stringify({ ); const commands = harness.commands; - assert.equal(commands.length, 4); - assert.match(commands[0].command, /gateway select nemoclaw/); - assert.match(commands[1].command, /provider get/); - assert.match(commands[2].command, /--type anthropic/); - assert.match(commands[2].command, /--credential ANTHROPIC_API_KEY/); - assert.doesNotMatch(commands[2].command, /sk-ant-TEST-NOT-A-REAL-VALUE/); - assert.match(commands[3].command, /--provider anthropic-prod/); + assert.equal(commands.length, 3); + assert.match(commands[0].command, /^provider get -g nemoclaw /); + assert.match(commands[1].command, /^provider create -g nemoclaw /); + assert.match(commands[1].command, /--type anthropic/); + assert.match(commands[1].command, /--credential ANTHROPIC_API_KEY/); + assert.doesNotMatch(commands[1].command, /sk-ant-TEST-NOT-A-REAL-VALUE/); + assert.match(commands[2].command, /^inference set -g nemoclaw /); + assert.match(commands[2].command, /--provider anthropic-prod/); }); }); it("updates OpenAI-compatible providers without passing an unsupported --type flag", async () => { @@ -1574,12 +1575,11 @@ console.log(JSON.stringify({ ); const commands = harness.commands; - assert.equal(commands.length, 4); - assert.match(commands[0].command, /gateway select nemoclaw/); - assert.match(commands[1].command, /provider get/); - assert.match(commands[2].command, /provider update openai-api/); - assert.doesNotMatch(commands[2].command, /--type/); - assert.match(commands[3].command, /inference set --no-verify/); + assert.equal(commands.length, 3); + assert.match(commands[0].command, /^provider get -g nemoclaw /); + assert.match(commands[1].command, /^provider update -g nemoclaw openai-api/); + assert.doesNotMatch(commands[1].command, /--type/); + assert.match(commands[2].command, /^inference set -g nemoclaw --no-verify/); }); }); it("re-prompts for credentials when openshell inference set fails with authorization errors", async () => { @@ -1703,7 +1703,7 @@ console.log(JSON.stringify({ "legacy credentials.json must survive the staging-only hydrate path", ); const providerUpdate = harness.commands.find((entry) => - entry.command.includes("provider update openai-api"), + entry.command.includes("provider update -g nemoclaw openai-api"), ); assert.ok(providerUpdate, "expected provider update command"); assert.equal(providerUpdate.env?.OPENAI_API_KEY, "sk-TEST-NOT-A-REAL-STORED-KEY"); @@ -3911,8 +3911,8 @@ const { createSandbox } = require(${onboardPath}); "OPENAI_API_KEY", ); - // gateway select + provider get + provider update + inference set - assert.equal(harness.commands.length, 4); + // provider get + provider update + inference set + assert.equal(harness.commands.length, 3); }); }); it("accepts gateway inference output that omits the Route line", async () => { @@ -3946,8 +3946,8 @@ const { createSandbox } = require(${onboardPath}); "OPENAI_API_KEY", ); - // gateway select + provider get + provider update + inference set - assert.equal(harness.commands.length, 4); + // provider get + provider update + inference set + assert.equal(harness.commands.length, 3); }); }); it("uses the sandbox-base registry in pullAndResolveBaseImageDigest (#1904)", () => { diff --git a/test/support/setup-inference-test-harness.ts b/test/support/setup-inference-test-harness.ts index 2b4030a9a06..4c474645c17 100644 --- a/test/support/setup-inference-test-harness.ts +++ b/test/support/setup-inference-test-harness.ts @@ -6,7 +6,11 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import { vi } from "vitest"; -import type { SetupInference, SetupInferenceDeps } from "../../src/lib/onboard/setup-inference.js"; +import { + createGatewayScopedOpenshellRunner, + type SetupInference, + type SetupInferenceDeps, +} from "../../src/lib/onboard/setup-inference.js"; const onboardProviderHelpers = require("../../src/lib/onboard/providers") as { upsertProvider: ( @@ -17,6 +21,7 @@ const onboardProviderHelpers = require("../../src/lib/onboard/providers") as { env: Record, runOpenshell: DirectRunOpenshell, ) => { ok: boolean; status?: number; message?: string }; + providerExistsInGateway: (name: string, runOpenshell: DirectRunOpenshell) => boolean; }; const localInferenceModule = require("../../src/lib/inference/local") as typeof import("../../src/lib/inference/local.js"); @@ -173,10 +178,16 @@ const setupCredentialBefore = process.env[credentialEnv] || null; /^provider (create|update) /.test(argv.join(" ")), ); if (!providerCommand) throw new Error("Production setupInference did not mutate a provider"); - const unscopedPatterns = [/^gateway select /, /^provider get /, /^inference set /]; - const unscopedCommands = unscopedPatterns - .map((pattern) => commands.find(({ argv }) => pattern.test(argv.join(" ")))) - .filter((command): command is ProductionOpenshellCommandRecord => command !== undefined); + const unscopedCommands = commands.filter(({ argv }) => { + if (argv[0] === "gateway" && argv[1] === "select") return true; + if (argv[0] !== "provider" && argv[0] !== "inference") return false; + return ( + !argv.some( + (arg, index) => + (arg === "-g" || arg === "--gateway") && typeof argv[index + 1] === "string", + ) && !argv.some((arg) => arg.startsWith("--gateway=")) + ); + }); const containsSecret = ({ env }: ProductionOpenshellCommandRecord) => Object.values(env).some((value) => value.includes(options.credentialValue)); const credentialEvidence = { @@ -278,18 +289,24 @@ export function createDirectSetupInferenceHarnessFactory( type: string, credentialEnv: string, baseUrl: string | null, - env: Record = {}, + env: Record | undefined, + gatewayName: string, ) => onboardProviderHelpers.upsertProvider( name, type, credentialEnv, baseUrl, - env, - runOpenshell, + env ?? {}, + createGatewayScopedOpenshellRunner(runOpenshell, gatewayName), ), verifyInferenceRoute, verifyOnboardInferenceSmoke, + providerExistsInGateway: (name: string, gatewayName: string) => + onboardProviderHelpers.providerExistsInGateway( + name, + createGatewayScopedOpenshellRunner(runOpenshell, gatewayName), + ), isNonInteractive: () => false, updateSandbox, resolveHermesNousApiKey: () => process.env.NOUS_API_KEY || null, From 1fe81bede5d9ee3ae49b0b1b9b28587d249ee8c8 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 13:44:30 -0700 Subject: [PATCH 13/31] fix(onboard): bind resume gateway before lifecycle Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 22 +++++++------- src/lib/onboard/gateway-binding.test.ts | 40 ++++++++++++------------- src/lib/onboard/gateway-binding.ts | 19 +++++++----- test/onboard-fsm-live-slices.test.ts | 3 ++ 4 files changed, 45 insertions(+), 39 deletions(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 61688f5bced..1e339802d17 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -4302,7 +4302,11 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { const recordedSandboxName = session?.steps?.sandbox?.status === "complete" ? session?.sandboxName || null : null; - + // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. + const onboardGateway = gatewayBinding.resolveCoreOnboardGatewayBinding({ authoritativeGateway, currentGateway: { name: GATEWAY_NAME, port: GATEWAY_PORT }, resume, sandbox: recordedSandboxName ? registry.getSandbox(recordedSandboxName) : null }); + // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. + ({ name: GATEWAY_NAME, port: GATEWAY_PORT } = onboardGateway); + process.env.OPENSHELL_GATEWAY = GATEWAY_NAME; console.log(""); console.log(` ${cliDisplayName()} Onboarding`); if (isNonInteractive()) note(" (non-interactive mode)"); @@ -4452,12 +4456,10 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { gpuPassthrough, }; // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. - const coreGatewayName = gatewayBinding.resolveCoreOnboardGatewayName({ authoritativeGatewayName: authoritativeGateway?.name, currentGatewayName: GATEWAY_NAME, resume, sandbox: sandboxName ? registry.getSandbox(sandboxName) : null }); - // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. - const runCoreGatewayOpenshell = setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, coreGatewayName); + const runCoreGatewayOpenshell = setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, GATEWAY_NAME); const [providerInferencePhase, sandboxPhase] = createCoreOnboardFlowPhases({ - gatewayName: coreGatewayName, + gatewayName: GATEWAY_NAME, forceProviderSelection: forceProviderSelectionForAgentChange, authoritativeResumeConfig: opts.authoritativeResumeConfig === true, env: process.env, @@ -4748,12 +4750,10 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { releaseOnboardLock(); onboardRuntimeBoundary.clear(); onboardTracing.finishOnboardTrace(onboardTrace, traceCompleted); - if (authoritativeGateway) { - GATEWAY_NAME = previousGatewayBinding.name; - GATEWAY_PORT = previousGatewayBinding.port; - if (previousOpenshellGateway === undefined) delete process.env.OPENSHELL_GATEWAY; - else process.env.OPENSHELL_GATEWAY = previousOpenshellGateway; - } + GATEWAY_NAME = previousGatewayBinding.name; + GATEWAY_PORT = previousGatewayBinding.port; + if (previousOpenshellGateway === undefined) delete process.env.OPENSHELL_GATEWAY; + else process.env.OPENSHELL_GATEWAY = previousOpenshellGateway; } } diff --git a/src/lib/onboard/gateway-binding.test.ts b/src/lib/onboard/gateway-binding.test.ts index 606cedf7dd5..129d71b7dcc 100644 --- a/src/lib/onboard/gateway-binding.test.ts +++ b/src/lib/onboard/gateway-binding.test.ts @@ -19,7 +19,7 @@ import { BASE_GATEWAY_NAME, BASE_GATEWAY_STATE_DIR_NAME, createDynamicGatewayRuntimeHelpers, - resolveCoreOnboardGatewayName, + resolveCoreOnboardGatewayBinding, resolveGatewayCompatContainerName, resolveGatewayName, resolveGatewayPortFromName, @@ -239,54 +239,54 @@ describe("resolveSandboxGatewayName", () => { }); }); -describe("resolveCoreOnboardGatewayName", () => { - const currentGatewayName = "nemoclaw"; +describe("resolveCoreOnboardGatewayBinding", () => { + const currentGateway = { name: "nemoclaw", port: DEFAULT_GATEWAY_PORT }; it("prefers the authoritative rebuild handoff when the registry row is gone", () => { expect( - resolveCoreOnboardGatewayName({ - authoritativeGatewayName: "nemoclaw-9090", - currentGatewayName, + resolveCoreOnboardGatewayBinding({ + authoritativeGateway: { name: "nemoclaw-9090", port: 9090 }, + currentGateway, resume: true, sandbox: null, }), - ).toBe("nemoclaw-9090"); + ).toEqual({ name: "nemoclaw-9090", port: 9090 }); }); it("uses the registered sandbox binding for an ordinary resume", () => { expect( - resolveCoreOnboardGatewayName({ - currentGatewayName, + resolveCoreOnboardGatewayBinding({ + currentGateway, resume: true, sandbox: { gatewayName: "nemoclaw-9090", gatewayPort: 9090 }, }), - ).toBe("nemoclaw-9090"); + ).toEqual({ name: "nemoclaw-9090", port: 9090 }); }); it("keeps the requested gateway for fresh or pre-registration flows", () => { expect( - resolveCoreOnboardGatewayName({ - currentGatewayName: "nemoclaw-9191", + resolveCoreOnboardGatewayBinding({ + currentGateway: { name: "nemoclaw-9191", port: 9191 }, resume: false, sandbox: { gatewayPort: 9090 }, }), - ).toBe("nemoclaw-9191"); + ).toEqual({ name: "nemoclaw-9191", port: 9191 }); expect( - resolveCoreOnboardGatewayName({ - currentGatewayName: "nemoclaw-9191", + resolveCoreOnboardGatewayBinding({ + currentGateway: { name: "nemoclaw-9191", port: 9191 }, resume: true, sandbox: null, }), - ).toBe("nemoclaw-9191"); + ).toEqual({ name: "nemoclaw-9191", port: 9191 }); }); it("uses the default for legacy rows and rejects invalid persisted bindings", () => { - expect(resolveCoreOnboardGatewayName({ currentGatewayName, resume: true, sandbox: {} })).toBe( - BASE_GATEWAY_NAME, + expect(resolveCoreOnboardGatewayBinding({ currentGateway, resume: true, sandbox: {} })).toEqual( + { name: BASE_GATEWAY_NAME, port: DEFAULT_GATEWAY_PORT }, ); expect(() => - resolveCoreOnboardGatewayName({ - currentGatewayName, + resolveCoreOnboardGatewayBinding({ + currentGateway, resume: true, sandbox: { gatewayName: "../other" }, }), diff --git a/src/lib/onboard/gateway-binding.ts b/src/lib/onboard/gateway-binding.ts index 9e43c352f1b..f5808b8fe77 100644 --- a/src/lib/onboard/gateway-binding.ts +++ b/src/lib/onboard/gateway-binding.ts @@ -141,16 +141,19 @@ export function resolveSandboxGatewayName( throw new Error(`Invalid persisted sandbox gateway binding (${detail.join(", ")})`); } -/** Resolve the core onboarding target without overriding an authoritative rebuild handoff. */ -export function resolveCoreOnboardGatewayName(options: { - authoritativeGatewayName?: string | null; - currentGatewayName: string; +/** Resolve one attempt-wide onboarding target without overriding an authoritative rebuild. */ +export function resolveCoreOnboardGatewayBinding(options: { + authoritativeGateway?: { name: string; port: number } | null; + currentGateway: { name: string; port: number }; resume: boolean; sandbox: SandboxGatewayBinding | null | undefined; -}): string { - if (options.authoritativeGatewayName) return options.authoritativeGatewayName; - if (!options.resume || !options.sandbox) return options.currentGatewayName; - return resolveSandboxGatewayName(options.sandbox); +}): { name: string; port: number } { + if (options.authoritativeGateway) return { ...options.authoritativeGateway }; + if (!options.resume || !options.sandbox) return { ...options.currentGateway }; + const name = resolveSandboxGatewayName(options.sandbox); + const port = resolveGatewayPortFromName(name); + if (port === null) throw new Error(`Invalid resolved onboarding gateway name: ${name}`); + return { name, port }; } /** diff --git a/test/onboard-fsm-live-slices.test.ts b/test/onboard-fsm-live-slices.test.ts index 47f34a59b7a..9edaf1f5bc6 100644 --- a/test/onboard-fsm-live-slices.test.ts +++ b/test/onboard-fsm-live-slices.test.ts @@ -259,6 +259,7 @@ gatewayHandlers.handleGatewayState = async (options) => { if (scenario.mode !== "resume-core-gateway" && scenario.mode !== "authoritative-core-gateway") { throw new Error("unexpected gateway compatibility handler"); } + called.push("gateway:" + options.gatewayName + ":" + process.env.OPENSHELL_GATEWAY); return { gatewayReuseState: "healthy", session: options.session, @@ -426,12 +427,14 @@ describe("live onboard FSM slice boundaries", () => { it("routes ordinary resume through the sandbox's recorded gateway", () => { assert.deepEqual(runSliceProbe({ slice: "core", mode: "resume-core-gateway" }), [ + "gateway:nemoclaw-9090:nemoclaw-9090", "provider-compat:nemoclaw-9090", ]); }); it("keeps an authoritative rebuild gateway after the registry row is removed", () => { assert.deepEqual(runSliceProbe({ slice: "core", mode: "authoritative-core-gateway" }), [ + "gateway:nemoclaw-9090:nemoclaw-9090", "provider-compat:nemoclaw-9090", ]); }); From 1319818bc750dfb6de82d97903ea3f9535cf0154 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 13:46:39 -0700 Subject: [PATCH 14/31] fix(inference): reject gateway endpoint overrides Signed-off-by: Aaron Erickson --- ...ence-set-gateway-route-containment.test.ts | 35 ++++++++++++- src/lib/actions/inference-set.ts | 12 +++++ .../sandbox/connect-route-containment.test.ts | 31 ++++++++++++ src/lib/actions/sandbox/connect.ts | 9 ++++ .../setup-inference-gateway-scope.test.ts | 50 +++++++++++++++++++ src/lib/onboard/setup-inference.ts | 12 ++++- src/lib/openshell-gateway-endpoint-guard.ts | 41 +++++++++++++++ 7 files changed, 188 insertions(+), 2 deletions(-) create mode 100644 src/lib/openshell-gateway-endpoint-guard.ts diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts index 417e574ec95..7ced5a66fe8 100644 --- a/src/lib/actions/inference-set-gateway-route-containment.test.ts +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -import { describe, expect, it } from "vitest"; +import { afterEach, describe, expect, it, vi } from "vitest"; import type { ConfigObject } from "../security/credential-filter"; import type { SandboxEntry } from "../state/registry"; import { runInferenceSet } from "./inference-set"; @@ -18,6 +18,39 @@ const entry = (name: string, overrides: Partial = {}): SandboxEntr }); describe("runtime shared gateway route containment", () => { + afterEach(() => { + vi.unstubAllEnvs(); + }); + + it("rejects an ambient gateway endpoint before OpenShell prep or state mutation", async () => { + vi.stubEnv("OPENSHELL_GATEWAY_ENDPOINT", "https://other.example.test"); + const deps = createDeps({ + config: {}, + entries: [entry("alpha")], + defaultSandbox: "alpha", + }); + + await expect( + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-b", sandboxName: "alpha" }, + deps, + ), + ).rejects.toMatchObject({ + message: expect.stringContaining("Unset OPENSHELL_GATEWAY_ENDPOINT"), + exitCode: 2, + }); + + expect(deps.calls.prepareRunOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.writeSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.recomputeSandboxConfigHash).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + expect(deps.calls.updateSession).not.toHaveBeenCalled(); + expect(deps.calls.appendAuditEntry).not.toHaveBeenCalled(); + }); + it("rejects a same-gateway conflict before OpenShell, config, or registry mutation (#6315)", async () => { const deps = createDeps({ config: {}, diff --git a/src/lib/actions/inference-set.ts b/src/lib/actions/inference-set.ts index 932d13595ea..68c3eef0810 100644 --- a/src/lib/actions/inference-set.ts +++ b/src/lib/actions/inference-set.ts @@ -20,6 +20,10 @@ import { parseGatewayProviderMetadata, } from "../onboard/gateway-provider-metadata"; import { ensureLocalProviderReachable } from "../onboard/local-inference-topology"; +import { + assertNoOpenShellGatewayEndpointOverride, + OpenShellGatewayEndpointOverrideError, +} from "../openshell-gateway-endpoint-guard"; import { type AgentConfigTarget, readSandboxConfig, @@ -747,6 +751,14 @@ export async function runInferenceSet( options: InferenceSetOptions, deps: InferenceSetDeps = defaultDeps(), ): Promise { + try { + assertNoOpenShellGatewayEndpointOverride(); + } catch (error) { + if (error instanceof OpenShellGatewayEndpointOverrideError) { + throw new InferenceSetError(error.message, 2); + } + throw error; + } // Resolve once before acquiring so a default-sandbox change cannot make the // protected callback mutate a different sandbox from the one whose lock we // hold. Prime the default OpenShell runner before acquiring too: its legacy diff --git a/src/lib/actions/sandbox/connect-route-containment.test.ts b/src/lib/actions/sandbox/connect-route-containment.test.ts index 1ea5da324f2..6d22fc92273 100644 --- a/src/lib/actions/sandbox/connect-route-containment.test.ts +++ b/src/lib/actions/sandbox/connect-route-containment.test.ts @@ -118,6 +118,37 @@ describe("connect route containment", () => { expect(exitSpy).toHaveBeenCalledWith(1); }); + it("exits before managed route reads or repairs when an endpoint override is ambient", async () => { + vi.stubEnv("OPENSHELL_GATEWAY_ENDPOINT", "https://other.example.test"); + const harness = createConnectHarness({ + registryEntry: { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + }, + }); + + await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( + "process.exit(1)", + ); + + expect(harness.captureOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + expect(harness.spawnSyncSpy).not.toHaveBeenCalledWith( + "openshell", + expect.any(Array), + expect.any(Object), + ); + const errorOutput = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); + expect(errorOutput).toContain("Unset OPENSHELL_GATEWAY_ENDPOINT"); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + it("exits before repairing a lone incomplete legacy custom route (#6315)", async () => { const harness = createConnectHarness({ inferenceGetOutput: diff --git a/src/lib/actions/sandbox/connect.ts b/src/lib/actions/sandbox/connect.ts index 0878065bf2a..8d2c20d3772 100644 --- a/src/lib/actions/sandbox/connect.ts +++ b/src/lib/actions/sandbox/connect.ts @@ -27,6 +27,10 @@ import { GatewayRouteConflictError } from "../../inference/gateway-route-compati import { findReachableOllamaHost, probeLocalProviderHealth } from "../../inference/local"; import { ensureOllamaAuthProxy, probeOllamaAuthProxyHealth } from "../../inference/ollama/proxy"; import { resolveSandboxGatewayName } from "../../onboard/gateway-binding"; +import { + assertNoOpenShellGatewayEndpointOverride, + OpenShellGatewayEndpointOverrideError, +} from "../../openshell-gateway-endpoint-guard"; import { isWsl } from "../../platform"; import { ROOT } from "../../runner"; import * as sandboxVersion from "../../sandbox/version"; @@ -713,6 +717,7 @@ function ensureSandboxInferenceRoute( // This projection is total; the catch below handles only later gateway and repair failures. inference = registry.getSandboxEntryInference(sb); if (inference.kind !== "configured") return { sandbox: sb, routeHealthy: null }; + assertNoOpenShellGatewayEndpointOverride(); const { provider, model } = inference; const gatewayName = resolveSandboxGatewayName(sb); const live = parseGatewayInference( @@ -768,6 +773,10 @@ function ensureSandboxInferenceRoute( return { sandbox: sb, routeHealthy: repairResult.healthy }; } catch (error) { if (!sb || inference?.kind !== "configured") return { sandbox: sb, routeHealthy: null }; + if (error instanceof OpenShellGatewayEndpointOverrideError) { + console.error(` Error: ${error.message}`); + process.exit(1); + } if (error instanceof GatewayRouteConflictError) { console.error(` Error: ${error.message}`); process.exit(1); diff --git a/src/lib/onboard/setup-inference-gateway-scope.test.ts b/src/lib/onboard/setup-inference-gateway-scope.test.ts index 982eb48863e..4f6771d4ccf 100644 --- a/src/lib/onboard/setup-inference-gateway-scope.test.ts +++ b/src/lib/onboard/setup-inference-gateway-scope.test.ts @@ -35,6 +35,46 @@ describe("gateway-scoped onboarding OpenShell commands", () => { ).toEqual(["sandbox", "exec", "-g", GATEWAY, "-n", "alpha", "--", "true"]); }); + it("does not treat gateway-like sandbox payload arguments as OpenShell options", () => { + expect( + scopeGatewayOpenshellArgs( + [ + "sandbox", + "exec", + "-n", + "alpha", + "--", + "tool", + "--gateway", + "payload-gateway", + "--gateway-endpoint=https://payload.example.test", + ], + GATEWAY, + ), + ).toEqual([ + "sandbox", + "exec", + "-g", + GATEWAY, + "-n", + "alpha", + "--", + "tool", + "--gateway", + "payload-gateway", + "--gateway-endpoint=https://payload.example.test", + ]); + }); + + it.each([ + ["--gateway-endpoint", "https://other.example.test"], + ["--gateway-endpoint=https://other.example.test"], + ])("rejects an explicit endpoint override before the payload separator: %j", (...endpointArgs) => { + expect(() => + scopeGatewayOpenshellArgs(["provider", "get", ...endpointArgs, "openai-api"], GATEWAY), + ).toThrow(/--gateway-endpoint may bypass the gateway recorded/); + }); + it.each([ ["-g", GATEWAY], ["--gateway", GATEWAY], @@ -70,6 +110,16 @@ describe("gateway-scoped onboarding OpenShell commands", () => { }); }); + it("rejects an ambient endpoint override before creating a scoped runner", () => { + const run = vi.fn(); + expect(() => + createGatewayScopedOpenshellRunner(run, GATEWAY, { + OPENSHELL_GATEWAY_ENDPOINT: "https://other.example.test", + }), + ).toThrow(/OPENSHELL_GATEWAY_ENDPOINT is set/); + expect(run).not.toHaveBeenCalled(); + }); + it("keeps an omitted provider env separate from the bound gateway", () => { const upsert = vi.fn(() => ({ ok: true })); bindGatewayUpsertProvider(upsert, GATEWAY)("openai-api", "openai", "OPENAI_API_KEY", null); diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index 8d43cff4327..d171a6245be 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -5,6 +5,11 @@ import { type CurrentGatewayRouteCompatibilityCheck, formatGatewayRouteConflict, } from "../inference/gateway-route-compatibility"; +import { + assertNoExplicitOpenShellGatewayEndpoint, + assertNoOpenShellGatewayEndpointOverride, + type OpenShellGatewayEndpointEnvironment, +} from "../openshell-gateway-endpoint-guard"; import type { HermesAuthMethod } from "./hermes-auth"; import type { CommonDeps, @@ -95,6 +100,7 @@ export type SetupInferenceDeps = ProviderBranchDeps & { export function scopeGatewayOpenshellArgs(args: string[], gatewayName: string): string[] { if (!gatewayName) throw new Error("OpenShell gateway name is required."); + assertNoExplicitOpenShellGatewayEndpoint(args); if (args[0] === "gateway" && args[1] === "select") { throw new Error("Gateway-scoped OpenShell operations must not change the selected gateway."); } @@ -103,7 +109,9 @@ export function scopeGatewayOpenshellArgs(args: string[], gatewayName: string): const sandboxProviderCommand = sandboxCommand && args[1] === "provider"; if (!providerCommand && !sandboxCommand) return [...args]; const gatewayFlagIndex = sandboxProviderCommand ? 3 : 2; - const gatewayTargets = args.flatMap((value, index) => { + const separatorIndex = args.indexOf("--"); + const optionEnd = separatorIndex === -1 ? args.length : separatorIndex; + const gatewayTargets = args.slice(0, optionEnd).flatMap((value, index) => { if (index < gatewayFlagIndex) return []; if (value === "-g" || value === "--gateway") return [args[index + 1] ?? ""]; return value.startsWith("--gateway=") ? [value.slice("--gateway=".length)] : []; @@ -126,7 +134,9 @@ export function scopeGatewayOpenshellArgs(args: string[], gatewayName: string): export function createGatewayScopedOpenshellRunner( runOpenshell: (args: string[], ...rest: Rest) => Result, gatewayName: string, + env: OpenShellGatewayEndpointEnvironment = process.env, ): (args: string[], ...rest: Rest) => Result { + assertNoOpenShellGatewayEndpointOverride(env); return (args, ...rest) => runOpenshell(scopeGatewayOpenshellArgs(args, gatewayName), ...rest); } diff --git a/src/lib/openshell-gateway-endpoint-guard.ts b/src/lib/openshell-gateway-endpoint-guard.ts new file mode 100644 index 00000000000..04d8fff4e57 --- /dev/null +++ b/src/lib/openshell-gateway-endpoint-guard.ts @@ -0,0 +1,41 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +export const OPENSHELL_GATEWAY_ENDPOINT_ENV = "OPENSHELL_GATEWAY_ENDPOINT"; + +export type OpenShellGatewayEndpointEnvironment = { + OPENSHELL_GATEWAY_ENDPOINT?: string; +}; + +export class OpenShellGatewayEndpointOverrideError extends Error { + constructor(message: string) { + super(message); + this.name = "OpenShellGatewayEndpointOverrideError"; + } +} + +export function assertNoOpenShellGatewayEndpointOverride( + env: OpenShellGatewayEndpointEnvironment = process.env, +): void { + const endpoint = env.OPENSHELL_GATEWAY_ENDPOINT; + if (typeof endpoint !== "string" || !endpoint.trim()) return; + + throw new OpenShellGatewayEndpointOverrideError( + `${OPENSHELL_GATEWAY_ENDPOINT_ENV} is set, so OpenShell may bypass the gateway recorded for this sandbox. ` + + `Unset ${OPENSHELL_GATEWAY_ENDPOINT_ENV} and retry.`, + ); +} + +export function assertNoExplicitOpenShellGatewayEndpoint(args: readonly string[]): void { + const separatorIndex = args.indexOf("--"); + const optionEnd = separatorIndex === -1 ? args.length : separatorIndex; + for (let index = 0; index < optionEnd; index += 1) { + const arg = args[index]; + if (arg === "--gateway-endpoint" || arg.startsWith("--gateway-endpoint=")) { + throw new OpenShellGatewayEndpointOverrideError( + "OpenShell --gateway-endpoint may bypass the gateway recorded for this sandbox. " + + "Remove --gateway-endpoint and retry.", + ); + } + } +} From 0ed48aa715e3616265beb32d705fa3df715e6344 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 13:48:31 -0700 Subject: [PATCH 15/31] fix(onboard): reject gateway endpoint override early Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 2 +- src/lib/onboard/setup-inference.ts | 3 +++ test/onboard-fsm-live-slices.test.ts | 19 +++++++++++++++++-- 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 1e339802d17..3d365396ae3 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -4078,6 +4078,7 @@ async function preflightAuthoritativeRebuildTarget( // ── Main ───────────────────────────────────────────────────────── const onboard = onboardEntryOptions.withNonInteractiveEnvironment(runOnboard); async function runOnboard(opts: OnboardOptions = {}): Promise { + setupInferenceFactory.assertNoOpenShellGatewayEndpointOverride(); const requestedToolDisclosure = toolDisclosureFlow.applyOnboardToolDisclosureRequest( opts.toolDisclosure, ); @@ -4151,7 +4152,6 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { console.error(` rm -f "${lockResult.lockFile}"`); process.exit(1); } - // Stage any pre-fix plaintext credentials.json into process.env so the // provider upserts later in this run can pick the values up. The file is // NOT removed here — the secure unlink runs only after onboarding diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index d171a6245be..7b964114aed 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -10,6 +10,9 @@ import { assertNoOpenShellGatewayEndpointOverride, type OpenShellGatewayEndpointEnvironment, } from "../openshell-gateway-endpoint-guard"; + +export { assertNoOpenShellGatewayEndpointOverride }; + import type { HermesAuthMethod } from "./hermes-auth"; import type { CommonDeps, diff --git a/test/onboard-fsm-live-slices.test.ts b/test/onboard-fsm-live-slices.test.ts index 9edaf1f5bc6..2611dd628ef 100644 --- a/test/onboard-fsm-live-slices.test.ts +++ b/test/onboard-fsm-live-slices.test.ts @@ -14,6 +14,7 @@ const probeTimeoutMs = 10_000; type SliceName = "initial" | "core" | "final"; type ProbeMode = | "fresh" + | "endpoint-override" | "resume-initial" | "resume-core-gateway" | "authoritative-core-gateway" @@ -352,7 +353,12 @@ const { onboard } = require(${onboardPath}); }); throw new Error("expected slice sentinel"); } catch (error) { - if (error === sentinel || error?.message === sentinel.message) { + if ( + error === sentinel || + error?.message === sentinel.message || + (scenario.mode === "endpoint-override" && + error?.name === "OpenShellGatewayEndpointOverrideError") + ) { console.log(JSON.stringify({ called })); return; } @@ -369,7 +375,12 @@ const { onboard } = require(${onboardPath}); { cwd: repoRoot, encoding: "utf-8", - env: probeEnvironment(tmpDir), + env: { + ...probeEnvironment(tmpDir), + ...(scenario.mode === "endpoint-override" + ? { OPENSHELL_GATEWAY_ENDPOINT: "http://127.0.0.1:65535" } + : {}), + }, timeout: probeTimeoutMs, }, ); @@ -404,6 +415,10 @@ describe("live onboard FSM slice boundaries", () => { assert.deepEqual(runSliceProbe({ slice: "initial" }), ["initial"]); }); + it("rejects an ambient gateway endpoint before entering the initial slice", () => { + assert.deepEqual(runSliceProbe({ slice: "initial", mode: "endpoint-override" }), []); + }); + it("enters the core slice after the initial slice reaches provider selection", () => { assert.deepEqual(runSliceProbe({ slice: "core" }), ["initial", "core"]); }); From 8ca48a80a66f0a5aadd94d5c75d05fbc3915ff9b Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 13:49:24 -0700 Subject: [PATCH 16/31] fix(inference): refresh route peers before mutation Signed-off-by: Aaron Erickson --- ...ence-set-gateway-route-containment.test.ts | 37 +++++++++++++++++++ .../inference-set-route-containment.ts | 4 +- src/lib/actions/inference-set.ts | 2 +- src/lib/onboard/inference-route.ts | 7 +++- .../setup-inference-gateway-scope.test.ts | 30 +++++++++++++++ 5 files changed, 75 insertions(+), 5 deletions(-) diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts index 7ced5a66fe8..bbc4a3a6145 100644 --- a/src/lib/actions/inference-set-gateway-route-containment.test.ts +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -148,6 +148,43 @@ describe("runtime shared gateway route containment", () => { expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); }); + it("refreshes peers after async endpoint validation before route mutation (#6315)", async () => { + const alpha = entry("alpha"); + const peer = entry("late-peer", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://peer.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + }); + const deps = createDeps({ config: {}, entries: [alpha], defaultSandbox: "alpha" }); + const listSandboxes = vi + .fn() + .mockReturnValueOnce({ sandboxes: [alpha], defaultSandbox: "alpha" }) + .mockReturnValue({ sandboxes: [alpha, peer], defaultSandbox: "alpha" }); + deps.listSandboxes = listSandboxes; + + await expect( + runInferenceSet( + { + provider: "compatible-endpoint", + model: "custom/model", + sandboxName: "alpha", + endpointUrl: "https://alpha.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + inferenceApi: "openai-completions", + }, + deps, + ), + ).rejects.toThrow("late-peer"); + + expect(listSandboxes).toHaveBeenCalledTimes(2); + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).toHaveBeenCalledOnce(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + }); + it("blocks an incomplete legacy custom target even without a peer (#6315)", async () => { const deps = createDeps({ config: {}, diff --git a/src/lib/actions/inference-set-route-containment.ts b/src/lib/actions/inference-set-route-containment.ts index 4989bebac19..40d4971f988 100644 --- a/src/lib/actions/inference-set-route-containment.ts +++ b/src/lib/actions/inference-set-route-containment.ts @@ -307,7 +307,7 @@ export async function finalizeInferenceSetRoute(options: { sandboxName: string; provider: string; model: string; - sandboxes: SandboxEntry[]; + getSandboxes: () => SandboxEntry[]; rewriteUrlWithDnsPinning: RewriteConfigUrlsWithDnsPinning; }): Promise<{ registryMetadata: RegistryInferenceMetadata; @@ -333,7 +333,7 @@ export async function finalizeInferenceSetRoute(options: { provider: options.provider, model: options.model, metadata: registryMetadata, - sandboxes: options.sandboxes, + sandboxes: options.getSandboxes(), }); return { registryMetadata, diff --git a/src/lib/actions/inference-set.ts b/src/lib/actions/inference-set.ts index 68c3eef0810..5f789b6135b 100644 --- a/src/lib/actions/inference-set.ts +++ b/src/lib/actions/inference-set.ts @@ -535,7 +535,7 @@ async function runInferenceSetWithoutHostLock( sandboxName, provider, model, - sandboxes: routeSandboxes, + getSandboxes: () => deps.listSandboxes().sandboxes, rewriteUrlWithDnsPinning: deps.rewriteConfigUrlsWithDnsPinning, }); diff --git a/src/lib/onboard/inference-route.ts b/src/lib/onboard/inference-route.ts index 3445b9dcf54..09dd25ec7dd 100644 --- a/src/lib/onboard/inference-route.ts +++ b/src/lib/onboard/inference-route.ts @@ -10,7 +10,10 @@ import { listSandboxes } from "../state/registry"; type RunCaptureOpenshell = (args: string[], options?: { ignoreError?: boolean }) => string | null; -export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpenshell) { +export function createInferenceRouteHelpers( + runCaptureOpenshell: RunCaptureOpenshell, + listSandboxesFn: typeof listSandboxes = listSandboxes, +) { function verifyInferenceRoute(gatewayName: string, _provider: string, _model: string): void { const output = runCaptureOpenshell(["inference", "get", "-g", gatewayName], { ignoreError: true, @@ -31,7 +34,7 @@ export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpens const checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck = (request) => checkGatewayRouteCompatibilityForRegistry({ ...request, - sandboxes: listSandboxes().sandboxes, + sandboxes: listSandboxesFn().sandboxes, }); return { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility }; diff --git a/src/lib/onboard/setup-inference-gateway-scope.test.ts b/src/lib/onboard/setup-inference-gateway-scope.test.ts index 4f6771d4ccf..4186e7954e2 100644 --- a/src/lib/onboard/setup-inference-gateway-scope.test.ts +++ b/src/lib/onboard/setup-inference-gateway-scope.test.ts @@ -154,4 +154,34 @@ describe("gateway-scoped inference route readers", () => { expect(call).toEqual([["inference", "get", "-g", GATEWAY], { ignoreError: true }]); } }); + + it("reads compatibility peers through the injected registry boundary", () => { + const listSandboxes = vi.fn(() => ({ + defaultSandbox: "alpha", + sandboxes: [ + { + name: "alpha", + gatewayName: GATEWAY, + gatewayPort: 9090, + provider: "openai-api", + model: "gpt-test", + gpuEnabled: false, + policies: [], + }, + ], + })); + const route = createInferenceRouteHelpers( + vi.fn(() => null), + listSandboxes, + ); + + expect( + route.checkGatewayRouteCompatibility({ + gatewayName: GATEWAY, + sandboxName: "alpha", + route: { provider: "openai-api", model: "gpt-test" }, + }), + ).toEqual({ ok: true }); + expect(listSandboxes).toHaveBeenCalledOnce(); + }); }); From ffa44a5658de7e3c9d6461f128987ed3947e34b5 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 15:19:01 -0700 Subject: [PATCH 17/31] fix(onboard): contain shared gateway route discovery Signed-off-by: Aaron Erickson --- bin/lib/nim-images.json | 1 + ci/test-file-size-budget.json | 2 +- .../sandbox/connect-route-containment.test.ts | 11 +- src/lib/actions/sandbox/connect.ts | 11 + .../gateway-route-compatibility.test.ts | 78 +++++ .../inference/gateway-route-compatibility.ts | 102 ++++++ src/lib/inference/nim-route-identity.test.ts | 13 + src/lib/inference/nim.ts | 6 + src/lib/inference/vllm.test.ts | 65 +++- src/lib/inference/vllm.ts | 31 +- src/lib/onboard.ts | 220 ++++++------- src/lib/onboard/inference-route.ts | 15 +- .../onboard/machine/core-flow-phases.test.ts | 8 + ...ovider-inference-route-containment.test.ts | 42 ++- .../provider-inference-route-containment.ts | 37 +++ .../handlers/provider-inference.test.ts | 26 +- .../machine/handlers/provider-inference.ts | 39 ++- src/lib/onboard/provider-host-state.test.ts | 36 +++ src/lib/onboard/provider-host-state.ts | 17 +- src/lib/onboard/setup-nim-flow.test.ts | 210 ++++++++++++- src/lib/onboard/setup-nim-flow.ts | 295 ++++++++++++------ src/lib/onboard/setup-nim-ollama.test.ts | 88 +++++- src/lib/onboard/setup-nim-ollama.ts | 53 +++- src/lib/onboard/setup-nim-selection.ts | 3 + src/lib/onboard/setup-nim-vllm.test.ts | 87 ++++++ src/lib/onboard/setup-nim-vllm.ts | 101 ++++++ test/onboard-fsm-live-slices.test.ts | 25 +- test/onboard-nim-route-containment.test.ts | 120 +++++++ ...d-remote-recreate-credential-reuse.test.ts | 4 +- test/onboard.test.ts | 16 +- test/support/connect-flow-test-harness.ts | 4 +- 31 files changed, 1497 insertions(+), 269 deletions(-) create mode 100644 src/lib/inference/nim-route-identity.test.ts create mode 100644 src/lib/onboard/setup-nim-vllm.test.ts create mode 100644 src/lib/onboard/setup-nim-vllm.ts create mode 100644 test/onboard-nim-route-containment.test.ts diff --git a/bin/lib/nim-images.json b/bin/lib/nim-images.json index f639a24454f..bd66b994416 100644 --- a/bin/lib/nim-images.json +++ b/bin/lib/nim-images.json @@ -7,6 +7,7 @@ }, { "name": "nvidia/nemotron-3-nano-30b-a3b", + "servedModel": "nvidia/nemotron-3-nano", "image": "nvcr.io/nim/nvidia/nemotron-3-nano:latest", "minGpuMemoryMB": 8192 }, diff --git a/ci/test-file-size-budget.json b/ci/test-file-size-budget.json index d0629733723..ff73414a542 100644 --- a/ci/test-file-size-budget.json +++ b/ci/test-file-size-budget.json @@ -11,7 +11,7 @@ "test/nemoclaw-start.test.ts": 4827, "test/onboard-messaging.test.ts": 2062, "test/onboard-selection.test.ts": 5835, - "test/onboard.test.ts": 4057, + "test/onboard.test.ts": 4045, "test/policies.test.ts": 2332 } } diff --git a/src/lib/actions/sandbox/connect-route-containment.test.ts b/src/lib/actions/sandbox/connect-route-containment.test.ts index 6d22fc92273..ad94f9cd1b9 100644 --- a/src/lib/actions/sandbox/connect-route-containment.test.ts +++ b/src/lib/actions/sandbox/connect-route-containment.test.ts @@ -104,6 +104,9 @@ describe("connect route containment", () => { "process.exit(1)", ); + expect(harness.ensureLiveSandboxSpy).not.toHaveBeenCalled(); + expect(harness.checkAndRecoverSpy).not.toHaveBeenCalled(); + expect(harness.captureOpenshellSpy).not.toHaveBeenCalled(); expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); @@ -131,10 +134,12 @@ describe("connect route containment", () => { }, }); - await expect(harness.connectSandbox("alpha", { probeOnly: true })).rejects.toThrow( - "process.exit(1)", - ); + await expect(harness.connectSandbox("alpha")).rejects.toThrow("process.exit(1)"); + expect(harness.preflightVllmSpy).not.toHaveBeenCalled(); + expect(harness.ensureLiveSandboxSpy).not.toHaveBeenCalled(); + expect(harness.checkAndRecoverSpy).not.toHaveBeenCalled(); + expect(harness.ensureOllamaAuthProxySpy).not.toHaveBeenCalled(); expect(harness.captureOpenshellSpy).not.toHaveBeenCalled(); expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); diff --git a/src/lib/actions/sandbox/connect.ts b/src/lib/actions/sandbox/connect.ts index 8d2c20d3772..3604439810c 100644 --- a/src/lib/actions/sandbox/connect.ts +++ b/src/lib/actions/sandbox/connect.ts @@ -886,6 +886,17 @@ export async function connectSandbox( sandboxName: string, { probeOnly = false }: SandboxConnectOptions = {}, ): Promise { + try { + assertNoOpenShellGatewayEndpointOverride(); + const registered = registry.getSandbox(sandboxName); + if (registered && registry.getSandboxEntryInference(registered).kind === "configured") { + const gatewayName = resolveSandboxGatewayName(registered); + assertSandboxGatewayRouteCompatible(sandboxName, registered, gatewayName); + } + } catch (error) { + console.error(` Error: ${error instanceof Error ? error.message : String(error)}`); + process.exit(1); + } // probe-only / recover never install or serve a model, so skip the // express-vLLM model preflight for them (it only steers the install path // and would otherwise hard-exit a recovery on a stale NEMOCLAW_VLLM_MODEL). diff --git a/src/lib/inference/gateway-route-compatibility.test.ts b/src/lib/inference/gateway-route-compatibility.test.ts index 14354869b22..f5fb019727d 100644 --- a/src/lib/inference/gateway-route-compatibility.test.ts +++ b/src/lib/inference/gateway-route-compatibility.test.ts @@ -7,6 +7,7 @@ import { checkGatewayRouteCompatibility, formatGatewayRouteConflict, type GatewayInferenceRoute, + preflightGatewayRouteDiscovery, } from "./gateway-route-compatibility"; const route = ( @@ -22,6 +23,14 @@ const route = ( ...overrides, }); +const discoveryRoute = ( + provider: string, + overrides: Partial = {}, +): Omit & { model: string | null } => ({ + ...route(provider, "discovery-pending", overrides), + model: null, +}); + const sandbox = (name: string, overrides: Partial = {}): SandboxEntry => ({ name, gatewayName: "nemoclaw", @@ -40,7 +49,76 @@ function check(requested: GatewayInferenceRoute, sandboxes: SandboxEntry[]) { }); } +function discover( + requested: Omit & { model: string | null }, + sandboxes: SandboxEntry[], +) { + return preflightGatewayRouteDiscovery({ + gatewayName: "nemoclaw", + sandboxName: "target", + route: requested, + sandboxes, + }); +} + describe("shared gateway inference route compatibility", () => { + it("allows unconstrained discovery when no configured same-gateway peer exists (#6315)", () => { + expect( + discover(discoveryRoute("nvidia-prod"), [ + sandbox("other", { gatewayName: "nemoclaw-9090", gatewayPort: 9090 }), + ]), + ).toEqual({ + ok: true, + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + }); + }); + + it("constrains discovery to the durable same-gateway model (#6315)", () => { + expect(discover(discoveryRoute("nvidia-prod"), [sandbox("stopped-peer")])).toEqual({ + ok: true, + requiredModel: "nvidia/model-a", + requiredEndpointUrl: null, + requiredInferenceApi: null, + }); + }); + + it("constrains custom discovery to the durable endpoint and API family (#6315)", () => { + expect( + discover(discoveryRoute("compatible-endpoint"), [ + sandbox("custom-peer", { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://example.test/v1", + preferredInferenceApi: "openai-completions", + }), + ]), + ).toEqual({ + ok: true, + requiredModel: "custom/model", + requiredEndpointUrl: "https://example.test/v1", + requiredInferenceApi: "openai-completions", + }); + }); + + it("blocks conflicting or unprovable discovery before a provider probe (#6315)", () => { + expect(discover(discoveryRoute("anthropic-prod"), [sandbox("stopped-peer")])).toMatchObject({ + ok: false, + result: { conflicts: [{ sandboxName: "stopped-peer", reason: "provider-model" }] }, + }); + expect( + discover(discoveryRoute("nvidia-prod"), [ + sandbox("unknown-gateway", { gatewayName: "not-a-nemoclaw-gateway", gatewayPort: null }), + ]), + ).toMatchObject({ + ok: false, + result: { + conflicts: [{ sandboxName: "unknown-gateway", reason: "invalid-gateway-binding" }], + }, + }); + }); + it("allows identical routes and ignores the target sandbox itself (#6315)", () => { expect( check(route("nvidia-prod", "nvidia/model-a"), [ diff --git a/src/lib/inference/gateway-route-compatibility.ts b/src/lib/inference/gateway-route-compatibility.ts index 7af03b2a2ac..528efc3b5ff 100644 --- a/src/lib/inference/gateway-route-compatibility.ts +++ b/src/lib/inference/gateway-route-compatibility.ts @@ -49,6 +49,22 @@ export type GatewayRouteCompatibilityResult = conflicts: GatewayRouteConflict[]; }; +export interface GatewayRouteDiscoveryConstraints { + requiredModel: string | null; + requiredEndpointUrl: string | null; + requiredInferenceApi: string | null; +} + +export type GatewayRouteDiscoveryResult = + | ({ ok: true } & GatewayRouteDiscoveryConstraints) + | { ok: false; result: Exclude }; + +export type CurrentGatewayRouteDiscoveryPreflight = ( + request: Omit & { + route: Omit & { model: string | null }; + }, +) => GatewayRouteDiscoveryResult; + const CUSTOM_ROUTE_PROVIDERS = new Set(["compatible-endpoint", "compatible-anthropic-endpoint"]); const SUPPORTED_INFERENCE_APIS = new Set([ @@ -96,6 +112,92 @@ function customRouteConflict( return null; } +/** + * Constrain read-only route discovery from durable same-gateway registry peers. + * Missing model/API fields are allowed only when the gateway has no configured + * peer, or when every peer supplies one identical value that discovery must + * subsequently verify with the exact compatibility guard. + */ +export function preflightGatewayRouteDiscovery( + request: Parameters[0] & { + sandboxes: readonly SandboxEntry[]; + }, +): GatewayRouteDiscoveryResult { + const provider = nonEmptyString(request.route.provider); + if (!provider) throw new Error("Requested gateway inference route requires a provider"); + const peers: SandboxEntry[] = []; + const invalidBindings: GatewayRouteConflict[] = []; + for (const sandbox of request.sandboxes) { + if (sandbox.name === request.sandboxName) continue; + let recordedGatewayName: string; + try { + recordedGatewayName = resolveSandboxGatewayName(sandbox); + } catch { + invalidBindings.push({ sandboxName: sandbox.name, reason: "invalid-gateway-binding" }); + continue; + } + if (recordedGatewayName === request.gatewayName && configuredRoute(sandbox)) { + peers.push(sandbox); + } + } + const requestedModel = nonEmptyString(request.route.model); + if (invalidBindings.length > 0) { + return { + ok: false, + result: { + ok: false, + gatewayName: request.gatewayName, + sandboxName: request.sandboxName, + route: { provider, model: requestedModel ?? "model discovery pending" }, + conflicts: invalidBindings, + }, + }; + } + if (peers.length === 0) { + return { + ok: true, + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + }; + } + const reference = peers[0]; + const recorded = configuredRoute(reference); + if (!recorded) throw new Error("Gateway route discovery peer is not configured"); + if (provider !== recorded.provider || (requestedModel && requestedModel !== recorded.model)) { + return { + ok: false, + result: { + ok: false, + gatewayName: request.gatewayName, + sandboxName: request.sandboxName, + route: { provider, model: requestedModel ?? recorded.model }, + conflicts: peers.map((sandbox) => ({ + sandboxName: sandbox.name, + reason: "provider-model" as const, + })), + }, + }; + } + const custom = CUSTOM_ROUTE_PROVIDERS.has(provider); + const candidate: GatewayInferenceRoute = { + ...request.route, + provider, + model: requestedModel ?? recorded.model, + endpointUrl: nonEmptyString(request.route.endpointUrl) ?? reference.endpointUrl, + preferredInferenceApi: + nonEmptyString(request.route.preferredInferenceApi) ?? reference.preferredInferenceApi, + }; + const compatibility = checkGatewayRouteCompatibility({ ...request, route: candidate }); + if (!compatibility.ok) return { ok: false, result: compatibility }; + return { + ok: true, + requiredModel: recorded.model, + requiredEndpointUrl: custom ? (nonEmptyString(reference.endpointUrl) ?? null) : null, + requiredInferenceApi: custom ? normalizedInferenceApi(reference.preferredInferenceApi) : null, + }; +} + /** * Compare a requested route with every configured registry row on the same * OpenShell gateway. Registry rows are intentionally used without a live-state diff --git a/src/lib/inference/nim-route-identity.test.ts b/src/lib/inference/nim-route-identity.test.ts new file mode 100644 index 00000000000..833baed04c3 --- /dev/null +++ b/src/lib/inference/nim-route-identity.test.ts @@ -0,0 +1,13 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it } from "vitest"; + +import { expectedServedModelId } from "./nim"; + +describe("NIM route identity", () => { + it("uses durable served IDs when they differ from catalog IDs (#6315)", () => { + expect(expectedServedModelId("nvidia/nemotron-3-nano-30b-a3b")).toBe("nvidia/nemotron-3-nano"); + expect(expectedServedModelId("meta/llama-3.1-8b-instruct")).toBe("meta/llama-3.1-8b-instruct"); + }); +}); diff --git a/src/lib/inference/nim.ts b/src/lib/inference/nim.ts index 5be3d0d902c..038ceeaadaf 100644 --- a/src/lib/inference/nim.ts +++ b/src/lib/inference/nim.ts @@ -39,6 +39,7 @@ export interface NimModel { name: string; image: string; minGpuMemoryMB: number; + servedModel?: string; } export type NvidiaPlatform = "spark" | "station" | "jetson" | "linux"; @@ -349,6 +350,11 @@ export function getImageForModel(modelName: string): string | null { return entry ? entry.image : null; } +export function expectedServedModelId(modelName: string): string { + const entry = nimImages.models.find((model: NimModel) => model.name === modelName); + return entry?.servedModel || modelName; +} + export function listModels(): NimModel[] { return nimImages.models.map((m: NimModel) => ({ name: m.name, diff --git a/src/lib/inference/vllm.test.ts b/src/lib/inference/vllm.test.ts index c4a8ba83462..d0846ab0348 100644 --- a/src/lib/inference/vllm.test.ts +++ b/src/lib/inference/vllm.test.ts @@ -27,7 +27,29 @@ vi.mock("./nim", () => ({ getGpuIndicesByName: mocks.getGpuIndicesByName, })); -import { buildVllmRunCommand, detectVllmProfile, installVllm, pullImage } from "./vllm"; +import { + buildVllmRunCommand, + detectVllmProfile, + installVllm, + pullImage, + resolveVllmServedModelId, +} from "./vllm"; + +describe("vLLM served route identity", () => { + it("uses one safe served-model override and rejects ambiguous aliases (#6315)", () => { + expect(resolveVllmServedModelId("catalog/model", [])).toBe("catalog/model"); + expect(resolveVllmServedModelId("catalog/model", ["--served-model-name", "served/model"])).toBe( + "served/model", + ); + expect(() => + resolveVllmServedModelId("catalog/model", [ + "--served-model-name", + "served/one", + "served/two", + ]), + ).toThrow("exactly one safe model ID"); + }); +}); describe("vLLM profile detection", () => { beforeEach(() => { @@ -160,6 +182,7 @@ describe("installVllm model resolution", () => { errSpy = vi.spyOn(console, "error").mockImplementation(() => {}); stdoutWrite = vi.spyOn(process.stdout, "write").mockImplementation(() => true); delete process.env.NEMOCLAW_VLLM_MODEL; + delete process.env.NEMOCLAW_VLLM_EXTRA_ARGS_JSON; delete process.env.HF_TOKEN; delete process.env.HUGGING_FACE_HUB_TOKEN; // Fail dockerPrereqsOk so the function returns before any docker work, @@ -244,4 +267,44 @@ describe("installVllm model resolution", () => { const errors = errSpy.mock.calls.map((c: unknown[]) => String(c[0])).join("\n"); expect(errors).toMatch(/gated on Hugging Face/); }); + + it("guards the effective served model before any docker work (#6315)", async () => { + process.env.NEMOCLAW_VLLM_EXTRA_ARGS_JSON = JSON.stringify([ + "--served-model-name", + "shared/served-model", + ]); + const profile = detectVllmProfile({ platform: "spark", type: "nvidia" })!; + const beforeInstall = vi.fn(); + + await installVllm(profile, { + hasImage: true, + nonInteractive: true, + promptFn: vi.fn(), + beforeInstall, + }); + + expect(beforeInstall).toHaveBeenCalledWith("shared/served-model"); + expect(beforeInstall.mock.invocationCallOrder[0]).toBeLessThan( + mocks.runCapture.mock.invocationCallOrder[0], + ); + }); + + it("performs no Docker work when the shared-gateway guard rejects installation (#6315)", async () => { + const profile = detectVllmProfile({ platform: "spark", type: "nvidia" })!; + + await expect( + installVllm(profile, { + hasImage: true, + nonInteractive: true, + promptFn: vi.fn(), + beforeInstall: () => { + throw new Error("route conflict"); + }, + }), + ).rejects.toThrow("route conflict"); + + expect(mocks.runCapture).not.toHaveBeenCalled(); + expect(mocks.dockerPullWithProgressWatchdog).not.toHaveBeenCalled(); + expect(mocks.dockerSpawn).not.toHaveBeenCalled(); + }); }); diff --git a/src/lib/inference/vllm.ts b/src/lib/inference/vllm.ts index 437f34aee24..f06b4cb8dbd 100644 --- a/src/lib/inference/vllm.ts +++ b/src/lib/inference/vllm.ts @@ -9,12 +9,13 @@ import { dockerCapture, dockerPullWithProgressWatchdog, dockerSpawn } from "../a import { buildValidatedCurlCommandArgs } from "../adapters/http/curl-args"; import { VLLM_PORT } from "../core/ports"; import { runCapture, runShell } from "../runner"; +import { isSafeModelId } from "../validation"; import { getGpuIndicesByName } from "./nim"; import { - VLLM_EXTRA_ARGS_ENV, - VLLM_MODELS, buildVllmServeCommand, parseVllmExtraServeArgs, + VLLM_EXTRA_ARGS_ENV, + VLLM_MODELS, type VllmModelDef, type VllmPlatform, } from "./vllm-models"; @@ -493,6 +494,29 @@ interface InstallVllmOptions { hasImage: boolean; nonInteractive: boolean; promptFn: (q: string) => Promise; + beforeInstall?: (modelId: string) => void; +} + +export function resolveVllmServedModelId(modelId: string, extraServeArgs: string[]): string { + let override: string | null = null; + for (let index = 0; index < extraServeArgs.length; index += 1) { + const arg = extraServeArgs[index]; + let values: string[] | null = null; + if (arg === "--served-model-name") { + values = []; + while (index + 1 < extraServeArgs.length && !extraServeArgs[index + 1].startsWith("-")) { + values.push(extraServeArgs[(index += 1)]); + } + } else if (arg.startsWith("--served-model-name=")) { + values = [arg.slice("--served-model-name=".length)]; + } + if (!values) continue; + if (override || values.length !== 1 || !isSafeModelId(values[0])) { + throw new Error("--served-model-name must specify exactly one safe model ID"); + } + override = values[0]; + } + return override ?? modelId; } // Public entry point. Returns ok=false on any prereq, pull, run, or load @@ -512,12 +536,15 @@ export async function installVllm( const { model, source: modelSource } = resolved; let extraServeArgs: string[]; + let servedModelId: string; try { extraServeArgs = parseVllmExtraServeArgs(); + servedModelId = resolveVllmServedModelId(model.id, extraServeArgs); } catch (err) { console.error(` vLLM install failed: ${(err as Error).message}`); return { ok: false }; } + opts.beforeInstall?.(servedModelId); console.log(""); console.log(` vLLM (${profile.name}):`); diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 3d365396ae3..b967c96d840 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -607,6 +607,7 @@ import { type SandboxGpuFlag, } from "./onboard/sandbox-gpu-mode"; import type { SelectionDrift } from "./onboard/selection-drift"; +import { createSetupNimVllmHandler } from "./onboard/setup-nim-vllm"; import { formatOnboardConfigSummary, formatSandboxBuildEstimateNote } from "./onboard/summary"; import type { ModelValidationResult, @@ -994,8 +995,12 @@ function upsertMessagingProviders( // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. const providerExistsInGateway = (name: string, gatewayName: string = GATEWAY_NAME) => onboardProviders.providerExistsInGateway(name, setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, gatewayName)); -const { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility } = - inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); +const { + verifyInferenceRoute, + isInferenceRouteReady, + checkGatewayRouteCompatibility, + preflightGatewayRouteDiscovery, +} = inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); const { inspectSandboxForCreate, pruneStaleSandboxEntry, @@ -1096,6 +1101,18 @@ const { assertOllamaUpgradeApplied, }); +const handleVllmSelection = createSetupNimVllmHandler({ + VLLM_PORT, + runCapture, + getLocalProviderBaseUrl, + getLocalProviderValidationBaseUrl, + isSafeModelId, + requireValue, + validateOpenAiLikeSelection, + applyVllmRuntimeContextWindow: localInference.applyVllmRuntimeContextWindow, + exitProcess: (code) => process.exit(code), +}); + const ollamaModelSize: typeof import("./inference/ollama/model-size") = require("./inference/ollama/model-size"); function isOpenshellInstalled(): boolean { @@ -3070,9 +3087,14 @@ type OllamaModelSelectionOutcome = async function selectAndValidateOllamaModel( gpu: ReturnType, provider: string, - defaults: { requestedModel: string | null; recoveredModel: string | null }, + defaults: { + requestedModel: string | null; + recoveredModel: string | null; + lockedModel?: string | null; + }, + onModelSelected?: (model: string) => void, ): Promise { - const { requestedModel, recoveredModel } = defaults; + const { requestedModel, recoveredModel, lockedModel } = defaults; const probeFailures = new OllamaProbeFailureTracker(); const confirm = (question: string, defaultIsYes: boolean) => promptYesNoOrDefault(question, null, defaultIsYes); @@ -3080,7 +3102,9 @@ async function selectAndValidateOllamaModel( while (true) { const installedModels = getOllamaModelOptions(); let model: string | typeof BACK_TO_SELECTION; - if (isNonInteractive()) { + if (lockedModel) { + model = lockedModel; + } else if (isNonInteractive()) { model = localInference.resolveNonInteractiveOllamaModel(requestedModel, recoveredModel, gpu); } else { model = await promptOllamaModel(gpu, { excludeModels: probeFailures.excludedModels() }); @@ -3091,6 +3115,7 @@ async function selectAndValidateOllamaModel( return { outcome: "back-to-selection" }; } const selectedModel = requireValue(model, "Expected an Ollama model selection"); + onModelSelected?.(selectedModel); if (!installedModels.includes(selectedModel)) { const lookup = ollamaModelSize.getOllamaModelSize(selectedModel); const sizeLabel = ollamaModelSize.formatModelSize(lookup); @@ -3167,77 +3192,6 @@ type SetupNimSelectionResult = "selected" | "retry-selection"; // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. type RemoteProviderSelectionArgs = { selected: ProviderChoice; requestedModel: string | null; recoveredFromSandbox: boolean; recoveredModel: string | null; sandboxName: string | null; gatewayName: string | null }; -async function handleVllmSelection( - state: SetupNimSelectionState, -): Promise { - console.log(` ✓ Using existing vLLM on localhost:${VLLM_PORT}`); - state.provider = "vllm-local"; - // Local vLLM uses an internal credential env, no user API key. - state.credentialEnv = null; - state.endpointUrl = getLocalProviderBaseUrl(state.provider); - if (!state.endpointUrl) { - console.error(" Local vLLM base URL could not be determined."); - process.exit(1); - } - - // Source boundary: local vLLM is an external process, so /v1/models can be - // unreachable, malformed, empty, or return an unsafe served id. setupNim is - // the last safe point before writing provider state, so fail closed here - // rather than returning a partially configured local provider. Remove this - // local guard only if the vLLM manager owns a typed, validated model probe. - const vllmModelsRaw = runCapture(["curl", "-sf", `http://127.0.0.1:${VLLM_PORT}/v1/models`], { - ignoreError: true, - }); - let vllmModels: { data?: Array<{ id?: unknown }> } = {}; - try { - vllmModels = JSON.parse(vllmModelsRaw); - if (vllmModels.data && vllmModels.data.length > 0) { - const detectedModel = - typeof vllmModels.data[0]?.id === "string" ? vllmModels.data[0].id : null; - state.model = detectedModel; - if (!detectedModel || !isSafeModelId(detectedModel)) { - console.error(` Detected model ID contains invalid characters: ${state.model}`); - process.exit(1); - } - console.log(` Detected model: ${state.model}`); - } else { - console.error(" Could not detect model from vLLM. Please specify manually."); - process.exit(1); - } - } catch { - console.error( - ` Could not query vLLM models endpoint. Is vLLM running on localhost:${VLLM_PORT}?`, - ); - process.exit(1); - } - - const validationBaseUrl = getLocalProviderValidationBaseUrl(state.provider); - if (!validationBaseUrl) { - console.error(" Local vLLM validation URL could not be determined."); - process.exit(1); - } - const validation = await validateOpenAiLikeSelection( - "Local vLLM", - validationBaseUrl, - requireValue(state.model as string | null | undefined, "Expected a detected vLLM model"), - null, - ); - if (validation.retry === "selection" || validation.retry === "model") { - return "retry-selection"; - } - if (!validation.ok) return "retry-selection"; - - localInference.applyVllmRuntimeContextWindow(vllmModels, state.model as string); - state.preferredInferenceApi = validation.api; - // Force chat completions — vLLM's /v1/responses endpoint does not run the - // --tool-call-parser, so tool calls arrive as raw text (#976). - if (state.preferredInferenceApi !== "openai-completions") { - console.log(" ℹ Using chat completions API (tool-call-parser requires /v1/chat/completions)"); - } - state.preferredInferenceApi = "openai-completions"; - return "selected"; -} - async function handleRoutedSelection( state: SetupNimSelectionState, ): Promise { @@ -3248,6 +3202,18 @@ async function handleRoutedSelection( return "retry-selection"; } + state.provider = bp.provider_name || "nvidia-router"; + state.model = bp.model; + const { HOST_GATEWAY_URL } = require("./inference/local"); + const routerEndpointUrl = bp.endpoint || ""; + state.endpointUrl = routerEndpointUrl; + if (routerEndpointUrl.match(/localhost|127\.0\.0\.1/)) { + const u = new URL(routerEndpointUrl); + state.endpointUrl = `${HOST_GATEWAY_URL}:${u.port}${u.pathname}`; + } + state.preferredInferenceApi = "openai-completions"; + state.assertRouteCompatible?.(); + const routerCredentialEnv = bp.router?.credential_env || bp.credential_env || DEFAULT_MODEL_ROUTER_CREDENTIAL_ENV; state.credentialEnv = routerCredentialEnv; @@ -3280,16 +3246,6 @@ async function handleRoutedSelection( } } - state.provider = bp.provider_name || "nvidia-router"; - state.model = bp.model; - const { HOST_GATEWAY_URL } = require("./inference/local"); - const routerEndpointUrl = bp.endpoint || ""; - state.endpointUrl = routerEndpointUrl; - if (routerEndpointUrl.match(/localhost|127\.0\.0\.1/)) { - const u = new URL(routerEndpointUrl); - state.endpointUrl = `${HOST_GATEWAY_URL}:${u.port}${u.pathname}`; - } - state.preferredInferenceApi = "openai-completions"; console.log(` ✓ Using Model Router: ${state.provider} / ${state.model}`); return "selected"; } @@ -3307,6 +3263,7 @@ async function handleNimLocalSelection( if (models.length === 0) { console.log(" No NIM models fit your GPU VRAM. Falling back to cloud API."); applyCloudFallbackSelection(state, REMOTE_PROVIDER_CONFIG.build); + state.assertRouteCompatible?.(); return "selected"; } @@ -3336,7 +3293,17 @@ async function handleNimLocalSelection( const modelChoice = await prompt(` Choose model [1]: `); sel = selectFromNumberedMenuOrExit(modelChoice, 1, models); } - state.model = sel.name; + const catalogModel = sel.name; + state.model = nim.expectedServedModelId(catalogModel); + state.provider = "vllm-local"; + state.credentialEnv = null; + state.endpointUrl = getLocalProviderBaseUrl(state.provider); + state.preferredInferenceApi = "openai-completions"; + if (!state.endpointUrl) { + console.error(" Local NVIDIA NIM base URL could not be determined."); + process.exit(1); + } + state.assertRouteCompatible?.(); let ngcApiKey: string | null = null; if (!nim.isNgcLoggedIn()) { @@ -3380,12 +3347,12 @@ async function handleNimLocalSelection( } } - console.log(` Pulling NIM image for ${state.model}...`); - nim.pullNimImage(state.model); + console.log(` Pulling NIM image for ${catalogModel}...`); + nim.pullNimImage(catalogModel); console.log(" Starting NIM container..."); const nimContainerNameLocal = nim.containerName(GATEWAY_NAME); - state.nimContainer = nim.startNimContainerByName(nimContainerNameLocal, state.model, undefined, { + state.nimContainer = nim.startNimContainerByName(nimContainerNameLocal, catalogModel, undefined, { ngcApiKey: ngcApiKey ?? undefined, }); @@ -3393,17 +3360,12 @@ async function handleNimLocalSelection( if (!nim.waitForNimHealth(undefined, undefined, { container: nimContainerNameLocal })) { console.error(" NIM failed to start. Falling back to cloud API."); applyCloudFallbackSelection(state, REMOTE_PROVIDER_CONFIG.build); + state.assertRouteCompatible?.(); return "selected"; } - state.provider = "vllm-local"; - state.credentialEnv = null; - state.endpointUrl = getLocalProviderBaseUrl(state.provider); - if (!state.endpointUrl) { - console.error(" Local NVIDIA NIM base URL could not be determined."); - process.exit(1); - } - state.model = nim.adoptServedModelId(state.model); + state.model = nim.adoptServedModelId(catalogModel); + state.assertRouteCompatible?.(); const nimValidationUrl = getLocalProviderValidationBaseUrl(state.provider) || state.endpointUrl; const validation = await validateOpenAiLikeSelection( "Local NVIDIA NIM", @@ -3434,6 +3396,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, state.credentialEnv = remoteConfig.credentialEnv; state.endpointUrl = remoteConfig.endpointUrl; state.preferredInferenceApi = null; + state.model = requestedModel || (recoveredFromSandbox ? recoveredModel : null); if (selected.key === "custom" || selected.key === "anthropicCompatible") { const kind = selected.key === "custom" ? "openai" : "anthropic"; @@ -3473,8 +3436,17 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, state.endpointUrl, ); } + const explicitApi = (process.env.NEMOCLAW_PREFERRED_API || "").trim().toLowerCase(); + state.preferredInferenceApi = selected.key === "custom" ? (explicitApi === "chat-completions" ? "openai-completions" : explicitApi || null) : null; + if (!state.preferredInferenceApi) { + state.preferredInferenceApi = + selected.key === "custom" || + bedrockRuntimeOnboard.needsBedrockRuntimeAdapter(state.endpointUrl) + ? "openai-completions" + : "anthropic-messages"; + } } - + state.assertRouteCompatible?.(); if (selected.key === "hermesProvider") { const selectedHermesAuthMethod = await promptHermesAuthMethod(); if (isBackToSelection(selectedHermesAuthMethod)) { @@ -3514,7 +3486,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, ); const defaultModel = - requestedModel || (recoveredFromSandbox && recoveredModel) || remoteConfig.defaultModel; + requestedModel || (typeof state.model === "string" && state.model) || remoteConfig.defaultModel; if (isNonInteractive()) { state.model = defaultModel; } else { @@ -3544,6 +3516,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, return "retry-selection"; } state.preferredInferenceApi = "openai-completions"; + state.assertRouteCompatible?.(); console.log(` Using ${remoteConfig.label} with model: ${state.model}`); return "selected"; } @@ -3563,7 +3536,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, await ensureApiKey(); } state.model = await state.nvidiaFeaturedModels!.select( - requestedModel, + requestedModel || (typeof state.model === "string" ? state.model : null), recoveredFromSandbox ? recoveredModel : null, isNonInteractive(), process.env.NEMOCLAW_MODEL, @@ -3579,6 +3552,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, const _envModelRemote = (process.env.NEMOCLAW_MODEL || "").trim(); const defaultModel = requestedModel || + (typeof state.model === "string" && state.model) || _envModelRemote || (recoveredFromSandbox && recoveredModel) || remoteConfig.defaultModel; @@ -3609,9 +3583,12 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, if (bedrockSelection.action === "selected") { state.model = bedrockSelection.model; state.preferredInferenceApi = bedrockSelection.preferredInferenceApi; + state.assertRouteCompatible?.(); return "selected"; } if (isNonInteractive()) { + state.model = defaultModel; + state.assertRouteCompatible?.(); // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. recoveredProviderReuse.resolveRecoveredProviderCredentialReuse( { selected, remoteConfig, state, selectedCredentialEnv, recoveredFromSandbox, selectedModel: defaultModel, sandboxName, recoveredRegistryRoute }, @@ -3630,21 +3607,27 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, let modelValidator: ((candidate: string) => ModelValidationResult) | null = null; if (selected.key === "openai" || selected.key === "gemini") { const modelAuthMode = getProbeAuthMode(state.provider); - modelValidator = (candidate) => - validateOpenAiLikeModel( + modelValidator = (candidate) => { + state.model = candidate; + state.assertRouteCompatible?.(); + return validateOpenAiLikeModel( remoteConfig.label, state.endpointUrl || remoteConfig.endpointUrl, candidate, getCredential(selectedCredentialEnv) || "", ...(modelAuthMode ? [{ authMode: modelAuthMode }] : []), ); + }; } else if (selected.key === "anthropic") { - modelValidator = (candidate) => - validateAnthropicModel( + modelValidator = (candidate) => { + state.model = candidate; + state.assertRouteCompatible?.(); + return validateAnthropicModel( state.endpointUrl || ANTHROPIC_ENDPOINT_URL, candidate, getCredential(selectedCredentialEnv) || "", ); + }; } while (true) { if (isNonInteractive()) { @@ -3664,6 +3647,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, console.log(""); return "retry-selection"; } + state.assertRouteCompatible?.(); const validationResult = state.reuseGatewayCredentialWithoutLocalKey ? "selected" @@ -3673,7 +3657,10 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, state, selectedCredentialEnv, }); - if (validationResult === "selected") break; + if (validationResult === "selected") { + state.assertRouteCompatible?.(); + break; + } if (validationResult === "retry-selection") return "retry-selection"; } } @@ -3683,6 +3670,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, isBackToSelection(state.model) ? null : state.model, `Missing model for ${remoteConfig.label}`, ); + state.assertRouteCompatible?.(); const buildValidation = await buildCredentialReuse.resolveBuildPreferredInferenceApi({ reuseGatewayCredentialWithoutLocalKey: state.skipHostInferenceSmoke === true, note, @@ -3703,6 +3691,7 @@ async function handleRemoteProviderSelection(args: RemoteProviderSelectionArgs, }); if (buildValidation.retrySelection) return "retry-selection"; state.preferredInferenceApi = buildValidation.preferredInferenceApi; + state.assertRouteCompatible?.(); } console.log(` Using ${remoteConfig.label} with model: ${state.model}`); @@ -3753,6 +3742,7 @@ function getSetupNimDeps(): SetupNimDeps { handleVllmSelection, handleRoutedSelection, coerceAgentInferenceApi: inferenceConfig.coerceAgentInferenceApi, + resolveAgentInferenceApi: inferenceConfig.resolveAgentInferenceApi, clearCompatibleEndpointReasoning: reasoningMode.clearCompatibleEndpointReasoning, maybePromptForInferenceInputCapability: (model) => inferenceInputCapability.maybePromptForInferenceInputCapability(model, { @@ -4302,8 +4292,9 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { const recordedSandboxName = session?.steps?.sandbox?.status === "complete" ? session?.sandboxName || null : null; + const gatewaySandboxName = resume ? (recordedSandboxName ?? requestedSandboxName) : null; // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. - const onboardGateway = gatewayBinding.resolveCoreOnboardGatewayBinding({ authoritativeGateway, currentGateway: { name: GATEWAY_NAME, port: GATEWAY_PORT }, resume, sandbox: recordedSandboxName ? registry.getSandbox(recordedSandboxName) : null }); + const onboardGateway = gatewayBinding.resolveCoreOnboardGatewayBinding({ authoritativeGateway, currentGateway: { name: GATEWAY_NAME, port: GATEWAY_PORT }, resume, sandbox: gatewaySandboxName ? registry.getSandbox(gatewaySandboxName) : null }); // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. ({ name: GATEWAY_NAME, port: GATEWAY_PORT } = onboardGateway); process.env.OPENSHELL_GATEWAY = GATEWAY_NAME; @@ -4312,7 +4303,6 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { if (isNonInteractive()) note(" (non-interactive mode)"); if (resume) note(" (resume mode)"); console.log(" ==================="); - const explicitSandboxGpuFlag = resolveSandboxGpuFlagFromOptions(opts); const recordedGpuPassthroughBeforePreflight = session?.gpuPassthrough === true; type InitialOnboardFlowContext = @@ -4470,9 +4460,19 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { }, providerDeps: { checkGatewayRouteCompatibility, + preflightGatewayRouteDiscovery, normalizeHermesAuthMethod, - setupNim: (g, s, a, recover, gateway) => - setupNim(g, s, a, recover, opts.rebuildRegistryInferenceRoute, gateway), + setupNim: (g, s, a, recover, gateway, assertRouteCompatible, canProbeRoute) => + setupNim( + g, + s, + a, + recover, + opts.rebuildRegistryInferenceRoute, + gateway, + assertRouteCompatible, + canProbeRoute, + ), setupInference, startRecordedStep, recordStepComplete, diff --git a/src/lib/onboard/inference-route.ts b/src/lib/onboard/inference-route.ts index 09dd25ec7dd..8a0467d8ebd 100644 --- a/src/lib/onboard/inference-route.ts +++ b/src/lib/onboard/inference-route.ts @@ -4,7 +4,9 @@ import { parseGatewayInference } from "../inference/config"; import { type CurrentGatewayRouteCompatibilityCheck, + type CurrentGatewayRouteDiscoveryPreflight, checkGatewayRouteCompatibility as checkGatewayRouteCompatibilityForRegistry, + preflightGatewayRouteDiscovery as preflightGatewayRouteDiscoveryForRegistry, } from "../inference/gateway-route-compatibility"; import { listSandboxes } from "../state/registry"; @@ -37,5 +39,16 @@ export function createInferenceRouteHelpers( sandboxes: listSandboxesFn().sandboxes, }); - return { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility }; + const preflightGatewayRouteDiscovery: CurrentGatewayRouteDiscoveryPreflight = (request) => + preflightGatewayRouteDiscoveryForRegistry({ + ...request, + sandboxes: listSandboxesFn().sandboxes, + }); + + return { + verifyInferenceRoute, + isInferenceRouteReady, + checkGatewayRouteCompatibility, + preflightGatewayRouteDiscovery, + }; } diff --git a/src/lib/onboard/machine/core-flow-phases.test.ts b/src/lib/onboard/machine/core-flow-phases.test.ts index ca570471037..617d30aa084 100644 --- a/src/lib/onboard/machine/core-flow-phases.test.ts +++ b/src/lib/onboard/machine/core-flow-phases.test.ts @@ -85,6 +85,12 @@ function createPhases( }, providerDeps: { checkGatewayRouteCompatibility: () => ({ ok: true }), + preflightGatewayRouteDiscovery: () => ({ + ok: true, + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + }), normalizeHermesAuthMethod: (value) => value === "oauth" || value === "api_key" ? value : null, setupNim: vi.fn(async () => ({ @@ -273,6 +279,8 @@ describe("core onboard flow phases", () => { { name: "openclaw" }, false, "nemoclaw", + expect.any(Function), + expect.any(Function), ); }); diff --git a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts index 3bc2d05a5b4..9b2eb0bb9d8 100644 --- a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts @@ -3,7 +3,10 @@ import { describe, expect, it, vi } from "vitest"; -import type { CurrentGatewayRouteCompatibilityCheck } from "../../../inference/gateway-route-compatibility"; +import type { + CurrentGatewayRouteCompatibilityCheck, + CurrentGatewayRouteDiscoveryPreflight, +} from "../../../inference/gateway-route-compatibility"; import { createSession, type Session, type SessionUpdates } from "../../../state/onboard-session"; import { handleProviderInferenceState, @@ -30,7 +33,26 @@ function createDeps() { checkGatewayRouteCompatibility: vi.fn(() => ({ ok: true, })), - setupNim: vi.fn(async () => ({ ...fallbackSelection })), + preflightGatewayRouteDiscovery: vi.fn(() => ({ + ok: true, + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + })), + selectionProbe: vi.fn(), + setupNim: vi.fn( + async (_gpu, _sandbox, _agent, _recover, _gateway, guard) => { + guard?.({ + provider: fallbackSelection.provider, + model: fallbackSelection.model, + endpointUrl: fallbackSelection.endpointUrl, + credentialEnv: fallbackSelection.credentialEnv, + preferredInferenceApi: fallbackSelection.preferredInferenceApi, + }); + calls.selectionProbe(); + return { ...fallbackSelection }; + }, + ), setupInference: vi.fn(async () => ({ ok: true as const })), recordStepComplete: vi.fn(async () => createSession()), surfaceReady: vi.fn(() => true), @@ -50,6 +72,7 @@ function createDeps() { }; const deps: Options["deps"] = { checkGatewayRouteCompatibility: calls.checkGatewayRouteCompatibility, + preflightGatewayRouteDiscovery: calls.preflightGatewayRouteDiscovery, normalizeHermesAuthMethod: () => null, setupNim: calls.setupNim, setupInference: calls.setupInference, @@ -139,6 +162,16 @@ function rejectRoute( route: { provider, model }, conflicts: [{ sandboxName: "existing-sandbox", reason: "provider-model" }], }); + calls.preflightGatewayRouteDiscovery.mockReturnValue({ + ok: false, + result: { + ok: false, + gatewayName: "nemoclaw-9090", + sandboxName: "target-sandbox", + route: { provider, model }, + conflicts: [{ sandboxName: "existing-sandbox", reason: "provider-model" }], + }, + }); } describe("provider route containment", () => { @@ -152,7 +185,7 @@ describe("provider route containment", () => { ).rejects.toThrow("exit 1"); expect(calls.setupNim).toHaveBeenCalledOnce(); - expect(calls.checkGatewayRouteCompatibility).toHaveBeenCalledWith({ + expect(calls.preflightGatewayRouteDiscovery).toHaveBeenCalledWith({ gatewayName: "nemoclaw-9090", sandboxName: null, route: { @@ -160,8 +193,11 @@ describe("provider route containment", () => { model: "nvidia/test", endpointUrl: "https://integrate.api.nvidia.com/v1", preferredInferenceApi: "openai-responses", + credentialEnv: "NVIDIA_INFERENCE_API_KEY", }, }); + expect(calls.checkGatewayRouteCompatibility).not.toHaveBeenCalled(); + expect(calls.selectionProbe).not.toHaveBeenCalled(); expect(calls.recordStepComplete).not.toHaveBeenCalled(); expect(calls.surfaceReady).not.toHaveBeenCalled(); expect(calls.setupInference).not.toHaveBeenCalled(); diff --git a/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts b/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts index f22285b1f94..e6ff6327a18 100644 --- a/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts +++ b/src/lib/onboard/machine/handlers/provider-inference-route-containment.ts @@ -3,16 +3,23 @@ import { type CurrentGatewayRouteCompatibilityCheck, + type CurrentGatewayRouteDiscoveryPreflight, formatGatewayRouteConflict, type GatewayInferenceRoute, + type GatewayRouteDiscoveryConstraints, } from "../../../inference/gateway-route-compatibility"; export interface ProviderInferenceRouteContainmentDeps { checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; + preflightGatewayRouteDiscovery: CurrentGatewayRouteDiscoveryPreflight; error(message: string): void; exitProcess(code: number): never; } +export type ProviderInferenceProbeRoute = Omit & { + model: string | null; +}; + export function assertProviderInferenceRouteCompatible( deps: ProviderInferenceRouteContainmentDeps, gatewayName: string, @@ -25,3 +32,33 @@ export function assertProviderInferenceRouteCompatible( deps.exitProcess(1); } } + +/** Constrain discovery from durable peers, then exact-check complete route identities. */ +export function guardProviderInferenceRouteSelection( + deps: ProviderInferenceRouteContainmentDeps, + gatewayName: string, + sandboxName: string | null, + route: ProviderInferenceProbeRoute, +): GatewayRouteDiscoveryConstraints { + const model = typeof route.model === "string" && route.model.trim() ? route.model : null; + const preflight = deps.preflightGatewayRouteDiscovery({ + gatewayName, + sandboxName, + route: { ...route, model }, + }); + if (!preflight.ok) { + deps.error(` Error: ${formatGatewayRouteConflict(preflight.result)}`); + deps.exitProcess(1); + } + const provider = typeof route.provider === "string" ? route.provider.trim() : ""; + const completeCustomRoute = + !["compatible-endpoint", "compatible-anthropic-endpoint"].includes(provider) || + (typeof route.endpointUrl === "string" && + route.endpointUrl.trim().length > 0 && + typeof route.preferredInferenceApi === "string" && + route.preferredInferenceApi.trim().length > 0); + if (model && completeCustomRoute) { + assertProviderInferenceRouteCompatible(deps, gatewayName, sandboxName, { ...route, model }); + } + return preflight; +} diff --git a/src/lib/onboard/machine/handlers/provider-inference.test.ts b/src/lib/onboard/machine/handlers/provider-inference.test.ts index d9b9d36356b..e8ec62ceabb 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.test.ts @@ -7,7 +7,10 @@ import path from "node:path"; import { describe, expect, it, vi } from "vitest"; -import type { CurrentGatewayRouteCompatibilityCheck } from "../../../inference/gateway-route-compatibility"; +import type { + CurrentGatewayRouteCompatibilityCheck, + CurrentGatewayRouteDiscoveryPreflight, +} from "../../../inference/gateway-route-compatibility"; import { createSession, type Session, type SessionUpdates } from "../../../state/onboard-session"; import { patchStagedDockerfile } from "../../dockerfile-patch"; import { clearCompatibleEndpointReasoning } from "../../reasoning-mode"; @@ -40,6 +43,12 @@ function createDeps( checkGatewayRouteCompatibility: vi.fn(() => ({ ok: true, })), + preflightGatewayRouteDiscovery: vi.fn(() => ({ + ok: true, + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + })), setupNim: vi.fn(async () => ({ ...baseSelection })), setupInference: vi.fn(async () => ({ ok: true as const })), startStep: vi.fn(async () => undefined), @@ -87,6 +96,7 @@ function createDeps( calls, deps: { checkGatewayRouteCompatibility: calls.checkGatewayRouteCompatibility, + preflightGatewayRouteDiscovery: calls.preflightGatewayRouteDiscovery, normalizeHermesAuthMethod: (value: string | null | undefined) => value === "oauth" || value === "api_key" ? value : null, setupNim: calls.setupNim, @@ -173,7 +183,15 @@ describe("handleProviderInferenceState", () => { const result = await handleProviderInferenceState(baseOptions(deps)); expect(calls.startStep).toHaveBeenNthCalledWith(1, "provider_selection"); - expect(calls.setupNim).toHaveBeenCalledWith({ type: "nvidia" }, null, null, true, "nemoclaw"); + expect(calls.setupNim).toHaveBeenCalledWith( + { type: "nvidia" }, + null, + null, + true, + "nemoclaw", + expect.any(Function), + expect.any(Function), + ); expect(calls.promptName).toHaveBeenCalledWith(null); expect(calls.log).toHaveBeenCalledWith("summary:nvidia-prod/nvidia/test/my-assistant"); expect(calls.startStep).toHaveBeenNthCalledWith(2, "inference", { @@ -443,6 +461,8 @@ describe("handleProviderInferenceState", () => { null, false, "nemoclaw", + expect.any(Function), + expect.any(Function), ); }); @@ -466,6 +486,8 @@ describe("handleProviderInferenceState", () => { null, false, "nemoclaw", + expect.any(Function), + expect.any(Function), ); expect(calls.setupInference).toHaveBeenCalled(); }); diff --git a/src/lib/onboard/machine/handlers/provider-inference.ts b/src/lib/onboard/machine/handlers/provider-inference.ts index c2aed2c440d..f76d75f8d06 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.ts @@ -2,12 +2,20 @@ // SPDX-License-Identifier: Apache-2.0 import { coerceAgentInferenceApi, resolveAgentInferenceApi } from "../../../inference/config"; -import type { CurrentGatewayRouteCompatibilityCheck } from "../../../inference/gateway-route-compatibility"; +import type { + CurrentGatewayRouteCompatibilityCheck, + CurrentGatewayRouteDiscoveryPreflight, + GatewayRouteDiscoveryConstraints, +} from "../../../inference/gateway-route-compatibility"; import type { WebSearchConfig } from "../../../inference/web-search"; import type { HermesAuthMethod, Session, SessionUpdates } from "../../../state/onboard-session"; import { withInferenceTrace, withProviderSelectionTrace } from "../../tracing"; import { advanceTo, type OnboardStateTransitionResult, retryTo } from "../result"; -import { assertProviderInferenceRouteCompatible } from "./provider-inference-route-containment"; +import { + assertProviderInferenceRouteCompatible, + guardProviderInferenceRouteSelection, + type ProviderInferenceProbeRoute, +} from "./provider-inference-route-containment"; export type ProviderInferenceRetry = { retry: "selection" } | { ok: true; retry?: undefined }; @@ -72,6 +80,7 @@ export interface ProviderInferenceStateOptions { }; deps: { checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; + preflightGatewayRouteDiscovery: CurrentGatewayRouteDiscoveryPreflight; normalizeHermesAuthMethod(value: string | null | undefined): HermesAuthMethod | null; setupNim( gpu: Gpu, @@ -79,6 +88,10 @@ export interface ProviderInferenceStateOptions { agent: Agent, allowRecordedProviderRecovery?: boolean, gatewayName?: string, + assertRouteCompatible?: ( + route: ProviderInferenceProbeRoute, + ) => GatewayRouteDiscoveryConstraints, + canProbeRoute?: (provider: string) => boolean, ): Promise; setupInference( sandboxName: string | null, @@ -397,7 +410,27 @@ export async function handleProviderInferenceState({ const selection = await withProviderSelectionTrace( sandboxName, (agent as { name?: string } | null)?.name, - () => deps.setupNim(gpu, sandboxName, agent, !fresh, gatewayName), + () => + deps.setupNim( + gpu, + sandboxName, + agent, + !fresh, + gatewayName, + (route) => guardProviderInferenceRouteSelection(deps, gatewayName, sandboxName, route), + (provider) => + deps.preflightGatewayRouteDiscovery({ + gatewayName, + sandboxName, + route: { + provider, + model: null, + endpointUrl: null, + preferredInferenceApi: null, + credentialEnv: null, + }, + }).ok, + ), ); model = selection.model; provider = selection.provider; diff --git a/src/lib/onboard/provider-host-state.test.ts b/src/lib/onboard/provider-host-state.test.ts index e2f2ec566bd..82693b2460c 100644 --- a/src/lib/onboard/provider-host-state.test.ts +++ b/src/lib/onboard/provider-host-state.test.ts @@ -57,6 +57,42 @@ function detectWithDeps( } describe("detectInferenceProviderHostState", () => { + it("suppresses local endpoint probes when route preflight disallows them (#6315)", () => { + const runCapture = vi.fn(() => "{}"); + const findReachableOllamaHost = vi.fn(() => "127.0.0.1"); + const deps = buildDeps({ + runCapture, + findReachableOllamaHost, + isWsl: vi.fn(() => true), + detectWindowsHostOllama: vi.fn(() => ({ + installed: true, + installedPath: "C:\\Ollama\\ollama.exe", + loopbackOnly: false, + })), + }); + + const state = detectInferenceProviderHostState({ + gpu: null, + experimental: false, + probeOllama: false, + probeVllm: false, + platform: "linux", + env: {}, + log: () => {}, + deps, + }); + + expect(findReachableOllamaHost).not.toHaveBeenCalled(); + expect(state.ollamaRunning).toBe(false); + expect(state.vllmRunning).toBe(false); + expect(state.windowsOllamaReachable).toBe(false); + expect( + runCapture.mock.calls.some(([command]) => + command.join(" ").match(/\/v1\/models|\/api\/tags/), + ), + ).toBe(false); + }); + it("collects local Ollama and vLLM state into one provider host snapshot", () => { const deps = buildDeps({ hostCommandExists: vi.fn((command) => command === "ollama"), diff --git a/src/lib/onboard/provider-host-state.ts b/src/lib/onboard/provider-host-state.ts index 8b50e5c9984..7fcea76a66f 100644 --- a/src/lib/onboard/provider-host-state.ts +++ b/src/lib/onboard/provider-host-state.ts @@ -17,7 +17,7 @@ import { getWindowsHostOllamaDockerRequirement, type WindowsHostOllamaDockerRequirement, } from "./local-inference-topology"; -import { resolveOllamaInstallMenuEntry, type OllamaInstallMenuResult } from "./ollama-install-menu"; +import { type OllamaInstallMenuResult, resolveOllamaInstallMenuEntry } from "./ollama-install-menu"; import { buildVllmMenuEntries, type VllmMenuEntry } from "./vllm-menu"; import { detectWindowsHostOllama, type WindowsHostOllamaState } from "./windows-host-ollama"; @@ -53,6 +53,8 @@ export interface InferenceProviderHostState { export interface DetectInferenceProviderHostStateInput { gpu: InferenceProviderHostGpu | null | undefined; experimental: boolean; + probeOllama?: boolean; + probeVllm?: boolean; platform?: NodeJS.Platform; env?: NodeJS.ProcessEnv; log?: (message?: string) => void; @@ -156,10 +158,10 @@ export function detectInferenceProviderHostState( const platform = input.platform ?? process.platform; const isWsl = deps.isWsl({ platform, env: input.env }); const hasOllama = deps.hostCommandExists("ollama"); - const ollamaHost = deps.findReachableOllamaHost(); + const ollamaHost = input.probeOllama === false ? null : deps.findReachableOllamaHost(); const ollamaRunning = ollamaHost !== null; const isWindowsHostOllama = ollamaHost === OLLAMA_HOST_DOCKER_INTERNAL; - const vllmRunning = probeVllmRunning(deps.runCapture); + const vllmRunning = input.probeVllm === false ? false : probeVllmRunning(deps.runCapture); const vllmProfile = deps.detectVllmProfile(input.gpu); const hasVllmImage = !!( vllmProfile && @@ -170,11 +172,10 @@ export function detectInferenceProviderHostState( ); const winOllamaState = deps.detectWindowsHostOllama(); const hasWindowsOllama = winOllamaState.installed; - const windowsOllamaReachable = probeWindowsOllamaReachable({ - isWsl, - isWindowsHostOllama, - runCapture: deps.runCapture, - }); + const windowsOllamaReachable = + input.probeOllama === false + ? false + : probeWindowsOllamaReachable({ isWsl, isWindowsHostOllama, runCapture: deps.runCapture }); maybeWarnAboutDuplicateOllamaDaemons({ isWsl, diff --git a/src/lib/onboard/setup-nim-flow.test.ts b/src/lib/onboard/setup-nim-flow.test.ts index 2d305f98d3d..fd111675659 100644 --- a/src/lib/onboard/setup-nim-flow.test.ts +++ b/src/lib/onboard/setup-nim-flow.test.ts @@ -123,6 +123,8 @@ function makeDeps(overrides: Partial = {}): SetupNimFlowDeps { handleVllmSelection: async () => unexpected("vLLM selection"), handleRoutedSelection: async () => unexpected("routed selection"), coerceAgentInferenceApi: (_agent, preferredInferenceApi) => preferredInferenceApi, + resolveAgentInferenceApi: (_agentName, _provider, preferredInferenceApi) => + preferredInferenceApi, clearCompatibleEndpointReasoning: () => null, maybePromptForInferenceInputCapability: vi.fn(async () => {}), }; @@ -221,6 +223,172 @@ describe("createSetupNim", () => { expect(result).toMatchObject({ model: "final-model", provider: "nvidia-prod" }); }); + it("suppresses unrelated local endpoint probes for an explicit remote selection (#6315)", async () => { + const detectInferenceProviderHostState = vi.fn(() => makeHostState()); + const canProbeRoute = vi.fn(() => true); + const handleRemoteProviderSelection = vi.fn( + async (_args, state) => { + state.model = "gpt-test"; + state.provider = "openai-api"; + state.endpointUrl = "https://api.openai.com/v1"; + state.credentialEnv = "OPENAI_API_KEY"; + return "selected"; + }, + ); + const setupNim = createSetupNim( + makeDeps({ + isNonInteractive: () => true, + getNonInteractiveProvider: () => "openai", + detectInferenceProviderHostState, + handleRemoteProviderSelection, + }), + ); + + await setupNim(null, null, null, true, null, "nemoclaw", undefined, canProbeRoute); + + expect(detectInferenceProviderHostState).toHaveBeenCalledWith({ + gpu: null, + experimental: false, + probeOllama: false, + probeVllm: false, + }); + expect(canProbeRoute).not.toHaveBeenCalled(); + }); + + it("checks shared-gateway compatibility before interactive local discovery probes (#6315)", async () => { + const events: string[] = []; + const canProbeRoute = vi.fn((provider: string) => { + events.push(`preflight:${provider}`); + return false; + }); + const detectInferenceProviderHostState = vi.fn((input) => { + events.push(`detect:${String(input.probeOllama)}:${String(input.probeVllm)}`); + return makeHostState(); + }); + const handleRemoteProviderSelection = vi.fn( + async (_args, state) => { + state.model = "nvidia/test"; + state.provider = "nvidia-prod"; + state.endpointUrl = "https://integrate.api.nvidia.com/v1"; + state.credentialEnv = "NVIDIA_INFERENCE_API_KEY"; + return "selected"; + }, + ); + const setupNim = createSetupNim( + makeDeps({ detectInferenceProviderHostState, handleRemoteProviderSelection }), + ); + + await setupNim(null, null, null, true, null, "nemoclaw", undefined, canProbeRoute); + + expect(events).toEqual([ + "preflight:ollama-local", + "preflight:vllm-local", + "detect:false:false", + ]); + }); + + it("rejects a known local route before host detection when its model conflicts (#6315)", async () => { + const detectInferenceProviderHostState = vi.fn(() => makeHostState()); + const routeGuard = vi.fn(() => { + throw new Error("route conflict"); + }); + const setupNim = createSetupNim( + makeDeps({ + isNonInteractive: () => true, + getNonInteractiveProvider: () => "ollama", + getNonInteractiveModel: () => "conflict/model", + detectInferenceProviderHostState, + }), + ); + + await expect(setupNim(null, null, null, true, null, "nemoclaw", routeGuard)).rejects.toThrow( + "route conflict", + ); + expect(routeGuard).toHaveBeenCalledWith( + expect.objectContaining({ provider: "ollama-local", model: "conflict/model" }), + ); + expect(detectInferenceProviderHostState).not.toHaveBeenCalled(); + }); + + it("applies same-gateway discovery constraints before a provider probe (#6315)", async () => { + const providerProbe = vi.fn(); + const routeGuard = vi.fn( + (route: { model: string | null; preferredInferenceApi?: string | null }) => ({ + requiredModel: route.model ? null : "shared/model", + requiredEndpointUrl: "https://shared.example.test/v1", + requiredInferenceApi: route.preferredInferenceApi ? null : "openai-responses", + }), + ); + const handleRemoteProviderSelection = vi.fn( + async (_args, state) => { + state.provider = "compatible-endpoint"; + state.model = null; + state.endpointUrl = "https://shared.example.test/v1"; + state.credentialEnv = "COMPATIBLE_API_KEY"; + state.preferredInferenceApi = null; + state.assertRouteCompatible?.(); + expect(state.model).toBe("shared/model"); + expect(state.preferredInferenceApi).toBe("openai-responses"); + providerProbe(); + return "selected"; + }, + ); + const setupNim = createSetupNim( + makeDeps({ + isNonInteractive: () => true, + getNonInteractiveProvider: () => "custom", + handleRemoteProviderSelection, + }), + ); + + const result = await setupNim(null, null, null, true, null, "nemoclaw", routeGuard); + + expect(providerProbe).toHaveBeenCalledOnce(); + expect(routeGuard).toHaveBeenLastCalledWith( + expect.objectContaining({ + provider: "compatible-endpoint", + model: "shared/model", + preferredInferenceApi: "openai-responses", + }), + ); + expect(result.model).toBe("shared/model"); + }); + + it("guards custom Anthropic routes with the final Hermes API identity (#6315)", async () => { + const agent = { name: "hermes" } as AgentDefinition; + const routeGuard = vi.fn((route) => { + expect(route.preferredInferenceApi).toBe("openai-completions"); + return { requiredModel: null, requiredEndpointUrl: null, requiredInferenceApi: null }; + }); + const handleRemoteProviderSelection = vi.fn( + async (_args, state) => { + state.provider = "compatible-anthropic-endpoint"; + state.model = "anthropic/model"; + state.endpointUrl = "https://anthropic.example.test"; + state.credentialEnv = "ANTHROPIC_COMPATIBLE_API_KEY"; + state.preferredInferenceApi = "anthropic-messages"; + state.assertRouteCompatible?.(); + return "selected"; + }, + ); + const setupNim = createSetupNim( + makeDeps({ + isNonInteractive: () => true, + getNonInteractiveProvider: () => "anthropicCompatible", + resolveAgentInferenceApi: (agentName, provider, preferredInferenceApi) => + agentName === "hermes" && provider === "compatible-anthropic-endpoint" + ? "openai-completions" + : preferredInferenceApi, + handleRemoteProviderSelection, + }), + ); + + const result = await setupNim(null, null, agent, true, null, "nemoclaw", routeGuard); + + expect(routeGuard).toHaveBeenCalled(); + expect(result.preferredInferenceApi).toBe("openai-completions"); + }); + it("recovers a recorded provider and model without prompting in non-interactive mode (#6245)", async () => { const prompt = vi.fn(async () => unexpected("interactive provider prompt")); const note = vi.fn(); @@ -342,9 +510,24 @@ describe("createSetupNim", () => { it("continues from a successful managed vLLM install into provider selection (#6245)", async () => { const profile = { name: "DGX Spark" } as VllmProfile; const prompt = vi.fn(async () => unexpected("provider prompt")); - const installVllm = vi.fn(async () => ({ ok: true })); + const detectInferenceProviderHostState = vi.fn(() => + makeHostState({ + vllmProfile: profile, + hasVllmImage: true, + vllmEntries: [{ key: "install-vllm", label: "Start vLLM (DGX Spark)" }], + }), + ); + const installVllm = vi.fn(async (_profile, options) => { + options.beforeInstall?.("vllm-model"); + return { ok: true }; + }); + const routeGuard = vi.fn(() => ({ + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + })); const handleVllmSelection = vi.fn(async (state) => { - state.model = "vllm-model"; + expect(state.model).toBe("vllm-model"); state.provider = "vllm"; state.endpointUrl = "http://127.0.0.1:8000/v1"; state.credentialEnv = null; @@ -356,25 +539,34 @@ describe("createSetupNim", () => { isNonInteractive: () => true, getNonInteractiveProvider: () => "install-vllm", prompt, - detectInferenceProviderHostState: () => - makeHostState({ - vllmProfile: profile, - hasVllmImage: true, - vllmEntries: [{ key: "install-vllm", label: "Start vLLM (DGX Spark)" }], - }), + detectInferenceProviderHostState, installVllm, handleVllmSelection, }), ); - const result = await setupNim(null); + const result = await setupNim(null, null, null, true, null, "nemoclaw", routeGuard); expect(installVllm).toHaveBeenCalledWith(profile, { hasImage: true, nonInteractive: true, promptFn: prompt, + beforeInstall: expect.any(Function), }); expect(prompt).not.toHaveBeenCalled(); + expect(detectInferenceProviderHostState).toHaveBeenCalledWith({ + gpu: null, + experimental: false, + probeOllama: false, + probeVllm: false, + }); + expect(routeGuard).toHaveBeenCalledWith({ + provider: "vllm-local", + model: "vllm-model", + endpointUrl: null, + preferredInferenceApi: "openai-completions", + credentialEnv: null, + }); expect(handleVllmSelection).toHaveBeenCalledOnce(); expect(result).toMatchObject({ model: "vllm-model", diff --git a/src/lib/onboard/setup-nim-flow.ts b/src/lib/onboard/setup-nim-flow.ts index dd06fd2875f..6fcdeeb08c2 100644 --- a/src/lib/onboard/setup-nim-flow.ts +++ b/src/lib/onboard/setup-nim-flow.ts @@ -2,13 +2,16 @@ // SPDX-License-Identifier: Apache-2.0 import type { AgentDefinition } from "../agent/defs"; +import type { GatewayRouteDiscoveryConstraints } from "../inference/gateway-route-compatibility"; import type { VllmProfile } from "../inference/vllm"; import { isBackToSelection } from "../navigation"; import type { HermesAuthMethod } from "./hermes-auth"; import type { ProviderSelectionResult } from "./machine/handlers/provider-inference"; +import type { ProviderInferenceProbeRoute } from "./machine/handlers/provider-inference-route-containment"; import type { NvidiaFeaturedModelSession } from "./nvidia-featured-model-selection"; import type { InferenceProviderHostGpu, InferenceProviderHostState } from "./provider-host-state"; import { buildInferenceProviderMenu, type ProviderMenuChoice } from "./provider-menu"; +import { providerNameToOptionKey } from "./provider-recovery"; import { resolveRequestedProviderSelection } from "./provider-selection"; import { reportProviderSelectionFailure } from "./provider-selection-failure"; import { promptForInferenceProviderSelection } from "./provider-selection-prompt"; @@ -42,6 +45,8 @@ export type SetupNim = ( recoverProvider?: boolean, rebuildRegistryInferenceRoute?: RebuildRouteHandoff | null, gatewayName?: string | null, + assertRouteCompatible?: (route: ProviderInferenceProbeRoute) => GatewayRouteDiscoveryConstraints, + canProbeRoute?: (provider: string) => boolean, ) => Promise; export interface SetupNimFlowDeps { @@ -57,6 +62,8 @@ export interface SetupNimFlowDeps { detectInferenceProviderHostState(input: { gpu: InferenceProviderHostGpu | null | undefined; experimental: boolean; + probeOllama?: boolean; + probeVllm?: boolean; }): InferenceProviderHostState; getAgentInferenceProviderOptions(agent: AgentDefinition | null | undefined): string[]; loadRoutedProfile(): { router?: { enabled?: boolean } } | null | undefined; @@ -121,6 +128,7 @@ export interface SetupNimFlowDeps { hasImage: boolean; nonInteractive: boolean; promptFn: (question: string) => Promise; + beforeInstall?: (modelId: string) => void; }, ): Promise<{ ok: boolean }>; handleVllmSelection(state: SetupNimSelectionState): Promise; @@ -129,6 +137,11 @@ export interface SetupNimFlowDeps { agent: AgentDefinition | null, preferredInferenceApi: string | null, ): string | null; + resolveAgentInferenceApi( + agentName: string | null, + provider: string, + preferredInferenceApi: string | null, + ): string | null; clearCompatibleEndpointReasoning(): null; maybePromptForInferenceInputCapability(model: string | null): Promise; } @@ -153,6 +166,121 @@ function clearReasoningUnlessCompatible( return deps.clearCompatibleEndpointReasoning(); } +function applyGatewayRouteDiscoveryConstraints( + state: SetupNimSelectionState, + constraints: GatewayRouteDiscoveryConstraints, +): void { + if (!state.model && constraints.requiredModel) { + state.model = constraints.requiredModel; + } + if (!state.endpointUrl && constraints.requiredEndpointUrl) { + state.endpointUrl = constraints.requiredEndpointUrl; + } + if (!state.preferredInferenceApi && constraints.requiredInferenceApi) { + state.preferredInferenceApi = constraints.requiredInferenceApi; + } +} + +const OLLAMA_PROBE_PROVIDER_KEYS = new Set([ + "ollama", + "install-ollama", + "start-windows-ollama", + "install-windows-ollama", +]); +const VLLM_ROUTE_PROVIDER_KEYS = new Set(["vllm", "install-vllm"]); +const VLLM_PROBE_PROVIDER_KEYS = new Set(["vllm"]); + +function localProviderProbeIntent(providerKey: string | null): { + ollama: boolean; + vllm: boolean; +} { + if (!providerKey) return { ollama: true, vllm: true }; + return { + ollama: OLLAMA_PROBE_PROVIDER_KEYS.has(providerKey), + vllm: VLLM_PROBE_PROVIDER_KEYS.has(providerKey), + }; +} + +function localProbeRouteProvider(providerKey: string | null): string | null { + if (providerKey && OLLAMA_PROBE_PROVIDER_KEYS.has(providerKey)) return "ollama-local"; + if (providerKey && VLLM_ROUTE_PROVIDER_KEYS.has(providerKey)) return "vllm-local"; + return null; +} + +function prepareProviderDiscovery(options: { + deps: SetupNimFlowDeps; + sandboxName: string | null; + recoverProvider: boolean; + rebuildRegistryInferenceRoute: RebuildRouteHandoff | null; + assertRouteCompatible?: (route: ProviderInferenceProbeRoute) => GatewayRouteDiscoveryConstraints; + canProbeRoute?: (provider: string) => boolean; +}): { + requestedProvider: string | null; + requestedModel: string | null; + recoveredRegistryRoute: RegistryInferenceRoute | null; + probeOllama: boolean; + probeVllm: boolean; +} { + const { + deps, + sandboxName, + recoverProvider, + rebuildRegistryInferenceRoute, + assertRouteCompatible, + canProbeRoute, + } = options; + const nonInteractive = deps.isNonInteractive(); + const requestedProvider = deps.getNonInteractiveProvider(); + const requestedModel = nonInteractive + ? deps.getNonInteractiveModel(requestedProvider || "build") + : null; + const recoveredRegistryRoute = + rebuildRegistryInferenceRoute?.sandboxName === sandboxName && + rebuildRegistryInferenceRoute.route.source === "registry" + ? rebuildRegistryInferenceRoute.route + : null; + const recoveredProbeProvider = + nonInteractive && !requestedProvider && recoverProvider + ? (recoveredRegistryRoute?.provider ?? deps.readRecordedProvider(sandboxName)) + : null; + const recoveredProbeKey = providerNameToOptionKey( + deps.remoteProviderConfig, + recoveredProbeProvider, + { + hasNimContainer: + recoveredProbeProvider === "vllm-local" && + Boolean(deps.readRecordedNimContainer(sandboxName)), + }, + ); + const providerIntentKey = + requestedProvider || recoveredProbeKey || (nonInteractive ? "build" : null); + const intent = localProviderProbeIntent(providerIntentKey); + const guardedProvider = localProbeRouteProvider(providerIntentKey); + if (guardedProvider && assertRouteCompatible) { + const recoveredModel = + recoveredRegistryRoute?.model ?? + (!requestedProvider && recoverProvider ? deps.readRecordedModel(sandboxName) : null); + assertRouteCompatible({ + provider: guardedProvider, + model: requestedModel || recoveredModel, + endpointUrl: null, + preferredInferenceApi: null, + credentialEnv: null, + }); + } + const ollamaPreflightPassed = + guardedProvider === "ollama-local" && Boolean(assertRouteCompatible); + const vllmPreflightPassed = guardedProvider === "vllm-local" && Boolean(assertRouteCompatible); + return { + requestedProvider, + requestedModel, + recoveredRegistryRoute, + probeOllama: + intent.ollama && (ollamaPreflightPassed || (canProbeRoute?.("ollama-local") ?? true)), + probeVllm: intent.vllm && (vllmPreflightPassed || (canProbeRoute?.("vllm-local") ?? true)), + }; +} + export function createSetupNim( defaults: SetupNimFlowDeps, overrides: Partial = {}, @@ -166,6 +294,10 @@ export function createSetupNim( recoverProvider = true, rebuildRegistryInferenceRoute: RebuildRouteHandoff | null = null, gatewayName: string | null = null, + assertRouteCompatible?: ( + route: ProviderInferenceProbeRoute, + ) => GatewayRouteDiscoveryConstraints, + canProbeRoute?: (provider: string) => boolean, ): Promise { deps.step(3, 8, "Configuring inference provider"); @@ -181,10 +313,60 @@ export function createSetupNim( let allowToolsIncompatible = false; let reuseGatewayCredential = false; const nvidiaFeaturedModels = deps.createNvidiaFeaturedModelSession(); + const createSelectionState = (): SetupNimSelectionState => { + const state: SetupNimSelectionState = { + model, + provider, + endpointUrl, + credentialEnv, + hermesAuthMethod, + hermesToolGateways, + preferredInferenceApi, + compatibleEndpointReasoning, + nimContainer, + allowToolsIncompatible, + nvidiaFeaturedModels, + }; + state.assertRouteCompatible = () => { + const effectiveInferenceApi = () => + deps.resolveAgentInferenceApi( + agent?.name ?? null, + state.provider, + deps.coerceAgentInferenceApi(agent, state.preferredInferenceApi), + ); + const route = (): ProviderInferenceProbeRoute => ({ + provider: state.provider, + model: typeof state.model === "string" && state.model.trim() ? state.model.trim() : null, + endpointUrl: state.endpointUrl, + preferredInferenceApi: effectiveInferenceApi(), + credentialEnv: state.credentialEnv, + }); + const constraints = assertRouteCompatible?.(route()) ?? { + requiredModel: null, + requiredEndpointUrl: null, + requiredInferenceApi: null, + }; + applyGatewayRouteDiscoveryConstraints(state, constraints); + assertRouteCompatible?.(route()); + return constraints; + }; + return state; + }; + const { requestedProvider, requestedModel, recoveredRegistryRoute, probeOllama, probeVllm } = + prepareProviderDiscovery({ + deps, + sandboxName, + recoverProvider, + rebuildRegistryInferenceRoute, + assertRouteCompatible, + canProbeRoute, + }); const providerHostState = deps.detectInferenceProviderHostState({ gpu, experimental: deps.experimental, + probeOllama, + probeVllm, }); const { hasOllama, @@ -204,15 +386,6 @@ export function createSetupNim( ollamaInstallMenu, gpuNimCapable, } = providerHostState; - const requestedProvider = deps.getNonInteractiveProvider(); - const requestedModel = deps.isNonInteractive() - ? deps.getNonInteractiveModel(requestedProvider || "build") - : null; - const recoveredRegistryRoute = - rebuildRegistryInferenceRoute?.sandboxName === sandboxName && - rebuildRegistryInferenceRoute.route.source === "registry" - ? rebuildRegistryInferenceRoute.route - : null; const agentProviderOptions = deps.getAgentInferenceProviderOptions(agent); const blueprintRouterCfg = deps.loadRoutedProfile(); @@ -253,6 +426,7 @@ export function createSetupNim( let selected: ProviderMenuChoice | undefined; let recoveredFromSandbox = false; let recoveredModel: string | null = null; + let preparedVllmState: SetupNimSelectionState | null = null; hermesAuthMethod = null; if (deps.isNonInteractive() || requestedProvider) { @@ -308,19 +482,7 @@ export function createSetupNim( } if (deps.remoteProviderConfig[selected.key]) { - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - compatibleEndpointReasoning, - nimContainer, - allowToolsIncompatible, - nvidiaFeaturedModels, - }; + const state = createSelectionState(); const result = await deps.handleRemoteProviderSelection( { selected, @@ -348,17 +510,7 @@ export function createSetupNim( if (result === "retry-selection") continue selectionLoop; break; } else if (selected.key === "nim-local") { - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - nimContainer, - allowToolsIncompatible, - }; + const state = createSelectionState(); const result = await deps.handleNimLocalSelection( gpu, { requestedModel, recoveredFromSandbox, recoveredModel }, @@ -380,17 +532,7 @@ export function createSetupNim( if (rejectWindowsHostOllama(selected.key, isWindowsHostOllama)) { continue selectionLoop; } - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - nimContainer, - allowToolsIncompatible, - }; + const state = createSelectionState(); const result = await deps.handleRunningOllamaSelection( gpu, requestedModel, @@ -412,17 +554,7 @@ export function createSetupNim( if (rejectWindowsHostOllama(selected.key, true)) { continue selectionLoop; } - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - nimContainer, - allowToolsIncompatible, - }; + const state = createSelectionState(); const result = await deps.handleWindowsHostOllamaSelection( gpu, selected.key, @@ -443,17 +575,7 @@ export function createSetupNim( if (result === "retry-selection") continue selectionLoop; break; } else if (selected.key === "install-ollama") { - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - nimContainer, - allowToolsIncompatible, - }; + const state = createSelectionState(); const result = await deps.handleInstallOllamaSelection( gpu, requestedModel, @@ -477,10 +599,20 @@ export function createSetupNim( if (deps.isNonInteractive()) deps.exitProcess(1); continue selectionLoop; } + const vllmState = createSelectionState(); + preparedVllmState = vllmState; const result = await deps.installVllm(vllmProfile, { hasImage: hasVllmImage, nonInteractive: deps.isNonInteractive(), promptFn: deps.prompt, + beforeInstall: (modelId) => { + vllmState.provider = "vllm-local"; + vllmState.model = modelId; + vllmState.endpointUrl = null; + vllmState.credentialEnv = null; + vllmState.preferredInferenceApi = "openai-completions"; + vllmState.assertRouteCompatible?.(); + }, }); if (!result.ok) { if (deps.isNonInteractive()) @@ -493,17 +625,8 @@ export function createSetupNim( }; } if (selected.key === "vllm") { - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - nimContainer, - allowToolsIncompatible, - }; + const state = preparedVllmState ?? createSelectionState(); + state.model = preparedVllmState?.model ?? requestedModel ?? recoveredModel; const result = await deps.handleVllmSelection(state); ({ model, @@ -517,17 +640,7 @@ export function createSetupNim( if (result === "retry-selection") continue selectionLoop; break; } else if (selected.key === "routed") { - const state: SetupNimSelectionState = { - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - preferredInferenceApi, - nimContainer, - allowToolsIncompatible, - }; + const state = createSelectionState(); const result = await deps.handleRoutedSelection(state); ({ model, @@ -558,7 +671,11 @@ export function createSetupNim( credentialEnv, hermesAuthMethod, hermesToolGateways, - preferredInferenceApi: deps.coerceAgentInferenceApi(agent, preferredInferenceApi), + preferredInferenceApi: deps.resolveAgentInferenceApi( + agent?.name ?? null, + provider, + deps.coerceAgentInferenceApi(agent, preferredInferenceApi), + ), compatibleEndpointReasoning, nimContainer, allowToolsIncompatible, diff --git a/src/lib/onboard/setup-nim-ollama.test.ts b/src/lib/onboard/setup-nim-ollama.test.ts index 69ead2d4023..efd93ece2ac 100644 --- a/src/lib/onboard/setup-nim-ollama.test.ts +++ b/src/lib/onboard/setup-nim-ollama.test.ts @@ -3,7 +3,7 @@ import assert from "node:assert/strict"; -import { describe, it, vi } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { createSetupNimOllamaHandlers } from "./setup-nim-ollama"; import type { SetupNimSelectionState } from "./setup-nim-selection"; @@ -61,6 +61,92 @@ function makeDeps(overrides: Partial = {}): Deps { } describe("createSetupNimOllamaHandlers", () => { + it("guards the selected route before systemd recovery and model preparation (#6315)", async () => { + const events: string[] = []; + const state = makeState(); + state.assertRouteCompatible = () => { + events.push(`guard:${String(state.model)}`); + return { + requiredModel: "required/model", + requiredEndpointUrl: null, + requiredInferenceApi: null, + }; + }; + const { handleRunningOllamaSelection } = createSetupNimOllamaHandlers( + makeDeps({ + ensureOllamaLoopbackSystemdOverride: () => { + events.push("systemd"); + return "unchanged"; + }, + selectAndValidateOllamaModel: async (_gpu, _provider, args, onModelSelected) => { + expect(args.lockedModel).toBe("required/model"); + events.push("prepare-model"); + onModelSelected?.("required/model"); + return { outcome: "selected", model: "required/model", allowToolsIncompatible: false }; + }, + }), + ); + + await handleRunningOllamaSelection(null, "required/model", null, true, state); + + expect(events).toEqual([ + "guard:required/model", + "systemd", + "prepare-model", + "guard:required/model", + ]); + }); + + it("does not install Ollama when shared-gateway preflight rejects", async () => { + const state = makeState(); + state.assertRouteCompatible = () => { + throw new Error("route conflict"); + }; + const install = vi.fn(() => ({ ok: true })); + const { handleInstallOllamaSelection } = createSetupNimOllamaHandlers( + makeDeps({ installOllamaOnLinux: install, installOllamaOnMacOS: install }), + ); + + await expect( + handleInstallOllamaSelection(null, "conflict/model", null, state, { + hasUpgradableOllama: false, + }), + ).rejects.toThrow("route conflict"); + expect(install).not.toHaveBeenCalled(); + }); + + it("does not switch, install, or restart Windows Ollama when preflight rejects", async () => { + const state = makeState(); + state.assertRouteCompatible = () => { + throw new Error("route conflict"); + }; + const switchHost = vi.fn(); + const install = vi.fn(async () => ({ ok: true })); + const restart = vi.fn(() => true); + const { handleWindowsHostOllamaSelection } = createSetupNimOllamaHandlers( + makeDeps({ + switchToWindowsOllamaHost: switchHost, + installOllamaOnWindowsHost: install, + setupWindowsOllamaWith0000Binding: restart, + }), + ); + + await expect( + handleWindowsHostOllamaSelection( + null, + "start-windows-ollama", + "conflict/model", + true, + false, + null, + state, + ), + ).rejects.toThrow("route conflict"); + expect(switchHost).not.toHaveBeenCalled(); + expect(install).not.toHaveBeenCalled(); + expect(restart).not.toHaveBeenCalled(); + }); + it("preserves accepted tools-incompatible state for running Ollama", async () => { const state = makeState(); const { handleRunningOllamaSelection } = createSetupNimOllamaHandlers(makeDeps()); diff --git a/src/lib/onboard/setup-nim-ollama.ts b/src/lib/onboard/setup-nim-ollama.ts index dd974950389..c6c29e70370 100644 --- a/src/lib/onboard/setup-nim-ollama.ts +++ b/src/lib/onboard/setup-nim-ollama.ts @@ -26,7 +26,12 @@ type SetupNimOllamaDeps = { selectAndValidateOllamaModel: ( gpu: any, provider: string, - args: { requestedModel: string | null; recoveredModel: string | null }, + args: { + requestedModel: string | null; + recoveredModel: string | null; + lockedModel?: string | null; + }, + onModelSelected?: (model: string) => void, ) => Promise< | { outcome: "back-to-selection" } | { outcome: "selected"; model: string; allowToolsIncompatible: boolean } @@ -83,11 +88,23 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { state: SetupNimSelectionState, requestedModel: string | null, recoveredModel: string | null, + lockedModel: string | null, ): Promise { - const result = await deps.selectAndValidateOllamaModel(gpu, state.provider, { - requestedModel, - recoveredModel, - }); + const constrainedModel = typeof state.model === "string" ? state.model : requestedModel; + const result = await deps.selectAndValidateOllamaModel( + gpu, + state.provider, + { + requestedModel: constrainedModel, + recoveredModel, + lockedModel, + }, + (model) => { + state.model = model; + state.preferredInferenceApi = "openai-completions"; + state.assertRouteCompatible?.(); + }, + ); if (result.outcome === "back-to-selection") return "retry-selection"; state.model = result.model; state.allowToolsIncompatible = result.allowToolsIncompatible; @@ -110,6 +127,7 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { state.provider = "ollama-local"; state.credentialEnv = null; state.endpointUrl = deps.getLocalProviderBaseUrl(state.provider); + state.preferredInferenceApi = "openai-completions"; state.skipHostInferenceSmoke = false; if (!state.endpointUrl) { console.error(" Local Ollama base URL could not be determined."); @@ -117,6 +135,16 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { } } + function preflightOllamaRoute( + state: SetupNimSelectionState, + requestedModel: string | null, + recoveredModel: string | null, + ): string | null { + configureOllamaState(state); + state.model = requestedModel || recoveredModel; + return state.assertRouteCompatible?.().requiredModel ?? null; + } + function applyOllamaFallbackState( state: SetupNimSelectionState, result: Extract["result"], @@ -143,6 +171,7 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { if (!deps.checkOllamaPortsOrWarn({ isNonInteractive: deps.isNonInteractive })) { return "retry-selection"; } + const lockedModel = preflightOllamaRoute(state, requestedModel, null); const isInstall = selectedKey === "install-windows-ollama"; const isSwitch = !isInstall && windowsOllamaReachable; const isRestart = !isInstall && !isSwitch && winOllamaLoopbackOnly; @@ -192,8 +221,7 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { } console.log(` ✓ Using Ollama on host.docker.internal:${deps.OLLAMA_PORT}`); } - configureOllamaState(state); - const result = await selectModel(gpu, state, requestedModel, null); + const result = await selectModel(gpu, state, requestedModel, null, lockedModel); if (result === "retry-selection") deps.resetOllamaHostCache(); return result; } @@ -208,6 +236,8 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { if (!deps.checkOllamaPortsOrWarn({ isNonInteractive: deps.isNonInteractive })) { return "retry-selection"; } + const initialState = { ...state, hermesToolGateways: [...state.hermesToolGateways] }; + const lockedModel = preflightOllamaRoute(state, requestedModel, recoveredModel); let ollamaReady = ollamaRunning; const overrideState = deps.ensureOllamaLoopbackSystemdOverride({ isNonInteractive: deps.isNonInteractive, @@ -236,13 +266,14 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { // Fallback crosses a provider boundary, so write a complete safe state // rather than merging over stale cloud/NIM/Ollama selection fields. applyOllamaFallbackState(state, startup.result); + state.assertRouteCompatible?.(); return "selected"; case "ready": startProxyOrAnnounceDirect(); - configureOllamaState(state); - return selectModel(gpu, state, requestedModel, recoveredModel); + return selectModel(gpu, state, requestedModel, recoveredModel, lockedModel); default: { const kind = (startup as { kind?: unknown }).kind; + Object.assign(state, initialState); console.error(` Unknown Ollama startup outcome: ${String(kind)}`); deps.process.exit(1); } @@ -259,6 +290,7 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { if (!deps.checkOllamaPortsOrWarn({ isNonInteractive: deps.isNonInteractive })) { return "retry-selection"; } + const lockedModel = preflightOllamaRoute(state, requestedModel, recoveredModel); const isUpgrade = ollamaInstallMenu.hasUpgradableOllama; const installResult = deps.process.platform === "darwin" @@ -276,8 +308,7 @@ export function createSetupNimOllamaHandlers(deps: SetupNimOllamaDeps): { return "retry-selection"; } startProxyOrAnnounceDirect(); - configureOllamaState(state); - return selectModel(gpu, state, requestedModel, recoveredModel); + return selectModel(gpu, state, requestedModel, recoveredModel, lockedModel); } return { diff --git a/src/lib/onboard/setup-nim-selection.ts b/src/lib/onboard/setup-nim-selection.ts index 174ccfb0ce3..5d38d0fbed8 100644 --- a/src/lib/onboard/setup-nim-selection.ts +++ b/src/lib/onboard/setup-nim-selection.ts @@ -1,6 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +import type { GatewayRouteDiscoveryConstraints } from "../inference/gateway-route-compatibility"; import type { NvidiaFeaturedModelSession } from "./nvidia-featured-model-selection"; export { createNvidiaFeaturedModelSession } from "./nvidia-featured-model-selection"; @@ -21,6 +22,8 @@ export type SetupNimSelectionState = { skipHostInferenceSmoke?: boolean; reuseGatewayCredentialWithoutLocalKey?: boolean; nvidiaFeaturedModels?: NvidiaFeaturedModelSession; + /** Attempt-wide shared-gateway guard, invoked after identity selection and before probes. */ + assertRouteCompatible?: () => GatewayRouteDiscoveryConstraints; }; export type CloudFallbackConfig = { diff --git a/src/lib/onboard/setup-nim-vllm.test.ts b/src/lib/onboard/setup-nim-vllm.test.ts new file mode 100644 index 00000000000..e4648d1d3aa --- /dev/null +++ b/src/lib/onboard/setup-nim-vllm.test.ts @@ -0,0 +1,87 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { SetupNimSelectionState } from "./setup-nim-flow"; +import { createSetupNimVllmHandler, type SetupNimVllmDeps } from "./setup-nim-vllm"; + +function state(model: string | null): SetupNimSelectionState { + return { + model, + provider: "nvidia-prod", + endpointUrl: null, + credentialEnv: null, + hermesAuthMethod: null, + hermesToolGateways: [], + preferredInferenceApi: null, + nimContainer: null, + allowToolsIncompatible: false, + }; +} + +function deps(overrides: Partial = {}): SetupNimVllmDeps { + return { + VLLM_PORT: 8000, + runCapture: () => JSON.stringify({ data: [{ id: "served/model" }] }), + getLocalProviderBaseUrl: () => "http://host.openshell.internal:8000/v1", + getLocalProviderValidationBaseUrl: () => "http://127.0.0.1:8000/v1", + isSafeModelId: () => true, + requireValue: (value) => { + if (value == null) throw new Error("missing value"); + return value; + }, + validateOpenAiLikeSelection: async () => ({ ok: true, api: "openai-completions" }), + applyVllmRuntimeContextWindow: vi.fn(), + exitProcess: (code) => { + throw new Error(`exit ${code}`); + }, + ...overrides, + }; +} + +beforeEach(() => { + vi.spyOn(console, "log").mockImplementation(() => {}); + vi.spyOn(console, "error").mockImplementation(() => {}); +}); + +afterEach(() => vi.restoreAllMocks()); + +describe("setupNim vLLM route containment", () => { + it("preflights before discovery and exact-checks the detected model before validation (#6315)", async () => { + const events: string[] = []; + const selection = state(null); + selection.assertRouteCompatible = () => { + events.push(selection.model ? "exact" : "preflight"); + return { requiredModel: null, requiredEndpointUrl: null, requiredInferenceApi: null }; + }; + const handler = createSetupNimVllmHandler( + deps({ + runCapture: () => { + events.push("probe"); + return JSON.stringify({ data: [{ id: "served/model" }] }); + }, + validateOpenAiLikeSelection: async () => { + events.push("validate"); + return { ok: true, api: "openai-completions" }; + }, + }), + ); + + await expect(handler(selection)).resolves.toBe("selected"); + expect(events).toEqual(["preflight", "probe", "exact", "validate"]); + }); + + it("rejects a detected model that differs from the durable shared route before validation", async () => { + const validate = vi.fn(async () => ({ ok: true })); + const selection = state("required/model"); + selection.assertRouteCompatible = () => ({ + requiredModel: "required/model", + requiredEndpointUrl: null, + requiredInferenceApi: null, + }); + const handler = createSetupNimVllmHandler(deps({ validateOpenAiLikeSelection: validate })); + + await expect(handler(selection)).rejects.toThrow("exit 1"); + expect(validate).not.toHaveBeenCalled(); + }); +}); diff --git a/src/lib/onboard/setup-nim-vllm.ts b/src/lib/onboard/setup-nim-vllm.ts new file mode 100644 index 00000000000..38e3624a18e --- /dev/null +++ b/src/lib/onboard/setup-nim-vllm.ts @@ -0,0 +1,101 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import type { SetupNimSelectionResult, SetupNimSelectionState } from "./setup-nim-flow"; + +type VllmModels = { data?: Array<{ id?: unknown }> }; + +export interface SetupNimVllmDeps { + VLLM_PORT: number; + runCapture(args: string[], options: { ignoreError: boolean }): string; + getLocalProviderBaseUrl(provider: string): string | null; + getLocalProviderValidationBaseUrl(provider: string): string | null; + isSafeModelId(model: string): boolean; + requireValue(value: T | null | undefined, message: string): T; + validateOpenAiLikeSelection( + label: string, + endpointUrl: string, + model: string, + credentialEnv: string | null, + ): Promise<{ ok: boolean; retry?: string; api?: string | null }>; + applyVllmRuntimeContextWindow(models: VllmModels, model: string): void; + exitProcess(code: number): never; +} + +export function createSetupNimVllmHandler( + deps: SetupNimVllmDeps, +): (state: SetupNimSelectionState) => Promise { + return async function handleVllmSelection( + state: SetupNimSelectionState, + ): Promise { + console.log(` ✓ Using existing vLLM on localhost:${deps.VLLM_PORT}`); + state.provider = "vllm-local"; + state.credentialEnv = null; + state.endpointUrl = deps.getLocalProviderBaseUrl(state.provider); + if (!state.endpointUrl) { + console.error(" Local vLLM base URL could not be determined."); + deps.exitProcess(1); + } + state.preferredInferenceApi = "openai-completions"; + state.assertRouteCompatible?.(); + const requiredModel = typeof state.model === "string" ? state.model : null; + + const raw = deps.runCapture(["curl", "-sf", `http://127.0.0.1:${deps.VLLM_PORT}/v1/models`], { + ignoreError: true, + }); + let models: VllmModels; + try { + models = JSON.parse(raw); + } catch { + console.error( + ` Could not query vLLM models endpoint. Is vLLM running on localhost:${deps.VLLM_PORT}?`, + ); + deps.exitProcess(1); + } + const detectedModel = + models.data && models.data.length > 0 && typeof models.data[0]?.id === "string" + ? models.data[0].id + : null; + if (!detectedModel) { + console.error(" Could not detect model from vLLM. Please specify manually."); + deps.exitProcess(1); + } + if (!deps.isSafeModelId(detectedModel)) { + console.error(" Detected vLLM model ID contains invalid characters."); + deps.exitProcess(1); + } + if (requiredModel && detectedModel !== requiredModel) { + console.error( + ` Detected vLLM model '${detectedModel}' does not match the shared gateway route '${requiredModel}'.`, + ); + deps.exitProcess(1); + } + state.model = detectedModel; + state.assertRouteCompatible?.(); + console.log(` Detected model: ${state.model}`); + + const validationBaseUrl = deps.getLocalProviderValidationBaseUrl(state.provider); + if (!validationBaseUrl) { + console.error(" Local vLLM validation URL could not be determined."); + deps.exitProcess(1); + } + const validation = await deps.validateOpenAiLikeSelection( + "Local vLLM", + validationBaseUrl, + deps.requireValue(state.model, "Expected a detected vLLM model"), + null, + ); + if (validation.retry === "selection" || validation.retry === "model" || !validation.ok) { + return "retry-selection"; + } + + deps.applyVllmRuntimeContextWindow(models, state.model); + if (validation.api !== "openai-completions") { + console.log( + " ℹ Using chat completions API (tool-call-parser requires /v1/chat/completions)", + ); + } + state.preferredInferenceApi = "openai-completions"; + return "selected"; + }; +} diff --git a/test/onboard-fsm-live-slices.test.ts b/test/onboard-fsm-live-slices.test.ts index 2611dd628ef..63cc6e2944e 100644 --- a/test/onboard-fsm-live-slices.test.ts +++ b/test/onboard-fsm-live-slices.test.ts @@ -17,6 +17,7 @@ type ProbeMode = | "endpoint-override" | "resume-initial" | "resume-core-gateway" + | "resume-incomplete-core-gateway" | "authoritative-core-gateway" | "ahead-core"; @@ -195,7 +196,7 @@ function machine(state, revision = 1) { return { version: 1, state, stateEnteredAt: null, revision }; } -function seedResumeSession(state) { +function seedResumeSession(state, sandboxComplete = true) { const session = onboardSession.createSession({ mode: "non-interactive", sandboxName: "fsm-sandbox", @@ -204,9 +205,10 @@ function seedResumeSession(state) { machine: machine(state), metadata: { gatewayName: "nemoclaw", fromDockerfile: null }, }); - for (const step of ["preflight", "gateway", "provider_selection", "sandbox"]) { + for (const step of ["preflight", "gateway", "provider_selection"]) { session.steps[step].status = "complete"; } + if (sandboxComplete) session.steps.sandbox.status = "complete"; onboardSession.saveSession(session); } @@ -235,7 +237,7 @@ function baseContext(context, overrides = {}) { } preflightHandlers.handlePreflightState = async (options) => { - if (scenario.mode === "resume-core-gateway" || scenario.mode === "authoritative-core-gateway") { + if (scenario.mode.includes("core-gateway")) { return { gpu: null, sandboxGpuConfig: { sandboxGpuEnabled: false, mode: "0" }, @@ -257,7 +259,7 @@ preflightHandlers.handlePreflightState = async (options) => { }; gatewayHandlers.handleGatewayState = async (options) => { - if (scenario.mode !== "resume-core-gateway" && scenario.mode !== "authoritative-core-gateway") { + if (!scenario.mode.includes("core-gateway")) { throw new Error("unexpected gateway compatibility handler"); } called.push("gateway:" + options.gatewayName + ":" + process.env.OPENSHELL_GATEWAY); @@ -269,7 +271,7 @@ gatewayHandlers.handleGatewayState = async (options) => { }; providerHandlers.handleProviderInferenceState = async (options) => { - if (!["ahead-core", "resume-core-gateway", "authoritative-core-gateway"].includes(scenario.mode)) { + if (scenario.mode !== "ahead-core" && !scenario.mode.includes("core-gateway")) { throw new Error("unexpected provider compatibility handler"); } called.push( @@ -319,10 +321,10 @@ flowSlices.runFinalOnboardFlowSequence = async ({ context }) => { if (scenario.mode === "resume-initial") { seedResumeSession("preflight"); } -if (scenario.mode === "resume-core-gateway" || scenario.mode === "authoritative-core-gateway") { - seedResumeSession("inference"); +if (scenario.mode.includes("core-gateway")) { + seedResumeSession("inference", scenario.mode !== "resume-incomplete-core-gateway"); } -if (scenario.mode === "resume-core-gateway") { +if (scenario.mode === "resume-core-gateway" || scenario.mode === "resume-incomplete-core-gateway") { registry.registerSandbox({ name: "fsm-sandbox", provider: "openai-api", @@ -447,6 +449,13 @@ describe("live onboard FSM slice boundaries", () => { ]); }); + it("routes an incomplete registered resume through its requested sandbox gateway", () => { + assert.deepEqual(runSliceProbe({ slice: "core", mode: "resume-incomplete-core-gateway" }), [ + "gateway:nemoclaw-9090:nemoclaw-9090", + "provider-compat:nemoclaw-9090", + ]); + }); + it("keeps an authoritative rebuild gateway after the registry row is removed", () => { assert.deepEqual(runSliceProbe({ slice: "core", mode: "authoritative-core-gateway" }), [ "gateway:nemoclaw-9090:nemoclaw-9090", diff --git a/test/onboard-nim-route-containment.test.ts b/test/onboard-nim-route-containment.test.ts new file mode 100644 index 00000000000..8d1b3b3abe8 --- /dev/null +++ b/test/onboard-nim-route-containment.test.ts @@ -0,0 +1,120 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +import { describe, it } from "vitest"; + +import { testTimeout } from "./helpers/timeouts"; + +describe("onboard provider-discovery route containment", () => { + it( + "rejects NIM and custom routes before provisioning, credentials, or endpoint probes (#6315)", + () => { + const repoRoot = path.join(import.meta.dirname, ".."); + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-nim-route-guard-")); + const scriptPath = path.join(tmpDir, "nim-route-guard.js"); + const onboardPath = JSON.stringify(path.join(repoRoot, "src", "lib", "onboard.ts")); + const credentialsPath = JSON.stringify( + path.join(repoRoot, "src", "lib", "credentials", "store.ts"), + ); + const runnerPath = JSON.stringify(path.join(repoRoot, "src", "lib", "runner.ts")); + const nimPath = JSON.stringify(path.join(repoRoot, "src", "lib", "inference", "nim.ts")); + const script = String.raw` +const runner = require(${runnerPath}); +const credentials = require(${credentialsPath}); +const nim = require(${nimPath}); +const calls = []; +runner.runCapture = (command) => { + const text = Array.isArray(command) ? command.join(" ") : String(command); + if (/https?:|\/v1\/models|\/api\/tags/.test(text)) calls.push("endpoint-probe"); + return ""; +}; +credentials.prompt = async () => { calls.push("credential-prompt"); return ""; }; +credentials.saveCredential = () => { calls.push("credential-save"); }; +nim.listModels = () => [{ + name: "nvidia/nemotron-3-nano-30b-a3b", + image: "fake", + minGpuMemoryMB: 8000, +}]; +nim.isNgcLoggedIn = () => { calls.push("ngc-login-check"); return true; }; +nim.dockerLoginNgc = () => { calls.push("ngc-login"); return true; }; +nim.pullNimImage = () => { calls.push("pull"); return "image"; }; +nim.containerName = () => { calls.push("container-name"); return "nim-test"; }; +nim.startNimContainerByName = () => { calls.push("start"); return "nim-test"; }; +nim.waitForNimHealth = () => { calls.push("health"); return true; }; +nim.adoptServedModelId = () => { calls.push("served-model"); return "served/model"; }; +const { setupNim } = require(${onboardPath}); +(async () => { + const originalLog = console.log; + console.log = () => {}; + const runGuarded = async (gpu) => { + calls.length = 0; + let message = ""; + let route = null; + try { + await setupNim(gpu, null, null, true, null, "nemoclaw", (candidate) => { + route = candidate; + calls.push("guard"); + throw new Error("route conflict"); + }, () => false); + } catch (error) { + message = error instanceof Error ? error.message : String(error); + } + return { calls: [...calls], message, route }; + }; + const nimResult = await runGuarded({ + type: "nvidia", + totalMemoryMB: 16000, + nimCapable: true, + }); + process.env.NEMOCLAW_PROVIDER = "custom"; + process.env.NEMOCLAW_MODEL = "custom/model"; + process.env.NEMOCLAW_ENDPOINT_URL = "https://custom.example.test/v1"; + const customResult = await runGuarded(null); + console.log = originalLog; + originalLog(JSON.stringify({ customResult, nimResult })); +})().catch((error) => { + console.error(error); + process.exit(1); +}); +`; + fs.writeFileSync(scriptPath, script); + try { + const result = spawnSync(process.execPath, [scriptPath], { + cwd: repoRoot, + encoding: "utf-8", + env: { + ...process.env, + HOME: tmpDir, + NEMOCLAW_EXPERIMENTAL: "1", + NEMOCLAW_MODEL: "nvidia/nemotron-3-nano-30b-a3b", + NEMOCLAW_NON_INTERACTIVE: "1", + NEMOCLAW_PROVIDER: "nim-local", + }, + }); + assert.equal(result.status, 0, result.stderr); + const payload = JSON.parse(result.stdout.trim()); + assert.equal(payload.nimResult.message, "route conflict"); + assert.deepEqual(payload.nimResult.calls, ["guard"]); + assert.equal(payload.nimResult.route.provider, "vllm-local"); + assert.equal(payload.customResult.message, "route conflict"); + assert.deepEqual(payload.customResult.calls, ["guard"]); + assert.deepEqual(payload.customResult.route, { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "https://custom.example.test/v1", + preferredInferenceApi: "openai-completions", + credentialEnv: "COMPATIBLE_API_KEY", + }); + } finally { + fs.rmSync(tmpDir, { recursive: true, force: true }); + } + }, + testTimeout(60_000), + ); +}); diff --git a/test/onboard-remote-recreate-credential-reuse.test.ts b/test/onboard-remote-recreate-credential-reuse.test.ts index 50c4b25a1d8..cc29ab1723a 100644 --- a/test/onboard-remote-recreate-credential-reuse.test.ts +++ b/test/onboard-remote-recreate-credential-reuse.test.ts @@ -172,8 +172,8 @@ const { setupNim, setupInference } = require(${onboardPath}); }); assert.deepEqual( curlUrls, - ["http://127.0.0.1:11434/api/tags", "http://127.0.0.1:8000/v1/models"], - `only exact loopback discovery probes may run without a local credential: ${curlLog}`, + [], + `remote recovery must not run unrelated local endpoint probes: ${curlLog}`, ); const openshellLog = fs.readFileSync(openshellLogPath, "utf8"); assert.match(openshellLog, /provider get -g nemoclaw compatible-endpoint/); diff --git a/test/onboard.test.ts b/test/onboard.test.ts index 0731ed67ed8..a6ce46ba717 100644 --- a/test/onboard.test.ts +++ b/test/onboard.test.ts @@ -1197,19 +1197,7 @@ const { onboard } = require(${onboardPath}); }); it("recovers the Ollama auth proxy on WSL when the sandbox needs proxy fronting", async () => { const proxyCalls: string[] = []; - let harness: ReturnType; - const applyLocalInferenceRoute = createLocalInferenceRouteApplier({ - runOpenshell: (args, options) => harness.runOpenshell(args, options), - isNonInteractive: () => false, - promptValidationRecovery: async () => "selection", - classifyApplyFailure: () => ({}) as never, - compactText: (value) => value.trim(), - redact: (value) => value, - localInferenceTimeoutSecs: 120, - error: vi.fn(), - exitProcess: () => assert.fail("unexpected exit"), - }); - harness = createDirectSetupInferenceHarness({ + const harness = createDirectSetupInferenceHarness({ runOpenshell: (args) => args.slice(0, 2).join(" ") === "provider get" ? { status: 1, stdout: "", stderr: "" } @@ -1230,7 +1218,7 @@ const { onboard } = require(${onboardPath}); persistAndProbeOllamaProxy: async (token: string) => { proxyCalls.push(`persist:${token}`); }, - applyLocalInferenceRoute, + applyLocalInferenceRoute: undefined, }, }); const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); diff --git a/test/support/connect-flow-test-harness.ts b/test/support/connect-flow-test-harness.ts index 4e169b3185d..cadf71019fe 100644 --- a/test/support/connect-flow-test-harness.ts +++ b/test/support/connect-flow-test-harness.ts @@ -25,6 +25,7 @@ export type ConnectHarness = { checkAndRecoverSpy: MockInstance; connectSandbox: ConnectSandbox; ensureOllamaAuthProxySpy: MockInstance; + ensureLiveSandboxSpy: MockInstance; errorSpy: MockInstance; logSpy: MockInstance; preflightVllmSpy: MockInstance; @@ -106,7 +107,7 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne const preflightVllmSpy = vi .spyOn(connectVllmPreflight, "preflightVllmModelEnvOrExit") .mockImplementation(() => undefined); - vi.spyOn(gatewayState, "ensureLiveSandboxOrExit").mockResolvedValue({ + const ensureLiveSandboxSpy = vi.spyOn(gatewayState, "ensureLiveSandboxOrExit").mockResolvedValue({ state: "present", output: "Name: alpha\nPhase: Ready\n", }); @@ -201,6 +202,7 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne checkAndRecoverSpy, connectSandbox: requireDist(connectModulePath).connectSandbox, ensureOllamaAuthProxySpy, + ensureLiveSandboxSpy, errorSpy, logSpy, preflightVllmSpy, From cbaf6227effc0a05630f7ed757615dfe60046916 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 16:11:01 -0700 Subject: [PATCH 18/31] test(onboard): reuse shared value guard Signed-off-by: Aaron Erickson --- src/lib/onboard/setup-nim-vllm.test.ts | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/lib/onboard/setup-nim-vllm.test.ts b/src/lib/onboard/setup-nim-vllm.test.ts index e4648d1d3aa..3713b534576 100644 --- a/src/lib/onboard/setup-nim-vllm.test.ts +++ b/src/lib/onboard/setup-nim-vllm.test.ts @@ -2,6 +2,7 @@ // SPDX-License-Identifier: Apache-2.0 import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { requireValue } from "../core/require-value"; import type { SetupNimSelectionState } from "./setup-nim-flow"; import { createSetupNimVllmHandler, type SetupNimVllmDeps } from "./setup-nim-vllm"; @@ -26,10 +27,7 @@ function deps(overrides: Partial = {}): SetupNimVllmDeps { getLocalProviderBaseUrl: () => "http://host.openshell.internal:8000/v1", getLocalProviderValidationBaseUrl: () => "http://127.0.0.1:8000/v1", isSafeModelId: () => true, - requireValue: (value) => { - if (value == null) throw new Error("missing value"); - return value; - }, + requireValue, validateOpenAiLikeSelection: async () => ({ ok: true, api: "openai-completions" }), applyVllmRuntimeContextWindow: vi.fn(), exitProcess: (code) => { From 38f7e0455b2e718c5b3665e32b5e0298e6e7379c Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 18:37:34 -0700 Subject: [PATCH 19/31] fix(inference): serialize shared gateway route mutations Signed-off-by: Aaron Erickson --- ...ence-set-gateway-route-containment.test.ts | 107 ++++++ src/lib/actions/inference-set.test-support.ts | 9 + src/lib/actions/inference-set.ts | 52 ++- .../sandbox/connect-route-containment.test.ts | 114 ++++++ src/lib/actions/sandbox/connect.ts | 55 ++- src/lib/actions/sandbox/snapshot.test.ts | 49 +++ src/lib/actions/sandbox/snapshot.ts | 73 +++- .../gateway-route-compatibility.test.ts | 8 +- .../inference/gateway-route-compatibility.ts | 21 +- .../gateway-route-mutation-lock.test.ts | 84 +++++ .../inference/gateway-route-mutation-lock.ts | 25 ++ src/lib/onboard.ts | 9 +- .../onboard/machine/core-flow-phases.test.ts | 34 +- src/lib/onboard/machine/core-flow-phases.ts | 1 + ...ovider-inference-route-containment.test.ts | 30 ++ .../handlers/provider-inference.test.ts | 4 + .../machine/handlers/provider-inference.ts | 40 ++- .../sandbox-route-mutation-lock.test.ts | 123 +++++++ .../machine/handlers/sandbox-test-fixtures.ts | 20 ++ .../onboard/machine/handlers/sandbox.test.ts | 18 + src/lib/onboard/machine/handlers/sandbox.ts | 259 ++++++++------ .../setup-inference-route-containment.test.ts | 99 +++++- src/lib/onboard/setup-inference.ts | 327 ++++++++++-------- src/lib/onboard/setup-nim-flow.test.ts | 2 +- src/lib/onboard/setup-nim-flow.ts | 2 +- src/lib/registry-recovery-action.ts | 112 +++++- .../registry-recovery-seeded-paths.test.ts | 22 +- .../state/registry-route-reservation.test.ts | 88 +++++ src/lib/state/registry.ts | 57 ++- test/onboard-fsm-live-slices.test.ts | 1 + test/onboard-messaging.test.ts | 8 +- test/onboard-prepared-gateway-handoff.test.ts | 4 +- test/support/connect-flow-test-harness.ts | 17 + 33 files changed, 1549 insertions(+), 325 deletions(-) create mode 100644 src/lib/inference/gateway-route-mutation-lock.test.ts create mode 100644 src/lib/inference/gateway-route-mutation-lock.ts create mode 100644 src/lib/onboard/machine/handlers/sandbox-route-mutation-lock.test.ts create mode 100644 src/lib/state/registry-route-reservation.test.ts diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts index bbc4a3a6145..c7349c37758 100644 --- a/src/lib/actions/inference-set-gateway-route-containment.test.ts +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -1,7 +1,11 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; import { afterEach, describe, expect, it, vi } from "vitest"; +import { withGatewayRouteMutationLock } from "../inference/gateway-route-mutation-lock"; import type { ConfigObject } from "../security/credential-filter"; import type { SandboxEntry } from "../state/registry"; import { runInferenceSet } from "./inference-set"; @@ -51,6 +55,26 @@ describe("runtime shared gateway route containment", () => { expect(deps.calls.appendAuditEntry).not.toHaveBeenCalled(); }); + it("rejects a pending onboarding route reservation before any mutation", async () => { + const deps = createDeps({ + config: {}, + entries: [entry("alpha", { pendingRouteReservation: true })], + defaultSandbox: "alpha", + }); + + await expect( + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-b", sandboxName: "alpha" }, + deps, + ), + ).rejects.toThrow("still being created by onboarding"); + + expect(deps.calls.prepareRunOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.writeSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + }); + it("rejects a same-gateway conflict before OpenShell, config, or registry mutation (#6315)", async () => { const deps = createDeps({ config: {}, @@ -185,6 +209,39 @@ describe("runtime shared gateway route containment", () => { expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); }); + it("rechecks a DNS-normalized endpoint before route mutation (#6315)", async () => { + const customRoute = { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: "http://public.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + } as const; + const deps = createDeps({ + config: {}, + entries: [entry("alpha", customRoute), entry("custom-peer", customRoute)], + defaultSandbox: "alpha", + rewriteConfigUrlsWithDnsPinning: async (value) => + typeof value === "string" ? "http://203.0.113.10/v1" : value, + }); + + await expect( + runInferenceSet( + { + ...customRoute, + sandboxName: "alpha", + inferenceApi: "openai-completions", + }, + deps, + ), + ).rejects.toThrow("custom-peer"); + + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).toHaveBeenCalledOnce(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + }); + it("blocks an incomplete legacy custom target even without a peer (#6315)", async () => { const deps = createDeps({ config: {}, @@ -319,4 +376,54 @@ describe("runtime shared gateway route containment", () => { expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); }); + + it("serializes same-gateway mutations and rechecks peers before the second write", async () => { + const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), "nemoclaw-route-lock-")); + try { + const entries = [ + entry("route-lock-alpha", { provider: null, model: null }), + entry("route-lock-beta", { provider: null, model: null }), + ]; + const deps = createDeps({ + config: { agents: { defaults: { model: {} } } }, + entries, + withGatewayRouteMutationLock: (gatewayName, operation) => + withGatewayRouteMutationLock(gatewayName, operation, { + stateDir, + pollIntervalMs: 1, + timeoutMs: 5_000, + }), + }); + deps.calls.updateSandbox.mockImplementation( + (sandboxName: string, updates: Partial) => { + const target = entries.find((candidate) => candidate.name === sandboxName); + expect(target).toBeDefined(); + Object.assign(target!, updates); + return true; + }, + ); + + const results = await Promise.allSettled([ + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-a", sandboxName: entries[0].name }, + deps, + ), + runInferenceSet( + { provider: "anthropic-prod", model: "claude-new", sandboxName: entries[1].name }, + deps, + ), + ]); + + expect(results.map((result) => result.status).sort()).toEqual(["fulfilled", "rejected"]); + expect( + deps.calls.captureOpenshell.mock.calls.filter( + ([args]) => args[0] === "inference" && args[1] === "set", + ), + ).toHaveLength(1); + expect(entries.filter((candidate) => candidate.provider && candidate.model)).toHaveLength(1); + expect(deps.calls.withGatewayRouteMutationLock).toHaveBeenCalledTimes(2); + } finally { + await fs.rm(stateDir, { recursive: true, force: true }); + } + }); }); diff --git a/src/lib/actions/inference-set.test-support.ts b/src/lib/actions/inference-set.test-support.ts index 6536fa83bf3..6e19b36d265 100644 --- a/src/lib/actions/inference-set.test-support.ts +++ b/src/lib/actions/inference-set.test-support.ts @@ -86,6 +86,7 @@ export function createDeps(options: { prepareRunOpenshell?: () => void; rewriteConfigUrlsWithDnsPinning?: (value: ConfigValue) => Promise; restartSandboxGateway?: InferenceSetDeps["restartSandboxGateway"]; + withGatewayRouteMutationLock?: InferenceSetDeps["withGatewayRouteMutationLock"]; }): InferenceSetDeps & { calls: { captureOpenshell: ReturnType; @@ -102,6 +103,7 @@ export function createDeps(options: { prepareRunOpenshell: ReturnType; rewriteConfigUrlsWithDnsPinning: ReturnType; restartSandboxGateway: ReturnType; + withGatewayRouteMutationLock: ReturnType; }; getSession: () => Session | null; } { @@ -149,6 +151,11 @@ export function createDeps(options: { forwardRecovered: true, })), ), + withGatewayRouteMutationLock: vi.fn( + options.withGatewayRouteMutationLock ?? + (async (_gatewayName: string, operation: () => Promise | unknown) => + await operation()), + ), }; return { getDefaultSandbox: () => defaultSandbox, @@ -173,6 +180,8 @@ export function createDeps(options: { resolveContextWindowForModel: calls.resolveContextWindowForModel, isSandboxConfigMutable: () => options.shieldsMutable ?? true, rewriteConfigUrlsWithDnsPinning: calls.rewriteConfigUrlsWithDnsPinning, + withGatewayRouteMutationLock: + calls.withGatewayRouteMutationLock as InferenceSetDeps["withGatewayRouteMutationLock"], restartSandboxGateway: calls.restartSandboxGateway, calls, getSession: () => session, diff --git a/src/lib/actions/inference-set.ts b/src/lib/actions/inference-set.ts index c8a32263d40..f41e042d8df 100644 --- a/src/lib/actions/inference-set.ts +++ b/src/lib/actions/inference-set.ts @@ -13,8 +13,10 @@ import { type SandboxInferenceConfig, } from "../inference/config"; import { resolveContextWindowForModel } from "../inference/context-window"; +import { withGatewayRouteMutationLock } from "../inference/gateway-route-mutation-lock"; import { type ValidationResult, validateLocalProvider } from "../inference/local"; import { inferenceSelectionRegistryFields } from "../inference/selection"; +import { resolveSandboxGatewayName } from "../onboard/gateway-binding"; import { matchesGatewayProviderBinding, parseGatewayProviderMetadata, @@ -52,15 +54,15 @@ import { readPreviousOpenClawInferenceApi, } from "./inference-set-gateway-restart"; import { buildInferenceSetFailure } from "./inference-set-provider-diagnostics"; +import { + applyOpenClawAnthropicReplyBudget, + readOpenClawPrimaryReplyBudget, +} from "./inference-set-reply-budget"; import { finalizeInferenceSetRoute, prepareInferenceSetRoute, type RegistryInferenceMetadata, } from "./inference-set-route-containment"; -import { - applyOpenClawAnthropicReplyBudget, - readOpenClawPrimaryReplyBudget, -} from "./inference-set-reply-budget"; export { normalizeCustomEndpointUrl } from "./inference-set-route-containment"; export { InferenceSetError }; @@ -118,6 +120,7 @@ export interface InferenceSetDeps extends InferenceGatewayRestartDeps { resolveContextWindowForModel: (provider: string, model: string) => number | null; isSandboxConfigMutable: (sandboxName: string) => boolean; rewriteConfigUrlsWithDnsPinning: (value: ConfigValue) => Promise; + withGatewayRouteMutationLock: typeof withGatewayRouteMutationLock; } const SUPPORTED_PROVIDER_NAMES = [ @@ -159,6 +162,7 @@ function defaultDeps(): InferenceSetDeps { ensureLocalProviderReachable, resolveContextWindowForModel, rewriteConfigUrlsWithDnsPinning, + withGatewayRouteMutationLock, restartSandboxGateway: defaultInferenceGatewayRestart, isSandboxConfigMutable: (sandboxName) => { const { isShieldsDown }: typeof import("../shields") = require("../shields"); @@ -186,6 +190,15 @@ function normalizeSandboxAgent(agentName: string | null | undefined): string { return (trimmed || "openclaw").toLowerCase(); } +function assertSandboxRouteReservationComplete(entry: SandboxEntry): void { + if (entry.pendingRouteReservation === true) { + throw new InferenceSetError( + `Sandbox '${entry.name}' is still being created by onboarding. Wait for onboarding to finish or remove the incomplete sandbox before changing inference.`, + 2, + ); + } +} + function resolveTargetSandbox( sandboxName: string | null | undefined, deps: Pick< @@ -199,6 +212,7 @@ function resolveTargetSandbox( if (!entry) { throw new InferenceSetError(`Sandbox '${explicitName}' is not registered.`, 2); } + assertSandboxRouteReservationComplete(entry); return { sandboxName: explicitName, entry, @@ -209,7 +223,10 @@ function resolveTargetSandbox( if (normalizeSandboxAgent(deps.getRequestedAgent()) === "hermes") { const hermesSandboxes = deps .listSandboxes() - .sandboxes.filter((entry) => normalizeSandboxAgent(entry.agent) === "hermes"); + .sandboxes.filter( + (entry) => + entry.pendingRouteReservation !== true && normalizeSandboxAgent(entry.agent) === "hermes", + ); if (hermesSandboxes.length === 1) { const entry = hermesSandboxes[0]; return { sandboxName: entry.name, entry, agentName: "hermes" }; @@ -240,6 +257,7 @@ function resolveTargetSandbox( if (!entry) { throw new InferenceSetError(`Sandbox '${targetName}' is not registered.`, 2); } + assertSandboxRouteReservationComplete(entry); return { sandboxName: targetName, entry, agentName: normalizeSandboxAgent(entry.agent) }; } @@ -461,7 +479,8 @@ function assertHermesCompatibleAnthropicOpenAiProvider( async function runInferenceSetWithoutHostLock( options: InferenceSetOptions, - deps: InferenceSetDeps = defaultDeps(), + deps: InferenceSetDeps, + expectedGatewayName: string, ): Promise> { const provider = trimRequired(options.provider, "provider"); const model = trimRequired(options.model, "model"); @@ -537,6 +556,13 @@ async function runInferenceSetWithoutHostLock( session: routeSession, sandboxes: routeSandboxes, }); + if (preparedRoute.gatewayName !== expectedGatewayName) { + throw new InferenceSetError( + `Sandbox '${sandboxName}' moved from OpenShell gateway '${expectedGatewayName}' to ` + + `'${preparedRoute.gatewayName}' while waiting for the route mutation lock. Retry the command.`, + 2, + ); + } const target = deps.resolveAgentConfig(sandboxName); const targetAgent = normalizeSandboxAgent(target.agentName); @@ -780,10 +806,16 @@ export async function runInferenceSet( const selected = resolveTargetSandbox(options.sandboxName, deps); deps.prepareRunOpenshell(); return withSandboxMutationLock(selected.sandboxName, async () => { - const mutation = await withTimerBoundShieldsMutationLockAsync( - selected.sandboxName, - "inference set", - () => runInferenceSetWithoutHostLock({ ...options, sandboxName: selected.sandboxName }, deps), + const lockedSelection = resolveTargetSandbox(selected.sandboxName, deps); + const gatewayName = resolveSandboxGatewayName(lockedSelection.entry); + const mutation = await deps.withGatewayRouteMutationLock(gatewayName, () => + withTimerBoundShieldsMutationLockAsync(selected.sandboxName, "inference set", () => + runInferenceSetWithoutHostLock( + { ...options, sandboxName: selected.sandboxName }, + deps, + gatewayName, + ), + ), ); // Release the config transition lock before the managed restart reacquires // it, but retain the outer sandbox lifecycle lock so another process cannot diff --git a/src/lib/actions/sandbox/connect-route-containment.test.ts b/src/lib/actions/sandbox/connect-route-containment.test.ts index ad94f9cd1b9..a95b9545e13 100644 --- a/src/lib/actions/sandbox/connect-route-containment.test.ts +++ b/src/lib/actions/sandbox/connect-route-containment.test.ts @@ -121,6 +121,100 @@ describe("connect route containment", () => { expect(exitSpy).toHaveBeenCalledWith(1); }); + it("rechecks peers after waiting for the shared gateway route lock", async () => { + let releaseLock!: () => void; + const released = new Promise((resolve) => { + releaseLock = resolve; + }); + let reportLockEntered!: () => void; + const lockEntered = new Promise((resolve) => { + reportLockEntered = resolve; + }); + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/model-a", + } as const; + const harness = createConnectHarness({ + registryEntry: alpha, + registryEntries: [alpha, { ...alpha, name: "peer" }], + withGatewayRouteMutationLock: async (_gatewayName, operation) => { + reportLockEntered(); + await released; + return await operation(); + }, + }); + + const connect = harness.connectSandbox("alpha", { probeOnly: true }); + await lockEntered; + const peer = harness.registryEntries.find((candidate) => candidate.name === "peer"); + expect(peer).toBeDefined(); + Object.assign(peer!, { provider: "anthropic-prod", model: "claude-new" }); + releaseLock(); + + await expect(connect).rejects.toThrow("process.exit(1)"); + expect(harness.withGatewayRouteMutationLockSpy).toHaveBeenCalledWith( + "nemoclaw", + expect.any(Function), + ); + expect(harness.captureOpenshellSpy).toHaveBeenCalledOnce(); + expect(harness.captureOpenshellSpy).toHaveBeenCalledWith( + ["inference", "get", "-g", "nemoclaw"], + { ignoreError: true, timeout: 15_000 }, + ); + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + + it("aborts before route reads or repairs when the target changes gateways while waiting", async () => { + let releaseLock!: () => void; + const released = new Promise((resolve) => { + releaseLock = resolve; + }); + let reportLockEntered!: () => void; + const lockEntered = new Promise((resolve) => { + reportLockEntered = resolve; + }); + const alpha = { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + gatewayPort: 8080, + provider: "nvidia-prod", + model: "nvidia/model-a", + } as const; + const harness = createConnectHarness({ + registryEntry: alpha, + registryEntries: [alpha], + withGatewayRouteMutationLock: async (_gatewayName, operation) => { + reportLockEntered(); + await released; + return await operation(); + }, + }); + + const connect = harness.connectSandbox("alpha", { probeOnly: true }); + await lockEntered; + Object.assign(harness.registryEntries[0], { + gatewayName: "nemoclaw-9090", + gatewayPort: 9090, + }); + releaseLock(); + + await expect(connect).rejects.toThrow("process.exit(1)"); + expect(harness.captureOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.applyVmDnsMonkeypatchSpy).not.toHaveBeenCalled(); + expect(harness.runSetupDnsProxySpy).not.toHaveBeenCalled(); + expect(harness.errorSpy).toHaveBeenCalledWith( + expect.stringContaining("changed OpenShell gateways while waiting"), + ); + expect(exitSpy).toHaveBeenCalledWith(1); + }); + it("exits before managed route reads or repairs when an endpoint override is ambient", async () => { vi.stubEnv("OPENSHELL_GATEWAY_ENDPOINT", "https://other.example.test"); const harness = createConnectHarness({ @@ -154,6 +248,26 @@ describe("connect route containment", () => { expect(exitSpy).toHaveBeenCalledWith(1); }); + it("rejects a pending onboarding reservation before liveness or route work", async () => { + const harness = createConnectHarness({ + registryEntry: { + name: "alpha", + pendingRouteReservation: true, + gatewayName: "nemoclaw", + provider: "nvidia-prod", + model: "nvidia/model-a", + }, + }); + + await expect(harness.connectSandbox("alpha")).rejects.toThrow("process.exit(1)"); + + const output = harness.errorSpy.mock.calls.map((call) => String(call[0] ?? "")).join("\n"); + expect(output).toContain("still being created by onboarding"); + expect(harness.ensureLiveSandboxSpy).not.toHaveBeenCalled(); + expect(harness.captureOpenshellSpy).not.toHaveBeenCalled(); + expect(harness.runOpenshellSpy).not.toHaveBeenCalled(); + }); + it("exits before repairing a lone incomplete legacy custom route (#6315)", async () => { const harness = createConnectHarness({ inferenceGetOutput: diff --git a/src/lib/actions/sandbox/connect.ts b/src/lib/actions/sandbox/connect.ts index 7bde34fc6dd..8e468adf110 100644 --- a/src/lib/actions/sandbox/connect.ts +++ b/src/lib/actions/sandbox/connect.ts @@ -24,6 +24,7 @@ import { sanitizeRouteValueForDisplay, } from "../../inference/config"; import { GatewayRouteConflictError } from "../../inference/gateway-route-compatibility"; +import { withGatewayRouteMutationLock } from "../../inference/gateway-route-mutation-lock"; import { findReachableOllamaHost, probeLocalProviderHealth } from "../../inference/local"; import { ensureOllamaAuthProxy, probeOllamaAuthProxyHealth } from "../../inference/ollama/proxy"; import { resolveSandboxGatewayName } from "../../onboard/gateway-binding"; @@ -224,15 +225,16 @@ function exitOnForwardRecoveryFailure( process.exit(1); } -function runSandboxConnectProbe(sandboxName: string): void { +async function runSandboxConnectProbe(sandboxName: string): Promise { const agent = agentRuntime.getSessionAgent(sandboxName); const agentName = agentRuntime.getAgentDisplayName(agent); if (agent && !agentRuntime.hasGatewayRuntime(agent)) { + const routeResult = await ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); runTerminalAgentConnectProbe({ agent, agentName, capture: captureOpenshell, - ensureInferenceRoute: (name, options) => ensureSandboxInferenceRoute(name, agent, options), + ensureInferenceRoute: () => routeResult, sandboxName, }); return; @@ -264,7 +266,7 @@ function runSandboxConnectProbe(sandboxName: string): void { ); } if (processCheck.wasRunning) { - ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); + await ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); // Defense-in-depth scope-upgrade approval on the probe-only / `recover` // path (#4504): the gateway is up, so deterministically clear any pending // allowlisted CLI/webchat scope upgrade. Best-effort; never throws. @@ -279,13 +281,13 @@ function runSandboxConnectProbe(sandboxName: string): void { return; } if (processCheck.recovered) { - ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); + await ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); // Same defense-in-depth approval after a recovery (#4504); best-effort. runConnectAutoPairApprovalPass(sandboxName); console.log(` Probe complete: recovered ${agentName} gateway in '${sandboxName}'.`); return; } - ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); + await ensureSandboxInferenceRoute(sandboxName, agent, { quiet: true }); console.error( ` Probe failed: ${agentName} gateway is not running in '${sandboxName}' and automatic recovery failed.`, ); @@ -704,7 +706,7 @@ function resetManagedInferenceRoute( ); } -function ensureSandboxInferenceRoute( +function ensureSandboxInferenceRouteUnlocked( sandboxName: string, agent: InferenceRouteProbeAgent, { quiet = false }: { quiet?: boolean } = {}, @@ -794,12 +796,38 @@ function ensureSandboxInferenceRoute( } } -function ensureSandboxInferenceRouteOrExit( +async function ensureSandboxInferenceRoute( sandboxName: string, agent: InferenceRouteProbeAgent, { quiet = false }: { quiet?: boolean } = {}, -): SandboxEntry | null { - const result = ensureSandboxInferenceRoute(sandboxName, agent, { quiet }); +): Promise { + const snapshot = registry.getSandbox(sandboxName); + if (!snapshot) return { sandbox: null, routeHealthy: null }; + if (registry.getSandboxEntryInference(snapshot).kind !== "configured") + return { sandbox: snapshot, routeHealthy: null }; + const gatewayName = resolveSandboxGatewayName(snapshot); + return withGatewayRouteMutationLock(gatewayName, () => { + const lockedSnapshot = registry.getSandbox(sandboxName); + if ( + lockedSnapshot && + registry.getSandboxEntryInference(lockedSnapshot).kind === "configured" && + resolveSandboxGatewayName(lockedSnapshot) !== gatewayName + ) { + console.error( + ` Error: sandbox '${sandboxName}' changed OpenShell gateways while waiting to verify its inference route. Retry the command.`, + ); + process.exit(1); + } + return ensureSandboxInferenceRouteUnlocked(sandboxName, agent, { quiet }); + }); +} + +async function ensureSandboxInferenceRouteOrExit( + sandboxName: string, + agent: InferenceRouteProbeAgent, + { quiet = false }: { quiet?: boolean } = {}, +): Promise { + const result = await ensureSandboxInferenceRoute(sandboxName, agent, { quiet }); if (result.routeHealthy === false) { process.exit(1); } @@ -884,6 +912,11 @@ export async function connectSandbox( try { assertNoOpenShellGatewayEndpointOverride(); const registered = registry.getSandbox(sandboxName); + if (registered?.pendingRouteReservation === true) { + throw new Error( + `Sandbox '${sandboxName}' is still being created by onboarding. Wait for onboarding to finish or remove the incomplete sandbox before connecting.`, + ); + } if (registered && registry.getSandboxEntryInference(registered).kind === "configured") { const gatewayName = resolveSandboxGatewayName(registered); assertSandboxGatewayRouteCompatible(sandboxName, registered, gatewayName); @@ -917,7 +950,7 @@ export async function connectSandbox( } if (probeOnly) { - return runSandboxConnectProbe(sandboxName); + return await runSandboxConnectProbe(sandboxName); } // Version staleness check — warn but don't block @@ -1081,7 +1114,7 @@ export async function connectSandbox( // cluster-wide inference.local route may still point at the other provider. // After the sandbox is Ready, verify and recover the route before SSH. const agent = agentRuntime.getSessionAgent(sandboxName); - sb = ensureSandboxInferenceRouteOrExit(sandboxName, agent); + sb = await ensureSandboxInferenceRouteOrExit(sandboxName, agent); maybeEnsureHermesToolGatewayBroker(sb); // ── Auto-pair late scope-upgrade approval (#4263) ─────────────── diff --git a/src/lib/actions/sandbox/snapshot.test.ts b/src/lib/actions/sandbox/snapshot.test.ts index 4634fb81e6d..d57c5f461b6 100644 --- a/src/lib/actions/sandbox/snapshot.test.ts +++ b/src/lib/actions/sandbox/snapshot.test.ts @@ -22,6 +22,8 @@ type SandboxRecord = { gatewayName?: string | null; imageTag?: string | null; openshellDriver?: string | null; + provider?: string | null; + model?: string | null; }; type DcodeProbeState = "active" | "idle" | "unverifiable" | "no-runtime"; @@ -209,6 +211,10 @@ vi.mock("../../state/gateway", () => ({ vi.mock("../../state/registry", () => ({ getCustomPolicies: getCustomPoliciesMock, getSandbox: getSandboxMock, + listSandboxes: () => ({ + sandboxes: ["alpha", "beta", "gamma"].map((name) => getSandboxMock(name)).filter(Boolean), + defaultSandbox: "alpha", + }), registerSandbox: registerSandboxMock, removeSandbox: vi.fn(), })); @@ -834,12 +840,16 @@ describe("runSandboxSnapshot", () => { agent: "openclaw", imageTag: "nemoclaw-alpha:test", openshellDriver: "docker", + provider: "nvidia-nim", + model: "nvidia/model-a", } : { name: "beta", agent: "openclaw", imageTag: "nemoclaw-beta:test", openshellDriver: "docker", + provider: "nvidia-nim", + model: "nvidia/model-a", }, ); parseLiveSandboxNamesMock.mockReturnValue(new Set(["alpha", "beta"])); @@ -883,6 +893,45 @@ describe("runSandboxSnapshot", () => { expect(restoreSandboxStateMock).toHaveBeenCalledWith("beta", "/tmp/backup-alpha"); }); + it("blocks auto-create before deleting a destination when a gateway peer conflicts", async () => { + const consoleError = vi.spyOn(console, "error").mockImplementation(() => {}); + getSandboxMock.mockImplementation((name) => ({ + name: name ?? "alpha", + agent: "openclaw", + gatewayName: "nemoclaw", + imageTag: `nemoclaw-${name}:test`, + openshellDriver: "docker", + provider: name === "gamma" ? "anthropic-prod" : "nvidia-nim", + model: name === "gamma" ? "claude-new" : "nvidia/model-a", + })); + parseLiveSandboxNamesMock.mockReturnValue(new Set(["alpha", "beta"])); + captureOpenshellMock.mockImplementation((args) => + openshellResponses(args, { + "sandbox exec": { status: 0, output: dcodeProbeOutput("no-runtime") }, + "sandbox list": { status: 0, output: "alpha Ready\nbeta Ready\n" }, + }), + ); + getLatestBackupMock.mockReturnValue({ + timestamp: "2026-06-15T00:00:00.000Z", + backupPath: "/tmp/backup-alpha", + }); + const { runSandboxSnapshot } = await import("./snapshot"); + + await expect( + runSandboxSnapshot("alpha", { + kind: "restore", + to: "beta", + force: true, + yes: true, + }), + ).rejects.toMatchObject({ exitCode: 1 }); + + expect(consoleError.mock.calls.flat().join("\n")).toContain("gamma"); + expect(lifecycleMock.events).not.toContain("delete"); + expect(streamSandboxCreateMock).not.toHaveBeenCalled(); + expect(registerSandboxMock).not.toHaveBeenCalled(); + }); + it("refuses snapshot creation before backup when the sandbox is not live", async () => { parseLiveSandboxNamesMock.mockReturnValue(new Set(["beta"])); const consoleError = vi.spyOn(console, "error").mockImplementation(() => {}); diff --git a/src/lib/actions/sandbox/snapshot.ts b/src/lib/actions/sandbox/snapshot.ts index 3976c507672..98faa982883 100644 --- a/src/lib/actions/sandbox/snapshot.ts +++ b/src/lib/actions/sandbox/snapshot.ts @@ -13,6 +13,11 @@ import { OPENSHELL_PROBE_TIMEOUT_MS } from "../../adapters/openshell/timeouts"; import { CLI_NAME } from "../../cli/branding"; import { prompt as askPrompt } from "../../credentials/store"; import { getSandboxDeleteOutcome } from "../../domain/sandbox/destroy"; +import { + checkGatewayRouteCompatibility, + formatGatewayRouteConflict, +} from "../../inference/gateway-route-compatibility"; +import { withGatewayRouteMutationLock } from "../../inference/gateway-route-mutation-lock"; import * as nim from "../../inference/nim"; import { listMessagingProviderSuffixes } from "../../messaging/channels"; import { resolveSandboxGatewayName } from "../../onboard/gateway-binding"; @@ -27,6 +32,7 @@ import { isSandboxReady } from "../../state/gateway"; import { withSandboxMutationLock } from "../../state/mcp-lifecycle-lock"; import type { SandboxEntry } from "../../state/registry"; import * as registry from "../../state/registry"; +import { getSandboxEntryInference } from "../../state/registry-entry-view"; import * as sandboxState from "../../state/sandbox"; import { cleanupShieldsDestroyArtifacts, removeSandboxRegistryEntry } from "./destroy"; import { @@ -780,16 +786,69 @@ async function runSnapshotRestoreUnlocked( snapshotExit(1); } } - if (targetEntry) { - verifyRestoreDestinationOnOwnGateway(targetSandbox); + } + const sourceGatewayName = resolveSandboxGatewayName(srcEntry); + await withGatewayRouteMutationLock(sourceGatewayName, async () => { + if (!targetExists && registry.getSandbox(targetSandbox)) { + console.error( + ` Destination sandbox '${targetSandbox}' was registered while this restore was waiting. Retry with --force only after reviewing that sandbox.`, + ); + snapshotExit(1); + } + const lockedSourceEntry = registry.getSandbox(sandboxName); + if (!lockedSourceEntry) { + console.error( + ` Cannot auto-create '${targetSandbox}': source '${sandboxName}' has no durable inference route metadata.`, + ); + snapshotExit(1); + } + if (getSandboxEntryInference(lockedSourceEntry).kind !== "configured") { + console.error( + ` Cannot auto-create '${targetSandbox}': source '${sandboxName}' has no complete durable inference route.`, + ); + snapshotExit(1); + } + const lockedFromImage = resolveSrcPodImage(sandboxName, lockedSourceEntry); + if (!lockedFromImage) { + console.error( + ` Cannot resolve the current image for source sandbox '${sandboxName}' — aborting before changing '${targetSandbox}'.`, + ); + snapshotExit(1); + } + const lockedGatewayName = resolveSandboxGatewayName(lockedSourceEntry); + if (lockedGatewayName !== sourceGatewayName) { + console.error( + ` Source sandbox '${sandboxName}' changed OpenShell gateways while waiting to restore. Retry the command.`, + ); + snapshotExit(1); + } + const compatibility = checkGatewayRouteCompatibility({ + gatewayName: sourceGatewayName, + sandboxName: targetSandbox, + route: lockedSourceEntry, + sandboxes: registry.listSandboxes().sandboxes, + }); + if (!compatibility.ok) { + console.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); + snapshotExit(1); + } + if (targetExists) { + if (targetEntry) { + verifyRestoreDestinationOnOwnGateway(targetSandbox); + } + deleteSandboxForRestore(targetSandbox); + requireLiveSandboxesOnSandboxGateway( + sandboxName, + " Failed to re-select source sandbox gateway after deleting destination.", + ); } - deleteSandboxForRestore(targetSandbox); - requireLiveSandboxesOnSandboxGateway( + await autoCreateSandboxFromSource( sandboxName, - " Failed to re-select source sandbox gateway after deleting destination.", + targetSandbox, + lockedSourceEntry, + lockedFromImage, ); - } - await autoCreateSandboxFromSource(sandboxName, targetSandbox, srcEntry, fromImage); + }); } withTimerBoundShieldsMutationLock(targetSandbox, "restore sandbox snapshot", () => { // Serialize filesystem restore, mutable-permission repair, and policy diff --git a/src/lib/inference/gateway-route-compatibility.test.ts b/src/lib/inference/gateway-route-compatibility.test.ts index f5fb019727d..49156cabfc3 100644 --- a/src/lib/inference/gateway-route-compatibility.test.ts +++ b/src/lib/inference/gateway-route-compatibility.test.ts @@ -282,7 +282,13 @@ describe("shared gateway inference route compatibility", () => { expect(result).toMatchObject({ ok: false, - conflicts: [{ sandboxName: "legacy-custom", reason: "incomplete-custom-route" }], + conflicts: [ + { + sandboxName: "legacy-custom", + reason: "incomplete-custom-route", + scope: "registered", + }, + ], }); expect(formatGatewayRouteConflict(result as Exclude)).toContain( "remove and re-onboard that sandbox with complete custom-route metadata", diff --git a/src/lib/inference/gateway-route-compatibility.ts b/src/lib/inference/gateway-route-compatibility.ts index 528efc3b5ff..feaed7ea93c 100644 --- a/src/lib/inference/gateway-route-compatibility.ts +++ b/src/lib/inference/gateway-route-compatibility.ts @@ -133,7 +133,11 @@ export function preflightGatewayRouteDiscovery( try { recordedGatewayName = resolveSandboxGatewayName(sandbox); } catch { - invalidBindings.push({ sandboxName: sandbox.name, reason: "invalid-gateway-binding" }); + invalidBindings.push({ + sandboxName: sandbox.name, + reason: "invalid-gateway-binding", + scope: "registered", + }); continue; } if (recordedGatewayName === request.gatewayName && configuredRoute(sandbox)) { @@ -175,6 +179,7 @@ export function preflightGatewayRouteDiscovery( conflicts: peers.map((sandbox) => ({ sandboxName: sandbox.name, reason: "provider-model" as const, + scope: "registered" as const, })), }, }; @@ -240,7 +245,11 @@ export function checkGatewayRouteCompatibility( try { recordedGatewayName = resolveSandboxGatewayName(sandbox); } catch { - conflicts.push({ sandboxName: sandbox.name, reason: "invalid-gateway-binding" }); + conflicts.push({ + sandboxName: sandbox.name, + reason: "invalid-gateway-binding", + scope: "registered", + }); continue; } if (recordedGatewayName !== request.gatewayName) continue; @@ -248,12 +257,16 @@ export function checkGatewayRouteCompatibility( if (!recorded) continue; if (recorded.provider !== requested.provider || recorded.model !== requested.model) { - conflicts.push({ sandboxName: sandbox.name, reason: "provider-model" }); + conflicts.push({ + sandboxName: sandbox.name, + reason: "provider-model", + scope: "registered", + }); continue; } if (CUSTOM_ROUTE_PROVIDERS.has(requested.provider)) { const reason = customRouteConflict(requested.provider, request.route, sandbox); - if (reason) conflicts.push({ sandboxName: sandbox.name, reason }); + if (reason) conflicts.push({ sandboxName: sandbox.name, reason, scope: "registered" }); } } diff --git a/src/lib/inference/gateway-route-mutation-lock.test.ts b/src/lib/inference/gateway-route-mutation-lock.test.ts new file mode 100644 index 00000000000..4185e35cc7c --- /dev/null +++ b/src/lib/inference/gateway-route-mutation-lock.test.ts @@ -0,0 +1,84 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { describe, expect, it } from "vitest"; +import { withGatewayRouteMutationLock } from "./gateway-route-mutation-lock"; + +describe("gateway route mutation lock", () => { + it("serializes separate operations for the same gateway", async () => { + const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), "nemoclaw-gateway-lock-")); + let releaseFirst!: () => void; + const firstReleased = new Promise((resolve) => { + releaseFirst = resolve; + }); + let reportFirstEntered!: () => void; + const firstEntered = new Promise((resolve) => { + reportFirstEntered = resolve; + }); + const events: string[] = []; + const options = { stateDir, pollIntervalMs: 1, timeoutMs: 5_000 }; + try { + const first = withGatewayRouteMutationLock( + "nemoclaw", + async () => { + events.push("first-enter"); + reportFirstEntered(); + await firstReleased; + events.push("first-exit"); + }, + options, + ); + await firstEntered; + const second = withGatewayRouteMutationLock( + "nemoclaw", + () => { + events.push("second-enter"); + }, + options, + ); + await new Promise((resolve) => setTimeout(resolve, 20)); + expect(events).toEqual(["first-enter"]); + releaseFirst(); + await Promise.all([first, second]); + expect(events).toEqual(["first-enter", "first-exit", "second-enter"]); + } finally { + releaseFirst(); + await fs.rm(stateDir, { recursive: true, force: true }); + } + }); + + it("allows operations for different gateways to overlap", async () => { + const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), "nemoclaw-gateway-lock-")); + let releaseFirst!: () => void; + const firstReleased = new Promise((resolve) => { + releaseFirst = resolve; + }); + let reportFirstEntered!: () => void; + const firstEntered = new Promise((resolve) => { + reportFirstEntered = resolve; + }); + const options = { stateDir, pollIntervalMs: 1, timeoutMs: 5_000 }; + try { + const first = withGatewayRouteMutationLock( + "nemoclaw", + async () => { + reportFirstEntered(); + await firstReleased; + }, + options, + ); + await firstEntered; + await expect( + withGatewayRouteMutationLock("nemoclaw-9090", () => "second", options), + ).resolves.toBe("second"); + releaseFirst(); + await first; + } finally { + releaseFirst(); + await fs.rm(stateDir, { recursive: true, force: true }); + } + }); +}); diff --git a/src/lib/inference/gateway-route-mutation-lock.ts b/src/lib/inference/gateway-route-mutation-lock.ts new file mode 100644 index 00000000000..1e64c3ed06a --- /dev/null +++ b/src/lib/inference/gateway-route-mutation-lock.ts @@ -0,0 +1,25 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { type McpLifecycleLockOptions, withMcpLifecycleLock } from "../state/mcp-lifecycle-lock"; + +const GATEWAY_ROUTE_LOCK_PREFIX = "gateway-route:"; + +/** + * Serializes host-side reads and writes of OpenShell's one-route-per-gateway + * inference state. The non-sandbox prefix keeps this lock namespace disjoint + * from user sandbox mutation locks while reusing their cross-process lease. + */ +export function withGatewayRouteMutationLock( + gatewayName: string, + operation: () => Promise | T, + options: McpLifecycleLockOptions = {}, +): Promise { + const normalizedGatewayName = gatewayName.trim(); + if (!normalizedGatewayName) throw new Error("OpenShell gateway name is required."); + return withMcpLifecycleLock( + `${GATEWAY_ROUTE_LOCK_PREFIX}${normalizedGatewayName}`, + operation, + options, + ); +} diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index b967c96d840..1df40ea052a 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -346,6 +346,8 @@ const { const registry: typeof import("./state/registry") = require("./state/registry"); const sandboxMutationLock: typeof import("./state/mcp-lifecycle-lock") = require("./state/mcp-lifecycle-lock"); +const gatewayRouteMutationLock: typeof import("./inference/gateway-route-mutation-lock") = + require("./inference/gateway-route-mutation-lock"); const { resolveSandboxImageTagFromCreateOutput } = require("./domain/sandbox/image-tag") as typeof import("./domain/sandbox/image-tag"); const nim: typeof import("./inference/nim") = require("./inference/nim"); @@ -3758,6 +3760,8 @@ const setupNim = setupNimFlow.createSetupNim(getSetupNimDeps()); function getSetupInferenceDeps(): SetupInferenceDeps { return { checkGatewayRouteCompatibility, + withGatewayRouteMutationLock: gatewayRouteMutationLock.withGatewayRouteMutationLock, + withSandboxMutationLock: sandboxMutationLock.withSandboxMutationLock, step, getGatewayName: () => GATEWAY_NAME, runOpenshell, @@ -3765,7 +3769,7 @@ function getSetupInferenceDeps(): SetupInferenceDeps { verifyInferenceRoute, verifyOnboardInferenceSmoke, isNonInteractive, - updateSandbox: registry.updateSandbox, + updateSandbox: registry.reserveSandboxInferenceRoute, hermesProviderAuth, getHermesToolGatewayBroker, providerExistsInGateway, @@ -4461,6 +4465,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { providerDeps: { checkGatewayRouteCompatibility, preflightGatewayRouteDiscovery, + withGatewayRouteMutationLock: gatewayRouteMutationLock.withGatewayRouteMutationLock, normalizeHermesAuthMethod, setupNim: (g, s, a, recover, gateway, assertRouteCompatible, canProbeRoute) => setupNim( @@ -4527,6 +4532,8 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { rootDir: ROOT, }, sandboxDeps: { + checkGatewayRouteCompatibility, + withGatewayRouteMutationLock: gatewayRouteMutationLock.withGatewayRouteMutationLock, resolvePath: preparedDcodeRuntime.resolveDockerfileProbePath, agentSupportsWebSearch, agentSupportsWebSearchProvider, diff --git a/src/lib/onboard/machine/core-flow-phases.test.ts b/src/lib/onboard/machine/core-flow-phases.test.ts index 617d30aa084..358e8ae5ccc 100644 --- a/src/lib/onboard/machine/core-flow-phases.test.ts +++ b/src/lib/onboard/machine/core-flow-phases.test.ts @@ -91,6 +91,10 @@ function createPhases( requiredEndpointUrl: null, requiredInferenceApi: null, }), + withGatewayRouteMutationLock: async ( + _gatewayName: string, + operation: () => Promise | T, + ) => await operation(), normalizeHermesAuthMethod: (value) => value === "oauth" || value === "api_key" ? value : null, setupNim: vi.fn(async () => ({ @@ -170,7 +174,17 @@ function createPhases( getDcodeSelectionDrift: () => ({ changed: false, unknown: false }), hasSandboxGpuDrift: () => false, getSandboxHermesToolGateways: () => [], - getSandboxRegistryEntry: () => null, + getSandboxRegistryEntry: () => ({ + name: "my-sandbox", + provider: "nim", + model: "nvidia/test", + endpointUrl: "https://example.test/v1", + credentialEnv: "NVIDIA_INFERENCE_API_KEY", + preferredInferenceApi: "chat", + gatewayName: "nemoclaw", + gpuEnabled: false, + policies: [], + }), normalizeHermesToolGatewaySelections: (value) => (Array.isArray(value) ? value : []), stringSetsEqual: (left, right) => left.length === right.length && left.every((item) => right.includes(item)), @@ -205,6 +219,11 @@ function createPhases( throw new Error(`exit ${code}`); }) as (code: number) => never, ...overrides.sandboxDeps, + checkGatewayRouteCompatibility: + overrides.sandboxDeps?.checkGatewayRouteCompatibility ?? (() => ({ ok: true })), + withGatewayRouteMutationLock: + overrides.sandboxDeps?.withGatewayRouteMutationLock ?? + (async (_gatewayName: string, operation: () => Promise | T) => await operation()), }, }); } @@ -295,6 +314,19 @@ describe("core onboard flow phases", () => { isInferenceRouteReady: (_gatewayName, _provider, _model) => true, setupInference, }, + sandboxDeps: { + getSandboxRegistryEntry: () => ({ + name: "my-sandbox", + provider: "hermes", + model: "nvidia/test", + endpointUrl: null, + credentialEnv: "HERMES_API_KEY", + preferredInferenceApi: null, + gatewayName: "nemoclaw", + gpuEnabled: false, + policies: [], + }), + }, }); const session = createSession({ model: "nvidia/test", diff --git a/src/lib/onboard/machine/core-flow-phases.ts b/src/lib/onboard/machine/core-flow-phases.ts index d734f177c51..f08f37fb440 100644 --- a/src/lib/onboard/machine/core-flow-phases.ts +++ b/src/lib/onboard/machine/core-flow-phases.ts @@ -106,6 +106,7 @@ export function createCoreOnboardFlowPhases< const sandboxStateResult = await handleSandboxState({ resume: context.resume, fresh: context.fresh, + gatewayName: options.gatewayName, authoritativeResumeConfig: options.authoritativeResumeConfig, resumeAgentChanged: options.sandbox.resumeAgentChanged, session: context.session, diff --git a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts index 9b2eb0bb9d8..a0d9b705ca4 100644 --- a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts @@ -73,6 +73,7 @@ function createDeps() { const deps: Options["deps"] = { checkGatewayRouteCompatibility: calls.checkGatewayRouteCompatibility, preflightGatewayRouteDiscovery: calls.preflightGatewayRouteDiscovery, + withGatewayRouteMutationLock: async (_gatewayName, operation) => await operation(), normalizeHermesAuthMethod: () => null, setupNim: calls.setupNim, setupInference: calls.setupInference, @@ -231,6 +232,35 @@ describe("provider route containment", () => { expect(calls.setupInference).not.toHaveBeenCalled(); }); + it("rechecks routed repair after waiting for the gateway lock", async () => { + const session = createSession({ provider: "nvidia-router", model: "router/model" }); + session.steps.provider_selection.status = "complete"; + const { calls, deps } = createDeps(); + let releaseLock!: () => void; + const lockReleased = new Promise((resolve) => { + releaseLock = resolve; + }); + let reportLockEntered!: () => void; + const lockEntered = new Promise((resolve) => { + reportLockEntered = resolve; + }); + deps.withGatewayRouteMutationLock = async (_gatewayName, operation) => { + reportLockEntered(); + await lockReleased; + return await operation(); + }; + + const repair = handleProviderInferenceState(resumeOptions(deps, session)); + await lockEntered; + rejectRoute(calls, "nvidia-router", "router/model"); + releaseLock(); + + await expect(repair).rejects.toThrow("exit 1"); + expect(calls.reconcileRouter).not.toHaveBeenCalled(); + expect(calls.reupsertRoutedProvider).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + }); + it("blocks compatible-endpoint messaging refresh before endpoint or gateway work", async () => { const session = createSession({ provider: "compatible-endpoint", diff --git a/src/lib/onboard/machine/handlers/provider-inference.test.ts b/src/lib/onboard/machine/handlers/provider-inference.test.ts index e8ec62ceabb..13c79e37dc7 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.test.ts @@ -97,6 +97,10 @@ function createDeps( deps: { checkGatewayRouteCompatibility: calls.checkGatewayRouteCompatibility, preflightGatewayRouteDiscovery: calls.preflightGatewayRouteDiscovery, + withGatewayRouteMutationLock: async ( + _gatewayName: string, + operation: () => Promise | T, + ) => await operation(), normalizeHermesAuthMethod: (value: string | null | undefined) => value === "oauth" || value === "api_key" ? value : null, setupNim: calls.setupNim, diff --git a/src/lib/onboard/machine/handlers/provider-inference.ts b/src/lib/onboard/machine/handlers/provider-inference.ts index f76d75f8d06..e2f422ad810 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.ts @@ -81,6 +81,10 @@ export interface ProviderInferenceStateOptions { deps: { checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; preflightGatewayRouteDiscovery: CurrentGatewayRouteDiscoveryPreflight; + withGatewayRouteMutationLock( + gatewayName: string, + operation: () => Promise | T, + ): Promise; normalizeHermesAuthMethod(value: string | null | undefined): HermesAuthMethod | null; setupNim( gpu: Gpu, @@ -563,23 +567,31 @@ export async function handleProviderInferenceState({ break; } if (deps.isRoutedInferenceProvider(provider)) { - try { - await deps.reconcileModelRouter(); - } catch (err) { - deps.error( - ` ✗ Failed to reconcile model router: ${err instanceof Error ? err.message : String(err)}`, - ); - deps.exitProcess(1); - } // #4564: re-upsert the gateway provider with the sandbox-facing // endpoint so a stale localhost base URL recorded by an earlier run is // repaired on resume instead of surviving and breaking inference.local. - const reupserted = deps.reupsertRoutedProvider( - gatewayName, - provider, - endpointUrl, - credentialEnv, - ); + const reupserted = await deps.withGatewayRouteMutationLock(gatewayName, async () => { + assertProviderInferenceRouteCompatible(deps, gatewayName, sandboxName, { + provider: selectedProvider, + model: selectedModel, + endpointUrl, + preferredInferenceApi, + }); + try { + await deps.reconcileModelRouter(); + } catch (err) { + deps.error( + ` ✗ Failed to reconcile model router: ${err instanceof Error ? err.message : String(err)}`, + ); + deps.exitProcess(1); + } + return deps.reupsertRoutedProvider( + gatewayName, + selectedProvider, + endpointUrl, + credentialEnv, + ); + }); if (!reupserted.ok) { deps.error( ` ${reupserted.message ?? "Failed to update the routed inference provider."}`, diff --git a/src/lib/onboard/machine/handlers/sandbox-route-mutation-lock.test.ts b/src/lib/onboard/machine/handlers/sandbox-route-mutation-lock.test.ts new file mode 100644 index 00000000000..941803dcbdc --- /dev/null +++ b/src/lib/onboard/machine/handlers/sandbox-route-mutation-lock.test.ts @@ -0,0 +1,123 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { describe, expect, it, vi } from "vitest"; +import { handleSandboxState } from "./sandbox"; +import { baseOptions, createDeps } from "./sandbox-test-fixtures"; + +describe("sandbox registration route transaction", () => { + it("rechecks compatibility after waiting for the gateway lock and before create", async () => { + let releaseGateway!: () => void; + const gatewayReleased = new Promise((resolve) => { + releaseGateway = resolve; + }); + let reportGatewayEntered!: () => void; + const gatewayEntered = new Promise((resolve) => { + reportGatewayEntered = resolve; + }); + const checkGatewayRouteCompatibility = vi.fn(() => ({ + ok: false as const, + gatewayName: "nemoclaw", + sandboxName: "my-assistant", + route: { provider: "provider", model: "model" }, + conflicts: [{ sandboxName: "peer", reason: "provider-model" as const }], + })); + const { calls, deps } = createDeps({ + checkGatewayRouteCompatibility, + withSandboxMutationLock: async (_sandboxName, operation) => await operation(), + withGatewayRouteMutationLock: async (_gatewayName, operation) => { + reportGatewayEntered(); + await gatewayReleased; + return await operation(); + }, + }); + + const onboard = handleSandboxState(baseOptions(deps)); + await gatewayEntered; + expect(checkGatewayRouteCompatibility).not.toHaveBeenCalled(); + releaseGateway(); + + await expect(onboard).rejects.toThrow("exit 1"); + expect(checkGatewayRouteCompatibility).toHaveBeenCalledWith( + expect.objectContaining({ gatewayName: "nemoclaw", sandboxName: null }), + ); + expect(calls.createSandbox).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + expect(calls.removeSandbox).not.toHaveBeenCalled(); + expect(calls.startStep).not.toHaveBeenCalled(); + expect(calls.updateSession).not.toHaveBeenCalled(); + expect(calls.error).toHaveBeenCalledWith(expect.stringContaining("peer")); + }); + + it("holds sandbox then gateway locks through sandbox creation and route registration", async () => { + const events: string[] = []; + const { deps } = createDeps({ + checkGatewayRouteCompatibility: () => { + events.push("guard"); + return { ok: true }; + }, + withSandboxMutationLock: async (_sandboxName, operation) => { + events.push("sandbox-lock"); + return await operation(); + }, + withGatewayRouteMutationLock: async (_gatewayName, operation) => { + events.push("gateway-lock"); + return await operation(); + }, + createSandbox: async () => { + events.push("create"); + return "my-assistant"; + }, + updateSandboxRegistry: () => { + events.push("registry"); + }, + }); + + await expect(handleSandboxState(baseOptions(deps))).resolves.toMatchObject({ + sandboxName: "my-assistant", + }); + expect(events).toEqual(["sandbox-lock", "gateway-lock", "guard", "create", "registry"]); + }); + + it("fails when a competing same-name registration changed routes", async () => { + const checkGatewayRouteCompatibility = vi.fn((request) => + request.sandboxName === null + ? { + ok: false as const, + gatewayName: "nemoclaw", + sandboxName: null, + route: { provider: "provider", model: "model" }, + conflicts: [{ sandboxName: "my-assistant", reason: "provider-model" as const }], + } + : { ok: true as const }, + ); + const { calls, deps } = createDeps({ + checkGatewayRouteCompatibility, + getSandboxRegistryEntry: () => ({ + name: "my-assistant", + provider: "other-provider", + model: "other-model", + }), + }); + + await expect(handleSandboxState(baseOptions(deps))).rejects.toThrow("exit 1"); + + expect(checkGatewayRouteCompatibility).toHaveBeenCalledWith( + expect.objectContaining({ sandboxName: null }), + ); + expect(calls.createSandbox).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + expect(calls.startStep).not.toHaveBeenCalled(); + }); + + it("fails when the route reservation disappears before creation", async () => { + const { calls, deps } = createDeps({ getSandboxRegistryEntry: () => null }); + + await expect(handleSandboxState(baseOptions(deps))).rejects.toThrow("exit 1"); + + expect(calls.error).toHaveBeenCalledWith(expect.stringContaining("disappeared")); + expect(calls.createSandbox).not.toHaveBeenCalled(); + expect(calls.updateSandbox).not.toHaveBeenCalled(); + expect(calls.startStep).not.toHaveBeenCalled(); + }); +}); diff --git a/src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts b/src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts index c6fd61367a2..8280b538a1c 100644 --- a/src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts +++ b/src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts @@ -96,6 +96,7 @@ export function createDeps( ) { let session = createSession(); const calls = { + checkGatewayRouteCompatibility: vi.fn(() => ({ ok: true as const })), note: vi.fn(), updateSession: vi.fn((mutator: (value: Session) => Session | void) => { session = mutator(session) ?? session; @@ -127,6 +128,17 @@ export function createDeps( exit: vi.fn((code: number): never => { throw new Error(`exit ${code}`); }), + withGatewayRouteMutationLock: vi.fn(), + }; + const runWithGatewayRouteMutationLock = async ( + gatewayName: string, + operation: () => Promise | T, + ): Promise => { + if (overrides.withGatewayRouteMutationLock) { + return await overrides.withGatewayRouteMutationLock(gatewayName, operation); + } + calls.withGatewayRouteMutationLock(gatewayName, operation); + return await operation(); }; return { calls, @@ -144,6 +156,10 @@ export function createDeps( getSandboxHermesToolGateways: () => [], getSandboxRegistryEntry: (name: string) => ({ name, + provider: "provider", + model: "model", + endpointUrl: null, + preferredInferenceApi: "openai-completions", webSearchEnabled: false, toolDisclosure: "progressive" as const, fromDockerfile: null, @@ -180,6 +196,9 @@ export function createDeps( error: calls.error, exitProcess: calls.exit, ...overrides, + checkGatewayRouteCompatibility: + overrides.checkGatewayRouteCompatibility ?? calls.checkGatewayRouteCompatibility, + withGatewayRouteMutationLock: runWithGatewayRouteMutationLock, }, getSession: () => session, }; @@ -207,6 +226,7 @@ export function baseOptions( resume: false, fresh: false, resumeAgentChanged: false, + gatewayName: "nemoclaw", session, sandboxName: null, model: "model", diff --git a/src/lib/onboard/machine/handlers/sandbox.test.ts b/src/lib/onboard/machine/handlers/sandbox.test.ts index ef1ed80aaac..560978b1d6b 100644 --- a/src/lib/onboard/machine/handlers/sandbox.test.ts +++ b/src/lib/onboard/machine/handlers/sandbox.test.ts @@ -144,6 +144,17 @@ describe("handleSandboxState", () => { const recordStateSkipped = vi.fn(async () => skippedSession); const { deps, calls } = createDeps({ getSandboxReuseState: () => "ready", + getSandboxRegistryEntry: () => ({ + name: "saved", + pendingRouteReservation: true, + provider: "provider", + model: "model", + endpointUrl: null, + preferredInferenceApi: "openai-completions", + toolDisclosure: "progressive", + fromDockerfile: null, + hermesAuthMethod: null, + }), recordStateSkipped, }); @@ -154,6 +165,9 @@ describe("handleSandboxState", () => { }); expect(calls.createSandbox).not.toHaveBeenCalled(); + expect(calls.updateSandbox).toHaveBeenCalledWith("saved", { + pendingRouteReservation: undefined, + }); expect(calls.skipped).toHaveBeenCalledWith("sandbox", "saved"); expect(recordStateSkipped).toHaveBeenCalledWith("sandbox", { reason: "resume", @@ -228,6 +242,10 @@ describe("handleSandboxState", () => { getSandboxReuseState: () => "ready", getSandboxRegistryEntry: (name) => ({ name, + provider: "provider", + model: "model", + endpointUrl: null, + preferredInferenceApi: "openai-completions", nemoclawVersion: "0.1.0", toolDisclosure: "progressive", }), diff --git a/src/lib/onboard/machine/handlers/sandbox.ts b/src/lib/onboard/machine/handlers/sandbox.ts index 111af3c7536..908addb1456 100644 --- a/src/lib/onboard/machine/handlers/sandbox.ts +++ b/src/lib/onboard/machine/handlers/sandbox.ts @@ -1,6 +1,10 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +import { + type CurrentGatewayRouteCompatibilityCheck, + formatGatewayRouteConflict, +} from "../../../inference/gateway-route-compatibility"; import { parseExplicitWebSearchProvider, type WebSearchConfig as SharedWebSearchConfig, @@ -13,7 +17,9 @@ import { import type { SandboxMessagingPlan } from "../../../messaging/manifest"; import type { HermesAuthMethod, Session, SessionUpdates } from "../../../state/onboard-session"; import type { SandboxEntry } from "../../../state/registry"; +import { getSandboxEntryInference } from "../../../state/registry-entry-view"; import { toolDisclosureOrDefault } from "../../../tool-disclosure"; +import { resolveSandboxGatewayName } from "../../gateway-binding"; import { withSandboxPhaseTrace } from "../../tracing"; import type { SandboxCreateIntent } from "../../types"; import { branchTo, type OnboardStateTransitionResult } from "../result"; @@ -40,6 +46,7 @@ export interface SandboxStateOptions< /** Internal rebuild mode: null web-search state is an authoritative disable, not a prompt. */ authoritativeResumeConfig?: boolean; resumeAgentChanged: boolean; + gatewayName: string; session: Session | null; sandboxName: string | null; model: string; @@ -60,6 +67,11 @@ export interface SandboxStateOptions< rootDir: string; env: NodeJS.ProcessEnv; deps: dcodeResume.Deps & { + checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; + withGatewayRouteMutationLock( + gatewayName: string, + operation: () => Promise | T, + ): Promise; resolvePath(value: string): string; agentSupportsWebSearch( agent: Agent, @@ -416,39 +428,86 @@ class SandboxStateFlow< return dcodeResume.preserveManagedDcodeRegistryEntry(this.options, decision); } + private assertGatewayRouteCompatible(sandboxName: string | null): void { + const targetEntry = sandboxName ? this.deps.getSandboxRegistryEntry(sandboxName) : null; + if (!sandboxName || !targetEntry) { + this.failGatewayRouteCheck( + ` Error: sandbox route reservation '${sandboxName ?? "unknown"}' disappeared while onboarding was in progress. Retry onboarding.`, + ); + } + if (getSandboxEntryInference(targetEntry).kind !== "configured") { + this.failGatewayRouteCheck( + ` Error: sandbox '${sandboxName}' has incomplete route metadata, so its shared-gateway compatibility cannot be proven. Remove and re-onboard that sandbox.`, + ); + } + if (resolveSandboxGatewayName(targetEntry) !== this.options.gatewayName) { + this.failGatewayRouteCheck( + ` Error: sandbox '${sandboxName}' changed OpenShell gateways while onboarding was in progress. Retry onboarding.`, + ); + } + const compatibility = this.deps.checkGatewayRouteCompatibility({ + gatewayName: this.options.gatewayName, + sandboxName: null, + route: { + provider: this.options.provider, + model: this.options.model, + endpointUrl: this.options.endpointUrl, + preferredInferenceApi: this.options.preferredInferenceApi, + credentialEnv: this.options.credentialEnv, + }, + }); + if (!compatibility.ok) { + this.failGatewayRouteCheck(` Error: ${formatGatewayRouteConflict(compatibility)}`); + } + } + + private failGatewayRouteCheck(message: string): never { + this.deps.error(message); + this.deps.exitProcess(1); + throw new Error("exitProcess returned while aborting an incompatible gateway route"); + } + private async reuseSandbox( state: SandboxStepState, ): Promise> { - if (state.webSearchConfig) { - const provider = webSearchProviderForConfig( - state.webSearchConfig as unknown as SharedWebSearchConfig, - ); - this.deps.note( - ` [resume] Reusing ${webSearchLabelFor(provider)} configuration already baked into the sandbox.`, + return this.deps.withGatewayRouteMutationLock(this.options.gatewayName, async () => { + this.assertGatewayRouteCompatible(state.sandboxName); + if (state.webSearchConfig) { + const provider = webSearchProviderForConfig( + state.webSearchConfig as unknown as SharedWebSearchConfig, + ); + this.deps.note( + ` [resume] Reusing ${webSearchLabelFor(provider)} configuration already baked into the sandbox.`, + ); + } + const messaging = reconcileReusedSandboxMessaging( + state.session?.messagingPlan ?? null, + this.options.agent, + this.deps, ); - } - const messaging = reconcileReusedSandboxMessaging( - state.session?.messagingPlan ?? null, - this.options.agent, - this.deps, - ); - if (messaging.changed) { - this.deps.updateSession((current) => { - current.messagingPlan = messaging.plan; - return current; + if (messaging.changed) { + this.deps.updateSession((current) => { + current.messagingPlan = messaging.plan; + return current; + }); + } + this.backfillReusedSandboxFidelity(state); + if (state.sandboxName) { + this.deps.updateSandboxRegistry(state.sandboxName, { + pendingRouteReservation: undefined, + }); + } + this.deps.skippedStepMessage("sandbox", state.sandboxName); + const skippedSession = await this.deps.recordStateSkipped("sandbox", { + reason: "resume", + sandboxName: state.sandboxName, }); - } - this.backfillReusedSandboxFidelity(state); - this.deps.skippedStepMessage("sandbox", state.sandboxName); - const skippedSession = await this.deps.recordStateSkipped("sandbox", { - reason: "resume", - sandboxName: state.sandboxName, + return { + ...state, + session: skippedSession, + selectedMessagingChannels: messaging.selectedChannels, + }; }); - return { - ...state, - session: skippedSession, - selectedMessagingChannels: messaging.selectedChannels, - }; } private backfillReusedSandboxFidelity(state: SandboxStepState): void { @@ -507,68 +566,85 @@ class SandboxStateFlow< this.options.hermesToolGateways, ); const resourceProfile = await this.deps.selectResourceProfileForSandbox(); - if (this.options.fresh) { - this.deps.stopStaleDashboardListenersForSandbox( - this.deps.listRegistrySandboxes().sandboxes, + const createAndRecord = async (): Promise> => { + this.assertGatewayRouteCompatible(requestedSandboxName); + await applySandboxResumeDecision(decision, state.sandboxName, this.deps); + await this.deps.startRecordedStep("sandbox", { + provider: this.options.provider, + model: this.options.model, + }); + if (this.options.fresh) { + this.deps.stopStaleDashboardListenersForSandbox( + this.deps.listRegistrySandboxes().sandboxes, + requestedSandboxName, + ); + } + this.deps.updateSession((current) => { + current.messagingPlan = messagingPlan; + return current; + }); + const sandboxName = await withSandboxPhaseTrace( requestedSandboxName, + this.options.provider, + this.options.model, + (this.options.agent as { name?: string } | null)?.name, + () => + this.deps.createSandbox( + this.options.gpu, + this.options.model, + this.options.provider, + this.options.preferredInferenceApi, + requestedSandboxName, + state.webSearchConfig, + state.selectedMessagingChannels, + this.options.fromDockerfile, + this.options.agent, + this.options.controlUiPort, + this.options.sandboxGpuConfig, + resourceProfile, + effectiveHermesToolGateways, + this.options.hermesAuthMethod, + { + recreate: decision.kind !== "create", + toolDisclosure: toolDisclosureOrDefault(state.session?.toolDisclosure), + }, + ), ); - } - const sandboxName = await withSandboxPhaseTrace( - requestedSandboxName, - this.options.provider, - this.options.model, - (this.options.agent as { name?: string } | null)?.name, - () => - this.deps.createSandbox( - this.options.gpu, - this.options.model, - this.options.provider, - this.options.preferredInferenceApi, - requestedSandboxName, - state.webSearchConfig, - state.selectedMessagingChannels, - this.options.fromDockerfile, - this.options.agent, - this.options.controlUiPort, - this.options.sandboxGpuConfig, - resourceProfile, - effectiveHermesToolGateways, - this.options.hermesAuthMethod, - { - recreate: decision.kind !== "create", - toolDisclosure: toolDisclosureOrDefault(state.session?.toolDisclosure), - }, - ), - ); - // createSandbox() owns the build fingerprint. In particular, reusing an - // image must not stamp it with the current version and hide build drift. - const { nemoclawVersion: _builtFingerprint, ...agentRegistryFields } = - this.deps.getSandboxAgentRegistryFields(this.options.agent, !this.options.fromDockerfile); - // Preserve the validated route and credential env-var name, never a credential value. - this.deps.updateSandboxRegistry(sandboxName, { - model: this.options.model, - provider: this.options.provider, - endpointUrl: this.options.endpointUrl, - credentialEnv: this.options.credentialEnv, - nimContainer: this.options.nimContainer, - preferredInferenceApi: this.options.preferredInferenceApi, - ...agentRegistryFields, - }); - // Finalization marks the default so a cancelled onboarding cannot leave a - // partially configured sandbox selected as the default. - const completedSession = await this.deps.recordStepComplete( - "sandbox", - this.deps.toSessionUpdates({ - sandboxName, - provider: this.options.provider, + // createSandbox() owns the build fingerprint. In particular, reusing an + // image must not stamp it with the current version and hide build drift. + const { nemoclawVersion: _builtFingerprint, ...agentRegistryFields } = + this.deps.getSandboxAgentRegistryFields(this.options.agent, !this.options.fromDockerfile); + // Preserve the validated route and credential env-var name, never a credential value. + this.deps.updateSandboxRegistry(sandboxName, { model: this.options.model, + provider: this.options.provider, + endpointUrl: this.options.endpointUrl, + credentialEnv: this.options.credentialEnv, nimContainer: this.options.nimContainer, - webSearchConfig: state.webSearchConfig, - messagingPlan, - hermesToolGateways: effectiveHermesToolGateways, - }), - ); - return { ...state, sandboxName, session: completedSession }; + preferredInferenceApi: this.options.preferredInferenceApi, + ...agentRegistryFields, + }); + // Finalization marks the default so a cancelled onboarding cannot leave a + // partially configured sandbox selected as the default. + const completedSession = await this.deps.recordStepComplete( + "sandbox", + this.deps.toSessionUpdates({ + sandboxName, + provider: this.options.provider, + model: this.options.model, + nimContainer: this.options.nimContainer, + webSearchConfig: state.webSearchConfig, + messagingPlan, + hermesToolGateways: effectiveHermesToolGateways, + }), + ); + return { ...state, sandboxName, session: completedSession }; + }; + const withGatewayLock = () => + this.deps.withGatewayRouteMutationLock(this.options.gatewayName, createAndRecord); + return this.deps.withSandboxMutationLock + ? this.deps.withSandboxMutationLock(requestedSandboxName, withGatewayLock) + : withGatewayLock(); } private async recreateSandbox( @@ -592,14 +668,6 @@ class SandboxStateFlow< state.webSearchConfig as unknown as SharedWebSearchConfig | null, webSearchConfig as unknown as SharedWebSearchConfig | null, ); - // Validate the replacement provider before any resume cleanup removes the - // still-live sandbox from the registry. A bad or missing credential must - // leave the existing sandbox recoverable. - await applySandboxResumeDecision(decision, state.sandboxName, this.deps); - await this.deps.startRecordedStep("sandbox", { - provider: this.options.provider, - model: this.options.model, - }); const requestedSandboxName = state.sandboxName ?? (await this.deps.promptValidatedSandboxName(this.options.agent)); const messaging = await reconcileSandboxMessaging({ @@ -609,14 +677,9 @@ class SandboxStateFlow< agent: this.options.agent, deps: this.deps, }); - const session = this.deps.updateSession((current) => { - current.messagingPlan = messaging.plan; - return current; - }); return this.createAndRecordSandbox( { ...state, - session, sandboxName: requestedSandboxName, webSearchConfig, webSearchConfigChanged, diff --git a/src/lib/onboard/setup-inference-route-containment.test.ts b/src/lib/onboard/setup-inference-route-containment.test.ts index e613eb5e567..5a7d8e50adb 100644 --- a/src/lib/onboard/setup-inference-route-containment.test.ts +++ b/src/lib/onboard/setup-inference-route-containment.test.ts @@ -2,6 +2,8 @@ // SPDX-License-Identifier: Apache-2.0 import { describe, expect, it, vi } from "vitest"; +import { checkGatewayRouteCompatibility } from "../inference/gateway-route-compatibility"; +import type { SandboxEntry } from "../state/registry"; import { createSetupInference, type SetupInferenceDeps } from "./setup-inference"; describe("onboard shared gateway route containment", () => { @@ -33,6 +35,15 @@ describe("onboard shared gateway route containment", () => { }); const setupInference = createSetupInference({ checkGatewayRouteCompatibility, + withSandboxMutationLock: async (_sandboxName: string, operation: () => Promise | T) => + await operation(), + withGatewayRouteMutationLock: async ( + _gatewayName: string, + operation: () => Promise | T, + ) => { + events.push("lock"); + return await operation(); + }, step: () => events.push("step"), getGatewayName, runOpenshell, @@ -54,7 +65,7 @@ describe("onboard shared gateway route containment", () => { ), ).rejects.toThrow("exit 1"); - expect(events[0]).toBe("guard"); + expect(events.slice(0, 2)).toEqual(["lock", "guard"]); expect(getGatewayName).toHaveBeenCalledOnce(); expect(checkGatewayRouteCompatibility).toHaveBeenCalledWith( expect.objectContaining({ gatewayName: "nemoclaw-9090" }), @@ -67,4 +78,90 @@ describe("onboard shared gateway route containment", () => { expect(error).toHaveBeenCalledWith(expect.stringContaining("stopped-sandbox")); expect(exitProcess).toHaveBeenCalledWith(1); }); + + it("reserves a fresh route before smoke failure lets another setup mutate it (#6315)", async () => { + const reservations: SandboxEntry[] = []; + let lockTail = Promise.resolve(); + const withGatewayRouteMutationLock = async ( + _gatewayName: string, + operation: () => Promise | T, + ): Promise => { + const previous = lockTail; + let release!: () => void; + lockTail = new Promise((resolve) => { + release = resolve; + }); + await previous; + try { + return await operation(); + } finally { + release(); + } + }; + const updateSandbox = vi.fn( + (name: string, route: Parameters[1]) => { + reservations.push({ name, ...route }); + return true; + }, + ); + const runOpenshell = vi.fn(() => ({ status: 0 })); + const exitProcess = vi.fn((code: number): never => { + throw new Error(`exit ${code}`); + }); + const setupInference = createSetupInference({ + checkGatewayRouteCompatibility: ( + request: Parameters[0], + ) => checkGatewayRouteCompatibility({ ...request, sandboxes: reservations }), + withSandboxMutationLock: async (_sandboxName: string, operation: () => Promise | T) => + await operation(), + withGatewayRouteMutationLock, + step: vi.fn(), + getGatewayName: () => "nemoclaw", + runOpenshell, + updateSandbox, + upsertProvider: vi.fn(() => ({ ok: true })), + verifyInferenceRoute: vi.fn(), + verifyOnboardInferenceSmoke: vi.fn(() => { + throw new Error("smoke failed"); + }), + isNonInteractive: () => true, + hermesProviderAuth: { HERMES_PROVIDER_NAME: "hermes-provider" }, + isRoutedInferenceProvider: () => true, + reconcileModelRouter: vi.fn(async () => undefined), + routedInference: { + upsertRoutedProvider: vi.fn(() => ({ + ok: true, + endpointUrl: "http://router.test/v1", + result: { ok: true }, + })), + }, + hydrateCredentialEnv: vi.fn(() => "secret"), + redact: (value: string) => value, + compactText: (value: string) => value, + log: vi.fn(), + error: vi.fn(), + exitProcess, + } as unknown as SetupInferenceDeps); + + const results = await Promise.allSettled([ + setupInference("alpha", "model-a", "router-a", "http://router-a.test/v1", "ROUTER_KEY"), + setupInference("beta", "model-b", "router-b", "http://router-b.test/v1", "ROUTER_KEY"), + ]); + + expect(results).toEqual([ + { status: "rejected", reason: expect.objectContaining({ message: "smoke failed" }) }, + { status: "rejected", reason: expect.objectContaining({ message: "exit 1" }) }, + ]); + expect(runOpenshell).toHaveBeenCalledTimes(1); + expect(updateSandbox).toHaveBeenCalledWith("alpha", { + provider: "router-a", + model: "model-a", + endpointUrl: "http://router-a.test/v1", + credentialEnv: "ROUTER_KEY", + preferredInferenceApi: null, + gatewayName: "nemoclaw", + }); + expect(reservations).toHaveLength(1); + expect(exitProcess).toHaveBeenCalledWith(1); + }); }); diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index 7b964114aed..38602dcfcb5 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -5,11 +5,13 @@ import { type CurrentGatewayRouteCompatibilityCheck, formatGatewayRouteConflict, } from "../inference/gateway-route-compatibility"; +import { withGatewayRouteMutationLock } from "../inference/gateway-route-mutation-lock"; import { assertNoExplicitOpenShellGatewayEndpoint, assertNoOpenShellGatewayEndpointOverride, type OpenShellGatewayEndpointEnvironment, } from "../openshell-gateway-endpoint-guard"; +import { withSandboxMutationLock } from "../state/mcp-lifecycle-lock"; export { assertNoOpenShellGatewayEndpointOverride }; @@ -71,6 +73,8 @@ type ProviderBranchDeps = Pick< export type SetupInferenceDeps = ProviderBranchDeps & { checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck; + withGatewayRouteMutationLock: typeof withGatewayRouteMutationLock; + withSandboxMutationLock: typeof withSandboxMutationLock; step: (current: number, total: number, label: string) => void; getGatewayName: () => string; runOpenshell: import("./openshell-cli").OpenshellCliHelpers["runOpenshell"]; @@ -85,7 +89,7 @@ export type SetupInferenceDeps = ProviderBranchDeps & { verifyInferenceRoute: (gatewayName: string, provider: string, model: string) => void; providerExistsInGateway: (name: string, gatewayName: string) => boolean; run: typeof import("../runner").run; - updateSandbox: CommonDeps["registry"]["updateSandbox"]; + updateSandbox: typeof import("../state/registry").reserveSandboxInferenceRoute; localInferenceTimeoutSecs: number; vllmLocalCredentialEnv: string; ollamaProxyCredentialEnv: string; @@ -199,153 +203,186 @@ export function createSetupInference( options: ProviderInferenceSetupOptions = {}, ): Promise { const gatewayName = options.gatewayName ?? deps.getGatewayName(); - const compatibility = deps.checkGatewayRouteCompatibility({ - gatewayName, - sandboxName, - route: { - provider, - model, - endpointUrl, - preferredInferenceApi: options.preferredInferenceApi ?? null, - }, - }); - if (!compatibility.ok) { - deps.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); - return deps.exitProcess(1); - } - deps.step(4, 8, "Setting up inference provider"); - const runGatewayOpenshell = createGatewayScopedOpenshellRunner(deps.runOpenshell, gatewayName); + const mutateGatewayRoute = (): Promise => + deps.withGatewayRouteMutationLock(gatewayName, async () => { + const compatibility = deps.checkGatewayRouteCompatibility({ + gatewayName, + sandboxName, + route: { + provider, + model, + endpointUrl, + preferredInferenceApi: options.preferredInferenceApi ?? null, + }, + }); + if (!compatibility.ok) { + deps.error(` Error: ${formatGatewayRouteConflict(compatibility)}`); + return deps.exitProcess(1); + } + deps.step(4, 8, "Setting up inference provider"); + const runGatewayOpenshell = createGatewayScopedOpenshellRunner( + deps.runOpenshell, + gatewayName, + ); + let routeReserved = false; + const reserveRoute = (name: string, selectedProvider: string, selectedModel: string) => { + if (routeReserved) return true; + const reserved = deps.updateSandbox(name, { + provider: selectedProvider, + model: selectedModel, + endpointUrl, + credentialEnv, + preferredInferenceApi: options.preferredInferenceApi ?? null, + gatewayName, + }); + routeReserved = reserved; + return reserved; + }; - const commonDeps = { - runOpenshell: runGatewayOpenshell, - upsertProvider: bindGatewayUpsertProvider(deps.upsertProvider, gatewayName), - verifyInferenceRoute: (selectedProvider: string, selectedModel: string) => - deps.verifyInferenceRoute(gatewayName, selectedProvider, selectedModel), - verifyOnboardInferenceSmoke: deps.verifyOnboardInferenceSmoke, - isNonInteractive: deps.isNonInteractive, - registry: { updateSandbox: deps.updateSandbox }, - exitProcess: deps.exitProcess, - error: deps.error, - log: deps.log, - } satisfies CommonDeps; + const commonDeps = { + runOpenshell: runGatewayOpenshell, + upsertProvider: bindGatewayUpsertProvider(deps.upsertProvider, gatewayName), + verifyInferenceRoute: (selectedProvider: string, selectedModel: string) => { + if (sandboxName) reserveRoute(sandboxName, selectedProvider, selectedModel); + deps.verifyInferenceRoute(gatewayName, selectedProvider, selectedModel); + }, + verifyOnboardInferenceSmoke: deps.verifyOnboardInferenceSmoke, + isNonInteractive: deps.isNonInteractive, + registry: { + updateSandbox: (name: string) => reserveRoute(name, provider, model), + }, + exitProcess: deps.exitProcess, + error: deps.error, + log: deps.log, + } satisfies CommonDeps; - if (provider === deps.hermesProviderAuth.HERMES_PROVIDER_NAME) { - return inferenceProviders.setupHermesProviderInference( - { - sandboxName, - model, - provider, - endpointUrl, - credentialEnv, - hermesAuthMethod, - hermesToolGateways, - }, - { - ...commonDeps, - hermesProviderAuth: deps.hermesProviderAuth, - getHermesToolGatewayBroker: deps.getHermesToolGatewayBroker, - providerExistsInGateway: (name: string) => - deps.providerExistsInGateway(name, gatewayName), - normalizeHermesAuthMethod: deps.normalizeHermesAuthMethod, - resolveHermesNousApiKey: deps.resolveHermesNousApiKey, - checkHermesProviderStoreReachable: deps.checkHermesProviderStoreReachable, - hermesAuthMethodLabel: deps.hermesAuthMethodLabel, - hermesConstants: deps.hermesConstants, - requireValue: deps.requireValue, - redact: deps.redact, - compactText: deps.compactText, - lookup: deps.lookup, - }, - ); - } + if (provider === deps.hermesProviderAuth.HERMES_PROVIDER_NAME) { + return inferenceProviders.setupHermesProviderInference( + { + sandboxName, + model, + provider, + endpointUrl, + credentialEnv, + hermesAuthMethod, + hermesToolGateways, + }, + { + ...commonDeps, + hermesProviderAuth: deps.hermesProviderAuth, + getHermesToolGatewayBroker: deps.getHermesToolGatewayBroker, + providerExistsInGateway: (name: string) => + deps.providerExistsInGateway(name, gatewayName), + normalizeHermesAuthMethod: deps.normalizeHermesAuthMethod, + resolveHermesNousApiKey: deps.resolveHermesNousApiKey, + checkHermesProviderStoreReachable: deps.checkHermesProviderStoreReachable, + hermesAuthMethodLabel: deps.hermesAuthMethodLabel, + hermesConstants: deps.hermesConstants, + requireValue: deps.requireValue, + redact: deps.redact, + compactText: deps.compactText, + lookup: deps.lookup, + }, + ); + } - if (inferenceProviders.isRemoteProviderName(provider)) { - const outcome = await inferenceProviders.setupRemoteProviderInference( - { - sandboxName, - model, - provider, - endpointUrl, - credentialEnv, - reuseGatewayCredentialWithoutLocalKey: - options.reuseGatewayCredentialWithoutLocalKey === true, - preferredInferenceApi: options.preferredInferenceApi ?? null, - }, - { - ...commonDeps, - REMOTE_PROVIDER_CONFIG: deps.REMOTE_PROVIDER_CONFIG, - hydrateCredentialEnv: deps.hydrateCredentialEnv, - promptValidationRecovery: deps.promptValidationRecovery, - classifyApplyFailure: deps.classifyApplyFailure, - LOCAL_INFERENCE_TIMEOUT_SECS: deps.localInferenceTimeoutSecs, - bedrockRuntimeOnboard: deps.bedrockRuntimeOnboard, - redact: deps.redact, - compactText: deps.compactText, - probeOpenAiLikeEndpoint: deps.probeOpenAiLikeEndpoint, - readGatewayProviderMetadata: deps.readGatewayProviderMetadata, - deleteGatewayProvider: deps.deleteGatewayProvider, - }, - ); - if (outcome.done) return outcome.result; - } else if (provider === "vllm-local") { - const outcome = await inferenceProviders.setupVllmLocalInference( - { model, provider }, - { - ...commonDeps, - validateLocalProvider: deps.validateLocalProvider, - getLocalProviderHealthCheck: deps.getLocalProviderHealthCheck, - getLocalProviderBaseUrl: deps.getLocalProviderBaseUrl, - applyLocalInferenceRoute: resolveLocalInferenceRouteApplier(deps, runGatewayOpenshell), - run: deps.run, - VLLM_LOCAL_CREDENTIAL_ENV: deps.vllmLocalCredentialEnv, - }, - ); - if (outcome.done) return outcome.result; - } else if (provider === "ollama-local") { - const outcome = await inferenceProviders.setupOllamaLocalInference( - { model, provider, allowToolsIncompatible: options.allowToolsIncompatible === true }, - { - ...commonDeps, - validateLocalProvider: deps.validateLocalProvider, - getLocalProviderBaseUrl: deps.getLocalProviderBaseUrl, - applyLocalInferenceRoute: resolveLocalInferenceRouteApplier(deps, runGatewayOpenshell), - getOllamaWarmupCommand: deps.getOllamaWarmupCommand, - run: deps.run, - shouldFrontOllamaWithProxy: deps.shouldFrontOllamaWithProxy, - ensureOllamaAuthProxy: deps.ensureOllamaAuthProxy, - isProxyHealthy: deps.isProxyHealthy, - getOllamaProxyToken: deps.getOllamaProxyToken, - persistAndProbeOllamaProxy: deps.persistAndProbeOllamaProxy, - localInference: deps.localInference, - OLLAMA_PROXY_CREDENTIAL_ENV: deps.ollamaProxyCredentialEnv, - }, - ); - if (outcome.done) return outcome.result; - } else if (deps.isRoutedInferenceProvider(provider)) { - await inferenceProviders.setupRoutedInference( - { model, provider, endpointUrl, credentialEnv }, - { - ...commonDeps, - reconcileModelRouter: deps.reconcileModelRouter, - routedInference: deps.routedInference, - hydrateCredentialEnv: deps.hydrateCredentialEnv, - redact: deps.redact, - compactText: deps.compactText, - }, - ); - } else { - deps.error(` Unsupported provider configuration: ${provider}`); - deps.exitProcess(1); - } + if (inferenceProviders.isRemoteProviderName(provider)) { + const outcome = await inferenceProviders.setupRemoteProviderInference( + { + sandboxName, + model, + provider, + endpointUrl, + credentialEnv, + reuseGatewayCredentialWithoutLocalKey: + options.reuseGatewayCredentialWithoutLocalKey === true, + preferredInferenceApi: options.preferredInferenceApi ?? null, + }, + { + ...commonDeps, + REMOTE_PROVIDER_CONFIG: deps.REMOTE_PROVIDER_CONFIG, + hydrateCredentialEnv: deps.hydrateCredentialEnv, + promptValidationRecovery: deps.promptValidationRecovery, + classifyApplyFailure: deps.classifyApplyFailure, + LOCAL_INFERENCE_TIMEOUT_SECS: deps.localInferenceTimeoutSecs, + bedrockRuntimeOnboard: deps.bedrockRuntimeOnboard, + redact: deps.redact, + compactText: deps.compactText, + probeOpenAiLikeEndpoint: deps.probeOpenAiLikeEndpoint, + readGatewayProviderMetadata: deps.readGatewayProviderMetadata, + deleteGatewayProvider: deps.deleteGatewayProvider, + }, + ); + if (outcome.done) return outcome.result; + } else if (provider === "vllm-local") { + const outcome = await inferenceProviders.setupVllmLocalInference( + { model, provider }, + { + ...commonDeps, + validateLocalProvider: deps.validateLocalProvider, + getLocalProviderHealthCheck: deps.getLocalProviderHealthCheck, + getLocalProviderBaseUrl: deps.getLocalProviderBaseUrl, + applyLocalInferenceRoute: resolveLocalInferenceRouteApplier( + deps, + runGatewayOpenshell, + ), + run: deps.run, + VLLM_LOCAL_CREDENTIAL_ENV: deps.vllmLocalCredentialEnv, + }, + ); + if (outcome.done) return outcome.result; + } else if (provider === "ollama-local") { + const outcome = await inferenceProviders.setupOllamaLocalInference( + { model, provider, allowToolsIncompatible: options.allowToolsIncompatible === true }, + { + ...commonDeps, + validateLocalProvider: deps.validateLocalProvider, + getLocalProviderBaseUrl: deps.getLocalProviderBaseUrl, + applyLocalInferenceRoute: resolveLocalInferenceRouteApplier( + deps, + runGatewayOpenshell, + ), + getOllamaWarmupCommand: deps.getOllamaWarmupCommand, + run: deps.run, + shouldFrontOllamaWithProxy: deps.shouldFrontOllamaWithProxy, + ensureOllamaAuthProxy: deps.ensureOllamaAuthProxy, + isProxyHealthy: deps.isProxyHealthy, + getOllamaProxyToken: deps.getOllamaProxyToken, + persistAndProbeOllamaProxy: deps.persistAndProbeOllamaProxy, + localInference: deps.localInference, + OLLAMA_PROXY_CREDENTIAL_ENV: deps.ollamaProxyCredentialEnv, + }, + ); + if (outcome.done) return outcome.result; + } else if (deps.isRoutedInferenceProvider(provider)) { + await inferenceProviders.setupRoutedInference( + { model, provider, endpointUrl, credentialEnv }, + { + ...commonDeps, + reconcileModelRouter: deps.reconcileModelRouter, + routedInference: deps.routedInference, + hydrateCredentialEnv: deps.hydrateCredentialEnv, + redact: deps.redact, + compactText: deps.compactText, + }, + ); + } else { + deps.error(` Unsupported provider configuration: ${provider}`); + deps.exitProcess(1); + } - commonDeps.verifyInferenceRoute(provider, model); - if (options.skipHostInferenceSmoke === true) - deps.log(" Reusing existing gateway credential; skipping host inference smoke."); - else deps.verifyOnboardInferenceSmoke({ provider, model, endpointUrl, credentialEnv }); - if (sandboxName) { - deps.updateSandbox(sandboxName, { model, provider }); - } - deps.log(` ✓ Inference route set: ${provider} / ${model}`); - return { ok: true }; + commonDeps.verifyInferenceRoute(provider, model); + if (options.skipHostInferenceSmoke === true) + deps.log(" Reusing existing gateway credential; skipping host inference smoke."); + else deps.verifyOnboardInferenceSmoke({ provider, model, endpointUrl, credentialEnv }); + if (sandboxName) { + commonDeps.registry.updateSandbox(sandboxName); + } + deps.log(` ✓ Inference route set: ${provider} / ${model}`); + return { ok: true as const }; + }); + return sandboxName + ? deps.withSandboxMutationLock(sandboxName, mutateGatewayRoute) + : mutateGatewayRoute(); }; } diff --git a/src/lib/onboard/setup-nim-flow.test.ts b/src/lib/onboard/setup-nim-flow.test.ts index fd111675659..96f1aac1461 100644 --- a/src/lib/onboard/setup-nim-flow.test.ts +++ b/src/lib/onboard/setup-nim-flow.test.ts @@ -558,7 +558,7 @@ describe("createSetupNim", () => { gpu: null, experimental: false, probeOllama: false, - probeVllm: false, + probeVllm: true, }); expect(routeGuard).toHaveBeenCalledWith({ provider: "vllm-local", diff --git a/src/lib/onboard/setup-nim-flow.ts b/src/lib/onboard/setup-nim-flow.ts index 6fcdeeb08c2..e8cb1cd050e 100644 --- a/src/lib/onboard/setup-nim-flow.ts +++ b/src/lib/onboard/setup-nim-flow.ts @@ -188,7 +188,7 @@ const OLLAMA_PROBE_PROVIDER_KEYS = new Set([ "install-windows-ollama", ]); const VLLM_ROUTE_PROVIDER_KEYS = new Set(["vllm", "install-vllm"]); -const VLLM_PROBE_PROVIDER_KEYS = new Set(["vllm"]); +const VLLM_PROBE_PROVIDER_KEYS = new Set(["vllm", "install-vllm"]); function localProviderProbeIntent(providerKey: string | null): { ollama: boolean; diff --git a/src/lib/registry-recovery-action.ts b/src/lib/registry-recovery-action.ts index 3c9139f2224..724efdda6e0 100644 --- a/src/lib/registry-recovery-action.ts +++ b/src/lib/registry-recovery-action.ts @@ -4,15 +4,23 @@ import { resolveOpenshell } from "./adapters/openshell/resolve"; import { captureOpenshell } from "./adapters/openshell/runtime"; import { OPENSHELL_PROBE_TIMEOUT_MS } from "./adapters/openshell/timeouts"; +import { GATEWAY_PORT } from "./core/ports"; import { getNamedGatewayLifecycleState, recoverNamedGatewayRuntime, } from "./gateway-runtime-action"; +import { + checkGatewayRouteCompatibility, + formatGatewayRouteConflict, +} from "./inference/gateway-route-compatibility"; +import { withGatewayRouteMutationLock } from "./inference/gateway-route-mutation-lock"; +import { resolveGatewayName, resolveSandboxGatewayName } from "./onboard/gateway-binding"; import { validateName } from "./runner"; import { parseLiveSandboxEntries } from "./runtime-recovery"; import * as onboardSession from "./state/onboard-session"; import type { SandboxEntry } from "./state/registry"; import * as registry from "./state/registry"; +import { getSandboxEntryInference } from "./state/registry-entry-view"; /** * #5714: a sandbox surfaced display-only by unseeded `nemoclaw list` recovery. @@ -72,7 +80,11 @@ function buildRecoveredSandboxEntry( * merging into an existing one. Returns true only when a new entry was created. * Invalid sandbox names are skipped (returns false). */ -function upsertRecoveredSandbox(name: string, metadata: RecoveredSandboxMetadata = {}) { +function upsertRecoveredSandbox( + name: string, + metadata: RecoveredSandboxMetadata = {}, + gatewayName = resolveGatewayName(GATEWAY_PORT), +) { let validName; try { validName = validateName(name, "sandbox name"); @@ -80,8 +92,34 @@ function upsertRecoveredSandbox(name: string, metadata: RecoveredSandboxMetadata return false; } - const entry = buildRecoveredSandboxEntry(validName, metadata); - if (registry.getSandbox(validName)) { + const existing = registry.getSandbox(validName); + if (existing && resolveSandboxGatewayName(existing) !== gatewayName) return false; + const recovered = buildRecoveredSandboxEntry(validName, metadata); + const entry = { + ...recovered, + provider: existing?.provider ?? recovered.provider ?? null, + model: existing?.model ?? recovered.model ?? null, + endpointUrl: existing?.endpointUrl ?? null, + credentialEnv: existing?.credentialEnv ?? null, + preferredInferenceApi: existing?.preferredInferenceApi ?? null, + gatewayName: existing?.gatewayName ?? gatewayName, + }; + const inference = getSandboxEntryInference(entry); + if (inference.kind === "configured") { + const compatibility = checkGatewayRouteCompatibility({ + gatewayName, + sandboxName: validName, + route: entry, + sandboxes: registry.listSandboxes().sandboxes, + }); + if (!compatibility.ok) { + console.warn( + ` Skipping unsafe registry recovery: ${formatGatewayRouteConflict(compatibility)}`, + ); + return false; + } + } + if (existing) { registry.updateSandbox(validName, entry); return false; } @@ -149,6 +187,7 @@ function seedRecoveryMetadata( current: { sandboxes: SandboxEntry[] }, session: Session | null, requestedSandboxName: string | null, + gatewayName = resolveGatewayName(GATEWAY_PORT), ) { const metadataByName = new Map( current.sandboxes.map((sandbox: SandboxEntry) => [sandbox.name, sandbox]), @@ -180,6 +219,7 @@ function seedRecoveryMetadata( recoveredFromSession = upsertRecoveredSandbox( session.sandboxName, metadataByName.get(session.sandboxName), + gatewayName, ); } return { metadataByName, recoveredFromSession }; @@ -239,7 +279,10 @@ interface LiveGatewayRecovery { */ async function recoverRegistryFromLiveGateway( metadataByName: Map, - { readOnly = false }: { readOnly?: boolean } = {}, + { + readOnly = false, + gatewayName = resolveGatewayName(GATEWAY_PORT), + }: { readOnly?: boolean; gatewayName?: string } = {}, ): Promise { if (!resolveOpenshell()) { return { recoveredFromGateway: 0, ephemeralSandboxes: [] }; @@ -290,7 +333,7 @@ async function recoverRegistryFromLiveGateway( recoveredFromGateway += 1; continue; } - if (upsertRecoveredSandbox(name, metadata)) { + if (upsertRecoveredSandbox(name, metadata, gatewayName)) { recoveredFromGateway += 1; } } @@ -327,19 +370,13 @@ function applyRecoveredDefault( * live sandboxes as display-only entries without persisting them. Returns the * registry listing plus `recoveredFromSession`/`recoveredFromGateway` markers. */ -export async function recoverRegistryEntries({ - requestedSandboxName = null, -}: { - requestedSandboxName?: string | null; -} = {}) { - const current = registry.listSandboxes(); - const session = onboardSession.loadSession(); - const recoveryCheck = shouldRecoverRegistryEntries(current, session, requestedSandboxName); - if (!recoveryCheck.shouldRecover) { - return { ...current, recoveredFromSession: false, recoveredFromGateway: 0 }; - } - - const seeded = seedRecoveryMetadata(current, session, requestedSandboxName); +async function recoverRegistryEntriesFromSnapshot( + current: ReturnType, + session: Session | null, + requestedSandboxName: string | null, + gatewayName: string, +) { + const seeded = seedRecoveryMetadata(current, session, requestedSandboxName, gatewayName); // A seed is any signal that the user expects a specific sandbox to exist: // existing registry entries, a *confirmed* onboard session, or an explicit // requested name. With a seed we allow active gateway recovery (which may @@ -357,6 +394,7 @@ export async function recoverRegistryEntries({ current.sandboxes.length > 0 || hasConfirmedSession || Boolean(requestedSandboxName); const gateway = await recoverRegistryFromLiveGateway(seeded.metadataByName, { readOnly: !hasRecoverySeed, + gatewayName, }); const recovered = applyRecoveredDefault(current.defaultSandbox, requestedSandboxName, session); // Merge display-only (ephemeral) live-gateway sandboxes that were not @@ -374,3 +412,41 @@ export async function recoverRegistryEntries({ recoveredFromGateway: gateway.recoveredFromGateway, }; } + +export async function recoverRegistryEntries({ + requestedSandboxName = null, +}: { + requestedSandboxName?: string | null; +} = {}) { + const current = registry.listSandboxes(); + const session = onboardSession.loadSession(); + const recoveryCheck = shouldRecoverRegistryEntries(current, session, requestedSandboxName); + if (!recoveryCheck.shouldRecover) { + return { ...current, recoveredFromSession: false, recoveredFromGateway: 0 }; + } + + const hasConfirmedSession = isSessionSandboxConfirmed(session) && Boolean(session?.sandboxName); + const hasRecoverySeed = + current.sandboxes.length > 0 || hasConfirmedSession || Boolean(requestedSandboxName); + const gatewayName = resolveGatewayName(GATEWAY_PORT); + if (!hasRecoverySeed) { + return recoverRegistryEntriesFromSnapshot(current, session, requestedSandboxName, gatewayName); + } + return withGatewayRouteMutationLock(gatewayName, async () => { + const lockedCurrent = registry.listSandboxes(); + const lockedSession = onboardSession.loadSession(); + const lockedCheck = shouldRecoverRegistryEntries( + lockedCurrent, + lockedSession, + requestedSandboxName, + ); + return lockedCheck.shouldRecover + ? recoverRegistryEntriesFromSnapshot( + lockedCurrent, + lockedSession, + requestedSandboxName, + gatewayName, + ) + : { ...lockedCurrent, recoveredFromSession: false, recoveredFromGateway: 0 }; + }); +} diff --git a/src/lib/registry-recovery-seeded-paths.test.ts b/src/lib/registry-recovery-seeded-paths.test.ts index cfef57cdb4f..4845ec09574 100644 --- a/src/lib/registry-recovery-seeded-paths.test.ts +++ b/src/lib/registry-recovery-seeded-paths.test.ts @@ -70,8 +70,8 @@ import { loadSession } from "./state/onboard-session.js"; const gammaEntry = (policies: string[]): SandboxEntry => ({ name: "gamma", - provider: "existing-provider", - model: "existing-model", + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", gpuEnabled: false, policies, }); @@ -130,6 +130,24 @@ describe("recoverRegistryEntries seeded recovery paths", () => { expect(mockRegistryState.defaultSandbox).toBe("gamma"); }); + it("fails closed instead of restoring a conflicting session route", async () => { + const consoleWarn = vi.spyOn(console, "warn").mockImplementation(() => {}); + mockRegistryState.sandboxes.gamma = { + ...gammaEntry([]), + provider: "existing-provider", + model: "existing-model", + }; + mockRegistryState.defaultSandbox = "gamma"; + vi.mocked(loadSession).mockReturnValue(completedSession("alpha", [])); + vi.mocked(parseLiveSandboxEntries).mockReturnValue([{ name: "alpha", phase: "Ready" }]); + + const result = await recoverRegistryEntries(); + + expect(result.recoveredFromSession).toBe(false); + expect(mockRegistryState.sandboxes.alpha).toBeUndefined(); + expect(consoleWarn.mock.calls.flat().join("\n")).toContain("gamma"); + }); + it("skips invalid session and live sandbox names during seeded recovery", async () => { mockRegistryState.sandboxes.gamma = gammaEntry([]); mockRegistryState.defaultSandbox = "gamma"; diff --git a/src/lib/state/registry-route-reservation.test.ts b/src/lib/state/registry-route-reservation.test.ts new file mode 100644 index 00000000000..7293c4175ab --- /dev/null +++ b/src/lib/state/registry-route-reservation.test.ts @@ -0,0 +1,88 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; + +describe("sandbox inference route reservation", () => { + afterEach(() => { + vi.unstubAllEnvs(); + vi.resetModules(); + }); + + it("persists a complete route without claiming the default sandbox", async () => { + const home = await fs.mkdtemp(path.join(os.tmpdir(), "nemoclaw-route-reservation-")); + vi.stubEnv("HOME", home); + vi.resetModules(); + try { + const registry = await import("./registry"); + + expect( + registry.reserveSandboxInferenceRoute("alpha", { + provider: "compatible-endpoint", + model: "model-a", + endpointUrl: "https://api.example.test/v1", + credentialEnv: "CUSTOM_API_KEY", + preferredInferenceApi: "openai-responses", + gatewayName: "nemoclaw-9090", + }), + ).toBe(true); + + expect(registry.listSandboxes()).toMatchObject({ + defaultSandbox: null, + sandboxes: [ + { + name: "alpha", + provider: "compatible-endpoint", + model: "model-a", + endpointUrl: "https://api.example.test/v1", + credentialEnv: "CUSTOM_API_KEY", + preferredInferenceApi: "openai-responses", + gatewayName: "nemoclaw-9090", + }, + ], + }); + expect(registry.getDefault()).toBeNull(); + expect(registry.setDefault("alpha")).toBe(false); + } finally { + await fs.rm(home, { recursive: true, force: true }); + } + }); + + it("retargets an existing row to the gateway protected by the reservation", async () => { + const home = await fs.mkdtemp(path.join(os.tmpdir(), "nemoclaw-route-reservation-")); + vi.stubEnv("HOME", home); + vi.resetModules(); + try { + const registry = await import("./registry"); + registry.registerSandbox({ + name: "alpha", + provider: "nvidia-prod", + model: "model-a", + gatewayName: "nemoclaw", + gatewayPort: 8080, + }); + + registry.reserveSandboxInferenceRoute("alpha", { + provider: "anthropic-prod", + model: "model-b", + endpointUrl: null, + credentialEnv: "ANTHROPIC_API_KEY", + preferredInferenceApi: "anthropic-messages", + gatewayName: "nemoclaw-9090", + }); + + expect(registry.getSandbox("alpha")).toMatchObject({ + gatewayName: "nemoclaw-9090", + provider: "anthropic-prod", + model: "model-b", + pendingRouteReservation: true, + }); + expect(registry.getSandbox("alpha")?.gatewayPort).toBeUndefined(); + } finally { + await fs.rm(home, { recursive: true, force: true }); + } + }); +}); diff --git a/src/lib/state/registry.ts b/src/lib/state/registry.ts index 2b72ae66a66..b39ba51b9df 100644 --- a/src/lib/state/registry.ts +++ b/src/lib/state/registry.ts @@ -5,7 +5,10 @@ import fs from "node:fs"; import path from "node:path"; import { isErrnoException } from "../core/errno"; import type { InferenceSelection } from "../inference/selection"; -import { inferenceSelectionRegistryFields } from "../inference/selection"; +import { + inferenceSelectionRegistryFields, + normalizeInferenceSelection, +} from "../inference/selection"; import { normalizeToolDisclosure, type ToolDisclosure } from "../tool-disclosure"; import { ensureConfigDir, readConfigFile, writeConfigFile } from "./config-io"; import { @@ -79,6 +82,8 @@ export interface SandboxGpuProofResult { export interface SandboxEntry extends Partial { name: string; + /** Route-only placeholder created before sandbox creation; never eligible as the default. */ + pendingRouteReservation?: true; createdAt?: string; gpuEnabled?: boolean; hostGpuDetected?: boolean; @@ -449,10 +454,16 @@ export function getSandbox(name: string): SandboxEntry | null { export function getDefault(): string | null { const data = load(); - if (data.defaultSandbox && data.sandboxes[data.defaultSandbox]) { + if ( + data.defaultSandbox && + data.sandboxes[data.defaultSandbox] && + data.sandboxes[data.defaultSandbox].pendingRouteReservation !== true + ) { return data.defaultSandbox; } - const names = Object.keys(data.sandboxes); + const names = Object.values(data.sandboxes) + .filter((sandbox) => sandbox.pendingRouteReservation !== true) + .map((sandbox) => sandbox.name); return names.length > 0 ? names[0] || null : null; } @@ -515,6 +526,42 @@ export function registerSandbox(entry: SandboxEntry): void { }); } +type SandboxInferenceRouteReservation = Pick< + InferenceSelection, + "provider" | "model" | "endpointUrl" | "credentialEnv" | "preferredInferenceApi" +> & { + gatewayName: string; +}; + +/** + * Persist a route dependency before releasing the shared-gateway mutation + * lock. A newly reserved row deliberately does not claim the default sandbox; + * normal sandbox registration replaces it after creation completes. + */ +export function reserveSandboxInferenceRoute( + name: string, + route: SandboxInferenceRouteReservation, +): boolean { + return withLock(() => { + const data = load(); + const existing = data.sandboxes[name]; + const normalized = normalizeInferenceSelection(route); + data.sandboxes[name] = { + ...(existing ?? { name, pendingRouteReservation: true as const }), + pendingRouteReservation: true, + provider: normalized.provider, + model: normalized.model, + endpointUrl: normalized.endpointUrl, + credentialEnv: normalized.credentialEnv, + preferredInferenceApi: normalized.preferredInferenceApi, + gatewayName: route.gatewayName, + gatewayPort: undefined, + }; + save(data); + return true; + }); +} + export function updateSandbox(name: string, updates: Partial): boolean { return withLock(() => { const data = load(); @@ -578,7 +625,9 @@ export function listSandboxes(): { sandboxes: SandboxEntry[]; defaultSandbox: st export function setDefault(name: string): boolean { return withLock(() => { - const registry = reversibleRemoval.setDefaultInRegistry(load(), name); + const current = load(); + if (current.sandboxes[name]?.pendingRouteReservation === true) return false; + const registry = reversibleRemoval.setDefaultInRegistry(current, name); if (!registry) return false; save(registry); return true; diff --git a/test/onboard-fsm-live-slices.test.ts b/test/onboard-fsm-live-slices.test.ts index 63cc6e2944e..c5bbc9a36e9 100644 --- a/test/onboard-fsm-live-slices.test.ts +++ b/test/onboard-fsm-live-slices.test.ts @@ -120,6 +120,7 @@ function probeEnvironment(tmpDir: string): NodeJS.ProcessEnv { HOME: tmpDir, TMPDIR: tmpDir, PATH: process.env.PATH || "/usr/bin:/bin", + NEMOCLAW_OPENSHELL_BIN: process.execPath, NODE_ENV: "test", NEMOCLAW_NON_INTERACTIVE: "1", NEMOCLAW_SANDBOX_NAME: "fsm-sandbox", diff --git a/test/onboard-messaging.test.ts b/test/onboard-messaging.test.ts index 445df67d002..3f5e70209b3 100644 --- a/test/onboard-messaging.test.ts +++ b/test/onboard-messaging.test.ts @@ -535,9 +535,9 @@ registry.registerSandbox({ runner.run = (command, opts = {}) => { const normalized = _n(command); commands.push({ command: normalized, env: opts.env || null }); - if (normalized.includes("provider get my-assistant-discord-bridge")) return { status: 0 }; - if (normalized.includes("provider get my-assistant-slack-bridge")) return { status: 0 }; - if (normalized.includes("provider get my-assistant-slack-app")) return { status: 0 }; + if (normalized.includes("provider get -g nemoclaw my-assistant-discord-bridge")) return { status: 0 }; + if (normalized.includes("provider get -g nemoclaw my-assistant-slack-bridge")) return { status: 0 }; + if (normalized.includes("provider get -g nemoclaw my-assistant-slack-app")) return { status: 0 }; if (normalized.includes("provider get")) return { status: 1 }; return { status: 0 }; }; @@ -699,7 +699,7 @@ registry.registerSandbox({ runner.run = (command, opts = {}) => { const normalized = _n(command); commands.push({ command: normalized, env: opts.env || null }); - if (normalized.includes("provider get my-assistant-telegram-bridge")) return { status: 0 }; + if (normalized.includes("provider get -g nemoclaw my-assistant-telegram-bridge")) return { status: 0 }; if (normalized.includes("provider get")) return { status: 1 }; return { status: 0 }; }; diff --git a/test/onboard-prepared-gateway-handoff.test.ts b/test/onboard-prepared-gateway-handoff.test.ts index 3d08ed1c0f1..7f1fec4d8a9 100644 --- a/test/onboard-prepared-gateway-handoff.test.ts +++ b/test/onboard-prepared-gateway-handoff.test.ts @@ -156,11 +156,11 @@ describe("prepared DCode gateway handoff", () => { }); }); - it("continues clearing an ordinary onboard run's ambient gateway (#6195)", () => { + it("scopes an ordinary onboard run to the default gateway (#6315)", () => { assert.deepEqual(runHandoffScenario("ordinary"), { error: null, flowCalls: 1, - gatewayAtInitialFlow: null, + gatewayAtInitialFlow: "nemoclaw", }); }); diff --git a/test/support/connect-flow-test-harness.ts b/test/support/connect-flow-test-harness.ts index cadf71019fe..a273b5adbe5 100644 --- a/test/support/connect-flow-test-harness.ts +++ b/test/support/connect-flow-test-harness.ts @@ -10,6 +10,8 @@ import type { SecretBoundaryRefusalReason } from "../../src/lib/actions/sandbox/ import type { SandboxEntry } from "../../src/lib/state/registry"; type ConnectSandbox = typeof import("../../src/lib/actions/sandbox/connect")["connectSandbox"]; +type GatewayRouteMutationLock = + typeof import("../../src/lib/inference/gateway-route-mutation-lock")["withGatewayRouteMutationLock"]; export const requireDist = createRequire(import.meta.url); export const connectModulePath = "../../src/lib/actions/sandbox/connect.js"; @@ -29,10 +31,12 @@ export type ConnectHarness = { errorSpy: MockInstance; logSpy: MockInstance; preflightVllmSpy: MockInstance; + registryEntries: SandboxEntry[]; runAutoPairSpy: MockInstance; runOpenshellSpy: MockInstance; runSetupDnsProxySpy: MockInstance; spawnSyncSpy: MockInstance; + withGatewayRouteMutationLockSpy: MockInstance; }; export type ConnectHarnessOptions = { @@ -58,6 +62,7 @@ export type ConnectHarnessOptions = { spawnSignal?: NodeJS.Signals | null; spawnStatus?: number | null; sttyThrows?: boolean; + withGatewayRouteMutationLock?: GatewayRouteMutationLock; }; function throwSttyFailure(): never { @@ -99,6 +104,9 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne "../../src/lib/actions/sandbox/gateway-failure-classifier.js", ); const ollamaProxy = requireDist("../../src/lib/inference/ollama/proxy.js"); + const gatewayRouteMutationLock = requireDist( + "../../src/lib/inference/gateway-route-mutation-lock.js", + ); const sandboxVersion = requireDist("../../src/lib/sandbox/version.js"); const registry = requireDist("../../src/lib/state/registry.js"); const sandboxSession = requireDist("../../src/lib/state/sandbox-session.js"); @@ -136,6 +144,13 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne return { status: 0, output: "" }; }); const runOpenshellSpy = vi.spyOn(runtime, "runOpenshell").mockReturnValue({ status: 0 }); + const withGatewayRouteMutationLockSpy = vi + .spyOn(gatewayRouteMutationLock, "withGatewayRouteMutationLock") + .mockImplementation( + (options.withGatewayRouteMutationLock ?? + (async (_gatewayName: string, operation: () => Promise | unknown) => + await operation())) as never, + ); const runSetupDnsProxySpy = vi.spyOn(dns, "runSetupDnsProxy").mockReturnValue({ exitCode: 0 }); const applyVmDnsMonkeypatchSpy = vi .spyOn(vmDnsMonkeypatch, "applyOpenShellVmDnsMonkeypatch") @@ -206,9 +221,11 @@ export function createConnectHarness(options: ConnectHarnessOptions = {}): Conne errorSpy, logSpy, preflightVllmSpy, + registryEntries, runAutoPairSpy, runOpenshellSpy, runSetupDnsProxySpy, spawnSyncSpy, + withGatewayRouteMutationLockSpy, }; } From 1f1fa611af3b365f2e7a76821a0147056b6d7da6 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 18:45:52 -0700 Subject: [PATCH 20/31] refactor(onboard): keep gateway wiring net neutral Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 1df40ea052a..ef173e718b6 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -997,12 +997,8 @@ function upsertMessagingProviders( // biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. const providerExistsInGateway = (name: string, gatewayName: string = GATEWAY_NAME) => onboardProviders.providerExistsInGateway(name, setupInferenceFactory.createGatewayScopedOpenshellRunner(runOpenshell, gatewayName)); -const { - verifyInferenceRoute, - isInferenceRouteReady, - checkGatewayRouteCompatibility, - preflightGatewayRouteDiscovery, -} = inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); +// biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. +const { verifyInferenceRoute, isInferenceRouteReady, checkGatewayRouteCompatibility, preflightGatewayRouteDiscovery } = inferenceRouteHelpers.createInferenceRouteHelpers(runCaptureOpenshell); const { inspectSandboxForCreate, pruneStaleSandboxEntry, @@ -1103,14 +1099,10 @@ const { assertOllamaUpgradeApplied, }); +// biome-ignore format: keep src/lib/onboard.ts net-neutral for growth guardrail. const handleVllmSelection = createSetupNimVllmHandler({ - VLLM_PORT, - runCapture, - getLocalProviderBaseUrl, - getLocalProviderValidationBaseUrl, - isSafeModelId, - requireValue, - validateOpenAiLikeSelection, + VLLM_PORT, runCapture, getLocalProviderBaseUrl, getLocalProviderValidationBaseUrl, + isSafeModelId, requireValue, validateOpenAiLikeSelection, applyVllmRuntimeContextWindow: localInference.applyVllmRuntimeContextWindow, exitProcess: (code) => process.exit(code), }); From c0c13a25a5661eeef95a21aa609ce29acff00f7f Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 19:01:47 -0700 Subject: [PATCH 21/31] test(inference): pin DNS finalization coverage Signed-off-by: Aaron Erickson --- docs/about/release-notes.mdx | 11 +++++ docs/reference/troubleshooting.mdx | 11 +++++ ...ence-set-gateway-route-containment.test.ts | 46 +++++++++++++++++++ .../inference-set-route-containment.ts | 12 +++++ 4 files changed, 80 insertions(+) diff --git a/docs/about/release-notes.mdx b/docs/about/release-notes.mdx index 5c469e3f3e1..61012a07b12 100644 --- a/docs/about/release-notes.mdx +++ b/docs/about/release-notes.mdx @@ -33,6 +33,17 @@ NemoClaw v0.0.74 upgrades the OpenShell policy boundary, adds managed MCP and pr - Shared OpenShell gateways now enforce a single compatible inference route across every registered sandbox, including stopped sandboxes. Onboarding, connect-time repair, and `inference set` reject provider/model conflicts; custom routes must also match the normalized endpoint and API family. As a migration requirement, custom switches must provide `--endpoint-url` and an unambiguous API family, and incomplete legacy custom-route metadata fails closed until the sandbox is removed and re-onboarded with complete metadata. + After backing up an affected workspace, an OpenAI-compatible route can be re-onboarded with complete metadata as follows (replace the example endpoint, model, and sandbox name): + + ```bash + $$nemoclaw legacy-sandbox destroy + NEMOCLAW_PROVIDER=custom \ + NEMOCLAW_ENDPOINT_URL=https://endpoint.example/v1 \ + NEMOCLAW_MODEL=your-model-id \ + NEMOCLAW_PREFERRED_API=openai-completions \ + $$nemoclaw onboard --name legacy-sandbox + ``` + Hermes deterministically selects `openai-completions` for `compatible-anthropic-endpoint`, so that one route may omit `--inference-api`; explicit incompatible values are rejected. Use a different `NEMOCLAW_GATEWAY_PORT` when sandboxes need independent routes. For more information, refer to [Switch Inference Providers](../inference/switch-inference-providers), [NemoClaw CLI Commands Reference](../reference/commands), and [Troubleshooting](../reference/troubleshooting). diff --git a/docs/reference/troubleshooting.mdx b/docs/reference/troubleshooting.mdx index 64e206dfbd4..ab1ad50c98e 100644 --- a/docs/reference/troubleshooting.mdx +++ b/docs/reference/troubleshooting.mdx @@ -339,6 +339,17 @@ An aligned group cannot move to a wholly new route one sandbox at a time because Back up each workspace, remove or migrate the peer sandboxes, switch the last sandbox on the original gateway, then re-onboard the peers with that same route or with separate gateway ports. If the error names incomplete legacy custom-route metadata, back up and remove the affected sandbox, then re-onboard it with an explicit custom endpoint and API family. +For an OpenAI-compatible route, replace the example endpoint, model, and sandbox name in this recovery sequence: + +```bash +$$nemoclaw legacy-sandbox destroy +NEMOCLAW_PROVIDER=custom \ + NEMOCLAW_ENDPOINT_URL=https://endpoint.example/v1 \ + NEMOCLAW_MODEL=your-model-id \ + NEMOCLAW_PREFERRED_API=openai-completions \ + $$nemoclaw onboard --name legacy-sandbox +``` + If the error names an invalid gateway binding, restore the affected row's known-good `gatewayName` and `gatewayPort` metadata from a trusted backup; otherwise back up and remove the sandbox, then re-onboard it. Do not guess or copy a binding from another sandbox because lifecycle commands use it to select the gateway. diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts index c7349c37758..3d1007a9780 100644 --- a/src/lib/actions/inference-set-gateway-route-containment.test.ts +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -10,6 +10,10 @@ import type { ConfigObject } from "../security/credential-filter"; import type { SandboxEntry } from "../state/registry"; import { runInferenceSet } from "./inference-set"; import { baseSession, createDeps, HERMES_TARGET } from "./inference-set.test-support"; +import { + finalizeInferenceSetRoute, + prepareInferenceSetRoute, +} from "./inference-set-route-containment"; const entry = (name: string, overrides: Partial = {}): SandboxEntry => ({ name, @@ -242,6 +246,48 @@ describe("runtime shared gateway route containment", () => { expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); }); + it("catches a DNS change between the preliminary and finalized gateway route checks", async () => { + const firstEndpoint = "https://first.example.test/v1"; + const secondEndpoint = "https://second.example.test/v1"; + const customRoute = { + provider: "compatible-endpoint", + model: "custom/model", + endpointUrl: firstEndpoint, + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + } as const; + const alpha = entry("alpha", customRoute); + const peer = entry("custom-peer", customRoute); + const prepared = prepareInferenceSetRoute({ + entry: alpha, + sandboxName: alpha.name, + provider: customRoute.provider, + model: customRoute.model, + customRoute: { + endpointUrl: firstEndpoint, + credentialEnv: customRoute.credentialEnv, + inferenceApi: customRoute.preferredInferenceApi, + }, + session: null, + sandboxes: [alpha, peer], + }); + const rewriteUrlWithDnsPinning = vi.fn().mockResolvedValueOnce(secondEndpoint); + + await expect( + finalizeInferenceSetRoute({ + prepared, + sandboxName: alpha.name, + provider: customRoute.provider, + model: customRoute.model, + getSandboxes: () => [alpha, peer], + rewriteUrlWithDnsPinning, + }), + ).rejects.toThrow("custom-peer"); + + expect(rewriteUrlWithDnsPinning).toHaveBeenCalledOnce(); + expect(rewriteUrlWithDnsPinning).toHaveBeenCalledWith(firstEndpoint); + }); + it("blocks an incomplete legacy custom target even without a peer (#6315)", async () => { const deps = createDeps({ config: {}, diff --git a/src/lib/actions/inference-set-route-containment.ts b/src/lib/actions/inference-set-route-containment.ts index 40d4971f988..86b627057a3 100644 --- a/src/lib/actions/inference-set-route-containment.ts +++ b/src/lib/actions/inference-set-route-containment.ts @@ -11,6 +11,18 @@ import type { Session } from "../state/onboard-session"; import type { SandboxEntry } from "../state/registry"; import { InferenceSetError } from "./inference-set-error"; +/** + * Custom-route compatibility is intentionally checked twice. The invalid state + * is a requested endpoint whose DNS-pinned identity differs from the route that + * passed the preliminary registry check. The source boundary is the + * operator-supplied `--endpoint-url`; DNS validation is asynchronous, so the + * synchronous preparation phase cannot safely pin it. Finalization therefore + * validates the pinned URL against a fresh registry snapshot before any route, + * config, or registry mutation. The DNS-change regression test in + * inference-set-gateway-route-containment.test.ts protects this boundary. + * Collapse these phases only when preparation can consume fully DNS-validated + * metadata without introducing an earlier mutation or endpoint probe. + */ export type RegistryInferenceMetadata = Pick< SandboxEntry, "endpointUrl" | "credentialEnv" | "preferredInferenceApi" | "nimContainer" From a878d6c4885fdc0bc1bd424f9e7c7bb63aa0da40 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 19:14:37 -0700 Subject: [PATCH 22/31] test(onboard): align route reservation fixtures Signed-off-by: Aaron Erickson --- .../handlers/sandbox-dcode-selection.test.ts | 30 +++++-- .../handlers/sandbox-tool-disclosure.test.ts | 87 +++++++++++-------- 2 files changed, 72 insertions(+), 45 deletions(-) diff --git a/src/lib/onboard/machine/handlers/sandbox-dcode-selection.test.ts b/src/lib/onboard/machine/handlers/sandbox-dcode-selection.test.ts index cc8aa2f93fd..285afcd467f 100644 --- a/src/lib/onboard/machine/handlers/sandbox-dcode-selection.test.ts +++ b/src/lib/onboard/machine/handlers/sandbox-dcode-selection.test.ts @@ -140,19 +140,33 @@ describe("handleSandboxState live DCode selection", () => { }); expect(getDcodeSelectionDrift).not.toHaveBeenCalled(); + expect(calls.createSandbox).not.toHaveBeenCalled(); + expect(calls.updateSandbox).toHaveBeenCalledWith("saved", { + pendingRouteReservation: undefined, + }); + }); + + it("fails closed for missing registry selection before live reuse (#6311)", async () => { + const getDcodeSelectionDrift = vi.fn(() => ({ changed: false, unknown: false })); + const { deps, calls } = createDeps({ + getSandboxReuseState: () => "ready", + getDcodeSelectionDrift, + getSandboxRegistryEntry: (name) => dcodeRegistryEntry(name, {}), + }); + + await expect(handleSandboxState(dcodeOptions(deps))).rejects.toThrow("exit 1"); + expect(calls.createSandbox).not.toHaveBeenCalled(); expect(calls.updateSandbox).not.toHaveBeenCalled(); }); - it.each([ - ["missing fields", {}], - ["stale", { provider: "old-provider", model: "old-model" }], - ])("backfills %s registry selection after verified live reuse (#6311)", async (_label, selection) => { + it("backfills stale registry selection after verified live reuse (#6311)", async () => { const getDcodeSelectionDrift = vi.fn(() => ({ changed: false, unknown: false })); const { deps, calls } = createDeps({ getSandboxReuseState: () => "ready", getDcodeSelectionDrift, - getSandboxRegistryEntry: (name) => dcodeRegistryEntry(name, selection), + getSandboxRegistryEntry: (name) => + dcodeRegistryEntry(name, { provider: "old-provider", model: "old-model" }), }); await handleSandboxState(dcodeOptions(deps)); @@ -162,8 +176,8 @@ describe("handleSandboxState live DCode selection", () => { provider: "provider", model: "model", }); - expect(getDcodeSelectionDrift.mock.invocationCallOrder[0]).toBeLessThan( - calls.updateSandbox.mock.invocationCallOrder[0], - ); + expect(calls.updateSandbox).toHaveBeenCalledWith("saved", { + pendingRouteReservation: undefined, + }); }); }); diff --git a/src/lib/onboard/machine/handlers/sandbox-tool-disclosure.test.ts b/src/lib/onboard/machine/handlers/sandbox-tool-disclosure.test.ts index 847bf6dd7b4..9c702635c15 100644 --- a/src/lib/onboard/machine/handlers/sandbox-tool-disclosure.test.ts +++ b/src/lib/onboard/machine/handlers/sandbox-tool-disclosure.test.ts @@ -11,8 +11,19 @@ vi.mock("../../messaging-channel-setup", () => ({ detectMessagingChannelsFromEnv: vi.fn(() => []), })); +const registeredEntry = (name: string, overrides: Record = {}) => ({ + name, + provider: "provider", + model: "model", + endpointUrl: null, + credentialEnv: null, + preferredInferenceApi: "openai-completions" as const, + gatewayName: "nemoclaw", + ...overrides, +}); + describe("handleSandboxState tool disclosure", () => { - it("does not claim an unregistered live sandbox as a managed legacy migration", async () => { + it("fails closed without claiming an unregistered live sandbox as a managed migration", async () => { const session = createSession({ sandboxName: "saved", toolDisclosure: "progressive" }); session.steps.sandbox.status = "complete"; const { deps, calls } = createDeps({ @@ -20,11 +31,13 @@ describe("handleSandboxState tool disclosure", () => { getSandboxRegistryEntry: () => null, }); - await handleSandboxState({ - ...baseOptions(deps, session), - resume: true, - sandboxName: "saved", - }); + await expect( + handleSandboxState({ + ...baseOptions(deps, session), + resume: true, + sandboxName: "saved", + }), + ).rejects.toThrow("exit 1"); expect(calls.createSandbox).not.toHaveBeenCalled(); expect(calls.removeSandbox).not.toHaveBeenCalled(); @@ -48,12 +61,12 @@ describe("handleSandboxState tool disclosure", () => { session.steps.sandbox.status = "complete"; const { deps, calls } = createDeps({ getSandboxReuseState: () => "ready", - getSandboxRegistryEntry: (name) => ({ - name, - nemoclawVersion: "0.1.0", - toolDisclosure: recorded, - fromDockerfile: null, - }), + getSandboxRegistryEntry: (name) => + registeredEntry(name, { + nemoclawVersion: "0.1.0", + toolDisclosure: recorded, + fromDockerfile: null, + }), }); await handleSandboxState({ @@ -78,11 +91,11 @@ describe("handleSandboxState tool disclosure", () => { updateSession: vi.fn( (mutator: (value: Session) => Session | void) => mutator(session) ?? session, ), - getSandboxRegistryEntry: (name) => ({ - name, - nemoclawVersion: "0.1.0", - toolDisclosure: recordedMode, - }), + getSandboxRegistryEntry: (name) => + registeredEntry(name, { + nemoclawVersion: "0.1.0", + toolDisclosure: recordedMode, + }), }); await handleSandboxState({ @@ -116,11 +129,11 @@ describe("handleSandboxState tool disclosure", () => { session.steps.sandbox.status = "complete"; const { deps, calls } = createDeps({ getSandboxReuseState: () => "ready", - getSandboxRegistryEntry: (name) => ({ - name, - nemoclawVersion: null, - fromDockerfile: "/tmp/Dockerfile.custom", - }), + getSandboxRegistryEntry: (name) => + registeredEntry(name, { + nemoclawVersion: null, + fromDockerfile: "/tmp/Dockerfile.custom", + }), }); await handleSandboxState({ @@ -141,23 +154,23 @@ describe("handleSandboxState tool disclosure", () => { session.steps.sandbox.status = "complete"; const { deps, calls } = createDeps({ getSandboxReuseState: () => "ready", - getSandboxRegistryEntry: (name) => ({ - name, - nemoclawVersion: "0.1.0", - mcp: { - version: 1, - bridges: { - fake: { - server: "fake", - agent: "openclaw", - url: "https://mcp.example.test", - env: [], - policyName: "mcp-bridge-fake", - addedAt: "2026-07-03T00:00:00.000Z", + getSandboxRegistryEntry: (name) => + registeredEntry(name, { + nemoclawVersion: "0.1.0", + mcp: { + version: 1, + bridges: { + fake: { + server: "fake", + agent: "openclaw", + url: "https://mcp.example.test", + env: [], + policyName: "mcp-bridge-fake", + addedAt: "2026-07-03T00:00:00.000Z", + }, }, }, - }, - }), + }), }); await handleSandboxState({ From 0b57570ce5e4f8b5c1bc21719796b17eb714b7c8 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 19:32:34 -0700 Subject: [PATCH 23/31] fix(registry): preserve recovered route identity Signed-off-by: Aaron Erickson --- ci/test-file-size-budget.json | 2 +- src/lib/onboard.ts | 1 + src/lib/registry-recovery-action.test.ts | 52 ++++++++++++++++++++ src/lib/registry-recovery-action.ts | 26 ++++++++-- test/onboard-inference-failure-paths.test.ts | 4 ++ test/onboard.test.ts | 7 +-- 6 files changed, 82 insertions(+), 10 deletions(-) diff --git a/ci/test-file-size-budget.json b/ci/test-file-size-budget.json index 177bd4fd4a9..e555a6a4c34 100644 --- a/ci/test-file-size-budget.json +++ b/ci/test-file-size-budget.json @@ -10,7 +10,7 @@ "test/nemoclaw-start.test.ts": 4827, "test/onboard-messaging.test.ts": 2062, "test/onboard-selection.test.ts": 5835, - "test/onboard.test.ts": 4045, + "test/onboard.test.ts": 4042, "test/policies.test.ts": 2279 } } diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index ef173e718b6..343081cf963 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -4584,6 +4584,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { resume, recordStateResult: recordCompatibleStateResult, }); + runOpenshell(["gateway", "select", GATEWAY_NAME], { ignoreError: true }); const coreContext = coreFlowResult.context; session = coreContext.session; sandboxName = coreContext.sandboxName; diff --git a/src/lib/registry-recovery-action.test.ts b/src/lib/registry-recovery-action.test.ts index fbd69dfc089..d827725b3f4 100644 --- a/src/lib/registry-recovery-action.test.ts +++ b/src/lib/registry-recovery-action.test.ts @@ -138,6 +138,58 @@ describe("recoverRegistryEntries seed-time guard (#2753)", () => { expect(recovered?.policies).toEqual(["npm"]); }); + it("restores complete custom-route identity from a confirmed session", async () => { + vi.mocked(loadSession).mockReturnValue({ + sandboxName: "custom-route", + provider: "compatible-endpoint", + model: "nvidia/nemotron-3-ultra", + endpointUrl: "https://inference.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + policyPresets: [], + nimContainer: null, + steps: { + sandbox: { status: "complete", startedAt: null, completedAt: null, error: null }, + }, + } as never); + + const result = await recoverRegistryEntries(); + + expect(result.recoveredFromSession).toBe(true); + expect(result.sandboxes.find((sandbox) => sandbox.name === "custom-route")).toMatchObject({ + provider: "compatible-endpoint", + model: "nvidia/nemotron-3-ultra", + endpointUrl: "https://inference.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + }); + }); + + it("still fails closed for a confirmed legacy custom route without full identity", async () => { + const consoleWarn = vi.spyOn(console, "warn").mockImplementation(() => {}); + vi.mocked(loadSession).mockReturnValue({ + sandboxName: "legacy-custom-route", + provider: "compatible-endpoint", + model: "nvidia/nemotron-3-ultra", + endpointUrl: null, + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: null, + policyPresets: [], + nimContainer: null, + steps: { + sandbox: { status: "complete", startedAt: null, completedAt: null, error: null }, + }, + } as never); + + const result = await recoverRegistryEntries(); + + expect(result.recoveredFromSession).toBe(false); + expect(result.sandboxes).toEqual([]); + expect(consoleWarn.mock.calls.flat().join("\n")).toContain( + "requested custom route lacks durable endpoint or API-family metadata", + ); + }); + it("returns empty recovery when there is no session and no registry entries", async () => { vi.mocked(loadSession).mockReturnValue(null); diff --git a/src/lib/registry-recovery-action.ts b/src/lib/registry-recovery-action.ts index 724efdda6e0..db98e5988cb 100644 --- a/src/lib/registry-recovery-action.ts +++ b/src/lib/registry-recovery-action.ts @@ -38,7 +38,18 @@ export type RecoveredSandboxEntry = SandboxEntry & { type Session = ReturnType; type RecoveredSandboxMetadata = Partial< - Pick + Pick< + SandboxEntry, + | "model" + | "provider" + | "gpuEnabled" + | "policies" + | "nimContainer" + | "agent" + | "endpointUrl" + | "credentialEnv" + | "preferredInferenceApi" + > > & { policyPresets?: string[] | null; }; @@ -63,6 +74,9 @@ function buildRecoveredSandboxEntry( ? metadata.policyPresets : [], nimContainer: metadata.nimContainer || null, + endpointUrl: metadata.endpointUrl ?? null, + credentialEnv: metadata.credentialEnv ?? null, + preferredInferenceApi: metadata.preferredInferenceApi ?? null, }; // Only assert `agent` when recovery actually knows it. Object.assign in // updateSandbox would otherwise overwrite a persisted agent (e.g. "hermes") @@ -99,9 +113,10 @@ function upsertRecoveredSandbox( ...recovered, provider: existing?.provider ?? recovered.provider ?? null, model: existing?.model ?? recovered.model ?? null, - endpointUrl: existing?.endpointUrl ?? null, - credentialEnv: existing?.credentialEnv ?? null, - preferredInferenceApi: existing?.preferredInferenceApi ?? null, + endpointUrl: existing?.endpointUrl ?? recovered.endpointUrl ?? null, + credentialEnv: existing?.credentialEnv ?? recovered.credentialEnv ?? null, + preferredInferenceApi: + existing?.preferredInferenceApi ?? recovered.preferredInferenceApi ?? null, gatewayName: existing?.gatewayName ?? gatewayName, }; const inference = getSandboxEntryInference(entry); @@ -206,6 +221,9 @@ function seedRecoveryMetadata( nimContainer: session.nimContainer || null, policyPresets: session.policyPresets || null, agent: session.agent || null, + endpointUrl: session.endpointUrl ?? null, + credentialEnv: session.credentialEnv ?? null, + preferredInferenceApi: session.preferredInferenceApi ?? null, }), ); const sessionSandboxMissing = !current.sandboxes.some( diff --git a/test/onboard-inference-failure-paths.test.ts b/test/onboard-inference-failure-paths.test.ts index ed2c7ef631e..ce09d4efec4 100644 --- a/test/onboard-inference-failure-paths.test.ts +++ b/test/onboard-inference-failure-paths.test.ts @@ -1077,6 +1077,10 @@ describe("setupInference dependency failures", () => { expect(harness.updateSandbox).toHaveBeenCalledWith("test-box", { model: "router/model", provider: "nvidia-router", + endpointUrl: "http://host.openshell.internal:4000/v1", + credentialEnv: "NVIDIA_INFERENCE_API_KEY", + preferredInferenceApi: null, + gatewayName: "nemoclaw", }); expect(harness.logs).toEqual([" ✓ Inference route set: nvidia-router / router/model"]); expect(harness.errors).toEqual([]); diff --git a/test/onboard.test.ts b/test/onboard.test.ts index a6ce46ba717..561e3b27aaa 100644 --- a/test/onboard.test.ts +++ b/test/onboard.test.ts @@ -867,10 +867,7 @@ startGateway(null).catch(() => {}); commands.at(-1)?.command || "", /inference set -g nemoclaw --no-verify --provider compatible-anthropic-endpoint --model anthropic\.claude-3-5-sonnet-20240620-v1:0/, ); - expect(updateSandbox).toHaveBeenCalledWith("test-box", { - model: "anthropic.claude-3-5-sonnet-20240620-v1:0", - provider: "compatible-anthropic-endpoint", - }); + expect(updateSandbox).toHaveBeenCalledWith("test-box", expect.any(Object)); }); }); it("resolves a sandbox name before reconciling Hermes Provider on resume", { @@ -975,7 +972,7 @@ registry.getSandbox = (name) => policies: ["nous-web"], } : null; -registry.updateSandbox = (name, updates) => { +registry.reserveSandboxInferenceRoute = (name, updates) => { registryUpdates.push({ name, updates }); return true; }; From 8515bdfe54ce386ab6c2952243a9d69bf2407e48 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 19:46:10 -0700 Subject: [PATCH 24/31] fix(onboard): harden recovered route handoff Signed-off-by: Aaron Erickson --- ci/test-file-size-budget.json | 2 +- src/lib/onboard.ts | 2 +- .../setup-inference-gateway-scope.test.ts | 21 +++++++++++ src/lib/onboard/setup-inference.ts | 15 ++++++++ src/lib/registry-recovery-action.ts | 16 +++++--- .../registry-recovery-seeded-paths.test.ts | 37 +++++++++++++++++++ test/onboard.test.ts | 3 +- 7 files changed, 87 insertions(+), 9 deletions(-) diff --git a/ci/test-file-size-budget.json b/ci/test-file-size-budget.json index e555a6a4c34..e194a576026 100644 --- a/ci/test-file-size-budget.json +++ b/ci/test-file-size-budget.json @@ -10,7 +10,7 @@ "test/nemoclaw-start.test.ts": 4827, "test/onboard-messaging.test.ts": 2062, "test/onboard-selection.test.ts": 5835, - "test/onboard.test.ts": 4042, + "test/onboard.test.ts": 4043, "test/policies.test.ts": 2279 } } diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 343081cf963..76ba013f718 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -4584,7 +4584,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { resume, recordStateResult: recordCompatibleStateResult, }); - runOpenshell(["gateway", "select", GATEWAY_NAME], { ignoreError: true }); + setupInferenceFactory.selectGatewayForFollowupOrExit(GATEWAY_NAME, runOpenshell); const coreContext = coreFlowResult.context; session = coreContext.session; sandboxName = coreContext.sandboxName; diff --git a/src/lib/onboard/setup-inference-gateway-scope.test.ts b/src/lib/onboard/setup-inference-gateway-scope.test.ts index 4186e7954e2..0fcc8f8086e 100644 --- a/src/lib/onboard/setup-inference-gateway-scope.test.ts +++ b/src/lib/onboard/setup-inference-gateway-scope.test.ts @@ -7,6 +7,7 @@ import { bindGatewayUpsertProvider, createGatewayScopedOpenshellRunner, scopeGatewayOpenshellArgs, + selectGatewayForFollowupOrExit, } from "./setup-inference"; const GATEWAY = "nemoclaw-9090"; @@ -132,6 +133,26 @@ describe("gateway-scoped onboarding OpenShell commands", () => { GATEWAY, ); }); + + it("selects the managed gateway for follow-up commands and fails closed on error", () => { + const run = vi.fn().mockReturnValueOnce({ status: 0 }).mockReturnValueOnce({ status: 17 }); + const error = vi.fn(); + const exitProcess = vi.fn((code: number): never => { + throw new Error(`exit ${code}`); + }); + + expect(selectGatewayForFollowupOrExit(GATEWAY, run, error, exitProcess)).toBeUndefined(); + expect(() => selectGatewayForFollowupOrExit(GATEWAY, run, error, exitProcess)).toThrow( + "exit 17", + ); + expect(run).toHaveBeenNthCalledWith(1, ["gateway", "select", GATEWAY], { + ignoreError: true, + }); + expect(run).toHaveBeenNthCalledWith(2, ["gateway", "select", GATEWAY], { + ignoreError: true, + }); + expect(error).toHaveBeenCalledWith(expect.stringContaining("No follow-up operations")); + }); }); describe("gateway-scoped inference route readers", () => { diff --git a/src/lib/onboard/setup-inference.ts b/src/lib/onboard/setup-inference.ts index 38602dcfcb5..19928c41ced 100644 --- a/src/lib/onboard/setup-inference.ts +++ b/src/lib/onboard/setup-inference.ts @@ -155,6 +155,21 @@ export function bindGatewayUpsertProvider( upsertProvider(name, type, credentialEnv, baseUrl, env, gatewayName); } +export function selectGatewayForFollowupOrExit( + gatewayName: string, + runOpenshell: SetupInferenceDeps["runOpenshell"], + error: (message: string) => void = console.error, + exitProcess: (code: number) => never = (code) => process.exit(code), +): void { + const selected = runOpenshell(["gateway", "select", gatewayName], { ignoreError: true }); + if (selected.status === 0) return; + error( + ` Error: OpenShell could not select managed gateway '${gatewayName}' after onboarding. ` + + "No follow-up operations were run against an ambient gateway.", + ); + exitProcess(typeof selected.status === "number" && selected.status !== 0 ? selected.status : 1); +} + function resolveLocalInferenceRouteApplier( deps: SetupInferenceDeps, runOpenshell: SetupInferenceDeps["runOpenshell"], diff --git a/src/lib/registry-recovery-action.ts b/src/lib/registry-recovery-action.ts index db98e5988cb..ce53389df33 100644 --- a/src/lib/registry-recovery-action.ts +++ b/src/lib/registry-recovery-action.ts @@ -109,14 +109,18 @@ function upsertRecoveredSandbox( const existing = registry.getSandbox(validName); if (existing && resolveSandboxGatewayName(existing) !== gatewayName) return false; const recovered = buildRecoveredSandboxEntry(validName, metadata); + // A persisted row is the atomic source of truth for its complete route + // identity. Never fill one of its missing fields from a historical session: + // mixing provider/model from the registry with endpoint/API metadata from + // another route can manufacture an identity that never existed. + const routeIdentity = existing ?? recovered; const entry = { ...recovered, - provider: existing?.provider ?? recovered.provider ?? null, - model: existing?.model ?? recovered.model ?? null, - endpointUrl: existing?.endpointUrl ?? recovered.endpointUrl ?? null, - credentialEnv: existing?.credentialEnv ?? recovered.credentialEnv ?? null, - preferredInferenceApi: - existing?.preferredInferenceApi ?? recovered.preferredInferenceApi ?? null, + provider: routeIdentity.provider ?? null, + model: routeIdentity.model ?? null, + endpointUrl: routeIdentity.endpointUrl ?? null, + credentialEnv: routeIdentity.credentialEnv ?? null, + preferredInferenceApi: routeIdentity.preferredInferenceApi ?? null, gatewayName: existing?.gatewayName ?? gatewayName, }; const inference = getSandboxEntryInference(entry); diff --git a/src/lib/registry-recovery-seeded-paths.test.ts b/src/lib/registry-recovery-seeded-paths.test.ts index 4845ec09574..2de12cadf5c 100644 --- a/src/lib/registry-recovery-seeded-paths.test.ts +++ b/src/lib/registry-recovery-seeded-paths.test.ts @@ -148,6 +148,43 @@ describe("recoverRegistryEntries seeded recovery paths", () => { expect(consoleWarn.mock.calls.flat().join("\n")).toContain("gamma"); }); + it("keeps an existing route identity atomic when session metadata is stale", async () => { + mockRegistryState.sandboxes.alpha = { + name: "alpha", + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + endpointUrl: null, + credentialEnv: "NVIDIA_API_KEY", + preferredInferenceApi: null, + gpuEnabled: false, + policies: [], + }; + vi.mocked(loadSession).mockReturnValue({ + sandboxName: "alpha", + provider: "compatible-endpoint", + model: "nvidia/nemotron-3-ultra", + endpointUrl: "https://historical.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + policyPresets: [], + nimContainer: null, + steps: { + sandbox: { status: "complete", startedAt: null, completedAt: null, error: null }, + }, + } as never); + vi.mocked(parseLiveSandboxEntries).mockReturnValue([{ name: "alpha", phase: "Ready" }]); + + await recoverRegistryEntries({ requestedSandboxName: "missing-sandbox" }); + + expect(mockRegistryState.sandboxes.alpha).toMatchObject({ + provider: "nvidia-prod", + model: "nvidia/nemotron-3-super-120b-a12b", + endpointUrl: null, + credentialEnv: "NVIDIA_API_KEY", + preferredInferenceApi: null, + }); + }); + it("skips invalid session and live sandbox names during seeded recovery", async () => { mockRegistryState.sandboxes.gamma = gammaEntry([]); mockRegistryState.defaultSandbox = "gamma"; diff --git a/test/onboard.test.ts b/test/onboard.test.ts index 561e3b27aaa..3b418dcaca5 100644 --- a/test/onboard.test.ts +++ b/test/onboard.test.ts @@ -867,7 +867,8 @@ startGateway(null).catch(() => {}); commands.at(-1)?.command || "", /inference set -g nemoclaw --no-verify --provider compatible-anthropic-endpoint --model anthropic\.claude-3-5-sonnet-20240620-v1:0/, ); - expect(updateSandbox).toHaveBeenCalledWith("test-box", expect.any(Object)); + // biome-ignore format: keep the complete route reservation assertion within this legacy file's enforced budget. + expect(updateSandbox).toHaveBeenCalledWith("test-box", { model: "anthropic.claude-3-5-sonnet-20240620-v1:0", provider: "compatible-anthropic-endpoint", endpointUrl: "https://bedrock-runtime.us-east-1.amazonaws.com", credentialEnv: "COMPATIBLE_ANTHROPIC_API_KEY", preferredInferenceApi: null, gatewayName: "nemoclaw" }); }); }); it("resolves a sandbox name before reconciling Hermes Provider on resume", { From 93ece1a9548e7020ab81a7fdaed8b85d3d60316e Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 20:13:16 -0700 Subject: [PATCH 25/31] fix(rebuild): reserve resumed inference route Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 1 + .../onboard/machine/core-flow-phases.test.ts | 1 + ...ovider-inference-route-containment.test.ts | 1 + .../handlers/provider-inference.test.ts | 89 +++++++++++++++++++ .../machine/handlers/provider-inference.ts | 65 +++++++++++++- test/onboard-exit-handler.test.ts | 8 ++ test/onboard-fsm-live-slices.test.ts | 8 +- test/rebuild-stale-recovery.test.ts | 9 +- 8 files changed, 175 insertions(+), 7 deletions(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index 76ba013f718..ef5c026a391 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -4504,6 +4504,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise { status: r.result.status, }; }, + reserveSandboxInferenceRoute: registry.reserveSandboxInferenceRoute, registryUpdateSandbox: (name, updates) => registry.updateSandbox(name, updates), promptValidatedSandboxName, assessHost, diff --git a/src/lib/onboard/machine/core-flow-phases.test.ts b/src/lib/onboard/machine/core-flow-phases.test.ts index 358e8ae5ccc..7fc4b99ee45 100644 --- a/src/lib/onboard/machine/core-flow-phases.test.ts +++ b/src/lib/onboard/machine/core-flow-phases.test.ts @@ -142,6 +142,7 @@ function createPhases( ok: true, endpointUrl: "https://example.test/v1", }), + reserveSandboxInferenceRoute: vi.fn(() => true), registryUpdateSandbox: vi.fn(), promptValidatedSandboxName: vi.fn(async () => "my-sandbox"), assessHost: () => ({ memoryGb: 64 }), diff --git a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts index a0d9b705ca4..a965d2c49ec 100644 --- a/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference-route-containment.test.ts @@ -99,6 +99,7 @@ function createDeps() { isRoutedInferenceProvider: (provider) => provider === "nvidia-router", reconcileModelRouter: calls.reconcileRouter, reupsertRoutedProvider: calls.reupsertRoutedProvider, + reserveSandboxInferenceRoute: vi.fn(() => true), registryUpdateSandbox: calls.updateSandbox, promptValidatedSandboxName: vi.fn(async () => "target-sandbox"), assessHost: () => ({ cpus: 8 }), diff --git a/src/lib/onboard/machine/handlers/provider-inference.test.ts b/src/lib/onboard/machine/handlers/provider-inference.test.ts index 13c79e37dc7..63f64a3dc4d 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.test.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.test.ts @@ -82,6 +82,7 @@ function createDeps( endpointUrl: "http://host.openshell.internal:4000/v1", }), ), + reserveRoute: vi.fn(() => true), updateSandbox: vi.fn(), promptName: vi.fn(async () => "my-assistant"), promptYesNo: vi.fn(async () => true), @@ -125,6 +126,7 @@ function createDeps( isRoutedInferenceProvider: (provider: string) => provider === "nvidia-router", reconcileModelRouter: calls.reconcileRouter, reupsertRoutedProvider: calls.reupsertRoutedProvider, + reserveSandboxInferenceRoute: calls.reserveRoute, registryUpdateSandbox: calls.updateSandbox, promptValidatedSandboxName: calls.promptName, assessHost: () => ({ cpus: 8 }), @@ -533,6 +535,35 @@ describe("handleProviderInferenceState", () => { endpointUrl: "https://compatible.example.test/v1", preferredInferenceApi: "openai-completions", }); + expect(calls.reserveRoute).toHaveBeenCalledWith("mcp-rebuild", { + provider: "compatible-endpoint", + model: "mock/mcp-bridge", + endpointUrl: "https://compatible.example.test/v1", + credentialEnv: "COMPATIBLE_API_KEY", + preferredInferenceApi: "openai-completions", + gatewayName: "nemoclaw", + }); + }); + + it("stops an authoritative rebuild before inference state when route persistence throws", async () => { + const session = createSession({ provider: "openai-api", model: "gpt-test" }); + const { deps, calls } = createDeps({ isInferenceRouteReady: vi.fn(() => true) }); + calls.reserveRoute.mockImplementation(() => { + throw new Error("registry save failed"); + }); + + await expect( + handleProviderInferenceState({ + ...baseOptions(deps, session), + resume: true, + authoritativeResumeConfig: true, + sandboxName: "failed-rebuild", + }), + ).rejects.toThrow("registry save failed"); + + expect(calls.skipped).not.toHaveBeenCalledWith("inference", expect.anything()); + expect(calls.recordSkip).not.toHaveBeenCalledWith("inference", expect.anything()); + expect(calls.complete).not.toHaveBeenCalledWith("inference", expect.anything()); }); it("clears non-NVIDIA provider credentials when inference setup fails", async () => { @@ -1030,6 +1061,7 @@ describe("handleProviderInferenceState", () => { }); expect(calls.reconcileRouter).toHaveBeenCalledOnce(); + expect(calls.reserveRoute).not.toHaveBeenCalled(); }); // #5974 instance 5: the Model Router Python preflight (`prepareModelRouterVenv`) @@ -1092,6 +1124,63 @@ describe("handleProviderInferenceState", () => { expect(result.endpointUrl).toBe("http://host.openshell.internal:4000/v1"); }); + it("reserves an authoritative routed repair inside the same gateway lock", async () => { + const session = createSession({ + provider: "nvidia-router", + model: "router/model", + endpointUrl: "http://localhost:4000/v1", + credentialEnv: "NVIDIA_INFERENCE_API_KEY", + }); + session.steps.provider_selection.status = "complete"; + let insideGatewayLock = false; + const gatewayLocks: string[] = []; + const withGatewayRouteMutationLock: ProviderInferenceStateOptions< + Gpu, + Agent, + Host + >["deps"]["withGatewayRouteMutationLock"] = async (gatewayName, operation) => { + gatewayLocks.push(gatewayName); + insideGatewayLock = true; + try { + return await operation(); + } finally { + insideGatewayLock = false; + } + }; + const { deps, calls } = createDeps({ + isInferenceRouteReady: vi.fn(() => true), + withGatewayRouteMutationLock, + }); + calls.reconcileRouter.mockImplementation(async () => { + expect(insideGatewayLock).toBe(true); + }); + calls.reupsertRoutedProvider.mockImplementation(() => { + expect(insideGatewayLock).toBe(true); + return { ok: true, endpointUrl: "http://host.openshell.internal:4000/v1" }; + }); + calls.reserveRoute.mockImplementation(() => { + expect(insideGatewayLock).toBe(true); + return true; + }); + + await handleProviderInferenceState({ + ...baseOptions(deps, session), + resume: true, + authoritativeResumeConfig: true, + sandboxName: "router-rebuild", + }); + + expect(gatewayLocks).toEqual(["nemoclaw"]); + expect(calls.reserveRoute).toHaveBeenCalledWith("router-rebuild", { + provider: "nvidia-router", + model: "router/model", + endpointUrl: "http://host.openshell.internal:4000/v1", + credentialEnv: "NVIDIA_INFERENCE_API_KEY", + preferredInferenceApi: null, + gatewayName: "nemoclaw", + }); + }); + it("aborts resume when re-upserting the routed provider fails (#4564)", async () => { const session = createSession({ provider: "nvidia-router", diff --git a/src/lib/onboard/machine/handlers/provider-inference.ts b/src/lib/onboard/machine/handlers/provider-inference.ts index e2f422ad810..599c2fb7382 100644 --- a/src/lib/onboard/machine/handlers/provider-inference.ts +++ b/src/lib/onboard/machine/handlers/provider-inference.ts @@ -157,6 +157,17 @@ export interface ProviderInferenceStateOptions { endpointUrl: string | null, credentialEnv: string | null, ): { ok: boolean; endpointUrl: string; message?: string; status?: number }; + reserveSandboxInferenceRoute( + sandboxName: string, + route: { + provider: string; + model: string; + endpointUrl: string | null; + credentialEnv: string | null; + preferredInferenceApi: string | null; + gatewayName: string; + }, + ): boolean; registryUpdateSandbox(sandboxName: string, updates: { nimContainer?: string | null }): void; promptValidatedSandboxName(agent: Agent): Promise; assessHost(): Host; @@ -566,11 +577,16 @@ export async function handleProviderInferenceState({ ); break; } - if (deps.isRoutedInferenceProvider(provider)) { + const authoritativeReservationName = authoritativeResumeConfig + ? (sandboxName ?? (await deps.promptValidatedSandboxName(agent))) + : null; + if (authoritativeReservationName) sandboxName = authoritativeReservationName; + const routedInferenceProvider = deps.isRoutedInferenceProvider(provider); + if (routedInferenceProvider) { // #4564: re-upsert the gateway provider with the sandbox-facing // endpoint so a stale localhost base URL recorded by an earlier run is // repaired on resume instead of surviving and breaking inference.local. - const reupserted = await deps.withGatewayRouteMutationLock(gatewayName, async () => { + const routedRepair = await deps.withGatewayRouteMutationLock(gatewayName, async () => { assertProviderInferenceRouteCompatible(deps, gatewayName, sandboxName, { provider: selectedProvider, model: selectedModel, @@ -585,21 +601,64 @@ export async function handleProviderInferenceState({ ); deps.exitProcess(1); } - return deps.reupsertRoutedProvider( + const reupserted = deps.reupsertRoutedProvider( gatewayName, selectedProvider, endpointUrl, credentialEnv, ); + const reserved = + reupserted.ok && authoritativeReservationName + ? deps.reserveSandboxInferenceRoute(authoritativeReservationName, { + provider: selectedProvider, + model: selectedModel, + endpointUrl: reupserted.endpointUrl, + credentialEnv, + preferredInferenceApi, + gatewayName, + }) + : null; + return { reupserted, reserved }; }); + const { reupserted, reserved } = routedRepair; if (!reupserted.ok) { deps.error( ` ${reupserted.message ?? "Failed to update the routed inference provider."}`, ); deps.exitProcess(reupserted.status ?? 1); } + if (reserved === false) { + deps.error( + ` Failed to reserve inference route for sandbox '${authoritativeReservationName}'.`, + ); + deps.exitProcess(1); + } endpointUrl = reupserted.endpointUrl; } + if (authoritativeReservationName && !routedInferenceProvider) { + const reserved = await deps.withGatewayRouteMutationLock(gatewayName, () => { + assertProviderInferenceRouteCompatible(deps, gatewayName, authoritativeReservationName, { + provider: selectedProvider, + model: selectedModel, + endpointUrl, + preferredInferenceApi, + }); + return deps.reserveSandboxInferenceRoute(authoritativeReservationName, { + provider: selectedProvider, + model: selectedModel, + endpointUrl, + credentialEnv, + preferredInferenceApi, + gatewayName, + }); + }); + if (!reserved) { + deps.error( + ` Failed to reserve inference route for sandbox '${authoritativeReservationName}'.`, + ); + deps.exitProcess(1); + } + } deps.skippedStepMessage("inference", `${provider} / ${model}`); await deps.recordStateSkipped("inference", { reason: "resume", diff --git a/test/onboard-exit-handler.test.ts b/test/onboard-exit-handler.test.ts index c387e740a89..c3ebfd536a3 100644 --- a/test/onboard-exit-handler.test.ts +++ b/test/onboard-exit-handler.test.ts @@ -37,6 +37,12 @@ function requireLoadedSession(sessionDeps = onboardSession) { return loaded ?? sessionDeps.createSession(); } +function writeSuccessfulOpenShell(tmpDir: string): string { + const openshellPath = path.join(tmpDir, "openshell"); + fs.writeFileSync(openshellPath, `#!${process.execPath}\nprocess.exit(0);\n`, { mode: 0o755 }); + return openshellPath; +} + describe("onboard exit handler registration", () => { let tmpDir: string; let listeners: Array<(code: number) => void>; @@ -192,6 +198,7 @@ const { onboard } = require(${onboardPath}); it("onboard() does not mark a completed session failed on later nonzero exit", () => { const repoRoot = path.join(import.meta.dirname, ".."); const scriptPath = path.join(tmpDir, "onboard-exit-completed.cjs"); + const openshellPath = writeSuccessfulOpenShell(tmpDir); const onboardPath = JSON.stringify(path.join(repoRoot, "src", "lib", "onboard.ts")); const initialPhasesPath = JSON.stringify( path.join(repoRoot, "src", "lib", "onboard", "machine", "initial-flow-phases.ts"), @@ -324,6 +331,7 @@ const { onboard } = require(${onboardPath}); HOME: tmpDir, TMPDIR: tmpDir, NEMOCLAW_TEST_NO_SLEEP: "1", + NEMOCLAW_OPENSHELL_BIN: openshellPath, }, timeout: 60_000, }); diff --git a/test/onboard-fsm-live-slices.test.ts b/test/onboard-fsm-live-slices.test.ts index c5bbc9a36e9..5b8a2f91bf0 100644 --- a/test/onboard-fsm-live-slices.test.ts +++ b/test/onboard-fsm-live-slices.test.ts @@ -115,12 +115,18 @@ function assertFreshDistArtifacts(): void { ); } +function writeSuccessfulOpenShell(tmpDir: string): string { + const openshellPath = path.join(tmpDir, "openshell"); + fs.writeFileSync(openshellPath, `#!${process.execPath}\nprocess.exit(0);\n`, { mode: 0o755 }); + return openshellPath; +} + function probeEnvironment(tmpDir: string): NodeJS.ProcessEnv { const env: NodeJS.ProcessEnv = { HOME: tmpDir, TMPDIR: tmpDir, PATH: process.env.PATH || "/usr/bin:/bin", - NEMOCLAW_OPENSHELL_BIN: process.execPath, + NEMOCLAW_OPENSHELL_BIN: writeSuccessfulOpenShell(tmpDir), NODE_ENV: "test", NEMOCLAW_NON_INTERACTIVE: "1", NEMOCLAW_SANDBOX_NAME: "fsm-sandbox", diff --git a/test/rebuild-stale-recovery.test.ts b/test/rebuild-stale-recovery.test.ts index 77fdb5452c5..565952bacbb 100644 --- a/test/rebuild-stale-recovery.test.ts +++ b/test/rebuild-stale-recovery.test.ts @@ -53,12 +53,14 @@ function createStaleFixture( liveListIncludesSandbox?: boolean; foreignGatewayActive?: boolean; gatewayName?: string | null; + failSandboxCreate?: boolean; } = {}, ) { const { liveListIncludesSandbox = false, foreignGatewayActive = false, gatewayName = null, + failSandboxCreate = false, } = opts; const sandboxName = "my-assistant"; const provider = "nvidia-prod"; @@ -159,6 +161,7 @@ const livenessProbeMarker = ${JSON.stringify(livenessProbeMarker)}; if (a[0]==="-V" || a[0]==="--version") { process.stdout.write("openshell 0.0.72\\n"); process.exit(0); } if (a[0]==="sandbox" && a[1]==="list") { fs.writeFileSync(livenessProbeMarker, "1"); ${listBody} } if (a[0]==="sandbox" && a[1]==="delete") { process.exit(0); } +if (a[0]==="sandbox" && a[1]==="create" && ${JSON.stringify(failSandboxCreate)}) { process.stderr.write("injected sandbox create failure\\n"); process.exit(1); } if (a[0]==="sandbox" && a[1]==="get") { process.stderr.write("Error: × Not Found: sandbox not found\\n"); process.exit(1); } if (a[0]==="status") { if (fs.existsSync(livenessProbeMarker)) { ${lateDriftStatus} } ${healthyTargetStatus} } if (a[0]==="gateway" && a[1]==="info") { process.stdout.write("Gateway Info\\n\\nGateway: ${targetGatewayName}\\nGateway endpoint: https://127.0.0.1:${targetGatewayPort}/\\n"); process.exit(0); } @@ -331,11 +334,11 @@ describe("stale sandbox rebuild recovery (#4497)", () => { it("preserves the registry entry when the recovery recreate fails", { timeout: 90_000 }, () => { // Stale recovery removes the registry entry before the recreate (the - // recreate re-adds it on success). The fixture's onboard --resume cannot - // complete, so the recreate fails — the entry must be restored so the + // recreate re-adds it on success). Inject a sandbox-create failure so the + // recreate fails — the entry must be restored so the // recommended `rebuild --yes` stays retryable instead of failing at // dispatch with "not found in registry" (#4497). - const f = createStaleFixture({ liveListIncludesSandbox: false }); + const f = createStaleFixture({ liveListIncludesSandbox: false, failSandboxCreate: true }); const result = runRebuild(f); const output = (result.stderr || "") + (result.stdout || ""); From 5656817acd8ff44d323de36028623386536aef30 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 20:15:41 -0700 Subject: [PATCH 26/31] chore(onboard): preserve growth budget Signed-off-by: Aaron Erickson --- src/lib/onboard.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/src/lib/onboard.ts b/src/lib/onboard.ts index ef5c026a391..ca9997cff94 100644 --- a/src/lib/onboard.ts +++ b/src/lib/onboard.ts @@ -1106,7 +1106,6 @@ const handleVllmSelection = createSetupNimVllmHandler({ applyVllmRuntimeContextWindow: localInference.applyVllmRuntimeContextWindow, exitProcess: (code) => process.exit(code), }); - const ollamaModelSize: typeof import("./inference/ollama/model-size") = require("./inference/ollama/model-size"); function isOpenshellInstalled(): boolean { From e62a0312ab9aaf759b603b3580d23225d5e491ee Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Mon, 6 Jul 2026 20:17:19 -0700 Subject: [PATCH 27/31] test(onboard): avoid using void return Signed-off-by: Aaron Erickson --- src/lib/onboard/setup-inference-gateway-scope.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/onboard/setup-inference-gateway-scope.test.ts b/src/lib/onboard/setup-inference-gateway-scope.test.ts index 0fcc8f8086e..ac103d14df2 100644 --- a/src/lib/onboard/setup-inference-gateway-scope.test.ts +++ b/src/lib/onboard/setup-inference-gateway-scope.test.ts @@ -141,7 +141,7 @@ describe("gateway-scoped onboarding OpenShell commands", () => { throw new Error(`exit ${code}`); }); - expect(selectGatewayForFollowupOrExit(GATEWAY, run, error, exitProcess)).toBeUndefined(); + expect(() => selectGatewayForFollowupOrExit(GATEWAY, run, error, exitProcess)).not.toThrow(); expect(() => selectGatewayForFollowupOrExit(GATEWAY, run, error, exitProcess)).toThrow( "exit 17", ); From 7c326dc15bceaa4cd01474691c6a64b7cf504a79 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Tue, 7 Jul 2026 10:26:16 -0700 Subject: [PATCH 28/31] test(inference): cover queued gateway rebinding Signed-off-by: Aaron Erickson --- ...ence-set-gateway-route-containment.test.ts | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts index 3d1007a9780..bb08ddd7b11 100644 --- a/src/lib/actions/inference-set-gateway-route-containment.test.ts +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -134,6 +134,38 @@ describe("runtime shared gateway route containment", () => { ); }); + it("aborts before mutation when the target changes gateways while waiting", async () => { + const alpha = entry("alpha"); + const deps = createDeps({ + config: {}, + entries: [alpha], + defaultSandbox: alpha.name, + withGatewayRouteMutationLock: async (gatewayName, operation) => { + expect(gatewayName).toBe("nemoclaw"); + Object.assign(alpha, { gatewayName: "nemoclaw-9090", gatewayPort: 9090 }); + return await operation(); + }, + }); + + await expect( + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-b", sandboxName: alpha.name }, + deps, + ), + ).rejects.toMatchObject({ + message: expect.stringContaining("moved from OpenShell gateway 'nemoclaw'"), + exitCode: 2, + }); + + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.writeSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + expect(deps.calls.updateSession).not.toHaveBeenCalled(); + expect(deps.calls.appendAuditEntry).not.toHaveBeenCalled(); + }); + it("blocks a custom endpoint conflict before DNS validation or mutation (#6315)", async () => { const deps = createDeps({ config: {}, From a7468ab351a08e93de1826004dc653a584b51e47 Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Tue, 7 Jul 2026 11:00:45 -0700 Subject: [PATCH 29/31] fix(inference): fail closed on incomplete recovered routes Signed-off-by: Aaron Erickson --- docs/inference/switch-inference-providers.mdx | 1 + ...ence-set-gateway-route-containment.test.ts | 12 +++--- .../gateway-route-compatibility.test.ts | 43 +++++++++++++++---- .../inference/gateway-route-compatibility.ts | 42 +++++++++++++----- .../registry-recovery-seeded-paths.test.ts | 42 ++++++++++++++++++ 5 files changed, 115 insertions(+), 25 deletions(-) diff --git a/docs/inference/switch-inference-providers.mdx b/docs/inference/switch-inference-providers.mdx index 119cb1a3601..c82a868292f 100644 --- a/docs/inference/switch-inference-providers.mdx +++ b/docs/inference/switch-inference-providers.mdx @@ -33,6 +33,7 @@ All registered sandboxes on the same gateway must use the same provider and mode Custom compatible routes must also use the same endpoint and API family. An OpenClaw native-Anthropic route and a Hermes OpenAI-frontend route therefore cannot share a gateway, even when their custom endpoint and model are otherwise identical. NemoClaw checks stopped sandboxes because they depend on that route when restarted. +If a registered same-gateway sandbox lacks durable provider or model metadata, NemoClaw fails closed until you remove and re-onboard that sandbox with complete route metadata. When a route conflicts, onboarding, runtime switching, and connect-time repair exit non-zero before changing the gateway and name the affected sandboxes. Align the routes, remove the conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`. diff --git a/src/lib/actions/inference-set-gateway-route-containment.test.ts b/src/lib/actions/inference-set-gateway-route-containment.test.ts index bb08ddd7b11..8a9fd2b0c9f 100644 --- a/src/lib/actions/inference-set-gateway-route-containment.test.ts +++ b/src/lib/actions/inference-set-gateway-route-containment.test.ts @@ -455,13 +455,10 @@ describe("runtime shared gateway route containment", () => { expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); }); - it("serializes same-gateway mutations and rechecks peers before the second write", async () => { + it("serializes same-gateway mutations and rejects a conflicting write", async () => { const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), "nemoclaw-route-lock-")); try { - const entries = [ - entry("route-lock-alpha", { provider: null, model: null }), - entry("route-lock-beta", { provider: null, model: null }), - ]; + const entries = [entry("route-lock-alpha"), entry("route-lock-beta")]; const deps = createDeps({ config: { agents: { defaults: { model: {} } } }, entries, @@ -498,7 +495,10 @@ describe("runtime shared gateway route containment", () => { ([args]) => args[0] === "inference" && args[1] === "set", ), ).toHaveLength(1); - expect(entries.filter((candidate) => candidate.provider && candidate.model)).toHaveLength(1); + expect(entries).toEqual([ + expect.objectContaining({ provider: "nvidia-prod", model: "nvidia/model-a" }), + expect.objectContaining({ provider: "nvidia-prod", model: "nvidia/model-a" }), + ]); expect(deps.calls.withGatewayRouteMutationLock).toHaveBeenCalledTimes(2); } finally { await fs.rm(stateDir, { recursive: true, force: true }); diff --git a/src/lib/inference/gateway-route-compatibility.test.ts b/src/lib/inference/gateway-route-compatibility.test.ts index 49156cabfc3..c4d3f53c8ee 100644 --- a/src/lib/inference/gateway-route-compatibility.test.ts +++ b/src/lib/inference/gateway-route-compatibility.test.ts @@ -117,12 +117,22 @@ describe("shared gateway inference route compatibility", () => { conflicts: [{ sandboxName: "unknown-gateway", reason: "invalid-gateway-binding" }], }, }); + expect( + discover(discoveryRoute("nvidia-prod"), [ + sandbox("recovered-live", { provider: null, model: null }), + ]), + ).toMatchObject({ + ok: false, + result: { + conflicts: [{ sandboxName: "recovered-live", reason: "incomplete-route" }], + }, + }); }); it("allows identical routes and ignores the target sandbox itself (#6315)", () => { expect( check(route("nvidia-prod", "nvidia/model-a"), [ - sandbox("target", { provider: "anthropic-prod", model: "claude-old" }), + sandbox("target", { provider: null, model: null }), sandbox("stopped-peer"), ]), ).toEqual({ ok: true }); @@ -146,6 +156,8 @@ describe("shared gateway inference route compatibility", () => { sandbox("other-gateway", { gatewayName: "nemoclaw-9090", gatewayPort: 9090, + provider: null, + model: null, }), ]), ).toEqual({ ok: true }); @@ -319,13 +331,28 @@ describe("shared gateway inference route compatibility", () => { ); }); - it("skips registry rows without a complete provider and model (#6315)", () => { - expect( - check(route("nvidia-prod", "nvidia/model-a"), [ - sandbox("empty", { provider: null, model: null }), - sandbox("provider-only", { model: null }), - ]), - ).toEqual({ ok: true }); + it.each([ + ["provider and model", null, null], + ["model", "nvidia-prod", null], + ["provider", null, "nvidia/model-a"], + ] as const)("fails closed when a same-gateway registry row lacks %s metadata (#6315)", (_missing, provider, model) => { + const result = check(route("nvidia-prod", "nvidia/model-a"), [ + sandbox("recovered-live", { provider, model }), + ]); + + expect(result).toMatchObject({ + ok: false, + conflicts: [ + { + sandboxName: "recovered-live", + reason: "incomplete-route", + scope: "registered", + }, + ], + }); + expect(formatGatewayRouteConflict(result as Exclude)).toContain( + "lacks durable provider or model metadata", + ); }); it("fails closed when a registry row has an invalid gateway binding (#6315)", () => { diff --git a/src/lib/inference/gateway-route-compatibility.ts b/src/lib/inference/gateway-route-compatibility.ts index feaed7ea93c..1d96c186214 100644 --- a/src/lib/inference/gateway-route-compatibility.ts +++ b/src/lib/inference/gateway-route-compatibility.ts @@ -30,6 +30,7 @@ export type GatewayRouteConflictReason = | "provider-model" | "custom-endpoint" | "custom-api" + | "incomplete-route" | "incomplete-custom-route" | "invalid-gateway-binding"; @@ -114,9 +115,9 @@ function customRouteConflict( /** * Constrain read-only route discovery from durable same-gateway registry peers. - * Missing model/API fields are allowed only when the gateway has no configured - * peer, or when every peer supplies one identical value that discovery must - * subsequently verify with the exact compatibility guard. + * Missing requested model/API fields are allowed only when the gateway has no + * configured peer, or when every peer supplies one identical value that + * discovery must subsequently verify with the exact compatibility guard. */ export function preflightGatewayRouteDiscovery( request: Parameters[0] & { @@ -126,26 +127,32 @@ export function preflightGatewayRouteDiscovery( const provider = nonEmptyString(request.route.provider); if (!provider) throw new Error("Requested gateway inference route requires a provider"); const peers: SandboxEntry[] = []; - const invalidBindings: GatewayRouteConflict[] = []; + const discoveryConflicts: GatewayRouteConflict[] = []; for (const sandbox of request.sandboxes) { if (sandbox.name === request.sandboxName) continue; let recordedGatewayName: string; try { recordedGatewayName = resolveSandboxGatewayName(sandbox); } catch { - invalidBindings.push({ + discoveryConflicts.push({ sandboxName: sandbox.name, reason: "invalid-gateway-binding", scope: "registered", }); continue; } - if (recordedGatewayName === request.gatewayName && configuredRoute(sandbox)) { - peers.push(sandbox); + if (recordedGatewayName !== request.gatewayName) continue; + if (configuredRoute(sandbox)) peers.push(sandbox); + else { + discoveryConflicts.push({ + sandboxName: sandbox.name, + reason: "incomplete-route", + scope: "registered", + }); } } const requestedModel = nonEmptyString(request.route.model); - if (invalidBindings.length > 0) { + if (discoveryConflicts.length > 0) { return { ok: false, result: { @@ -153,7 +160,7 @@ export function preflightGatewayRouteDiscovery( gatewayName: request.gatewayName, sandboxName: request.sandboxName, route: { provider, model: requestedModel ?? "model discovery pending" }, - conflicts: invalidBindings, + conflicts: discoveryConflicts, }, }; } @@ -204,7 +211,7 @@ export function preflightGatewayRouteDiscovery( } /** - * Compare a requested route with every configured registry row on the same + * Compare a requested route with every durable registry row on the same * OpenShell gateway. Registry rows are intentionally used without a live-state * filter because stopped sandboxes still depend on the gateway route when they * restart. The requested route must already carry the target agent's effective @@ -254,7 +261,14 @@ export function checkGatewayRouteCompatibility( } if (recordedGatewayName !== request.gatewayName) continue; const recorded = configuredRoute(sandbox); - if (!recorded) continue; + if (!recorded) { + conflicts.push({ + sandboxName: sandbox.name, + reason: "incomplete-route", + scope: "registered", + }); + continue; + } if (recorded.provider !== requested.provider || recorded.model !== requested.model) { conflicts.push({ @@ -305,6 +319,9 @@ export function formatGatewayRouteConflict( const hasIncompleteCustomRoute = result.conflicts.some( (conflict) => conflict.reason === "incomplete-custom-route", ); + const hasIncompleteRoute = result.conflicts.some( + (conflict) => conflict.reason === "incomplete-route", + ); const hasInvalidGatewayBinding = result.conflicts.some( (conflict) => conflict.reason === "invalid-gateway-binding", ); @@ -312,6 +329,9 @@ export function formatGatewayRouteConflict( hasIncompleteCustomRoute ? "At least one custom route lacks durable endpoint or API-family metadata, so compatibility cannot be proven; remove and re-onboard that sandbox with complete custom-route metadata." : null, + hasIncompleteRoute + ? "At least one registered sandbox lacks durable provider or model metadata, so same-gateway compatibility cannot be proven; remove and re-onboard that sandbox with complete route metadata." + : null, hasInvalidGatewayBinding ? "At least one registry row has an invalid gateway binding, so gateway separation cannot be proven; restore its known-good gateway binding or remove and re-onboard that sandbox." : null, diff --git a/src/lib/registry-recovery-seeded-paths.test.ts b/src/lib/registry-recovery-seeded-paths.test.ts index 2de12cadf5c..fd4c3d6fcc7 100644 --- a/src/lib/registry-recovery-seeded-paths.test.ts +++ b/src/lib/registry-recovery-seeded-paths.test.ts @@ -3,6 +3,8 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; +import { runInferenceSet } from "./actions/inference-set.js"; +import { createDeps } from "./actions/inference-set.test-support.js"; import type { SandboxEntry } from "./state/registry.js"; interface MockRegistryState { @@ -261,4 +263,44 @@ describe("recoverRegistryEntries seeded recovery paths", () => { expect(mockRegistryState.sandboxes.beta).toBeUndefined(); expect(mockRegistryState.defaultSandbox).toBeNull(); }); + + it("blocks route mutation after seeded recovery persists a live row without route metadata (#6315)", async () => { + mockRegistryState.sandboxes.gamma = gammaEntry([]); + mockRegistryState.defaultSandbox = "gamma"; + vi.mocked(parseLiveSandboxEntries).mockReturnValue([ + { name: "recovered-live", phase: "Ready" }, + ]); + + await recoverRegistryEntries({ requestedSandboxName: "missing-sandbox" }); + expect(mockRegistryState.sandboxes["recovered-live"]).toMatchObject({ + gatewayName: "nemoclaw", + provider: null, + model: null, + }); + + const deps = createDeps({ + config: {}, + entries: Object.values(mockRegistryState.sandboxes), + defaultSandbox: "gamma", + }); + await expect( + runInferenceSet( + { provider: "nvidia-prod", model: "nvidia/model-b", sandboxName: "gamma" }, + deps, + ), + ).rejects.toMatchObject({ + message: expect.stringMatching(/recovered-live.*lacks durable provider or model metadata/s), + exitCode: 2, + }); + + expect(deps.calls.rewriteConfigUrlsWithDnsPinning).not.toHaveBeenCalled(); + expect(deps.calls.captureOpenshell).not.toHaveBeenCalled(); + expect(deps.calls.readSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.writeSandboxConfig).not.toHaveBeenCalled(); + expect(deps.calls.recomputeSandboxConfigHash).not.toHaveBeenCalled(); + expect(deps.calls.updateSandbox).not.toHaveBeenCalled(); + expect(deps.calls.updateSession).not.toHaveBeenCalled(); + expect(deps.calls.appendAuditEntry).not.toHaveBeenCalled(); + expect(deps.calls.restartSandboxGateway).not.toHaveBeenCalled(); + }); }); From 81f9e394ddc7fdba1403bb690a709b242619a68e Mon Sep 17 00:00:00 2001 From: Aaron Erickson Date: Tue, 7 Jul 2026 11:11:24 -0700 Subject: [PATCH 30/31] test(inference): complete Hermes alias route fixtures Signed-off-by: Aaron Erickson --- src/lib/actions/inference-set-hermes-run.test.ts | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/lib/actions/inference-set-hermes-run.test.ts b/src/lib/actions/inference-set-hermes-run.test.ts index 0c26f58d38d..4d19276c136 100644 --- a/src/lib/actions/inference-set-hermes-run.test.ts +++ b/src/lib/actions/inference-set-hermes-run.test.ts @@ -329,8 +329,20 @@ describe("runInferenceSet Hermes routing", () => { const deps = createDeps({ config, entries: [ - { name: "alpha", agent: "openclaw" }, - { name: "hermes-one", agent: "hermes" }, + { + name: "alpha", + agent: "openclaw", + gatewayName: "nemoclaw-9090", + gatewayPort: 9090, + provider: "nvidia-prod", + model: "nvidia/model-a", + }, + { + name: "hermes-one", + agent: "hermes", + provider: "hermes-provider", + model: "z-ai/glm-5.1", + }, ], defaultSandbox: "alpha", requestedAgent: "hermes", From 7df7274c138efe4080993378863a21b4567db842 Mon Sep 17 00:00:00 2001 From: Carlos Villela Date: Tue, 7 Jul 2026 12:43:52 -0700 Subject: [PATCH 31/31] docs(inference): clarify reusable custom routes Signed-off-by: Carlos Villela --- docs/inference/switch-inference-providers.mdx | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/inference/switch-inference-providers.mdx b/docs/inference/switch-inference-providers.mdx index 6964a3c4392..4fa8c8dffc9 100644 --- a/docs/inference/switch-inference-providers.mdx +++ b/docs/inference/switch-inference-providers.mdx @@ -33,7 +33,7 @@ All registered sandboxes on the same gateway must use the same provider and mode Custom compatible routes must also use the same endpoint and API family. An OpenClaw native-Anthropic route and a Hermes OpenAI-frontend route therefore cannot share a gateway, even when their custom endpoint and model are otherwise identical. NemoClaw checks stopped sandboxes because they depend on that route when restarted. -If a registered same-gateway sandbox lacks durable provider or model metadata, NemoClaw fails closed until you remove and re-onboard that sandbox with complete route metadata. +If a registered same-gateway sandbox lacks durable provider or model metadata, or a custom route lacks durable endpoint or API-family metadata, NemoClaw fails closed until you remove and re-onboard that sandbox with complete route metadata. When a route conflicts, onboarding, runtime switching, and connect-time repair exit non-zero before changing the gateway and name the affected sandboxes. Align the routes, remove the conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`. @@ -115,7 +115,11 @@ $$nemoclaw inference set --provider compatible-endpoint --model $$nemoclaw inference set --provider compatible-anthropic-endpoint --model ``` -To change only the model on a sandbox that is already on this provider, omit `--endpoint-url` — `inference set` reuses the endpoint that onboarding established and does not repoint the gateway route. Any `--endpoint-url` you do pass is always validated by the host-side SSRF guard, so a URL that resolves to a private or internal address is rejected even if it is the same one onboarding recorded; omit the flag to keep the established endpoint. To point the sandbox at a genuinely different endpoint, re-run `$$nemoclaw onboard` with the new endpoint (rebuild reuses the recorded endpoint and cannot change it). +To change only the model on a sandbox that is already on this provider, omit `--endpoint-url` when its durable registry entry contains both endpoint and API-family metadata. +`inference set` reuses that recorded route and does not repoint the gateway. +If the durable metadata is incomplete, NemoClaw fails closed and tells you to re-run onboarding. +Any `--endpoint-url` you pass is always validated by the host-side SSRF guard, so a URL that resolves to a private or internal address is rejected even if it matches the recorded endpoint. +To point the sandbox at a different endpoint, re-run `$$nemoclaw onboard` with the new endpoint because rebuild reuses the recorded endpoint and cannot change it.