-
Notifications
You must be signed in to change notification settings - Fork 3.1k
fix(onboard): recover portable Ollama lifecycle #8506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1020,10 +1020,21 @@ Set `NEMOCLAW_NO_CONNECT_HINT=1` to suppress the hint in scripted workflows. | |
| If the sandbox is running an outdated agent version, a non-blocking warning prints before connecting with a `$$nemoclaw <name> rebuild` hint. | ||
| If another terminal is already connected to the sandbox, `connect` prints a note with the number of existing sessions before proceeding. Multiple concurrent sessions are allowed. | ||
|
|
||
| `connect` does not pull or serve a model itself, but it does inspect managed-vLLM install variables such as `NEMOCLAW_VLLM_MODEL` and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` if you exported them in the same shell. | ||
| `connect` does not pull a model itself, but it does inspect managed-vLLM install variables such as `NEMOCLAW_VLLM_MODEL` and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` if you exported them in the same shell. | ||
| An unknown model slug, malformed extra-args JSON, or a gated model (for example `deepseek-r1-distill-70b`) with no `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN` exits non-zero with the same error the installer would emit, before any sandbox readiness probe or SSH attach. | ||
| Unset the managed-vLLM variable, or fix the value, before retrying. | ||
|
Comment on lines
+1023
to
1025
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Document the These lines state that every 🤖 Prompt for AI Agents |
||
|
|
||
| <AgentOnly variant="openclaw"> | ||
| For a portable experimental-profile sandbox with the recorded `ollama-local` provider, `connect --probe-only` probes `http://127.0.0.1:11434/api/tags` before it decides whether to start Ollama. | ||
| If the API is unhealthy and the ownership receipt is valid, it starts the fixed user-local Ollama executable with `serve`. | ||
| It refuses to launch a duplicate when another `ollama` process exists but the API remains unhealthy. | ||
| After a launch, it waits up to 30 seconds for valid `/api/tags` JSON with a `models` array. | ||
| It then requires the existing sandbox `inference.local` route probe to return a trusted reachable result before it exits successfully. | ||
| When route repair needs the local Ollama dependency check, it verifies or recovers the authenticated proxy before it retries `inference.local`. | ||
| If Ollama does not become healthy within 30 seconds, the command identifies the exact receipt-bound executable and its `serve` argument, then tells you to retry recovery. | ||
| The command does not take over a system service or an unrelated user-managed Ollama daemon. | ||
| </AgentOnly> | ||
|
|
||
| Before reading or changing the live OpenShell gateway inference route, `connect` verifies the shared provider and sandbox metadata. | ||
| When the live route differs and the metadata is compatible, `connect` warns and re-points the route to the target sandbox's recorded provider and model. | ||
| Refer to [Use Shared Gateway Routes](../inference/manage-inference/use-shared-gateway-routes) for provider-global identity, route drift, and hard-error recovery. | ||
|
|
@@ -1378,6 +1389,12 @@ $$nemoclaw my-assistant recover | |
|
|
||
| </AgentOnly> | ||
|
|
||
| <AgentOnly variant="openclaw"> | ||
|
|
||
| For a portable experimental-profile sandbox with the recorded `ollama-local` provider, `recover` also runs the ownership-bound Ollama and `inference.local` route reachability checks described for [`connect --probe-only`](#$$nemoclaw-name-connect). | ||
|
|
||
| </AgentOnly> | ||
|
|
||
| <AgentOnly variant="hermes"> | ||
|
|
||
| `recover` re-evaluates the documented Hermes secret boundary against `/sandbox/.hermes/.env` and the supervisor runtime environment on every run, including when the gateway is already healthy. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import fs from "node:fs"; | ||
| import os from "node:os"; | ||
| import path from "node:path"; | ||
|
|
||
| import { afterEach, describe, expect, it } from "vitest"; | ||
|
|
||
| import { | ||
| loadUserLocalOllamaOwnership, | ||
| recordUserLocalOllamaOwnership, | ||
| removeUserLocalOllamaOwnership, | ||
| userLocalOllamaOwnershipInternals, | ||
| } from "./ollama-user-local-runtime"; | ||
|
|
||
| const temporaryDirectories: string[] = []; | ||
|
|
||
| function createFixture(): { homeDir: string; stateDir: string; binPath: string } { | ||
| const homeDir = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-ollama-owner-")); | ||
| temporaryDirectories.push(homeDir); | ||
| return { | ||
| homeDir, | ||
| stateDir: path.join(homeDir, ".nemoclaw"), | ||
| binPath: path.join(homeDir, ".local", "bin", "ollama"), | ||
| }; | ||
| } | ||
|
|
||
| afterEach(() => { | ||
| for (const directory of temporaryDirectories.splice(0)) { | ||
| fs.rmSync(directory, { force: true, recursive: true }); | ||
| } | ||
| }); | ||
|
|
||
| describe("user-local Ollama ownership receipt", () => { | ||
| it("records and reloads only the fixed NemoClaw user-local path (#8502)", () => { | ||
| const fixture = createFixture(); | ||
|
|
||
| recordUserLocalOllamaOwnership(fixture.binPath, fixture); | ||
|
|
||
| expect(loadUserLocalOllamaOwnership(fixture)).toBe(fixture.binPath); | ||
| const receipt = userLocalOllamaOwnershipInternals.receiptPath(fixture); | ||
| expect(fs.statSync(receipt).mode & 0o777).toBe(0o600); | ||
| }); | ||
|
|
||
| it("refuses to record an Ollama path outside the fixed user-local install (#8502)", () => { | ||
| const fixture = createFixture(); | ||
|
|
||
| expect(() => recordUserLocalOllamaOwnership("/usr/local/bin/ollama", fixture)).toThrow( | ||
| "unexpected user-local Ollama path", | ||
| ); | ||
| }); | ||
|
|
||
| it("rejects a receipt that redirects recovery to another executable (#8502)", () => { | ||
| const fixture = createFixture(); | ||
| const receipt = userLocalOllamaOwnershipInternals.receiptPath(fixture); | ||
| fs.mkdirSync(path.dirname(receipt), { recursive: true }); | ||
| fs.writeFileSync( | ||
| receipt, | ||
| `${JSON.stringify({ schemaVersion: 1, binPath: "/tmp/unrelated" })}\n`, | ||
| { mode: 0o600 }, | ||
| ); | ||
|
|
||
| expect(() => loadUserLocalOllamaOwnership(fixture)).toThrow("ownership receipt is invalid"); | ||
| }); | ||
|
|
||
| it("removes obsolete ownership after a system installation (#8502)", () => { | ||
| const fixture = createFixture(); | ||
| recordUserLocalOllamaOwnership(fixture.binPath, fixture); | ||
|
|
||
| removeUserLocalOllamaOwnership(fixture); | ||
|
|
||
| expect(loadUserLocalOllamaOwnership(fixture)).toBeNull(); | ||
| }); | ||
| }); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import fs from "node:fs"; | ||
| import os from "node:os"; | ||
| import path from "node:path"; | ||
|
|
||
| import { openRegularFileNoFollow } from "../../adapters/fs/regular-file"; | ||
| import { OLLAMA_PORT } from "../../core/ports"; | ||
| import { ensureConfigDir } from "../../state/config-io"; | ||
|
|
||
| export { OLLAMA_PORT }; | ||
|
|
||
| const RECEIPT_DIRECTORY = "ollama"; | ||
| const RECEIPT_FILE = "user-local-ownership.json"; | ||
| const MAX_RECEIPT_BYTES = 1024; | ||
|
|
||
| interface UserLocalOllamaOwnershipReceipt { | ||
| schemaVersion: 1; | ||
| binPath: string; | ||
| } | ||
|
|
||
| export interface UserLocalOllamaOwnershipDeps { | ||
| homeDir?: string; | ||
| stateDir?: string; | ||
| } | ||
|
|
||
| function resolveHomeDir(deps: UserLocalOllamaOwnershipDeps): string { | ||
| return deps.homeDir ?? os.homedir(); | ||
| } | ||
|
|
||
| function resolveStateDir(deps: UserLocalOllamaOwnershipDeps): string { | ||
| return deps.stateDir ?? path.join(resolveHomeDir(deps), ".nemoclaw"); | ||
| } | ||
|
|
||
| function receiptPath(deps: UserLocalOllamaOwnershipDeps): string { | ||
| return path.join(resolveStateDir(deps), RECEIPT_DIRECTORY, RECEIPT_FILE); | ||
| } | ||
|
|
||
| function expectedBinPath(deps: UserLocalOllamaOwnershipDeps): string { | ||
| return path.join(resolveHomeDir(deps), ".local", "bin", "ollama"); | ||
| } | ||
|
|
||
| function parseReceipt( | ||
| value: unknown, | ||
| deps: UserLocalOllamaOwnershipDeps, | ||
| ): UserLocalOllamaOwnershipReceipt { | ||
| if (typeof value !== "object" || value === null || Array.isArray(value)) { | ||
| throw new Error("NemoClaw-managed user-local Ollama ownership receipt is malformed"); | ||
| } | ||
| const receipt = value as Record<string, unknown>; | ||
| if ( | ||
| Object.keys(receipt).sort().join(",") !== "binPath,schemaVersion" || | ||
| receipt.schemaVersion !== 1 || | ||
| receipt.binPath !== expectedBinPath(deps) | ||
| ) { | ||
| throw new Error("NemoClaw-managed user-local Ollama ownership receipt is invalid"); | ||
| } | ||
| return receipt as unknown as UserLocalOllamaOwnershipReceipt; | ||
| } | ||
|
|
||
| /** Record the fixed user-local binary only after NemoClaw starts it successfully. */ | ||
| export function recordUserLocalOllamaOwnership( | ||
| binPath: string, | ||
| deps: UserLocalOllamaOwnershipDeps = {}, | ||
| ): void { | ||
| if (binPath !== expectedBinPath(deps)) { | ||
| throw new Error("NemoClaw refused to record an unexpected user-local Ollama path"); | ||
| } | ||
| const target = receiptPath(deps); | ||
| ensureConfigDir(path.dirname(target)); | ||
| let file; | ||
| try { | ||
| file = openRegularFileNoFollow(target, { writable: true }); | ||
| } catch (error) { | ||
| if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; | ||
| file = openRegularFileNoFollow(target, { create: true, mode: 0o600, writable: true }); | ||
| } | ||
| try { | ||
| const receipt: UserLocalOllamaOwnershipReceipt = { schemaVersion: 1, binPath }; | ||
| file.replaceUtf8(`${JSON.stringify(receipt, null, 2)}\n`, 0o600); | ||
| } finally { | ||
| file.close(); | ||
| } | ||
| } | ||
|
|
||
| /** Load the exact receipt-bound user-local path, or null when no receipt exists. */ | ||
| export function loadUserLocalOllamaOwnership( | ||
| deps: UserLocalOllamaOwnershipDeps = {}, | ||
| ): string | null { | ||
| let file; | ||
| try { | ||
| file = openRegularFileNoFollow(receiptPath(deps)); | ||
| } catch (error) { | ||
| if ((error as NodeJS.ErrnoException).code === "ENOENT") return null; | ||
| throw error; | ||
| } | ||
| try { | ||
| const receipt = parseReceipt(JSON.parse(file.readUtf8(MAX_RECEIPT_BYTES)), deps); | ||
| return receipt.binPath; | ||
| } catch (error) { | ||
| if (error instanceof SyntaxError) { | ||
| throw new Error("NemoClaw-managed user-local Ollama ownership receipt is malformed"); | ||
| } | ||
| throw error; | ||
| } finally { | ||
| file.close(); | ||
| } | ||
| } | ||
|
|
||
| /** Remove stale user-local ownership after a successful system installation. */ | ||
| export function removeUserLocalOllamaOwnership(deps: UserLocalOllamaOwnershipDeps = {}): void { | ||
| try { | ||
| fs.unlinkSync(receiptPath(deps)); | ||
| } catch (error) { | ||
| if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; | ||
| } | ||
| } | ||
|
|
||
| export const userLocalOllamaOwnershipInternals = { receiptPath }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Scope the automatic-start statement to the applicable flow.
Line 54 says that an ordinary user-local daemon needs a manual restart after reboot. The earlier unqualified statement says NemoClaw starts any installed stopped daemon. Scope that statement to installation or onboarding, or document the portable receipt-owned recovery exception there.
🤖 Prompt for AI Agents