Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 40 additions & 4 deletions docs/inference/set-up-llama-cpp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ Onboarding performs these actions:
- Reuses a verified GGUF from the shared `~/.cache/huggingface/` cache, or acquires the exact file through the existing Hugging Face mechanism when it is absent or invalid.
- Verifies the GGUF size and SHA-256 digest before it starts the runtime.
- Runs the authenticated container as the current non-root user with one NVIDIA GPU and no CPU fallback.
- Publishes port `8081` on `127.0.0.1` and connects the container to a Docker internal network.
- Starts a private host bridge on `127.0.0.1:8081` and connects the container to a Docker internal network without publishing a container port.
- Registers `llama-cpp-local` and routes agent traffic through `https://inference.local/v1`.

The runtime cannot download a model and has no egress on its internal Docker network.
Expand All @@ -147,16 +147,14 @@ Selecting either runtime fails before model acquisition or runtime mutation and

## Verify the Managed Runtime

Run the managed runtime, route, and host-port checks:
Run the managed runtime and route checks:

```bash
$$nemoclaw my-assistant status
$$nemoclaw my-assistant doctor
docker port nemoclaw-llama-cpp
```

Accept the result when `status` reports `Managed llama.cpp: running`, the inference route is `reachable`, and `doctor` exits with status `0`.
The Docker port output must contain `127.0.0.1:8081`.
These checks do not establish agent and model qualification.

`status` reports the recipe ID, model digest, image reference, endpoint, and lifecycle state without exposing the API key.
Expand All @@ -165,6 +163,44 @@ Refer to [CLI Commands](../../reference/commands) for complete command behavior.

## Recover the Managed Runtime

During managed installation, NemoClaw first checks internal runtime readiness and host-loopback health.
It repeats these checks when onboarding resumes.
It then runs the authoritative OpenShell Docker bridge probe on fixed port `8081`.
If that bridge probe cannot connect or times out after the earlier checks pass, onboarding stops without changing UFW.
When the bridge reports a valid narrow subnet and a gateway IP address inside that subnet, the error reports these values:

- The detected Docker network.
- The source subnet.
- The gateway IP address.
- The fixed port `8081`.
- The exact narrow UFW command.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

If the bridge topology is missing or invalid, onboarding fails closed without printing topology or a UFW command.
Inspect the OpenShell Docker bridge configuration before you retry onboarding.

<Warning>
The printed command changes UFW.
Confirm that the reported Docker network, source subnet, and gateway IP address belong to the OpenShell Docker bridge before you run it.
The rule allows TCP traffic only from that subnet to port `8081` on that gateway IP address.
</Warning>

Run the exact command from the onboarding error.
It has this form:

```bash
sudo ufw allow from <subnet> to <gateway-ip> port 8081 proto tcp
```

After you apply the rule, rerun the same onboarding selection.

If onboarding still reports the bridge failure, or when the managed runtime no longer needs the rule, remove the exact rule that you added:

```bash
sudo ufw --force delete allow from <subnet> to <gateway-ip> port 8081 proto tcp
```

Use the same subnet and gateway IP address that appeared in the original command.

If onboarding stops, rerun the same provider and recipe selection.
NemoClaw resumes only the exact persisted Docker authority, image, recipe, model digest, network, and runtime identity.
It reconciles an unfinished create journal before it starts a new runtime.
Expand Down
21 changes: 21 additions & 0 deletions src/lib/onboard/host-service-reachability.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,27 @@ describe("formatHostServiceUnreachableMessage", () => {
expect(msg).toContain("nemoclaw onboard");
});

it.each([
"nemohermes",
"nemo-deepagents",
])("uses the invoked %s CLI in the recovery command (#8712)", (invokedAs) => {
vi.stubEnv("NEMOCLAW_INVOKED_AS", invokedAs);

const msg = formatHostServiceUnreachableMessage(
{
ok: false,
reason: "tcp_failed",
port: 8081,
networkName: "openshell-docker",
subnet: "172.18.0.0/16",
gatewayIp: "172.18.0.1",
},
{ serviceLabel: "managed llama.cpp server" },
);

expect(msg).toContain(`Then rerun \`${invokedAs} onboard\`.`);
});

it("falls back to result.port when no explicit port option is given", () => {
const msg = formatHostServiceUnreachableMessage(
{
Expand Down
3 changes: 2 additions & 1 deletion src/lib/onboard/host-service-reachability.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
*/

import { dockerCapture, dockerRun } from "../adapters/docker/run";
import { cliName } from "./branding";

export const DEFAULT_PROBE_NETWORK = "openshell-docker";
const HOST_INTERNAL_NAME = "host.openshell.internal";
Expand Down Expand Up @@ -252,7 +253,7 @@ export function formatHostServiceUnreachableMessage(
" A host firewall may be blocking traffic from the OpenShell Docker bridge.",
" To allow it:",
allowCmd,
" Then re-run `nemoclaw onboard`.",
` Then rerun \`${cliName()} onboard\`.`,
].join("\n");
}

Expand Down
Loading
Loading