Skip to content

fix: start the on-device model on demand and stop implying it is active #367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
KrasimirKralev merged 1 commit into from
Aug 11, 2026
Merged

fix: start the on-device model on demand and stop implying it is active #367

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions src/app/setup-api/ai-models/configure/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,15 @@ export async function POST(request: Request) {
clawaiTier?: string;
model?: string;
oauthHandoff?: boolean;
/**
* Explicit "make this the model that answers", as distinct from "install
* it and keep it available". Enabling a local model deliberately does NOT
* take over from the provider the customer chose, so the Settings panel's
* "Switch to Gemma 4" button had no way to actually switch — it ran the
* same enable flow and silently left the harness where it was. This flag
* is that missing intent; omitted, the promote policy is unchanged.
*/
activate?: boolean;
};
try {
body = await request.json();
Expand Down Expand Up @@ -607,7 +616,10 @@ export async function POST(request: Request) {
);
}

const shouldPromoteLocalToPrimary = isLocalScope && !configStore.ai_model_configured;
// A fresh device promotes its first local model automatically; an existing
// device only promotes when the user explicitly asked to switch to it.
const shouldPromoteLocalToPrimary =
isLocalScope && (!configStore.ai_model_configured || body.activate === true);
// Resolve the ClawBox AI tier once and reuse it for both the primary
// model selection (below) and the config-store write (further down).
// Inlining the same `?? storedTier ?? DEFAULT_TIER` chain in two
Expand Down Expand Up @@ -963,7 +975,19 @@ export async function POST(request: Request) {
// quietly take the device off the provider the customer chose.
if ((ocProvider === "llamacpp" || ocProvider === "ollama") && (await getActiveHarness()) === "hermes") {
try {
await applyLocalAiToHermes({ provider: ocProvider, model: config.defaultModel });
await applyLocalAiToHermes({
provider: ocProvider,
// Hermes wants the bare model id, not the `llamacpp/…` qualified
// form — matching the openclaw-absent branch above.
model: config.defaultModel.replace(/^(?:llamacpp|ollama)\//, ""),
// This branch runs on the `dual` SKU, where OpenClaw exists but
// Hermes is the harness actually answering. Without carrying the
// promotion through, "Switch to Gemma 4" moved OpenClaw's primary
// and left Hermes pointed at its old provider — the same
// configured-but-not-active split this change exists to remove,
// reproduced on the one SKU that has both.
makeDefault: shouldPromoteLocalToPrimary,
});
} catch (err) {
// Non-fatal: the local model is configured and running either way.
console.error("[ai-models/configure] Hermes local provider registration failed:", err);
Expand Down
4 changes: 4 additions & 0 deletions src/app/setup-api/ai-models/status/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ const PROVIDER_LABELS: Record<string, string> = {
openrouter: "OpenRouter",
ollama: "Ollama Local",
llamacpp: "llama.cpp Local",
// Hermes' id for the on-device model. Without an entry here the raw
// "clawlocal" leaked into the UI as a provider name. Matches the wording in
// lib/hermes-providers.ts so the same model isn't called two things.
clawlocal: "Gemma 4 (on-device)",
};

const CLAWBOX_AI_TIER_CONFIG_KEY = "clawai_tier";
Expand Down
16 changes: 12 additions & 4 deletions src/app/setup-api/llamacpp/install/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,11 @@ function startLlamaCpp(spec: ReturnType<typeof getLlamaCppLaunchSpec>, alias: st
);
}

async function configureLlamaCpp(alias: string, scope: ConfigureScope): Promise<{ ok: boolean; error?: string }> {
async function configureLlamaCpp(
alias: string,
scope: ConfigureScope,
activate: boolean,
): Promise<{ ok: boolean; error?: string }> {
const req = new Request("http://localhost/setup-api/ai-models/configure", {
method: "POST",
headers: { "Content-Type": "application/json" },
Expand All @@ -211,6 +215,7 @@ async function configureLlamaCpp(alias: string, scope: ConfigureScope): Promise<
apiKey: alias,
authMode: "local",
scope,
activate,
}),
});
const res = await configureAiModel(req);
Expand All @@ -222,7 +227,7 @@ async function configureLlamaCpp(alias: string, scope: ConfigureScope): Promise<
}

export async function POST(request: Request) {
let body: { model?: string; scope?: ConfigureScope };
let body: { model?: string; scope?: ConfigureScope; activate?: boolean };
try {
body = await request.json();
} catch {
Expand All @@ -231,6 +236,9 @@ export async function POST(request: Request) {

const alias = body.model?.trim() || getDefaultLlamaCppModel();
const scope = body.scope === "local" ? "local" : "primary";
// Only an explicit "switch to it" click sets this; a plain enable leaves the
// customer's chosen provider in place.
const activate = body.activate === true;
if (!MODEL_ID_RE.test(alias)) {
return NextResponse.json({ error: "Invalid llama.cpp model ID" }, { status: 400 });
}
Expand All @@ -246,7 +254,7 @@ export async function POST(request: Request) {
const existingModels = await queryLlamaCppModels(spec.baseUrl);
if (existingModels.includes(alias)) {
emit(controller, { status: "llama.cpp is already running. Applying configuration..." });
const configured = await configureLlamaCpp(alias, scope);
const configured = await configureLlamaCpp(alias, scope, activate);
if (!configured.ok) {
emit(controller, { error: configured.error });
controller.close();
Expand Down Expand Up @@ -324,7 +332,7 @@ export async function POST(request: Request) {
const models = await queryLlamaCppModels(spec.baseUrl);
if (models.includes(alias)) {
emit(controller, { status: "llama.cpp is ready. Applying ClawBox configuration..." });
const configured = await configureLlamaCpp(alias, scope);
const configured = await configureLlamaCpp(alias, scope, activate);
if (!configured.ok) {
emit(controller, { error: configured.error });
controller.close();
Expand Down
12 changes: 11 additions & 1 deletion src/components/AIModelsStep.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ interface AIModelsStepProps {
defaultProviderId?: string;
currentProviderId?: string | null;
currentModel?: string | null;
/**
* Whether the local model is the harness's ACTIVE selection, as opposed to
* merely installed. `currentProviderId` only ever reported the latter, so the
* llama.cpp panel showed an "already configured" pill — and hid its own
* switch button — on devices that were not actually using the model.
* Undefined keeps the old provider-id-derived behaviour for callers that
* don't know the difference (the setup wizard).
*/
localAiIsActive?: boolean;
openClawAIOfferRequest?: number;
requestedProviderId?: string | null;
providerSelectionRequest?: number;
Expand Down Expand Up @@ -400,6 +409,7 @@ export default function AIModelsStep({
defaultProviderId,
currentProviderId = null,
currentModel = null,
localAiIsActive,
openClawAIOfferRequest = 0,
requestedProviderId = null,
providerSelectionRequest = 0,
Expand Down Expand Up @@ -1820,7 +1830,7 @@ export default function AIModelsStep({
<LlamaCppModelPanel
llamaCppRunning={llamaCppRunning}
llamaCppInstalled={llamaCppInstalled}
llamaCppIsActive={normalizedCurrentProvider === "llamacpp"}
llamaCppIsActive={localAiIsActive ?? normalizedCurrentProvider === "llamacpp"}
llamaCppSaving={llamaCppSaving}
llamaCppProgress={llamaCppProgress}
selectedLlamaCppModel={selectedLlamaCppModel}
Expand Down
10 changes: 7 additions & 3 deletions src/components/LlamaCppModelPanel.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ interface LlamaCppModelPanelProps {
llamaCppProgress?: string | null;
selectedLlamaCppModel: string;
setSelectedLlamaCppModel: (model: string) => void;
saveLlamaCppConfig: (model: string) => void;
saveLlamaCppConfig: (model: string, options?: { activate?: boolean }) => void;
buttonClassName?: string;
buttonSpinner?: ReactNode;
}
Expand Down Expand Up @@ -63,7 +63,7 @@ export default function LlamaCppModelPanel({

let buttonLabel: string;
if (llamaCppSaving) {
buttonLabel = "Enabling Gemma 4...";
buttonLabel = canSwitchToGemma ? "Switching to Gemma 4..." : "Enabling Gemma 4...";
} else if (canSwitchToGemma) {
buttonLabel = "Switch to Gemma 4";
} else {
Expand Down Expand Up @@ -101,7 +101,11 @@ export default function LlamaCppModelPanel({
) : (
<button
type="button"
onClick={() => saveLlamaCppConfig(selectedLlamaCppModel)}
// When the model is already installed, this button's job is to make
// it the active one — so say so to the server. Without the flag it
// re-ran the enable flow and left the harness pointed elsewhere,
// i.e. a "Switch to Gemma 4" button that did not switch.
onClick={() => saveLlamaCppConfig(selectedLlamaCppModel, { activate: canSwitchToGemma })}
disabled={!!llamaCppSaving}
className={buttonClassName}
>
Expand Down
95 changes: 77 additions & 18 deletions src/components/SettingsApp.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import { useClawboxLogin } from "@/lib/use-clawbox-login";
import { I18nProvider, useT, LANGUAGES, type Locale } from "@/lib/i18n";
import { cachedActiveHarness, fetchHarness } from "@/lib/client-harness";
import { isPairingToken, normalizePairingToken, samePairingToken } from "@/lib/telegram-pairing-token";
import { lastModelSegment } from "@/lib/chat-header-pills";
import { QRCodeSVG } from "qrcode.react";
import type { UpdateState } from "@/lib/updater";
import { RESTART_STEP_ID } from "@/lib/update-constants";
Expand Down Expand Up @@ -90,6 +91,24 @@ const NAV_ITEMS: { id: Section; icon: string; labelKey?: string; label?: string
];

/* ── Helpers ── */
// Hermes registers the on-device model under this single provider id whichever
// local runtime backs it (see HERMES_LOCAL_PROVIDER in lib/hermes-local-ai.ts).
// OpenClaw instead names the runtime directly ("llamacpp" / "ollama"), so
// "is the local model the active provider" has to accept either spelling.
const HERMES_LOCAL_PROVIDER_ID = "clawlocal";

// Copy for the four Local AI states, kept as data so the status line and the
// card below cannot drift apart. "Selected" vs "available" is the distinction
// that matters: installing the on-device model does not make it the one that
// answers, and saying "sleeping until needed" when it was never selected read
// as though it were.
const LOCAL_AI_STATUS_SUFFIX = {
offline: "endpoint not responding",
available: "available, not currently selected",
standby: "selected · sleeping until needed",
running: "selected · running",
} as const;

function formatBytes(b: number): string {
if (!b) return "0 B";
const u = ["B", "KB", "MB", "GB", "TB"];
Expand Down Expand Up @@ -971,7 +990,10 @@ export default function SettingsApp({ ui }: SettingsAppProps) {
/* ── AI Provider ── */
const [aiProvider, setAiProvider] = useState<{ connected: boolean; provider: string | null; providerLabel: string | null; mode: string | null; model: string | null; clawaiTier: "flash" | "pro" | null } | null>(null);
useEffect(() => {
if (section !== "ai" && !isMobile) return;
// The Local AI panel needs this too: it is the only source that knows which
// provider the ACTIVE harness is really set to, which is what separates
// "the on-device model is installed" from "it is what answers".
if (section !== "ai" && section !== "localAi" && !isMobile) return;
fetch("/setup-api/ai-models/status", { cache: "no-store" }).then(r => r.json()).then(setAiProvider).catch(() => {});
}, [section, isMobile]);
// Which agent consumes the local model. Named the harness outright, and said
Expand Down Expand Up @@ -1016,6 +1038,34 @@ export default function SettingsApp({ ui }: SettingsAppProps) {
setLocalAiStatus({ configured: false, provider: null, model: null, running: null, standbyEnabled: false });
}
}, []);
// Is the on-device model the provider the active harness will actually answer
// with? `localAiStatus` alone can never say — it is built from the
// config-store keys written when the model was installed, and installing one
// deliberately does not take over from the provider the customer chose. The
// harness's own selection (via /setup-api/ai-models/status) is the only proof.
const localAiIsActive = !!localAiStatus?.configured
&& !!aiProvider?.provider
&& (aiProvider.provider === HERMES_LOCAL_PROVIDER_ID || aiProvider.provider === localAiStatus.provider);
Comment on lines +1046 to +1048

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Match the active model before marking Local AI as selected.

clawlocal identifies the Hermes local-provider slot. It does not identify a specific local model. This condition reports the configured model as active whenever the provider matches, even if aiProvider.model still selects another local model. The panel can then show standby or running state and hide the switch action incorrectly.

Compare the normalized active model with localAiStatus.model in addition to the provider.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/components/SettingsApp.tsx` around lines 1046 - 1048, Update the
localAiIsActive condition in SettingsApp so it compares the normalized active
model against localAiStatus.model in addition to the existing configured and
provider checks; keep the Hermes provider-slot matching behavior while ensuring
a different local model does not mark Local AI as selected.


/**
* The four states the Local AI cards render, resolved once so the status line
* and the card copy cannot drift apart:
* offline — configured, but the endpoint isn't answering and there is no standby
* available — installed and healthy, but something else is answering
* standby — selected, asleep to free RAM until it is needed
* running — selected and resident
*/
const localAiState: "offline" | "available" | "standby" | "running" | null = !localAiStatus?.configured
? null
: localAiStatus.running === false && !localAiStatus.standbyEnabled
? "offline"
: !localAiIsActive
? "available"
: localAiStatus.running === false
? "standby"
: "running";
const localAiOffline = localAiState === "offline";

const disableLocalAi = useCallback(async () => {
setLocalAiDisabling(true);
setLocalAiError(null);
Expand Down Expand Up @@ -2222,25 +2272,25 @@ export default function SettingsApp({ ui }: SettingsAppProps) {
</div>
) : localAiStatus.configured ? (
<div className={`flex items-center gap-4 rounded-xl px-4 py-3.5 border ${
localAiStatus.running === false && !localAiStatus.standbyEnabled
localAiOffline
? "bg-amber-500/[0.06] border-amber-500/15"
: "bg-cyan-500/[0.06] border-cyan-500/15"
}`}>
<div className={`relative w-10 h-10 rounded-full border flex items-center justify-center shrink-0 ${
localAiStatus.running === false && !localAiStatus.standbyEnabled
localAiOffline
? "bg-amber-500/10 border-amber-400/10"
: "bg-cyan-500/10 border-cyan-400/10"
}`}>
<AIProviderIcon provider={localAiStatus.provider} size={24} />
<span className={`absolute -right-1 -bottom-1 w-5 h-5 rounded-full border flex items-center justify-center ${
localAiStatus.running === false && !localAiStatus.standbyEnabled
localAiOffline
? "bg-[#2a1d10] border-amber-500/25"
: "bg-[#10212a] border-cyan-500/25"
}`}>
<span className={`material-symbols-rounded ${
localAiStatus.running === false && !localAiStatus.standbyEnabled ? "text-amber-300" : "text-cyan-300"
localAiOffline ? "text-amber-300" : "text-cyan-300"
}`} style={{ fontSize: 14 }}>
{localAiStatus.running === false && !localAiStatus.standbyEnabled ? "warning" : "check"}
{localAiOffline ? "warning" : "check"}
</span>
</span>
</div>
Expand All @@ -2250,16 +2300,14 @@ export default function SettingsApp({ ui }: SettingsAppProps) {
</div>
<div className="flex items-center gap-1.5 mt-0.5">
<span className={`w-1.5 h-1.5 rounded-full animate-pulse ${
localAiStatus.running === false && !localAiStatus.standbyEnabled ? "bg-amber-300" : "bg-cyan-300"
localAiOffline ? "bg-amber-300" : "bg-cyan-300"
}`} />
<span className={`text-xs ${
localAiStatus.running === false && !localAiStatus.standbyEnabled ? "text-amber-300/80" : "text-cyan-300/80"
localAiOffline ? "text-amber-300/80" : "text-cyan-300/80"
}`}>
{localAiStatus.running === false && !localAiStatus.standbyEnabled
? `${localAiStatus.model ? localAiStatus.model.split("/").pop() : "Configured"} · endpoint not responding`
: localAiStatus.running === false
? `${localAiStatus.model ? localAiStatus.model.split("/").pop() : "Configured"} · sleeping until needed`
: (localAiStatus.model ? localAiStatus.model.split("/").pop() : "Ready as fallback")}
{`${localAiStatus.model ? lastModelSegment(localAiStatus.model) : "Configured"} · ${
localAiState ? LOCAL_AI_STATUS_SUFFIX[localAiState] : ""
}`}
</span>
</div>
</div>
Expand Down Expand Up @@ -2329,11 +2377,17 @@ export default function SettingsApp({ ui }: SettingsAppProps) {
{localAiStatus.provider === "llamacpp" ? "Gemma 4" : "Ollama"}
</div>
<p className="text-sm text-[var(--text-secondary)] mt-1">
{localAiStatus.running === false && !localAiStatus.standbyEnabled
{localAiState === "offline"
? "Configured, but currently offline."
: localAiStatus.running === false
? `Enabled with on-demand standby to free RAM until ${harnessLabel} needs it.`
: "Enabled and ready as your local backup model."}
: localAiState === "available"
// Installed and ready, but the harness is pointed
// elsewhere — name what IS answering so the state is
// unambiguous. The label comes from the same endpoint
// that reports the active provider, so it can't drift.
? `Installed and ready, but ${harnessLabel} is currently set to ${aiProvider?.providerLabel || aiProvider?.provider || "another provider"}.`
: localAiState === "standby"
? `Selected. Kept in on-demand standby to free RAM until ${harnessLabel} needs it.`
: "Selected and running as your on-device model."}
</p>
</div>
<button
Expand All @@ -2357,9 +2411,14 @@ export default function SettingsApp({ ui }: SettingsAppProps) {
defaultProviderId="llamacpp"
currentProviderId={localAiStatus?.provider ?? null}
currentModel={localAiStatus?.model ?? null}
// Installed is not selected. Without this the panel rendered the
// green "already configured" pill and hid its own switch button,
// so a device that had Gemma installed but unselected offered no
// way to actually start using it.
localAiIsActive={localAiIsActive}
title="Set Up Local AI"
description={localAiStatus?.configured
? "Gemma 4 is configured as your private on-device fallback."
? "Gemma 4 is installed as your private on-device model."
Comment on lines +2414 to +2421

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Refresh aiProvider after local-model activation.

localAiIsActive derives from aiProvider, but the onConfigured callback refreshes only localAiStatus. After a successful switch, this component retains the old active-provider result until remount. The panel can continue to show “Switch to Gemma 4” after Hermes or OpenClaw already selected it.

Fetch /setup-api/ai-models/status and update aiProvider in this callback.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/components/SettingsApp.tsx` around lines 2414 - 2421, Update the
local-model activation onConfigured callback in SettingsApp so it refreshes both
localAiStatus and aiProvider after success. Fetch /setup-api/ai-models/status in
that callback, derive the current provider from the response, and update
aiProvider so the panel immediately reflects Gemma 4 activation without
requiring a remount.

: "Turn on a local model so ClawBox always has a private on-device backup."}
configureScope="local"
testId="settings-local-ai-step"
Expand Down
Loading
Loading