fix: address issue #5456

[jason-ma-nv]

Summary

nemoclaw inference set now recomputes the model context window on a model switch instead of carrying the previous model's contextWindow into the new model's config. Root cause: buildProviderConfig cloned the prior models[0] and only overrode id/name/compat, preserving the stale contextWindow for any provider switch (onboard computed a per-provider window, but inference set never did). Added a resolveContextWindow dependency to runInferenceSet whose default mirrors onboard per provider: probe the Ollama runtime context length (floored at MIN_AUTODETECTED_OLLAMA_CONTEXT_WINDOW=16384) for ollama-local, read max_model_len from /v1/models for vllm-local, and fall back to the flat 131072 cloud default otherwise. patchOpenClawInferenceConfig/buildProviderConfig now accept the recomputed window: a positive number replaces the stale value, null drops it (so a failed local probe falls back to the agent default rather than a wrong stale window), and undefined preserves the existing value for direct callers (e.g. connect route repair). Added a pure resolveVllmRuntimeContextWindow helper in vllm-runtime-context.ts and a host-probe wrapper in local.ts, plus a shared DEFAULT_CLOUD_CONTEXT_WINDOW constant in config.ts. The fix is OpenClaw-specific because the contextWindow field lives only in the OpenClaw config; Hermes config has no per-model contextWindow field.

Related Issue

Fixes #5456

Changes

• Automated Claude Code fix selected by auto_fix/auto_fix_recent_issues.py.
• See the commits on this branch for the exact file-level changes.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
• Targeted tests pass for changed behavior
• Full npm test passes (broad runtime changes only)
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

Verification details reported by Claude Code:

• Wrote 3 failing unit tests first (TDD) in src/lib/actions/inference-set.test.ts reproducing the stale-window carry-over: two patchOpenClawInferenceConfig cases (replace stale window with recomputed value; drop stale window when none available) and one runInferenceSet case (cloud->ollama switch recomputes window). Confirmed all 3 failed with the stale 131072 still present.
• Implemented the fix and reran: src/lib/actions/inference-set.test.ts passes 23/23.
• Ran the full inference test surface after rebuilding dist: 466/466 passing across src/lib/inference + inference-set.
• Ran related regression files (connect-route-repair, vllm-runtime-context, ollama-runtime-context, openclaw-config-merge): 54/54 passing.
• CLI build (build:cli) and type-check (typecheck:cli) both clean.
• Biome check/format clean on all changed source files.
• Updated docs/inference/switch-inference-providers.mdx to document that inference set recomputes the context window per provider on a model switch.
• npx vitest run --project cli src/lib/actions/inference-set.test.ts
• npm run build:cli
• npx vitest run --project cli src/lib/inference/ src/lib/actions/inference-set.test.ts
• npm run typecheck:cli
• npx vitest run --project cli src/lib/actions/inference-set.test.ts src/lib/actions/sandbox/connect-route-repair.test.ts src/lib/inference/vllm-runtime-context.test.ts src/lib/inference/ollama-runtime-context.test.ts src/lib/state/openclaw-config-merge.test.ts
• npx biome check --write src/lib/actions/inference-set.ts src/lib/actions/inference-set.test.ts src/lib/inference/local.ts src/lib/inference/vllm-runtime-context.ts src/lib/inference/config.ts

---

Signed-off-by: Jason Ma jama@nvidia.com

Summary by CodeRabbit

• New Features

  • Context window is automatically recomputed when switching between inference providers or models

• Bug Fixes

  • Resolved issue where stale context window values persisted when switching to a different inference provider

• Documentation

  • Clarified how context windows are determined for different inference provider types (ollama-local, vllm-local, and cloud defaults)

[coderabbitai]

📝 Walkthrough

Walkthrough

inference set now recomputes the model context window on every model switch instead of carrying over the stale value from the previous model. A new resolveContextWindow dependency is added to InferenceSetDeps, backed by defaultResolveContextWindow which probes Ollama, reads vLLM max_model_len, or falls back to a DEFAULT_CLOUD_CONTEXT_WINDOW constant (131072). buildProviderConfig and patchOpenClawInferenceConfig are extended to set or delete contextWindow based on the resolved value.

Changes

Context-window recomputation on model switch

Layer / File(s)	Summary
vLLM context-window resolver and shared constant src/lib/inference/vllm-runtime-context.ts, src/lib/inference/config.ts, src/lib/inference/local.ts	Adds resolveVllmRuntimeContextWindowFromModels pure helper extracting max_model_len from a /v1/models response, exports DEFAULT_CLOUD_CONTEXT_WINDOW = 131072, and adds a resolveVllmRuntimeContextWindow probe that curls the local vLLM endpoint and delegates to the pure resolver. Also re-exports MIN_AUTODETECTED_OLLAMA_CONTEXT_WINDOW from local.ts.
InferenceSetDeps contract, defaultResolveContextWindow, and patchOpenClawInferenceConfig wiring src/lib/actions/inference-set.ts	Adds resolveContextWindow(provider, model): number | null to InferenceSetDeps, implements defaultResolveContextWindow dispatching across ollama-local/vllm-local/cloud, extends buildProviderConfig and patchOpenClawInferenceConfig to accept the resolved value (setting or deleting the field), and calls deps.resolveContextWindow in runInferenceSet.
Tests and documentation src/lib/actions/inference-set.test.ts, docs/inference/switch-inference-providers.mdx	Extends createDeps with optional resolveContextWindow, adds unit tests for the recompute-and-replace and stale-value-drop paths in patchOpenClawInferenceConfig, adds an integration test asserting runInferenceSet uses the recomputed window, and adds docs describing per-backend detection on inference set.

Sequence Diagram(s)

sequenceDiagram
  actor User
  participant runInferenceSet
  participant defaultResolveContextWindow
  participant resolveVllmRuntimeContextWindow as resolveVllmRuntimeContextWindow (local.ts)
  participant vLLM as vLLM /v1/models
  participant patchOpenClawInferenceConfig
  participant buildProviderConfig

  User->>runInferenceSet: inference set --provider vllm-local --model mymodel
  runInferenceSet->>defaultResolveContextWindow: provider=vllm-local, model=mymodel
  defaultResolveContextWindow->>resolveVllmRuntimeContextWindow: modelId=mymodel
  resolveVllmRuntimeContextWindow->>vLLM: GET /v1/models (curl)
  vLLM-->>resolveVllmRuntimeContextWindow: JSON with max_model_len
  resolveVllmRuntimeContextWindow-->>defaultResolveContextWindow: number | null
  defaultResolveContextWindow-->>runInferenceSet: contextWindow
  runInferenceSet->>patchOpenClawInferenceConfig: config, provider, model, contextWindow
  patchOpenClawInferenceConfig->>buildProviderConfig: ..., contextWindow
  buildProviderConfig-->>patchOpenClawInferenceConfig: sets or deletes contextWindow field
  patchOpenClawInferenceConfig-->>runInferenceSet: updated OpenClaw config

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 No more stale windows carried from before,
The claw now probes each runtime at the door.
For Ollama it asks, for vLLM it peeks,
And cloud gets 131k as the default it seeks.
The context is fresh with each model we set —
No overflow silently lurking, no tokens misread yet! 🪟✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 23.08% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	❓ Inconclusive	The title 'fix: address issue #5456' is overly generic and vague, referencing only an issue number without describing what the fix addresses or what problem it solves.	Use a more descriptive title that summarizes the actual fix, such as 'fix: recompute context window on inference provider switches' to convey the core change.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	The PR fully implements the requirements from issue #5456: context window recomputation for provider switches (ollama-local, vllm-local, and cloud), matching onboard behavior, with comprehensive test coverage and documentation updates.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to fixing the context window recomputation issue: adding resolveContextWindow dependency, modifying patchOpenClawInferenceConfig and buildProviderConfig, and implementing vllm/local runtime probing helpers.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch auto/fix-5456-inference-cli-inference-set-does-not

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the auto/fix-5456-infere... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	main	auto/fix-5456-infere... 3e084f6	+/-
nemoclaw/src/se...cret-scanner.ts	—	100%	—
nemoclaw/src/commands/slash.ts	—	100%	—
nemoclaw/src/li...bprocess-env.ts	—	100%	—
nemoclaw/src/bl...eprint/state.ts	—	98%	—
nemoclaw/src/onboard/config.ts	—	98%	—
nemoclaw/src/bl...int/snapshot.ts	—	97%	—
nemoclaw/src/bl...print/runner.ts	—	95%	—
nemoclaw/src/co...ration-state.ts	—	94%	—
nemoclaw/src/bl...ate-networks.ts	—	94%	—
nemoclaw/src/index.ts	—	94%	—

TypeScript / code-coverage/cli

The overall coverage in the auto/fix-5456-infere... branch is 44%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	main	auto/fix-5456-infere... 3e084f6	+/-
src/lib/state/o...oard-session.ts	—	90%	—
src/lib/inference/local.ts	—	75%	—
src/lib/sandbox/config.ts	—	72%	—
src/lib/inference/nim.ts	—	72%	—
src/lib/onboard/preflight.ts	—	64%	—
src/lib/state/sandbox.ts	—	55%	—
src/lib/onboard...er-gpu-patch.ts	—	50%	—
src/lib/actions...licy-channel.ts	—	49%	—
src/lib/policy/index.ts	—	48%	—
src/lib/onboard.ts	—	17%	—

---

_{Updated June 15, 2026 09:20 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-5458.docs.buildwithfern.com/nemoclaw

[github-actions]

E2E Advisor Recommendation

Required E2E: openclaw-inference-switch-vitest
Optional E2E: gpu-e2e, openclaw-inference-switch-e2e

Dispatch hint: openclaw-inference-switch-vitest

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• openclaw-inference-switch-vitest (high): Direct coverage for the changed OpenClaw nemoclaw inference set path. It exercises a real install/OpenShell/Docker/live-provider boundary and verifies route, openclaw.json, registry, inference.local, and live agent behavior after switching.

Optional E2E

• gpu-e2e (high): Useful adjacent confidence for the local Ollama inference path touched by this PR, especially runtime context-window behavior, but it validates local Ollama onboarding/inference rather than the exact inference set switch path and requires a GPU runner.
• openclaw-inference-switch-e2e (high): Nightly script-based counterpart for OpenClaw inference switching. Optional if openclaw-inference-switch-vitest runs, because the Vitest job is the current migrated live scenario for this flow.

New E2E recommendations

• local inference model switching (high): Existing E2E coverage appears to validate OpenClaw inference switching for hosted providers and local Ollama onboarding/inference, but not switching an already-running OpenClaw sandbox to ollama-local or vllm-local and asserting the recomputed contextWindow in openclaw.json.
  • Suggested test: Add a hermetic or lightweight live E2E scenario that mocks local Ollama/vLLM runtime metadata, runs nemoclaw inference set --provider ollama-local and --provider vllm-local, and verifies stale context windows are replaced or dropped in the running OpenClaw config.

Dispatch hint

• Workflow: .github/workflows/e2e-vitest-scenarios.yaml
• jobs input: openclaw-inference-switch-vitest

[github-actions]

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: ubuntu-repo-cloud-openclaw
Optional Vitest E2E scenarios: None

Dispatch required Vitest E2E scenarios:

• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required Vitest E2E scenarios

• ubuntu-repo-cloud-openclaw: The PR changes OpenClaw inference switching and context-window recomputation logic. This live-supported Ubuntu cloud OpenClaw scenario exercises the inference path and is the smallest available Vitest scenario covering the affected OpenClaw inference surface.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw

Optional Vitest E2E scenarios

• None.

Relevant changed files

• src/lib/actions/inference-set.ts
• src/lib/inference/config.ts
• src/lib/inference/local.ts
• src/lib/inference/vllm-runtime-context.ts

[github-actions]

PR Review Advisor

Findings: 0 needs attention, 3 worth checking, 1 nice ideas
Top item: Add direct tests for the default context-window resolver fallbacks

Review findings

🛠️ Needs attention

• None.

🔎 Worth checking

• Source-of-truth review needed: src/lib/inference/vllm-runtime-context.ts and src/lib/inference/local.ts vLLM context-window fallback: The advisor marked localized patch analysis as needs_followup.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Evidence: `local.ts` returns `null` for empty curl output and invalid JSON; `vllm-runtime-context.ts` returns `null` for missing entries, missing/blank `max_model_len`, malformed values, or over-ceiling values.
• Default resolver fallback behavior is not directly covered (src/lib/inference/vllm-runtime-context.ts:23): The new `runInferenceSet` test injects `resolveContextWindow`, so it proves plumbing but not the default provider-specific behavior added in this PR. The new pure vLLM resolver and the local host-probe wrapper also return `null` for external invalid states without direct tests on those new call paths. Existing `applyVllmRuntimeContextWindow` tests cover similar parsing semantics, but the new resolver could diverge without failing.
  • Recommendation: Add focused tests for the new resolver paths: cloud switches reset a stale local window to 131072, Ollama detections below the floor become `MIN_AUTODETECTED_OLLAMA_CONTEXT_WINDOW`, the vLLM pure resolver handles matched/missing model ids and invalid/over-ceiling `max_model_len`, and the local vLLM wrapper returns `null` for empty or invalid JSON curl output.
  • Evidence: `runInferenceSet` passes `deps.resolveContextWindow(provider, model)` into `patchOpenClawInferenceConfig`, but the added caller test supplies `resolveContextWindow: () => 16384`; `vllm-runtime-context.test.ts` imports only `applyVllmRuntimeContextWindow`, not the new `resolveVllmRuntimeContextWindow`.
• Active duplicate PR appears to patch the same issue and files: The drift context reports open PR fix(inference): recompute context window on model switch #5457 with the same linked issue [Inference][CLI] inference set does not recompute the context window on model switch — stale window carries over (any provider) #5456 and overlapping core files. That creates a higher chance of contradictory implementations or duplicated review effort.
  • Recommendation: Before merging, compare this implementation against the active overlapping PR and decide which branch should be the source of truth for the [Inference][CLI] inference set does not recompute the context window on model switch — stale window carries over (any provider) #5456 fix.
  • Evidence: Open PR overlap: fix(inference): recompute context window on model switch #5457 is titled `fix(inference): recompute context window on model switch`, links issue [Inference][CLI] inference set does not recompute the context window on model switch — stale window carries over (any provider) #5456, and overlaps `src/lib/actions/inference-set.test.ts`, `src/lib/actions/inference-set.ts`, `src/lib/inference/config.ts`, and `src/lib/inference/vllm-runtime-context.ts`.

🌱 Nice ideas

• Consider extracting the context-window resolver out of the growing action module (src/lib/actions/inference-set.ts:120): `inference-set.ts` is already a large action module and this PR adds provider-specific probing knowledge to it. The behavior is small today, but keeping the provider resolver near inference runtime helpers would reduce future action-module growth.
  • Recommendation: If this area changes again, consider moving `defaultResolveContextWindow` into a focused inference context-window module and keeping `runInferenceSet` responsible only for orchestration.
  • Evidence: The drift context flags `src/lib/actions/inference-set.ts` growing by 51 lines and `src/lib/inference/local.ts` by 44 lines; the new `defaultResolveContextWindow` starts at `src/lib/actions/inference-set.ts:120`.

Consider writing more tests for

• **Runtime validation** — Verify `runInferenceSet` or an exported default resolver resets an OpenClaw model from a stale Ollama-sized `contextWindow` to 131072 when switching to `nvidia-prod`.. The PR changes runtime inference switching and local network probing. Unit tests cover the config mutation and dependency-injected caller plumbing, but provider-specific default resolver behavior and new fallback paths are not directly pinned.
• **Runtime validation** — Verify an Ollama switch floors a detected below-floor runtime context length to `MIN_AUTODETECTED_OLLAMA_CONTEXT_WINDOW` rather than preserving the prior cloud window.. The PR changes runtime inference switching and local network probing. Unit tests cover the config mutation and dependency-injected caller plumbing, but provider-specific default resolver behavior and new fallback paths are not directly pinned.
• **Runtime validation** — Verify `resolveVllmRuntimeContextWindow` returns the matched model's `max_model_len`, falls back to the first entry when the target id is absent, and returns `null` for missing, malformed, non-positive, or over-ceiling values.. The PR changes runtime inference switching and local network probing. Unit tests cover the config mutation and dependency-injected caller plumbing, but provider-specific default resolver behavior and new fallback paths are not directly pinned.
• **Runtime validation** — Verify the local vLLM host-probe wrapper invokes curl against `http://127.0.0.1:<VLLM\_PORT>/v1/models\` with bounded timeouts and returns `null` for empty output or invalid JSON.. The PR changes runtime inference switching and local network probing. Unit tests cover the config mutation and dependency-injected caller plumbing, but provider-specific default resolver behavior and new fallback paths are not directly pinned.
• **Runtime validation** — Consider a targeted runtime/integration validation that switches a sandbox cloud→Ollama and Ollama→cloud and inspects the written OpenClaw config context window, without relying on external E2E job status in review.. The PR changes runtime inference switching and local network probing. Unit tests cover the config mutation and dependency-injected caller plumbing, but provider-specific default resolver behavior and new fallback paths are not directly pinned.
• **Default resolver fallback behavior is not directly covered** — Add focused tests for the new resolver paths: cloud switches reset a stale local window to 131072, Ollama detections below the floor become `MIN_AUTODETECTED_OLLAMA_CONTEXT_WINDOW`, the vLLM pure resolver handles matched/missing model ids and invalid/over-ceiling `max_model_len`, and the local vLLM wrapper returns `null` for empty or invalid JSON curl output.
• **Acceptance clause:** This is PROVIDER-AGNOSTIC — the prior window is kept for any switch. — add test evidence or identify existing coverage. The production code recomputes for OpenClaw switches through `deps.resolveContextWindow(provider, model)`, including cloud, Ollama, and vLLM branches. Test coverage directly exercises cloud-to-Ollama through a mocked resolver, but does not explicitly cover Ollama-to-cloud or vLLM default resolution.
• **Acceptance clause:** vllm → applyVllmRuntimeContextWindow (reads /v1/models max_model_len) — add test evidence or identify existing coverage. `resolveVllmRuntimeContextWindow` curls `/v1/models` and the pure resolver reads `max_model_len`; however the new resolver functions are not directly covered by tests.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/actions/inference-set.ts (1)

455-457: Run the targeted inference-switch E2E jobs before merge.

Because this path updates live route switching and config patching, run the two recommended jobs to validate both agent flows end to end:

gh workflow run nightly-e2e.yaml --ref <branch> -f jobs=openclaw-inference-switch-e2e,hermes-inference-switch-e2e

As per coding guidelines: src/lib/actions/inference-set.ts includes an explicit E2E recommendation for openclaw-inference-switch-e2e and hermes-inference-switch-e2e.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/inference-set.ts` around lines 455 - 457, Before merging this
pull request, run the specified E2E test workflows to validate the inference
switching and context window configuration changes in the resolveContextWindow
call path. Execute the workflow command provided in the comment using the gh CLI
with your branch name, running both the openclaw-inference-switch-e2e and
hermes-inference-switch-e2e jobs to ensure agent flows work end-to-end with the
live route switching and config patching modifications.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/actions/inference-set.ts`:
- Around line 455-457: Before merging this pull request, run the specified E2E
test workflows to validate the inference switching and context window
configuration changes in the resolveContextWindow call path. Execute the
workflow command provided in the comment using the gh CLI with your branch name,
running both the openclaw-inference-switch-e2e and hermes-inference-switch-e2e
jobs to ensure agent flows work end-to-end with the live route switching and
config patching modifications.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ac9dd04c-6279-4c63-a049-b5f36c937d62

📥 Commits

Reviewing files that changed from the base of the PR and between f4f3c58 and 3e084f6.

📒 Files selected for processing (6)

• docs/inference/switch-inference-providers.mdx
• src/lib/actions/inference-set.test.ts
• src/lib/actions/inference-set.ts
• src/lib/inference/config.ts
• src/lib/inference/local.ts
• src/lib/inference/vllm-runtime-context.ts

[jason-ma-nv]

Closing in favor of #5457, which fixes the same issue (#5456).

Both PRs recompute the context window on an inference set model switch and stop buildProviderConfig from carrying the previous model's contextWindow. The deciding difference is the Ollama path:

• resolveOllamaRuntimeContextWindow only returns a value when the model is already loaded (runtimeStatus.loaded ? contextLength : null).
• fix(inference): recompute context window on model switch #5457 warms the model first, then probes — so in the canonical repro (cloud → ollama-local on a freshly-pulled, not-yet-loaded model) it loads the model and writes the correct runtime window (~16384), mirroring onboard.
• This PR probes without warming, so for a not-yet-loaded model the probe returns null and it drops the stale window (falling back to the agent default) rather than setting the correct value. It's a valid root-cause fix for the cloud/vLLM paths but a less complete fix for the headline Ollama case.

#5457 also surfaces the recomputed window to the user (logs the value / warns to rebuild on failure), which the issue explicitly flagged as missing.

A few pieces from this PR are worth carrying into #5457 — noted in a review comment there (Ollama floor at 16384, vLLM probe timeouts, and the docs paragraph).

This PR was auto-created by the issue auto-fix pipeline.