fix(inference): keep first Local Ollama TUI turn under OpenClaw compaction budget (#5468)

[yimoj]

Summary

Local Ollama small-context models (e.g. qwen2.5:0.5b) could not answer a first
hello in the OpenClaw TUI: the turn failed with "Auto-compaction could not
recover this turn." NemoClaw floors a Local Ollama runtime window to 16k, but
OpenClaw 2026.5.27's default 20k compaction reserve leaves only ~8k of first-turn
prompt budget — too small for the agent base prompt + tool catalogue (~7.4k) —
so the first turn overflowed and preemptive compaction (no prior history to
compact) failed. This bakes a context-aware compaction reserve into the
generated openclaw.json for small Local Ollama windows so the first turn fits.

Related Issue

Fixes #5468

Changes

• scripts/generate-openclaw-config.mts: For the Local Ollama upstream
  provider with a context window at or below 28k (OpenClaw's 20k reserve floor +
  its 8k minimum prompt budget), emit agents.defaults.compaction with
  reserveTokens and reserveTokensFloor lowered to the model's reply budget
  (maxTokens). This raises promptBudgetBeforeReserve from ~8k to
  contextWindow - maxTokens (12288 for a 16k window). Both keys are set because
  OpenClaw takes max(reserveTokens, reserveTokensFloor), so lowering the floor
  alone would let the 20k default pull the reserve back up. The reserve is
  clamped so a pathological maxTokens can never make the window worse than the
  default. Larger windows keep OpenClaw's default safeguard untouched.
• Dockerfile: Add ARG NEMOCLAW_UPSTREAM_PROVIDER and promote it to ENV
  so the config generator can see the user-selected upstream provider
  (ollama-local), which is otherwise collapsed to the managed inference
  provider key. dockerfile-patch.ts already writes this ARG (previously a
  no-op for the OpenClaw image, which lacked it).
• test/ollama-local-openclaw-config-propagation.test.ts: Unit coverage for
  upstream-provider propagation through the staged Dockerfile and for the
  small-context compaction policy (small window applies, non-ollama/large window
  does not, threshold boundary, pathological-maxTokens clamp).
• test/e2e/test-gpu-e2e.sh: New Phase 5.5 asserting the baked
  openclaw.json carries the small-context reserve policy and that the real
  openclaw tui first turn is not blocked by preemptive auto-compaction.

Type of Change

• Code change (feature, bug fix, or refactor)

Verification

• Targeted tests pass for changed behavior
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed

End-to-end (real nemoclaw onboard + real openclaw tui, Local Ollama, qwen2.5:0.5b)

Reproduced the exact reporter workflow on a Linux host (CPU; the bug is a
config/budget interaction, not GPU-specific) through the worktree CLI:
node ./bin/nemoclaw.js onboard (Local Ollama / qwen2.5:0.5b) → real sandbox →
openclaw tui → send hello.

Pre-fix (baked config: contextWindow=16384, no compaction key):

⚠️ Auto-compaction could not recover this turn. I kept this conversation mapped
to the current session. Please try again, use /compact, or use /new to start a
fresh session.
 agent main | session main | inference/qwen2.5:0.5b | tokens ?/16k

Post-fix (same fresh nemoclaw onboard --fresh; baked config now adds
agents.defaults.compaction {reserveTokens: 4096, reserveTokensFloor: 4096},
raising promptBudget from 16384 → 12288):

 ⠦ streaming • 0s | connected
Hello! How can I assist you today?
 connected | idle
 agent main | session main | inference/qwen2.5:0.5b | tokens ?/16k

The first turn returns a real assistant reply with no auto-compaction recovery
failure.

---

Signed-off-by: Yimo Jiang yimoj@nvidia.com

Summary by CodeRabbit

Release Notes

• New Features

  • Automatic context compaction policy is now applied for small-context local Ollama models (≤28,000 token context windows) to improve first-turn reliability and reduce context overflow failures.

• Build/Configuration

  • Added a new build-time upstream provider setting that is carried into the container environment for config generation, alongside the existing provider key behavior.

• Tests

  • Expanded end-to-end coverage for small-context first-turn behavior.
  • Added unit coverage for compaction policy decisions and upstream provider configuration propagation.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3edf6120-7513-4ce9-8d38-1fc18b44a3a8

📥 Commits

Reviewing files that changed from the base of the PR and between 3fd115c and 47cee64.

📒 Files selected for processing (1)

• scripts/generate-openclaw-config.mts

---

📝 Walkthrough

Walkthrough

Adds NEMOCLAW_UPSTREAM_PROVIDER as a Docker build ARG and container ENV variable. Introduces a buildLocalOllamaSmallContextCompaction() helper in the config-generation script that computes a clamped reserveTokens compaction override for ollama-local models at or below a 28k-token context threshold, applied conditionally inside buildConfig(). Unit and E2E tests validate the policy.

Changes

Local Ollama Small-Context Compaction Fix

Layer / File(s)	Summary
Dockerfile: NEMOCLAW_UPSTREAM_PROVIDER ARG and ENV Dockerfile	Declares ARG NEMOCLAW_UPSTREAM_PROVIDER= with an empty default and adds it to the ENV block so the config script can read the user-selected provider at build/runtime.
Config script: constants, helper, and buildConfig integration scripts/generate-openclaw-config.mts	Documents NEMOCLAW_UPSTREAM_PROVIDER as a main input, defines LOCAL_OLLAMA_UPSTREAM_PROVIDER and small-context threshold constants, exports buildLocalOllamaSmallContextCompaction() which returns a clamped { reserveTokens, reserveTokensFloor } object when provider is ollama-local and contextWindow <= 28000, and wires the result into agentDefaults.compaction inside buildConfig().
Unit and E2E tests test/ollama-local-openclaw-config-propagation.test.ts, test/e2e/test-gpu-e2e.sh	Adds a Dockerfile-staging test asserting NEMOCLAW_UPSTREAM_PROVIDER is preserved, a describe block covering compaction override presence/absence and reserve clamping at boundary context sizes, and an E2E Phase 5.5 that inspects the baked openclaw.json and optionally drives the OpenClaw TUI via expect to verify no auto-compaction failure on first turn.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• NVIDIA/NemoClaw#5154: Changes to the patchStagedDockerfile implementation are directly tested and validated by the new staged-Dockerfile test in this PR.

Suggested labels

bug-fix, area: inference, area: local-models, integration: openclaw, nightly-e2e

Suggested reviewers

• cv

Poem

🐇 A context too small, the TUI would stall,
No "hello" got through—compaction ate all!
But now with a clamp on the reserve floor,
Small Ollama windows can chat once more.
The tokens align, the thresholds just right—
Qwen2.5 finally speaks into the night! 🌙

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately reflects the main fix: enabling Local Ollama small-context models to work via OpenClaw TUI by adjusting the compaction budget.
Linked Issues check	✅ Passed	All code changes directly address issue #5468: config generator applies small-context compaction policy for Local Ollama models, Dockerfile propagates upstream provider, and tests verify both the policy and end-to-end TUI behavior.
Out of Scope Changes check	✅ Passed	All changes are scoped to the small-context Ollama compaction fix: Dockerfile build args, config generation logic, unit tests, and e2e validation for the reported model/context window combination.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}