fix(inference): show registered providers and onboard tip on provider-not-found by kagura-agent · Pull Request #5983 · NVIDIA/NemoClaw

kagura-agent · 2026-06-29T10:23:28Z

Description

When nemoclaw inference set fails because the requested provider is not registered in the gateway, the error message now includes the list of registered providers and a tip to run nemoclaw onboard.

Before

× provider 'openai-api' not found

After

OpenShell inference route update failed with exit 1.
Registered providers: nvidia-prod
Tip: register a new provider with `nemoclaw onboard`.

Changes

Capture openshell inference set output (pipe stdio instead of inherit) to detect provider-not-found errors
When detected, query the sandbox registry for known providers and format an enhanced error message
Defensive try-catch around the registry query so a registry failure doesn't mask the original error

Testing

3 new vitest test cases in inference-set.test.ts:
1. Provider-not-found with registered providers → shows list + tip
2. Provider-not-found with no registered providers → shows 'No providers registered' + tip
3. Non-provider-not-found failure → unchanged generic error
All 34 tests pass, typecheck clean

Closes #5924

Signed-off-by: kagura-agent kagura.agent.ai@gmail.com

Summary by CodeRabbit

Bug Fixes
- Improved OpenShell inference route update errors with clearer, user-focused messaging for “provider not found” cases.
- When applicable, errors now include a deduplicated “registered providers” list and an onboarding tip to register providers via the CLI.
- If registry details can’t be retrieved, messaging falls back to “No providers registered,” while still showing the onboarding tip.
- For other failures, error text stays generic and avoids including sensitive stderr output.
Tests
- Expanded inference-set tests to verify the updated classification and messaging (including registry lookup failure and non-provider-not-found cases).

…-not-found (NVIDIA#5924) When `openshell inference set` fails with a provider-not-found error, the error message now lists registered providers and suggests running `nemoclaw onboard` to register a new one. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com> Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com>

copy-pr-bot · 2026-06-29T10:23:31Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2026-06-29T10:23:45Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9c4fb6fd-9ea1-42a1-a94e-64fc94eb6a41

📥 Commits

Reviewing files that changed from the base of the PR and between f9f873b and 1e59a23.

📒 Files selected for processing (1)

src/lib/actions/inference-set.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

src/lib/actions/inference-set.test.ts

📝 Walkthrough

Walkthrough

runOpenshell now accepts stdio, and runInferenceSet uses piped stderr plus provider-pattern matching to throw provider-aware errors with sandbox-derived provider details or onboarding guidance. The tests were updated for the new call shape and expanded error-message coverage.

Changes

Provider-not-found error enrichment

Layer / File(s)	Summary
runOpenshell contract and failure handling `src/lib/actions/inference-set.ts`	`runOpenshell` opts now accepts `stdio: StdioOptions`. The non-zero exit path pipes stdout/stderr, formats redacted failure details, matches provider-not-found patterns, and throws `InferenceSetError` messages that include sandbox-derived provider information or a fallback when registry listing fails.
Updated inference-set error tests `src/lib/actions/inference-set.test.ts`	Existing OpenClaw and Hermes expectations include `stdio: ["ignore", "pipe", "pipe"]`. New tests cover provider-not-found, non-matching stderr, missing provider metadata, and registry listing failures.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: provider-not-found guidance with registered providers and onboard tip.
Linked Issues check	✅ Passed	The PR matches `#5924` by adding registered-provider context and a `nemoclaw onboard` tip for unregistered provider failures.
Out of Scope Changes check	✅ Passed	The extra error-detail handling and tests support the same provider-not-found workflow and do not appear unrelated.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

src/lib/actions/inference-set.test.ts (1)
1113-1180: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a regression case for listSandboxes() throwing.

The new fallback branch in runInferenceSet is still untested. A case where listSandboxes() throws would lock in the intended public error output for registry failures instead of only the happy-path lookup variants. As per path instructions, “Review tests for behavioral confidence rather than implementation lock-in.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/inference-set.test.ts` around lines 1113 - 1180, Add a
regression test in inference-set.test.ts for the new fallback path in
runInferenceSet when listSandboxes() throws during provider lookup. Use
createDeps and mock the deps call that powers listSandboxes to throw, then
assert the surfaced Error message matches the intended public registry-failure
output and still includes the onboard tip when appropriate. Keep the test
alongside the existing runInferenceSet provider-not-found cases so it covers the
fallback branch without locking into implementation details.
Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/actions/inference-set.ts`:
- Around line 620-639: In the InferenceSetError construction inside the
inference-set flow, avoid defaulting to “No providers registered” when
deps.listSandboxes() fails. Track whether the provider registry lookup succeeded
in the try block, and only populate providerList from the registeredProviders
list when it does; if the lookup throws, leave out provider details entirely and
keep the fallback tip text. Use the existing deps.listSandboxes,
registeredProviders, and InferenceSetError block to locate the fix.

---

Nitpick comments:
In `@src/lib/actions/inference-set.test.ts`:
- Around line 1113-1180: Add a regression test in inference-set.test.ts for the
new fallback path in runInferenceSet when listSandboxes() throws during provider
lookup. Use createDeps and mock the deps call that powers listSandboxes to
throw, then assert the surfaced Error message matches the intended public
registry-failure output and still includes the onboard tip when appropriate.
Keep the test alongside the existing runInferenceSet provider-not-found cases so
it covers the fallback branch without locking into implementation details.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3cfcd6b2-0113-4a48-ae6b-b4750208f12b

📥 Commits

Reviewing files that changed from the base of the PR and between c6113be and fac6dc5.

📒 Files selected for processing (2)

src/lib/actions/inference-set.test.ts
src/lib/actions/inference-set.ts

Address CodeRabbit review: - Don't report 'No providers registered' when listSandboxes() throws - Add regression test for listSandboxes() throwing Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com>

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/actions/inference-set.test.ts`:
- Around line 1186-1195: The test setup in runInferenceSet is using a mocked
missing provider that does not match the provider under test, so the failure
path is not validating the intended behavior. Update the stderr fixture and the
runInferenceSet call to use the same provider name, referencing the
runInferenceSet test case and its provider/model inputs, so the assertion
exercises the public error enrichment for the requested provider directly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2c5b7b90-942e-4f9a-99c5-1ce544c9d33c

📥 Commits

Reviewing files that changed from the base of the PR and between fac6dc5 and 30d0c86.

📒 Files selected for processing (2)

src/lib/actions/inference-set.test.ts
src/lib/actions/inference-set.ts

🚧 Files skipped from review as they are similar to previous changes (1)

src/lib/actions/inference-set.ts

wscurran · 2026-06-29T14:43:04Z

✨ Thanks for the fix. The enhanced error message with registered providers list and onboard tip is a clear UX improvement for nemoclaw inference set.

Related open issues:

#5924 [All Platforms][Inference] nemoclaw inference set with unregistered provider returns provider '<X>' not found with no hint to run nemoclaw onboard

…t-provider-hint

cv · 2026-06-30T06:21:27Z

Superseded by #6023, which preserves this exact verified head SHA in NVIDIA/NemoClaw so the mandatory PR Review Advisor can run. No commits were rewritten or cherry-picked.

…-not-found (#6023)  ## Summary  When `nemoclaw inference set` fails because the requested provider is not registered, preserve a bounded, fully redacted failure diagnostic and add the registered provider list plus an onboarding recovery tip. This same-repository replacement preserves #5983's verified contributor history so the mandatory PR Review Advisor can run; fork-origin advisor jobs are skipped by policy. ## Related Issue  Closes #5924 Supersedes #5983 ## Changes  - Capture `openshell inference set` output and enrich only a quoted provider-not-found diagnostic naming the provider actually requested. - Fully redact, whitespace-compact, and cap generic and enhanced OpenShell failure details at 500 characters so diagnostics remain useful without exposing credentials or unbounded subprocess output. - Query the sandbox registry for known providers without allowing registry lookup failures to hide the original route-update error; emit only a static safe warning when that lookup fails. - Isolate parsing and message formatting in `inference-set-error.ts`, with focused security/correctness tests separate from the already-large action test. - Cover matching, mismatched, malformed, stderr, and stdout diagnostics; registered and empty registries; registry failures; full secret-shape redaction; enhanced-path output bounds; and generic failures. - Compatibility boundary: OpenShell 0.0.71 exposes this condition only as subprocess text. Unknown or drifted formats stay generic; replace this parser when OpenShell provides a structured provider-not-found error carrying the missing provider as a field. - Preserve #5983's exact four GitHub-Verified commits without rewriting or cherry-picking, then add two signed maintainer hardening/refactor commits on top. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates  - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: this enriches an existing failure path without changing commands, configuration, or successful behavior. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: full redaction, exact-provider correlation, bounded output, malformed-input behavior, safe registry-warning text, and focused regressions were manually reviewed. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification  - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [x] Targeted tests pass for changed behavior — 38/38 focused tests - [x] `npm run typecheck:cli` passes - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the style guide (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Exact-head E2E: https://github.com/NVIDIA/NemoClaw/actions/runs/28425898821 ---  Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com>  ## Summary by CodeRabbit * **New Features** * Enhanced “provider not found” failure diagnostics when switching inference providers, including a registered-provider list (or “No providers registered”) and an onboarding tip to run `nemoclaw onboard`. * Improved visibility into command output while redacting sensitive details. * **Bug Fixes** * More reliable detection of provider-missing failures and clearer, actionable error reporting; non-provider-related failures remain generic. * When the requested provider can’t be found, existing route/sandbox state remains unchanged. * **Documentation** * Added prerequisites and troubleshooting guidance for cross-provider switching, including what happens when OpenShell can’t locate the requested provider.  --------- Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> Signed-off-by: Prek Shiv <prekshiv@nvidia.com> Co-authored-by: kagura-agent <kagura.agent.ai@gmail.com> Co-authored-by: Claude Opus 4 <noreply@anthropic.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>

…-not-found (NVIDIA#6023)  ## Summary  When `nemoclaw inference set` fails because the requested provider is not registered, preserve a bounded, fully redacted failure diagnostic and add the registered provider list plus an onboarding recovery tip. This same-repository replacement preserves NVIDIA#5983's verified contributor history so the mandatory PR Review Advisor can run; fork-origin advisor jobs are skipped by policy. ## Related Issue  Closes NVIDIA#5924 Supersedes NVIDIA#5983 ## Changes  - Capture `openshell inference set` output and enrich only a quoted provider-not-found diagnostic naming the provider actually requested. - Fully redact, whitespace-compact, and cap generic and enhanced OpenShell failure details at 500 characters so diagnostics remain useful without exposing credentials or unbounded subprocess output. - Query the sandbox registry for known providers without allowing registry lookup failures to hide the original route-update error; emit only a static safe warning when that lookup fails. - Isolate parsing and message formatting in `inference-set-error.ts`, with focused security/correctness tests separate from the already-large action test. - Cover matching, mismatched, malformed, stderr, and stdout diagnostics; registered and empty registries; registry failures; full secret-shape redaction; enhanced-path output bounds; and generic failures. - Compatibility boundary: OpenShell 0.0.71 exposes this condition only as subprocess text. Unknown or drifted formats stay generic; replace this parser when OpenShell provides a structured provider-not-found error carrying the missing provider as a field. - Preserve NVIDIA#5983's exact four GitHub-Verified commits without rewriting or cherry-picking, then add two signed maintainer hardening/refactor commits on top. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates  - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: this enriches an existing failure path without changing commands, configuration, or successful behavior. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: full redaction, exact-provider correlation, bounded output, malformed-input behavior, safe registry-warning text, and focused regressions were manually reviewed. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification  - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [x] Targeted tests pass for changed behavior — 38/38 focused tests - [x] `npm run typecheck:cli` passes - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the style guide (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Exact-head E2E: https://github.com/NVIDIA/NemoClaw/actions/runs/28425898821 ---  Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com>  ## Summary by CodeRabbit * **New Features** * Enhanced “provider not found” failure diagnostics when switching inference providers, including a registered-provider list (or “No providers registered”) and an onboarding tip to run `nemoclaw onboard`. * Improved visibility into command output while redacting sensitive details. * **Bug Fixes** * More reliable detection of provider-missing failures and clearer, actionable error reporting; non-provider-related failures remain generic. * When the requested provider can’t be found, existing route/sandbox state remains unchanged. * **Documentation** * Added prerequisites and troubleshooting guidance for cross-provider switching, including what happens when OpenShell can’t locate the requested provider.  --------- Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> Signed-off-by: Prek Shiv <prekshiv@nvidia.com> Co-authored-by: kagura-agent <kagura.agent.ai@gmail.com> Co-authored-by: Claude Opus 4 <noreply@anthropic.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>

coderabbitai Bot reviewed Jun 29, 2026

View reviewed changes

Comment thread src/lib/actions/inference-set.ts Outdated

fix: handle registry errors and add listSandboxes throw test

30d0c86

Address CodeRabbit review: - Don't report 'No providers registered' when listSandboxes() throws - Add regression test for listSandboxes() throwing Signed-off-by: kagura-agent <kagura.agent.ai@gmail.com>

coderabbitai Bot reviewed Jun 29, 2026

View reviewed changes

Comment thread src/lib/actions/inference-set.test.ts

wscurran added area: cli Command line interface, flags, terminal UX, or output area: inference Inference routing, serving, model selection, or outputs area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow bug-fix PR fixes a bug or regression labels Jun 29, 2026

wscurran added the v0.0.71 label Jun 29, 2026

wscurran requested a review from prekshivyas June 29, 2026 14:43

cv added 2 commits June 29, 2026 21:27

fix(inference): preserve captured failure details

f9f873b

Merge remote-tracking branch 'origin/main' into fix/5924-inference-se…

1e59a23

…t-provider-hint

cv mentioned this pull request Jun 30, 2026

fix(inference): show registered providers and onboard tip on provider-not-found #6023

Merged

22 tasks

cv closed this Jun 30, 2026

Navigation Menu

Uh oh!

fix(inference): show registered providers and onboard tip on provider-not-found - #5983

fix(inference): show registered providers and onboard tip on provider-not-found#5983
kagura-agent wants to merge 4 commits into
NVIDIA:mainfrom
kagura-agent:fix/5924-inference-set-provider-hint

kagura-agent commented Jun 29, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

copy-pr-bot Bot commented Jun 29, 2026

Uh oh!

coderabbitai Bot commented Jun 29, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

wscurran commented Jun 29, 2026

Uh oh!

cv commented Jun 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kagura-agent commented Jun 29, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Before

After

Changes

Testing

Summary by CodeRabbit

Uh oh!

copy-pr-bot Bot commented Jun 29, 2026

Uh oh!

coderabbitai Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wscurran commented Jun 29, 2026

Uh oh!

cv commented Jun 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kagura-agent commented Jun 29, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 29, 2026 •

edited

Loading