Skip to content

fix(onboard): probe registered extra providers exactly - #6596

Merged
cv merged 35 commits into
mainfrom
codex/6501-strict-extra-provider-reconciliation
Jul 10, 2026
Merged

fix(onboard): probe registered extra providers exactly#6596
cv merged 35 commits into
mainfrom
codex/6501-strict-extra-provider-reconciliation

Conversation

@apurvvkumaria

@apurvvkumaria apurvvkumaria commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes the post-merge #6587 gap where one paginated provider-list page could omit healthy registry extras beyond entry 100. Reconciliation now uses bounded, gateway-scoped per-provider probes, recognizes the exact wrapped OpenShell #6501 diagnostic, and preserves every indeterminate result.

Related Issue

Follow-up to #6501 and #6587. Supersedes #6518.

Changes

  • Probe every recorded extra with argv-safe provider get -g <gateway> <name> instead of treating one list page as authoritative.
  • Omit only exit-1 diagnostics that bind the exact, case-sensitive quoted provider name to a tightly anchored not-found shape, including the wrapped not-found-and-unrecognized form reported in [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501.
  • Fail open on gateway, transport, authentication, timeout, signal, ambiguous, name-mismatched, or command-spoofed output.
  • Read canonical stderr/stdout without duplicating Node's composite output array, and treat process errors or capture-limit diagnostics as indeterminate.
  • Bound each probe to 5 seconds and 64 KiB, cap total reconciliation time at 15 seconds, and never mutate the local registry.
  • Cover stale providers after index 100, the literal wrapped issue diagnostic in a realistic spawnSync result, capture truncation, mixed-case names, multi-line diagnostic spoofing, gateway failures, aggregate timeout, and stable repeated sandbox-create provider arguments.

The create-time filter is intentionally non-destructive. A provider omitted from one sandbox create remains in the user-owned registry for a later retry; --fresh does not purge it. Indeterminate probes preserve the attachment, and the final sandbox create remains authoritative and reports any concrete attachment failure. This avoids deleting healthy user configuration because of a transient gateway or transport failure.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: internal provider-state reconciliation only; no CLI, prompt, configuration, policy, or required user action changes.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent nine-category security review passed after exact-name, whole-diagnostic, subprocess-bound, and fail-open checks.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: CLI unit 14/14 and spawn integration 1/1 passed; the spawn test uses a realistic composite process result, executes two create attempts, and proves identical filtered provider arguments.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: not applicable to this focused reconciliation change; npm run typecheck:cli, npm run check:diff, source-shape, test-size, and test-conditional scans passed.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Exact-head live validation for onboard-repair, onboard-resume, and cloud-onboard passed at workflow run 29048610487 for head a0ea6e60f98c6e15a6084dba7a5ba9ff1e89215a.


Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

  • New Features

    • Onboarding and repair now detect and remove stale extra-provider records.
    • Resume operations can explicitly recreate an existing sandbox while preserving its registry entry.
    • Sandbox creation and retries now use a consistent, deduplicated provider list.
  • Bug Fixes

    • Providers are retained when availability checks are inconclusive, preventing accidental removal.
    • Reconciliation now avoids relying on potentially stale provider-list snapshots and handles diagnostic variations more reliably.

Replace the first-page snapshot with bounded, gateway-scoped per-provider probes.

Remove only exact absence; preserve indeterminate results and local registry state.

Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria apurvvkumaria added area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow bug-fix PR fixes a bug or regression v0.0.79 labels Jul 9, 2026
@apurvvkumaria apurvvkumaria self-assigned this Jul 9, 2026
@github-code-quality

github-code-quality Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/6501-strict-ex... branch remains at 77%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.
File main 90d5cdf codex/6501-strict-ex... bc4a194 +/-
src/lib/messagi...ls/wechat/qr.ts 92% 74% -18%
src/lib/state/s...dbox-session.ts 79% 64% -15%
src/lib/actions...er-preflight.ts 93% 84% -9%
src/lib/shields...nsition-lock.ts 87% 82% -5%
src/lib/onboard...cy-selection.ts 87% 82% -5%
src/lib/messagi...agent-config.ts 64% 69% +5%
src/lib/runner.ts 72% 80% +8%
src/lib/onboard...conciliation.ts 44% 85% +41%
src/lib/onboard...ostic-parser.ts 0% 91% +91%
src/lib/onboard...est-fixtures.ts 0% 100% +100%

Updated July 10, 2026 19:55 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Extra-provider reconciliation now probes each recorded provider, removes only exact not-found matches, preserves indeterminate results, and bounds diagnostics and probe time. Onboarding resume flows accept explicit sandbox recreation, while unit, integration, and live tests cover stale-provider pruning and filtered sandbox creation.

Changes

Extra-provider reconciliation

Layer / File(s) Summary
Probe contract and diagnostic matching
src/lib/onboard/extra-provider-reconciliation.ts, src/lib/onboard/extra-provider-diagnostic-parser.ts
Probe results, timing, warning hooks, diagnostic normalization, and exact provider-not-found matching were added.
Per-provider reconciliation loop
src/lib/onboard/extra-provider-reconciliation.ts, src/lib/onboard/extra-provider-reconciliation.test-fixtures.ts, src/lib/onboard/extra-provider-reconciliation.test.ts
Recorded providers are probed individually within a shared deadline; only definitive exact not-found results remove names.
Diagnostic and probe validation
src/lib/onboard/*reconciliation*.test.ts
Tests cover probe failures, time limits, gateway preconditions, diagnostic variants, ambiguity, redaction, and bounded parsing.

Sandbox recreation and stale-provider filtering

Layer / File(s) Summary
Explicit sandbox recreation flow
src/lib/onboard.ts, src/lib/onboard/machine/core-flow-phases.ts, src/lib/onboard/machine/handlers/*
The recreate callback is propagated through onboarding and causes resume decisions to recreate sandboxes while preserving registry entries.
Filtered creation and live registry scenarios
src/lib/onboard/sandbox-create-plan-extra-providers.test.ts, test/onboard-extra-provider-reconciliation.test.ts, test/e2e/fixtures/extra-providers-registry.ts, test/e2e/live/*
Creation retries deduplicate providers, registry fixtures persist provider state, and live repair/resume tests verify stale providers are pruned and excluded from gateway configuration.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Onboard
  participant Reconciliation
  participant OpenShell
  participant Gateway
  Onboard->>Reconciliation: reconcile recorded providers
  Reconciliation->>OpenShell: provider get for each name
  OpenShell->>Gateway: query provider status
  Gateway-->>OpenShell: status and diagnostics
  OpenShell-->>Reconciliation: probe result
  Reconciliation-->>Onboard: filtered provider names
Loading

Suggested labels: refactor

Suggested reviewers: cv, jyaunches

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.85% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: exact probing of registered extra providers during onboarding reconciliation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/6501-strict-extra-provider-reconciliation

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: cloud-onboard, onboard-repair, onboard-resume
Optional E2E: gateway-guard-recovery

Dispatch hint: cloud-onboard,onboard-repair,onboard-resume

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • cloud-onboard (high): Required by the deterministic risk plan for platform-install/full hosted onboarding risk. Changes in core onboarding and sandbox setup can affect clean-host install, hosted inference setup, sandbox creation, and the real assistant user flow.
  • onboard-repair (medium-high): Required by the deterministic risk plan and the onboarding resume rule. The PR changes resume/repair sandbox lifecycle paths and extra-provider reconciliation; live repair validates recreating a missing recorded sandbox, preserving live provider attachments, pruning stale attachments, and converging persisted state with runtime state.
  • onboard-resume (medium): Required by the deterministic risk plan and the onboarding resume rule. The PR changes resume orchestration, sandbox recreation resume handling, and create-intent extra-provider propagation; live resume validates interruption recovery, cached setup reuse, sandbox recreation, and selective extra-provider reconciliation.

Optional E2E

  • gateway-guard-recovery (medium): Useful adjacent confidence for the gateway-recovery.ts changes, especially selected-gateway recovery and post-recovery runtime handling. Not merge-blocking because the deterministic floor is already covered by cloud-onboard, onboard-repair, and onboard-resume.

New E2E recommendations

  • None.

Dispatch hint

  • Workflow: .github/workflows/e2e.yaml
  • jobs input: cloud-onboard,onboard-repair,onboard-resume

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Recommendation

Required E2E targets: cloud-onboard, onboard-repair, onboard-resume
Optional E2E targets: None

Dispatch required E2E targets:

  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=cloud-onboard
  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-repair
  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-resume

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E targets

  • cloud-onboard: Installer and platform changes must work on a clean supported host with the pinned runtime dependencies.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=cloud-onboard
  • onboard-repair: Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-repair
  • onboard-resume: Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-resume

Optional E2E targets

  • None.

Relevant changed files

  • src/lib/onboard.ts
  • src/lib/onboard/extra-provider-diagnostic-parser.ts
  • src/lib/onboard/extra-provider-reconciliation.ts
  • src/lib/onboard/gateway-recovery.ts
  • src/lib/onboard/machine/core-flow-phases.ts
  • src/lib/onboard/machine/handlers/sandbox-resume.ts
  • src/lib/onboard/machine/handlers/sandbox.ts
  • src/lib/onboard/types.ts
  • test/e2e/fixtures/extra-providers-registry.ts
  • test/e2e/fixtures/gateway-providers.ts
  • test/e2e/live/onboard-repair.test.ts
  • test/e2e/live/onboard-resume.test.ts
  • test/e2e/mock-parity.json
  • src/lib/onboard/extra-provider-reconciliation.test-fixtures.ts
  • src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 29045865605
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 0 passed, 0 failed, 3 cancelled, 0 skipped

Job Result
cloud-onboard ⚠️ cancelled
onboard-repair ⚠️ cancelled
onboard-resume ⚠️ cancelled

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/onboard-extra-provider-reconciliation.test.ts (1)

60-67: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Vestigial provider list mock branch.

Lines 66-67 still stub provider list -g nemoclaw --names, but the assertion at Lines 170-175 asserts this command is never invoked. This branch is now dead code in this test — harmless, but worth pruning or adding a comment explaining it's a defensive fallback in case of unexpected calls.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/onboard-extra-provider-reconciliation.test.ts` around lines 60 - 67, The
test has a dead mock branch for provider list -g nemoclaw --names that is no
longer exercised because the existing assertions in the same test expect that
command to never run. Remove the obsolete branch from the test setup, or if you
want to keep it as a safety net, add a short comment near the mocked command
handling in onboard-extra-provider-reconciliation.test.ts explaining that it is
a defensive fallback and not part of the expected flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/onboard-extra-provider-reconciliation.test.ts`:
- Around line 60-67: The test has a dead mock branch for provider list -g
nemoclaw --names that is no longer exercised because the existing assertions in
the same test expect that command to never run. Remove the obsolete branch from
the test setup, or if you want to keep it as a safety net, add a short comment
near the mocked command handling in
onboard-extra-provider-reconciliation.test.ts explaining that it is a defensive
fallback and not part of the expected flow.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b8cb7512-d4a7-42e8-9aef-2663f8fc5d53

📥 Commits

Reviewing files that changed from the base of the PR and between 614122b and b8204d2.

📒 Files selected for processing (3)
  • src/lib/onboard/extra-provider-reconciliation.test.ts
  • src/lib/onboard/extra-provider-reconciliation.ts
  • test/onboard-extra-provider-reconciliation.test.ts

Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 29046117436
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 0 passed, 0 failed, 3 cancelled, 0 skipped

Job Result
cloud-onboard ⚠️ cancelled
onboard-repair ⚠️ cancelled
onboard-resume ⚠️ cancelled

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29046258405
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: Add or justify PRA-T1 and any related test follow-ups.
Open items: 0 required · 0 warnings · 0 suggestions · 8 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Action checklist

  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Runtime validation
  • PRA-T6 Add or justify test follow-up: Acceptance clause
  • PRA-T7 Add or justify test follow-up: Acceptance clause
  • PRA-T8 Add or justify test follow-up: Acceptance clause
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — Run the `cloud-onboard` E2E job for Installer and platform changes must work on a clean supported host with the pinned runtime dependencies. Matched files: `src/lib/onboard/machine/core-flow-phases.ts`, `src/lib/onboard/machine/handlers/sandbox-resume.ts`, `src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts`, `src/lib/onboard/machine/handlers/sandbox.ts`.. Deterministic regression risks require live validation: lifecycle-state, platform-install. Checked-in unit, mocked-spawn, and live E2E definitions cover the changed lifecycle and platform-install invariants, but the deterministic risk plan lists `cloud-onboard`, `onboard-repair`, and `onboard-resume` as required runtime validation floors. This review did not execute or verify those jobs.
  • PRA-T2 Runtime validation — Run or verify exact-head `cloud-onboard` to validate clean-host installer/platform behavior reaches a usable agent with pinned runtime dependencies.. Deterministic regression risks require live validation: lifecycle-state, platform-install. Checked-in unit, mocked-spawn, and live E2E definitions cover the changed lifecycle and platform-install invariants, but the deterministic risk plan lists `cloud-onboard`, `onboard-repair`, and `onboard-resume` as required runtime validation floors. This review did not execute or verify those jobs.
  • PRA-T3 Runtime validation — Run the `onboard-repair` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-diagnostic-parser.ts`, `src/lib/onboard/extra-provider-reconciliation.test-fixtures.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/gateway-recovery.ts`.. Deterministic regression risks require live validation: lifecycle-state, platform-install. Checked-in unit, mocked-spawn, and live E2E definitions cover the changed lifecycle and platform-install invariants, but the deterministic risk plan lists `cloud-onboard`, `onboard-repair`, and `onboard-resume` as required runtime validation floors. This review did not execute or verify those jobs.
  • PRA-T4 Runtime validation — Run or verify exact-head `onboard-repair` with stale and live extra-provider seeding; confirm repair converges, stale provider attachment is absent, live provider attachment remains present, registry state is pruned as expected, and sandbox status succeeds.. Deterministic regression risks require live validation: lifecycle-state, platform-install. Checked-in unit, mocked-spawn, and live E2E definitions cover the changed lifecycle and platform-install invariants, but the deterministic risk plan lists `cloud-onboard`, `onboard-repair`, and `onboard-resume` as required runtime validation floors. This review did not execute or verify those jobs.
  • PRA-T5 Runtime validation — Run the `onboard-resume` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-diagnostic-parser.ts`, `src/lib/onboard/extra-provider-reconciliation.test-fixtures.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/gateway-recovery.ts`.. Deterministic regression risks require live validation: lifecycle-state, platform-install. Checked-in unit, mocked-spawn, and live E2E definitions cover the changed lifecycle and platform-install invariants, but the deterministic risk plan lists `cloud-onboard`, `onboard-repair`, and `onboard-resume` as required runtime validation floors. This review did not execute or verify those jobs.
  • PRA-T6 Acceptance clause — Sandbox <code>test</code> is created successfully with no web-search provider configured. — add test evidence or identify existing coverage. Behavior is covered by checked-in tests, but not with the literal sandbox name `test`: `test/onboard-extra-provider-reconciliation.test.ts` calls `createSandbox(null, "gpt-5.4")` with no web-search config and asserts the sandbox create command omits `--provider tavily-search`; live resume/repair E2E definitions use `e2e-resume` and `e2e-repair` sandboxes.
  • PRA-T7 Acceptance clause — <code>openshell sandbox list</code> shows <code>test</code> in a healthy/running state — add test evidence or identify existing coverage. Checked-in live E2E definitions assert `nemoclaw <sandbox> status` exits 0 after resume/repair and probe sandbox provider attachments through OpenShell, but the reviewed assertions do not literally run `openshell sandbox list` for sandbox name `test`.
  • PRA-T8 Acceptance clause — no <code>tavily-search</code> provider is referenced anywhere in the gateway configuration (since option 1, "No web search", was selected). — add test evidence or identify existing coverage. `test/onboard-extra-provider-reconciliation.test.ts` seeds stale `tavily-search`, returns an exact not-found diagnostic, and asserts sandbox create omits `--provider tavily-search` and registry extras no longer contain it. Live E2E definitions assert seeded stale extra providers are absent from recreated sandbox provider attachments while live providers remain present; they do not inspect every gateway configuration surface literally for `tavily-search`.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor (Nemotron Ultra) — Changes requested

Merge posture: Do not merge yet
Primary next action: Resolve or justify PRA-1: Workaround limitation not surfaced in user-facing documentation or --fresh behavior.
Open items: 0 required · 2 warnings · 0 suggestions · 6 test follow-ups
Since last review: 2 prior items resolved · 2 still apply · 2 new items found

Action checklist

  • PRA-1 Resolve or justify: Workaround limitation not surfaced in user-facing documentation or --fresh behavior in src/lib/onboard/extra-provider-reconciliation.ts:154
  • PRA-2 Resolve or justify: Missing retry stability test for prepareSandboxCreatePlan provider list idempotency in src/lib/onboard/sandbox-create-plan.test.ts:654
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Missing retry stability test for prepareSandboxCreatePlan provider list idempotency
  • PRA-T5 Add or justify test follow-up: Acceptance clause
  • PRA-T6 Add or justify test follow-up: extra-provider-reconciliation.ts create-time filter workaround

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify acceptance src/lib/onboard/extra-provider-reconciliation.ts:154 Either: (a) make --fresh purge extraProviders from the registry, or (b) add a CLI warning/log when reconciliation omits providers due to gateway not-found diagnostics, or (c) document the limitation in onboard --help or docs. Prefer (a) if registry cleanup is straightforward.
PRA-2 Resolve/justify tests src/lib/onboard/sandbox-create-plan.test.ts:654 Add a unit test in sandbox-create-plan.test.ts that calls prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers like 'tavily-search') and asserts identical createArgs.extraProviders both times. This validates invariant 1 without requiring full E2E.
Review findings by urgency: 0 required fixes, 2 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Workaround limitation not surfaced in user-facing documentation or --fresh behavior

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:154
  • Category: acceptance
  • Problem: The extra-provider-reconciliation.ts JSDoc documents the removal condition: 'delete this defensive prune once OpenShell/NemoClaw gateway reset owns extra-provider lifecycle cleanup before sandbox creation ([NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501).' However, the --fresh flag does not purge extraProviders from the local registry, and no user-facing documentation or CLI help indicates that stale extra providers persist in the registry and may affect other flows. The workaround is effective for sandbox create but incomplete for full cleanup.
  • Impact: Users running --fresh onboarding may believe all stale state is cleared, but extraProviders remain in the registry. Other callers of listExtraProviders (if any) could be affected by stale entries. Operators have no visibility into the workaround's scope.
  • Recommended action: Either: (a) make --fresh purge extraProviders from the registry, or (b) add a CLI warning/log when reconciliation omits providers due to gateway not-found diagnostics, or (c) document the limitation in onboard --help or docs. Prefer (a) if registry cleanup is straightforward.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Missing regression test: Test that --fresh onboarding with prior stale extraProviders succeeds and registry is cleared (or document why it intentionally isn't).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Evidence: extra-provider-reconciliation.ts JSDoc line 15-17: 'Removal condition: delete this defensive prune once OpenShell/NemoClaw gateway reset owns extra-provider lifecycle cleanup' Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501: 'reused gateway has stale state from a prior onboarding attempt' Prior review PRA-2 and PRA-T7 flagged this workaround limitation

PRA-2 Resolve/justify — Missing retry stability test for prepareSandboxCreatePlan provider list idempotency

  • Location: src/lib/onboard/sandbox-create-plan.test.ts:654
  • Category: tests
  • Problem: The lifecycle-state risk plan invariant 1 requires 'partial failure and retry converge without ghost resources or stale ports.' prepareSandboxCreatePlan calls reconcileRegisteredExtraProviders internally. On retry/resume, the gateway provider list must return the same filtered result for convergence. No existing test calls prepareSandboxCreatePlan twice with identical inputs (including stale managed providers) and asserts identical createArgs.extraProviders. The new sandbox-recreate-resume.test.ts tests handleSandboxState with explicit recreate but does not test prepareSandboxCreatePlan idempotency directly.
  • Impact: If reconciliation behaves non-deterministically across retries (e.g., gateway list changes between calls, or probe ordering varies), onboarding resume could produce different sandbox create args, violating lifecycle-state invariant 1 and potentially causing ghost resources or stale port conflicts.
  • Recommended action: Add a unit test in sandbox-create-plan.test.ts that calls prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers like 'tavily-search') and asserts identical createArgs.extraProviders both times. This validates invariant 1 without requiring full E2E.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read sandbox-create-plan.test.ts and confirm no test calls prepareSandboxCreatePlan twice with same inputs and asserts createArgs.extraProviders stability.
  • Missing regression test: Add test: prepareSandboxCreatePlan(webSearchConfig, extraProvidersWithStale) called twice, assert createArgs.extraProviders identical both times.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read sandbox-create-plan.test.ts and confirm no test calls prepareSandboxCreatePlan twice with same inputs and asserts createArgs.extraProviders stability.
  • Evidence: src/lib/onboard/sandbox-create-plan.ts calls reconcileRegisteredExtraProviders at line 2746 (onboard.ts:2746 in diff) src/lib/onboard/sandbox-create-plan.test.ts has 654 lines but no retry idempotency test riskPlan lifecycle-state invariant 1: 'partial failure and retry converge without ghost resources or stale ports' Prior review PRA-5 flagged this exact gap

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — Run the `cloud-onboard` E2E job for Installer and platform changes must work on a clean supported host with the pinned runtime dependencies. Matched files: `src/lib/onboard/machine/core-flow-phases.ts`, `src/lib/onboard/machine/handlers/sandbox-resume.ts`, `src/lib/onboard/machine/handlers/sandbox-test-fixtures.ts`, `src/lib/onboard/machine/handlers/sandbox.ts`.. Deterministic regression risks require live validation: lifecycle-state, platform-install.
  • PRA-T2 Runtime validation — Run the `onboard-repair` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-diagnostic-parser.ts`, `src/lib/onboard/extra-provider-reconciliation.test-fixtures.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/gateway-recovery.ts`.. Deterministic regression risks require live validation: lifecycle-state, platform-install.
  • PRA-T3 Runtime validation — Run the `onboard-resume` E2E job for Onboarding and sandbox state must converge across persisted metadata, reported status, and the live runtime. Matched files: `src/lib/onboard.ts`, `src/lib/onboard/extra-provider-diagnostic-parser.ts`, `src/lib/onboard/extra-provider-reconciliation.test-fixtures.ts`, `src/lib/onboard/extra-provider-reconciliation.ts`, `src/lib/onboard/gateway-recovery.ts`.. Deterministic regression risks require live validation: lifecycle-state, platform-install.
  • PRA-T4 Missing retry stability test for prepareSandboxCreatePlan provider list idempotency — Add a unit test in sandbox-create-plan.test.ts that calls prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers like 'tavily-search') and asserts identical createArgs.extraProviders both times. This validates invariant 1 without requiring full E2E.
  • PRA-T5 Acceptance clause — Workaround limitation surfaced to users (--fresh purges or documents stale extraProviders) — add test evidence or identify existing coverage. Documented in extra-provider-reconciliation.ts JSDoc with removal condition, but --fresh does not purge extraProviders and no user-facing documentation/CLI warning exists; maps to F-001
  • PRA-T6 extra-provider-reconciliation.ts create-time filter workaround — Test that --fresh onboarding with prior stale extraProviders succeeds and registry is cleared (or document why it intentionally isn't). extra-provider-reconciliation.ts JSDoc line 15-17 documents removal condition; Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501 root cause is reused gateway stale state
Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Workaround limitation not surfaced in user-facing documentation or --fresh behavior

  • Location: src/lib/onboard/extra-provider-reconciliation.ts:154
  • Category: acceptance
  • Problem: The extra-provider-reconciliation.ts JSDoc documents the removal condition: 'delete this defensive prune once OpenShell/NemoClaw gateway reset owns extra-provider lifecycle cleanup before sandbox creation ([NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501).' However, the --fresh flag does not purge extraProviders from the local registry, and no user-facing documentation or CLI help indicates that stale extra providers persist in the registry and may affect other flows. The workaround is effective for sandbox create but incomplete for full cleanup.
  • Impact: Users running --fresh onboarding may believe all stale state is cleared, but extraProviders remain in the registry. Other callers of listExtraProviders (if any) could be affected by stale entries. Operators have no visibility into the workaround's scope.
  • Recommended action: Either: (a) make --fresh purge extraProviders from the registry, or (b) add a CLI warning/log when reconciliation omits providers due to gateway not-found diagnostics, or (c) document the limitation in onboard --help or docs. Prefer (a) if registry cleanup is straightforward.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Missing regression test: Test that --fresh onboarding with prior stale extraProviders succeeds and registry is cleared (or document why it intentionally isn't).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check registry.ts for listExtraProviders implementation and whether --fresh clears extraProviders. Verify no other callers of listExtraProviders are affected by stale entries.
  • Evidence: extra-provider-reconciliation.ts JSDoc line 15-17: 'Removal condition: delete this defensive prune once OpenShell/NemoClaw gateway reset owns extra-provider lifecycle cleanup' Issue [NemoClaw][onboard][All Platforms] nemoclaw onboard fails creating sandbox when web search is disabled — tries to configure nonexistent tavily-search provider #6501: 'reused gateway has stale state from a prior onboarding attempt' Prior review PRA-2 and PRA-T7 flagged this workaround limitation

PRA-2 Resolve/justify — Missing retry stability test for prepareSandboxCreatePlan provider list idempotency

  • Location: src/lib/onboard/sandbox-create-plan.test.ts:654
  • Category: tests
  • Problem: The lifecycle-state risk plan invariant 1 requires 'partial failure and retry converge without ghost resources or stale ports.' prepareSandboxCreatePlan calls reconcileRegisteredExtraProviders internally. On retry/resume, the gateway provider list must return the same filtered result for convergence. No existing test calls prepareSandboxCreatePlan twice with identical inputs (including stale managed providers) and asserts identical createArgs.extraProviders. The new sandbox-recreate-resume.test.ts tests handleSandboxState with explicit recreate but does not test prepareSandboxCreatePlan idempotency directly.
  • Impact: If reconciliation behaves non-deterministically across retries (e.g., gateway list changes between calls, or probe ordering varies), onboarding resume could produce different sandbox create args, violating lifecycle-state invariant 1 and potentially causing ghost resources or stale port conflicts.
  • Recommended action: Add a unit test in sandbox-create-plan.test.ts that calls prepareSandboxCreatePlan twice with identical webSearchConfig and extraProviders (including stale managed providers like 'tavily-search') and asserts identical createArgs.extraProviders both times. This validates invariant 1 without requiring full E2E.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read sandbox-create-plan.test.ts and confirm no test calls prepareSandboxCreatePlan twice with same inputs and asserts createArgs.extraProviders stability.
  • Missing regression test: Add test: prepareSandboxCreatePlan(webSearchConfig, extraProvidersWithStale) called twice, assert createArgs.extraProviders identical both times.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read sandbox-create-plan.test.ts and confirm no test calls prepareSandboxCreatePlan twice with same inputs and asserts createArgs.extraProviders stability.
  • Evidence: src/lib/onboard/sandbox-create-plan.ts calls reconcileRegisteredExtraProviders at line 2746 (onboard.ts:2746 in diff) src/lib/onboard/sandbox-create-plan.test.ts has 654 lines but no retry idempotency test riskPlan lifecycle-state invariant 1: 'partial failure and retry converge without ghost resources or stale ports' Prior review PRA-5 flagged this exact gap

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

apurvvkumaria and others added 2 commits July 9, 2026 13:23
Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29047798544
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@apurvvkumaria
apurvvkumaria requested a review from jyaunches July 9, 2026 20:45
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29048610487
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria

Copy link
Copy Markdown
Collaborator Author

Advisor follow-up for cb7977f289d0487d1d15357b6c475581ff6c1160:

  • Nemotron PRA-10 addressed: indeterminate provider probes now emit one aggregate operational warning. It contains only a sorted reason-class set and affected provider count; gateway names, provider names, raw diagnostics, and exception text are never emitted. Throw, process error, timeout/signal, unexpected exit, ambiguous diagnostic, capture limit, and aggregate-budget exhaustion are all classified. There is no onboarding metrics/counter substrate to extend without introducing new infrastructure, so this focused change follows the existing injectable warn convention instead.
  • Focused validation: CLI unit 15/15, realistic repeated-create integration 1/1, CLI type-check, commit hooks, and push hooks passed. Boundary coverage now proves fail-open at exactly 64 KiB, exact parsing immediately below the limit, bounded adversarial input, one warning across every indeterminate class, and no gateway/provider-name disclosure.
  • Nemotron PRA-3 evidence: run 29048610487 passed onboard-repair, onboard-resume, and cloud-onboard at the immediately preceding behavior head a0ea6e60f98c6e15a6084dba7a5ba9ff1e89215a. The new commit changes only warning visibility and tests, not provider filtering or sandbox-create arguments. For exact-current-head confirmation, the same three targets are running in run 29049630922.
  • GPT PRA-1 rationale: the dedicated mutable live stale-provider fixture is deferred. Checked-in coverage precisely seeds stale tavily-search, preserves multiple live custom providers and registry state, executes two create attempts, and asserts identical filtered attachment arguments; the lifecycle floor then exercises real repair, resume, and cloud onboarding. Adding a new state-seeding live fixture would expand shared E2E infrastructure beyond this narrow reconciliation fix. This is an explicit acceptance of that residual test-fixture gap, not a claim that the generic lifecycle run itself seeds stale provider state.

The commit is signed, GitHub-verified, and retains Shawn Xie as co-author.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 29049630922
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: onboard-repair,onboard-resume,cloud-onboard
Summary: 0 passed, 0 failed, 3 cancelled, 0 skipped

Job Result
cloud-onboard ⚠️ cancelled
onboard-repair ⚠️ cancelled
onboard-resume ⚠️ cancelled

@cv cv added v0.0.80 and removed v0.0.79 labels Jul 9, 2026
@cv

cv commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Exact-head maintainer sweep (cb7977f): keeping this held for the two substantive gaps already identified by the advisors: split/table-drive the 94→412-line reconciliation test body, and add checked-in stale-plus-live extra-provider resume/repair convergence coverage. The required exact-head live run was cancelled; the prior green run was on ancestor a0ea6e6, so please rerun onboard-repair, onboard-resume, and cloud-onboard after the fixes, together with both advisors and CodeRabbit.

One disposition to avoid redundant work: Nemotron PRA-3 (missing retry idempotency) appears already covered by test/onboard-extra-provider-reconciliation.test.ts, which performs two create attempts and asserts the filtered provider arguments remain identical. The structure and lifecycle-coverage findings remain blocking.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ❌ Some jobs failed

Run: 29114463308
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 0 passed, 1 failed, 2 cancelled, 0 skipped

Job Result
cloud-onboard ⚠️ cancelled
onboard-repair ⚠️ cancelled
onboard-resume ❌ failure

Failed jobs: onboard-resume. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ❌ Some jobs failed

Run: 29115029731
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 1 passed, 2 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ❌ failure
onboard-resume ❌ failure

Failed jobs: onboard-repair, onboard-resume. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ⚠️ Run cancelled — no signal

Run: 29115764584
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: (selector rejected by workflow validation)
Summary: 0 passed, 0 failed, 75 cancelled, 0 skipped

Job Result
agent-turn-latency ⚠️ cancelled
bedrock-runtime-compatible-anthropic ⚠️ cancelled
brave-search ⚠️ cancelled
channels-add-remove ⚠️ cancelled
channels-stop-start ⚠️ cancelled
cloud-inference ⚠️ cancelled
cloud-onboard ⚠️ cancelled
common-egress-agent ⚠️ cancelled
concurrent-gateway-ports ⚠️ cancelled
credential-migration ⚠️ cancelled
credential-sanitization ⚠️ cancelled
cron-preflight-inference-local ⚠️ cancelled
device-auth-health ⚠️ cancelled
diagnostics ⚠️ cancelled
docs-validation ⚠️ cancelled
double-onboard ⚠️ cancelled
full-e2e ⚠️ cancelled
gateway-drift-preflight ⚠️ cancelled
gateway-guard-recovery ⚠️ cancelled
gateway-health-honest ⚠️ cancelled
gpu-double-onboard ⚠️ cancelled
gpu-e2e ⚠️ cancelled
hermes-dashboard ⚠️ cancelled
hermes-discord ⚠️ cancelled
hermes-e2e ⚠️ cancelled
hermes-gpu-startup ⚠️ cancelled
hermes-inference-switch ⚠️ cancelled
hermes-shields-config ⚠️ cancelled
hermes-slack ⚠️ cancelled
inference-routing ⚠️ cancelled
issue-2478-crash-loop-recovery ⚠️ cancelled
issue-4434-tui-unreachable-inference ⚠️ cancelled
issue-4462-scope-upgrade-approval ⚠️ cancelled
jetson-nvmap-gpu ⚠️ cancelled
kimi-inference-compat ⚠️ cancelled
launchable-smoke ⚠️ cancelled
live ⚠️ cancelled
mcp-bridge ⚠️ cancelled
mcp-bridge-dev ⚠️ cancelled
messaging-compatible-endpoint ⚠️ cancelled
messaging-providers ⚠️ cancelled
model-router-provider-routed-inference ⚠️ cancelled
network-policy ⚠️ cancelled
ollama-auth-proxy ⚠️ cancelled
onboard-negative-paths ⚠️ cancelled
onboard-repair ⚠️ cancelled
onboard-resume ⚠️ cancelled
openclaw-discord-pairing ⚠️ cancelled
openclaw-inference-switch ⚠️ cancelled
openclaw-plugin-runtime-exdev ⚠️ cancelled
openclaw-skill-cli ⚠️ cancelled
openclaw-slack-pairing ⚠️ cancelled
openclaw-tui-chat-correlation ⚠️ cancelled
openshell-gateway-auth-contract ⚠️ cancelled
openshell-gateway-upgrade ⚠️ cancelled
openshell-version-pin ⚠️ cancelled
overlayfs-autofix ⚠️ cancelled
rebuild-hermes ⚠️ cancelled
rebuild-hermes-stale-base ⚠️ cancelled
rebuild-openclaw ⚠️ cancelled
sandbox-operations ⚠️ cancelled
sandbox-rebuild ⚠️ cancelled
sandbox-rlimits-connect ⚠️ cancelled
sandbox-survival ⚠️ cancelled
security-posture ⚠️ cancelled
sessions-agents-cli ⚠️ cancelled
shields-config ⚠️ cancelled
skill-agent ⚠️ cancelled
snapshot-commands ⚠️ cancelled
spark-install ⚠️ cancelled
state-backup-restore ⚠️ cancelled
telegram-injection ⚠️ cancelled
token-rotation ⚠️ cancelled
tunnel-lifecycle ⚠️ cancelled
upgrade-stale-sandbox ⚠️ cancelled

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ❌ Some jobs failed

Run: 29115969006
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 1 passed, 2 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ❌ failure
onboard-resume ❌ failure

Failed jobs: onboard-repair, onboard-resume. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29116257938
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29116735646
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29117568248
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29118167293
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29119108431
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All requested jobs passed

Run: 29119308633
Workflow ref: codex/6501-strict-extra-provider-reconciliation
Requested targets: (default — all supported)
Requested jobs: cloud-onboard,onboard-repair,onboard-resume
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
cloud-onboard ✅ success
onboard-repair ✅ success
onboard-resume ✅ success

@cjagwani

Copy link
Copy Markdown
Collaborator

Exact-head follow-up for bc4a19498ba980d713c89e7a29139ddc3bae5cfb:

  • All 44 ordinary checks are green, all 35 commits are Verified, CodeRabbit has no unresolved threads, and exact-head live runs 29119108431 and 29119308633 passed cloud-onboard, onboard-repair, and onboard-resume.
  • The two Nemotron warnings do not reproduce on this head: confirmed-missing providers are removed by reconcileRegisteredExtraProviders(), and retry idempotency is covered by both the plan test and the production create-path integration test.
  • The old review at 5bc41db10 appears addressed by the later exact wrapped-diagnostic, repair/resume, retry, and live coverage.

One audit-record blocker remains: the PR description still says reconciliation “never mutate[s] the local registry,” is intentionally non-destructive, and that --fresh does not purge an entry. The current implementation and tests now remove a confirmed-missing provider from the registry. @apurvvkumaria, please update the summary/changes/verification and exact-head live-run link so the description matches the shipped behavior. @cv, please re-review the current head after that correction.

The branch is also two commits behind main (no conflict; merge-ref CI is green). This is not yet an admin-merge request while the human change request and description mismatch remain.

@cv
cv merged commit 7c955d4 into main Jul 10, 2026
200 checks passed
@cv
cv deleted the codex/6501-strict-extra-provider-reconciliation branch July 10, 2026 21:37
cv pushed a commit that referenced this pull request Jul 11, 2026
## Summary

Release-prep documentation for **v0.0.80**. Adds the `## v0.0.80`
section to `docs/about/release-notes.mdx` summarizing user-facing
changes since v0.0.79, each bullet linking to the relevant deeper page.

Produced via `nemoclaw-contributor-update-docs` (pre-tag path): scanned
`v0.0.79..HEAD`, applied the docs skip list (no violations), and
confirmed the 8 commits that already shipped in-PR docs are complete. No
new pages needed.

## Source summary

- #6507 -> `docs/about/release-notes.mdx`: Hermes v0.18 + Slack Block
Kit (rich rendering, digest-pinned base image).
- #6584 / #6616 -> `docs/about/release-notes.mdx`: host-local OpenRouter
runtime attribution adapter (port `11437`,
`NEMOCLAW_OPENROUTER_RUNTIME_ADAPTER_PORT`) and native Deep Agents
`openrouter` provider.
- #6210 / #6292 -> `docs/about/release-notes.mdx`: host corporate proxy
CA import into sandbox trust (`NEMOCLAW_CORPORATE_CA_BUNDLE`,
`NEMOCLAW_CORPORATE_CA_IMPORT`).
- #6624 / #6623 / #6656 -> `docs/about/release-notes.mdx`:
release-matched base-image selection, surfaced cluster-image build
diagnostics, preserved Nemotron profile registration.
- #6629 / #6637 -> `docs/about/release-notes.mdx`: bare `connect`
default-sandbox behavior and route-probe hardening.
- #6634 / #6626 / #6596 / #5569 / #6610 / #6655 ->
`docs/about/release-notes.mdx`: onboarding/recovery preservation,
stale-gateway-PID fix, installer backup message, vLLM label on managed
platforms.
- #6578 / #5670 -> `docs/about/release-notes.mdx`: automatic Hermes
light terminal skin and non-interactive `npx` MCP server startup.

## Verification

`npm run docs`: 0 errors, all internal links resolve (2 pre-existing
hidden-page warnings). `_build/` variants for OpenClaw, Hermes, and Deep
Agents all regenerate with the v0.0.80 section.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
  * Added release notes for v0.0.80.
  * Documented Hermes upgrades, including Slack Block Kit rendering.
  * Added details on OpenRouter traffic routing and attribution headers.
* Documented improved proxy certificate handling and sandbox
reliability.
* Highlighted enhanced connection defaults, route-probing safeguards,
onboarding recovery, and terminal/MCP startup behavior.
  * Added references to relevant user-guide documentation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Fixes the post-merge NVIDIA#6587 gap where one paginated provider-list page
could omit healthy registry extras beyond entry 100. Reconciliation now
uses bounded, gateway-scoped per-provider probes, recognizes the exact
wrapped OpenShell NVIDIA#6501 diagnostic, and preserves every indeterminate
result.

## Related Issue
Follow-up to NVIDIA#6501 and NVIDIA#6587. Supersedes NVIDIA#6518.

## Changes
- Probe every recorded extra with argv-safe `provider get -g <gateway>
<name>` instead of treating one list page as authoritative.
- Omit only exit-1 diagnostics that bind the exact, case-sensitive
quoted provider name to a tightly anchored not-found shape, including
the wrapped not-found-and-unrecognized form reported in NVIDIA#6501.
- Fail open on gateway, transport, authentication, timeout, signal,
ambiguous, name-mismatched, or command-spoofed output.
- Read canonical stderr/stdout without duplicating Node's composite
`output` array, and treat process errors or capture-limit diagnostics as
indeterminate.
- Bound each probe to 5 seconds and 64 KiB, cap total reconciliation
time at 15 seconds, and never mutate the local registry.
- Cover stale providers after index 100, the literal wrapped issue
diagnostic in a realistic `spawnSync` result, capture truncation,
mixed-case names, multi-line diagnostic spoofing, gateway failures,
aggregate timeout, and stable repeated sandbox-create provider
arguments.

The create-time filter is intentionally non-destructive. A provider
omitted from one sandbox create remains in the user-owned registry for a
later retry; `--fresh` does not purge it. Indeterminate probes preserve
the attachment, and the final `sandbox create` remains authoritative and
reports any concrete attachment failure. This avoids deleting healthy
user configuration because of a transient gateway or transport failure.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: internal provider-state
reconciliation only; no CLI, prompt, configuration, policy, or required
user action changes.
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: independent
nine-category security review passed after exact-name, whole-diagnostic,
subprocess-bound, and fail-open checks.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: CLI
unit 14/14 and spawn integration 1/1 passed; the spawn test uses a
realistic composite process result, executes two create attempts, and
proves identical filtered provider arguments.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: not applicable to this
focused reconciliation change; `npm run typecheck:cli`, `npm run
check:diff`, source-shape, test-size, and test-conditional scans passed.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Exact-head live validation for `onboard-repair`, `onboard-resume`, and
`cloud-onboard` passed at [workflow run
29048610487](https://github.com/NVIDIA/NemoClaw/actions/runs/29048610487)
for head `a0ea6e60f98c6e15a6084dba7a5ba9ff1e89215a`.

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Onboarding and repair now detect and remove stale extra-provider
records.
* Resume operations can explicitly recreate an existing sandbox while
preserving its registry entry.
* Sandbox creation and retries now use a consistent, deduplicated
provider list.

* **Bug Fixes**
* Providers are retained when availability checks are inconclusive,
preventing accidental removal.
* Reconciliation now avoids relying on potentially stale provider-list
snapshots and handles diagnostic variations more reliably.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: Shawn Xie <shaxie@nvidia.com>
Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
## Summary

Release-prep documentation for **v0.0.80**. Adds the `## v0.0.80`
section to `docs/about/release-notes.mdx` summarizing user-facing
changes since v0.0.79, each bullet linking to the relevant deeper page.

Produced via `nemoclaw-contributor-update-docs` (pre-tag path): scanned
`v0.0.79..HEAD`, applied the docs skip list (no violations), and
confirmed the 8 commits that already shipped in-PR docs are complete. No
new pages needed.

## Source summary

- NVIDIA#6507 -> `docs/about/release-notes.mdx`: Hermes v0.18 + Slack Block
Kit (rich rendering, digest-pinned base image).
- NVIDIA#6584 / NVIDIA#6616 -> `docs/about/release-notes.mdx`: host-local OpenRouter
runtime attribution adapter (port `11437`,
`NEMOCLAW_OPENROUTER_RUNTIME_ADAPTER_PORT`) and native Deep Agents
`openrouter` provider.
- NVIDIA#6210 / NVIDIA#6292 -> `docs/about/release-notes.mdx`: host corporate proxy
CA import into sandbox trust (`NEMOCLAW_CORPORATE_CA_BUNDLE`,
`NEMOCLAW_CORPORATE_CA_IMPORT`).
- NVIDIA#6624 / NVIDIA#6623 / NVIDIA#6656 -> `docs/about/release-notes.mdx`:
release-matched base-image selection, surfaced cluster-image build
diagnostics, preserved Nemotron profile registration.
- NVIDIA#6629 / NVIDIA#6637 -> `docs/about/release-notes.mdx`: bare `connect`
default-sandbox behavior and route-probe hardening.
- NVIDIA#6634 / NVIDIA#6626 / NVIDIA#6596 / NVIDIA#5569 / NVIDIA#6610 / NVIDIA#6655 ->
`docs/about/release-notes.mdx`: onboarding/recovery preservation,
stale-gateway-PID fix, installer backup message, vLLM label on managed
platforms.
- NVIDIA#6578 / NVIDIA#5670 -> `docs/about/release-notes.mdx`: automatic Hermes
light terminal skin and non-interactive `npx` MCP server startup.

## Verification

`npm run docs`: 0 errors, all internal links resolve (2 pre-existing
hidden-page warnings). `_build/` variants for OpenClaw, Hermes, and Deep
Agents all regenerate with the v0.0.80 section.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
  * Added release notes for v0.0.80.
  * Documented Hermes upgrades, including Slack Block Kit rendering.
  * Added details on OpenRouter traffic routing and attribution headers.
* Documented improved proxy certificate handling and sandbox
reliability.
* Highlighted enhanced connection defaults, route-probing safeguards,
onboarding recovery, and terminal/MCP startup behavior.
  * Added references to relevant user-guide documentation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants