Skip to content

fix(ci): route advisors to inference API - #5384

Merged
cv merged 1 commit into
mainfrom
codex/fix-pr-advisor-config
Jun 13, 2026
Merged

fix(ci): route advisors to inference API#5384
cv merged 1 commit into
mainfrom
codex/fix-pr-advisor-config

Conversation

@cv

@cv cv commented Jun 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Route the PR and E2E advisor SDK configuration to the hosted OpenAI-compatible inference-api.nvidia.com service so the injected advisor secrets are used against the intended endpoint. The advisor wrappers now surface provider errors in artifacts/comments instead of collapsing empty SDK output into a JSON parse failure.

Changes

  • Point the shared advisor provider config at https://inference-api.nvidia.com/v1 with conservative OpenAI-compatible request flags.
  • Remove stale PI_PR_REVIEW_ADVISOR_API_KEY and OPENAI_API_KEY advisor fallbacks from workflows and docs.
  • Capture assistant provider errors from the Pi SDK session stream and report them through PR/E2E advisor failure artifacts.
  • Extend PR advisor workflow-boundary coverage for the single-secret contract and update advisor README guidance.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved error detection and reporting in advisor tools; errors occurring during advisor turns are now properly captured and reported.
  • Documentation

    • Updated setup instructions to reflect current credential configuration requirements for advisor tools.
    • Clarified advisor model and endpoint configuration details.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this Jun 13, 2026
@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR consolidates advisor credential handling by migrating from optional credential fallbacks to explicit environment variable sources, implementing per-turn error tracking in the advisor session, and enforcing credential boundaries in CI workflows. The session module now captures assistant errors, workflow steps use dedicated API key secrets, and analyzers validate error state before proceeding.

Changes

Advisor credential consolidation and error tracking

Layer / File(s) Summary
Session module error tracking and configuration
tools/advisors/session.mts
Exports ADVISOR_OPENAI_COMPATIBLE_BASE_URL constant and extends RunAdvisorResult with turnErrors: string[]. Updates advisorModel to accept and return optional compat configuration. Implements per-turn error capture during streaming and event handling by normalizing assistant errors. Removes fallback to OPENAI_API_KEY and loads credential via process.env[credentialEnv]?.trim().
Analyzer error validation and failure handling
tools/e2e-advisor/analyze.mts, tools/e2e-advisor/scenarios.mts, tools/pr-review-advisor/analyze.mts
Both e2e-advisor and pr-review-advisor analyzers check sdkResult.turnErrors after advisor completion and exit with status 1, writing failure artifacts if any errors are present.
Workflow environment credential updates
.github/workflows/e2e-advisor.yaml, .github/workflows/pr-review-advisor.yaml
E2E advisor and PR review advisor workflow steps now use E2E_ADVISOR_API_KEY and PR_REVIEW_ADVISOR_API_KEY respectively, sourced from dedicated secrets without fallback chains.
Workflow boundary credential enforcement
tools/pr-review-advisor/workflow-boundary.mts, test/pr-review-advisor.test.ts
Workflow boundary validator enforces PR_REVIEW_ADVISOR_API_KEY originates only from secrets.PR_REVIEW_ADVISOR_API_KEY and forbids OPENAI_API_KEY in the step environment. Test fixture and assertions validate the tighter requirements.
Test coverage and documentation updates
test/pr-review-advisor.test.ts, tools/e2e-advisor/README.md, tools/pr-review-advisor/README.md
Test suite imports advisor configuration helpers and adds validation for OpenAI-compatible configuration wiring. E2E and PR review advisor READMEs document fixed model/endpoint usage (openai/openai/gpt-5.5 via NVIDIA inference API) and credential setup without mentioning OPENAI_API_KEY fallbacks.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested labels

area: ci

Suggested reviewers

  • prekshivyas

🐰 A rabbit's ode to credentials clear,
No fallbacks left, the path is here,
Each turn's error caught and tracked with care,
Workflows now enforce what's fair!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(ci): route advisors to inference API' directly summarizes the main change: reconfiguring advisors to use the hosted inference API endpoint instead of fallbacks.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-pr-advisor-config

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Failed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-advisor-raw-output.txt

@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: None
Optional Vitest E2E scenarios: None

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Failed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-scenario-advisor-raw-output.txt

@github-actions

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 1 worth checking, 0 nice ideas
Top item: PR review advisor unavailable

Review findings

🛠️ Needs attention

  • None.

🔎 Worth checking

  • PR review advisor unavailable: The automated advisor could not complete: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt
    • Recommendation: Re-run the PR Review Advisor or perform a manual review.
    • Evidence: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt

🌱 Nice ideas

  • None.
Consider writing more tests for
  • **Mocked behavioral coverage** — Add or confirm behavioral tests with mocked filesystem/network/process boundaries.. Changed code has I/O, state, credentials, provider, or config behavior that should be covered with behavioral mocks: tools/advisors/session.mts.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@github-code-quality

github-code-quality Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the codex/fix-pr-advisor... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main codex/fix-pr-advisor... dc6df54 +/-
nemoclaw/src/se...cret-scanner.ts 100%
nemoclaw/src/commands/slash.ts 100%
nemoclaw/src/li...bprocess-env.ts 100%
nemoclaw/src/bl...eprint/state.ts 98%
nemoclaw/src/onboard/config.ts 98%
nemoclaw/src/bl...int/snapshot.ts 97%
nemoclaw/src/bl...print/runner.ts 95%
nemoclaw/src/co...ration-state.ts 94%
nemoclaw/src/bl...ate-networks.ts 94%
nemoclaw/src/index.ts 94%

TypeScript / code-coverage/cli

The overall coverage in the codex/fix-pr-advisor... branch is 44%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main codex/fix-pr-advisor... dc6df54 +/-
src/lib/state/o...oard-session.ts 90%
src/lib/inference/local.ts 77%
src/lib/sandbox/config.ts 72%
src/lib/inference/nim.ts 72%
src/lib/onboard/preflight.ts 64%
src/lib/state/sandbox.ts 55%
src/lib/onboard...er-gpu-patch.ts 50%
src/lib/actions...licy-channel.ts 49%
src/lib/policy/index.ts 48%
src/lib/onboard.ts 17%

Updated June 13, 2026 09:32 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tools/advisors/session.mts (1)

143-153: 💤 Low value

Consider guarding against duplicate error capture within the same turn.

The currentTurnError ||= normalized pattern captures only the first error per turn, which is reasonable. However, subsequent errors for the same turn are still logged to raw but silently discarded from turnErrors. If multiple distinct errors occur in a turn, only the first is surfaced to callers.

This is likely intentional (first error is usually the root cause), but worth confirming the design intent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/advisors/session.mts` around lines 143 - 153, The current
captureTurnError function sets currentTurnError only once but continues to
append every subsequent error to raw and never records them in turnErrors; to
guard against duplicate capture, change captureTurnError (the function using
currentTurnError, currentTurnText, turnErrors, raw and options.logPrefix) so
that if currentTurnError is already set it returns immediately (or at least
skips raw.append and any push into turnErrors), and only when currentTurnError
is undefined should it set currentTurnError ||= normalized, append the formatted
message to raw and add the normalized error to turnErrors; this ensures only the
first distinct error per turn is logged/recorded while preventing duplicate
captures for the same turn.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tools/advisors/session.mts`:
- Around line 143-153: The current captureTurnError function sets
currentTurnError only once but continues to append every subsequent error to raw
and never records them in turnErrors; to guard against duplicate capture, change
captureTurnError (the function using currentTurnError, currentTurnText,
turnErrors, raw and options.logPrefix) so that if currentTurnError is already
set it returns immediately (or at least skips raw.append and any push into
turnErrors), and only when currentTurnError is undefined should it set
currentTurnError ||= normalized, append the formatted message to raw and add the
normalized error to turnErrors; this ensures only the first distinct error per
turn is logged/recorded while preventing duplicate captures for the same turn.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 387c9d76-c34c-41b1-9624-50be428d0385

📥 Commits

Reviewing files that changed from the base of the PR and between 52323e5 and dc6df54.

📒 Files selected for processing (10)
  • .github/workflows/e2e-advisor.yaml
  • .github/workflows/pr-review-advisor.yaml
  • test/pr-review-advisor.test.ts
  • tools/advisors/session.mts
  • tools/e2e-advisor/README.md
  • tools/e2e-advisor/analyze.mts
  • tools/e2e-advisor/scenarios.mts
  • tools/pr-review-advisor/README.md
  • tools/pr-review-advisor/analyze.mts
  • tools/pr-review-advisor/workflow-boundary.mts
💤 Files with no reviewable changes (1)
  • .github/workflows/e2e-advisor.yaml

@cv
cv merged commit 39fd60a into main Jun 13, 2026
47 checks passed
@cv
cv deleted the codex/fix-pr-advisor-config branch June 13, 2026 09:37
@cv cv added the v0.0.65 label Jun 13, 2026
@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: providers Inference provider integrations and provider behavior bug-fix PR fixes a bug or regression labels Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: providers Inference provider integrations and provider behavior bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants