fix(advisor): make ledger commits atomic - #6566
Merged
Merged
Conversation
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
📝 WalkthroughWalkthroughThis PR adds atomic terminal-tool turn handling and repair flow for advisor sessions, and updates PR review advisor schema, prompting, normalization, and tests to require ChangesAdvisor atomic turn protocol and session integration
Ledger findingId consistency and stage-turn restructuring
Estimated code review effort: 4 (Complex) | ~75 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
E2E Advisor RecommendationRequired E2E: None Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
|
Contributor
E2E Target RecommendationRequired E2E targets: None Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage remains at 96%, unchanged from the TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most impacted files.
Updated |
Contributor
PR Review Advisor (Nemotron Ultra) — InformationalMerge posture: Informational / low confidence Action checklist
Findings index
Review findings by urgency: 0 required fixes, 1 item to resolve/justify, 0 in-scope improvements
|
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
PR Review Advisor — InformationalMerge posture: Informational / low confidence Action checklist
Findings index
Review findings by urgency: 0 required fixes, 1 item to resolve/justify, 0 in-scope improvements
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tools/pr-review-advisor/analyze.mts (1)
673-714: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider extracting the sourceOfTruthReview validation loop.
The added per-item validation block (lines 697-712) pushes
reviewLedgerConsistencyIssueswell past a simple aggregation into a function juggling findings-count checks, per-index diffing, and now a 3-branch sourceOfTruthReview check. Extracting the new loop into a smallsourceOfTruthReviewLedgerIssues(review, index, openFindingIds)helper would keep this function's complexity manageable.As per coding guidelines,
**/*.{js,ts}: "Keep function complexity low in JavaScript and TypeScript code."♻️ Suggested extraction
+function sourceOfTruthReviewLedgerIssues( + review: SourceOfTruthReview, + index: number, + openFindingIds: ReadonlySet<string>, +): string[] { + const unresolved = review.status === "missing" || review.status === "needs_followup"; + if (unresolved && !review.findingId) { + return [`sourceOfTruthReview[${index + 1}] ${review.surface} must reference an open ledger finding`]; + } + if (unresolved && !openFindingIds.has(review.findingId!)) { + return [`sourceOfTruthReview[${index + 1}] ${review.surface} references non-open ledger finding ${review.findingId}`]; + } + if (!unresolved && review.findingId) { + return [`sourceOfTruthReview[${index + 1}] ${review.surface} must use findingId=null for status=${review.status}`]; + } + return []; +}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/pr-review-advisor/analyze.mts` around lines 673 - 714, The reviewLedgerConsistencyIssues function is becoming too complex because the new sourceOfTruthReview per-item validation is mixed in with the existing findings checks. Extract the loop logic into a small helper such as sourceOfTruthReviewLedgerIssues(review, index, openFindingIds) and have reviewLedgerConsistencyIssues aggregate its results, keeping the existing behavior for open-finding validation and status/findingId rules intact.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tools/pr-review-advisor/analyze.mts`:
- Around line 673-714: The reviewLedgerConsistencyIssues function is becoming
too complex because the new sourceOfTruthReview per-item validation is mixed in
with the existing findings checks. Extract the loop logic into a small helper
such as sourceOfTruthReviewLedgerIssues(review, index, openFindingIds) and have
reviewLedgerConsistencyIssues aggregate its results, keeping the existing
behavior for open-finding validation and status/findingId rules intact.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 3aa3cecb-49b4-4fc0-8be7-a78a1fdffc93
📒 Files selected for processing (12)
.github/workflows/pr-review-advisor.yamltest/advisor-session-context-tools.test.tstest/advisor-session-runner.test.tstest/pr-review-advisor-ledger-tools.test.tstest/pr-review-advisor-workflow-boundary.test.tstest/pr-review-advisor.test.tstools/advisors/README.mdtools/advisors/session.mtstools/advisors/turn-protocol.mtstools/pr-review-advisor/README.mdtools/pr-review-advisor/analyze.mtstools/pr-review-advisor/schema.json
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
test/pr-review-advisor-workflow-boundary.test.ts (1)
134-136: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider asserting
installis defined before non-null dereference.
install!.run!.replace(...)will throw a rawTypeError(rather than a clear test failure message) if the step is ever renamed andinstallisundefined. Aexpect(install).toBeDefined()before the mutation would make failures more diagnosable.🧪 Optional diagnostic assertion
const install = workflow.jobs.review.steps.find((step) => step.name === "Install Pi SDK"); + expect(install).toBeDefined(); install!.run = install!.run!.replace('"ripgrep=${RIPGREP_VERSION}"', "ripgrep");🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/pr-review-advisor-workflow-boundary.test.ts` around lines 134 - 136, The test mutates the “Install Pi SDK” step via install!.run!.replace(...), but it should first assert that the step lookup succeeded so a rename fails clearly. In test/pr-review-advisor-workflow-boundary.test.ts, add an explicit expectation that install is defined before dereferencing it, then keep the existing run replacement and workflow write logic unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@test/pr-review-advisor-workflow-boundary.test.ts`:
- Around line 134-136: The test mutates the “Install Pi SDK” step via
install!.run!.replace(...), but it should first assert that the step lookup
succeeded so a rename fails clearly. In
test/pr-review-advisor-workflow-boundary.test.ts, add an explicit expectation
that install is defined before dereferencing it, then keep the existing run
replacement and workflow write logic unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0113fa26-0c1a-4f60-9ae4-cb4324153456
📒 Files selected for processing (6)
.github/workflows/pr-review-advisor.yamltest/pr-review-advisor-workflow-boundary.test.tstest/pr-review-advisor.test.tstools/pr-review-advisor/README.mdtools/pr-review-advisor/analyze.mtstools/pr-review-advisor/workflow-boundary.mts
🚧 Files skipped from review as they are similar to previous changes (4)
- .github/workflows/pr-review-advisor.yaml
- tools/pr-review-advisor/README.md
- test/pr-review-advisor.test.ts
- tools/pr-review-advisor/analyze.mts
25 tasks
Collaborator
Author
|
Automated-review and CI follow-up:
|
Collaborator
Author
|
Exact-head maintainer follow-up at
GitHub's rollup still displays superseded cancelled duplicates from rapid consecutive head updates, but later successful runs exist for those same commit-lint, DCO, and maintainer-edit contexts. The only remaining merge gate is independent review; I am not self-approving a |
This was referenced Jul 9, 2026
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
<!-- markdownlint-disable MD041 --> ## Summary Make the PR Review Advisor's six analysis stages observable and its ledger writes atomic. This follows a census of every advisor run after NVIDIA#6547: useful findings were being hidden by protocol failures, especially when models omitted, retried, or mixed prose with ledger calls. ## Related Issue Follow-up to NVIDIA#6547 and NVIDIA#6446. ## Changes - Split every stage into a visible analysis turn and a strict tool-only commit turn, with one bounded repair attempt when no ledger mutation settles. - Add a reusable turn-protocol abstraction so stage ordering, validation, and repair behavior are defined once. - Preserve valid committed findings when a later stage fails, require explicit source-of-truth finding IDs, and remove prose-derived synthetic findings. - Recognize multi-issue `Refs`/`References`/`Follow-up to` relations and provide `rg` from a trusted runner binary or an exact package pin. - Add regression coverage for omitted, failed, retried, malformed, empty, and late-failure model behavior; update the advisor maintainer documentation. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: this changes internal PR-review automation; the two maintainer READMEs were updated, with no end-user behavior change. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — [GPT advisor review](NVIDIA#6566 (comment)) identified the package provenance gap; the runner/package are now pinned and enforced by a semantic workflow-boundary mutation test. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 131/131 focused advisor tests passed; the final workflow boundary suite passed 8/8. - [x] Applicable broad gate passed — the clean-checkout CI matrix and aggregate `checks` gate passed. The earlier local `npm test` run completed with 15,432 passed, 39 skipped, 1 todo, and 27 environment-only failures caused by the borrowed dependency symlink and ambient local `umask`/`SSH_AUTH_SOCK`; all advisor tests passed. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — completed with zero errors and only pre-existing warnings. - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com> --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: J. Yaunches <jyaunches@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make the PR Review Advisor's six analysis stages observable and its ledger writes atomic. This follows a census of every advisor run after #6547: useful findings were being hidden by protocol failures, especially when models omitted, retried, or mixed prose with ledger calls.
Related Issue
Follow-up to #6547 and #6446.
Changes
Refs/References/Follow-up torelations and providergfrom a trusted runner binary or an exact package pin.Type of Change
Quality Gates
Verification
Verifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablechecksgate passed. The earlier localnpm testrun completed with 15,432 passed, 39 skipped, 1 todo, and 27 environment-only failures caused by the borrowed dependency symlink and ambient localumask/SSH_AUTH_SOCK; all advisor tests passed.npm run docsbuilds without warnings (doc changes only) — completed with zero errors and only pre-existing warnings.Signed-off-by: Carlos Villela cvillela@nvidia.com