fix(#3864): preserve prior review remediations

[shairevivo]

Summary

• Treat direct, trusted prior-review remediations as expected work during re-review without exempting unrelated changes.
• Materialize the prior-review-to-HEAD diff and provide remediation candidates, provenance, and delta context to intent-coherence.
• Keep intent-coherence lightweight when it is re-dispatched solely for remediation or unmatched-delta scope checks, preserving the performance guard from Scope re-review sub-agent dispatch to dimensions with prior findings #354.
• Keep ordinary linked-issue authorization available for changes not covered by a prior finding.
• Add a re-review functional-eval fixture and synchronize GitHub/GitLab provenance documentation.

Fixes fullsend-ai/fullsend#3864.

Verification

• bash scripts/pr-review-remediation-test.sh
• bash eval/lint-cases.sh review
• make check-bundle
• make script-test
• make lint

A prepared single-case live functional eval was not started because this machine has no Vertex AI credentials; no temporary GitHub repo or PR was created.

[github-actions]

Functional tests did not run

Functional tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer must add the ok-to-test label after the latest push.

[qodo-code-review]

PR Summary by Qodo

Preserve verified review remediations during re-review

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

AI Description

• Exempts app-verified prior-finding remediations from scope-creep findings.
• Keeps unrelated changes subject to scope review and remediation correctness checks.
• Adds regression coverage for remediation provenance and review guidance.

Diagram

graph TD
  A["Prior findings"] --> C{"Verified match?"} -->|Yes| D["Candidate context"] --> E["Intent agent"] --> F{"Direct remediation?"} -->|Yes| G["Continued reviews"]
  B["Changed files"] --> C
  C -->|No| H["Scope review"]
  F -->|No| H

Loading

High-Level Assessment

The targeted, provenance-gated exemption is the appropriate approach. A blanket exemption for all changes after review would hide genuine scope creep, while inferring remediation solely from commit history would provide weaker authorization than matching app-verified findings by category and file.

Files changed (5) +75 / -3

Bug fix (2) +26 / -0

SKILL.mdPass verified remediation candidates into review context +13/-0

Pass verified remediation candidates into review context

• Defines how app-verified prior findings are matched to changed files during re-review. Adds remediation candidates and prior-review provenance to sub-agent context while retaining reviews for unmatched and substantive changes.

skills/pr-review/SKILL.md

intent-coherence.mdExclude direct remediations from scope-creep findings +13/-0

Exclude direct remediations from scope-creep findings

• Instructs the intent-coherence reviewer not to flag verified, directly matched remediation as scope creep. Unverified, unrelated, or excessive changes remain reportable, and remediation still receives correctness and completeness review.

skills/pr-review/sub-agents/intent-coherence.md

Tests (2) +46 / -0

MakefileAdd remediation regression test to script suite +1/-0

Add remediation regression test to script suite

• Runs the new PR review remediation contract test as part of the standard 'script-test' target.

Makefile

pr-review-remediation-test.shVerify remediation review guidance contract +45/-0

Verify remediation review guidance contract

• Adds assertions covering verified provenance, category-and-file matching, candidate context propagation, scope preservation, and continued correctness review.

scripts/pr-review-remediation-test.sh

Other (1) +3 / -3

.skillsaw-baseline.jsonRefresh review-skill context budget baseline +3/-3

Refresh review-skill context budget baseline

• Regenerates the baseline timestamp and records the increased token estimate for the expanded PR review skill.

.skillsaw-baseline.json

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (1)

1. Unrelated re-review changes go unchecked ✓ Resolved 🐞 Bug ≡ Correctness

Description

The new remediation rule says unmatched changes remain under scope-creep review, but the existing
re-review dispatch condition skips intent-coherence unless changed files implement behavior
claimed by the linked issue. When an app-verified re-review contains remediation plus an unrelated
file, that file does not re-qualify the agent and therefore receives no scope-authorization review.

Code

skills/pr-review/SKILL.md[R244-245]

+`app-verified` prior finding's category and file; unmatched changes remain in
+scope for scope-creep review and matched changes still receive other reviews.

Relevance

●●● Strong

Concrete dispatch gap leaves unrelated re-review changes without the required scope-authorization
check.

PR-#49

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The added guidance explicitly keeps unmatched changes in scope, while the re-review dispatch rules
skip intent-coherence when that dimension had no prior finding unless the incremental files
implement behavior claimed by the issue. Intent-coherence exclusively owns scope authorization and
scope creep, so correctness and style dispatch cannot perform the omitted check.

skills/pr-review/SKILL.md[240-245]
skills/pr-review/SKILL.md[316-339]
skills/pr-review/SKILL.md[354-367]
skills/pr-review/sub-agents/intent-coherence.md[17-23]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The remediation guidance promises scope-creep review for unmatched changes, but re-review dispatch can skip the intent-coherence agent that exclusively owns that review.

## Fix Focus Areas
- skills/pr-review/SKILL.md[240-245]
- skills/pr-review/SKILL.md[316-339]

## Recommended Fix
Update re-review dispatch so intent-coherence is selected whenever remediation candidates or unmatched changed files exist. Ensure it receives the complete incremental diff so it can distinguish direct remediation from unrelated edits, including extra edits within a matched file.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. GitLab remediations remain scope creep ✓ Resolved 🔗 Cross-repo conflict ≡ Correctness

Description

The candidate-selection guidance accepts only app-verified provenance, but fullsend's GitLab
workflow emits bot-verified after validating and loading a prior review. Every GitLab re-review
therefore supplies no matched remediation candidates to intent-coherence, so direct fixes can still
be reported as scope creep.

Code

skills/pr-review/SKILL.md[244]

+`app-verified` prior finding's category and file; unmatched changes remain in

Relevance

●● Moderate

Concrete provenance mismatch is plausible, but cross-workflow integration behavior lacks a close
team precedent.

PR-#1038

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The changed guidance requires the GitHub-specific app-verified value. Fullsend's GitLab
integration instead assigns bot-verified after checking the note author and exports that distinct
value to the review harness, so GitLab can never satisfy the new condition.

skills/pr-review/SKILL.md[240-245]
skills/pr-review/SKILL.md[364-367]
External repo: fullsend-ai/fullsend, internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/fullsend-agent.yml [396-400]
External repo: fullsend-ai/fullsend, internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/fullsend-agent.yml [445-452]
External repo: fullsend-ai/fullsend, internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/fullsend-agent.yml [476-478]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
GitLab re-reviews receive `bot-verified` provenance from fullsend, while remediation candidate construction requires `app-verified`, preventing prior-review remediations from receiving the new scope-creep exemption.

## Fix Focus Areas
- skills/pr-review/SKILL.md[240-245]
- skills/pr-review/sub-agents/intent-coherence.md[25-36]
- scripts/pr-review-remediation-test.sh[25-38]

## Recommended Fix
Define the trusted provenance values by forge, accepting fullsend's `bot-verified` value for GitLab remediation candidates while continuing to reject `none` and all `unverifiable-*` values. Document GitLab's weaker author-based trust level and add a contract test covering both GitHub `app-verified` and GitLab `bot-verified` re-reviews.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Protected review logic needs approval 📜 Skill insight § Compliance

Description

This PR changes scripts/pr-review-remediation-test.sh and two files under skills/, all of which
are protected governance paths. Issue #3864 explains the work, so the changes are justified, but
they still require human review rather than automated approval.

Code

skills/pr-review/SKILL.md[R240-243]

+#### 3a-2. Prior-finding remediation candidates
+
+On re-review, pass a `Prior-finding remediation candidates` section to the
+intent-coherence sub-agent. Include only changed files matching an

Relevance

● Weak

Protected-path approval guidance was explicitly rejected in a closely matching precedent.

PR-#753

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

Compliance rule 1538392 requires a finding whenever protected paths such as scripts/ or skills/
are modified. The cited additions change the primary review skill, its intent-coherence sub-agent,
and a review test script; the linked issue supplies justification but does not remove the
human-approval requirement.

skills/pr-review/SKILL.md[240-245]
skills/pr-review/sub-agents/intent-coherence.md[25-36]
scripts/pr-review-remediation-test.sh[1-45]
Skill: pr-review

---

Context sources

✅ Compliance rules (platform): 57 rules

✅ Skills: 4 invoked

  code-review

  code-implementation

  pr-review

  docs-review

✅ Cross-repo context — repo relationships

  Explored: repo: fullsend-ai/fullsend (sha: ec55e9ca)

Review mode: ⚖️ Balanced: This changes executable review guidance and adds a regression test affecting re-review scope handling, so it warrants a careful single-pass review but is not sufficiently bug-dense for extended.

Tip of the day

💡 Did you know, you can turn these tips off under Display preferences

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[waynesun09]

Review-only pass on 3a4b0a2 — no approval or change request implied, and I have not touched assignment or labels.

Nine findings; seven are anchored inline. The three below have no line inside the diff, so they are here with explicit paths.

---

HIGH — Dispatch-examples table still omits intent-coherence, contradicting the new always-dispatch rule

skills/pr-review/SKILL.md:381-382

The PR rewrites lines 336-341 to "intent-coherence — always re-qualifies when changed_since_prior is non-empty". Both re-review rows of the Dispatch examples table are unchanged and still list agent sets without intent-coherence:

• line 381 — Re-review after fix (prior findings in correctness only) | correctness (full scope), style-conventions (trivial scope), challenger
• line 382 — Re-review after fix (prior findings in security only) | correctness (full scope), security (normal scope), style-conventions (trivial scope), challenger

Verified both against head 3a4b0a2. Those rows were added by cc6af96 alongside the old narrowed rule and were not updated here. Every re-review-after-fix has a non-empty delta, so both rows now describe dispatch sets the rule 45 lines above forbids. An orchestrator reading the concrete examples (the more actionable of the two) will skip intent-coherence — the exact sub-agent that owns the new remediation exemption this PR exists to add — so the fix silently does nothing on the canonical scenario.

Suggestion: Add intent-coherence to both re-review rows (with whatever scope constraint the rule ends up carrying), and grep the rest of the table for any other row describing a re-review path.

---

MEDIUM — Step 3d context-package field list not updated for the two new fields

skills/pr-review/SKILL.md:637-659

The step 4 prompt template gained two new sections at lines 791-795 (### Prior-finding remediation candidates and ### Prior review provenance), but the step 3d "Prepare context packages" field list was not touched: it still enumerates only diff, pr_head, head_sha, repo_full_name, changed_files, prior_findings, prior_review_sha, changed_since_prior, pr_metadata, issue_context, cross_repo_context, scope_constraint.

3d is the step that says what to assemble; step 4 only renders it. An orchestrator implementing 3d literally never computes the candidates or the provenance value, so the template renders "none" on every re-review and the exemption never fires.

Suggestion: Add remediation_candidates (intent-coherence only, per 3a-2) and prior_review_provenance to the 3d field list so assembly and rendering agree.

---

MEDIUM — agents/review.md provenance enum omits bot-verified and unverifiable-wrong-user

agents/review.md:40-47

SKILL.md now declares (lines 204-206) that trusted provenance is "app-verified (GitHub) or bot-verified (GitLab)", and the new test asserts both literals. I verified bot-verified is real: fullsend's GitLab scaffold internal/scaffold/fullsend-repo-gitlab/.gitlab/ci/fullsend-agent.yml:451 sets PRIOR_REVIEW_PROVENANCE="bot-verified" after an author.id check (the comment at :396 notes it is a weaker check), emits unverifiable-wrong-user on mismatch, and scaffold_gitlab_test.go:237 asserts it.

But agents/review.md:40-47 — the runner-facing documentation of that same env var — still lists only none, app-verified, unverifiable-no-app, unverifiable-wrong-app. The skill and the agent contract doc now disagree on the value vocabulary, so a reader of the contract treats bot-verified as an unknown (therefore untrusted) value. This file is not in the diff, hence the body placement.

Suggestion: Add bot-verified (GitLab, author.id-based, weaker than the GitHub app check) and unverifiable-wrong-user to the enum in agents/review.md so the contract matches the scaffold and the skill.

[shairevivo]

Addressed the three body-only findings in 04144a0: the re-review dispatch examples include bounded intent-coherence; step 3d now assembles remediation candidates, provenance, and the incremental-diff path; and agents/review.md documents GitLab bot-verified and unverifiable-wrong-user provenance values. The protected-path item remains subject to human merge review.