feat(#1762): prioritize functional correctness over surface-level checks in review - #1902
Conversation
…cks in review Add runtime mechanism verification instructions to the code-review skill's correctness dimension, budget allocation priority to the pr-review orchestrator's triage step, and a runtime mechanism checklist to the correctness sub-agent. These changes address a gap where the review agent spent tokens on surface-level consistency checks (stale terminology, naming mismatches) while missing functional bugs that human reviewers caught — such as guard mechanisms that would never activate, missing output format contracts between components, and unhandled failure paths. Three files changed: - code-review/SKILL.md: new "Runtime mechanism verification" bullet in the correctness dimension instructing reviewers to trace guards from producer to consumer and verify they function at runtime. - pr-review/SKILL.md: new "Budget allocation priority" section (3a-1) establishing priority order: functional correctness > security > intent coherence > docs/style/contracts. - pr-review/sub-agents/correctness.md: new "Runtime mechanism checklist" with three items covering producer-consumer tracing, format expectation matching, and failure path handling. Note: make lint could not run due to Go toolchain permission error in sandbox (unrelated to these markdown-only changes). Closes #1762
Site previewPreview: https://0e145244-site.fullsend-ai.workers.dev Commit: |
ReviewFindingsLow
Info
|
| When evaluating tests, check git history of modified test files for | ||
| assertion loosening or coverage reduction that coincides with production | ||
| changes — this is a security-adjacent concern (split-payload pattern). | ||
|
|
There was a problem hiding this comment.
[low] test-inadequate
The runtime mechanism checklist is added as prompt-level instructions with no regression test to verify effectiveness. Without replaying PR #1550 or a similar test case through the updated agent, there is no empirical validation these instructions work.
| @@ -205,6 +205,29 @@ dimension by keyword, or to `correctness` as a fallback. | |||
|
|
|||
| Each sub-agent receives ONLY the prior findings for its own dimension. | |||
|
|
|||
There was a problem hiding this comment.
[low] logic-error
Section 3a-1 introduces budget allocation priority as advisory guidance, but the orchestrator has no mechanism to enforce it. Sub-agents run independently with identical context packages. The priority ordering can only influence synthesis, which already determines outcome by severity.
Add runtime mechanism verification instructions to the code-review skill's correctness dimension, budget allocation priority to the pr-review orchestrator's triage step, and a runtime mechanism checklist to the correctness sub-agent.
These changes address a gap where the review agent spent tokens on surface-level consistency checks (stale terminology, naming mismatches) while missing functional bugs that human reviewers caught — such as guard mechanisms that would never activate, missing output format contracts between components, and unhandled failure paths.
Three files changed:
in the correctness dimension instructing reviewers to trace guards
from producer to consumer and verify they function at runtime.
(3a-1) establishing priority order: functional correctness >
security > intent coherence > docs/style/contracts.
checklist" with three items covering producer-consumer tracing,
format expectation matching, and failure path handling.
Note: make lint could not run due to Go toolchain permission error in sandbox (unrelated to these markdown-only changes).
Closes #1762
Post-script verification
agent/1762-review-correctness-priority)7e768afe50ea067e39066e4adc5451f22f85cec0..HEAD)