fix(#1527): warn when automated review misses initial PR window - #1531
Conversation
When the review agent's pre-fetch step finds no prior automated review, check whether human reviewers have already submitted formal PR reviews. If so, emit a ::warning:: annotation to surface cases where the automated review dispatch was delayed (e.g. by a transient webhook delivery failure) and a human had to review first. This provides observability into the gap described in the issue without changing dispatch behavior. Note: make lint could not run due to a Go toolchain permission error in the sandbox (not related to this change). The script passes bash -n syntax validation. Closes #1527
Site previewPreview: https://9aaf1556-site.fullsend-ai.workers.dev Commit: |
ReviewFindingsLow
|
|
|
||
| # Observability: warn if human reviewers already commented before the | ||
| # first automated review dispatch. This surfaces cases where the | ||
| # automated review missed the initial PR-creation window (e.g. due to |
There was a problem hiding this comment.
[low] correctness
The --paginate --jq pattern applies the jq expression per-page. If reviews span multiple pages (30+), the command outputs one integer per page and the -gt comparison silently fails on the multiline string, suppressing the warning. The existing code in this same script (lines 18-23) correctly handles pagination by piping --jq .[] to jq -s.
Suggested fix: Follow the existing pagination pattern: --jq '.[]' | jq -s '[.[] | select(.user.type != "Bot")] | length'
Retro: PR #1531 — Review dispatch observability warningTimeline: Issue #1527 was filed by the retro agent on 2026-05-26 about a 6-hour gap where automated review missed PR #1487's initial window. The code agent created PR #1531 within 12 minutes, adding a What went well: Fast code agent turnaround (12 min from issue to PR). Review agent correctly identified the pagination bug. Clean single-file change. What fell through: The review agent's finding about Existing issue coverage (skipped proposals):
1 proposal filed for the concrete shipped bug that has no existing issue tracking it. Proposals filed
|
When the review agent's pre-fetch step finds no prior automated review, check whether human reviewers have already submitted formal PR reviews. If so, emit a ::warning:: annotation to surface cases where the automated review dispatch was delayed (e.g. by a transient webhook delivery failure) and a human had to review first. This provides observability into the gap described in the issue without changing dispatch behavior.
Note: make lint could not run due to a Go toolchain permission error in the sandbox (not related to this change). The script passes bash -n syntax validation.
Closes #1527
Post-script verification
agent/1527-review-dispatch-observability)f6c8d3509682ca18940bad7e0c9cef510572fe4b..HEAD)