Repository navigation
feat: implement issue #469 — pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) - #471
Conversation
…ly lost to ci-pending skip (self-referential check runs)
|
Warning Review limit reached
More reviews will be available in 42 minutes and 56 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new helper script scripts/lib/ci-status.sh to compute CI gate status while filtering out the PR Review Agent's own check runs, preventing self-blocking cascades. It updates scripts/review-one-pr.sh to use this helper and adds a polling mechanism for mention-triggered runs when CI is pending. Comprehensive unit tests are also added in tests/test_ci_status.bats. The review feedback suggests several improvements: using Bash here-strings (<<<) instead of echo to safely pass JSON variables to jq without risk of backslash corruption, adding robust error and rate-limit handling for gh pr view calls during polling, and passing the comment body directly to gh pr comment via the --body flag instead of writing to a temporary file.
There was a problem hiding this comment.
Pull request overview
This PR addresses issue #469 where mention-triggered (FORCE_REVIEW=true) PR-review runs could be incorrectly skipped due to CI appearing “pending” from the PR Review Agent’s own self-referential check runs, resulting in a silent no-op. It introduces a dedicated CI-status classifier that filters out the agent’s own checks and adds a short polling window for forced reviews to avoid transient rollup churn swallowing mention-triggered runs.
Changes:
- Added
compute_ci_statushelper to classify CI state while filtering the PR-review cascade’s own check runs. - Updated
review-one-pr.shto use the new CI classifier and to poll briefly onFORCE_REVIEW=truewhen CI is pending, posting a visible comment if still pending. - Added Bats coverage for CI-status classification and wired the new test into the lint workflow.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tests/test_ci_status.bats |
Adds unit tests for CI rollup classification and own-check filtering behavior. |
scripts/review-one-pr.sh |
Replaces inline CI classification with compute_ci_status and adds forced-review polling + visible pending ack. |
scripts/lib/ci-status.sh |
Implements CI rollup classification with filtering for self-referential PR-review checks. |
.github/workflows/lint.yml |
Runs the new Bats test as part of the lint workflow. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0dce08435f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
@dev-lead please process and advance this PR — fix any failing CI, resolve outstanding review threads, and enable auto-merge once it's green and approvable. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 97611356b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: 76b22b0af6a6aee5d3630065f95b2db2b1f14b43
Review mode: triage-approved (single reviewer)
Summary
Confirms the triage tier's low-risk classification. PR refactors CI-status classification into scripts/lib/ci-status.sh so the cascade can filter its own check runs (the root cause from #469), and adds a polling loop in scripts/review-one-pr.sh for FORCE_REVIEW=true so mention-triggered runs survive transient check-rollup churn. The new helper is covered by 315 lines of bats tests, and lint.yml is updated to run them.
Linked issue analysis
Issue #469 lists three acceptance criteria, all satisfied:
- Mention during check churn never produces a silent no-op — when
FORCE_REVIEW=trueand CI is pending, the script now polls (10 × 30s default) and, if still pending, posts a visibleci-pending-ackcomment before exiting. - Own check runs never cause
ci-pending/ci-failingclassification —compute_ci_statusfilters entries namedreview/dispatch/ci-relay/pr-review-mention/mention-ack(both bare andworkflow / jobforms) before classifying. - Bats coverage for the rollup-filtering logic —
tests/test_ci_status.batscovers empty rollups, pending/failing/passing single checks, all the own-check name forms (bare + workflow-prefixed), mixed pending-own + external states, StatusContext entries, and malformed JSON inputs.
Findings
Nothing blocking. The earlier automated reviewers (gemini, copilot, codex) commented on commit 0dce0843 with minor style/robustness suggestions; CodeRabbit's subsequent review on the head SHA was an APPROVED state, indicating threads were resolved or judged non-blocking. Polling loop has bounded iterations (max ~5 min), temp file is cleaned up after use, and the polling block properly unsets its local vars.
CI status
All required checks passing:
- CI workflow: Lint, ShellCheck, Compile agentic workflows, Secret scan (gitleaks), Agent Security Scan — SUCCESS
- Lint workflow: shellcheck, bats, validate-agent-profiles, gh-aw-compile — SUCCESS
- CodeQL (actions, python) — SUCCESS
- SonarCloud, agent-shield, dependency-audit, unit-tests, PR Review Agent — SUCCESS
- Dependabot ecosystems (cargo/govuln/npm/pip/pnpm) — SKIPPED (no manifests)
Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.
Dev-Lead — on-mention (applied)Changes committed and pushed. |
b87f198
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6932d103da
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
|
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: c17345ad638dd8fc5ee8d6b86074d10bafad422b
Review mode: triage-approved (single reviewer)
Summary
Implements issue #469: mention-triggered (FORCE_REVIEW=true) PR-review runs no longer get silently swallowed by their own pending check runs. The fix extracts CI classification into scripts/lib/ci-status.sh with a compute_ci_status helper that filters the cascade's own check (review, plus reusable/mention-trigger workflow forms) before classifying, adds a short polling window when FORCE_REVIEW=true and CI is still pending, and posts a visible "CI still running — re-mention when green" comment if it doesn't settle. 342 lines of bats coverage exercise the filter and the dev-lead non-filter cases.
Linked issue analysis
Issue #469 acceptance criteria:
- A
@donpetry-bot reviewmention during check churn produces either a review or a visible explanatory comment — never a silent no-op. ✓ — polling loop inscripts/review-one-pr.sh(up to 10×30s = 5 min) followed by an explicitci-pending-ackcomment posted viagh pr commentwhen CI remains pending. - The PR Review Agent's own check runs never cause
ci-pending/ci-failingclassification. ✓ —is_own_checkfilters barereviewand the three known PR Review workflow prefixes; dev-lead bare names (dispatch,ci-relay) are deliberately NOT filtered to avoid racing branch-mutating runs, with tests asserting this. - Bats coverage for the rollup-filtering logic. ✓ —
tests/test_ci_status.batsadds 32 tests covering empty/passing/pending/failing rollups, bare and compound name filtering, dev-lead non-filtering, status-context entries, and malformed input.
Findings
No blocking findings. The implementation went through multiple iterations addressing prior reviewer feedback:
echo | jqreplaced with here-strings (<<<) throughout to avoid backslash corruption from user-generated content in check names.gh pr viewcalls inside the polling loop wrapped withis_rate_limitederror handling and clean exit-100 on rate limits.- Env-var overrides (
FORCE_REVIEW_POLL_ATTEMPTS,FORCE_REVIEW_POLL_INTERVAL_SEC) validated as positive integers with default fallbacks. - Own-check filter narrowed from broad bare-name matching (which could have silently bypassed target-repo
review/dispatch/Build / reviewgates) to only the PR Review workflow names, with explicit comments documenting why dev-lead bare checks remain in the gate. gh pr commentbody passed via--body "$_msg"(no disk I/O).
Optional follow-up (non-blocking): if other org workflows ever add a bare review job, the filter would mask it; this is acknowledged in the inline comments and is the correct trade-off for the current repo layout.
CI status
All checks green: Lint, bats, ShellCheck, CodeQL (actions & python), gitleaks, SonarCloud, agent-shield, validate-agent-profiles, Compile agentic workflows, gh-aw-compile, dependency-audit, CodeRabbit. Dependabot-automerge and ci-relay correctly SKIPPED.
Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c17345ad63
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| (.name // .context // "") as $n | | ||
| ($n == "review") or | ||
| ($n | test("^PR Review (Agent|Reusable) /")) or | ||
| ($n | test("^PR Review — Mention Trigger /")); |
There was a problem hiding this comment.
For mention-triggered reviews, this repo's caller workflow has a PR check job named pr-review-mention (.github/workflows/pr-review-mention.yml), but the helper only filters the compound PR Review — Mention Trigger / ... form while the file's own comment notes that gh pr view --json statusCheckRollup can expose only the job name. In that bare-name shape, an in-progress or failed mention-trigger check remains in $ext, so the repository_dispatch review can still wait for/potentially skip on the workflow that just triggered it instead of proceeding once external CI is green.
Useful? React with 👍 / 👎.
| (.name // .context // "") as $n | | ||
| ($n == "review") or |
There was a problem hiding this comment.
Because this derives $n from .context as well as .name, any legacy commit status context called review (or matching one of the PR Review prefixes) is filtered even though the PR Review Agent's own run is a GitHub Actions check run, not a commit status. In a target repo with a required external status context named review, a pending or failing status would be removed from $ext, causing compute_ci_status to report passing and let the cascade review before that real gate is green.
Useful? React with 👍 / 👎.
…ly lost to ci-pending skip (self-referential check runs) (#471) * feat: implement issue #469 — pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) * chore: apply manual instructions [skip ci-relay] * chore: apply manual instructions [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…ly lost to ci-pending skip (self-referential check runs) (#471) * feat: implement issue #469 — pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) * chore: apply manual instructions [skip ci-relay] * chore: apply manual instructions [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…ly lost to ci-pending skip (self-referential check runs) (#471) * feat: implement issue #469 — pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) * chore: apply manual instructions [skip ci-relay] * chore: apply manual instructions [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…ly lost to ci-pending skip (self-referential check runs) (#471) * feat: implement issue #469 — pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) * chore: apply manual instructions [skip ci-relay] * chore: apply manual instructions [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
…ly lost to ci-pending skip (self-referential check runs) (#471) * feat: implement issue #469 — pr-review: mention-triggered runs silently lost to ci-pending skip (self-referential check runs) * chore: apply manual instructions [skip ci-relay] * chore: apply manual instructions [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] * fix(reviews): address review comments [skip ci-relay] --------- Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
Closes #469
Implemented by dev-lead agent. Please review.