Skip to content

ci: stale-PR automation — bestaxbot closer + generic stale (#275) #279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
allxsmith merged 8 commits into from
Jul 11, 2026
+200 −8
Merged

ci: stale-PR automation — bestaxbot closer + generic stale (#275) #279

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
acb6549
ci: stale-PR automation — bestaxbot closer + generic stale (#275)
claude Jul 10, 2026
f49942a
ci: exempt neverstale-labeled PRs from the bestaxbot stale closer (#275)
claude Jul 10, 2026
973ce0e
ci: exempt claude-assisted PRs from the 30/14 stale sweep; widen the …
claude Jul 11, 2026
4f095a3
ci: backfill claude-assisted on bot PRs, revalidate before close, inc…
claude Jul 11, 2026
492bd4e
ci: raise gh pr list limits, stop swallowing label-strip failures (#275)
claude Jul 11, 2026
7f7b60a
Merge remote-tracking branch 'origin/main' into ci/stale-automation
claude Jul 11, 2026
b7a9446
Merge remote-tracking branch 'origin/main' into ci/stale-automation
claude Jul 11, 2026
d4ffd23
ci: serialize the bestaxbot stale closer with a concurrency group
claude Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 124 additions & 0 deletions .github/workflows/close-stale-bestaxbot-prs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
name: Close Stale Claude-Assisted PRs

# Closes Claude-assisted PRs (bestaxbot-authored or labeled
# `claude-assisted`) untouched for 90 days. Mirrors oven-sh/bun's
# close-stale-robobun-prs.yml. Zero Claude usage — plain shell + gh, so it
# can never spend AI budget or act on untrusted content.
#
# This is the ONLY closer for these PRs: stale.yml exempts the
# `claude-assisted` label from its generic 30/14-day sweep, so AI-assisted
# PRs get the longer leash (they routinely sit waiting on human review).
# The sweep is the union of author:bestaxbot and label:claude-assisted —
# the label (auto-applied by auto-label-claude-prs.yml) also covers
# human-authored PRs with Claude attribution, and the author query covers
# bot PRs opened before the labeler existed. A first step backfills the
# claude-assisted label onto any unlabeled bestaxbot PR so the generic
# sweep can't close pre-labeler bot PRs at day 44: this job's 03:41 cron
Comment on lines +11 to +16

@claude claude Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

auto-label-claude-prs.yml doesn't exist — human-authored Claude PRs are never protected — 🟠 Major · Correctness

What: Both new workflows repeatedly cite auto-label-claude-prs.yml as the source of the claude-assisted label (here, and in stale.yml:11). That workflow does not exist anywhere in the repo — the only thing that ever applies claude-assisted is the backfill step below, and it is scoped to --author bestaxbot.

Why it matters: The stated design is a union: author:bestaxbot plus label:claude-assisted so the label leg "also covers human-authored PRs with Claude attribution." In reality no human-authored PR ever receives the label, so:

  • stale.yml's exempt-pr-labels: …,claude-assisted never matches a human PR → it gets marked stale at 30 days and closed at 44.
  • This 90-day closer's --label claude-assisted leg only ever returns bestaxbot PRs (which the author leg already covers), so it's currently redundant.

Net: the entire "longer leash for human Claude-assisted PRs" story is inoperative. The bot path works (author query + backfill); the human path silently does not.

Fix: Either ship the auto-label-claude-prs.yml labeler in this PR (so the label is actually applied on human PRs with Claude attribution), or drop the claims/leg that depend on it and document that only bestaxbot-authored PRs get the 90-day leash.

Verification 
$ ls .github/workflows | grep -i label      → (nothing)
$ grep -rn "auto-label-claude-prs" .         → only the two new files, as dangling references
$ grep -rn 'claude-assisted' .github/workflows
    → applied only by the backfill step (author bestaxbot); exempted in stale.yml

# fires before stale.yml's 04:17 sweep.
#
# PRs labeled `neverstale` are exempt from this closer too — the label means
# "exempt from stale automation", and that covers the 90-day sweep as well
# as the generic 30/14-day stale.yml layer.
#
# Loop labels (ai-loop / ai-loop-paused / needs-human-review) are stripped
# BEFORE the close (same pattern as on-slop.yml) so the PR exits the AI
# loop's state machine cleanly instead of lingering as a closed-but-labeled
# zombie the watchdog might still try to sweep.

on:
schedule:
# 03:41 UTC daily — deliberately offset from the loop watchdog's
# `23 */2 * * *` so the closer never races a sweep re-dispatch.
- cron: '41 3 * * *'
workflow_dispatch:
Comment on lines +28 to +33

@coderabbitai coderabbitai Bot Jul 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## File list\n'
git ls-files '.github/workflows/*' | sed 's#^`#-` #'

printf '\n## Target workflow\n'
wc -l .github/workflows/close-stale-bestaxbot-prs.yml
cat -n .github/workflows/close-stale-bestaxbot-prs.yml

printf '\n## Search for concurrency and watchdog references\n'
rg -n "concurrency|23 \\*/2 \\* \\* \\*|close-stale|bestaxbot|watchdog|re-dispatch" .github/workflows

Repository: allxsmith/bestax

Length of output: 9279

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## claude-pr-loop relevant sections\n'
sed -n '1,120p' .github/workflows/claude-pr-loop.yml

printf '\n## on-slop relevant sections\n'
sed -n '1,90p' .github/workflows/on-slop.yml

Repository: allxsmith/bestax

Length of output: 8538

Add a concurrency group to the closer workflow. The cron offset doesn’t prevent overlap with manual dispatches or another in-flight run, so duplicate comments and competing label/close operations can still happen. If this needs to serialize with the watchdog too, use the same group naming scheme there.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/close-stale-bestaxbot-prs.yml around lines 12 - 17, Add a
workflow-level concurrency group to the closer workflow identified by its `on`
schedule and `workflow_dispatch` triggers, using a stable group name and
cancel-in-progress behavior that prevents overlapping runs, including manual
dispatches. If serialization with the loop watchdog is required, configure both
workflows to use the same concurrency group naming scheme.


# No workflow-level grants; the single job takes exactly what it needs.
permissions: {}

# One sweep at a time: a manual workflow_dispatch overlapping the daily cron
# must not race a second closer over the same PRs (double comments, or a
# second `gh pr close` aborting under set -e).
concurrency:
group: close-stale-bestaxbot-prs
cancel-in-progress: false

jobs:
close:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
pull-requests: write # comment, edit labels, close
issues: write # issue-comment API for the PR conversation
steps:
# Self-heal: bestaxbot PRs opened before auto-label-claude-prs.yml
# existed (or where it misfired) lack the claude-assisted label, so
# stale.yml's generic 30/14-day sweep would close them at day 44 before
# this 90-day closer ever saw them. Label them here; the REST call
# auto-creates the label on first use, same as the labeler.
- name: Backfill claude-assisted onto unlabeled bestaxbot PRs
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
gh pr list --repo "$REPO" --author bestaxbot --state open \
Comment thread
coderabbitai[bot] marked this conversation as resolved.
--limit 1000 --json number,labels \
| jq -r '.[] | select(([.labels[].name] | index("claude-assisted")) | not) | .number' \
| while read -r PR; do
echo "Labeling PR #$PR claude-assisted"
gh api -X POST "repos/$REPO/issues/$PR/labels" \
-f "labels[]=claude-assisted" || true
done

- name: Close Claude-assisted PRs inactive for 90+ days
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
CUTOFF=$(date -u -d '90 days ago' +%Y-%m-%dT%H:%M:%SZ)
echo "Closing Claude-assisted PRs not updated since $CUTOFF"

# Union of the two selectors, deduped by PR number. A label with
# no matches (or that doesn't exist yet) yields an empty list from
# `gh pr list`, not an error. <= makes the 90-day cutoff inclusive.
{
gh pr list --repo "$REPO" --author bestaxbot --state open \
--limit 1000 --json number,updatedAt,labels
gh pr list --repo "$REPO" --label claude-assisted --state open \
--limit 1000 --json number,updatedAt,labels
} \
| jq -rs --arg cutoff "$CUTOFF" \
'add | unique_by(.number) | .[] | select(.updatedAt <= $cutoff) | select(([.labels[].name] | index("neverstale")) | not) | .number' \
| while read -r PR; do
# Re-check eligibility immediately before acting: the listing
# snapshot ages while earlier PRs in the loop are processed,
# and a maintainer may have pushed activity or applied
# neverstale in the meantime. Any fetch failure counts as
# not-eligible — closing on stale data is worse than retrying
# tomorrow.
ELIGIBLE=$(gh pr view "$PR" --repo "$REPO" \
--json state,updatedAt,labels \
| jq -r --arg cutoff "$CUTOFF" \
'if .state == "OPEN" and .updatedAt <= $cutoff and (([.labels[].name] | index("neverstale")) | not) then "yes" else "no" end' \
|| echo no)
if [ "$ELIGIBLE" != "yes" ]; then
echo "Skipping PR #$PR — no longer eligible"
continue
fi

echo "Closing stale PR #$PR"
gh pr comment "$PR" --repo "$REPO" --body "Closing this PR after 90 days of inactivity. If it's still wanted, reopen it — for AI-loop PRs, you can instead restart the loop per the instructions in the PR's \`ai-loop-state\` status comment."
Comment thread
coderabbitai[bot] marked this conversation as resolved.

# Strip loop labels FIRST so the PR leaves the loop's state
# machine cleanly. gh succeeds when a label is absent, so a
# failure here is a real API/auth error — let it abort (set -e)
# rather than close the PR with loop labels still attached;
# the daily cron retries tomorrow.
gh pr edit "$PR" --repo "$REPO" \
--remove-label ai-loop \
--remove-label ai-loop-paused \
--remove-label needs-human-review
Comment on lines +118 to +121

@claude claude Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Un-guarded label strip can halt the whole closer — 🟡 Minor · Correctness

What: This diverges from the on-slop.yml pattern it claims to mirror — that one ends the same three --remove-label calls with || true (on-slop.yml:67); here it runs bare under set -e. gh pr edit --remove-label X is a silent no-op only when X still exists as a repo label; if any of ai-loop / ai-loop-paused / needs-human-review has been deleted or renamed, gh exits non-zero with 'X' not found.

Why it matters: The comment justifies dropping the guard as "a failure here is a real API/auth error" — but a label removed from the repo is neither, yet it aborts the step on the first PR in the loop, so no PRs get closed that day (and every subsequent day until the label is restored). Most 90-day-stale PRs carry none of these loop labels, making them the common case that trips it.

Fix: restore the tolerant form so a stray label can't wedge the sweep, while still surfacing genuine auth failures via the daily retry:

Suggested change
gh pr edit "$PR" --repo "$REPO" \
--remove-label ai-loop \
--remove-label ai-loop-paused \
--remove-label needs-human-review
gh pr edit "$PR" --repo "$REPO" \
--remove-label ai-loop \
--remove-label ai-loop-paused \
--remove-label needs-human-review || true


gh pr close "$PR" --repo "$REPO"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
done
63 changes: 63 additions & 0 deletions .github/workflows/stale.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Stale

# Marks PRs with no activity for 30 days as `stale`, then closes them 14
# days later if the inactivity continues. Mirrors oven-sh/bun's stale.yaml.
# Zero Claude usage — just actions/stale.
#
# Escape hatch: a maintainer applies the `neverstale` label to exempt a PR
# permanently. Any activity (commit, comment, review) also removes `stale`
# and resets the clock.
#
# Claude-assisted PRs (the `claude-assisted` label, auto-applied by
# auto-label-claude-prs.yml) are exempt from this sweep: they get the longer
# 90-day leash enforced by close-stale-bestaxbot-prs.yml instead, since they
# routinely sit waiting on human review. Without this exemption the 30/14
# sweep would close them at day 44 and the 90-day closer could never fire.
#
# Labels: `stale` is auto-created by actions/stale on first application
# (suggested color #ededed). `neverstale` (#c2e0c6) is human-applied and
# created at rollout.
#
# Issues are deliberately NOT processed (a deviation from bun, whose issue
# side keys off a needs-repro/waiting-for-author triage process this repo
# doesn't run).

on:
schedule:
# 04:17 UTC daily — a distinct minute from both the loop watchdog
# (`23 */2 * * *`) and the bestaxbot closer (`41 3 * * *`).
- cron: '17 4 * * *'
workflow_dispatch:

# No workflow-level grants; the single job takes exactly what it needs.
permissions: {}

jobs:
stale:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
pull-requests: write # label, comment, close PRs
issues: write # actions/stale needs both even with issues disabled
steps:
- uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9.1.0
with:
days-before-pr-stale: 30
days-before-pr-close: 14
stale-pr-label: stale
exempt-pr-labels: neverstale,claude-assisted
remove-stale-when-updated: true
stale-pr-message: >
This PR has had no activity for 30 days and has been marked
`stale`. It will close in 14 days unless activity resumes — any
commit, comment, or review removes the label and resets the
clock. If this PR should never be swept, ask a maintainer to
apply the `neverstale` label.
close-pr-message: >
Closing after 14 further days of inactivity. If this is still
wanted, reopen it or open a fresh PR — and consider asking a
maintainer for the `neverstale` label to exempt it from stale
automation.
# Issues disabled — see the header comment.
days-before-issue-stale: -1
days-before-issue-close: -1
3 changes: 3 additions & 0 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,3 +96,6 @@ label. Kill switches: remove `ai-loop` (per PR) or set repo
variable `AI_LOOP_ENABLED=false` (whole system). The `<!-- ai-loop-state … -->` PR comment
is machine-managed — never reformat its first line. The loop refuses PRs that touch
`.github/**` or the jest/commitlint/release/pnpm-workspace configs.
Stale automation: PRs go `stale` at 30 days and close 14 days later — except Claude-assisted
PRs (`claude-assisted` label or bestaxbot author), which skip that sweep and instead close
after 90 days of inactivity; `neverstale` exempts a PR from both layers.
18 changes: 10 additions & 8 deletions docs/docs/guides/getting-started/ai-development.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,16 @@ the review-time requirements (Storybook story for UI changes, docs page for API

PRs authored by the loop move through a small label lifecycle:

| Label | Where | Meaning |
| -------------------- | ------ | ---------------------------------------------------------------------------------------------- |
| `claude-fix` | issues | Maintainer-approved: Claude implements this issue and opens a PR |
| `ai-loop` | PRs | The PR is inside the autonomous review/fix loop |
| `needs-human-review` | PRs | The loop converged (or hit a disagreement) — awaiting maintainer review/merge |
| `ai-loop-paused` | PRs | The loop hit its iteration cap or a guardrail — a maintainer must intervene |
| `deep-review` | PRs | Opt-in: a triage+ user applies it to run the Claude deep review on any PR (re-apply to re-run) |
| `claude-assisted` | PRs | Auto-applied provenance for AI-assisted PRs (bestaxbot or the Claude footer) |
| Label | Where | Meaning |
| -------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `claude-fix` | issues | Maintainer-approved: Claude implements this issue and opens a PR |
| `ai-loop` | PRs | The PR is inside the autonomous review/fix loop |
| `needs-human-review` | PRs | The loop converged (or hit a disagreement) — awaiting maintainer review/merge |
| `ai-loop-paused` | PRs | The loop hit its iteration cap or a guardrail — a maintainer must intervene |
| `deep-review` | PRs | Opt-in: a triage+ user applies it to run the Claude deep review on any PR (re-apply to re-run) |
| `claude-assisted` | PRs | Auto-applied provenance for AI-assisted PRs (bestaxbot or the Claude footer) |
| `stale` | PRs | Auto-applied after 30 days of inactivity; closes 14 days later unless activity resumes. Claude-assisted PRs skip this sweep — a separate closer sweeps them after 90 days instead |
| `neverstale` | PRs | Exempts a PR from all stale automation (both the 30/14-day sweep and the 90-day Claude-assisted closer) |

The loop itself: Claude implements the issue and opens the PR → CodeRabbit reviews it and a
second, independent Claude review (a stronger model than the implementer) does a deep pass →
Expand Down
Loading