Skip to content

fix: remove tier-1 dry-run, self-provision ai:blocked - #2899

Merged
alexbelgium merged 6 commits into
masterfrom
fix/ai-triage-live-readiness
Jul 23, 2026
Merged

alexbelgium merged 6 commits into
masterfrom
fix/ai-triage-live-readiness

Conversation

@alexbelgium

@alexbelgium alexbelgium commented Jul 23, 2026 •

Copy link
Copy Markdown
Owner

Two small follow-ups now that AI_PR_TOKEN is configured and tier 1 has been watched running in dry-run for a while.

  1. Remove DRY_RUN from on_issues_ai_triage.yaml entirely — it was scaffolding for the initial rollout, not meant to stay. Verdicts now apply labels/comments unconditionally instead of only logging what they'd do.

  2. daily_ai_fix.yaml: self-provision ai:blocked — every other control label (ai-triage, ai:classified, ai:needs-human, ai:fixed, ai:upstream) gets created up front via the existing "ensure labels exist" step, but ai:blocked never did; the forbidden-paths guard applies it directly with no prior gh label create. Under set -euo pipefail, gh pr edit --add-label on a label that doesn't exist yet fails and aborts that step's loop immediately — silently skipping every remaining PR behind the one that failed. Folded it into the same pre-creation step (renamed "Ensure control labels exist" to reflect that it's no longer only about the per-issue relabel).

Both files pass actionlint clean; no other DRY_RUN references remain anywhere in .github/.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added manual issue re-triage from workflow dispatch with a required issue number.
    • Improved issue triage label handling, including support for blocked issues and cleanup of stale labels.
  • Bug Fixes

    • Prevented low-confidence add-on bug classifications from receiving incorrect triage labels.
    • Improved workflow behavior for owned issues and manual re-triage runs.
    • Enabled secure authentication for automated AI fixes.

- on_issues_ai_triage.yaml: remove DRY_RUN entirely. AI_PR_TOKEN is now
  configured, tier 1 has been watched in dry-run, and the toggle was
  meant to be temporary scaffolding, not a permanent code path — verdicts
  now apply labels/comments unconditionally.
- daily_ai_fix.yaml: fold ai:blocked into the existing "ensure labels
  exist up front" step (renamed to reflect that). It was the one control
  label neither workflow ever created: the forbidden-paths guard applies
  it directly, and under set -euo pipefail a missing label there aborts
  that step's loop entirely, silently skipping every remaining PR behind
  the one that failed. No repo had hit this yet only because no label in
  the ai:*/ai-* namespace existed at all before now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 23, 2026 13:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026 •

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Changes

AI workflow updates

Layer / File(s) Summary
Control label provisioning
.github/workflows/daily_ai_fix.yaml
Adds OIDC permission access and ensures ai:blocked is included with the existing AI control labels.
Manual retriage entry
.github/workflows/on_issues_ai_triage.yaml
Adds manual issue dispatch, uses the supplied issue number throughout the workflow, bypasses automatic-event guards, and skips the issue-event wait.
Triage verdict application
.github/workflows/on_issues_ai_triage.yaml
Updates confidence-based labeling and control-label cleanup, removes the dry-run exit path, and uses best-effort label creation without --force.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

Suggested labels: bug, enhancement

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title concisely reflects the main workflow changes: removing dry-run behavior and adding self-provisioning for the ai:blocked control label.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 105543690e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/on_issues_ai_triage.yaml Outdated
Comment thread .github/workflows/on_issues_ai_triage.yaml Outdated
alexbelgium and others added 2 commits July 23, 2026 15:35
A manual AI fix sweep failed with "Could not fetch an OIDC token. Did you
remember to add id-token: write to your workflow permissions?". The action
mints a GitHub OIDC token to authenticate the claude_code_oauth_token flow,
which needs id-token: write — absent from both jobs' permissions. Tier 2
failed on it now; tier 1 would have failed identically the first time it
ran live. Added to both.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a workflow_dispatch trigger with a required `issue` input to
on_issues_ai_triage.yaml, so a specific (existing) issue can be triaged
on demand instead of only on issues.opened.

- Every issue-number reference now reads
  `github.event.issue.number || inputs.issue`, so it resolves from the
  event on the auto path and from the input on manual dispatch.
- The job's auto-trigger guards (skip the maintainer's own issues, honour
  no-ai) are bypassed on workflow_dispatch — a manual run is a deliberate
  override.
- The 60s ping_submitter wait is skipped on manual dispatch; there's no
  race to lose against an issue whose ping already landed.

The input flows only through env vars and expression contexts, never
inline into a run: block, so there's no shell-injection surface; a bad
number just fails `gh issue view` cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alexbelgium and others added 3 commits July 23, 2026 15:38
…spatch

feat: manual tier-1 triage of a single issue number
Two findings from Codex review of #2899, both live now that DRY_RUN is gone:

- A low-confidence `addon-bug` had ai:needs-human set by the low branch and
  then ai-triage appended right back unconditionally, so it would enter the
  unattended tier-2 fix pass despite Rule 2 saying an uncertain call should
  only flag a human. Guard the ai-triage add on CONF != low.
- The label-create loop used `--force`, which updates existing labels; with
  a model-supplied cosmetic label like `bug` that already exists, triage
  recolored it to ededed as a side effect. Drop `--force` so existing labels
  are left untouched (create fails harmlessly via || true) while missing
  ones are still created.

tier 2's own label step keeps --force intentionally: its list is a fixed
set of workflow-owned ai:* labels meant to be gray, not model input.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified: gh issue edit --add-label=... is purely additive, and the
"owned" branch exited without touching labels at all — so a manual
workflow_dispatch re-triage that changes the verdict (e.g. a prior
addon-bug run now comes back needs-info, upstream-bug, or owned) left
the old ai-triage label in place, and daily_ai_fix.yaml would still pick
the issue up for the unattended fix pass despite the fresh verdict.

Both label-applying paths now also remove whichever of
ai-triage/ai:classified/ai:needs-human this run did NOT re-apply, as a
separate best-effort call that can't block the add. Simulated every
verdict/confidence transition, including the reported case (addon-bug ->
needs-info): ai-triage is now correctly removed instead of left stale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/on_issues_ai_triage.yaml:
- Around line 130-139: Update the verdict-label handling around the `ai-triage`
append guard to remove any existing `ai-triage` label whenever the verdict is
not eligible: `VERDICT` must be `addon-bug` and `CONF` must not be `low`.
Preserve adding the label for eligible verdicts and keep `ai:classified`
assignment unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d3b285d-5072-4c3a-af25-9bfe1beb0e6b

📥 Commits

Reviewing files that changed from the base of the PR and between 2ff5753 and 82085bc.

📒 Files selected for processing (2)
  • .github/workflows/daily_ai_fix.yaml
  • .github/workflows/on_issues_ai_triage.yaml

Comment thread .github/workflows/on_issues_ai_triage.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants