ci: use a PAT to resolve stale comment-cop review threads

[robobun]

Problem

Comment Cop is supposed to auto-resolve its own review threads once the flagged comment block disappears from the PR diff. That has never worked: every resolveReviewThread call fails with

Request failed due to following response errors:
 - Resource not accessible by integration

so stale bot threads accumulate on every claude-labeled PR and get cleaned up by hand. Example: run 31002896415 on #36956 logged 64 consecutive resolveReviewThread failed warnings; earlier runs show the same, so it is not transient.

Cause

GitHub rejects the resolveReviewThread GraphQL mutation specifically for the Actions GITHUB_TOKEN, even though the workflow grants pull-requests: write (posting review comments through REST works fine with the same token). The restriction does not apply to app tokens in general (CodeRabbit's installation token resolves threads on this repo) or to user tokens: I verified by running the exact mutation as robobun against one of the threads from that run (PRRT_kwDOFVKCyc6Wol1B), which succeeded, while the workflow's attempt on the same thread had failed.

Fix

Split thread resolution out of the scan step:

• The scan step keeps using GITHUB_TOKEN to read the diff and post review comments, and now exports the stale thread ids as a step output instead of resolving them.
• A new step runs the resolveReviewThread mutations authenticated with the existing ROBOBUN_TOKEN repo secret, the same PAT release.yml already uses. The step is non-blocking (continue-on-error), but if every mutation fails it marks itself failed so a dead or under-scoped token surfaces as an error annotation in the run instead of repeating the silent-warning failure mode this PR fixes.

Since the resolve step now acts with real write authority, thread selection also got stricter: a thread only counts as comment-cop's if its root comment is authored by github-actions and carries the <!-- comment-cop:... --> marker. Previously the marker alone was trusted, and anyone can paste the marker into their own review comment.

The PAT's exposure stays narrow: the workflow checks out no PR code, the resolve step receives only GitHub-generated thread ids through env (never interpolated into the script), and the only operation it performs is resolving threads the bot itself opened. Fork PRs cannot trigger this workflow on their own: the claude label requires triage access to add, and the auto-labeler runs with a read-only token on fork PRs.

Token choice

ROBOBUN_TOKEN is used because it already exists and demonstrably works. It is broader than this job needs (release.yml uses it to push to other oven-sh repos). If narrower credentials are preferred, either of these is a drop-in replacement for github-token in the resolve step, and both need someone with org access to provision:

• a fine-grained PAT on robobun scoped to oven-sh/bun with Pull requests read/write, or
• a dedicated GitHub App via actions/create-github-app-token (app installation tokens can run this mutation; only the Actions token is blocked).

Verification

• Ran the mutation as a robobun user token against a real thread from the failing run: resolve and unresolve both succeed, confirming the restriction is specific to the Actions token.
• Confirmed ROBOBUN_TOKEN is a plain repo secret already consumed by non-environment jobs in release.yml, so it is available here.
• Dry-ran the new thread-selection logic against the live thread set of transpiler: compile zod v4 schemas into lazy wrappers with flat validators #36956: exactly the 64 genuine cop threads match (the same 64 the failing run tried to resolve), and the 5 human and other-bot threads are skipped.
• Validated the YAML and syntax-checked both embedded scripts.

Note: pull_request_target runs the workflow from the base branch, so CI on this PR cannot exercise the change; it takes effect once merged. The CI failures on this PR are pre-existing on main (asan lane) and unrelated to this workflow-only diff.

---

no test proof · iteration 0 · build/CI scripts only; test-proof not applicable

[coderabbitai]

Walkthrough

The workflow identifies trusted stale review threads by root-comment author, emits their IDs from the scan step, and resolves them in a separate PAT-authenticated step with partial-failure reporting.

Changes

Comment-cop stale-thread handling

Layer / File(s)	Summary
Scan and emit trusted stale threads .github/workflows/comment-cop.yml	The scan queries root-comment authors and emits unresolved stale thread IDs only for threads opened by github-actions.
Resolve stale threads with PAT authentication .github/workflows/comment-cop.yml	A conditional step uses ROBOBUN_TOKEN to resolve stale threads, reports individual failures, fails when all mutations fail, and continues the job.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: using a PAT to resolve stale Comment Cop review threads.
Description check	✅ Passed	The description explains the problem, cause, fix, security considerations, token choice, and verification steps in sufficient detail.

---

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

I didn't find any bugs, and the security posture looks sound (no PR-code checkout, thread IDs are GitHub-generated node IDs passed via env: rather than interpolated into the script, label-gated trigger, continue-on-error). That said, this introduces a PAT into a pull_request_target workflow — the classic pwn-request surface — so a human should sign off on the credential exposure.

What was reviewed

• Injection: STALE_THREADS is passed via env: and read with process.env, not ${{ }} inside the script; IDs originate from GitHub's GraphQL reviewThreads.nodes[].id.
• Scope: PAT step only runs resolveReviewThread on IDs whose root comment carries the <!-- comment-cop:... --> marker; scan step still uses GITHUB_TOKEN.
• Control flow: core.setOutput runs before the early return, and an empty toResolve yields '' so the if: guard skips the resolve step.
• Confirmed ROBOBUN_TOKEN is an existing repo secret already consumed by release.yml.

Extended reasoning...

Overview

This PR modifies .github/workflows/comment-cop.yml — a pull_request_target workflow that scans claude-labeled PRs for multi-line comment blocks and posts review comments. The change splits the previously-broken "resolve stale threads" logic into its own step authenticated with the ROBOBUN_TOKEN PAT instead of the Actions GITHUB_TOKEN, because GitHub rejects the resolveReviewThread GraphQL mutation from integration tokens. The scan step now exports stale thread IDs via core.setOutput; a new step consumes them via an env: var and runs the mutation with per-ID try/catch and continue-on-error.

Security risks

This is the crux. pull_request_target workflows run with base-repo secrets and can be triggered by fork PRs, so exposing a PAT here is the canonical pwn-request risk surface. I walked through it and the design is defensible:

• No PR code is checked out or executed. The scan step reads the diff via github.rest.pulls.listFiles and parses patch text; nothing from the PR reaches a shell or eval.
• The PAT step receives only GitHub-generated node IDs (reviewThreads.nodes[].id, e.g. PRRT_...), filtered to threads whose root comment carries the bot's own <!-- comment-cop:KEY --> marker. An attacker cannot control these values.
• No script injection: the output is passed via env: STALE_THREADS: ${{ ... }} and read with process.env.STALE_THREADS.split(' '), not interpolated into the script body.
• Label-gated: the job only runs on PRs carrying the claude label, which requires triage/write to add.
• Blast radius of misuse is tiny: even if an attacker could somehow inject arbitrary IDs, the only operation is resolveReviewThread — resolving review threads, not reading secrets or writing code.

I did not find a path by which PR-controlled content reaches the PAT-authenticated step in a way that could exfiltrate or misuse the token. Nonetheless, granting a workflow access to a user PAT in a fork-triggerable context is exactly the kind of decision a maintainer should explicitly ratify rather than have auto-approved.

Level of scrutiny

High. Workflow files under pull_request_target that touch secrets are one of the most common sources of supply-chain compromise in GitHub Actions. The change itself is mechanically simple (~30 lines net), and the correctness is easy to verify — core.setOutput is called before the early return, empty toResolve produces '' which is falsy in the if: expression so the step is skipped, and .split(' ').filter(Boolean) handles edges. But "simple" and "security-sensitive" are orthogonal.

Other factors

• ROBOBUN_TOKEN is already a repo secret consumed by release.yml (lines 209, 312, 317), so this isn't introducing a new secret — but release.yml is not fork-triggerable, whereas this workflow is.
• No CODEOWNERS entry covers .github/workflows/.
• The PR description accurately describes the security posture and the author clearly thought it through; I'm deferring on the principle that credential-scope changes in pull_request_target warrant human sign-off, not because I found a flaw.

[robobun]

Agreed that the credential question is the part that needs a human decision, which is why the PR is assigned to Jarred rather than self-merging. To restate the trade-off in one place: the workflow never checks out or executes PR code, the PAT step receives only GitHub-generated thread ids through env (never interpolated into the script), and the only mutation it can run is resolveReviewThread on threads the bot itself opened. If exposing ROBOBUN_TOKEN here is still unwanted, the fallback would be deleting the stale bot comments over REST with the integration token, at the cost of losing thread history and any human replies.

[robobun]

Pushed a second commit addressing what came out of a self-review pass:

• Thread selection now requires the root comment to be authored by github-actions, not just to carry the marker. The marker is forgeable (anyone can paste it into their own review comment), and with a PAT behind the resolve step that would have let someone point the bot at their own threads, or suppress new cop comments by replaying a live key. Dry-ran the stricter filter against the 69 threads of transpiler: compile zod v4 schemas into lazy wrappers with flat validators #36956: the 64 genuine cop threads match, the 5 human and other-bot threads are skipped.
• If every mutation fails, the resolve step now marks itself failed (still non-blocking through continue-on-error), so an expired or revoked token surfaces as an error annotation instead of the silent warnings that hid the original breakage.
• Corrected the cause wording in the body: the rejection is specific to the Actions GITHUB_TOKEN. App installation tokens can run this mutation (CodeRabbit resolves threads on this repo), so a dedicated GitHub App is a viable narrower-credential option alongside a scoped fine-grained PAT; both are listed under token choice in the body and need org-side provisioning.

On CI: the two red tests on the asan lane (worker-transfer-terminate-stress SIGABRT, AsyncLocalStorage-tracking leak) are pre-existing on main and unrelated to this workflow-only diff, which Buildkite does not execute anyway.

[claude]

No issues found in the implementation, but this exposes ROBOBUN_TOKEN (a broad PAT that release.yml uses to push to other oven-sh repos) to a pull_request_target workflow — a credential-scope decision that a maintainer should sign off on, as the PR description itself notes.

What was reviewed:

• Injection surface: thread IDs are read from process.env, not interpolated into the script; the scan step passes only GitHub-generated GraphQL node IDs.
• pull_request_target exposure: workflow file runs from base branch, no checkout of PR code, gated on the claude label.
• Author-check hardening (github-actions login) correctly narrows which threads the PAT step will act on.

Extended reasoning...

Overview

This PR modifies .github/workflows/comment-cop.yml to fix stale-thread auto-resolution, which has never worked because GitHub's Actions GITHUB_TOKEN cannot call the resolveReviewThread GraphQL mutation. The fix splits resolution into a separate step authenticated with secrets.ROBOBUN_TOKEN, passes stale thread IDs via a step output, adds an author check so only threads opened by github-actions are considered, and marks the resolve step failed (non-blocking via continue-on-error) if every mutation fails.

Security risks

The material change is exposing ROBOBUN_TOKEN to a pull_request_target-triggered job. That token is currently used in release.yml to push to oven-sh/homebrew-bun, so it has write access beyond this repo. The mitigations are sound: the workflow file runs from the base branch (attackers can't edit it), no PR code is checked out or executed, the resolve step only receives GitHub-generated node IDs via env (never string-interpolated into the script body), and thread selection now requires both the github-actions author and the marker. The claude label gate requires triage access. I did not find an injection or exfiltration path. However, the PR description explicitly flags that ROBOBUN_TOKEN is broader than needed and offers two narrower alternatives (a fine-grained PAT or a GitHub App installation token) that need org-side provisioning — that policy trade-off is a human call.

Level of scrutiny

High. Any change that puts a repo secret with cross-repo write scope behind a PR-triggered event deserves a maintainer's explicit approval, even when the implementation is careful. The author's own comment on the PR ("the credential question is the part that needs a human decision, which is why the PR is assigned to Jarred rather than self-merging") agrees.

Other factors

The logic changes themselves look correct: core.setOutput + if: steps.scan.outputs.stale-threads skips the resolve step when the output is empty, .split(' ').filter(Boolean) is defensive against that anyway, and the total-failure check surfaces a dead token without blocking the job. The author-login check tightens a real (if minor) forgery gap that mattered more once a PAT is behind the action. No automated test is expected for a workflow-only change; the PR documents manual verification against a live failing run.

[robobun]

Heads-up from #39183: that PR makes comment-cop's scan step REST-only. It replaces the reviewThreads GraphQL query (the step was crashing on it whenever the repo's shared GraphQL quota was exhausted, 33 runs on Aug 15) with pulls.listReviewComments for dedup, and deletes the GITHUB_TOKEN resolve loop, since those mutations all fail today as this PR describes.

If #39183 lands first, the rebase here would need the reviewThreads lookup to move into the ROBOBUN_TOKEN step as well (the scan step can hand over the stale keys, i.e. the markers on the PR whose block is no longer in the diff, instead of thread ids), which also keeps that query off the exhausted quota.

[robobun]

The premise here needs a correction: the Actions token is not blocked from resolveReviewThread. It is under-scoped. GitHub gates that mutation on the contents permission, not on pull-requests, and this workflow grants contents: read.

Evidence: open-code-review#27 ran one script twice, a minute apart, against its own review thread. With contents: read + pull-requests: write the mutation failed with Resource not accessible by integration (run 31245782000). With contents: write + pull-requests: write it succeeded (run 31245826522). Community discussion 44650 reports the same for GitHub App tokens.

So the PAT is not needed. #40468 makes the one-line permission change, counts the mutations that succeed in the summary line, and adds a source lint that pins the permission. The GITHUB_TOKEN expires with the job and is scoped to this repo, which is a smaller exposure than ROBOBUN_TOKEN in a pull_request_target workflow. I suggest closing this PR in favor of #40468.

[robobun]

Correction to my comment above: #40468 is closed. A review of that change found the stale threads already get resolved without the workflow's help (all 61 comment-cop threads on the 15 most recent merged claude PRs are resolved, 60 of them by robobun), so neither a PAT nor a wider GITHUB_TOKEN buys anything. The direction for this workflow is #39183, which removes the resolve loop. I suggest closing this PR in favor of #39183.

[robobun]

Closing this PR in favor of #39183.

This PR makes the comment-cop resolve loop work by running the resolveReviewThread mutation with a PAT. #39183 removes that loop and the GraphQL query behind it, so the scan step survives GraphQL quota exhaustion (33 failed runs on Aug 15). The two changes cannot both land.

The review of #40468 showed the loop is not needed. On the 15 most recent merged claude PRs, all 61 comment-cop threads are resolved, 60 of them by robobun's own review sweep and 1 by a maintainer. The set this loop targets (unresolved and outdated) was empty on every PR sampled. So neither a PAT nor a wider GITHUB_TOKEN changes the outcome, and a long-lived PAT in a pull_request_target workflow is the larger exposure of the two.

For the record, the original failure has a known cause: GitHub gates resolveReviewThread on contents: write, not on pull-requests: write.