Skip to content

fix(review): collect GitHub Copilot reviewer threads #3353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stranske merged 2 commits into from
Sep 4, 2026
Merged

fix(review): collect GitHub Copilot reviewer threads #3353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e2fa993
fix(review): collect GitHub Copilot threads
Sep 4, 2026
6df141a
fix(review): cover Copilot reviewer bot variant
Sep 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/scripts/__tests__/bot-comment-handler.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ const REGISTRY_PATH = path.resolve(__dirname, '..', '..', 'agents', 'registry.ym
test('default bot author allowlist recognizes canonical review bots', () => {
for (const login of [
'copilot[bot]',
'copilot-pull-request-reviewer',
'copilot-pull-request-reviewer[bot]',
'github-actions[bot]',
'coderabbitai[bot]',
Comment thread
Copilot marked this conversation as resolved.
'chatgpt-codex-connector',
Expand Down
2 changes: 2 additions & 0 deletions .github/scripts/bot-comment-handler.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ const MAX_CONTROLLER_COMMENT_LENGTH = 60000;
const MAX_COLLECTED_COMMENT_OUTPUT_LENGTH = 450000;
const DEFAULT_BOT_AUTHORS = Object.freeze([
'copilot[bot]',
'copilot-pull-request-reviewer',
'copilot-pull-request-reviewer[bot]',
'github-actions[bot]',
'coderabbitai[bot]',
'chatgpt-codex-connector',
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/agents-bot-comment-handler.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,7 @@ jobs:
with:
pr_number: ${{ needs.resolve.outputs.pr_number }}
dry_run: ${{ inputs.dry_run == true }}
ignored_paths: '.agents/,scripts/langchain/prompts/,docs/,templates/'
ignored_paths: '.agents/,scripts/langchain/prompts/,docs/'
secrets:
service_bot_pat: ${{ secrets.SERVICE_BOT_PAT }}
gh_app_client_id: ${{ secrets.WORKFLOWS_APP_CLIENT_ID }}
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/reusable-bot-comment-handler.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ on:
required: false
type: string
default: >-
Copilot,copilot[bot],github-actions[bot],coderabbitai[bot],
Copilot,copilot[bot],copilot-pull-request-reviewer,
copilot-pull-request-reviewer[bot],
github-actions[bot],coderabbitai[bot],
chatgpt-codex-connector,chatgpt-codex-connector[bot]
Comment thread
Copilot marked this conversation as resolved.
skip_if_human_replied:
description: 'Legacy option: skip active threads after any non-bot reply'
Expand Down Expand Up @@ -68,7 +70,9 @@ on:
required: false
type: string
default: >-
Copilot,copilot[bot],github-actions[bot],coderabbitai[bot],
Copilot,copilot[bot],copilot-pull-request-reviewer,
copilot-pull-request-reviewer[bot],
github-actions[bot],coderabbitai[bot],
chatgpt-codex-connector,chatgpt-codex-connector[bot]
skip_if_human_replied:
description: 'Legacy option: skip active threads after any non-bot reply'
Expand Down
2 changes: 1 addition & 1 deletion docs/WORKFLOW_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ _Inline Gate helper_
- **`reusable-20-pr-meta.yml`** — PR metadata/keepalive reusable for consumer repos. Dual-checks out the consumer repo + Workflows scripts, detects keepalive activations from comments/Gate runs, updates PR body sections, and dispatches the keepalive orchestrator using the provided PATs/default token (no App mint required).
- **`reusable-70-orchestrator-init.yml`** — Preflight stage for Agents 70 orchestrator. Probes the load-balanced credential pool via `setup-api-client` + `checkRateLimitStatus` (fail-closed to `unknown` on probe errors), then repeats the gate on the exact wrapped client consumed by idle-precheck. Both gates preserve a 15% pool reserve plus forecast demand; the workflow exports the consuming `credential_pool_id` and `rate_limit_state` and resolves the remaining orchestrator parameters.
- **`reusable-70-orchestrator-main.yml`** — Executes the orchestrator stages (keepalive gate, readiness, preflight, diagnostics, bootstrap, watchdog, keepalive sweep, belt scan/automerge) using the outputs from the init workflow. The belt scan creates one wrapped client, probes that exact client, and only then runs the scan through the same instance; unknown or under-reserve pools start no scan. Requires the GitHub App token when PATs aren’t available so keepalive writes still run under `agents-workflows-bot`.
- **`reusable-bot-comment-handler.yml`** — Collects the complete active, non-outdated bot `reviewThreads` set, generates an exact-head per-agent prompt, and persists the thread IDs and acceptance criteria in size-bounded controller comments. Immediately before the triggering assignment it re-queries both the head and complete active-thread set and fails closed for recollection if either changed. Manual callers retain `autofix:bot-comments` when that dispatch aborts, so a later event can recollect instead of losing the trigger. Human replies do not hide active review debt by default, uneditable comments from older credentials cannot block dispatch, and generic top-level reviews are not completion evidence. The handler prefers GitHub App client ID auth, records the selected App auth mode, keeps a warning-only legacy App ID fallback, and still falls back to `service_bot_pat` or `GITHUB_TOKEN` so consumer repos don’t have to configure extra secrets.
- **`reusable-bot-comment-handler.yml`** — Collects the complete active, non-outdated bot `reviewThreads` set, including GitHub Copilot's `copilot-pull-request-reviewer` identity variants, generates an exact-head per-agent prompt, and persists the thread IDs and acceptance criteria in size-bounded controller comments. Immediately before the triggering assignment it re-queries both the head and complete active-thread set and fails closed for recollection if either changed. Manual Workflows callers keep source-template review threads in scope and retain `autofix:bot-comments` when dispatch aborts, so a later event can recollect instead of losing the trigger. Human replies do not hide active review debt by default, uneditable comments from older credentials cannot block dispatch, and generic top-level reviews are not completion evidence. The handler prefers GitHub App client ID auth, records the selected App auth mode, keeps a warning-only legacy App ID fallback, and still falls back to `service_bot_pat` or `GITHUB_TOKEN` so consumer repos don’t have to configure extra secrets.
- **`reusable-codex-run.yml`** — Codex execution wrapper that checks out the target PR branch, installs the pinned Codex CLI, runs the prompt, and pushes commits when the GitHub App token is available (otherwise drops to read-only mode using `GITHUB_TOKEN`).
- **`reusable-model-profile-trial.yml`** — Dedicated read-only Codex trial
worker. It executes an immutable helper checkout separately from the target,
Expand Down
2 changes: 2 additions & 0 deletions docs/bot-comment-handler.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ The workflow reads the PR's labels to determine which agent to use:

By default, the workflow processes comments from:
- `copilot[bot]` - GitHub Copilot code review
- `copilot-pull-request-reviewer` - GitHub Copilot pull-request review
- `copilot-pull-request-reviewer[bot]` - GitHub Copilot reviewer bot variant
- `github-actions[bot]` - GitHub Actions (lint, type check suggestions)
- `coderabbitai[bot]` - CodeRabbit AI review
- `chatgpt-codex-connector` - Codex connector review
Expand Down
2 changes: 2 additions & 0 deletions templates/consumer-repo/.github/scripts/bot-comment-handler.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ const MAX_CONTROLLER_COMMENT_LENGTH = 60000;
const MAX_COLLECTED_COMMENT_OUTPUT_LENGTH = 450000;
const DEFAULT_BOT_AUTHORS = Object.freeze([
'copilot[bot]',
'copilot-pull-request-reviewer',
'copilot-pull-request-reviewer[bot]',
'github-actions[bot]',
'coderabbitai[bot]',
'chatgpt-codex-connector',
Expand Down
10 changes: 10 additions & 0 deletions tests/workflows/test_bot_comment_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,19 @@ def test_reusable_bot_comment_handler_ignores_agents_paths() -> None:
assert ".agents/" in ignored_paths.split(",")

bot_authors = inputs.get("bot_authors", {}).get("default", "")
assert "copilot-pull-request-reviewer" in bot_authors
assert "copilot-pull-request-reviewer[bot]" in bot_authors
assert "chatgpt-codex-connector[bot]" in bot_authors
Comment thread
Copilot marked this conversation as resolved.


def test_canonical_bot_comment_handler_keeps_source_templates_in_scope() -> None:
workflow = _load_yaml(ROOT / ".github/workflows/agents-bot-comment-handler.yml")
ignored_paths = workflow["jobs"]["handle"]["with"]["ignored_paths"].split(",")

assert ".agents/" in ignored_paths
assert "templates/" not in ignored_paths


def test_reusable_bot_comment_handler_revalidates_active_threads_before_dispatch() -> None:
workflow_path = ROOT / ".github/workflows/reusable-bot-comment-handler.yml"
workflow = _load_yaml(workflow_path)
Expand Down
Loading