Skip to content

Determine sub-agent count in pre-script based on PR size#510

Closed
Copilot wants to merge 4 commits intomainfrom
copilot/determine-agent-count
Closed

Determine sub-agent count in pre-script based on PR size#510
Copilot wants to merge 4 commits intomainfrom
copilot/determine-agent-count

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 1, 2026

This change makes PR review fan-out deterministic by deciding sub-agent usage in the pre-script from PR size, then writing explicit runtime instructions for the reviewer to follow.

What changed

1) Pre-script now computes review strategy from file count

In .github/workflows/gh-aw-fragments/pr-context.md, the pre-script now:

  • Computes FILE_COUNT from /tmp/pr-context/files.json
  • Computes AGENT_COUNT using fixed thresholds
  • Writes /tmp/pr-context/agent_count.txt
  • Writes /tmp/pr-context/review-strategy.md with exact instructions for one of three modes:
    • Direct review (no sub-agents)
    • 2 sub-agents (A→Z and Z→A)
    • 3 sub-agents (A→Z, Z→A, largest-first)

It also adds agent_count.txt and review-strategy.md to the generated /tmp/pr-context/README.md manifest.

2) Review prompt now consumes precomputed strategy

In .github/workflows/gh-aw-pr-review.md:

  • Step 2 is simplified to read and follow /tmp/pr-context/review-strategy.md (instead of hard-coding 3 sub-agents)
  • Step 3 now states dedup/merge is conditional on sub-agents being used

Thresholds

Files changed Sub-agents Strategy
<= 10 0 Parent agent reviews directly
11-20 2 A→Z + Z→A
> 20 3 A→Z + Z→A + largest-first

Generated workflow updates

  • .github/workflows/gh-aw-pr-review.lock.yml includes the compiled logic/prompt updates above
  • Other lock files that import the updated fragment were regenerated and only had metadata hash refreshes

Fixes elastic/ai-github-actions#508.

Generated by Update PR Body for issue #510

Add dynamic sub-agent count to the pr-context pre-script:
- ≤10 files: direct review (no sub-agents)
- 11-20 files: 2 sub-agents (A→Z, Z→A)
- >20 files: 3 sub-agents (A→Z, Z→A, largest-first)

The pre-script writes /tmp/pr-context/review-strategy.md with
precise instructions so the agent follows a deterministic path.
The PR review workflow reads this strategy file instead of always
spawning 3 sub-agents.

Recompiled all affected lock files.

Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>
Copilot AI changed the title [WIP] Add agent count determination in pre-script Determine sub-agent count in pre-script based on PR size Mar 1, 2026
@strawgate strawgate marked this pull request as ready for review March 1, 2026 02:52
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 1, 2026

📝 Walkthrough

Walkthrough

This change introduces dynamic sub-agent sizing for PR reviews based on the number of changed files. The PR context generation now calculates a FILE_COUNT and assigns AGENT_COUNT values (0 for ≤10 files, 2 for 11–20 files, 3 for >20 files), writing these values to agent_count.txt. A new review-strategy.md artifact is generated with conditional strategy sections for Direct Review (no sub-agents), 2-agent review, or 3-agent review depending on PR size. Workflow files are updated to consume this strategy document instead of hard-coded instructions, and the PR context manifest is expanded to include these new precomputed artifacts.

Possibly related PRs

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed PR implements all core requirements from issue #508: computes PR size in prescript, applies three-tier thresholds (≤10, 11-20, >20 files), generates deterministic strategy instructions, and integrates with review workflow.
Out of Scope Changes check ✅ Passed All changes align with issue #508 objectives: pre-script sizing logic, strategy document generation, manifest updates, and review workflow alignment. No unrelated modifications detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch copilot/determine-agent-count

Comment @coderabbitai help to get the list of available commands and usage tips.

- Instruction to read changed files from the workspace (the PR branch is checked out)

Each sub-agent returns a structured findings list. They do NOT leave inline comments.
Read `/tmp/pr-context/review-strategy.md` for the pre-computed review strategy. The strategy is determined by PR size and specifies the exact number of sub-agents (0, 2, or 3). Follow the instructions in that file exactly.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[HIGH] Conflicting instructions still force three sub-agents

This new strategy step says to follow /tmp/pr-context/review-strategy.md (which can require 0 or 2 agents), but the workflow still imports gh-aw-fragments/pick-three-keep-many.md, whose text explicitly says “Spawn 3 sub-agents” and “Wait for all 3 sub-agents”.

Because both instructions are present in the same prompt, the model can still follow the hardcoded “3 sub-agents” rule, which defeats the deterministic sizing behavior introduced here.

Please make the prompt sources consistent (for example, gate/remove the pick-three-keep-many fragment when strategy says direct review, or rewrite that fragment to be conditional on review-strategy.md).

@strawgate
Copy link
Copy Markdown
Contributor

/ai resolve pr issues

github-actions Bot and others added 2 commits March 1, 2026 03:08
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
.github/workflows/gh-aw-pr-review.md (1)

125-125: ⚠️ Potential issue | 🟠 Major

Resolve conflicting sub-agent instructions in the prompt.

Line 125 makes orchestration dynamic, but this prompt still imports gh-aw-fragments/pick-three-keep-many.md (Line 16). If that fragment still contains fixed 3-agent orchestration text, the agent receives contradictory directives and deterministic PR-size fan-out can fail.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/gh-aw-pr-review.md at line 125, The prompt imports a fixed
3-agent fragment (gh-aw-fragments/pick-three-keep-many.md) while orchestration
is supposed to be dynamic via the pre-computed review strategy (read
`/tmp/pr-context/review-strategy.md`), causing conflicting directives; update
the prompt assembly in gh-aw-pr-review.md so the pick-three-keep-many fragment
is only included when the strategy explicitly requires three sub-agents, or
replace the fixed fragment import with logic that reads
`/tmp/pr-context/review-strategy.md` and conditionally includes the appropriate
fragment (0, 2, or 3-agent fragment names) to ensure the chosen orchestration is
deterministic and consistent with the review strategy.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In @.github/workflows/gh-aw-pr-review.md:
- Line 125: The prompt imports a fixed 3-agent fragment
(gh-aw-fragments/pick-three-keep-many.md) while orchestration is supposed to be
dynamic via the pre-computed review strategy (read
`/tmp/pr-context/review-strategy.md`), causing conflicting directives; update
the prompt assembly in gh-aw-pr-review.md so the pick-three-keep-many fragment
is only included when the strategy explicitly requires three sub-agents, or
replace the fixed fragment import with logic that reads
`/tmp/pr-context/review-strategy.md` and conditionally includes the appropriate
fragment (0, 2, or 3-agent fragment names) to ensure the chosen orchestration is
deterministic and consistent with the review strategy.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4e93b38 and 7b2304a.

📒 Files selected for processing (9)
  • .github/workflows/gh-aw-fragments/pr-context.md
  • .github/workflows/gh-aw-mention-in-pr-by-id.lock.yml
  • .github/workflows/gh-aw-mention-in-pr-no-sandbox.lock.yml
  • .github/workflows/gh-aw-mention-in-pr.lock.yml
  • .github/workflows/gh-aw-pr-review-addresser.lock.yml
  • .github/workflows/gh-aw-pr-review.lock.yml
  • .github/workflows/gh-aw-pr-review.md
  • github/workflows/gh-aw-fragments/pick-three-keep-many.md
  • github/workflows/gh-aw-pr-review.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants