feat: add agents-verifier workflow for consumer repos#192
Conversation
- Create reusable-agents-verifier.yml that consumer repos can call - Add agents-verifier.yml thin caller to consumer template - Uses dual checkout pattern to get scripts from Workflows repo The verifier runs post-merge on PRs to check if acceptance criteria were met. If not, it opens a follow-up issue in the consumer repo.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Automated Status SummaryHead SHA: 04ef412
Coverage Overview
Coverage Trend
Updated automatically; will refresh on subsequent CI/Docker completions. Keepalive checklistScopeNo scope information available Tasks
Acceptance criteria
|
🤖 Keepalive Loop StatusPR #192 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
There was a problem hiding this comment.
Pull request overview
This PR adds the Agents Verifier workflow to consumer repository templates, enabling automated post-merge verification of acceptance criteria. The workflow was previously only in the Workflows repository but is now being made available to consumer repos where the original issues and PRs exist.
Key Changes
- Created a new reusable workflow that waits for CI completion, builds verification context from merged PRs and linked issues, runs Codex in verifier mode, and opens follow-up issues if criteria aren't met
- Added a consumer template that calls the reusable workflow with appropriate permissions and configuration
- Integrated metrics collection and reporting for verifier runs
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
.github/workflows/reusable-agents-verifier.yml |
New reusable workflow implementing the complete verifier logic including CI wait, context building, Codex execution, follow-up issue creation, and metrics collection |
templates/consumer-repo/.github/workflows/agents-verifier.yml |
Consumer template that triggers on merged PRs and delegates to the reusable workflow with configured CI workflows to wait for |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Add new workflow to: - EXPECTED_NAMES in test_workflow_naming.py - Reusable workflow table in WORKFLOWS.md - Prose documentation in WORKFLOWS.md - Primary workflows list in WORKFLOW_SYSTEM.md
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
|
Status | ✅ autofix updates applied |
- Add continue-on-error: true to Run verifier step so subsequent parsing and issue creation steps run even if Codex crashes - Detect Codex failures and set verdict='error' to create follow-up issues for infrastructure failures, not just criteria failures - Fix checks_run counting to use checkbox syntax (- [x]) instead of counting all bullet points, matching the intended behavior - Update count_checkboxes regex to match both dash (-) and asterisk (*) list markers for consistency with the JavaScript implementation Addresses comments from copilot-pull-request-reviewer.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Summary
Adds the Agents Verifier workflow to the consumer repo template so it runs post-merge in consumer repos.
Changes
.github/workflows/reusable-agents-verifier.yml- New reusable workflow that:templates/consumer-repo/.github/workflows/agents-verifier.yml- Thin caller that consumer repos useWhy
The verifier workflow was only in the Workflows repo, but it makes more sense in consumer repos because:
Usage
Consumer repos need to:
agents-verifier.ymlto their.github/workflows/CODEX_AUTH_JSONsecret is setThe workflow automatically triggers when PRs are merged.