Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 42 additions & 22 deletions .github/workflows/ci-failure-analyst.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
---
on:
workflow_run:
workflows: [CI, Lint, Tests]
check_run:
types: [completed]

permissions:
actions: read
checks: read
contents: read
pull-requests: read

engine: claude

Expand All @@ -24,38 +22,60 @@ network: defaults
safe-outputs:
add-comment:
max: 1
staged: true
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
---

# CI Failure Analyst

When a CI workflow run completes, diagnose failures and post a root-cause comment on the associated pull request.
When a CI check run completes with `conclusion: failure` on a non-fork PR, diagnose the failure
and post a single root-cause comment on the associated pull request.

> **Deployed runtime.** The live CI Failure Analyst in this repo is **not** a compiled
> gh-aw workflow — it is the byte-frozen ring-0 thin caller
> `.github/workflows/ci-failure-analyst.lock.yml`, which triggers on `check_run:[completed]`,
> posts **live** (declaring `issues: write` + `pull-requests: write`), and delegates all logic to
> `ci-failure-analyst-reusable.yml@ci-failure-analyst/v1-stable`. This `.md` is the gh-aw source
> retained alongside that stub (the `gh-aw-compile` orphaned-lock detector requires a sibling
> `.md` for every `*.lock.yml`). The spec's `on:` trigger and comment posture are kept in sync
> with the deployed stub — `check_run:[completed]`, one comment per failing head SHA — but
> permission models differ: the deployed stub carries live `issues: write` + `pull-requests: write`,
> while this spec uses `safe-outputs.add-comment` (gh-aw strict mode disallows raw write
> permissions). Do **not** edit the frozen `.lock.yml` block (caller-stub-freeze,
> AGENTS.md). For the full deployed architecture and permission table see
> [`docs/aw/ci-failure-analyst.md`](../../docs/aw/ci-failure-analyst.md); for the machine-readable
> interaction contract see
> [`interaction-contracts/ci-failure-analyst.yml`](../../interaction-contracts/ci-failure-analyst.yml).

## Instructions

Follow these steps in order.

### 1. Guard: conclusion must be `failure`
### 1. Guard: conclusion must be `failure` and this must not be the analyst's own check

Check `github.event.workflow_run.conclusion`. If it is anything other than `failure`
Check `github.event.check_run.conclusion`. If it is anything other than `failure`
(e.g. `cancelled`, `skipped`, `success`, `neutral`), output a `noop` action and stop.
Do not post any comment.

### 2. Guard: workflow run must be associated with a PR
Also skip the analyst's own check run to prevent loops: if
`github.event.check_run.name` starts with `CI Failure Analyst`, output a `noop` and stop.

Check `github.event.workflow_run.pull_requests`. If the array is empty, the workflow run is
not attached to any open pull request. Output a `noop` and stop. Do not comment on
issues or any other thread.
### 2. Guard: the check run must be associated with a non-fork PR

Check `github.event.check_run.pull_requests`. If the array is empty, resolve the PR from the
commits-to-pulls API (`repos/{owner}/{repo}/commits/{head_sha}/pulls`) as a fallback. If no open
PR is found, output a `noop` and stop. Do not comment on issues or any other thread.

Filter to **non-fork** PRs only (the PR's head repo must equal the base repo). A fork PR is a
`noop`.

### 3. Identify the target PR

Use the first entry in `github.event.workflow_run.pull_requests`. Note:
Use the first non-fork PR. Note:

- PR number
- Head SHA: `github.event.workflow_run.head_sha`
- Workflow name: `github.event.workflow_run.name`
- Details URL: `github.event.workflow_run.html_url`
- Workflow run ID: `github.event.workflow_run.id`
- Head SHA: `github.event.check_run.head_sha`
- Check run name: `github.event.check_run.name`
- Details URL: `github.event.check_run.details_url` (fall back to `github.event.check_run.html_url`)
- Check run ID: `github.event.check_run.id`

### 4. Check for an existing analyst comment (idempotency)

Expand All @@ -65,16 +85,16 @@ List comments on the PR and search for one that starts with the exact marker:
<!-- ci-analyst sha=HEAD_SHA -->
```

where `HEAD_SHA` is the workflow run's `head_sha`. If a comment with this marker already
where `HEAD_SHA` is the check run's `head_sha`. If a comment with this marker already
exists, output a `noop` and stop. Do not post a duplicate comment.

### 5. Fetch failure details

Use the workflow run ID (`github.event.workflow_run.id`) with the actions toolset to retrieve
the workflow run logs. Parse the logs to find the failing job and step.
Use the failed check run / its workflow run with the actions toolset to retrieve the run logs.
Parse the logs to find the failing job and step.

If logs are unavailable (e.g. the run has expired), note this in the comment and
provide a best-guess diagnosis from the workflow name and any available context.
provide a best-guess diagnosis from the check run name and any available context.

### 6. Identify the failing step and error

Expand All @@ -96,7 +116,7 @@ Choose the single best-fit category:
| **Lint/style** | `eslint`, `shellcheck`, `markdownlint`, `yamllint`, `ruff`, `golangci-lint` errors |
| **Build error** | Compilation error, `npm ERR!`, `cargo build` failed, dependency resolution failure |

If the workflow name is `lint`, `shellcheck`, `markdownlint`, `yamllint`, or similar, the
If the check run name is `lint`, `shellcheck`, `markdownlint`, `yamllint`, or similar, the
category **must** be Lint/style — do not classify it as Test failure.

### 8. Post the diagnostic comment
Expand All @@ -105,7 +125,7 @@ Post a comment to the PR (identified in Step 3 by PR number) using exactly this

```markdown
<!-- ci-analyst sha=HEAD_SHA -->
## CI Failure: WORKFLOW_NAME
## CI Failure: CHECK_RUN_NAME

**Step:** FAILING_STEP_NAME
**Root cause:** ROOT_CAUSE_CATEGORY
Expand Down
21 changes: 21 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,27 @@ committed baseline (`tests/fixtures/caller-stub-freeze/*.block`).
- PR-triggered on the existing Lint workflow — **no new cron/scheduled workload.** It must not be removed by
template syncs. If the org template gains an equivalent, remove this exception and defer.

### Agentic interaction model

How agentic roles are triggered and how they interact is a **normative repo-local standard**,
[`docs/agentic-interaction-model.md`](./docs/agentic-interaction-model.md). It codifies the
event-first principle, the three trigger classes (event-driven / backstop-timer /
scheduled-origin), the `GITHUB_TOKEN` event-boundary rule and its two sanctioned bridges, the
timer contract, and the #860 runaway rules. Read it before adding or changing an agentic
workflow's `on:` block, a `schedule.cron`, or a cross-workflow dispatch — a new role must add a
row to its §4 classification table and declare a machine-readable per-role interaction contract
(`interaction-contracts/<name>.yml` and/or `personas/<id>/interaction.yml`), both CI-verified.

- **Adding a role?** Follow the step-by-step runbook
[`docs/adding-an-agentic-role.md`](./docs/adding-an-agentic-role.md); each step cites the
governing section of the standard.
- This section is consistent with, not competing with, "Scheduled workflows" below (which
governs *when* a timer may fire) and the "Ring-0 caller-stub freeze" above (which byte-freezes
the self-host caller stubs the standard classifies as Class 1). Like the off-peak scheduling
standard, it is **repo-local first**, shaped for promotion into an org-wide
`standards/agent-standards.md` in
[`petry-projects/.github`](https://github.com/petry-projects/.github).

### Scheduled workflows

- **Never schedule at minute 0.** A scheduled workflow must not use a `0 * * * *`
Expand Down
18 changes: 18 additions & 0 deletions docs/aw/ci-failure-analyst.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,24 @@ repo-x (check_run: failure)

**Why reusable workflow (not dispatch)?** CI Failure Analyst is read-only — it only posts a comment using the caller's own `GITHUB_TOKEN`. No `GH_PAT_WORKFLOWS` is needed. The reusable pattern is simpler: one file, `secrets: inherit`, and GitHub automatically scopes the token to the caller's repo.

### Interaction contract

CI Failure Analyst is a **Class 1 event-driven role** (a single reaction to
`check_run:[completed]`, no schedule) under the
[agentic interaction model](../agentic-interaction-model.md) (the normative standard). It has
no persona manifest — its trigger lives entirely in the byte-frozen thin caller
`.github/workflows/ci-failure-analyst.lock.yml` — so its pattern is declared in one
machine-readable, CI-verified contract:

- **Runtime lens** — [`interaction-contracts/ci-failure-analyst.yml`](../../interaction-contracts/ci-failure-analyst.yml)
(the `check_run` trigger, the `<!-- ci-analyst sha=... -->` emit, the per-SHA idempotency key
and concurrency lane, and why it is not bound by `pr-automation-budget`).

It structurally cannot loop — one comment per failing head SHA, deduped by the `ci-analyst`
marker and serialized by the per-head-SHA concurrency lane — so it carries no `stop_markers` and
`budget: none`. When the caller stub's `on:` block changes, update that contract and the
standard's §4 classification table in the same change.

### Files

| File | Role |
Expand Down
15 changes: 15 additions & 0 deletions docs/dev-lead/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,21 @@ When a CI check fails, a trusted bot leaves a review, or a human mentions `@clau

The agent is the reactive counterpart to the [PR Review Agent](../pr-review-agent/pr-review-agent.md) (which is proactive and read-only). The dev-lead agent is reactive and write-enabled.

### Interaction contract

dev-lead pairs a **Class 1 event-driven role** with a **Class 2 self-heal backstop timer** (dev-lead-retry.yml) under the
[agentic interaction model](../agentic-interaction-model.md) (the normative standard). Its
trigger/emit/idempotency pattern is declared as machine-readable, CI-verified contracts, not
prose in this spec:

- **Runtime lens** — [`interaction-contracts/dev-lead.yml`](../../interaction-contracts/dev-lead.yml)
(the deployed workflow's `on:` events, emits, idempotency key, concurrency lane, and budget).
- **Role lens** — [`personas/dev-lead/interaction.yml`](../../personas/dev-lead/interaction.yml)
(the persona-manifest view of the same pattern).

When this spec's event taxonomy (§4) or the workflow triggers (§7.1) change, update those
contracts and the standard's §4 classification table in the same change.

---

## 2. Design Principles
Expand Down
17 changes: 17 additions & 0 deletions docs/pr-review-agent/pr-review-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,23 @@ and **Copilot**.
knows the PR has new commits since the last review and runs the cascade
again — handling iterative review cycles cleanly.

## Interaction contract

pr-review pairs a **Class 1 event fast-path** (the `pr-review-trigger.yml` caller stub) with a
**Class 2 backstop timer** (`pr-review-sweep.yml`) under the
[agentic interaction model](../agentic-interaction-model.md) (the normative standard). Its
triggers, timers, emits, and stop markers are declared as machine-readable, CI-verified
contracts rather than prose:

- **Runtime lens** — [`interaction-contracts/pr-review.yml`](../../interaction-contracts/pr-review.yml)
(the deployed trigger/sweep `on:` blocks, the backstop timer's `stop_condition` +
`event_fast_path`, emits, and budget).
- **Role lens** — [`personas/pr-review/interaction.yml`](../../personas/pr-review/interaction.yml)
(the persona-manifest view of the same pattern).

When the trigger or sweep workflow's `on:` block changes, update those contracts and the
standard's §4 classification table in the same change.

## Setup

### Reviewer identity
Expand Down
Loading