feat: implement issue #233 — Phase 2: CI Failure Analyst gh-aw workflow

.gitattributes

@@ -0,0 +1 @@
+.github/workflows/*.lock.yml linguist-generated=true merge=ours

.github/aw/actions-lock.json

@@ -0,0 +1,14 @@
+{
+  "entries": {
+    "actions/github-script@v9.0.0": {
+      "repo": "actions/github-script",
+      "version": "v9.0.0",
+      "sha": "3a2844b7e9c422d3c10d287c895573f7108da1b3"
+    },
+    "github/gh-aw-actions/setup@v0.74.4": {
+      "repo": "github/gh-aw-actions/setup",
+      "version": "v0.74.4",
+      "sha": "d3abfe96a194bce3a523ed2093ddedd5704cdf62"
+    }
+  }
+}

.github/dependabot.yml

@@ -1,11 +1,12 @@
-version: 2
 updates:
-  # GitHub Actions — version updates (keep actions current)
-  - package-ecosystem: "github-actions"
-    directory: "/"
-    schedule:
-      interval: "weekly"
-    open-pull-requests-limit: 10
-    labels:
-      - "security"
-      - "dependencies"
+- directory: /
+  ignore:
+  - dependency-name: "github/gh-aw-actions/*" # Managed by gh aw compile. Version-locked to the gh-aw compiler; do not bump.
+  labels:
+  - security
+  - dependencies
+  open-pull-requests-limit: 10
+  package-ecosystem: github-actions
+  schedule:
+    interval: weekly
+version: 2

.github/workflows/ci-failure-analyst.md

@@ -0,0 +1,126 @@
+---
+on:
+  workflow_run:
+    workflows: [CI, Lint, Tests]
+    types: [completed]
+
+permissions:
+  actions: read
+  checks: read
+  contents: read
+  pull-requests: read
+
+engine: claude
+
+models:
+  claude: [claude-sonnet-4-6]
+
+tools:
+  github:
+    toolsets: [context, pull_requests, actions]
+
+network: defaults
+
+safe-outputs:
+  add-comment:
+    max: 1
+    staged: true
+---
+
+# CI Failure Analyst
+
+When a CI workflow run completes, diagnose failures and post a root-cause comment on the associated pull request.
+
+## Instructions
+
+Follow these steps in order.
+
+### 1. Guard: conclusion must be `failure`
+
+Check `github.event.workflow_run.conclusion`. If it is anything other than `failure`
+(e.g. `cancelled`, `skipped`, `success`, `neutral`), output a `noop` action and stop.
+Do not post any comment.
+
+### 2. Guard: workflow run must be associated with a PR
+
+Check `github.event.workflow_run.pull_requests`. If the array is empty, the workflow run is
+not attached to any open pull request. Output a `noop` and stop. Do not comment on
+issues or any other thread.
+
+### 3. Identify the target PR
+
+Use the first entry in `github.event.workflow_run.pull_requests`. Note:
+
+- PR number
+- Head SHA: `github.event.workflow_run.head_sha`
+- Workflow name: `github.event.workflow_run.name`
+- Details URL: `github.event.workflow_run.html_url`
+- Workflow run ID: `github.event.workflow_run.id`
+
+### 4. Check for an existing analyst comment (idempotency)
+
+List comments on the PR and search for one that starts with the exact marker:
+
+```
+<!-- ci-analyst sha=HEAD_SHA -->
+```
+
+where `HEAD_SHA` is the workflow run's `head_sha`. If a comment with this marker already
+exists, output a `noop` and stop. Do not post a duplicate comment.
+
+### 5. Fetch failure details
+
+Use the workflow run ID (`github.event.workflow_run.id`) with the actions toolset to retrieve
+the workflow run logs. Parse the logs to find the failing job and step.
+
+If logs are unavailable (e.g. the run has expired), note this in the comment and
+provide a best-guess diagnosis from the workflow name and any available context.
+
+### 6. Identify the failing step and error
+
+From the logs, extract:
+
+- The name of the specific step that failed
+- The key error message or assertion that caused the failure
+
+### 7. Classify the root cause
+
+Choose the single best-fit category:
+
+| Category | Indicators |
+|---|---|
+| **Test failure** | Assertion failed, test output shows `FAIL`, `AssertionError`, `Expected … got …` |
+| **Env issue** | Missing secret, missing env var, `command not found`, `permission denied`, authentication error |
+| **Flaky test** | Timeout, connection reset, port conflict, `ECONNREFUSED`, intermittent HTTP error |
+| **Config error** | Malformed workflow YAML, missing required field, `invalid syntax`, wrong runner |
+| **Lint/style** | `eslint`, `shellcheck`, `markdownlint`, `yamllint`, `ruff`, `golangci-lint` errors |
+| **Build error** | Compilation error, `npm ERR!`, `cargo build` failed, dependency resolution failure |
+
+If the workflow name is `lint`, `shellcheck`, `markdownlint`, `yamllint`, or similar, the
+category **must** be Lint/style — do not classify it as Test failure.
+
+### 8. Post the diagnostic comment
+
+Post a comment to the PR (identified in Step 3 by PR number) using exactly this structure:
+
+```markdown
+<!-- ci-analyst sha=HEAD_SHA -->
+## CI Failure: WORKFLOW_NAME
+
+**Step:** FAILING_STEP_NAME
+**Root cause:** ROOT_CAUSE_CATEGORY
+
+BRIEF_EXPLANATION (2–3 sentences: what failed, why it likely failed, what the error means)
+
+**Suggested fix:** ONE_CONCRETE_ACTION
+
+[View run logs](DETAILS_URL)
+```
+
+Rules:
+- The `<!-- ci-analyst sha=... -->` marker **must** be the very first line of the comment.
+- Root cause category must be one of the six categories defined above.
+- Suggested fix must be one specific, actionable step (e.g. "Run `npm test` locally and fix the failing assertion in `src/auth.test.js`", not "Fix the tests").
+- Keep the entire comment under 300 words.
+- Do not include any information beyond this structure.
+- Pass the PR number explicitly when invoking the `add-comment` safe output.

.github/workflows/ci.yml

@@ -93,6 +93,24 @@ jobs:
         run: |
           npx --yes --ignore-scripts ecc-agentshield@1.4.0 scan --path . --format json --min-severity high
 
+  aw-compile:
+    name: Compile agentic workflows
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Install gh aw
+        run: gh extension install github/gh-aw --pin v0.74.4
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Compile agentic workflows
+        run: gh aw compile --no-emit --approve
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
   secret-scan:
     name: Secret scan (gitleaks)
     runs-on: ubuntu-latest

docs/aw/ci-failure-analyst.md

@@ -0,0 +1,93 @@
+# CI Failure Analyst
+
+Agentic workflow that diagnoses CI failures and posts a root-cause comment on the associated pull request within 2 minutes.
+
+## What it does
+
+When a check run completes with `conclusion: failure`, the workflow:
+
+1. Fetches the failed run logs from the GitHub API
+2. Identifies the specific step that failed and its error message
+3. Classifies the root cause into one of six categories
+4. Posts a structured diagnostic comment on the PR (or updates an existing analyst comment for the same SHA — no duplicates)
+
+If the check run is not associated with a PR, or if the conclusion is anything other than `failure`, the workflow is a no-op.
+
+## Trigger
+
+```yaml
+on:
+  workflow_run:
+    workflows: [CI, Lint, Tests]
+    types: [completed]
+```
+
+The workflow handles `conclusion != failure` as a no-op in its instructions. The `workflow_run` trigger fires for GitHub Actions workflow completions, including this repo's own CI workflows (`ci.yml`, `lint.yml`, `test.yml`). The analyst itself guards on conclusion.
+
+## Root cause categories
+
+| Category | Indicators |
+|---|---|
+| **Test failure** | Assertion failed, `FAIL`, `AssertionError`, `Expected … got …` |
+| **Env issue** | Missing secret/env var, `command not found`, `permission denied`, auth error |
+| **Flaky test** | Timeout, `ECONNREFUSED`, connection reset, intermittent HTTP error |
+| **Config error** | Malformed YAML, missing required field, wrong runner, `invalid syntax` |
+| **Lint/style** | `eslint`, `shellcheck`, `markdownlint`, `yamllint`, `ruff` errors |
+| **Build error** | Compilation error, `npm ERR!`, dependency resolution failure |
+
+## Comment format
+
+The analyst posts (or updates) a comment on the PR with this structure:
+
+```markdown
+<!-- ci-analyst sha=<HEAD_SHA> -->
+## CI Failure: <check name>
+
+**Step:** <failing step name>
+**Root cause:** <category>
+
+<2–3 sentence explanation>
+
+**Suggested fix:** <one concrete action>
+
+[View run logs](<details_url>)
+```
+
+The `<!-- ci-analyst sha=... -->` marker enables idempotency: if the workflow triggers again for the same SHA, it updates the existing comment instead of creating a duplicate.
+
+## Permissions
+
+| Permission | Reason |
+|---|---|
+| `actions: read` | Fetch workflow run logs via the actions toolset |
+| `checks: read` | Read check run details and annotations |
+| `contents: read` | Check out the repository for agent context |
+| `pull-requests: read` | List PR comments for idempotency check |
+
+Write access (posting the comment) is handled by the `safe-outputs` job using the `GH_AW_GITHUB_TOKEN` secret (or `GITHUB_TOKEN` as a fallback). The `safe-outputs` job runs with `permissions: {}` and relies on an external token for the write operation, so `GITHUB_TOKEN` alone is not sufficient without `GH_AW_GITHUB_TOKEN` set.
+
+## Engine and model
+
+- **Engine:** Claude (via `engine: claude`)
+- **Model:** Configured via the `GH_AW_MODEL_AGENT_CLAUDE` repository variable. Set it to `claude-sonnet-4-6` to use the recommended model. If unset, the engine defaults to `auto`.
+
+## Staged mode
+
+The `add-comment` safe-output is configured with `staged: true`. During the initial rollout, posted comments are queued for human review before being published. Remove `staged: true` from the `add-comment` block in the workflow frontmatter once the output has been verified against the scenario spec.
+
+## Setup
+
+1. Ensure the `ANTHROPIC_API_KEY` secret is set in the repository (or org-level secrets).
+2. Set `GH_AW_GITHUB_TOKEN` to a PAT with `pull-requests: write` permission. This token is used by the `safe-outputs` job to post the diagnostic comment. Without it, `GITHUB_TOKEN` will be used as a fallback but will lack write permission (the `safe-outputs` job runs with `permissions: {}`).
+3. Optionally set `GH_AW_GITHUB_MCP_SERVER_TOKEN` to a PAT for MCP GitHub API access. Falls back to `GH_AW_GITHUB_TOKEN`, then `GITHUB_TOKEN`.
+4. Set the `GH_AW_MODEL_AGENT_CLAUDE` repository variable to `claude-sonnet-4-6`.
+5. Compile the workflow: `gh aw compile ci-failure-analyst --approve`.
+6. Commit both `ci-failure-analyst.md` and `ci-failure-analyst.lock.yml`.
+
+## Scenario spec
+
+See [`tests/aw/ci-failure-analyst/scenarios.md`](../../tests/aw/ci-failure-analyst/scenarios.md) for the full scenario spec used to verify correct behavior, including edge cases for non-failure conclusions, PRs without associated check runs, and duplicate-comment suppression.
+
+## Relationship to dev-lead
+
+The CI Failure Analyst is a **read-only diagnostic** agent. It posts comments but does not push commits or open PRs. The `dev-lead` agent retains exclusive write authority for applying actual fixes to failing CI.